Thread: proposal: persistent plpgsql plugin info - field plugin_info for plpgsql_function structure

Hello

I am working on plpgsql_check and I would to write a protection against repeated check - so I need to mark a compiled (cached) function. Now, plpgsql extension can store own data to exec state, and I would to some similar for plpgsql_function. I believe so it can be useful for any plpgsql extension that collects data per signature (and recreating) means so refresh is necessary.

Ideas, objections?

Regards

Pavel
Pavel Stehule escribió:
> Hello
> 
> I am working on plpgsql_check and I would to write a protection against
> repeated check - so I need to mark a compiled (cached) function. Now,
> plpgsql extension can store own data to exec state, and I would to some
> similar for plpgsql_function. I believe so it can be useful for any plpgsql
> extension that collects data per signature (and recreating) means so
> refresh is necessary.

I'm not sure I understand this.  Do you want to avoid running the
checker if a previous run was seen as successful and function has not
changed?  Suppose the function depends on a table.  I invoke the check
(it succeeds), then drop the table, then invoke the check again.  What
should happen?  Conversely, if I invoke the check and it fails, then I
create the table and invoke the check again, what should happen?  How
does this idea of yours know when to invalidate the cached result of the
check when unrelated objects, which the function depends on, are
dropped/created/modified?

-- 
Álvaro Herrera                http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services






2013/12/31 Alvaro Herrera <alvherre@2ndquadrant.com>
Pavel Stehule escribió:
> Hello
>
> I am working on plpgsql_check and I would to write a protection against
> repeated check - so I need to mark a compiled (cached) function. Now,
> plpgsql extension can store own data to exec state, and I would to some
> similar for plpgsql_function. I believe so it can be useful for any plpgsql
> extension that collects data per signature (and recreating) means so
> refresh is necessary.

I'm not sure I understand this.  Do you want to avoid running the
checker if a previous run was seen as successful and function has not
changed?  Suppose the function depends on a table.  I invoke the check
(it succeeds), then drop the table, then invoke the check again.  What
should happen?  Conversely, if I invoke the check and it fails, then I
create the table and invoke the check again, what should happen?  How
does this idea of yours know when to invalidate the cached result of the
check when unrelated objects, which the function depends on, are
dropped/created/modified?

plpgsql_check is designed for interactive (active) mode and passive mode. In interactive mode a enhanced checking is started by explicit request - explicit using plpgsql_check function - this feature is taken from patches that I sent to mailing list. In this mode a check is executed always (when checking is enabled) - and it is called repeatedly (when user requests it).

Passive mode is taken from plpgsql_lint extension. It is plpgsql extension based on begin_func callback. plpgsql_lint doesn't support fatal_errors option - every detected error is fatal, that stops execution. plpgsql_check allows fatal_errors = false (plpgsql_check raises warnings only], and I am searching way how to minimize repeated warning messages. It is one motivation. Second motivation is increasing speed of regression tests by removing repeated check. Good idea is a function that removes mark from compiled function - so anybody can do recheck without leaving of session.

Requested feature doesn't help me implement this concept 100%, but helps with check If I worked with some instance of function or not. And inside core a implementation is cheap. Outside core it is a magic with hash and checking transaction id (about 200 lines). When I worked on extension for coverage calculation I had to solve same task, so I think so this variable can be useful generally for similar tasks.

Pavel Stehule <pavel.stehule@gmail.com> writes:
> Requested feature doesn't help me implement this concept 100%, but helps
> with check If I worked with some instance of function or not. And inside
> core a implementation is cheap. Outside core it is a magic with hash and
> checking transaction id (about 200 lines). When I worked on extension for
> coverage calculation I had to solve same task, so I think so this variable
> can be useful generally for similar tasks.

Are you proposing a reserved-for-plugins "void*" in struct
PLpgSQL_function similar to the existing one in struct PLpgSQL_execstate?

If so, while it sounds harmless in itself, I think your argument above is
actually the strongest reason *not* to do it.  The existing plpgsql plugin
infrastructure is incapable of supporting more than one plugin at a time,
and the more attractive we make it, the more likely we're going to have
conflicts.  It was never meant to support anything but the plpgsql
debugger.  Before we start aiding and abetting the development of other
plugins, we need a design that allows more than one of them to be
installed.
        regards, tom lane






2013/12/31 Tom Lane <tgl@sss.pgh.pa.us>
Pavel Stehule <pavel.stehule@gmail.com> writes:
> Requested feature doesn't help me implement this concept 100%, but helps
> with check If I worked with some instance of function or not. And inside
> core a implementation is cheap. Outside core it is a magic with hash and
> checking transaction id (about 200 lines). When I worked on extension for
> coverage calculation I had to solve same task, so I think so this variable
> can be useful generally for similar tasks.

Are you proposing a reserved-for-plugins "void*" in struct
PLpgSQL_function similar to the existing one in struct PLpgSQL_execstate?

If so, while it sounds harmless in itself, I think your argument above is
actually the strongest reason *not* to do it.  The existing plpgsql plugin
infrastructure is incapable of supporting more than one plugin at a time,
and the more attractive we make it, the more likely we're going to have
conflicts.  It was never meant to support anything but the plpgsql
debugger.  Before we start aiding and abetting the development of other
plugins, we need a design that allows more than one of them to be
installed.

ok, what we can do better?

Can be solution a callback on plpgsql_HashTableInsert and plpgsql_HashTableInsert? With these callbacks a relation between function and some plugin data can be implemented more simply.

Regards

Pavel

 

                        regards, tom lane

On Wed, Jan 1, 2014 at 12:52 AM, Pavel Stehule <pavel.stehule@gmail.com> wrote:
> 2013/12/31 Alvaro Herrera <alvherre@2ndquadrant.com>
>>
>> I'm not sure I understand this.  Do you want to avoid running the
>> checker if a previous run was seen as successful and function has not
>> changed?  Suppose the function depends on a table.  I invoke the check
>> (it succeeds), then drop the table, then invoke the check again.  What
>> should happen?  Conversely, if I invoke the check and it fails, then I
>> create the table and invoke the check again, what should happen?  How
>> does this idea of yours know when to invalidate the cached result of the
>> check when unrelated objects, which the function depends on, are
>> dropped/created/modified?
>
>
> plpgsql_check is designed for interactive (active) mode and passive mode. In
> interactive mode a enhanced checking is started by explicit request -
> explicit using plpgsql_check function - this feature is taken from patches
> that I sent to mailing list. In this mode a check is executed always (when
> checking is enabled) - and it is called repeatedly (when user requests it).
>
> Passive mode is taken from plpgsql_lint extension. It is plpgsql extension
> based on begin_func callback. plpgsql_lint doesn't support fatal_errors
> option - every detected error is fatal, that stops execution. plpgsql_check
> allows fatal_errors = false (plpgsql_check raises warnings only], and I am
> searching way how to minimize repeated warning messages. It is one
> motivation. Second motivation is increasing speed of regression tests by
> removing repeated check. Good idea is a function that removes mark from
> compiled function - so anybody can do recheck without leaving of session.

Will it really help by adding different modes for plpgsql_check functionality,
is that going to help in overcoming the concerns raised by committer
which I understand was mainly about code duplication, improvement
in messages and comments in code or am I missing something?


With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com