Thread: Plan invalidation design

Plan invalidation design

From
Tom Lane
Date:
I'm starting to think about the long-wanted plan invalidation mechanism.
Here's a sketch --- anyone see any problems?

* Create a new module, say src/backend/utils/cache/plancache.c, that we
will put in charge of all long-lived plans --- or at least those cached by
PREPARE, plpgsql, and RI triggers.  I'm unsure whether we should make all
SPI plans work this way or not; it's possible that doing so would change
SPI's API behavior enough to break user-written code.  Any thoughts on
that?

* plancache.c will have two basic functions:

1. Given a query's raw parse tree (that is, the raw output of gram.y),
analyze and plan the query.  Store both the parse tree and plan in a
backend-local cache table, and return a handle for the table entry as well
as the plan tree.

2. Given a handle for a previously stored query, check to see if the plan
is still up to date; if not, regenerate it from the raw parse tree (note
this could result in failure, eg if a column used by the query has been
dropped).  Then return the plan tree.

We probably want to return a direct pointer to the cached plan tree
instead of making a copy.  This should be safe, because the executor now
treats plan trees as read-only, but it does mean that when plan
invalidation occurs the cached plan tree might still be in use.  We'll
probably need to have a notion of a reference count: so the two functions
above would increment the plan's refcount and there would be a third
"ReleasePlanCache" function to call when done using a plan (and, hence,
these references would need to be supported by the ResourceManager
mechanism).

Note that the source object for caching is a raw parse tree.  This should
work since we already require that gram.y not look into the database
during its processing; therefore, the raw tree need never be invalidated.
It'd be conceptually simpler if we passed in a query string instead, but
I don't think that works for PREPARE, because it might be embedded in a
multi-command string.  (We do probably want to pass in the original query
string too, if available, because it's needed for syntax error reporting.)
nodes/copyfuncs.c will need some expansion, as I don't believe it has
coverage for all raw-parse-tree node types.

Invalidation will be detected by having plancache.c watch for relcache
invalidation events, using the existing inval.c callback mechanism.
On any relcache inval, traverse the plan cache looking for plans that
mention the invalidated relation in their rangetables, and mark them as
needing to be regenerated before next use.  (If they currently have
refcount zero, we could delete the plan part of the cache entry
immediately.)

Relcache inval casts a fairly wide net; for example, adding or dropping an
index will invalidate all plans using the index's table whether or not
they used that particular index, and I believe that VACUUM will also
result in a relcache inval due to updating the table's pg_class row.
I think this is a good thing though --- for instance, after adding an
index it seems a good idea to replan to see if the new index is useful,
and replanning after a VACUUM is useful if the table has changed size
enough to warrant a different plan.  OTOH this might mean that plans on a
high-update-traffic table never survive very long because of autovacuum's
efforts.  If that proves to be a problem in practice we can look at ways
to dial down the number of replans, but for the moment I think it's more
important to be sure we *can* replan at need than to find ways to avoid
replans.

Note that I'm currently intending to detect only relcache invals, not
changes to functions or operators used in the plan.  (Relcache inval will
cover view redefinitions, though.)  We could extend it to handle that
later, but it looks like a lot more mechanism and overhead for not a lot
of gain.  AFAICS there are only three cases where there'd be a benefit:
* if you redefine an immutable function, any places where its result has been pre-computed by constant-folding wouldn't
getupdated without inval.
 
* if you have a SQL function that's been inlined into a plan, a change in the function definition wouldn't get
reflectedinto the plan without inval.
 
* if you alter a function and change its volatility property, that might possibly affect the shape of plans that use
thefunction (for instance some optimization transformation might now be allowed or not).
 
To my memory none of these problems have been complained of from the
field.  Making the cache module able to detect function-related
invalidations would be a bit of work --- for example, if a function has
been inlined, there is no recognizable reference to it at all in the plan
tree, so we'd have to modify the planner to track such things and report
them somehow.  (The corresponding problem for views doesn't exist, because
there is still a rangetable entry for a view after it's been expanded.)
So I think this is a "maybe do someday" part, not something to do in the
first release.

One interesting point is that to avoid race conditions, the function that
checks for is-plan-update-required will have to acquire locks on the
tables mentioned in the plan before it can be sure there's not a pending
invalidation event on them.  This doesn't seem like a problem, though it
might mean we want to refactor the executor API a bit to avoid duplicate
effort.

Comments?
        regards, tom lane


Re: Plan invalidation design

From
Lukas Kahwe Smith
Date:
Tom Lane wrote:

> Relcache inval casts a fairly wide net; for example, adding or dropping an
> index will invalidate all plans using the index's table whether or not
> they used that particular index, and I believe that VACUUM will also
> result in a relcache inval due to updating the table's pg_class row.
> I think this is a good thing though --- for instance, after adding an
> index it seems a good idea to replan to see if the new index is useful,
> and replanning after a VACUUM is useful if the table has changed size
> enough to warrant a different plan.  OTOH this might mean that plans on a
> high-update-traffic table never survive very long because of autovacuum's
> efforts.  If that proves to be a problem in practice we can look at ways
> to dial down the number of replans, but for the moment I think it's more
> important to be sure we *can* replan at need than to find ways to avoid
> replans.

I remember that there was discussion about invalidating plans who's 
estimated cost turn out to be severely off when executed. That is 
probably a more reliable metric (than invalidating with every VACCUM - 
unless of course the amount of changed rows is considered), though it 
will probably put a fixed overhead on all relevant queries. So it might 
not be feasible. Of course this checking after a query runs longer than 
expected also means that at least one execution will in fact have to run 
slow instead of preempting this from happening at all.

Also while not directly related it might be thing to keep in mind. It 
would also be cool to support multiple plans for different sets of 
parameters, since obviously the data distribution and therefore the 
optimal plan will potentially vary greatly with different parameters.

regards,
Lukas

PS: I moved "Plan invalidation" to confirmed on the wishlist ..


Re: Plan invalidation design

From
Tom Lane
Date:
Lukas Kahwe Smith <smith@pooteeweet.org> writes:
> I remember that there was discussion about invalidating plans who's 
> estimated cost turn out to be severely off when executed.

That's something we might think about after the infrastructure is in
place.  But the question to answer is why the re-plan won't yield
just the same plan as before.
        regards, tom lane


Re: Plan invalidation design

From
Lukas Kahwe Smith
Date:
Tom Lane wrote:
> Lukas Kahwe Smith <smith@pooteeweet.org> writes:
>> I remember that there was discussion about invalidating plans who's 
>> estimated cost turn out to be severely off when executed.
> 
> That's something we might think about after the infrastructure is in
> place.  But the question to answer is why the re-plan won't yield
> just the same plan as before.

Yeah, also invalidating plans like this only really makes sense once we 
have the ability to keep multiple plans around for different sets of 
parameters. Otherwise we could also end up in a situation where after 
every execution we determine that a re-plan is necessary because the 
parameters used differ in distribution.

regards,
Lukas


Re: Plan invalidation design

From
Lukas Kahwe Smith
Date:
Tom Lane wrote:

> place.  But the question to answer is why the re-plan won't yield
> just the same plan as before.

oh and when the estimated cost repeatedly do not match the actual cost, 
we of course want to generate an email with all relevant information 
that is send to this list ;)

regards,
Lukas


Re: Plan invalidation design

From
Russell Smith
Date:
Tom Lane wrote:
> I'm starting to think about the long-wanted plan invalidation mechanism.
> Here's a sketch --- anyone see any problems?
>
> * Create a new module, say src/backend/utils/cache/plancache.c, that we
> will put in charge of all long-lived plans --- or at least those cached by
> PREPARE, plpgsql, and RI triggers.  I'm unsure whether we should make all
> SPI plans work this way or not; it's possible that doing so would change
> SPI's API behavior enough to break user-written code.  Any thoughts on
> that?
>
> * plancache.c will have two basic functions:
>
> 1. Given a query's raw parse tree (that is, the raw output of gram.y),
> analyze and plan the query.  Store both the parse tree and plan in a
> backend-local cache table, and return a handle for the table entry as well
> as the plan tree.
>
> 2. Given a handle for a previously stored query, check to see if the plan
> is still up to date; if not, regenerate it from the raw parse tree (note
> this could result in failure, eg if a column used by the query has been
> dropped).  Then return the plan tree.
>   
What do we do in the case of failure?  Die in the same way we do now 
when you can't use the plan that's been made?
> We probably want to return a direct pointer to the cached plan tree
> instead of making a copy.  This should be safe, because the executor now
> treats plan trees as read-only, but it does mean that when plan
> invalidation occurs the cached plan tree might still be in use.  We'll
> probably need to have a notion of a reference count: so the two functions
> above would increment the plan's refcount and there would be a third
> "ReleasePlanCache" function to call when done using a plan (and, hence,
> these references would need to be supported by the ResourceManager
> mechanism).
>   
excuse my ignorance here, but under what circumstances is a plan in use 
for a single backend at the same time as it's invalidated.
What potential failures does this introduce?  If you are using the old 
plan, and the new plan fails as mentioned above.  Where are we then?
> Note that the source object for caching is a raw parse tree.  This should
> work since we already require that gram.y not look into the database
> during its processing; therefore, the raw tree need never be invalidated.
> It'd be conceptually simpler if we passed in a query string instead, but
> I don't think that works for PREPARE, because it might be embedded in a
> multi-command string.  (We do probably want to pass in the original query
> string too, if available, because it's needed for syntax error reporting.)
> nodes/copyfuncs.c will need some expansion, as I don't believe it has
> coverage for all raw-parse-tree node types.
>   
If the syntax has become invalid, that is because the columns in the 
query, or tables have changed.  Is this information not available in the 
plan tree?
What other circumstances could you have a syntax error from a query that 
has been successfully planned and parsed?
I've read this paragraph 3 times now and am still quite unclear about 
the requirements for the original query to be stored.  Is the plan cache 
going to replace the syntax check which I thought would have been done 
in gram.y. 
> Invalidation will be detected by having plancache.c watch for relcache
> invalidation events, using the existing inval.c callback mechanism.
> On any relcache inval, traverse the plan cache looking for plans that
> mention the invalidated relation in their rangetables, and mark them as
> needing to be regenerated before next use.  (If they currently have
> refcount zero, we could delete the plan part of the cache entry
> immediately.)
>
> Relcache inval casts a fairly wide net; for example, adding or dropping an
> index will invalidate all plans using the index's table whether or not
> they used that particular index, and I believe that VACUUM will also
> result in a relcache inval due to updating the table's pg_class row.
> I think this is a good thing though --- for instance, after adding an
> index it seems a good idea to replan to see if the new index is useful,
> and replanning after a VACUUM is useful if the table has changed size
> enough to warrant a different plan.  OTOH this might mean that plans on a
> high-update-traffic table never survive very long because of autovacuum's
> efforts.  If that proves to be a problem in practice we can look at ways
> to dial down the number of replans, but for the moment I think it's more
> important to be sure we *can* replan at need than to find ways to avoid
> replans.
>
> Note that I'm currently intending to detect only relcache invals, not
> changes to functions or operators used in the plan.  (Relcache inval will
> cover view redefinitions, though.)  We could extend it to handle that
> later, but it looks like a lot more mechanism and overhead for not a lot
> of gain.  AFAICS there are only three cases where there'd be a benefit:
> * if you redefine an immutable function, any places where its result has
>   been pre-computed by constant-folding wouldn't get updated without inval.
>   
If you replan and immutable function, aren't you possibly messing up a 
functional index that is using the old function.  Hey, if you change an 
immutable function that has an index, you are in trouble already.
So the implication of a immutable function change are much more wide 
ranging that plan invalidation problems.

> * if you have a SQL function that's been inlined into a plan, a change
>   in the function definition wouldn't get reflected into the plan without
>   inval.
> * if you alter a function and change its volatility property, that might
>   possibly affect the shape of plans that use the function (for instance
>   some optimization transformation might now be allowed or not).
>   
Replanning pl/pgsql with CREATE TEMP TABLE would be a good use here.  
You loose the preplanning benefits, but we remove the ongoing problem 
where people report that their temp-table isn't working.

Even function alterations to pl/pgsql should a replan.  But of more 
interest is being able to use the old function for currently running 
transactions when the function is changed.  Last time I tried to edit a 
pl/pgsql function while it was being used by a transaction, the 
transaction failed because the function definition changed.  I'm not 
100% sure of the use case here as I'm writing this email at too late an 
hour.
> To my memory none of these problems have been complained of from the
> field.  Making the cache module able to detect function-related
> invalidations would be a bit of work --- for example, if a function has
> been inlined, there is no recognizable reference to it at all in the plan
> tree, so we'd have to modify the planner to track such things and report
> them somehow.  (The corresponding problem for views doesn't exist, because
> there is still a rangetable entry for a view after it's been expanded.)
> So I think this is a "maybe do someday" part, not something to do in the
> first release.
>
> One interesting point is that to avoid race conditions, the function that
> checks for is-plan-update-required will have to acquire locks on the
> tables mentioned in the plan before it can be sure there's not a pending
> invalidation event on them.  This doesn't seem like a problem, though it
> might mean we want to refactor the executor API a bit to avoid duplicate
> effort.
>   
Is the race condition here any more likely to happen than the failure of 
a re plan when something has changed from underneath the original query?
My very brief thought gives me the impression that they are the same 
thing, however they may not be.
> Comments?
>   
Again, as a person who has only a limited understand of the code, I'm 
happy to be wrong about anything I have written.

Regards

Russell Smith


Re: Plan invalidation design

From
Tom Lane
Date:
Russell Smith <mr-russ@pws.com.au> writes:
> Tom Lane wrote:
>> 2. Given a handle for a previously stored query, check to see if the plan
>> is still up to date; if not, regenerate it from the raw parse tree (note
>> this could result in failure, eg if a column used by the query has been
>> dropped).  Then return the plan tree.
>> 
> What do we do in the case of failure?  Die in the same way we do now 
> when you can't use the plan that's been made?

Well, the difference is that at plan use you might get an error that
currently could only occur at initial query parsing.  I don't see that
this is a big deal, but it will be a change in behavior.

One thing I realized since yesterday is that it'll have to be possible
for the caller to tell whether the plan has changed since he last saw
it (perhaps via a re-plan counter included in the cache entry).  It's
entirely possible that the set of output columns will have changed,
and so the caller may need to re-do derived work.  For example plpgsql
will need to re-do its analysis of whether a plan is "simple".

What we might want in some cases is for the caller to decide to error
out if the set of output columns changes.  I think this is likely
appropriate for queries prepared via the Parse protocol message, because
we may have already told the client what the column set is, and it won't
be prepared to deal with getting a different set of columns back.  I'm
not sure now whether that's appropriate for every call site, but if it
is then we could avoid some of these definitional issues.

>> We probably want to return a direct pointer to the cached plan tree
>> instead of making a copy.  This should be safe, because the executor now
>> treats plan trees as read-only, but it does mean that when plan
>> invalidation occurs the cached plan tree might still be in use.

> excuse my ignorance here, but under what circumstances is a plan in use 
> for a single backend at the same time as it's invalidated.

There shouldn't be any structural changes in a table once you've
acquired lock on it, but there could be statistics changes, eg from
VACUUM; and the relcache inval mechanism currently doesn't distinguish
those cases.  We'd need some such concept anyway if we ever extend
the invalidation to cover functions, because there's no locking on them.

> What other circumstances could you have a syntax error from a query that 
> has been successfully planned and parsed?

DROP COLUMN, DROP FUNCTION, ... lots of possibilities.

> I've read this paragraph 3 times now and am still quite unclear about 
> the requirements for the original query to be stored.  Is the plan cache 
> going to replace the syntax check which I thought would have been done 
> in gram.y. 

We don't need to re-do that syntax check, precisely because it's purely
a syntax check and doesn't involve any database state.

> If you replan and immutable function, aren't you possibly messing up a 
> functional index that is using the old function.  Hey, if you change an 
> immutable function that has an index, you are in trouble already.

True.

> Replanning pl/pgsql with CREATE TEMP TABLE would be a good use here.  
> You loose the preplanning benefits, but we remove the ongoing problem 
> where people report that their temp-table isn't working.

Yeah, that's one of the main arguments why this is worth the trouble.

> Even function alterations to pl/pgsql should a replan.  But of more 
> interest is being able to use the old function for currently running 
> transactions when the function is changed.  Last time I tried to edit a 
> pl/pgsql function while it was being used by a transaction, the 
> transaction failed because the function definition changed.

I fixed a couple of bugs in that area recently --- the current behavior
should be that any active execution of a plpgsql function will finish
out using the function definition that was current when it started.  But
that's something that's local to the PL function manager and doesn't
really have anything to do with plans using the function.  Inlined SQL
functions are the exception to the rule that a plan doesn't know exactly
what a function it calls does.

> Is the race condition here any more likely to happen than the failure of 
> a re plan when something has changed from underneath the original query?

It's not really the same thing --- the problem is making sure that your
check for table changes is accurate, and doesn't miss a change that
commits just after you look.
        regards, tom lane


Re: Plan invalidation design

From
Gregory Stark
Date:
"Tom Lane" <tgl@sss.pgh.pa.us> writes:

> Russell Smith <mr-russ@pws.com.au> writes:
>> Tom Lane wrote:
>>> 2. Given a handle for a previously stored query, check to see if the plan
>>> is still up to date; if not, regenerate it from the raw parse tree (note
>>> this could result in failure, eg if a column used by the query has been
>>> dropped).  Then return the plan tree.
>>> 
>> What do we do in the case of failure?  Die in the same way we do now 
>> when you can't use the plan that's been made?
>
> Well, the difference is that at plan use you might get an error that
> currently could only occur at initial query parsing.  I don't see that
> this is a big deal, but it will be a change in behavior.
>
> One thing I realized since yesterday is that it'll have to be possible
> for the caller to tell whether the plan has changed since he last saw
> it (perhaps via a re-plan counter included in the cache entry).  It's
> entirely possible that the set of output columns will have changed,
> and so the caller may need to re-do derived work.  For example plpgsql
> will need to re-do its analysis of whether a plan is "simple".

Hm. The set of output columns could change? How?

If you prepare "select *" and add a column, you're saying the query should
start failing? That seems strange given the behaviour of views, which is that
once parsed the list of columns is written in stone. It seems prepared queries
should work the same way that views work and remember which physical column
they were referring to previously. (Personally I don't like that behaviour but
it feels like this should be consistent with it.)

I guess you do have a serious problem if you redefine the type of a column or
redefine a view (though I think you would have to drop and recreate it, CREATE
OR REPLACE wouldn't let you change the output columns).

>>> We probably want to return a direct pointer to the cached plan tree
>>> instead of making a copy.  This should be safe, because the executor now
>>> treats plan trees as read-only, but it does mean that when plan
>>> invalidation occurs the cached plan tree might still be in use.
>
>> excuse my ignorance here, but under what circumstances is a plan in use 
>> for a single backend at the same time as it's invalidated.

Invalidation messages can occur at certain. If you access any new table or
object while the plan is still running, either because you're in a plpgsql
loop fetching records from it, or because some function you're calling in the
query runs some other sql against another table then you'll receive any
pending invalidation messages. 

It should only be possible to receive messages from operations that are legal
to execute while someone is using the object. So, for example, creating new
indexes. So if you're actively in the process of using the plan it shouldn't
be necessary to junk it.

Perhaps that means it would be handy to have two kinds of invalidation
messages. Hard invalidations mean that anybody with cached plans should
immediately junk them and throw up nasty errors and assertion failures if
they're in a state when that shouldn't happen. And Soft invalidations mean you
shouldn't start any new queries but any that are executing are still ok.

--  Gregory Stark EnterpriseDB          http://www.enterprisedb.com


Re: Plan invalidation design

From
Tom Lane
Date:
Gregory Stark <stark@enterprisedb.com> writes:
> If you prepare "select *" and add a column, you're saying the query should
> start failing?

Either fail or change output; which you like better?  The whole point of
this exercise is to support plpgsql functions that do something like
create temp table foo ...select * into rec from foo ...drop table foo ...

and from the system's point of view successive executions of this
sequence are talking about completely different tables.  There's no
reason to suppose they have the same expansion of "*".  I'd also like
to think that the semantics of a plpgsql function are not going to
be different the first time it's executed in a session than subsequent
times, which suggests that indeed it ought to cope with "*" expanding
differently from last time.

To do what you're suggesting, we'd have to redesign parse analysis
so that expansion of "*" was a completely separate step from everything
else, producing a parse tree that's still raw except for that one change;
and I'm not sure what other strange decisions we'd have to make.  I
don't find that an attractive idea.
        regards, tom lane


Re: Plan invalidation design

From
"Merlin Moncure"
Date:
On 2/19/07, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Gregory Stark <stark@enterprisedb.com> writes:
> > If you prepare "select *" and add a column, you're saying the query should
> > start failing?
>
> Either fail or change output; which you like better?  The whole point of
> this exercise is to support plpgsql functions that do something like
>
>         create temp table foo ...
>         select * into rec from foo ...
>         drop table foo ...

If that's the case, do you think there is a simpler way to handle this
problem than plan invalidation?  Maybe I'm oversimplifying things a
bit here, but how about something like:

create local table foo ...
select * into rec from foo ...

this isn't completely unsurprising...we have 'SET LOCAL, etc.

merlin


Re: Plan invalidation design

From
Russell Smith
Date:
Gregory Stark wrote:


[snip]

>
> Hm. The set of output columns could change? How?
>
> If you prepare "select *" and add a column, you're saying the query should
> start failing? That seems strange given the behaviour of views, which is that
> once parsed the list of columns is written in stone. It seems prepared queries
> should work the same way that views work and remember which physical column
> they were referring to previously. (Personally I don't like that behaviour but
> it feels like this should be consistent with it.)
>
> I guess you do have a serious problem if you redefine the type of a column or
> redefine a view (though I think you would have to drop and recreate it, CREATE
> OR REPLACE wouldn't let you change the output columns).
>   

I would think it best to move towards changing views to not have output 
columns set in stone.  It seems unreasonable that you can add/drop/alter 
columns in a table as much as you like, but you can't touch a view.  I 
also not excited about the current view restrictions.  Which means we 
don't want to start backing the idea by putting in more code that acts 
in the same way.

I'm guessing from what Tom is saying, that the reason we have views set 
in stone is because they are/can be an example of inlined SQL.  
Particularly when views are built on views.  Any further enlightenment 
welcome, but probably off topic for this thread.

Russell Smith


Re: Plan invalidation design

From
"Simon Riggs"
Date:
On Sat, 2007-02-17 at 12:48 -0500, Tom Lane wrote:

> Relcache inval casts a fairly wide net; for example, adding or dropping an
> index will invalidate all plans using the index's table whether or not
> they used that particular index, and I believe that VACUUM will also
> result in a relcache inval due to updating the table's pg_class row.
> I think this is a good thing though --- for instance, after adding an
> index it seems a good idea to replan to see if the new index is useful,
> and replanning after a VACUUM is useful if the table has changed size
> enough to warrant a different plan.  OTOH this might mean that plans on a
> high-update-traffic table never survive very long because of autovacuum's
> efforts.  If that proves to be a problem in practice we can look at ways
> to dial down the number of replans, but for the moment I think it's more
> important to be sure we *can* replan at need than to find ways to avoid
> replans.

Just some info on that: In an update-intensive scenario, I'm seeing
VACUUMs every 2 minutes on the heaviest hit tables on CVS HEAD on a
medium-powered 4-CPU server. Re-planning multiple queries on 100+
sessions every few minutes would not be good.

It seems a reasonable working assumption that HOT will reduce that
requirement considerably, but its something to watch. Thanks for drawing
attention  to it.

Presumably ANALYZE would have the same effect?

--  Simon Riggs              EnterpriseDB   http://www.enterprisedb.com




Re: Plan invalidation design

From
Alvaro Herrera
Date:
Simon Riggs wrote:
> On Sat, 2007-02-17 at 12:48 -0500, Tom Lane wrote:
> 
> > Relcache inval casts a fairly wide net; for example, adding or dropping an
> > index will invalidate all plans using the index's table whether or not
> > they used that particular index, and I believe that VACUUM will also
> > result in a relcache inval due to updating the table's pg_class row.
> > I think this is a good thing though --- for instance, after adding an
> > index it seems a good idea to replan to see if the new index is useful,
> > and replanning after a VACUUM is useful if the table has changed size
> > enough to warrant a different plan.  OTOH this might mean that plans on a
> > high-update-traffic table never survive very long because of autovacuum's
> > efforts.  If that proves to be a problem in practice we can look at ways
> > to dial down the number of replans, but for the moment I think it's more
> > important to be sure we *can* replan at need than to find ways to avoid
> > replans.
> 
> Just some info on that: In an update-intensive scenario, I'm seeing
> VACUUMs every 2 minutes on the heaviest hit tables on CVS HEAD on a
> medium-powered 4-CPU server. Re-planning multiple queries on 100+
> sessions every few minutes would not be good.

I would think the inval would be sent if relpages changed by more than a
certain threshold, say 10%.  In steady state, a high-update table that's
under continuous vacuum should not change size much, thus no replan.

But clearly the point here is to get the inval to be sent at all, and
look for inhibitions mechanisms later.

> Presumably ANALYZE would have the same effect?

It would be nice to have a way to calculate a delta from the previous
statistics snapshot and send an inval if it's appropriate.  Can it be
done?

-- 
Alvaro Herrera                                http://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.


Re: Plan invalidation design

From
"Andrew Hammond"
Date:
On Feb 18, 9:35 am, t...@sss.pgh.pa.us (Tom Lane) wrote:
> Russell Smith <mr-r...@pws.com.au> writes:
>
> > If you replan and immutable function, aren't you possibly messing up a
> > functional index that is using the old function.  Hey, if you change an
> > immutable function that has an index, you are in trouble already.
>
> True.

While I agree that if you change an immutable function used by an
index, your index will break, I do not understand how re-planning it
will cause problems. Is the worry that the index will not pick up on
the new plan?

Andrew