Thread: Selectivity estimation for intarray

Selectivity estimation for intarray

From
Alexander Korotkov
Date:
Hackers,

currently built-in &&, @>, <@ array operators have selectivity estimations while same operator in intarray contrib haven't them. Problem is that operators in intarray contrib still use contsel and contjoinsel functions for selectivity estimation even when built-in operators receive their specific selectivity estimations.

Attached patch adds selectivity estimation functions to  &&, @>, <@ operators in intarray contrib. In order to have less code duplication they are just wrappers over built-in selectivity estimation functions.

However, I faced a problem of migration scripts. Currently, ALTER OPERATOR can only change owner and schema of operator not operator parameters themselves. Change pg_operator directly is also not an option. At first, it would be kludge. At second, in order to correctly find corresponding operator in pg_operator migration script need to know schema where extension is installed. But it can't refer @extschema@ because extension is relocatable.

My proposal is to let ALTER OPERATOR change restrict and join selectivity functions of the operator. Also it would be useful to be able to change commutator and negator of operator: extension could add commutators and negators in further versions. Any thoughts?

------
With best regards,
Alexander Korotkov.
Attachment

Re: Selectivity estimation for intarray

From
Oleg Bartunov
Date:
Any chance to have this patch in 9.5 ? Many intarray users will be happy.

On Wed, Apr 29, 2015 at 1:48 PM, Alexander Korotkov <aekorotkov@gmail.com> wrote:
Hackers,

currently built-in &&, @>, <@ array operators have selectivity estimations while same operator in intarray contrib haven't them. Problem is that operators in intarray contrib still use contsel and contjoinsel functions for selectivity estimation even when built-in operators receive their specific selectivity estimations.

Attached patch adds selectivity estimation functions to  &&, @>, <@ operators in intarray contrib. In order to have less code duplication they are just wrappers over built-in selectivity estimation functions.

However, I faced a problem of migration scripts. Currently, ALTER OPERATOR can only change owner and schema of operator not operator parameters themselves. Change pg_operator directly is also not an option. At first, it would be kludge. At second, in order to correctly find corresponding operator in pg_operator migration script need to know schema where extension is installed. But it can't refer @extschema@ because extension is relocatable.

My proposal is to let ALTER OPERATOR change restrict and join selectivity functions of the operator. Also it would be useful to be able to change commutator and negator of operator: extension could add commutators and negators in further versions. Any thoughts?

------
With best regards,
Alexander Korotkov.


--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: Selectivity estimation for intarray

From
Tom Lane
Date:
Alexander Korotkov <aekorotkov@gmail.com> writes:
> My proposal is to let ALTER OPERATOR change restrict and join selectivity
> functions of the operator. Also it would be useful to be able to change
> commutator and negator of operator: extension could add commutators and
> negators in further versions. Any thoughts?

I'm pretty dubious about this, because we lack any mechanism for undoing
parser/planner decisions based on operator properties.  And there's quite
a lot of stuff that is based on the assumption that operator properties
will never change.

An example of the pitfalls here is that we can never allow ALTER OPERATOR
RENAME, because for example if you rename '<' to '~<~' that will change
its precedence, and we have no way to fix the parse trees embedded in
stored views to reflect that.

For the specific cases you mention, perhaps it would be all right if we
taught plancache.c to blow away *all* cached plans upon seeing any change
in pg_operator; but that seems like a brute-force solution.
        regards, tom lane



Re: Selectivity estimation for intarray

From
Tom Lane
Date:
Oleg Bartunov <obartunov@gmail.com> writes:
> Any chance to have this patch in 9.5 ? Many intarray users will be happy.

Considering how desperately behind we are on reviewing/committing patches
that were submitted by the deadline, I don't think it would be appropriate
or fair to add on major new patches that came in months late.  Please add
this to the first 9.6 commitfest, instead.
        regards, tom lane



Re: Selectivity estimation for intarray

From
Stephen Frost
Date:
* Tom Lane (tgl@sss.pgh.pa.us) wrote:
> Alexander Korotkov <aekorotkov@gmail.com> writes:
> > My proposal is to let ALTER OPERATOR change restrict and join selectivity
> > functions of the operator. Also it would be useful to be able to change
> > commutator and negator of operator: extension could add commutators and
> > negators in further versions. Any thoughts?
>
> I'm pretty dubious about this, because we lack any mechanism for undoing
> parser/planner decisions based on operator properties.  And there's quite
> a lot of stuff that is based on the assumption that operator properties
> will never change.
>
> An example of the pitfalls here is that we can never allow ALTER OPERATOR
> RENAME, because for example if you rename '<' to '~<~' that will change
> its precedence, and we have no way to fix the parse trees embedded in
> stored views to reflect that.
>
> For the specific cases you mention, perhaps it would be all right if we
> taught plancache.c to blow away *all* cached plans upon seeing any change
> in pg_operator; but that seems like a brute-force solution.

Agreed that it is- but is that really a problem...?  I've not run into
many (any?) systems where pg_operator is getting changed often...  The
worst part would be adding new operators/extensions, but perhaps we
could exclude that specific case from triggering the cache invalidation?
Thanks!
    Stephen

Re: Selectivity estimation for intarray

From
Tom Lane
Date:
Stephen Frost <sfrost@snowman.net> writes:
> * Tom Lane (tgl@sss.pgh.pa.us) wrote:
>> For the specific cases you mention, perhaps it would be all right if we
>> taught plancache.c to blow away *all* cached plans upon seeing any change
>> in pg_operator; but that seems like a brute-force solution.

> Agreed that it is- but is that really a problem...?

Perhaps it isn't; we certainly have assumptions that pg_amop, for
instance, changes seldom enough that it's not worth tracking individual
changes.  The same might be true of pg_operator.  I'm not sure though.

The core point I'm trying to make is that making pg_operator entries
mutable is something that's going to require very careful review.
        regards, tom lane