pgsql: Improve selectivity estimation for assorted match-style operator - Mailing list pgsql-committers

From Tom Lane
Subject pgsql: Improve selectivity estimation for assorted match-style operator
Date
Msg-id E1jJeQ4-0002k8-HN@gemulon.postgresql.org
Whole thread Raw
List pgsql-committers
Improve selectivity estimation for assorted match-style operators.

Quite a few matching operators such as JSONB's @> used "contsel" and
"contjoinsel" as their selectivity estimators.  That was a bad idea,
because (a) contsel is only a stub, yielding a fixed default estimate,
and (b) that default is 0.001, meaning we estimate these operators as
five times more selective than equality, which is surely pretty silly.

There's a good model for improving this in ltree's ltreeparentsel():
for any "var OP constant" query, we can try applying the operator
to all of the column's MCV and histogram values, taking the latter
as being a random sample of the non-MCV values.  That code is
actually 100% generic, except for the question of exactly what
default selectivity ought to be plugged in when we don't have stats.

Hence, migrate the guts of ltreeparentsel() into the core code, provide
wrappers "matchingsel" and "matchingjoinsel" with a more-appropriate
default estimate, and use those for the non-geometric operators that
formerly used contsel (mostly JSONB containment operators and tsquery
matching).

Also apply this code to some match-like operators in hstore, ltree, and
pg_trgm, including the former users of ltreeparentsel as well as ones
that improperly used contsel.  Since commit 911e70207 just created new
versions of those extensions that we haven't released yet, we can sneak
this change into those new versions instead of having to create an
additional generation of update scripts.

Patch by me, reviewed by Alexey Bashtanov

Discussion: https://postgr.es/m/12237.1582833074@sss.pgh.pa.us

Branch
------
master

Details
-------
https://git.postgresql.org/pg/commitdiff/a80818605e5447b9b846590c3d3fab99060cb53e

Modified Files
--------------
contrib/hstore/hstore--1.6--1.7.sql   |  15 ++++
contrib/ltree/ltree--1.1--1.2.sql     |  81 +++++++++++++++++
contrib/ltree/ltree_op.c              | 102 ++--------------------
contrib/pg_trgm/expected/pg_trgm.out  |  19 ++--
contrib/pg_trgm/pg_trgm--1.4--1.5.sql |  11 +++
doc/src/sgml/xoper.sgml               |  13 +++
src/backend/utils/adt/selfuncs.c      | 160 ++++++++++++++++++++++++++++++++++
src/include/catalog/catversion.h      |   2 +-
src/include/catalog/pg_operator.dat   |  35 ++++----
src/include/catalog/pg_proc.dat       |   9 ++
src/include/utils/selfuncs.h          |   6 ++
11 files changed, 332 insertions(+), 121 deletions(-)


pgsql-committers by date:

Previous
From: Peter Eisentraut
Date:
Subject: pgsql: Refactor code to look up local replication tuple
Next
From: Tom Lane
Date:
Subject: pgsql: Check equality semantics for unique indexes on partitioned table