Thread: pgsql: Implement operator class parameters
Implement operator class parameters PostgreSQL provides set of template index access methods, where opclasses have much freedom in the semantics of indexing. These index AMs are GiST, GIN, SP-GiST and BRIN. There opclasses define representation of keys, operations on them and supported search strategies. So, it's natural that opclasses may be faced some tradeoffs, which require user-side decision. This commit implements opclass parameters allowing users to set some values, which tell opclass how to index the particular dataset. This commit doesn't introduce new storage in system catalog. Instead it uses pg_attribute.attoptions, which is used for table column storage options but unused for index attributes. In order to evade changing signature of each opclass support function, we implement unified way to pass options to opclass support functions. Options are set to fn_expr as the constant bytea expression. It's possible due to the fact that opclass support functions are executed outside of expressions, so fn_expr is unused for them. This commit comes with some examples of opclass options usage. We parametrize signature length in GiST. That applies to multiple opclasses: tsvector_ops, gist__intbig_ops, gist_ltree_ops, gist__ltree_ops, gist_trgm_ops and gist_hstore_ops. Also we parametrize maximum number of integer ranges for gist__int_ops. However, the main future usage of this feature is expected to be json, where users would be able to specify which way to index particular json parts. Catversion is bumped. Discussion: https://postgr.es/m/d22c3a18-31c7-1879-fc11-4c1ce2f5e5af%40postgrespro.ru Author: Nikita Glukhov, revised by me Reviwed-by: Nikolay Shaplov, Robert Haas, Tom Lane, Tomas Vondra, Alvaro Herrera Branch ------ master Details ------- https://git.postgresql.org/pg/commitdiff/911e70207703799605f5a0e8aad9f06cff067c63 Modified Files -------------- contrib/bloom/bloom.h | 3 +- contrib/bloom/blutils.c | 1 + contrib/bloom/blvalidate.c | 5 + contrib/hstore/Makefile | 1 + contrib/hstore/expected/hstore.out | 45 + contrib/hstore/hstore--1.6--1.7.sql | 12 + contrib/hstore/hstore.control | 2 +- contrib/hstore/hstore_gist.c | 192 ++-- contrib/hstore/sql/hstore.sql | 13 + contrib/intarray/Makefile | 3 +- contrib/intarray/_int.h | 45 +- contrib/intarray/_int_bool.c | 34 +- contrib/intarray/_int_gist.c | 30 +- contrib/intarray/_int_tool.c | 4 +- contrib/intarray/_intbig_gist.c | 189 ++-- contrib/intarray/expected/_int.out | 160 +++ contrib/intarray/intarray--1.2--1.3.sql | 20 + contrib/intarray/intarray.control | 2 +- contrib/intarray/sql/_int.sql | 36 + contrib/ltree/Makefile | 2 +- contrib/ltree/_ltree_gist.c | 182 ++-- contrib/ltree/expected/ltree.out | 154 +++ contrib/ltree/ltree--1.1--1.2.sql | 21 + contrib/ltree/ltree.control | 2 +- contrib/ltree/ltree.h | 54 +- contrib/ltree/ltree_gist.c | 257 ++--- contrib/ltree/sql/ltree.sql | 35 + contrib/pg_trgm/Makefile | 2 +- contrib/pg_trgm/expected/pg_trgm.out | 1157 ++++++++++++++++++++++ contrib/pg_trgm/pg_trgm--1.4--1.5.sql | 12 + contrib/pg_trgm/pg_trgm.control | 2 +- contrib/pg_trgm/sql/pg_trgm.sql | 14 + contrib/pg_trgm/trgm.h | 17 +- contrib/pg_trgm/trgm_gist.c | 227 +++-- doc/src/sgml/hstore.sgml | 17 + doc/src/sgml/indices.sgml | 2 +- doc/src/sgml/intarray.sgml | 25 +- doc/src/sgml/ltree.sgml | 37 +- doc/src/sgml/pgtrgm.sgml | 17 + doc/src/sgml/ref/create_index.sgml | 16 +- doc/src/sgml/textsearch.sgml | 13 +- src/backend/access/brin/brin.c | 1 + src/backend/access/brin/brin_validate.c | 3 + src/backend/access/common/reloptions.c | 488 +++++++-- src/backend/access/gin/ginutil.c | 1 + src/backend/access/gin/ginvalidate.c | 6 +- src/backend/access/gist/gist.c | 1 + src/backend/access/gist/gistvalidate.c | 6 +- src/backend/access/hash/hash.c | 1 + src/backend/access/hash/hashvalidate.c | 4 + src/backend/access/index/amvalidate.c | 11 + src/backend/access/index/indexam.c | 77 +- src/backend/access/nbtree/nbtree.c | 1 + src/backend/access/nbtree/nbtvalidate.c | 3 + src/backend/access/spgist/spgvalidate.c | 5 + src/backend/catalog/heap.c | 8 +- src/backend/catalog/index.c | 22 +- src/backend/catalog/toasting.c | 1 + src/backend/commands/indexcmds.c | 65 +- src/backend/commands/opclasscmds.c | 55 +- src/backend/commands/tablecmds.c | 2 +- src/backend/nodes/copyfuncs.c | 1 + src/backend/nodes/equalfuncs.c | 1 + src/backend/nodes/makefuncs.c | 3 + src/backend/nodes/outfuncs.c | 1 + src/backend/optimizer/util/plancat.c | 3 + src/backend/parser/gram.y | 60 +- src/backend/parser/parse_utilcmd.c | 8 + src/backend/utils/adt/ruleutils.c | 135 ++- src/backend/utils/adt/selfuncs.c | 23 +- src/backend/utils/adt/tsgistidx.c | 274 ++--- src/backend/utils/cache/lsyscache.c | 35 + src/backend/utils/cache/relcache.c | 143 ++- src/backend/utils/fmgr/fmgr.c | 53 + src/include/access/amapi.h | 2 + src/include/access/amvalidate.h | 1 + src/include/access/brin_internal.h | 1 + src/include/access/genam.h | 3 + src/include/access/gin.h | 3 +- src/include/access/gist.h | 22 +- src/include/access/hash.h | 3 +- src/include/access/nbtree.h | 3 +- src/include/access/reloptions.h | 47 + src/include/access/spgist.h | 3 +- src/include/catalog/catversion.h | 2 +- src/include/catalog/heap.h | 1 + src/include/catalog/pg_amproc.dat | 3 + src/include/catalog/pg_proc.dat | 3 + src/include/fmgr.h | 7 + src/include/nodes/execnodes.h | 2 + src/include/nodes/parsenodes.h | 1 + src/include/nodes/pathnodes.h | 1 + src/include/utils/lsyscache.h | 1 + src/include/utils/rel.h | 1 + src/include/utils/relcache.h | 3 + src/include/utils/ruleutils.h | 1 + src/test/regress/expected/alter_generic.out | 18 +- src/test/regress/expected/btree_index.out | 3 + src/test/regress/expected/opr_sanity.out | 2 +- src/test/regress/expected/tsearch.out | 176 ++++ src/test/regress/input/create_function_1.source | 5 + src/test/regress/output/create_function_1.source | 4 + src/test/regress/regress.c | 7 + src/test/regress/sql/alter_generic.sql | 11 +- src/test/regress/sql/btree_index.sql | 3 + src/test/regress/sql/opr_sanity.sql | 2 +- src/test/regress/sql/tsearch.sql | 45 + src/tools/pgindent/typedefs.list | 11 + 108 files changed, 4063 insertions(+), 901 deletions(-)
Hi, On 2020-03-30 16:17:53 +0000, Alexander Korotkov wrote: > Implement operator class parameters > > PostgreSQL provides set of template index access methods, where opclasses have > much freedom in the semantics of indexing. These index AMs are GiST, GIN, > SP-GiST and BRIN. There opclasses define representation of keys, operations on > them and supported search strategies. So, it's natural that opclasses may be > faced some tradeoffs, which require user-side decision. This commit implements > opclass parameters allowing users to set some values, which tell opclass how to > index the particular dataset. > > This commit doesn't introduce new storage in system catalog. Instead it uses > pg_attribute.attoptions, which is used for table column storage options but > unused for index attributes. > > In order to evade changing signature of each opclass support function, we > implement unified way to pass options to opclass support functions. Options > are set to fn_expr as the constant bytea expression. It's possible due to the > fact that opclass support functions are executed outside of expressions, so > fn_expr is unused for them. > > This commit comes with some examples of opclass options usage. We parametrize > signature length in GiST. That applies to multiple opclasses: tsvector_ops, > gist__intbig_ops, gist_ltree_ops, gist__ltree_ops, gist_trgm_ops and > gist_hstore_ops. Also we parametrize maximum number of integer ranges for > gist__int_ops. However, the main future usage of this feature is expected > to be json, where users would be able to specify which way to index particular > json parts. > > Catversion is bumped. > > Discussion: https://postgr.es/m/d22c3a18-31c7-1879-fc11-4c1ce2f5e5af%40postgrespro.ru > Author: Nikita Glukhov, revised by me > Reviwed-by: Nikolay Shaplov, Robert Haas, Tom Lane, Tomas Vondra, Alvaro Herrera This triggers a few new (harmless) warnings for me: In file included from /home/andres/src/postgresql/src/include/postgres.h:46, from /home/andres/src/postgresql/src/backend/access/index/indexam.c:44: /home/andres/src/postgresql/src/backend/access/index/indexam.c: In function ‘index_getprocid’: /home/andres/src/postgresql/src/backend/access/index/indexam.c:782:17: warning: comparison is always true due to limitedrange of data type [-Wtype-limits] 782 | Assert(procnum >= 0 && procnum <= (uint16) nproc); | ^~ /home/andres/src/postgresql/src/backend/access/index/indexam.c:782:2: note: in expansion of macro ‘Assert’ 782 | Assert(procnum >= 0 && procnum <= (uint16) nproc); | ^~~~~~ /home/andres/src/postgresql/src/backend/access/index/indexam.c: In function ‘index_getprocinfo’: /home/andres/src/postgresql/src/backend/access/index/indexam.c:818:17: warning: comparison is always true due to limitedrange of data type [-Wtype-limits] 818 | Assert(procnum >= 0 && procnum <= (uint16) nproc); | ^~ /home/andres/src/postgresql/src/include/c.h:782:9: note: in definition of macro ‘Assert’ 782 | if (!(condition)) \ | ^~~~~~~~~ PostgreSQL installation complete. Greetings, Andres Freund
On Mon, Mar 30, 2020 at 9:36 PM Andres Freund <andres@anarazel.de> wrote: > This triggers a few new (harmless) warnings for me: > In file included from /home/andres/src/postgresql/src/include/postgres.h:46, > from /home/andres/src/postgresql/src/backend/access/index/indexam.c:44: > /home/andres/src/postgresql/src/backend/access/index/indexam.c: In function ‘index_getprocid’: > /home/andres/src/postgresql/src/backend/access/index/indexam.c:782:17: warning: comparison is always true due to limitedrange of data type [-Wtype-limits] > 782 | Assert(procnum >= 0 && procnum <= (uint16) nproc); > | ^~ > /home/andres/src/postgresql/src/backend/access/index/indexam.c:782:2: note: in expansion of macro ‘Assert’ > 782 | Assert(procnum >= 0 && procnum <= (uint16) nproc); > | ^~~~~~ > /home/andres/src/postgresql/src/backend/access/index/indexam.c: In function ‘index_getprocinfo’: > /home/andres/src/postgresql/src/backend/access/index/indexam.c:818:17: warning: comparison is always true due to limitedrange of data type [-Wtype-limits] > 818 | Assert(procnum >= 0 && procnum <= (uint16) nproc); > | ^~ > /home/andres/src/postgresql/src/include/c.h:782:9: note: in definition of macro ‘Assert’ > 782 | if (!(condition)) \ > | ^~~~~~~~~ > PostgreSQL installation complete. Thank you! This reveals to me that this commit contains rudiments of allowing procnum == 0. I'll commit the fix soon. BTW, what version of compiler (and options) do you use? At the first glance gcc and clang don't show these warnings to me. ------ Alexander Korotkov Postgres Professional: http://www.postgrespro.com The Russian Postgres Company
Hi, On 2020-03-30 23:11:06 +0300, Alexander Korotkov wrote: > BTW, what version of compiler (and options) do you use? At the first > glance gcc and clang don't show these warnings to me. In this case it was a snapshot of gcc-10, but I think it'd have been there with older compilers too. The warning is enabled by -Wextra (or the more specific -Wtype-limits of course). Regards, Andres