Thread: pgsql: Support subscripting of arbitrary types, not only arrays.

pgsql: Support subscripting of arbitrary types, not only arrays.

From
Tom Lane
Date:
Support subscripting of arbitrary types, not only arrays.

This patch generalizes the subscripting infrastructure so that any
data type can be subscripted, if it provides a handler function to
define what that means.  Traditional variable-length (varlena) arrays
all use array_subscript_handler(), while the existing fixed-length
types that support subscripting use raw_array_subscript_handler().
It's expected that other types that want to use subscripting notation
will define their own handlers.  (This patch provides no such new
features, though; it only lays the foundation for them.)

To do this, move the parser's semantic processing of subscripts
(including coercion to whatever data type is required) into a
method callback supplied by the handler.  On the execution side,
replace the ExecEvalSubscriptingRef* layer of functions with direct
calls to callback-supplied execution routines.  (Thus, essentially
no new run-time overhead should be caused by this patch.  Indeed,
there is room to remove some overhead by supplying specialized
execution routines.  This patch does a little bit in that line,
but more could be done.)

Additional work is required here and there to remove formerly
hard-wired assumptions about the result type, collation, etc
of a SubscriptingRef expression node; and to remove assumptions
that the subscript values must be integers.

One useful side-effect of this is that we now have a less squishy
mechanism for identifying whether a data type is a "true" array:
instead of wiring in weird rules about typlen, we can look to see
if pg_type.typsubscript == F_ARRAY_SUBSCRIPT_HANDLER.  For this
to be bulletproof, we have to forbid user-defined types from using
that handler directly; but there seems no good reason for them to
do so.

This patch also removes assumptions that the number of subscripts
is limited to MAXDIM (6), or indeed has any hard-wired limit.
That limit still applies to types handled by array_subscript_handler
or raw_array_subscript_handler, but to discourage other dependencies
on this constant, I've moved it from c.h to utils/array.h.

Dmitry Dolgov, reviewed at various times by Tom Lane, Arthur Zakirov,
Peter Eisentraut, Pavel Stehule

Discussion: https://postgr.es/m/CA+q6zcVDuGBv=M0FqBYX8DPebS3F_0KQ6OVFobGJPM507_SZ_w@mail.gmail.com
Discussion: https://postgr.es/m/CA+q6zcVovR+XY4mfk-7oNk-rF91gH0PebnNfuUjuuDsyHjOcVA@mail.gmail.com

Branch
------
master

Details
-------
https://git.postgresql.org/pg/commitdiff/c7aba7c14efdbd9fc1bb44b4cb83bedee0c6a6fc

Modified Files
--------------
contrib/postgres_fdw/deparse.c            |  15 +-
doc/src/sgml/catalogs.sgml                |  38 +-
doc/src/sgml/ref/create_type.sgml         |  76 +++-
src/backend/catalog/aclchk.c              |   4 +-
src/backend/catalog/dependency.c          |  16 +
src/backend/catalog/heap.c                |   2 +
src/backend/catalog/pg_type.c             |  11 +-
src/backend/commands/typecmds.c           |  95 ++++-
src/backend/executor/execExpr.c           | 150 ++++----
src/backend/executor/execExprInterp.c     | 238 +-----------
src/backend/jit/llvm/llvmjit_expr.c       |  78 ++--
src/backend/jit/llvm/llvmjit_types.c      |   5 +-
src/backend/nodes/copyfuncs.c             |   1 +
src/backend/nodes/equalfuncs.c            |   1 +
src/backend/nodes/nodeFuncs.c             |  11 +-
src/backend/nodes/outfuncs.c              |   1 +
src/backend/nodes/readfuncs.c             |   1 +
src/backend/optimizer/util/clauses.c      |  43 ++-
src/backend/parser/parse_coerce.c         |   5 +-
src/backend/parser/parse_collate.c        |  23 ++
src/backend/parser/parse_expr.c           |   6 +-
src/backend/parser/parse_node.c           | 231 ++++--------
src/backend/parser/parse_target.c         |  48 ++-
src/backend/utils/adt/Makefile            |   1 +
src/backend/utils/adt/arrayfuncs.c        |   6 +-
src/backend/utils/adt/arraysubs.c         | 577 ++++++++++++++++++++++++++++++
src/backend/utils/adt/format_type.c       |   8 +-
src/backend/utils/adt/jsonfuncs.c         |   3 +-
src/backend/utils/cache/lsyscache.c       |  67 +++-
src/backend/utils/cache/typcache.c        |   2 +
src/bin/pg_dump/pg_dump.c                 |  15 +
src/include/c.h                           |   8 -
src/include/catalog/catversion.h          |   2 +-
src/include/catalog/pg_proc.dat           |   8 +
src/include/catalog/pg_type.dat           |  39 +-
src/include/catalog/pg_type.h             |  30 +-
src/include/executor/execExpr.h           |  56 +--
src/include/nodes/primnodes.h             |  42 ++-
src/include/nodes/subscripting.h          | 167 +++++++++
src/include/parser/parse_node.h           |   6 +-
src/include/utils/array.h                 |   5 +
src/include/utils/lsyscache.h             |   6 +
src/include/utils/typcache.h              |   1 +
src/pl/plperl/plperl.c                    |   6 +-
src/pl/plpgsql/src/pl_comp.c              |   4 +-
src/pl/plpython/plpy_typeio.c             |   8 +-
src/test/regress/expected/arrays.out      |   8 +-
src/test/regress/expected/opr_sanity.out  |   6 +-
src/test/regress/expected/type_sanity.out |  43 ++-
src/test/regress/sql/arrays.sql           |   2 +-
src/test/regress/sql/opr_sanity.sql       |   6 +-
src/test/regress/sql/type_sanity.sql      |  32 +-
52 files changed, 1552 insertions(+), 711 deletions(-)


Re: pgsql: Support subscripting of arbitrary types, not only arrays.

From
Alexander Korotkov
Date:
On Wed, Dec 9, 2020 at 8:40 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Support subscripting of arbitrary types, not only arrays.

Nice to see this committed!

BTW, it seems typedefs.list needs to be adjusted.

------
Regards,
Alexander Korotkov

Attachment

Re: pgsql: Support subscripting of arbitrary types, not only arrays.

From
Tom Lane
Date:
Alexander Korotkov <aekorotkov@gmail.com> writes:
> BTW, it seems typedefs.list needs to be adjusted.

There's no particular consensus so far that typedefs.list should be
maintained on-the-fly.  Checking the version in git against what
the buildfarm is reporting shows a *lot* of diffs besides this,
so I'm hardly the only one not updating it.

(I'd be happy to sign onto such maintenance if we were also doing
something to keep HEAD pgindent'd on a more continuous basis.
See prior discussions.)

            regards, tom lane



Re: pgsql: Support subscripting of arbitrary types, not only arrays.

From
Alexander Korotkov
Date:
On Thu, Dec 10, 2020 at 6:14 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Alexander Korotkov <aekorotkov@gmail.com> writes:
> > BTW, it seems typedefs.list needs to be adjusted.
>
> There's no particular consensus so far that typedefs.list should be
> maintained on-the-fly.  Checking the version in git against what
> the buildfarm is reporting shows a *lot* of diffs besides this,
> so I'm hardly the only one not updating it.
>
> (I'd be happy to sign onto such maintenance if we were also doing
> something to keep HEAD pgindent'd on a more continuous basis.
> See prior discussions.)

OK, thank you for pointing!

------
Regards,
Alexander Korotkov