Thread: [COMMITTERS] pgsql: Support XMLTABLE query expression

[COMMITTERS] pgsql: Support XMLTABLE query expression

From
Alvaro Herrera
Date:
Support XMLTABLE query expression

XMLTABLE is defined by the SQL/XML standard as a feature that allows
turning XML-formatted data into relational form, so that it can be used
as a <table primary> in the FROM clause of a query.

This new construct provides significant simplicity and performance
benefit for XML data processing; what in a client-side custom
implementation was reported to take 20 minutes can be executed in 400ms
using XMLTABLE.  (The same functionality was said to take 10 seconds
using nested PostgreSQL XPath function calls, and 5 seconds using
XMLReader under PL/Python).

The implemented syntax deviates slightly from what the standard
requires.  First, the standard indicates that the PASSING clause is
optional and that multiple XML input documents may be given to it; we
make it mandatory and accept a single document only.  Second, we don't
currently support a default namespace to be specified.

This implementation relies on a new executor node based on a hardcoded
method table.  (Because the grammar is fixed, there is no extensibility
in the current approach; further constructs can be implemented on top of
this such as JSON_TABLE, but they require changes to core code.)

Author: Pavel Stehule, Álvaro Herrera
Extensively reviewed by: Craig Ringer
Discussion: https://postgr.es/m/CAFj8pRAgfzMD-LoSmnMGybD0WsEznLHWap8DO79+-GTRAPR4qA@mail.gmail.com

Branch
------
master

Details
-------
http://git.postgresql.org/pg/commitdiff/fcec6caafa2346b6c9d3ad5065e417733bd63cd9

Modified Files
--------------
contrib/pg_stat_statements/pg_stat_statements.c |  12 +
doc/src/sgml/func.sgml                          | 203 ++++++++-
src/backend/commands/explain.c                  |  24 +
src/backend/executor/Makefile                   |   3 +-
src/backend/executor/execAmi.c                  |   6 +
src/backend/executor/execProcnode.c             |  14 +
src/backend/executor/nodeTableFuncscan.c        | 502 +++++++++++++++++++++
src/backend/nodes/copyfuncs.c                   |  91 ++++
src/backend/nodes/equalfuncs.c                  |  60 +++
src/backend/nodes/makefuncs.c                   |   8 +-
src/backend/nodes/nodeFuncs.c                   |  66 +++
src/backend/nodes/outfuncs.c                    |  73 ++++
src/backend/nodes/print.c                       |   4 +
src/backend/nodes/readfuncs.c                   |  47 ++
src/backend/optimizer/path/allpaths.c           |  37 ++
src/backend/optimizer/path/costsize.c           |  81 ++++
src/backend/optimizer/plan/createplan.c         |  77 +++-
src/backend/optimizer/plan/initsplan.c          |   2 +
src/backend/optimizer/plan/planner.c            |  37 +-
src/backend/optimizer/plan/setrefs.c            |  14 +
src/backend/optimizer/plan/subselect.c          |   6 +
src/backend/optimizer/prep/prepjointree.c       |   6 +
src/backend/optimizer/util/pathnode.c           |  26 ++
src/backend/optimizer/util/plancat.c            |   6 +-
src/backend/optimizer/util/relnode.c            |   5 +-
src/backend/parser/analyze.c                    |   9 +
src/backend/parser/gram.y                       | 189 +++++++-
src/backend/parser/parse_clause.c               | 245 ++++++++++-
src/backend/parser/parse_coerce.c               |  33 +-
src/backend/parser/parse_relation.c             |  79 +++-
src/backend/parser/parse_target.c               |   7 +
src/backend/rewrite/rewriteHandler.c            |   4 +
src/backend/utils/adt/ruleutils.c               | 133 +++++-
src/backend/utils/adt/xml.c                     | 558 +++++++++++++++++++++++-
src/include/catalog/catversion.h                |   2 +-
src/include/executor/nodeTableFuncscan.h        |  24 +
src/include/executor/tablefunc.h                |  67 +++
src/include/nodes/execnodes.h                   |  25 ++
src/include/nodes/nodes.h                       |   5 +
src/include/nodes/parsenodes.h                  |  39 ++
src/include/nodes/plannodes.h                   |  10 +
src/include/nodes/primnodes.h                   |  22 +
src/include/optimizer/cost.h                    |   5 +
src/include/optimizer/pathnode.h                |   4 +
src/include/parser/kwlist.h                     |   3 +
src/include/parser/parse_coerce.h               |   4 +
src/include/parser/parse_relation.h             |   5 +
src/include/utils/xml.h                         |   3 +
src/test/regress/expected/xml.out               | 504 +++++++++++++++++++++
src/test/regress/expected/xml_1.out             | 475 ++++++++++++++++++++
src/test/regress/expected/xml_2.out             | 504 +++++++++++++++++++++
src/test/regress/sql/xml.sql                    | 288 ++++++++++++
52 files changed, 4606 insertions(+), 50 deletions(-)


Re: [COMMITTERS] pgsql: Support XMLTABLE query expression

From
Tom Lane
Date:
Alvaro Herrera <alvherre@alvh.no-ip.org> writes:
> Support XMLTABLE query expression

This fails immediately on RHEL6:

$ ./configure --with-libxml [ other usual arguments ]
$ make -s
xml.c: In function 'XmlTableGetValue':
xml.c:4434: warning: implicit declaration of function 'xmlXPathSetContextNode'
utils/adt/xml.o: In function `XmlTableGetValue':
/home/postgres/pgsql/src/backend/utils/adt/xml.c:4434: undefined reference to `xmlXPathSetContextNode'
collect2: ld returned 1 exit status
make[2]: *** [postgres] Error 1
make[1]: *** [all-backend-recurse] Error 2
make: *** [all-src-recurse] Error 2

Relevant package versions are

libxml2-2.7.6-21.el6_8.1.x86_64
libxml2-devel-2.7.6-21.el6_8.1.x86_64
xml-common-0.6.3-33.el6.noarch

            regards, tom lane


Re: [COMMITTERS] pgsql: Support XMLTABLE query expression

From
Alvaro Herrera
Date:
Tom Lane wrote:
> Alvaro Herrera <alvherre@alvh.no-ip.org> writes:
> > Support XMLTABLE query expression
>
> This fails immediately on RHEL6:

Yeah, i noticed on termite as well.  Apparently xmlXPathSetContextNode
appeared in 2.9.1 only, which was released in 2013.  :-(

The easiest for now is to disable XMLTABLE completely with older
libxml2.

--
Álvaro Herrera                https://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services


Re: [COMMITTERS] pgsql: Support XMLTABLE query expression

From
Tom Lane
Date:
Alvaro Herrera <alvherre@2ndquadrant.com> writes:
> Tom Lane wrote:
>> This fails immediately on RHEL6:

> Yeah, i noticed on termite as well.  Apparently xmlXPathSetContextNode
> appeared in 2.9.1 only, which was released in 2013.  :-(

> The easiest for now is to disable XMLTABLE completely with older
> libxml2.

That does not sound like an acceptable solution.  These aren't
obsolete platforms we're talking about.

            regards, tom lane


Re: [COMMITTERS] pgsql: Support XMLTABLE query expression

From
Alvaro Herrera
Date:
Tom Lane wrote:
> Alvaro Herrera <alvherre@2ndquadrant.com> writes:
> > Tom Lane wrote:
> >> This fails immediately on RHEL6:
>
> > Yeah, i noticed on termite as well.  Apparently xmlXPathSetContextNode
> > appeared in 2.9.1 only, which was released in 2013.  :-(
>
> > The easiest for now is to disable XMLTABLE completely with older
> > libxml2.
>
> That does not sound like an acceptable solution.  These aren't
> obsolete platforms we're talking about.

Apparently, it's a one line change to support that.  I'm testing it now.

--
Álvaro Herrera                https://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services