pgsql: Fix lexing of standard multi-character operators in edge cases. - Mailing list pgsql-committers

From Andrew Gierth
Subject pgsql: Fix lexing of standard multi-character operators in edge cases.
Date
Msg-id E1fswXG-0001On-Qg@gemulon.postgresql.org
Whole thread Raw
List pgsql-committers
Fix lexing of standard multi-character operators in edge cases.

Commits c6b3c939b (which fixed the precedence of >=, <=, <> operators)
and 865f14a2d (which added support for the standard => notation for
named arguments) created a class of lexer tokens which look like
multi-character operators but which have their own token IDs distinct
from Op. However, longest-match rules meant that following any of
these tokens with another operator character, as in (1<>-1), would
cause them to be incorrectly returned as Op.

The error here isn't immediately obvious, because the parser would
usually still find the correct operator via the Op token, but there
were more subtle problems:

1. If immediately followed by a comment or +-, >= <= <> would be given
   the old precedence of Op rather than the correct new precedence;

2. If followed by a comment, != would be returned as Op rather than as
   NOT_EQUAL, causing it not to be found at all;

3. If followed by a comment or +-, the => token for named arguments
   would be lexed as Op, causing the argument to be mis-parsed as a
   simple expression, usually causing an error.

Fix by explicitly checking for the operators in the {operator} code
block in addition to all the existing special cases there.

Backpatch to 9.5 where the problem was introduced.

Analysis and patch by me; review by Tom Lane.
Discussion: https://postgr.es/m/87va851ppl.fsf@news-spur.riddles.org.uk

Branch
------
REL9_5_STABLE

Details
-------
https://git.postgresql.org/pg/commitdiff/af988d13012f899631d41ab5120e36d8069b1444

Modified Files
--------------
src/backend/parser/scan.l                     | 28 ++++++++++
src/bin/psql/psqlscan.l                       |  9 ++++
src/interfaces/ecpg/preproc/pgc.l             | 28 ++++++++++
src/test/regress/expected/create_operator.out | 74 +++++++++++++++++++++++++++
src/test/regress/expected/polymorphism.out    | 36 +++++++++++++
src/test/regress/sql/create_operator.sql      | 31 +++++++++++
src/test/regress/sql/polymorphism.sql         | 15 ++++++
7 files changed, 221 insertions(+)


pgsql-committers by date:

Previous
From: Tom Lane
Date:
Subject: pgsql: In libpq, don't look up all the hostnames at once.
Next
From: Andres Freund
Date:
Subject: pgsql: Deduplicate code between slot_getallattrs() andslot_getsomeattr