BUG #19748: tsvectorrecv accepts duplicate lexemes and position 0 that tsvectorin rejects - Mailing list pgsql-bugs

The following bug has been logged on the website:

Bug reference:      19748
Logged by:          Ke
Email address:      kehan5800@gmail.com
PostgreSQL version: 18.6
Operating system:   Ubuntu 22.04.2 x86_64
Description:

tsvectorrecv() (src/backend/utils/adt/tsvector.c:446) has this comment
(tsvector.c:552-556):

    * Enforce that datalen is still within MAXSTRPOS, ie the last lexeme
    * didn't go past that.  We could allow that, since no "pos" field
    * overflowed, but tsvectorrecv shouldn't accept values that other
    * tsvector-constructing routines wouldn't.

Two checks are still missing relative to tsvectorin():

1. Duplicate lexemes are not merged. tsvectorin() runs uniqueentry()
   (tsvector.c:265). tsvectorrecv() detects "compareentry(...) <= 0"
   (tsvector.c:514-517), but only sorts (tsvector.c:563); equal entries
   stay. The result is a tsvector with repeated lexemes, which no other
   constructor produces, and which silently changes on a text round trip.

2. The first position is never checked. tsvectorin() rejects position 0
   ("wrong position info in tsvector", tsvector_parser.c:334); the receive
   loop only checks that positions ascend for j > 0 (tsvector.c:540-545),
   so a leading position 0 is accepted.

Self-contained reproduction (bash + psql; the files are COPY BINARY files
with one tsvector column):

    psql -c "CREATE TABLE tv (v tsvector)"

    # one row: tsvector with 2 entries, 'a' (no positions), 'a' (no
positions)
    printf
'PGCOPY\n\377\r\n\0\0\0\0\0\0\0\0\0\0\001\0\0\0\014\0\0\0\002a\0\0\0a\0\0\0\377\377'
> dup.bin
    # one row: tsvector with 1 entry, 'a' with one position, 0
    printf
'PGCOPY\n\377\r\n\0\0\0\0\0\0\0\0\0\0\001\0\0\0\012\0\0\0\001a\0\0\001\0\0\377\377'
> pos0.bin

    psql -c "\copy tv from 'dup.bin' with (format binary)"     -- COPY 1
    psql -c "\copy tv from 'pos0.bin' with (format binary)"    -- COPY 1

    SELECT v, length(v), pg_input_is_valid(v::text, 'tsvector') FROM tv;
        v    | length | pg_input_is_valid
    ---------+--------+-------------------
     'a':0   |      1 | f
     'a' 'a' |      2 | t

    SELECT v, length(v), v::text::tsvector AS reparsed,
           length(v::text::tsvector), v = v::text::tsvector AS same
    FROM tv WHERE length(v) = 2;
        v    | length | reparsed | length | same
    ---------+--------+----------+--------+------
     'a' 'a' |      2 | 'a'      |      1 | f

    SELECT v::text::tsvector FROM tv WHERE length(v) = 1;
    ERROR:  wrong position info in tsvector: "'a':0"

Consequences through pg_dump (plain format, psql restore), from the
attached transcript:

    duplicates: 'a' 'a' -> 'a', 'a':1 'a':2 -> 'a':1,2, 'a' 'a' 'b' -> 'a'
'b';
                restore reports no error and length(v) changes
    position 0: ERROR: wrong position info in tsvector: "'a':0";
                0 of 2 rows of that table restored

Expected: tsvectorrecv() rejects (or normalises, as tsvectorin() does for
duplicates) values that tsvectorin() would not produce, as its own comment
says it should.

Actual: both are accepted; the duplicate case later changes silently on a
text dump/restore, and the position-0 case makes the dump unrestorable.

Suggested fix:
  - after the qsort in tsvectorrecv(), merge adjacent equal entries the way
    uniqueentry() does (merging and de-duplicating their position lists),
    or reject them with ERRCODE_INVALID_BINARY_REPRESENTATION;
  - in the position loop, also reject WEP_GETPOS(wepptr[0]) == 0.
The existing elog(ERROR, ...) calls in tsvectorrecv() could at the same
time become ereport(ERROR, errcode(ERRCODE_INVALID_BINARY_REPRESENTATION),
...), since they are reachable from client input (COPY ... FORMAT binary,
binary-format parameters).