Re: TID Range Scans can return wrong results after scanning backwards - Mailing list pgsql-bugs

From Ewan Young
Subject Re: TID Range Scans can return wrong results after scanning backwards
Date
Msg-id CAON2xHNLx+ds7xfp6-Gdz7Nr+grLKe9CGYR8uJdTbVY-_Kq9tg@mail.gmail.com
Whole thread
In response to TID Range Scans can return wrong results after scanning backwards  (Jeff Davis <pgsql@j-davis.com>)
List pgsql-bugs
Hi,

Thanks for the report!
I can reproduce this on master and on REL_19_STABLE (12 rows instead
of 32, as in your example), and the diagnosis matches what I see:
heapgettup_advance_block() spends rs_numblocks as a budget in both
directions, so a scan that reverses mid-way runs out early.  It goes
back to when TID Range Scans made bidirectional limited scans possible
(v14); the budget logic itself is older, from the BRIN work, and the
2021 fix for the backward *start* block (16dfe253e31) didn't consider
direction changes.

The attached patch treats the limit as a fixed window of rs_numblocks
blocks starting at rs_startblock instead of a budget: the forward
direction ends when it reaches the block just past the window, and the
backward direction already ends at rs_startblock, since
heapgettup_initial_block() starts a backward scan at the window's last
block.  rs_numblocks is no longer modified during the scan.  TID Range
Scans are the only caller that scans a limited range in both
directions, so other heap_setscanlimits() users see no change.

Besides your example, I ran a randomized comparison of SCROLL cursor
sequences (random MOVE/FETCH FORWARD/BACKWARD/ALL/ABSOLUTE/RELATIVE
over random ctid ranges) between the TID Range Scan and a seqscan of
the same query: 135 of 1500 sequences differ on unpatched master, none
with the patch.

On Thu, Sep 17, 2026 at 4:07 AM Jeff Davis <pgsql@j-davis.com> wrote:
>
> AI reported a bug (appended to this email) where TID Range Scans can
> return the wrong results after changing the direction of the scan.
>
> Regards,
>         Jeff Davis
>
>
>
> SQL repro
> ---------
>
>     CREATE TABLE t (id int, data text) WITH (fillfactor = 10);
>     -- 5 rows per page, 20 pages: ctids (0,1) .. (19,5)
>     INSERT INTO t SELECT i, repeat('x', 100)
>       FROM generate_series(1, 100) i;
>     SET enable_seqscan = off;
>     BEGIN;
>     DECLARE c SCROLL CURSOR FOR
>       SELECT ctid FROM t WHERE ctid >= '(2,1)' AND ctid <= '(11,5)';
>     MOVE FORWARD 35 c;      -- cursor is now on (8,5)
>     FETCH BACKWARD 2 c;     -- (8,4), (8,3): correct
>     FETCH BACKWARD ALL c;   -- should return 32 rows, returns 12
>     COMMIT;
>
>
> Diagnosis
> ---------
>
> heap_setscanlimits() stores the number of blocks to scan in
> rs_numblocks. heapgettup_advance_block() then treats that field as a
> remaining budget: it decrements it once for every block it hands out,
> in either direction, and ends the scan when it reaches zero.  That
> model only holds for a scan that moves in one direction and consumes
> exactly the blocks it requests.  A change in direction breaks that.
>
>
>
>
>


--
Regards,
Ewan Young

Attachment

pgsql-bugs by date:

Previous
From: Daniel Gustafsson
Date:
Subject: Re: Postmaster crashes on SIGHUP when oauth_validator_libraries holds only whitespace
Next
From: PG Bug reporting form
Date:
Subject: BUG #19692: Generic partition-pruning plan delays statement_timeout cancellation