Re: BUG: ReadStream look-ahead exhausts local buffers when effective_io_concurrency>=64 - Mailing list pgsql-hackers

From Melanie Plageman
Subject Re: BUG: ReadStream look-ahead exhausts local buffers when effective_io_concurrency>=64
Date
Msg-id CAAKRu_ZNDH0=RZ1mCQi2jT0UbHCNK2iAD+P0om+fgHOawPYpQA@mail.gmail.com
Whole thread
In response to Re: BUG: ReadStream look-ahead exhausts local buffers when effective_io_concurrency>=64  (Xuneng Zhou <xunengzhou@gmail.com>)
Responses Re: BUG: ReadStream look-ahead exhausts local buffers when effective_io_concurrency>=64
List pgsql-hackers
On Thu, Jul 2, 2026 at 7:50 PM Xuneng Zhou <xunengzhou@gmail.com> wrote:
>
> Thanks for reporting this issue. It smells similar to the bug reported
> by Alexander earlier. [1] The root cause of them seems the same: we
> give read stream too much budget for local buffer pins. The
> fix(da6874635db by Melanie) is to cut the budget to 1/4.
>
> > Attached is a patch that addresses this by limiting ReadStream's max_pinned_buffers for temp tables to 75% of the
availablelocal buffers. It also introduces a cap on max_ios for temp tables to DEFAULT_EFFECTIVE_IO_CONCURRENCY, to
accountfor multiple sequential scan look-aheads happening simultaneously. 
>
> If that's the case, I'm wondering whether it makes sense to backpatch
> this fix to 18. I tried to do this for the local tree and the
> reproducer passed. That said, it might not be safe to do so for a
> stable version. It would be helpful to hear Melanie's and Andres's
> thoughts on this.

da6874635db fixes the originally reported bug in this thread on 18. It
makes sense to backpatch it to 18. I hadn't thought of the TOAST case.
I'm thinking about whether backpatching to 17 makes sense or not. It
would be much more difficult to hit an issue in 17, but I'll need to
think more about it.

I wonder if in 20, we should move toward Andres' suggested formula
     Max(1, (num_temp_buffers − NLocalPinnedBuffers) / 4)
instead of the existing
     num_temp_buffers / 4 − NLocalPinnedBuffers

Which would mean we got a quarter of available buffers instead of a
hard cap of a quarter of total buffers.
This wouldn't have been any better at solving the issue in this
thread, but I'm wondering if it is just a better formula in general
going forward (not for backpatching).

- Melanie



pgsql-hackers by date:

Previous
From: Nisha Moond
Date:
Subject: Re: Support EXCEPT for TABLES IN SCHEMA publications
Next
From: Antonin Houska
Date:
Subject: Re: Remove redundant DISTINCT when GROUP BY already guarantees uniqueness