Hi,
On 2025-07-18 23:25:38 -0400, Peter Geoghegan wrote:
> On Fri, Jul 18, 2025 at 10:47 PM Andres Freund <andres@anarazel.de> wrote:
> > > (Within an index AM, there is a 1:1 correspondence between batches and leaf
> > > pages, and batches need to hold on to a leaf page buffer pin for a
> > > time. None of this should really matter to the table AM.)
> >
> > To some degree the table AM will need to care about the index level batching -
> > we have to be careful about how many pages we keep pinned overall. Which is
> > something that both the table and the index AM have some influence over.
>
> Can't they operate independently?
I'm somewhat doubtful. Read stream is careful to limit how many things it
pins, lest we get errors about having too many buffers pinned. Somehow the
number of pins held within the index needs to be limited too, and how much
that needs to be limited depends on how many buffers are pinned in the read
stream :/
> > > At a high level, the table AM (and/or its read stream) asks for so
> > > many heap blocks/TIDs. Occasionally, index AM implementation details
> > > (i.e. the fact that many index leaf pages have to be read to get very
> > > few TIDs) will result in that request not being honored. The interface
> > > that the table AM uses must therefore occasionally answer "I'm sorry,
> > > I can only reasonably give you so many TIDs at this time". When that
> > > happens, the table AM has to make do. That can be very temporary, or
> > > it can happen again and again, depending on implementation details
> > > known only to the index AM side (though typically it'll never happen
> > > even once).
> >
> > I think that requirement will make things more complicated. Why do we need to
> > have it?
>
> What if it turns out that there is a large run of contiguous leaf
> pages that contain no more than 2 or 3 matching index tuples?
I think that's actually likely a case where you want *deeper* prefetching, as
it makes it more likely that the table tuples are on different pages, i.e. you
need a lot more in-flight IOs to avoid stalling on IO.
> What if there's no matches across many leaf pages?
We don't need to keep leaf nodes without matches pinned in that case, so I
don't think there's really an issue?
Greetings,
Andres Freund
