Re: index prefetching - Mailing list pgsql-hackers

From Andres Freund
Subject Re: index prefetching
Date
Msg-id qlgap72f3diyspc55ser4ydoxhipnblvxsmb2os7ucvcccurg6@sli433mqd7xl
Whole thread Raw
In response to Re: index prefetching  (Tomas Vondra <tomas@vondra.me>)
List pgsql-hackers
Hi,

On 2025-08-26 17:06:11 +0200, Tomas Vondra wrote:
> On 8/26/25 01:48, Andres Freund wrote:
> > Hi,
> > 
> > On 2025-08-25 15:00:39 +0200, Tomas Vondra wrote:
> >> Thanks. Based on the testing so far, the patch seems to be a substantial
> >> improvement. What's needed to make this prototype committable?
> > 
> > Mainly some testing infrastructure that can trigger this kind of stream. The
> > logic is too finnicky for me to commit it without that.
> > 
> 
> So, what would that look like?

I'm thinking of something like an SQL function that accepts a relation and a
series of block numbers, which creates a read stream reading the passed in
block numbers.  Combined with the injection points that are already used in
test_aio, that should allow to test things that I don't know how to test
without that.  E.g. encountering an already-in-progress multi-block IO that
only completes partially.


> Another approach would be to test this at C level, sidestepping the
> query execution entirely. We'd have a "stream generator" that just
> generates a sequence of blocks of our own choosing (could be hard-coded,
> some pattern, read from a file ...), and feed it into a read stream.
> 
> But how would we measure success for these tests? I don't think we want
> to look at query duration, that's very volatile.

Yea, the performance effects would be harder to test, what I care more about
is the error paths. Those are really hard to test interactively.

Greetings,

Andres Freund



pgsql-hackers by date:

Previous
From: Bertrand Drouvot
Date:
Subject: Re: Adding per backend commit and rollback counters
Next
From: Tomas Vondra
Date:
Subject: Re: Changing the state of data checksums in a running cluster