Thread: Finalizing read stream users' flag choices

Hi,

Over the course of the last two releases, we have added many read
stream users. Each user specifies any number of flags (defined at the
top of read_stream.h) which govern different aspects of the read
stream behavior.

There are a few inconsistencies (many caused by me) that I want to
iron out and gain consensus on.

The first is whether maintenance_io_concurerency or
effective_io_concurrency affects the readahead distance.

We've said before that maintenance_io_concurrency should govern work
done on behalf of many different sessions. That was said to include at
least vacuum and recovery. I need to change the index vacuum users to
use READ_STREAM_MAINTENANCE. But I wonder about the other users like
amcheck and autoprewarm.

Another related question is if/how we should document which of these
are controlled by effective_io_concurrency or
maintenance_io_concurrency.

The second is related to how they ramp up the size of IOs and the
number read ahead:
READ_STREAM_DEFAULT ramps up the prefetch distance gradually.
READ_STREAM_FULL starts at full distance immediately

Some of the users specify DEFAULT and others don't (it is defined as 0
so this is fine technically). Perhaps that should be explicit for all
of them? Separately, Thomas Munro has mentioned he thinks we should
remove READ_STREAM_FULL.

And a somewhat related point, with buffered IO, READ_STREAM_SEQUENTIAL
disables prefetching to encourage OS readahead. I don't know if any
other users than sequential scan should do this.

Other than the obvious issue with index vacuuming read stream users
needing to set READ_STREAM_MAINTENANCE, the other questions are
subjective. Below are all of the read stream users in master and their
current flags.

sequential scan: READ_STREAM_SEQUENTIAL | READ_STREAM_USE_BATCHING

bitmap heap scan: READ_STREAM_DEFAULT | READ_STREAM_USE_BATCHING

phase I heap vacuum: READ_STREAM_MAINTENANCE

phase II index vacuuming:
  btree index vacuuming: READ_STREAM_FULL | READ_STREAM_USE_BATCHING
  spgist vacuum: READ_STREAM_FULL | READ_STREAM_USE_BATCHING
  gist vacuum: READ_STREAM_FULL | READ_STREAM_USE_BATCHING

phase III heap vacuuming: READ_STREAM_MAINTENANCE | READ_STREAM_USE_BATCHING

analyze: READ_STREAM_MAINTENANCE | READ_STREAM_USE_BATCHING

amcheck:
  with skipping: READ_STREAM_DEFAULT
  without skipping:  READ_STREAM_SEQUENTIAL | READ_STREAM_FULL |
READ_STREAM_USE_BATCHING;

autoprewarm: READ_STREAM_DEFAULT | READ_STREAM_USE_BATCHING

pg_prewarm: READ_STREAM_FULL | READ_STREAM_USE_BATCHING

pg_visibility:
  collect visibility data: READ_STREAM_FULL | READ_STREAM_USE_BATCHING
  collect corrupt items: READ_STREAM_FULL

createdb: READ_STREAM_FULL | READ_STREAM_USE_BATCHING

- Melanie

On Tue, Apr 8, 2025 at 12:07 PM Melanie Plageman
<melanieplageman@gmail.com> wrote:
> We've said before that maintenance_io_concurrency should govern work
> done on behalf of many different sessions. That was said to include at
> least vacuum and recovery. I need to change the index vacuum users to
> use READ_STREAM_MAINTENANCE. But I wonder about the other users like
> amcheck and autoprewarm.

autoprewarm, and possibly regular prewarm, seem like maintenance to
me. amcheck does not.

Otherwise, the choices you've made about what is maintenance seem
reasonable to me, based on the list you included further down in the
email.

I don't know enough to opine on the questions about full vs. default,
or sequential scans.

--
Robert Haas
EDB: http://www.enterprisedb.com

hi,

On 2025-04-08 12:06:47 -0400, Melanie Plageman wrote:
> And a somewhat related point, with buffered IO, READ_STREAM_SEQUENTIAL
> disables prefetching to encourage OS readahead. I don't know if any
> other users than sequential scan should do this.

Worth adding that prefetches are only issued when io_method==sync and thus
READ_STREAM_SEQUENTIAL only has an effect if io_method==sync.

I suspect we should retire READ_STREAM_SEQUENTIAL in 19 or so:

a) The sequential-ness detection has gotten smarter / more granular, reducing
   the need for forcing read_stream's hand.
b) There are plenty cases where READ_STREAM_SEQUENTIAL *hurts* with seqscans
   and io_method==sync, e.g. if there are plenty pre-existing buffers in s_b.
c) It doesn't have an effect with io_method != sync

But it'll depend a bit on our experiences.

Greetings,

Andres Freund