Re: AIO v2.5 - Mailing list pgsql-hackers

Hi,

Attached is v2.10, with the following changes:

- committed core AIO infrastructure patch

  A few cosmetic changes


- committed io_method=worker

  Two non-cosmetic changes:

  - The shmem allocation functions over-estimated the amount of shared memory
    required.

  - pgaio_worker_shmem_init() should initialize up to MAX_IO_WORKERS, not just
    io_workers, since the latter is intentionally PGC_SIGHUP (found by Thomas)


- Bunch of typo fixes found by searching for repeated words

  Thomas found one and then I searched for more.


- Added Melanie's patch to allow effective_io_concurrency to be set on windows etc


- Fixed a reference to an outdated function reference (thanks to Bilal)


- Reordered patches to be a bit more in dependency order

  E.g. "bufmgr: Implement AIO read support" doesn't depend on Thomas' "buffer
  forwarding" patches and thus can be commited before those go in.


Next steps:

- Decide what to do about the smgr interrupt issue

  I guess it could be deferred, based on the argument it only matters with a
  sufficiently high debug level, but I don't feel comfortable with that.

  I think it'd be reasonable to just go with the patch I sent on the other
  thread (and included here).


- Get somebody to look at
  "bufmgr: Improve stats when buffer was read in concurrently"

  This arguably fixes a bug, or just weird behaviour, on master.


- Address the set_max_safe_fds() issue and once resolved, commit io_uring
  method

  Can happen concurrently with next steps


- Commit "aio: Implement support for reads in smgr/md/fd"


- Get somebody to do one more pass at bufmgr related commits, I think they
  could use a less in-the-weeds eye.

  That's the following commits:
  - localbuf: Track pincount in BufferDesc as well
  - bufmgr: Implement AIO read support
  - bufmgr: Use AIO in StartReadBuffers()
  - aio: Basic read_stream adjustments for real AIO


Questions / Unresolved:

- Write support isn't going to land in 18, but there is a tiny bit of code
  regarding writes in the code for bufmgr IO. I guess I could move that to a
  later commit?

  I'm inclined to leave it, the structure of the code only really makes
  knowing that it's going to be shared between reads & writes.


- pg_aios view name


Greetings,

Andres Freund