Thread: libpq pipeline/batch mode

libpq pipeline/batch mode

From
Daniele Varrazzo
Date:
Hello,

I am pleased to see that the pipeline/batch mode is progressing and it
might be released in pg14: https://commitfest.postgresql.org/30/2724/

psycopg3 uses only async functions, even for seemingly blocking calls,
so I think it is in a good position to make use of it. I don't know
yet if it should be a special operation mode, or the normal way to
work, replacing the lock to coordinate cursor interaction with the
connection with a pair of send/receive queues, if we could make
execute only send and fetch receive (ending up giving the async fetch
a proper I/O use, to reconnect to the recent conversation about it).

However I am getting stretched pretty thinly with different directions
of work: main tasks in my queue are:

- connection pool
- binary numeric/datetime adapters
- server-side cursors
- jsonb binary protocol (something to design server-side too)

so I doubt I will be able to hack on a branch of psycopg and a branch
of the libpq to study and design the pipeline possibilities too.

If anyone would like to take up this research task, or even would like
to collaborate on any other task listed above (or on the project
board: https://github.com/psycopg/psycopg3/projects/1) I would be
grateful!

Cheers!

-- Daniele