Re: Asynchronous execution support for Custom Scan - Mailing list pgsql-hackers

From Ronan Dunklau
Subject Re: Asynchronous execution support for Custom Scan
Date
Msg-id 7453168.EvYhyI6sBW@aivenlaptop
Whole thread Raw
In response to Re: Asynchronous execution support for Custom Scan  (Etsuro Fujita <etsuro.fujita@gmail.com>)
Responses Re: Asynchronous execution support for Custom Scan
List pgsql-hackers
Le mardi 6 septembre 2022, 11:29:55 CET Etsuro Fujita a écrit :
> On Mon, Sep 5, 2022 at 10:32 PM Kazutaka Onishi <onishi@heterodb.com> wrote:
> > I'm sorry for my error on your name...
>
> No problem.
>
> > >  IIUC, it uses the proposed
> > >
> > > APIs, but actually executes ctidscans *synchronously*, so it does not
> > > improve performance.  Right?
> >
> > Exactly.
> > The actual CustomScan that supports asynchronous execution will
> > start processing in CustomScanAsyncRequest,
> > configure to detect completion via file descriptor in
> > CustomScanAsyncConfigureWait,
> > and receive the result in CustomScanAsyncNotify.
>
> Ok, thanks!

Thanks for this patch, seems like a useful addition to the CustomScan API.
Just to nitpick: there are extraneous tabs in createplan.c on a blank line.

Sorry for the digression, but I know your ctidscan module had been proposed
for inclusion in contrib a long time ago, and I wonder if the rationale for
not including it could have changed. We still don't have tests which cover
CustomScan, and I can think of at least a few use cases where this customscan
is helpful and not merely testing code.

One of those use case is when performing migrations on a table, and one wants
to update the whole table by filling a new column with a computed value. You
obviously don't want to do it in a single transaction, so you end up batching
updates using an index looking for null values. If you want to do this, it's
much faster to update rows in a range of block, performing a first series of
batch updating all such block ranges, and then finally update the ones we
missed transactionally (inserted in a block we already processed while in the
middle of the batch, or in new blocks resulting from a relation extension).

Best regards,

--
Ronan Dunklau





pgsql-hackers by date:

Previous
From: Aleksander Alekseev
Date:
Subject: Re: Add 64-bit XIDs into PostgreSQL 15
Next
From: "Fujii.Yuki@df.MitsubishiElectric.co.jp"
Date:
Subject: RE: [CAUTION!! freemail] Re: Partial aggregates pushdown