Re: Parallel INSERT (INTO ... SELECT ...) - Mailing list pgsql-hackers

From Greg Nancarrow
Subject Re: Parallel INSERT (INTO ... SELECT ...)
Date
Msg-id CAJcOf-f761GTeoNyWgPuNHJ-1LCGqOnzarXaWAZ1maGTWoYbeQ@mail.gmail.com
Whole thread Raw
In response to Re: Parallel INSERT (INTO ... SELECT ...)  (Thomas Munro <thomas.munro@gmail.com>)
Responses Re: Parallel INSERT (INTO ... SELECT ...)
List pgsql-hackers
On Fri, Oct 9, 2020 at 8:41 AM Thomas Munro <thomas.munro@gmail.com> wrote:
> One thing I noticed is that you have logic, variable names and
> assertions all over the tree that assume that we can only do parallel
> *inserts*.  I agree 100% with your plan to make Parallel Insert work
> first, it is an excellent goal and if we get it in it'll be a headline
> feature of PG14 (along with COPY etc).  That said, I wonder if it
> would make sense to use more general naming (isParallelModifyLeader?),
> be more liberal where you really mean "is it DML", and find a way to
> centralise the logic about which DML commands types are currently
> allowed (ie insert only for now) for assertions and error checks etc,
> so that in future we don't have to go around and change all these
> places and rename things again and again.
>

Fair points.
I agree, it would make more sense to generalise the naming and
centralise the DML-command-type checks, rather than everything being
insert-specific.
It was getting a bit ugly. I'll work on that.

> While contemplating that, I couldn't resist taking a swing at the main
> (?) show stopper for Parallel Update and Parallel Delete, judging by
> various clues left in code comments by Robert: combo command IDs
> created by other processes.  Here's a rapid prototype to make that
> work (though perhaps not as efficiently as we'd want, not sure).  With
> that in place, I wonder what else we'd need to extend your patch to
> cover all three operations... it can't be much!  Of course I don't
> want to derail your work on Parallel Insert, I'm just providing some
> motivation for my comments on the (IMHO) shortsightedness of some of
> the coding.
>

Thanks for your prototype code for coordination of combo command IDs
with the workers.
It does give me the incentive to look beyond that issue and see
whether parallel Update and parallel Delete are indeed possible. I'll
be sure to give it a go!

> PS Why not use git format-patch to create patches?

Guess I was being a bit lazy - will use git format-patch in future.


Regards,
Greg Nancarrow
Fujitsu Australia



pgsql-hackers by date:

Previous
From: "tsunakawa.takay@fujitsu.com"
Date:
Subject: RE: Transactions involving multiple postgres foreign servers, take 2
Next
From: Kyotaro Horiguchi
Date:
Subject: Re: Expansion of our checks for connection-loss errors