Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation) - Mailing list pgsql-hackers

From Peter Geoghegan
Subject Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)
Date
Msg-id CAH2-Wz=q+E1=e0zFv7OTMX+Ef=ucW6hmhDhnKby7wNs7RHYBOw@mail.gmail.com
Whole thread Raw
In response to Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)  (Amit Kapila <amit.kapila16@gmail.com>)
Responses Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)
List pgsql-hackers
On Fri, Jan 19, 2018 at 6:52 PM, Amit Kapila <amit.kapila16@gmail.com> wrote:
> Or reverse is also possible which means the workers won't get chance
> to run the plan in which case we can use parallel_leader_participation
> = off to test workers behavior.  As said before, I see only that as
> the reason to keep parallel_leader_participation in this patch.  If we
> decide to do that way, then I think we should remove the code that
> specifically disallows a "degenerate parallel CREATE INDEX" as that
> seems to be confusing.   If we go this way, then I think we should use
> the wording suggested by Robert in one of its email [1] to describe
> the usage of parallel_leader_participation.

I agree that parallel_leader_participation is only useful for testing
in the context of parallel CREATE INDEX. My concern with allowing a
"degenerate parallel CREATE INDEX" to go ahead is that
parallel_leader_participation generally isn't just intended for
testing by hackers (if it was, then I wouldn't care). But I'm now more
than willing to let this go.

> BTW, is there any other way for "parallel create index" to force that
> the work is done by workers?  I am insisting on having something which
> can test the code path in workers because we have found quite a few
> bugs using that idea.

I agree that this is essential (more so than supporting
parallel_leader_participation). You can use the parallel_workers table
storage parameter for this. When the storage param has been set, we
don't care about the amount of memory available to each worker. You
can stress-test the implementation as needed. (The storage param does
care about max_parallel_maintenance_workers, but you can set that as
high as you like.)

-- 
Peter Geoghegan


pgsql-hackers by date:

Previous
From: Amit Kapila
Date:
Subject: Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)
Next
From: Amit Kapila
Date:
Subject: Re: [HACKERS] parallel.c oblivion of worker-startup failures