Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation) - Mailing list pgsql-hackers

From Amit Kapila
Subject Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)
Date
Msg-id CAA4eK1KgmdT3ivm1vG+rJzKOKeYQU2XLhp6ma5LMHxaG89brsA@mail.gmail.com
Whole thread Raw
In response to Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)  (Peter Geoghegan <pg@bowt.ie>)
Responses Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)
List pgsql-hackers
On Fri, Jan 26, 2018 at 12:00 PM, Peter Geoghegan <pg@bowt.ie> wrote:
> On Thu, Jan 25, 2018 at 10:00 PM, Amit Kapila <amit.kapila16@gmail.com> wrote:
>>> At this point, my preferred solution is for someone to go implement
>>> Amit's WaitForParallelWorkersToAttach() idea [1] (Amit himself seems
>>> like the logical person for the job).
>>>
>>
>> I can implement it and share a prototype patch with you which you can
>> use to test parallel sort stuff.
>
> That would be great. Thank you.
>
>> I would like to highlight the
>> difference which you will see with WaitForParallelWorkersToAttach as
>> compare to WaitForParallelWorkersToFinish() is that the former will
>> give you how many of nworkers_launched workers are actually launched
>> whereas latter gives an error if any of the expected workers is not
>> launched.  I feel former is good and your proposed way of calling it
>> after the leader is done with its work has alleviated the minor
>> disadvantage of this API which is that we need for workers to startup.
>
> I'm not sure that it makes much difference, though, since in the end
> WaitForParallelWorkersToFinish() is called anyway, much like
> nodeGather.c. Have I missed something?
>

Nopes, you are right.  I had in my mind that if we have something like
what I am proposing, then we don't even need to detect failures in
WaitForParallelWorkersToFinish and we can finish the work without
failing.

> I had imagined that WaitForParallelWorkersToAttach() would give me an
> error in the style of WaitForParallelWorkersToFinish(), without
> actually waiting for the parallel workers to finish.
>

I think that is also doable.  I will give it a try and report back if
I see any problem with it.  However, it might take me some time as I
am busy with few other things and I am planning to take two days off
for some personal reasons, OTOH if it turns out to be a simple (which
I expect it should be), then I will report back early.

-- 
With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com


pgsql-hackers by date:

Previous
From: Peter Moser
Date:
Subject: Re: [HACKERS] [PROPOSAL] Temporal query processing with range types
Next
From: David Rowley
Date:
Subject: Re: STATISTICS retained in CREATE TABLE ... LIKE (INCLUDING ALL)?