Re: Buffers from parallel workers not accumulated to upper nodes with gather merge - Mailing list pgsql-bugs

From Jehan-Guillaume de Rorthais
Subject Re: Buffers from parallel workers not accumulated to upper nodes with gather merge
Date
Msg-id 20200723110749.7aa0e15a@firost
Whole thread Raw
In response to Re: Buffers from parallel workers not accumulated to upper nodes with gather merge  (Jehan-Guillaume de Rorthais <jgdr@dalibo.com>)
Responses Re: Buffers from parallel workers not accumulated to upper nodes with gather merge
List pgsql-bugs
On Tue, 21 Jul 2020 17:34:57 +0200
Jehan-Guillaume de Rorthais <jgdr@dalibo.com> wrote:
> On Mon, 20 Jul 2020 15:29:00 +0530
> Amit Kapila <amit.kapila16@gmail.com> wrote:
> > On Mon, Jul 20, 2020 at 1:29 PM Jehan-Guillaume de Rorthais
> > <jgdr@dalibo.com> wrote:  
> > > On Mon, 20 Jul 2020 11:30:34 +0530
> > > Amit Kapila <amit.kapila16@gmail.com> wrote:
[...]
> > > > I think irrespective of whether we want to call
> > > > ExecShutdownGatherMergeWorkers in gather_merge_getnext (when we know
> > > > we are done aka binaryheap_empty) to fix this particular issue, it is
> > > > better to shutdown the workers as soon as we are done similar to what
> > > > we do for Gather node.  It is good to release resources as soon as we
> > > > can.  
> > >
> > > Indeed. I was focusing on the bug and I didn't thought about that. The
> > > patch I test was really just a quick test. I'll have a closer look at
> > > this, but I suppose this might be considered as a separate commit?
> > >  
> > 
> > Good Question.  Initially, I thought we will have it in a same commit,
> > but now on again thinking about it, we might want to keep this one for
> > the HEAD only and ExecShutdownNode related change in the back-branches.  
> 
> OK.
> 
> I'll send a patch proposal for Gather Merge on HEAD tomorrow.

Please, find in attachment the patch to release Gather Merge workers as soon as
they are exhausted.

I was wondering how interesting it would be to release each worker individually
of each others? This might be interesting for long running queries and/or large
number of workers. This would apply to Gather as well. Do you think it worth to
give it try? Maybe it has already been discussed in the past?

Regards,

Attachment

pgsql-bugs by date:

Previous
From: Slawomir Nowakiewicz
Date:
Subject: RE: BUG #16549: "CASE" not work properly , the function works properly on PostgreSQL 9.6.8
Next
From: Amit Kapila
Date:
Subject: Re: Buffers from parallel workers not accumulated to upper nodes with gather merge