Re: Re: parallel distinct union and aggregate support patch - Mailing list pgsql-hackers

From bucoo@sohu.com
Subject Re: Re: parallel distinct union and aggregate support patch
Date
Msg-id 202103301732449250842@sohu.com
Whole thread Raw
In response to parallel distinct union and aggregate support patch  ("bucoo@sohu.com" <bucoo@sohu.com>)
Responses Re: Re: parallel distinct union and aggregate support patch  (David Rowley <dgrowleyml@gmail.com>)
List pgsql-hackers
> This patch has not gotten any review in the last two CFs and is unlikely
> to be committed for PG14 so I have moved it to the 2021-07 CF. A rebase
> is also required so marked Waiting for Author.
>  
> I can see this is a work in progress, but you may want to consider the
> several suggestions that an unbuffered approach might be better.

I have written a plan with similar functions, It is known that the following two situations do not work well.
1. Under "Parallel Append" plan
  Gather
  -> Parallel Append
      -> Agg
          -> Parallel Redistribute(1)
              -> ...
      -> Agg
          -> Parallel Redistribute(2)
              -> ...
  when parallel worker 1 execute "Parallel Redistribute(1)" and worker execute "Parallel Redistribute(2)",
  both "Parallel Redistribute" plan can not send tuples to other worker(both worker are stuck),
  because outher worker's memory buffer run out soon.

2. Under "Nestloop" plan
  Gather
  -> Nestloop(1)
      -> Nestloop(2)
          -> Parallel Redistribute
              -> ...
          -> IndexScan
      -> Agg
  At some point might be the case: parallel worker 1 executing Agg and "Parallel Redistribute" plan's memory buffer is full,
  worker 2 executing "Parallel Redistribute" and it waiting worker 1 eat "Parallel Redistribute" plan's memory buffer,
  it's stuck.



bucoo@sohu.com

pgsql-hackers by date:

Previous
From: Bharath Rupireddy
Date:
Subject: Re: Use consistent terminology for tablesync slots.
Next
From: Markus Wanner
Date:
Subject: Re: [PATCH] Provide more information to filter_prepare