Re: Re: parallel distinct union and aggregate support patch - Mailing list pgsql-hackers

From bucoo@sohu.com
Subject Re: Re: parallel distinct union and aggregate support patch
Date
Msg-id 2020102314284168669733@sohu.com
Whole thread Raw
In response to parallel distinct union and aggregate support patch  ("bucoo@sohu.com" <bucoo@sohu.com>)
Responses Re: Re: parallel distinct union and aggregate support patch
List pgsql-hackers
> Interesting idea.  So IIUC, whenever a worker is scanning the tuple it
> will directly put it into the respective batch(shared tuple store),
> based on the hash on grouping column and once all the workers are
> doing preparing the batch then each worker will pick those baches one
> by one, perform sort and finish the aggregation.  I think there is a
> scope of improvement that instead of directly putting the tuple to the
> batch what if the worker does the partial aggregations and then it
> places the partially aggregated rows in the shared tuple store based
> on the hash value and then the worker can pick the batch by batch.  By
> doing this way, we can avoid doing large sorts.  And then this
> approach can also be used with the hash aggregate, I mean the
> partially aggregated data by the hash aggregate can be put into the
> respective batch.

Good idea. Batch sort suitable for large aggregate result rows,
in large aggregate result using partial aggregation maybe out of memory,
and all aggregate functions must support partial(using batch sort this is unnecessary).

Actually i written a batch hash store for hash aggregate(for pg11) like this idea,
but not write partial aggregations to shared tuple store, it's write origin tuple and hash value
to shared tuple store, But it's not support parallel grouping sets.
I'am trying to write parallel hash aggregate support using batch shared tuple store for PG14,
and need support parallel grouping sets hash aggregate.

pgsql-hackers by date:

Previous
From: "kuroda.hayato@fujitsu.com"
Date:
Subject: RE: ECPG: proposal for new DECLARE STATEMENT
Next
From: Heikki Linnakangas
Date:
Subject: Re: partition routing layering in nodeModifyTable.c