Re: [HACKERS] Partition-wise aggregation/grouping - Mailing list pgsql-hackers

From Konstantin Knizhnik
Subject Re: [HACKERS] Partition-wise aggregation/grouping
Date
Msg-id c777af01-9b4b-b352-5b36-9f649c1daa1b@postgrespro.ru
Whole thread Raw
In response to Re: [HACKERS] Partition-wise aggregation/grouping  (Jeevan Chalke <jeevan.chalke@enterprisedb.com>)
Responses Re: [HACKERS] Partition-wise aggregation/grouping  (David Rowley <david.rowley@2ndquadrant.com>)
List pgsql-hackers


On 15.11.2017 13:35, Jeevan Chalke wrote:

As explained by Ashutosh Bapat in reply
https://www.postgresql.org/message-id/CAFjFpRdpeMTd8kYbM_x0769V-aEKst5Nkg3+coG=8ki7s8Zqjw@mail.gmail.com
we cannot rely on just aggtype==aggtranstype.

Obviously this check (aggtype==aggtranstype) is not correct criteria for all user defined aggregates.
I just did it as temporary work around for standard aggregates.


However, I have tried pushing partial aggregation over remote server and also
submitted a PoC patch here:
https://www.postgresql.org/message-id/CAM2+6=UakP9+TSJuh2fbhHWNJc7OYFL1_gvu7mt2fXtVt6GY3g@mail.gmail.com

I have later removed these patches from Partition-wise-Aggregation patch set
as it is altogether a different issue than this mail thread. We might need to
discuss on it separately.
...
Interesting idea of "asynchronous append". However, IMHO it deserves its own
email-chain.

The main problem IMHO is that there are a lot of different threads and patches related with this topic:(
And it is very difficult to combine all of them together to achieve the final goal: efficient execution of OLAP queries on sharded table.
It will be nice if somebody who is making the most contribution in this direction can somehow maintain it...
I just faced with particular problem with our pg_shardman extension and now (thanks to your patch) I have some working solution for it.
But certainly I prefer to have this support in mainstream version of Postgres.

There are two open questions, which I wan to discuss (sorry, may be one again this is not the right thread for it):

1. Parallel append and FDW/postgres_fdw: should FDW support parallel scan and do we really need it to support concurrent execution of query on local and remote partitions?
"Asynchronous append" partly solves this problem, but only for remote partitions. I do not completely understand all complexity of alternative approaches.

2. Right now partition-wise aggregation/grouping works only for tables partitioned using new PG 10 partitioning mechanism. But it doesn't work for inherited tables, although
there seems to be not so much difference between this two cases. Do you think that sometimes it will be also supported for standard inheritance mechanism or there is no sense in it?


-- 
Konstantin Knizhnik
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company 

pgsql-hackers by date:

Previous
From: Tom Lane
Date:
Subject: Updated macOS start scripts
Next
From: Mark Dilger
Date:
Subject: Re: Updated macOS start scripts