Re: Is Hash Agg being used? 7.4 seems to handle this query worse than 7.3 - Mailing list pgsql-general

From Tom Lane
Subject Re: Is Hash Agg being used? 7.4 seems to handle this query worse than 7.3
Date
Msg-id 27528.1044992198@sss.pgh.pa.us
Whole thread Raw
In response to Re: Is Hash Agg being used? 7.4 seems to handle this query worse than 7.3  (Greg Stark <gsstark@mit.edu>)
Responses Re: Is Hash Agg being used? 7.4 seems to handle this query worse than 7.3  (Greg Stark <gsstark@mit.edu>)
List pgsql-general
Greg Stark <gsstark@mit.edu> writes:
> It was always a bit mysterious to me how postgres could implement
> count(distinct) without introducing a separate sort and aggregate for each
> occurrence.

It can't.  There's a sort + uniq + aggregate process done behind the
scenes in the executor for each DISTINCT aggregate.  This doesn't show
up on the EXPLAIN output, because the planner has nothing to do with it.

I thought about doing this via a separate hashtable for each group ...
for about a minute.  The trouble is you have to run those things in
parallel if you're doing hashed aggregation, so the resources required
are really out of the question in most cases.  With the group approach,
the executor is only processing the values for one outer group at a
time, so it only has to run one inner sort + uniq + agg process at a
time.

I suppose we could consider introducing two implementations (hash or
sort/uniq) for a DISTINCT agg within the executor, but it's code that
remains unwritten...

            regards, tom lane

pgsql-general by date:

Previous
From: Greg Stark
Date:
Subject: Re: Is Hash Agg being used? 7.4 seems to handle this query worse than 7.3
Next
From: Dmitry Tkach
Date:
Subject: Re: translating filenames into tablenames?