Re: Memory-Bounded Hash Aggregation - Mailing list pgsql-hackers

From Jeff Davis
Subject Re: Memory-Bounded Hash Aggregation
Date
Msg-id 7e58d8c57690373d66e0a07e8cf683f7d141c74b.camel@j-davis.com
Whole thread Raw
In response to Re: Memory-Bounded Hash Aggregation  (Jeff Davis <pgsql@j-davis.com>)
Responses Re: Memory-Bounded Hash Aggregation  (Jeff Davis <pgsql@j-davis.com>)
List pgsql-hackers
On Mon, 2020-02-03 at 18:24 -0800, Jeff Davis wrote:
> On Mon, 2020-02-03 at 10:29 -0800, Jeff Davis wrote:
> > I ended up converting the freelist to a min heap.
> > 
> > Attached is a patch which makes three changes to better support
> > HashAgg:
> 
> And now I'm attaching another version of the main Hash Aggregation
> patch to be applied on top of the logtape.c patch.
> 
> Not a lot of changes from the last version; mostly some cleanup and
> rebasing. But it's faster now with the logtape.c changes.

Attaching latest version (combined logtape changes along with main
HashAgg patch).

I believe I've addressed all of the comments, except for Heikki's
question about changing the logtape.c API. I think big changes to the
API (such as Heikki's proposal) are out of scope for this patch,
although I do favor the changes in general. This patch just includes
the LogicalTapeSetExtend() API by Adam Lee, which is less intrusive.

I noticed (and fixed) a small regression for some in-memory hashagg
queries due to the way I was choosing the number of buckets when
creating the hash table. I don't think that it is necessarily worse in
general, but given that there is at least one case of a regression, I
made it more closely match the old behavior, and the regression
disappared.

I improved costing by taking into account the actual number of
partitions and the memory limits, at least for the first pass (in
recursive passes the number of partitions can change).

Aside from that, just some cleanup and rebasing.

Regards,
    Jeff Davis


Attachment

pgsql-hackers by date:

Previous
From: Quan Zongliang
Date:
Subject: Re: Restore replication settings when modifying a field type
Next
From: Ranier Vilela
Date:
Subject: Re: Postgres 32 bits client compilation fail. Win32 bits client is supported?