Re: Spilling hashed SetOps and aggregates to disk - Mailing list pgsql-hackers

From Andres Freund
Subject Re: Spilling hashed SetOps and aggregates to disk
Date
Msg-id 20180605125732.jdbvz54z6q36aud7@alap3.anarazel.de
Whole thread Raw
In response to Re: Spilling hashed SetOps and aggregates to disk  (David Rowley <david.rowley@2ndquadrant.com>)
Responses Re: Spilling hashed SetOps and aggregates to disk
Re: Spilling hashed SetOps and aggregates to disk
List pgsql-hackers
On 2018-06-06 00:53:42 +1200, David Rowley wrote:
> On 6 June 2018 at 00:45, Andres Freund <andres@anarazel.de> wrote:
> > On 2018-06-05 09:35:13 +0200, Tomas Vondra wrote:
> >> I wonder if an aggregate might use a custom context
> >> internally (I don't recall anything like that). The accounting capability
> >> seems potentially useful for other places, and those might not use AllocSet
> >> (or at least not directly).
> >
> > Yea, that seems like a big issue.
> 
> Unfortunately, at least one of the built-in ones do. See initArrayResultArr.

I think it's ok to only handle this gracefully if serialization is
supported.

But I think my proposal to continue use a hashtable for the already
known groups, and sorting for additional groups would largely address
that largely, right?  We couldn't deal with groups becoming too large,
but easily with the number of groups becoming too large.

Greetings,

Andres Freund


pgsql-hackers by date:

Previous
From: Dmitry Dolgov
Date:
Subject: Re: why partition pruning doesn't work?
Next
From: Andres Freund
Date:
Subject: Re: plans for PostgreSQL 12