Re: Combining Aggregates - Mailing list pgsql-hackers

From David Rowley
Subject Re: Combining Aggregates
Date
Msg-id CAKJS1f8=RzEE+3Lc_oR5+9gY759xtVtFCvCRNH+PZfjuLeHnkg@mail.gmail.com
Whole thread Raw
In response to Re: Combining Aggregates  (Tomas Vondra <tomas.vondra@2ndquadrant.com>)
List pgsql-hackers
On 19 January 2016 at 18:04, Tomas Vondra <tomas.vondra@2ndquadrant.com> wrote:
Hi,

On 01/19/2016 05:00 AM, David Rowley wrote:
On 19 January 2016 at 06:03, Pavel Stehule <pavel.stehule@gmail.com
<mailto:pavel.stehule@gmail.com>> wrote:

...

    It is strange, why hashaggregate is too slow?


Good question. I looked at this and found my VM was swapping like crazy.
Upon investigation it appears that's because, since the patch creates a
memory context per aggregated group, and in this case I've got 1 million
of them, it means we create 1 million context, which are
ALLOCSET_SMALL_INITSIZE (1KB) in size, which means about 1GB of memory,
which is more than my VM likes.

Really? Where do we create the memory context? IIRC string_agg uses the aggcontext directly, and indeed that's what I see in string_agg_transfn and makeStringAggState.


Yeah, all this is talk is relating to Robert's expandedstring-v1.patch which changes string_agg to use text and expanded-objects. This also means that a memory context is created per group, which is rather a big overhead.
 

--
 David Rowley                   http://www.2ndQuadrant.com/
 PostgreSQL Development, 24x7 Support, Training & Services

pgsql-hackers by date:

Previous
From: Tomas Vondra
Date:
Subject: Re: Combining Aggregates
Next
From: Tomas Vondra
Date:
Subject: Re: Combining Aggregates