Re: Re: custom hash-based COUNT(DISTINCT) aggregate - unexpectedly high memory consumption - Mailing list pgsql-hackers

From Tomas Vondra
Subject Re: Re: custom hash-based COUNT(DISTINCT) aggregate - unexpectedly high memory consumption
Date
Msg-id 152db6b33e29e83c1d1e6957b5a7c198.squirrel@sq.gransy.com
Whole thread Raw
In response to Re: Re: custom hash-based COUNT(DISTINCT) aggregate - unexpectedly high memory consumption  (Atri Sharma <atri.jiit@gmail.com>)
List pgsql-hackers
On 8 Říjen 2013, 13:52, Atri Sharma wrote:
> On Tue, Oct 8, 2013 at 4:15 PM, Tomas Vondra <tv@fuzzy.cz> wrote:
>> On 8 Říjen 2013, 11:42, Atri Sharma wrote:
>>>>
>>>> I've made some significant improvements in the chaining version (in
>>>> the
>>>> master branch), now getting about the memory consumption I've
>>>> estimated.
>>>>
>>> I agree, we can hope to reduce the memory consumption by making changes
>>> in
>>> the current chaining implementation. I would like to look into changing
>>> the data structure used for chaining from singly linked list to maybe
>>> skip
>>> list or something else.
>>
>> Just to be sure - I haven't been messing with the HashAggregate
>> implementation directly, but with a custom aggregate. But feel free to
>> tweak the built-in hash table ;-)
>>
>> Tomas
>>
>
> Heh.
>
> Do you mind if I try it out on the custom agg you built? I assume it
> is on the github repo link you shared?

Not at all, that's why I pushed that into a public repo. The "master"
branch contains the regular chained hash table, the open addressing is in
a separate branch (also in the repo).

Tomas




pgsql-hackers by date:

Previous
From: Bruce Momjian
Date:
Subject: Release note fix for timeline item
Next
From: Alexander Korotkov
Date:
Subject: Re: GIN improvements part 1: additional information