Re: Parallel grouping sets - Mailing list pgsql-hackers

From Pengzhou Tang
Subject Re: Parallel grouping sets
Date
Msg-id CAG4reAR+xM1fpZbw7uYUm-3LO0pkhcP6W0_pO4k9F4+fr5R_yQ@mail.gmail.com
Whole thread Raw
In response to Re: Parallel grouping sets  (Tomas Vondra <tomas.vondra@2ndquadrant.com>)
Responses Re: Parallel grouping sets  (Daniel Gustafsson <daniel@yesql.se>)
List pgsql-hackers

The memory context stats from a running process before it gets killed by
OOM look like this

   TopMemoryContext: 101560 total in 6 blocks; 7336 free (6 chunks); 94224 used
     TopTransactionContext: 73816 total in 4 blocks; 11624 free (0 chunks); 62192 used
       ExecutorState: 1375731712 total in 174 blocks; 5391392 free (382 chunks); 1370340320 used
         HashAgg meta context: 315784 total in 10 blocks; 15400 free (2 chunks); 300384 used
           ExprContext: 8192 total in 1 blocks; 7928 free (0 chunks); 264 used
           ExprContext: 8192 total in 1 blocks; 7928 free (0 chunks); 264 used
           ExprContext: 8192 total in 1 blocks; 7928 free (0 chunks); 264 used
           ...

That's 1.3GB allocated in ExecutorState - that doesn't seem right.

FWIW there are only very few groups (each attribute has fewer than 30
distinct values, so there's only about ~1000 groups. On master it works
just fine, of course.


Thanks a lot, the patch has a memory leak in the lookup_hash_entries, it uses a list_concat there
and causes a 64-byte leak for every tuple, has fixed that.

Also, resolved conflicts and rebased the code.

Thanks,
Pengzhou  
Attachment

pgsql-hackers by date:

Previous
From: Konstantin Knizhnik
Date:
Subject: Re: weird hash plan cost, starting with pg10
Next
From: Ibrar Ahmed
Date:
Subject: Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits