Re: Reducing the chunk header sizes on all memory context types - Mailing list pgsql-hackers

On Tue, 6 Sept 2022 at 01:41, David Rowley <dgrowleyml@gmail.com> wrote:
>
> On Fri, 2 Sept 2022 at 20:11, David Rowley <dgrowleyml@gmail.com> wrote:
> >
> > On Thu, 1 Sept 2022 at 12:46, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> > >
> > > David Rowley <dgrowleyml@gmail.com> writes:
> > > > Maybe we should just consider always making room for a sentinel for
> > > > chunks that are on dedicated blocks. At most that's an extra 8 bytes
> > > > in some allocation that's either over 1024 or 8192 (depending on
> > > > maxBlockSize).
> > >
> > > Agreed, if we're not doing that already then we should.
> >
> > Here's a patch to that effect.
>
> If there are no objections, then I plan to push that patch soon.

I've now pushed the patch which adds the sentinel space in more cases.

The final analysis I did on the stats gathered during make
installcheck show that we'll now allocate about 19MBs more over the
entire installcheck run out of about 26GBs total allocations.

That analysis looks something like:

Before:

SELECT CASE
         WHEN pow2_size > 0
              AND pow2_size = size THEN 'No'
         WHEN pow2_size = 0
              AND size = maxalign_size THEN 'No'
         ELSE 'Yes'
       END                    AS has_sentinel,
       Count(*)               AS n_allocations,
       Sum(CASE
             WHEN pow2_size > 0 THEN pow2_size
             ELSE maxalign_size
           END) / 1024 / 1024 mega_bytes_alloc
FROM   memstats
GROUP  BY 1;
has_sentinel | n_allocations | mega_bytes_alloc
--------------+---------------+------------------
 No           |      26445855 |            21556
 Yes          |      37602052 |             5044

After:

SELECT CASE
         WHEN pow2_size > 0
              AND pow2_size = size THEN 'No'
         WHEN pow2_size = 0
              AND size = maxalign_size THEN 'Yes' -- this part changed
         ELSE 'Yes'
       END                    AS has_sentinel,
       Count(*)               AS n_allocations,
       Sum(CASE
             WHEN pow2_size > 0 THEN pow2_size
             WHEN size = maxalign_size THEN maxalign_size + 8
             ELSE maxalign_size
           END) / 1024 / 1024 mega_bytes_alloc
FROM   memstats
GROUP  BY 1;
has_sentinel | n_allocations | mega_bytes_alloc
--------------+---------------+------------------
 No           |      23980527 |             2177
 Yes          |      40067380 |            24442

That amounts to previously having about 58.7% of allocations having a
sentinel up to 62.6% currently, during the installcheck run.

It seems a pretty large portion of allocation request sizes are
power-of-2 sized and use AllocSet.

David