Re: copy.c allocation constant - Mailing list pgsql-hackers

From Andres Freund
Subject Re: copy.c allocation constant
Date
Msg-id 20180124201320.q67fptt6diotcnsa@alap3.anarazel.de
Whole thread Raw
In response to Re: copy.c allocation constant  (Alvaro Herrera <alvherre@alvh.no-ip.org>)
List pgsql-hackers
Hi,

On 2018-01-24 17:07:01 -0300, Alvaro Herrera wrote:
> Andres Freund wrote:
> > glibc's malloc does add a header. My half-informed suspicion is that
> > most newer malloc backing allocators will have a header, because
> > maintaining a shared lookup-by-address table is pretty expensive to
> > maintain. A bit of metadata indicating size and/or source of the
> > allocation makes using thread-local information a lot easier.
>
> Sounds like it'd be smart to allocate something closer to
> M_MMAP_THRESHOLD (which with typical values would be about double the
> amount of memory the current RAW_BUF_SIZE value), minus a few dozen
> bytes to allow for palloc's and malloc's respective headers.  That way
> we bet for a mmap()ed allocation with minimum space wastage across all
> layers.

In general there might be cases where that's worthwhile, although
M_MMAP_THRESHOLD IIRC isn't a constant anymore, complicating things. The
specific case this thread is discussing doesn't seem worthy of such
attention though, there's afaict no actual practical problem here.


> Not sure whether we want to try to minimize wastage through
> clever use of malloc_usable_size() on each backend's first allocation --
> that'd probably just be overengineering.

Likely also dangerous, I don't think this is a constant.

Greetings,

Andres Freund


pgsql-hackers by date:

Previous
From: Thomas Munro
Date:
Subject: Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)
Next
From: Alvaro Herrera
Date:
Subject: Re: pgsql: Add parallel-aware hash joins.