PGC_SIGHUP shared_buffers? - Mailing list pgsql-hackers

Hi,

I remember Magnus making a comment many years ago to the effect that
every setting that is PGC_POSTMASTER is a bug, but some of those bugs
are very difficult to fix. Perhaps the use of the word bug is
arguable, but I think the sentiment is apt, especially with regard to
shared_buffers. Changing without a server restart would be really
nice, but it's hard to figure out how to do it. I can think of a few
basic approaches, and I'd like to know (a) which ones people think are
good and which ones people think suck (maybe they all suck) and (b) if
anybody's got any other ideas not mentioned here.

1. Complicate the Buffer->pointer mapping. Right now, BufferGetBlock()
is basically just BufferBlocks + (buffer - 1) * BLCKSZ, which means
that we're expecting to find all of the buffers in a single giant
array. Years ago, somebody proposed changing the implementation to
essentially WhereIsTheBuffer[buffer], which was heavily criticized on
performance grounds, because it requires an extra memory access. A
gentler version of this might be something like
WhereIsTheChunkOfBuffers[buffer/CHUNK_SIZE]+(buffer%CHUNK_SIZE)*BLCKSZ;
i.e. instead of allowing every single buffer to be at some random
address, manage chunks of the buffer pool. This makes the lookup array
potentially quite a lot smaller, which might mitigate performance
concerns. For example, if you had one chunk per GB of shared_buffers,
your mapping array would need only a handful of cache lines, or a few
handfuls on really big systems.

(I am here ignoring the difficulties of how to orchestrate addition of
or removal of chunks as a SMOP[1]. Feel free to criticize that
hand-waving, but as of this writing, I feel like moderate
determination would suffice.)

2. Make a Buffer just a disguised pointer. Imagine something like
typedef struct { Page bp; } *buffer. WIth this approach,
BufferGetBlock() becomes trivial. The tricky part with this approach
is that you still need a cheap way of finding the buffer header. What
I imagine might work here is to again have some kind of chunked
representation of shared_buffers, where each chunk contains a bunch of
buffer headers at, say, the beginning, followed by a bunch of buffers.
Theoretically, if the chunks are sufficiently strong-aligned, you can
figure out what offset you're at within the chunk without any
additional information and the whole process of locating the buffer
header is just math, with no memory access. But in practice, getting
the chunks to be sufficiently strongly aligned sounds hard, and this
also makes a Buffer 64 bits rather than the current 32. A variant on
this concept might be to make the Buffer even wider and include two
pointers in it i.e. typedef struct { Page bp; BufferDesc *bd; }
Buffer.

3. Reserve lots of address space and then only use some of it. I hear
rumors that some forks of PG have implemented something like this. The
idea is that you convince the OS to give you a whole bunch of address
space, but you try to avoid having all of it be backed by physical
memory. If you later want to increase shared_buffers, you then get the
OS to back more of it by physical memory, and if you later want to
decrease shared_buffers, you hopefully have some way of giving the OS
the memory back. As compared with the previous two approaches, this
seems less likely to be noticeable to most PG code. Problems include
(1) you have to somehow figure out how much address space to reserve,
and that forms an upper bound on how big shared_buffers can grow at
runtime and (2) you have to figure out ways to reserve address space
and back more or less of it with physical memory that will work on all
of the platforms that we currently support or might want to support in
the future.

4. Give up on actually changing the size of shared_buffer per se, but
stick some kind of resizable secondary cache in front of it. Data that
is going to be manipulated gets brought into a (perhaps small?) "real"
shared_buffers that behaves just like today, but you have some larger
data structure which is designed to be easier to resize and maybe
simpler in some other ways that sits between shared_buffers and the OS
cache. This doesn't seem super-appealing because it requires a lot of
data copying, but maybe it's worth considering as a last resort.

Thoughts?

--
Robert Haas
EDB: http://www.enterprisedb.com

[1] https://en.wikipedia.org/wiki/Small_matter_of_programming