Re: Better shared data structure management and resizable shared data structures - Mailing list pgsql-hackers

On 12/03/2026 19:03, Robert Haas wrote:
> On Fri, Mar 6, 2026 at 9:13 AM Heikki Linnakangas <hlinnaka@iki.fi> wrote:
>> 1. _PG_init() gets called early at postmaster startup, if the library is
>> in shared_preload_libraries. If it's not in shared_preload_libraries, it
>> gets called whenever the module is loaded.
>>
>> 2. The library can install a shmem_request_hook, which gets called early
>> at postmaster startup, but after initializing the MaxBackends GUC. It
>> only gets called when the library is loaded via shared_preload_libraries.
>>
>> 3. The library can install a shmem_startup_hook. It gets called later at
>> postmaster startup, after the shared memory segment has been allocated.
>> In EXEC_BACKEND mode it also gets called at backend startup. It does not
>> get called if the library is not listed in shared_preload_libraries.
>>
>> None of these is quite the right moment to call the new
>> ShmemRegisterStruct() function. _PG_init() is too early if the extension
>> needs MaxBackends for sizing the shared memory area. shmem_request_hook
>> is otherwise good, but in EXEC_BACKEND mode, the ShmemRegisterStruct()
>> function needs to also be called backend startup and shmem_request_hook
>> is not called at backend startup. shmem_startup_hook() is too late.
> 
> I believe that the design goal of
> 4f2400cb3f10aa79f99fba680c198237da28dd38 was to make it so that people
> who had working extensions already didn't need to change their code,
> but those for whom the restrictions of doing things in _PG_init were
> annoying would have a workable alternative. I think that's a pretty
> good goal, although I don't feel we absolutely have to stick to it. It
> could easily be worth breaking that if we get something cool out of
> it. But is there a reason we can't make it so that this new mechanism
> can be used either from _PG_init() or shmem_startup_hook()? (I assume
> there is or you likely would have done it already, but it's not clear
> to me what that reason is.)

shmem_startup_hook() is too late. The shmem structs need to be 
registered at postmaster startup before the shmem segment is allocated, 
so that we can calculate the total size needed.

I'm currently leaning towards _PG_init(), except for allocations that 
depend on MaxBackends. For those, you can install a shmem_request_hook 
that sets the size in the descriptor. In other words, you can leave the 
'size' as empty in _PG_init(), but set it later in the shmem_request_hook.

Another option is to add a new bespoken callback in the descriptor for 
such size adjustments, which would get called at the same time as 
shmem_request_hook. That might be a little more ergonomic, there would 
no longer be any need for extensions to use the old 
shmem_request/startup_hooks with the new ShmemRegisterStruct() mechanism.

Except that you'd still need them for RequestNamedLWLockTranche(). I 
wonder if we should recommend extensions to embed the LWLock struct into 
their shared memory struct and use the LWLockInitialize() and 
LWLockNewTrancheId() functions instead. That fits the new 
ShmemRegisterStruct() API a little better than RequestNamedLWLockTranche().

- Heikki