Re: background processes vs. hot standby - Mailing list pgsql-hackers

Hi,

On 2013-05-24 09:48:03 -0400, Robert Haas wrote:
> CheckRequiredParameterValues() has some code that, when hot standby is
> in use, checks the values of max_connections,
> max_prepared_transactions, and max_locks_per_transaction against the
> master.   The comment says "we must have at least as many backend
> slots as the primary" ... but the code no longer enforces that,
> because we now compute MaxBackends like this:
> 
>         MaxBackends = MaxConnections + autovacuum_max_workers + 1 +
>                 GetNumShmemAttachedBgworkers();
> 
> If GetNumShmemAttachedBgworkers() returns a lower value on the standby
> than it did on the master, then we might well have fewer backend slots
> on the standby.  I'm having trouble remembering why it's a problem to
> have fewer backend slots on the standby than the master, but if we
> need to prevent that then this code is no longer adequate to the task.

It's afair important because we need to allocate shared memory which can
keep track of the maximum number of xids (toplevel *
max_non_suboverflowed_subxids) in progress. That's the
KnownAssignedXids* stuff in procarray.c.

> The comment doesn't explain why we check max_locks_per_transaction.  I
> thought the reason for that check was that we needed to ensure that
> there were at least as many lock table slots on the standby as there
> were on the master, to prevent bad things from happening later.  That
> was already not true, since the existing code didn't enforce any
> limitation on autovacuum_max_workers on the standby side.  Maybe that
> doesn't matter, since autovacuum workers can't run in hot standby
> mode; not sure.  But the addition of background workers to MaxBackends
> provides another way for that to be not true.  Here's how we compute
> the size of the lock table:

Yea, we need it exactly for that reason. I think its unlikely to cause
actual problems since we only ship access exclusive locks to the standby
and its hard to see scenarios where we have that many AEL on the
primary. But we probably should fix it anyway.

I think fixing the autovacuum_max_workers case is entirely reasonable
and relatively unlikely to cause problems. I don't think we can easily
do it in a minor release though since I don't see a way to transport
knowledge about it via the WAL without breaking either the WAL format
entirely or change the meaning of MaxConnections in ControlFile which
would cause problems with upgrading the primary first.

I am less excited about doing something similar for the background
worker case. Requiring just as many background workers on the standby
sounds like a bad idea to me, there seem to be too many cases where that
doesn't seem to make sense.
I wonder if we shouldn't make background workers use connections slots
from max_connections similar to how superuser_reserved_connections
work. That would mean we don't need to care about it for HS.

Greetings,

Andres Freund