Re: [HACKERS] Race conditions with WAL sender PID lookups - Mailing list pgsql-hackers

From Thomas Munro
Subject Re: [HACKERS] Race conditions with WAL sender PID lookups
Date
Msg-id CAEepm=1wSjXgyTFuMj9wJuZcY2-Diou1D2iewXiGMN6_wh02ng@mail.gmail.com
Whole thread Raw
In response to Re: [HACKERS] Race conditions with WAL sender PID lookups  (Michael Paquier <michael.paquier@gmail.com>)
List pgsql-hackers
On Fri, May 19, 2017 at 2:05 PM, Michael Paquier
<michael.paquier@gmail.com> wrote:
> On Thu, May 18, 2017 at 1:43 PM, Thomas Munro
> <thomas.munro@enterprisedb.com> wrote:
>> On Thu, May 11, 2017 at 1:48 PM, Michael Paquier
>> <michael.paquier@gmail.com> wrote:
>>> I had my eyes on the WAL sender code this morning, and I have noticed
>>> that walsender.c is not completely consistent with the PID lookups it
>>> does in walsender.c. In two code paths, the PID value is checked
>>> without holding the WAL sender spin lock (WalSndRqstFileReload and
>>> pg_stat_get_wal_senders), which looks like a very bad idea contrary to
>>> what the new WalSndWaitStopping() does and what InitWalSenderSlot() is
>>> doing for ages.
>>
>> There is also code that accesses shared walsender state without
>> spinlocks over in syncrep.c.  I think that file could use a few words
>> of explanation for why it's OK to access pid, state and flush without
>> synchronisation.
>
> Yes, that is read during the quorum and priority sync evaluation.
> Except sync_standby_priority, all the other variables should be
> protected using the spin lock of the WAL sender. walsender_private.h
> is clear regarding that. So the current coding is inconsistent even
> there. Attached is an updated patch.

I don't claim that that stuff is wrong, just that it would be good to
hear an analysis.  I think the question is: is there a way for syncrep
to hang because of a perfectly timed walsender transition, however
vanishingly unlikely?  I'm thinking of something like: syncrep skips a
walsender slot because it's looking at arbitrarily old 'pid' from
before a walsender connected, or gets a torn read of 'flush' that
comes out as 0 but was actually non-0.

Incidentally, I suspect that a couple of places where 'volatile' is
used it's superfluous (accessing things protected by an LWLock that is
held).

I don't see any of this as 'open item' material, it's interesting to
look into but it's preexisting code.  As for unlocked reads used for
pg_stat_X views, it seems well established that we're OK with that (at
least for things that the project has decided can be read atomically,
to wit aligned 32 bit values).

-- 
Thomas Munro
http://www.enterprisedb.com



pgsql-hackers by date:

Previous
From: Alvaro Herrera
Date:
Subject: Re: [HACKERS] Segmentation fault when creating a BRIN, 10beta1
Next
From: Tom Lane
Date:
Subject: Re: [HACKERS] syscache entries out of order