Re: [GENERAL] postmaster deadlock while logging after syslogger exited - Mailing list pgsql-general

From Michael Paquier
Subject Re: [GENERAL] postmaster deadlock while logging after syslogger exited
Date
Msg-id CAB7nPqSG7pCLeSuZ=0nQ2vMrZXYw99wfy0aK7W=f4jVgAuFPVw@mail.gmail.com
Whole thread Raw
In response to Re: [GENERAL] postmaster deadlock while logging after sysloggerexited  (Andres Freund <andres@anarazel.de>)
Responses Re: [GENERAL] postmaster deadlock while logging after sysloggerexited
List pgsql-general
On Fri, Nov 17, 2017 at 10:50 AM, Andres Freund <andres@anarazel.de> wrote:
> On 2017-11-06 15:35:03 -0500, Tom Lane wrote:
>> David Pacheco <dap@joyent.com> writes:
>> > I ran into what appears to be a deadlock in the logging subsystem.  It
>> > looks like what happened was that the syslogger process exited because it
>> > ran out of memory.  But before the postmaster got a chance to handle the
>> > SIGCLD to restart it, it handled a SIGUSR1 to start an autovacuum worker.
>> > That also failed, and the postmaster went to log a message about it, but
>> > it's blocked on the pipe that's normally connected to the syslogger,
>> > presumably because the pipe is full because the syslogger is gone and
>> > hasn't read from it.
>>
>> Ugh.
>
> I'm somewhat inclined to say that one has to live with this if the
> system is so resource constrainted that processes barely using memory
> get killed.
>
> We could work around a situation like that if we made postmaster use a
> *different* pipe as stderr than the one we're handing to normal
> backends. If postmaster created a new pipe and closed the read end
> whenever forking a syslogger, we should get EPIPEs when writing after
> syslogger died and could fall back to proper stderr or such.

I don't have the code on top of my mind, but isn't a custom fd causing
a small penalty when redirection_done is switched to true because the
first process generating a message to the syslogger pipe needs to open
it first if not done yet? So you'd need proper locking to save from
race conditions. Or is the first message redirected message always
generated by the postmaster or the syslogger? I don't recall that this
is actually true..
-- 
Michael


-- 
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

pgsql-general by date:

Previous
From: Andres Freund
Date:
Subject: Re: [GENERAL] postmaster deadlock while logging after sysloggerexited
Next
From: Andres Freund
Date:
Subject: Re: [GENERAL] postmaster deadlock while logging after sysloggerexited