Re: [HACKERS] log_destination=file - Mailing list pgsql-hackers

From Magnus Hagander
Subject Re: [HACKERS] log_destination=file
Date
Msg-id CABUevEwFWUeB1pp=xHVMKfQeKx2qZ3PGh1ijU-jYESoaG-apsA@mail.gmail.com
Whole thread Raw
In response to Re: [HACKERS] log_destination=file  (Greg Stark <stark@mit.edu>)
Responses Re: [HACKERS] log_destination=file
List pgsql-hackers


On Fri, Sep 1, 2017 at 6:00 PM, Greg Stark <stark@mit.edu> wrote:
On 31 August 2017 at 13:49, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Magnus Hagander <magnus@hagander.net> writes:
>> On Thu, Aug 31, 2017 at 2:34 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Yes, it's pretty important, because of assorted stuff not-under-our-
> control that doesn't know about ereport and will just print to stderr
> anyway.  Some examples: dynamic linker can't-resolve-symbol failure
> messages, glibc malloc corruption error messages, just about any external
> module in plperl or plpython.  I don't find this to be negotiable.

So what happens now with these messages? My understanding is that
they're missing from the CSV logs and are simply inserted into the
text logs without any log_line_prefix? The logging collector doesn't
recognize these messages and reformat them for the CSV logs does it?

Yeah, that's pretty much it.

Fixing that is also on my plan, but for later :)

 
I'm actually asking because I'm more concerned with JSON logs or
msgpack logs. Currently these are supported with an emit_log_hook but
they can't capture these non-ereport logs either.

Also the CSV and emit_log_hook based logs don't have any convenient
way to turn them on and off and control the location and filename of
the logs. It would be nice if we could have something like

log_destinations='stderr=text,syslog=json,postgresql-%Y-%m-%d_%H%M%S.csv=csv'

That could definitely be an interesting improvement as well. Decoupling the format from the destination would make a lot of sense. (Arguments can certainly be made for example for csv-to-syslog, when you have a custom syslog server)

 
>> Are you actually asking for a benchmark of if logging gets slower?
>
> Yes.

Personally I don't think it's "performance" so much as operational
issues that are more concerning. For all we know there are people out
there who tried to use the logging collector and found it didn't work
well on some system -- perhaps it interacted with systemd or something
else on the system -- and they switched back to just using stderr. I
don't know how to flush these users out though if there are any. Just
making this change early in a release cycle is the best we can do.

The most common use of stderr I think is debian/ubuntu which uses that and then pg_ctl based redirect. Or at least used to (I remember being annoyed by that at least once..)

AFAIK all other major platforms already use the logging collector in the standard packages.
 
--

pgsql-hackers by date:

Previous
From: Robert Haas
Date:
Subject: Re: [HACKERS] [PATCH] Assert that the correct locks are held whencalling PageGetLSN()
Next
From: Tom Lane
Date:
Subject: Re: [HACKERS] Missing SIZE_MAX