Thread: Log rotation?
With all the great work put into allowing true 24/7 operation, as distributed we're still unable to rotate the log file. While the log file tends to be smaller than the database system as a whole, this is still going to annoy people because they can't control disk usage without taking the server down. I've been playing with a little program I wrote whose sole purpose is to write its stdin to a file and close and reopen that file when it receives a signal. I figured this could work well when integrated transparently into pg_ctl. So, is log rotation a concern? Is this a reasonable solution? Other ideas? (No Grand Unified Logging Solutions please. And no, "use syslog" doesn't count.) -- Peter Eisentraut peter_e@gmx.net http://funkturm.homeip.net/~peter
> So, is log rotation a concern? Is this a reasonable solution? Other > ideas? > > (No Grand Unified Logging Solutions please. And no, "use syslog" doesn't > count.) What's the problem with using newsyslog or logrotate at the moment? (ie. use the system log rotator) Chris
Peter Eisentraut <peter_e@gmx.net> writes:
> With all the great work put into allowing true 24/7 operation, as
> distributed we're still unable to rotate the log file. While the log file
> tends to be smaller than the database system as a whole, this is still
> going to annoy people because they can't control disk usage without taking
> the server down.
> I've been playing with a little program I wrote whose sole purpose is to
> write its stdin to a file and close and reopen that file when it receives
> a signal. I figured this could work well when integrated transparently
> into pg_ctl.
Aren't there log-rotation utilities out there already? (I seem to
recall mention that Apache has one, for instance.) Seems like this
is a wheel we shouldn't have to reinvent.
Also, I kinda thought the long-range solution was to encourage everyone
to migrate to syslog logging ...
> And no, "use syslog" doesn't count.
Why not?
regards, tom lane
Tom Lane <tgl@sss.pgh.pa.us> writes: > > And no, "use syslog" doesn't count. > > Why not? The standard implementations of syslog lose log entries under heavy load, because they rely on a daemon which reads from a named pipe with a limited buffer space. This is not acceptable in a production system, since heavy load is often just the time you need to see the log entries. It would be possible to implement the syslog(3) interface in a different way, of course, which did not use syslogd. I don't know of any such implementation. (My personal preference these days is an approach like DJB's daemontools, which separates the handling of log entries from the program doing the logging.) Ian
Ian Lance Taylor <ian@airs.com> writes:
> Tom Lane <tgl@sss.pgh.pa.us> writes:
>>> And no, "use syslog" doesn't count.
>>
>> Why not?
> The standard implementations of syslog lose log entries under heavy
> load,
Okay, that's a sufficient answer for that point.
> (My personal preference these days is an approach like DJB's
> daemontools, which separates the handling of log entries from the
> program doing the logging.)
That still leads back to my first question, which is whether we can't
rely on someone else's logrotation code.
regards, tom lane
Tom Lane writes: > Aren't there log-rotation utilities out there already? (I seem to > recall mention that Apache has one, for instance.) Seems like this > is a wheel we shouldn't have to reinvent. I'm aware of the Apache rotatelogs utility, but I'm not completely satisfied with it. 1. It tries to do the rotating itself. I'd rather rely on the OS' rotating and archiving facilities. 2. Only offers a time-based rotate, no manual intervention possible (via signal). 3. We don't want to have to tell people to install Apache and patch their pg_ctl. 4. We don't want to include it in our distribution because the license contains an advertisement clause. It's not like what I wrote is going to look wildly different than theirs. There's only so much variation you can put into 100 lines of code. > > And no, "use syslog" doesn't count. > > Why not? 1. Might not be available (?) 2. Might not be reliable 3. Might not have root access 4. Not all messages will go through elog. This is a bug, but not trivial to fix. -- Peter Eisentraut peter_e@gmx.net http://funkturm.homeip.net/~peter
Christopher Kings-Lynne writes: > What's the problem with using newsyslog or logrotate at the moment? (ie. > use the system log rotator) The postmaster will never close the output file, so you can rotate all you want, the original file will never be abandoned. -- Peter Eisentraut peter_e@gmx.net http://funkturm.homeip.net/~peter
Thus spake Tom Lane
> Also, I kinda thought the long-range solution was to encourage everyone
> to migrate to syslog logging ...
>
> > And no, "use syslog" doesn't count.
>
> Why not?
Well, one "why not" might be that syslog is not a guaranteed delivery
logging system. It might be good enough for some applications but I
don't think that it should be forced on everyone.
--
D'Arcy J.M. Cain <darcy@{druid|vex}.net> | Democracy is three wolves
http://www.druid.net/darcy/ | and a sheep voting on
+1 416 425 1212 (DoD#0082) (eNTP) | what's for dinner.
Peter Eisentraut <peter_e@gmx.net> writes:
> Tom Lane writes:
>> Aren't there log-rotation utilities out there already? (I seem to
>> recall mention that Apache has one, for instance.) Seems like this
>> is a wheel we shouldn't have to reinvent.
> I'm aware of the Apache rotatelogs utility, but I'm not completely
> satisfied with it.
Okay, those are reasonable points. Given that it's only ~100 lines of
code, I'll withdraw my objection to rolling our own. Let's just do it.
regards, tom lane
At 08:54 PM 9/5/2001 -0700, Ian Lance Taylor wrote: >Tom Lane <tgl@sss.pgh.pa.us> writes: > > > > And no, "use syslog" doesn't count. > > > > Why not? > >The standard implementations of syslog lose log entries under heavy >load, because they rely on a daemon which reads from a named pipe with >a limited buffer space. This is not acceptable in a production >system, since heavy load is often just the time you need to see the >log entries. > >It would be possible to implement the syslog(3) interface in a >different way, of course, which did not use syslogd. I don't know of >any such implementation. > >(My personal preference these days is an approach like DJB's >daemontools, which separates the handling of log entries from the >program doing the logging.) > >Ian Greetings, Kind of ironic, I have been working on a similar logging system for Apache that works with PostgreSQL, and I just released 2.0-beta last night. My post to announcements was delayed, but you can check it out here: http://www.digitalstratum.com/pglogd/ If pgLOGd looks like something similar to what you are looking for, I could probably modify it to log for PostgreSQL. Two of its requirements during development were fast and robust, and similar to what you described above it does not "process" the entries, that is done later. You also got me thinking that maybe syslogd needs an overhaul too... Matthew
You may be interested in http://www.ietf.org/internet-drafts/draft-ietf-syslog-reliable-12.txt which builds a reliable syslog protocol on top of BEEP. There are free implementations of BEEP in C and Java at http://beepcore.org ----- Original Message ----- From: "Matthew Hagerty" <mhagerty@voyager.net> To: "Ian Lance Taylor" <ian@airs.com>; "Tom Lane" <tgl@sss.pgh.pa.us> Cc: "Peter Eisentraut" <peter_e@gmx.net>; "PostgreSQL Development" <pgsql-hackers@postgresql.org> Sent: Thursday, September 06, 2001 10:08 AM Subject: Re: [HACKERS] Log rotation? > At 08:54 PM 9/5/2001 -0700, Ian Lance Taylor wrote: > >Tom Lane <tgl@sss.pgh.pa.us> writes: > > > > > > And no, "use syslog" doesn't count. > > > > > > Why not? > > > >The standard implementations of syslog lose log entries under heavy > >load, because they rely on a daemon which reads from a named pipe with > >a limited buffer space. This is not acceptable in a production > >system, since heavy load is often just the time you need to see the > >log entries. > > > >It would be possible to implement the syslog(3) interface in a > >different way, of course, which did not use syslogd. I don't know of > >any such implementation. > > > >(My personal preference these days is an approach like DJB's > >daemontools, which separates the handling of log entries from the > >program doing the logging.) > > > >Ian > > Greetings, > > Kind of ironic, I have been working on a similar logging system for Apache > that works with PostgreSQL, and I just released 2.0-beta last night. My > post to announcements was delayed, but you can check it out here: > http://www.digitalstratum.com/pglogd/ > > If pgLOGd looks like something similar to what you are looking for, I could > probably modify it to log for PostgreSQL. Two of its requirements during > development were fast and robust, and similar to what you described above > it does not "process" the entries, that is done later. You also got me > thinking that maybe syslogd needs an overhaul too... > > Matthew > > > ---------------------------(end of broadcast)--------------------------- > TIP 1: subscribe and unsubscribe commands go to majordomo@postgresql.org
Yeah, I use FreeBSD's wonderful newsyslog utility, and I do my logging like
this:
su -l pgsql -c '[ -d ${PGDATA} ] && exec /usr/local/bin/pg_ctl
start -s -w -o "-i" -l /var/log/pgsql.log'
And my /etc/newsyslog.conf entry:
/var/log/pgsql.log pgsql:pgsql 600 3 4096 * Z
Chris
> -----Original Message-----
> From: pgsql-hackers-owner@postgresql.org
> [mailto:pgsql-hackers-owner@postgresql.org]On Behalf Of Peter Eisentraut
> Sent: Thursday, 6 September 2001 6:04 PM
> To: Christopher Kings-Lynne
> Cc: PostgreSQL Development
> Subject: Re: [HACKERS] Log rotation?
>
>
> Christopher Kings-Lynne writes:
>
> > What's the problem with using newsyslog or logrotate at the
> moment? (ie.
> > use the system log rotator)
>
> The postmaster will never close the output file, so you can rotate all you
> want, the original file will never be abandoned.
>
> --
> Peter Eisentraut peter_e@gmx.net http://funkturm.homeip.net/~peter
>
>
> ---------------------------(end of broadcast)---------------------------
> TIP 6: Have you searched our list archives?
>
> http://www.postgresql.org/search.mpl
>
Christopher Kings-Lynne writes:
> Yeah, I use FreeBSD's wonderful newsyslog utility, and I do my logging like
> this:
>
> su -l pgsql -c '[ -d ${PGDATA} ] && exec /usr/local/bin/pg_ctl
> start -s -w -o "-i" -l /var/log/pgsql.log'
>
> And my /etc/newsyslog.conf entry:
>
> /var/log/pgsql.log pgsql:pgsql 600 3 4096 * Z
Sorry, this does not convey any information to me. Does this newsyslog
thing do anything that's so smart that we should know about it?
--
Peter Eisentraut peter_e@gmx.net http://funkturm.homeip.net/~peter
Hi Peter, > I've been playing with a little program I wrote whose sole purpose is to > write its stdin to a file and close and reopen that file when it receives > a signal. I figured this could work well when integrated transparently > into pg_ctl. > > So, is log rotation a concern? Is this a reasonable solution? Other > ideas? There was a discussion of this over a year ago. After I contributed to the discussion Tom Lane suggested I write something, and in the tradition of software development I did so but never quite finished it ... I got to the point that it works for me, then got distracted before completing a test suite. You may well prefer your own code, but this one supports rotation on size and/or time basis as well as on receipt of SIGHUP, and places a timestamp on each line written. It's also pretty careful about errors, which is one of the things that was disliked about the Apache program last time it was discussed. I am happy to contribute the code using the standard PostgreSQL license if it's wanted. (If anyone wants it under a BSD license or GPL for another purpose that's fine too.) I use the code on HP-UX, and developed it on NetBSD. There shouldn't be too many portability problems lurking, other than the usual hassles of what % escape to use in printf() for off_t. I doubt anyone wants log files larger than a couple of GB anyway? :-) ftp://ftp.nemeton.com.au/pub/src/logwrite-1.0alpha.tar.gz > (No Grand Unified Logging Solutions please. And no, "use syslog" doesn't > count.) <grin> One improvement I suggest is that the postmaster be taught to start (and restart if necessary) the log program. This avoids fragile startup scripts and also avoids taking down PostgreSQL if someone sends the wrong signal to the log program. Cheers, Giles