Thread: [7.0.2] rotating log files ...

[7.0.2] rotating log files ...

From
The Hermit Hacker
Date:
can someone recommend on how to rotate the following log files without
having to stop/start the server?

postgres 85333 pgsql    1w  VREG  13,131084       995 762037 /pgsql/logs/postmaster.5432.61308
postgres 85333 pgsql    2w  VREG  13,131084 316501241 762038 /pgsql/logs/5432.61308

its the 316meg one and growing that I'm more interested in doing, obviously ...

I'm starting my postmaster as:

#!/bin/tcsh
setenv PORT 5432
setenv POSTMASTER /pgsql/bin/postmaster
unlimit
${POSTMASTER} -B 384 -N 192 \
              -o "-F -o /pgsql/logs/${PORT}.$$ -S 32768" \
              -i -p ${PORT} -D/pgsql/data >& /pgsql/logs/postmaster.${PORT}.$$ &

where I was hoping that the '-o /pgsql/logs/${PORT}.$$' would create a
'per postgres process' log file, but $$ is taking on the PID of the parent
process, not the children ...

suggestions?

Marc G. Fournier                   ICQ#7615664               IRC Nick: Scrappy
Systems Administrator @ hub.org
primary: scrappy@hub.org           secondary: scrappy@{freebsd|postgresql}.org


Re: [7.0.2] rotating log files ...

From
"Mitch Vincent"
Date:
Can you move the logfile then touch the original file to create an empty
one?

*shrug* just an idea :-)

-Mitch

----- Original Message -----
From: "The Hermit Hacker" <scrappy@hub.org>
To: <pgsql-admin@postgresql.org>
Sent: Monday, August 28, 2000 10:59 AM
Subject: [ADMIN] [7.0.2] rotating log files ...


>
> can someone recommend on how to rotate the following log files without
> having to stop/start the server?
>
> postgres 85333 pgsql    1w  VREG  13,131084       995 762037
/pgsql/logs/postmaster.5432.61308
> postgres 85333 pgsql    2w  VREG  13,131084 316501241 762038
/pgsql/logs/5432.61308
>
> its the 316meg one and growing that I'm more interested in doing,
obviously ...
>
> I'm starting my postmaster as:
>
> #!/bin/tcsh
> setenv PORT 5432
> setenv POSTMASTER /pgsql/bin/postmaster
> unlimit
> ${POSTMASTER} -B 384 -N 192 \
>               -o "-F -o /pgsql/logs/${PORT}.$$ -S 32768" \
>               -i -p ${PORT} -D/pgsql/data >&
/pgsql/logs/postmaster.${PORT}.$$ &
>
> where I was hoping that the '-o /pgsql/logs/${PORT}.$$' would create a
> 'per postgres process' log file, but $$ is taking on the PID of the parent
> process, not the children ...
>
> suggestions?
>
> Marc G. Fournier                   ICQ#7615664               IRC Nick:
Scrappy
> Systems Administrator @ hub.org
> primary: scrappy@hub.org           secondary:
scrappy@{freebsd|postgresql}.org
>
>


Re: [7.0.2] rotating log files ...

From
Lamar Owen
Date:
The Hermit Hacker wrote:
[snipped log stuff]
> suggestions?

Change to run with syslog.  As of 7.0.2, syslog is working quite well --
and syslog targets are easily rollable.

Since I've switched to syslog here, I haven't had any troubles with
logs.

--
Lamar Owen
WGCR Internet Radio
1 Peter 4:11

Re: [7.0.2] rotating log files ...

From
Alfred Perlstein
Date:
* The Hermit Hacker <scrappy@hub.org> [000828 11:19] wrote:
>
> can someone recommend on how to rotate the following log files without
> having to stop/start the server?
>
> postgres 85333 pgsql    1w  VREG  13,131084       995 762037 /pgsql/logs/postmaster.5432.61308
> postgres 85333 pgsql    2w  VREG  13,131084 316501241 762038 /pgsql/logs/5432.61308
>
> its the 316meg one and growing that I'm more interested in doing, obviously ...

This may not work: make a copy of it and try running a program that
calls ftruncate() on the file.

I think postgresql needs some utility to do this, I may whip one up when
time allows, it should be trivial.

-Alfred

Re: [7.0.2] rotating log files ...

From
Alfred Perlstein
Date:
* Mitch Vincent <mitch@venux.net> [000828 11:28] wrote:
> Can you move the logfile then touch the original file to create an empty
> one?
>
> *shrug* just an idea :-)

that won't work, the shell has attached the process's stdout/err to
the file, you can even remove the file and the server will still have
an open reference to it and the space won't be reclaimed.

-Alfred

Re: [7.0.2] rotating log files ...

From
"Mitch Vincent"
Date:
Yeah, I thought about that after I sent the email. What can I say, I think
with my index finger sometimes instead of my brain :-)

-Mitch
----- Original Message -----
From: "Alfred Perlstein" <bright@wintelcom.net>
To: "Mitch Vincent" <mitch@venux.net>
Cc: "The Hermit Hacker" <scrappy@hub.org>; <pgsql-admin@postgresql.org>
Sent: Monday, August 28, 2000 11:30 AM
Subject: Re: [ADMIN] [7.0.2] rotating log files ...


> * Mitch Vincent <mitch@venux.net> [000828 11:28] wrote:
> > Can you move the logfile then touch the original file to create an empty
> > one?
> >
> > *shrug* just an idea :-)
>
> that won't work, the shell has attached the process's stdout/err to
> the file, you can even remove the file and the server will still have
> an open reference to it and the space won't be reclaimed.
>
> -Alfred
>


Re: [7.0.2] rotating log files ...

From
Chris Ryan
Date:
It's not normally a good idea to move/remove files that a process has an
open file handle for. The safest way is to copy the file to a new
location/name then cat /dev/null > logfile

Chris

Mitch Vincent wrote:
>
> Can you move the logfile then touch the original file to create an empty
> one?
>
> *shrug* just an idea :-)
>
> -Mitch
>
> ----- Original Message -----
> From: "The Hermit Hacker" <scrappy@hub.org>
> To: <pgsql-admin@postgresql.org>
> Sent: Monday, August 28, 2000 10:59 AM
> Subject: [ADMIN] [7.0.2] rotating log files ...
>
> >
> > can someone recommend on how to rotate the following log files without
> > having to stop/start the server?
> >
> > postgres 85333 pgsql    1w  VREG  13,131084       995 762037
> /pgsql/logs/postmaster.5432.61308
> > postgres 85333 pgsql    2w  VREG  13,131084 316501241 762038
> /pgsql/logs/5432.61308
> >
> > its the 316meg one and growing that I'm more interested in doing,
> obviously ...
> >
> > I'm starting my postmaster as:
> >
> > #!/bin/tcsh
> > setenv PORT 5432
> > setenv POSTMASTER /pgsql/bin/postmaster
> > unlimit
> > ${POSTMASTER} -B 384 -N 192 \
> >               -o "-F -o /pgsql/logs/${PORT}.$$ -S 32768" \
> >               -i -p ${PORT} -D/pgsql/data >&
> /pgsql/logs/postmaster.${PORT}.$$ &
> >
> > where I was hoping that the '-o /pgsql/logs/${PORT}.$$' would create a
> > 'per postgres process' log file, but $$ is taking on the PID of the parent
> > process, not the children ...
> >
> > suggestions?
> >
> > Marc G. Fournier                   ICQ#7615664               IRC Nick:
> Scrappy
> > Systems Administrator @ hub.org
> > primary: scrappy@hub.org           secondary:
> scrappy@{freebsd|postgresql}.org
> >
> >

Re: [7.0.2] rotating log files ...

From
Dana Hudes
Date:
That won't work in and of itself since the filehandle to the open
file remains. Possibly do that and HUP the owning process.
I've no idea what happens when that particular process gets SIGHUP
(just because I have the source doesn't mean I read it for  no reason)

On Mon, 28 Aug 2000, Mitch Vincent wrote:

> Can you move the logfile then touch the original file to create an empty
> one?
>
> *shrug* just an idea :-)
>
> -Mitch
>
> ----- Original Message -----
> From: "The Hermit Hacker" <scrappy@hub.org>
> To: <pgsql-admin@postgresql.org>
> Sent: Monday, August 28, 2000 10:59 AM
> Subject: [ADMIN] [7.0.2] rotating log files ...
>
>
> >
> > can someone recommend on how to rotate the following log files without
> > having to stop/start the server?
> >
> > postgres 85333 pgsql    1w  VREG  13,131084       995 762037
> /pgsql/logs/postmaster.5432.61308
> > postgres 85333 pgsql    2w  VREG  13,131084 316501241 762038
> /pgsql/logs/5432.61308
> >
> > its the 316meg one and growing that I'm more interested in doing,
> obviously ...
> >
> > I'm starting my postmaster as:
> >
> > #!/bin/tcsh
> > setenv PORT 5432
> > setenv POSTMASTER /pgsql/bin/postmaster
> > unlimit
> > ${POSTMASTER} -B 384 -N 192 \
> >               -o "-F -o /pgsql/logs/${PORT}.$$ -S 32768" \
> >               -i -p ${PORT} -D/pgsql/data >&
> /pgsql/logs/postmaster.${PORT}.$$ &
> >
> > where I was hoping that the '-o /pgsql/logs/${PORT}.$$' would create a
> > 'per postgres process' log file, but $$ is taking on the PID of the parent
> > process, not the children ...
> >
> > suggestions?
> >
> > Marc G. Fournier                   ICQ#7615664               IRC Nick:
> Scrappy
> > Systems Administrator @ hub.org
> > primary: scrappy@hub.org           secondary:
> scrappy@{freebsd|postgresql}.org
> >
> >
>


Re: [7.0.2] rotating log files ...

From
Ragnar Kjørstad
Date:
On Mon, Aug 28, 2000 at 02:31:29PM -0400, Chris Ryan wrote:
> It's not normally a good idea to move/remove files that a process has an
> open file handle for. The safest way is to copy the file to a new
> location/name then cat /dev/null > logfile

What's wrong with moving the file when it's in use?

Copying the file will take much longer, and you might loose log-entries
that are written after cp but before truncate.


--
Ragnar Kjorstad

Re: [7.0.2] rotating log files ...

From
Dana Hudes
Date:
*falls over in astonishment, gagging*
*switch to CS professor mode*

Filesystem corruption result highly likely.
Reason: while you are  touching the file, so is the process.
It has a file handle open.  Actually should be locked and it would
be best if mv would honor that.
Under other operating systems such as VMS, the file would be locked
only SYSADMIN could mess with it.
Think -- Linux is a Unix variant, a true multitasking
multiprocessing operating  (with thread (lightweight process) support, no
less) system. While you are typing your mv command the process is running
unless you suspend it somehow. Therefore it is busy writing to disk
while you mess with its file. Even if you catch between writes, there is
the write cache / ouptut buffers (which is why a close is often preceded
by a fflush).

REally you need to take some basic operating systems course (or at least
read a book on the subject) if you do not understand.

I will soon have my recommended books list going and will have a book on
o/s there...


On Mon, 28 Aug 2000, [iso-8859-1] Ragnar Kj�rstad wrote:

> On Mon, Aug 28, 2000 at 02:31:29PM -0400, Chris Ryan wrote:
> > It's not normally a good idea to move/remove files that a process has an
> > open file handle for. The safest way is to copy the file to a new
> > location/name then cat /dev/null > logfile
>
> What's wrong with moving the file when it's in use?
>
> Copying the file will take much longer, and you might loose log-entries
> that are written after cp but before truncate.
>
>
>


Re: [7.0.2] rotating log files ...

From
Alfred Perlstein
Date:
> > On Mon, Aug 28, 2000 at 02:31:29PM -0400, Chris Ryan wrote:
> > > It's not normally a good idea to move/remove files that a process has an
> > > open file handle for. The safest way is to copy the file to a new
> > > location/name then cat /dev/null > logfile
> >
> > What's wrong with moving the file when it's in use?
> >
> > Copying the file will take much longer, and you might loose log-entries
> > that are written after cp but before truncate.

* Dana Hudes <dhudes@hudes.org> [000828 13:07] wrote:
> *falls over in astonishment, gagging*
> *switch to CS professor mode*
>
> Filesystem corruption result highly likely.
> Reason: while you are  touching the file, so is the process.
> It has a file handle open.  Actually should be locked and it would
> be best if mv would honor that.
> Under other operating systems such as VMS, the file would be locked
> only SYSADMIN could mess with it.
> Think -- Linux is a Unix variant, a true multitasking
> multiprocessing operating  (with thread (lightweight process) support, no
> less) system. While you are typing your mv command the process is running
> unless you suspend it somehow. Therefore it is busy writing to disk
> while you mess with its file. Even if you catch between writes, there is
> the write cache / ouptut buffers (which is why a close is often preceded
> by a fflush).

This is half wrong, if the rename operation happens on the same
filesystem then there is no chance of corruption.  The OS will take
care of this.  However if it's a mv to another filesystem then what
you're saying is a possibility.

I'm not sure if you miss-parsed Chris Ryan's question which I take
as a rename within the same filesystem, but I would not be so eager
to jump down his throat if you're not completely familiar with the
OS internals or not sure of the question.

      (ponders how trivial it might be to write a program that swaps)
. o O (out open files out from under a running program)

This is a good way of fixing the problem:

cp logfile someplaceelse
killall -STOP postresql
# append any extra data written to the log
truncate -s 0 logfile  # FreeBSD provides this as a userland tool
                       # you may have to write your own to call the OS's
                       # truncate/ftruncate routine
                       # or simply : cat /dev/null > logfile (my work)
killall -CONT postresql

you shouldn't loose any data that way.

--
-Alfred Perlstein - [bright@wintelcom.net|alfred@freebsd.org]
"I have the heart of a child; I keep it in a jar on my desk."

RE: [7.0.2] rotating log files ...

From
Neil Toronto
Date:
The process still has an open file handle, and will continue to do so even
after you move it.  So, if your file is /var/log/messages, and you do a mv
/var/log/messages /var/log/messages.old or something (I know that's stupid,
but this is an example), the process will continue to write to
/var/log/messages.old.

The best way is what Chris said: copy the file, and cat /dev/null > logfile.
That'll truncate it well.

-----Original Message-----
From: Ragnar Kjørstad [mailto:postgres@ragnark.vestdata.no]

What's wrong with moving the file when it's in use?

Copying the file will take much longer, and you might loose log-entries
that are written after cp but before truncate.


--
Ragnar Kjorstad

RE: [7.0.2] rotating log files ...

From
The Hermit Hacker
Date:
Just as an FYI ... I just enabled the syslog facility so that it all goes
through there instead ...



On Mon, 28 Aug 2000, Neil Toronto wrote:

> The process still has an open file handle, and will continue to do so even
> after you move it.  So, if your file is /var/log/messages, and you do a mv
> /var/log/messages /var/log/messages.old or something (I know that's stupid,
> but this is an example), the process will continue to write to
> /var/log/messages.old.
>
> The best way is what Chris said: copy the file, and cat /dev/null > logfile.
> That'll truncate it well.
>
> -----Original Message-----
> From: Ragnar Kj�rstad [mailto:postgres@ragnark.vestdata.no]
>
> What's wrong with moving the file when it's in use?
>
> Copying the file will take much longer, and you might loose log-entries
> that are written after cp but before truncate.
>
>
> --
> Ragnar Kjorstad
>

Marc G. Fournier                   ICQ#7615664               IRC Nick: Scrappy
Systems Administrator @ hub.org
primary: scrappy@hub.org           secondary: scrappy@{freebsd|postgresql}.org


Re: [7.0.2] rotating log files ...

From
Ragnar Kjørstad
Date:
On Mon, Aug 28, 2000 at 03:53:11PM -0400, Dana Hudes wrote:
> Filesystem corruption result highly likely.
> Reason: while you are  touching the file, so is the process.
> It has a file handle open.  Actually should be locked and it would
> be best if mv would honor that.
> Under other operating systems such as VMS, the file would be locked
> only SYSADMIN could mess with it.
> Think -- Linux is a Unix variant, a true multitasking
> multiprocessing operating  (with thread (lightweight process) support, no
> less) system. While you are typing your mv command the process is running
> unless you suspend it somehow. Therefore it is busy writing to disk
> while you mess with its file. Even if you catch between writes, there is
> the write cache / ouptut buffers (which is why a close is often preceded
> by a fflush).

I think you may have gotten this wrong...

1. On linux (and most other unixes, I believe), the file-operations
(like read, write and so on) are not tied with the filenames at all.
They all work on an open filedescriptor, wich is linked to an inode.
(via a dentry, but I don't think that's rellevant). If the filename
is changed, or even removed, they don't care. Actually, if you look at
how tmpfiles are implemented, you'll find that in most systems they do
(open, unlink) and then go on working with a file.

2. rename is guaranteed to be atomic by posix, if I'm not mistaken.


The problem you're probably refering to is when deleted files stay open
and the system crashes. On non-journaling filesystems the blocks will
not be freed until fsck is run. A few filesystems have had bugs that
prevented the space from beeing freed up again - I'm not sure if theese
problems were only caused when the system crashed or also under normal
usage.

> REally you need to take some basic operating systems course (or at least
> read a book on the subject) if you do not understand.

Actually I have, and I'we been working with linux-kernel development for
some time (e.g. the reiserfs filesystem for linux) so I believe I have
some idea of what I'm talking about. I'm not too arrogant to learn
though, so let me know if there are errors in my argument.


--
Ragnar Kjorstad

Re: [7.0.2] rotating log files ...

From
Ragnar Kjørstad
Date:
On Mon, Aug 28, 2000 at 02:37:24PM -0600, Neil Toronto wrote:
> The process still has an open file handle, and will continue to do so even
> after you move it.  So, if your file is /var/log/messages, and you do a mv
> /var/log/messages /var/log/messages.old or something (I know that's stupid,
> but this is an example), the process will continue to write to
> /var/log/messages.old.

Naturally, you need to HUP the process when you're done moving the file.
(and ofcourse, you need to have the rotated version on the same
filesystem)

> The best way is what Chris said: copy the file, and cat /dev/null > logfile.
> That'll truncate it well.

Copying the file will take much longer, and you might loose log-entries
that are written after cp but before truncate.

--
Ragnar Kjorstad

Re: [7.0.2] rotating log files ...

From
Palle Girgensohn
Date:
Hi,

Apart from using syslog, which apparently work fine (?) with
7.0.2, the 'rotatelogs' utility from apache's support dir is
great (No need to hack one up, as Alfred Perlstein suggest;
this babe been around and is quite stable).

Of course, you need to stop the server once, but I guess other
ways have been thouroughly dicussed in already :)

Enclosed is a Bournce shell script showing how I start postgres
on developers' machines [debug on] (run as root, will su pgsql)

On syslog... is syslog support reliable now? If so,
admin/pg-options.htm should be updated:

...Unfortunately many functions call directly printf() to print
their messages to stdout or stderr and this output can't be
redirected to syslog or have timestamps in it....

Cheers,
Palle

The Hermit Hacker wrote:
>
> can someone recommend on how to rotate the following log files without
> having to stop/start the server?
...
> suggestions?
>
> Marc G. Fournier                   ICQ#7615664               IRC Nick: Scrappy
> Systems Administrator @ hub.org
> primary: scrappy@hub.org           secondary: scrappy@{freebsd|postgresql}.org

Attachment

Re: [7.0.2] rotating log files ...

From
Bruce Momjian
Date:
> Hi,
>
> Apart from using syslog, which apparently work fine (?) with
> 7.0.2, the 'rotatelogs' utility from apache's support dir is
> great (No need to hack one up, as Alfred Perlstein suggest;
> this babe been around and is quite stable).
>
> Of course, you need to stop the server once, but I guess other
> ways have been thouroughly dicussed in already :)
>
> Enclosed is a Bournce shell script showing how I start postgres
> on developers' machines [debug on] (run as root, will su pgsql)
>
> On syslog... is syslog support reliable now? If so,
> admin/pg-options.htm should be updated:
>
> ...Unfortunately many functions call directly printf() to print
> their messages to stdout or stderr and this output can't be
> redirected to syslog or have timestamps in it....

Would someone be interested in converting our fprintf(stderr,...) calls
to elog and submitting a patch, please?

--
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026