Thread: log file rotate

log file rotate

From
Bruce Momjian
Date:
Tom, you didn't like Andreas' idea of allowing the user to rotate the
log files on demand.  Isn't that standard functionality for any logging
program in case you want to manually start a new log file?  Is there no
way to do this simply?  Is this a TODO?

--  Bruce Momjian                        |  http://candle.pha.pa.us pgman@candle.pha.pa.us               |  (610)
359-1001+  If your life is a hard drive,     |  13 Roberts Road +  Christ can be your backup.        |  Newtown Square,
Pennsylvania19073
 


Re: log file rotate

From
Tom Lane
Date:
Bruce Momjian <pgman@candle.pha.pa.us> writes:
> Tom, you didn't like Andreas' idea of allowing the user to rotate the
> log files on demand.

Give me a use case that requires that, and is sufficiently interesting
to justify even a marginal decrease in the reliability of the log
process.

Frankly, I do not believe that database users should have anything to do
with the log rotation process.  Do we have a TODO for allowing users to
force switching to a new WAL file segment?

> Is this a TODO?

IMHO, no.
        regards, tom lane


Re: log file rotate

From
Bruce Momjian
Date:
Tom Lane wrote:
> Bruce Momjian <pgman@candle.pha.pa.us> writes:
> > Tom, you didn't like Andreas' idea of allowing the user to rotate the
> > log files on demand.
> 
> Give me a use case that requires that, and is sufficiently interesting
> to justify even a marginal decrease in the reliability of the log
> process.
> 
> Frankly, I do not believe that database users should have anything to do
> with the log rotation process.  Do we have a TODO for allowing users to
> force switching to a new WAL file segment?

I thought rotatelogs supported it so we should in cases where someone
wanted to make a new log file to delete an unusually large one, like a 1
gig log file caused by some runaway process.  However, I see rotatelogs
doesn't have that capability so I guess we don't need it either.

--  Bruce Momjian                        |  http://candle.pha.pa.us pgman@candle.pha.pa.us               |  (610)
359-1001+  If your life is a hard drive,     |  13 Roberts Road +  Christ can be your backup.        |  Newtown Square,
Pennsylvania19073
 


Re: log file rotate

From
Tom Lane
Date:
Bruce Momjian <pgman@candle.pha.pa.us> writes:
> I thought rotatelogs supported it so we should in cases where someone
> wanted to make a new log file to delete an unusually large one, like a 1
> gig log file caused by some runaway process.

Hm?  We have a rotate-on-size parameter, so that's not going to happen.
        regards, tom lane


Re: log file rotate

From
Bruce Momjian
Date:
Tom Lane wrote:
> Bruce Momjian <pgman@candle.pha.pa.us> writes:
> > I thought rotatelogs supported it so we should in cases where someone
> > wanted to make a new log file to delete an unusually large one, like a 1
> > gig log file caused by some runaway process.
> 
> Hm?  We have a rotate-on-size parameter, so that's not going to happen.

Ah, OK.

--  Bruce Momjian                        |  http://candle.pha.pa.us pgman@candle.pha.pa.us               |  (610)
359-1001+  If your life is a hard drive,     |  13 Roberts Road +  Christ can be your backup.        |  Newtown Square,
Pennsylvania19073
 


Re: log file rotate

From
Andreas Pflug
Date:
Tom Lane wrote:
> Bruce Momjian <pgman@candle.pha.pa.us> writes:
> 
>>Tom, you didn't like Andreas' idea of allowing the user to rotate the
>>log files on demand.
> 
> 
> Give me a use case that requires that, and is sufficiently interesting
> to justify even a marginal decrease in the reliability of the log
> process.

- Debugging some stuff, a distinct period of work, I'd like to start 
with a fresh serverlog.
- When doing a backup, I'd like to continue with a fresh serverlog so I 
can see what was happening since that point in time.

MSSQL calls this sp_cycle_errorlog.

I can't see even a marginal decrease of reliability, since the logger 
already accepts SIGHUP, and will reevaluate postgresql.conf to set the 
rotation_requested flag accordingly; I'm just asking for setting the 
flag directly.


Regards,
Andreas


Re: log file rotate

From
Andreas Pflug
Date:
Tom Lane wrote:
>
> Give me a use case that requires that, and is sufficiently interesting
> to justify even a marginal decrease in the reliability of the log
> process.
> 
> Frankly, I do not believe that database users should have anything to do
> with the log rotation process.

The (super)user will sometimes purge old files, so he *does* have to do 
something with the rotation process' output. It's up to him what he 
regards as old, maybe everything before 'now'.

YAUC (yet another use case): rotating every day at 3 am.

Regards,
Andreas



switch WAL segment

From
Andreas Pflug
Date:
Tom Lane wrote:

>Do we have a TODO for allowing users to
> force switching to a new WAL file segment?

Together with PITR, this might make sense?

Regards,
Andreas


Re: switch WAL segment

From
Bruce Momjian
Date:
Andreas Pflug wrote:
> Tom Lane wrote:
> 
> >Do we have a TODO for allowing users to
> > force switching to a new WAL file segment?
> 
> Together with PITR, this might make sense?

I thought about this, and it would perhaps help, but it would bloat the
archive location if used repeatedly, perhaps every minute so you have a
recent version.  It also might not work on server shutdown because there
might not be time to do the archive.  I think we are going to need a
solution to archive the current file in a different way.

--  Bruce Momjian                        |  http://candle.pha.pa.us pgman@candle.pha.pa.us               |  (610)
359-1001+  If your life is a hard drive,     |  13 Roberts Road +  Christ can be your backup.        |  Newtown Square,
Pennsylvania19073
 


Re: switch WAL segment

From
Tom Lane
Date:
Bruce Momjian <pgman@candle.pha.pa.us> writes:
> Andreas Pflug wrote:
>> Tom Lane wrote:
>>> Do we have a TODO for allowing users to
>>> force switching to a new WAL file segment?
>> 
>> Together with PITR, this might make sense?

> I thought about this, and it would perhaps help, but it would bloat the
> archive location if used repeatedly, perhaps every minute so you have a
> recent version.  It also might not work on server shutdown because there
> might not be time to do the archive.  I think we are going to need a
> solution to archive the current file in a different way.

More to the point, if we did think that was worth doing then we'd just
make pg_start_backup do it automatically.  There's still no need for
user control.
        regards, tom lane


Re: switch WAL segment

From
Manfred Spraul
Date:
Andreas Pflug wrote:

> Tom Lane wrote:
>
>> Do we have a TODO for allowing users to
>> force switching to a new WAL file segment?
>
>
> Together with PITR, this might make sense?
>
Another idea:
Has anyone tried to put the WAL segment directory on a cluster 
filesystem and use that for cold (perhaps even hot) failover?
The archive script could apply completed wal segments to the backup 
node. If the primary node fails, the last (partial) segment is applied 
as well and the backup node is activated.

--   Manfred