Re: serverlog rotation/functions - Mailing list pgsql-hackers

Bruce Momjian wrote:
>>Ok, no limit (but a default maximum of 50k remains). And since it's 
>>superuser only, he hopefully knows what he does.
> 
> 
> Huh?  Why have a default maximum?

Just for convenience. Both start and size are optional parameters, but 
with start=0 and size=50000. Well, it's a very special function anyway, 
so we could require the user to supply all parameters. I'll remove it.

> 
> 
> Ah, I think it needs to close that as soon as it starts.  Don't other
> subprocesses do that?  That shared memory is very fragile and we don't
> want an errant pointer poking in there.

The result of an errant pointer writing to that shred mem would be

1) wrong pid for SysLogger, so it can't be signalled to rotate from backends
2) wrong timestamp, so backends don't know the latest logfile.

Nothing particularly crash prone really.


> This seems clean and fast enough to me:
> 
>     SELECT filename
>     FROM pg_dir_ls('/var/log')
>     ORDER BY 1 DESC
>     LIMIT 1

For a logfile listing function, this would look

SELECT MAX(startdate)  FROM pg_logfile_ls()


> 
> Considering that any query from a client is going to have to go through
> the parser and be executed, an 'ls' in a directory just isn't a
> measurable performance hit.
> 
> If you want run a test that does an 'ls' and one that doesn't to see
> that there is no measurable performance difference.
> 
So while a simple PG_RETURN_TIMESTAMP(logfiletimestamp) compared to a 
lengthy setof returning function is drastically faster, this appears 
much less drastic with parser overhead.

> I would not worry about the clock going backward.  PostgreSQL would have
> enough problems with timestamp columns moving backward that the file log
> times are the least of our problems.

I see, so the admin is in trouble anyway (what about PITR? Data column 
deviations appear harmless compared to restoration based on timestamps).

> 
> 
> You can't know that you aren't reading corrupt data if you read shared
> memory without a lock.  What if the write is happening as you read?

I thought about this quite a while.

If the shmem fields aren't written atomically (one is 32bit, one 64 bit, 
probably on dword boundaries so writing will happen at least processor 
bus wide, do we support any 16 bit processor?) the corruption 
consequences as above apply. In the case of the timestamp, the high word 
will rarely change anyway, only every 2^32 seconds...

Concurrent access on the logger pid would mean to call 
pg_logfile_rotate() while a killed logger is being restarted, which is 
creating a new logfile then anyway. This would send a SIGINT into outer 
space, maybe to the bgwriter triggering a checkpoint, or the postmaster 
shutting it down (gracefully, still unwanted).

BTW, the consequences of a trigger flag in shmem would be less because 
all that could happen was a log rotation (which appends to existing 
files, just in case syslogger died in the milliseconds after a rotation).

> 
> The only clean solution I can think of is to write an operating system
> file that contains the current log filename and read from that.  I
> believe such writes are atomic.  But again, this seems like overkill to
> me.

Ah wait.
Digging further behind SIGUSR1 I now *do* see a solution without pid in 
shmem, using SendPostmasterSignal. Well, a little hint from gurus would 
have helped...

I'll convert to this, *dropping* all shmem.


Regards,
Andreas