Re: Hard limit on WAL space used (because PANIC sucks) - Mailing list pgsql-hackers

From MauMau
Subject Re: Hard limit on WAL space used (because PANIC sucks)
Date
Msg-id D117F8165E894FA1B805302504C6F1D7@maumau
Whole thread Raw
In response to Re: Hard limit on WAL space used (because PANIC sucks)  (Josh Berkus <josh@agliodbs.com>)
Responses Re: Hard limit on WAL space used (because PANIC sucks)
List pgsql-hackers
From: "Josh Berkus" <josh@agliodbs.com>
> There's actually three potential failure cases here:
>
> - One Volume: WAL is on the same volume as PGDATA, and that volume is
> completely out of space.
>
> - XLog Partition: WAL is on its own partition/volume, and fills it up.
>
> - Archiving: archiving is failing or too slow, causing the disk to fill
> up with waiting log segments.

I think there is one more case.  Is this correct?

- Failure of a disk containing data directory or tablespace
If checkpoint can't write buffers to disk because of disk failure, 
checkpoint cannot complete, thus WAL files accumulate in pg_xlog/.
This means that one disk failure will lead to postgres shutdown.


> xLog Partition
> --------------
>
> As Heikki pointed, out, a full dedicated WAL drive is hard to fix once
> it gets full, since there's nothing you can safely delete to clear
> space, even enough for a checkpoint record.

This sounds very scary.  Is it possible to complete recovery and start up 
postmaster with either or both of the following modifications?

[Idea 1]
During recovery, force archiving a WAL file and delete/recycle it in 
pg_xlog/ as soon as its contents are applied.

[Idea 2]
During recovery, when disk full is encountered at end-of-recovery 
checkpoint, force archiving all unarchived WAL files and delete/recycle them 
at that time.


Regards
MauMau





pgsql-hackers by date:

Previous
From: Robert Haas
Date:
Subject: Re: Optimising Foreign Key checks
Next
From: Robert Haas
Date:
Subject: Re: Cost limited statements RFC