Re: PITR: XLog File compression on Archive - Mailing list pgsql-hackers

From Tom Lane
Subject Re: PITR: XLog File compression on Archive
Date
Msg-id 20810.1093301730@sss.pgh.pa.us
Whole thread Raw
In response to PITR: XLog File compression on Archive  ("Simon Riggs" <simon@2ndquadrant.com>)
List pgsql-hackers
"Simon Riggs" <simon@2ndquadrant.com> writes:
> As a result, I have thought that there may be a way to remove those pages
> from the xlog files immediately before being copied away to archive, without
> effecting crash recovery logic AT ALL.

This isn't all that easy.  The main problem I can see is that you have
to maintain (or be able to reconstruct) the absolute WAL offset of each
WAL record, because that number shows up in page LSNs in the data files.
There's also the question of keeping track of page boundaries in the WAL
files.  And random access to a WAL segment would go by the wayside ---
I think you'd have to scan forward from the file start to locate the
checkpoint record you intend to replay from.  The reader code would need
some nontrivial alterations to deal with all this, and you would
*definitely* need to know whether you were reading a compressed or
uncompressed WAL file.

I have zero interest in touching this problem for 8.0 ...
        regards, tom lane

PS: but something you *could* do in 8.0 is replace "cp" by "gzip" to
archive compressed files that way.


pgsql-hackers by date:

Previous
From: Gaetano Mendola
Date:
Subject: Re: PITR: XLog File compression on Archive
Next
From: Alvaro Herrera
Date:
Subject: Re: PITR: XLog File compression on Archive