Re: Avoid full page images in streaming replication? - Mailing list pgsql-hackers

From Michael Paquier
Subject Re: Avoid full page images in streaming replication?
Date
Msg-id CAB7nPqQJ7VnB4U_MzbS4vLs05v2gqum7FJv7JJwN6ZBAR-Jk=g@mail.gmail.com
Whole thread Raw
In response to Re: Avoid full page images in streaming replication?  (Jim Nasby <Jim.Nasby@BlueTreble.com>)
List pgsql-hackers
On Fri, Oct 23, 2015 at 7:37 AM, Jim Nasby wrote:
> What I'm wondering is how compressible a 'normal' FPI is. Certainly if the
> hole is zero'd out and the page is mostly empty you'll get great
> compression. What about other workloads? For reference, if a 'FPI
> placeholder' WAL record is 16 bytes, that's 51,200% compression. If it's 12
> bytes, it's 68,200% compression. (I'm assuming we write the hole too, but
> maybe that's not true?)

Well, to begin with FPI usually avoid to include the page hole in the
middle. Now, regarding the compressibility of a page taken without its
hole, that's highly schema-dependent. Based on some measurements I did
some time ago a page with repetitive data could compress up to 40%,
with less compressible stuff like UUID I recall it to be 20~25%. I
hacked out for the FPW compression patch a module able to work
directly on raw pages to test their compressibility:
https://github.com/michaelpq/pg_plugins/tree/master/compress_test
get_raw_page() has been taken from pageinspect and I added to it an
option to remove the hole in the middle of the page. Using that you
are able to guess how much pages can get compressed.
-- 
Michael



pgsql-hackers by date:

Previous
From: Tom Lane
Date:
Subject: Re: Making tab-complete.c easier to maintain
Next
From: Michael Paquier
Date:
Subject: Re: Avoid full page images in streaming replication?