Re: XLog size reductions: smaller XLRec block header for PG17 - Mailing list pgsql-hackers

On 18/05/2023 17:59, Matthias van de Meent wrote:
> Attached is a patch that reduces this overhead by up to 2 bytes by
> encoding how large the block data length field is into the block ID,
> and thus optionally reducing the block data's length field to 0 bytes.
> Examples: cross-page update records will now be 2 bytes shorter,
> because the record never registers any data for the new block of the
> update; pgbench transactions are now either 6 or 8 bytes smaller
> depending on whether the update crosses a page boundary (in xlog
> record size; after alignment it is 0 or 4/8 bytes, depending on
> MAXALIGN and whether the updates are cross-page updates).
> 
> It changes the block IDs used to fit in 6 bits, using the upper 2 bits
> of the block_id field to store how much data is contained in the
> record (0, <=UINT8_MAX, or <=UINT16_MAX bytes).

Perhaps we should introduce a few generic inline functions to do varint 
encoding. That could be useful in many places, while this scheme is very 
tailored for XLogRecordBlockHeader.

We could replace XLogRecordDataHeaderShort and XLogRecordDataHeaderLong 
with this too. With just one XLogRecordDataHeader, with a 
variable-length length field.

> This is part 1 of a series of patches trying to decrease the size of
> WAL; see also [0], [1] and [2] for more info on what's still to go.
> I'm working on a separate, much more involved patch for the XLogRecord
> header itself, similar to the patch in [1], which I expect to send
> sometime soon as well.
> Unless someone thinks the patches should be discussed as one series,
> I'm planning on posting that in another thread, as I don't see any
> meaningful dependencies between the patches, and the XLR header patch
> will be quite a bit larger than this one.
> 
> Kind regards,
> 
> Matthias van de Meent
> Neon, Inc.
> 
> [0] https://wiki.postgresql.org/wiki/Updating_the_WAL_infrastructure

Good ideas here. Eliminating the two padding bytes from XLogRecord in 
particular seems like a pure win.

> PS. Benchmark results on my system (5950x with other light tasks
> running) don't show an obviously negative effect in a 10-minute run
> with these arbitrary pgbench settings on a fresh cluster with default
> configuration:
> 
> ./pg_install/bin/pgbench postgres -j 2 -c 6 -T 600 -M prepared
> [...]
> master: tps = 375
> patched: tps = 381

That was probably not CPU limited, so that any overhead in generating 
the WAL would not show up. Try PGOPTIONS="-csynchronous_commit=off" and 
pgbench -N option. And make sure the scale is large enough that there is 
no lock contention. Also would be good to measure the overhead in 
replaying the WAL.

How much space saving does this yield?

- Heikki