Re: Compression of bigger WAL records - Mailing list pgsql-hackers

From Andrey Borodin
Subject Re: Compression of bigger WAL records
Date
Msg-id CCA2457E-6E5B-42C6-A5BD-E79385E6A71C@yandex-team.ru
Whole thread
In response to Re: Compression of bigger WAL records  (Michael Paquier <michael@paquier.xyz>)
Responses Re: Compression of bigger WAL records
List pgsql-hackers
Hi Michael!

On 7 Oct 2026, at 08:46 UTC, Michael Paquier wrote:
> I would be interested in any scripts you may have, and attempt to
> reproduce the numbers you have posted.

Attached is a cleaned-up version of the replay script. It creates six
million rows with repeat(md5(g::text), 5), checkpoints, and scans the
table to log hint-bit FPIs with wal_log_hints=on. Then it crashes the
server and measures the "redo done" elapsed time, excluding the
end-of-recovery checkpoint. The original runs used -O2, no assertions,
fsync=off and shared_buffers=8GB.

There's synchronous insert before the immediate shutdown to ensure
the scan's WAL is flushed.

I found the original logs: 4.34 and 4.14 seconds without the patch and
2.32 and 2.33 with it. lz4 was just an unchanged control: 1.59 seconds in
all four runs. The 200MB figure was compressed WAL, not raw page images.
I have not remeasured these numbers on v10.

Tomas also tested context reuse in the wal_compression default thread
[0]: recovery went from 158 to 147 seconds, versus 145 with lz4.
ZSTD_createDCtx_internal disappeared from the profile, where it had
accounted for about 9% of sampled CPU time.

A void pointer under USE_ZSTD is fine with me, and I agree with keeping
the context in the reader.

Thank you!


Best regards, Andrey Borodin.

[0] https://www.postgresql.org/message-id/18d632e6-6fa9-42fc-9dda-bad166374aa5@vondra.me

Attachment

pgsql-hackers by date:

Previous
From: Tomás Senart
Date:
Subject: Re: [PATCH v1] Batch B-tree TIDs when building a bitmap
Next
From: Fujii Masao
Date:
Subject: Re: 041_checkpoint_at_promote.pl might fail due to race condition on child kill