Re: Compress prune/freeze records with Delta Frame of Reference algorithm - Mailing list pgsql-hackers

From Andrey Borodin
Subject Re: Compress prune/freeze records with Delta Frame of Reference algorithm
Date
Msg-id 2D414DAA-99D6-4566-86EE-E5FC8C21F4EA@yandex-team.ru
Whole thread Raw
In response to Compress prune/freeze records with Delta Frame of Reference algorithm  (Evgeny Voropaev <evgeny.voropaev@tantorlabs.com>)
Responses Re: Compress prune/freeze records with Delta Frame of Reference algorithm
List pgsql-hackers

> On 7 Mar 2026, at 18:56, Evgeny Voropaev <evgeny.voropaev@tantorlabs.com> wrote:
>
> Looking forward to discussion!

Great idea and nice library for DFoR!

Prune/freeze records can be large, and reducing their size would help streaming
replication a lot. The algorithm itself (Delta Frame of Reference bit-packing) is
sound for sorted integer sequences.

Eventually we will have wholesale WAL compression, but proposed here type of
compression is more perspective for particular case, because it exploits knowledge
about nature of compressed data.

The patchset is, obviously, a prototype that worth working on further.
Perhaps, first thing to start is to fix CI failures [0].

I do not know if we can budle LGPL lib. Luckily it's only for tests, when all
design questions are resolved we can just re-implement tests with something else.

As a minor nit: do not use stdlib assert(), use capital Assert() :)

Are we 100% sure qsort() won't allocate something anywhere? sort_template.h seems
to be allocation-free, but just in case...


Best regards, Andrey Borodin.


[0] https://github.com/x4m/postgres_g/runs/67140025428


pgsql-hackers by date:

Previous
From: Mahendra Singh Thalor
Date:
Subject: Re: pg_restore: remove unnecessary code from restore_all_databases function
Next
From: Lakshmi N
Date:
Subject: Re: Add missing CHECK_FOR_INTERRUPTS calls in dblink module