Re: Compress ReorderBuffer spill files using LZ4 - Mailing list pgsql-hackers

From Amit Kapila
Subject Re: Compress ReorderBuffer spill files using LZ4
Date
Msg-id CAA4eK1Kd6vP1g5pJXoiPgfLQsn54hecsc06hxpNWV-9_xMa+5Q@mail.gmail.com
Whole thread Raw
In response to Compress ReorderBuffer spill files using LZ4  (Julien Tachoires <julmon@gmail.com>)
Responses Re: Compress ReorderBuffer spill files using LZ4
Re: Compress ReorderBuffer spill files using LZ4
Re: Compress ReorderBuffer spill files using LZ4
List pgsql-hackers
On Thu, Jun 6, 2024 at 4:28 PM Julien Tachoires <julmon@gmail.com> wrote:
>
> When the content of a large transaction (size exceeding
> logical_decoding_work_mem) and its sub-transactions has to be
> reordered during logical decoding, then, all the changes are written
> on disk in temporary files located in pg_replslot/<slot_name>.
> Decoding very large transactions by multiple replication slots can
> lead to disk space saturation and high I/O utilization.
>

Why can't one use 'streaming' option to send changes to the client
once it reaches the configured limit of 'logical_decoding_work_mem'?

>
> 2. Do we want a GUC to switch compression on/off?
>

It depends on the overhead of decoding. Did you try to measure the
decoding overhead of decompression when reading compressed files?

--
With Regards,
Amit Kapila.



pgsql-hackers by date:

Previous
From: Hannu Krosing
Date:
Subject: Re: Function and Procedure with same signature?
Next
From: Andrew Dunstan
Date:
Subject: Re: ssl tests fail due to TCP port conflict