Re: zstd compression for pg_dump - Mailing list pgsql-hackers

From Tomas Vondra
Subject Re: zstd compression for pg_dump
Date
Msg-id f3947107-a95a-348c-263c-284ac7275ae0@enterprisedb.com
Whole thread Raw
In response to Re: zstd compression for pg_dump  (Justin Pryzby <pryzby@telsasoft.com>)
Responses Re: zstd compression for pg_dump
List pgsql-hackers
On 4/1/23 15:36, Justin Pryzby wrote:
>
> ...
>
>> If there are no concerns, why disable it outside Windows? I don't have a
>> good idea how beneficial the multi-threaded compression is, so I can't
>> quite judge the risk/benefits tradeoff.
> 
> Because it's a minor/fringe feature, and it's annoying to have platform
> differences (would we plan on relaxing the restriction in v17, or is it
> more likely we'd forget ?).
> 
> I realized how little I've tested with zstd workers myself.  And I think
> on cirrusci, the macos and freebsd tasks have zstd libraries with
> threading support, but it wasn't being exercised (because using :workers
> would cause the patch to fail unless it's supported everywhere).  So I
> updated the "for CI only" patch to 1) use meson wraps to compile zstd
> library with threading on linux and windows; and, 2) use zstd:workers=3
> "opportunistically" (but avoid failing if threads are not supported,
> since the autoconf task still doesn't have access to a library with
> thread support).  That's a great step, but it still seems bad that the
> thread stuff has been little exercised until now.  (Also, the windows
> task failed; I think that's due to a transient network issue).
> 

Agreed, let's leave the threading for PG17, depending on how beneficial
it turns out to be for pg_dump.

> Feel free to mess around with threads (but I'd much rather see the patch
> progress for zstd:long).

OK, understood. The long mode patch is pretty simple. IIUC it does not
change the format, i.e. in the worst case we could leave it for PG17
too. Correct?


regards

-- 
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company



pgsql-hackers by date:

Previous
From: Justin Pryzby
Date:
Subject: Re: Option to not use ringbuffer in VACUUM, using it in failsafe mode
Next
From: Andres Freund
Date:
Subject: Re: Non-superuser subscription owners