Re: refactoring basebackup.c (zstd workers) - Mailing list pgsql-hackers

From Robert Haas
Subject Re: refactoring basebackup.c (zstd workers)
Date
Msg-id CA+TgmoYfV1ASq0QVR1QHaJnXfr+fPLoPByQL1SfWqjUuKtX_UA@mail.gmail.com
Whole thread Raw
In response to Re: refactoring basebackup.c (zstd workers)  (Justin Pryzby <pryzby@telsasoft.com>)
Responses Re: refactoring basebackup.c (zstd workers)  (Robert Haas <robertmhaas@gmail.com>)
List pgsql-hackers
On Mon, Mar 14, 2022 at 1:11 PM Justin Pryzby <pryzby@telsasoft.com> wrote:
> Internally, I was thinking they'd all be handled as first-class options, with
> separate struct fields and separate replication protocol options.  If an option
> isn't known, it'd be rejected on the client side, rather than causing an error
> on the server.

There's some appeal to that, but one downside is that it means that
the client can't be used to fetch data that is compressed in a way
that the server knows about and the client doesn't. I don't think
that's great. Why should, for example, pg_basebackup need to be
compiled with zstd support in order to request zstd compression on the
server side? If the server knows about the brand new
justin-magic-sauce compression algorithm, maybe the client should just
be able to request it and, when given various .jms files by the
server, shrug its shoulders and accept them for what they are. That
doesn't work if -Fp is involved, or similar, but it should work fine
for simple cases if we set things up right.

> Maybe there'd be an option parser for this in common/ (I think that might
> require having new data structure there too, maybe one for each compression
> method, or maybe a union{} to handles them all).  Most of the ~100 lines to
> support wal_compression='zstd:N' are to parse out the N.

Yes, it's actually a very simple feature now that we've got the rest
of the infrastructure set up correctly for it.

-- 
Robert Haas
EDB: http://www.enterprisedb.com



pgsql-hackers by date:

Previous
From: Andres Freund
Date:
Subject: Re: Non-replayable WAL records through overflows and >MaxAllocSize lengths
Next
From: Robert Haas
Date:
Subject: Re: obsolete reference to a SubPlan field