Re: multithreaded zstd backup compression for client and server - Mailing list pgsql-hackers

From Robert Haas
Subject Re: multithreaded zstd backup compression for client and server
Date
Msg-id CA+Tgmobd5VMBrQmPD=vpGPOYqupsPcDaJpEFB9cTS3Sp4rNrxg@mail.gmail.com
Whole thread Raw
In response to Re: multithreaded zstd backup compression for client and server  (Justin Pryzby <pryzby@telsasoft.com>)
Responses Re: multithreaded zstd backup compression for client and server
List pgsql-hackers
On Wed, Mar 23, 2022 at 5:52 PM Justin Pryzby <pryzby@telsasoft.com> wrote:
> Also because the library may not be compiled with threading.  A few days ago, I
> tried to rebase the original "parallel workers" patch over the COMPRESS DETAIL
> patch but then couldn't test it, even after trying various versions of the zstd
> package and trying to compile it locally.  I'll try again soon...

Ah. Right, I can update the comment to mention that.

> I think you should also test the return value when setting the compress level.
> Not only because it's generally a good idea, but also because I suggested to
> support negative compression levels.  Which weren't allowed before v1.3.4, and
> then the range is only defined since 1.3.6 (ZSTD_minCLevel).  At some point,
> the range may have been -7..22 but now it's -131072..22.

Yeah, I was thinking that might be a good change. It would require
adjusting some other code though, because right now only compression
levels 1..22 are accepted anyhow.

> lib/compress/zstd_compress.c:int ZSTD_minCLevel(void) { return (int)-ZSTD_TARGETLENGTH_MAX; }
> lib/zstd.h:#define ZSTD_TARGETLENGTH_MAX    ZSTD_BLOCKSIZE_MAX
> lib/zstd.h:#define ZSTD_BLOCKSIZE_MAX     (1<<ZSTD_BLOCKSIZELOG_MAX)
> lib/zstd.h:#define ZSTD_BLOCKSIZELOG_MAX  17
> ; -1<<17
>         -131072

So does that, like, compress the value by making it way bigger? :-)

-- 
Robert Haas
EDB: http://www.enterprisedb.com



pgsql-hackers by date:

Previous
From: Andres Freund
Date:
Subject: Re: ubsan
Next
From: Tom Lane
Date:
Subject: Re: [PATCH] Expose port->authn_id to extensions and triggers