Re: Add LZ4 compression in pg_dump - Mailing list pgsql-hackers

From Justin Pryzby
Subject Re: Add LZ4 compression in pg_dump
Date
Msg-id ZAfosKXCewHVENaZ@telsasoft.com
Whole thread Raw
In response to Re: Add LZ4 compression in pg_dump  (Tomas Vondra <tomas.vondra@enterprisedb.com>)
Responses Re: Add LZ4 compression in pg_dump  (Tomas Vondra <tomas.vondra@enterprisedb.com>)
List pgsql-hackers
On Wed, Mar 01, 2023 at 04:52:49PM +0100, Tomas Vondra wrote:
> Thanks. That seems correct to me, but I find it somewhat confusing,
> because we now have
> 
>  DeflateCompressorInit vs. InitCompressorGzip
> 
>  DeflateCompressorEnd vs. EndCompressorGzip
> 
>  DeflateCompressorData - The name doesn't really say what it does (would
>                          be better to have a verb in there, I think).
> 
> I wonder if we can make this somehow clearer?

To move things along, I updated Georgios' patch:

Rename DeflateCompressorData() to DeflateCompressorCommon();
Rearrange functions to their original order allowing a cleaner diff to the prior code;
Change pg_fatal() to an assertion+comment;
Update the commit message and fix a few typos;

> Also, InitCompressorGzip says this:
> 
>    /*
>     * If the caller has defined a write function, prepare the necessary
>     * state. Avoid initializing during the first write call, because End
>     * may be called without ever writing any data.
>     */
>     if (cs->writeF)
>         DeflateCompressorInit(cs);
>
> Does it actually make sense to not have writeF defined in some cases?

InitCompressor is being called for either reading or writing, either of
which could be null:

src/bin/pg_dump/pg_backup_custom.c:     ctx->cs = AllocateCompressor(AH->compression_spec,
src/bin/pg_dump/pg_backup_custom.c-                                                              NULL,
src/bin/pg_dump/pg_backup_custom.c-                                                              _CustomWriteFunc);
--
src/bin/pg_dump/pg_backup_custom.c:     cs = AllocateCompressor(AH->compression_spec,
src/bin/pg_dump/pg_backup_custom.c-                                                     _CustomReadFunc, NULL);

It's confusing that the comment says "Avoid initializing...".  What it
really means is "Initialize eagerly...".  But that makes more sense in
the context of the commit message for this bugfix than in a comment.  So
I changed that too.

+       /* If deflation was initialized, finalize it */
                    
 
+       if (cs->private_data)
                    
 
+               DeflateCompressorEnd(AH, cs);
                    
 

Maybe it'd be more clear if this used "if (cs->writeF)", like in the
init function ?

-- 
Justin

Attachment

pgsql-hackers by date:

Previous
From: Andres Freund
Date:
Subject: Re: buildfarm + meson
Next
From: Andres Freund
Date:
Subject: Re: buildfarm + meson