Re: astreamer_lz4: fix bug of output pointer advancement in decompressor - Mailing list pgsql-hackers

From Tom Lane
Subject Re: astreamer_lz4: fix bug of output pointer advancement in decompressor
Date
Msg-id 1531718.1772644615@sss.pgh.pa.us
Whole thread Raw
In response to astreamer_lz4: fix bug of output pointer advancement in decompressor  (Chao Li <li.evan.chao@gmail.com>)
Responses Re: astreamer_lz4: fix bug of output pointer advancement in decompressor
Re: astreamer_lz4: fix bug of output pointer advancement in decompressor
List pgsql-hackers
Chao Li <li.evan.chao@gmail.com> writes:
> There have been a couple of LZ4-related patches recently, so I spent some time playing with the LZ4 path and found a
bugin astreamer_lz4_decompressor_content(). 

Yup, that's clearly wrong.  I failed to reproduce a crash with the
test hack you suggested, but no matter.  Pushed with some cosmetic
editorialization.

The track record of all this client-side-compression logic is
really quite awful :-(.  Another thing that I'm looking askance
at is the error handling, or rather lack of it:

        ret = LZ4F_decompress(mystreamer->dctx,
                              next_out, &out_size,
                              next_in, &read_size, NULL);

        if (LZ4F_isError(ret))
            pg_log_error("could not decompress data: %s",
                         LZ4F_getErrorName(ret));

        ... continue on our merry way ...

I suspect whoever wrote this thought pg_log_error is equivalent
to elog(ERROR), but it's not; it just prints a message.  It seems
highly unlikely to me that continuing onwards will result in a
good outcome.  I'm a bit inclined to s/pg_log_error/pg_fatal/
throughout these files, at least in places where there's no
visible effort to handle the error.

            regards, tom lane



pgsql-hackers by date:

Previous
From: Jacob Champion
Date:
Subject: Re: pgsql: Change default value of default_toast_compression to "lz4", when
Next
From: Fujii Masao
Date:
Subject: Re: Shutdown indefinitely stuck due to unflushed FPI_FOR_HINT record