pgsql: Fix poorly-sized buffers in astreamer compression modules. - Mailing list pgsql-committers

From Tom Lane
Subject pgsql: Fix poorly-sized buffers in astreamer compression modules.
Date
Msg-id E1w54RV-001G7r-06@gemulon.postgresql.org
Whole thread Raw
List pgsql-committers
Fix poorly-sized buffers in astreamer compression modules.

astreamer_gzip.c and astreamer_lz4.c left their decompression
output buffers at StringInfo's default allocation, merely 1kB.
This results in a lot of ping-ponging between the decompressor
and the next astreamer filter.  This patch increases these buffer
sizes to 256kB.  In a simple test this had a small but measurable
effect (saving a few percent) on the overall runtime of pg_waldump
for the gzipped-data case; I didn't bother measuring for lz4.

astreamer_zstd.c used ZSTD_DStreamOutSize() to size its
compression output buffer, but the libzstd API says you should use
ZSTD_CStreamOutSize(); ZSTD_DStreamOutSize() is for decompression.
The two functions seem to produce the same value (256kB) here, so
this is just cosmetic, but nonetheless we should play by the rules.

While these issues are old, they don't seem significant enough to
warrant back-patching.

Discussion: https://postgr.es/m/3424809.1774234940@sss.pgh.pa.us

Branch
------
master

Details
-------
https://git.postgresql.org/pg/commitdiff/6e243d81c54966e640870c96d73576376ec125c6

Modified Files
--------------
src/fe_utils/astreamer_gzip.c | 2 ++
src/fe_utils/astreamer_lz4.c  | 2 ++
src/fe_utils/astreamer_zstd.c | 2 +-
3 files changed, 5 insertions(+), 1 deletion(-)


pgsql-committers by date:

Previous
From: Álvaro Herrera
Date:
Subject: pgsql: Don't include storage/lock.h in so many headers
Next
From: Álvaro Herrera
Date:
Subject: pgsql: Avoid including clog.h in proc.h