Hi!
> On Jan 4, 2021, at 11:04 AM, Andrey Borodin <x4mmm@yandex-team.ru> wrote:
>
> Daniil, is levels definition compatible with libpq compression patch?
> +typedef struct Compress {
> + CompressionAlgorithm alg;
> + int level;
> + /* Is a nondefault level set ? This is useful since different compression
> + * methods have different "default" levels. For now we assume the levels
> + * are all integer, though.
> + */
> + bool level_set;
> +} Compress;
Similarly to this patch, it is also possible to define the compression level at the initialization stage in libpq
compressionpatch.
The difference is that in libpq compression patch the default compression level always equal to 1, independently of the
chosencompression algorithm.
> On Jan 4, 2021, at 11:04 AM, Andrey Borodin <x4mmm@yandex-team.ru> wrote:
>
> Libpq compression encountered some problems with memory consumption which required some extra config efforts.
> On Jan 4, 2021, at 12:06 PM, Justin Pryzby <pryzby@telsasoft.com> wrote:
>
> RAM use is not significantly different from zlib, except that zstd --long adds
> more memory.
Regarding ZSTD memory usage:
Recently I’ve made a couple of tests of libpq compression with different ZLIB/ZSTD compression levels which shown that
compressing/decompressingZSTD w/ high compression levels
require to allocate more virtual (Commited_AS) memory, which may be exploited by malicious clients:
https://www.postgresql.org/message-id/62527092-16BD-479F-B503-FA527AF3B0C2%40yandex-team.ru
We can avoid high memory usage by limiting the max window size to 8MB. This should effectively disable the support of
compressionlevels above 19:
https://www.postgresql.org/message-id/6A45DFAA-1682-4EF2-B835-C5F46615EC49%40yandex-team.ru
So maybe it is worthwhile to use similar restrictions in this patch.
—
Daniil Zakhlystov
