Re: basebackup: add missing deflateEnd() in gzip compression sink - Mailing list pgsql-hackers
| From | Jianghua Yang |
|---|---|
| Subject | Re: basebackup: add missing deflateEnd() in gzip compression sink |
| Date | |
| Msg-id | CAAZLFmQOH+fy6=ybEQ-xGdX9Zee2xkXhfuzCfbWweXxJ_6iD-A@mail.gmail.com Whole thread |
| In response to | Re: basebackup: add missing deflateEnd() in gzip compression sink (Jianghua Yang <yjhjstz@gmail.com>) |
| Responses |
Re: basebackup: add missing deflateEnd() in gzip compression sink
|
| List | pgsql-hackers |
Hi Michael,
Thanks for the review and backpatch suggestion.
Here is v1 as a two-patch series. The first patch is unchanged from the
original submission. I've added a second patch that fixes another resource
leak I noticed nearby in pg_basebackup:
v1-0001: basebackup: add missing deflateEnd() calls in gzip compression sink
v1-0002: pg_basebackup: add missing close() for incremental manifest file
In the incremental backup code path, the file descriptor opened for reading
the manifest is never closed after the upload completes. Since the backup
can run for a long time after this point, the leaked descriptor remains
open for the entire duration.
Thanks for the review and backpatch suggestion.
Here is v1 as a two-patch series. The first patch is unchanged from the
original submission. I've added a second patch that fixes another resource
leak I noticed nearby in pg_basebackup:
v1-0001: basebackup: add missing deflateEnd() calls in gzip compression sink
v1-0002: pg_basebackup: add missing close() for incremental manifest file
In the incremental backup code path, the file descriptor opened for reading
the manifest is never closed after the upload completes. Since the backup
can run for a long time after this point, the leaked descriptor remains
open for the entire duration.
Regards,
Jianghua Yang
Jianghua Yang
Jianghua Yang <yjhjstz@gmail.com> 于2026年3月21日周六 06:09写道：
Thanks for reviewing, Michael.
> based on the argument of the permanent connection my take is that
> this cleanup warrants a backpatch.
Agreed. The current code leaks zlib internal state on every archive,
and on a long-lived replication connection those allocations just
pile up with no cleanup until the connection ends.Michael Paquier <michael@paquier.xyz> 于2026年3月20日周五 23:09写道：On Fri, Mar 20, 2026 at 10:02:44AM -0700, Jianghua Yang wrote:
> While reviewing the backup compression backends, I noticed that the gzip
> sink (basebackup_gzip.c) never calls deflateEnd(), unlike the lz4 and
> zstd sinks which properly free their compression contexts.
>
> This means each archive (one per tablespace) leaks ~256KB of zlib
> internal state until the backup's memory context is destroyed. On the
> ERROR path, the zlib context is not released at all since there is no
> gzip-specific cleanup callback.
Yes, you are right on this one. This code could be better in cleaning
up the resources it is working on (and note that all the other
gzip-related code paths call the end() part if doing an init() or an
init2()). Leaking that once per archive may not look like a big deal,
but this is backend-side code we are talking about, and on a
persistent replication connection it is not really cool to lose the
context of this data with garbage coming from zlib piling up, so based
on the argument of the permanent connection my take is that this
cleanup warrants a backpatch.
--
Michael
Attachment
pgsql-hackers by date: