Memory leak in XLOG reader facility when decoding records (caused by WAL refactoring) - Mailing list pgsql-bugs

From Michael Paquier
Subject Memory leak in XLOG reader facility when decoding records (caused by WAL refactoring)
Date
Msg-id CAB7nPqQFqCDXX_R9qde3bjmzXVvuYnnXgTf-u5K3oAqk0VbG0Q@mail.gmail.com
Whole thread Raw
Responses Re: Memory leak in XLOG reader facility when decoding records (caused by WAL refactoring)  (Andres Freund <andres@anarazel.de>)
List pgsql-bugs
Hi all,

While running valgring on pg_rewind, I have noticed that each new call
to XLogReadRecord leaks XLogReaderState->main_data and
XLogReaderState->blocks[block_id].data because each one of them is
palloc'd to store respectively the main data of the record and the
data attached to each block. I think that the most correct fix to
prevent the leak is to free those pointers when calling ResetDecoder()
which is used to reset a reader state between two records.
==46805== 154 bytes in 1 blocks are definitely lost in loss record 103 of 167
==46805==    at 0x6B1B: malloc (in
/usr/local/Cellar/valgrind/3.10.1/lib/valgrind/vgpreload_memcheck-amd64-darwin.so)
==46805==    by 0x10000DB2D: pg_malloc_internal (fe_memutils.c:30)
==46805==    by 0x10000DE79: palloc (fe_memutils.c:118)
==46805==    by 0x1000052A0: DecodeXLogRecord (xlogreader.c:1203)
==46805==    by 0x100003AA9: XLogReadRecord (xlogreader.c:461)
==46805==    by 0x1000022F0: extractPageMap (parsexlog.c:78)
==46805==    by 0x10000188C: main (pg_rewind.c:280)

This leak is rather bad particularly when a node is in recovery,
because the recovering node can fail with an OOM after replaying a
certain amount of records, and even worse for a standby node which is
aimed to replay records indefinitely.

Attached is a patch fixing the leak, and I checked that it got fixed
after re-running valgrind. I am adding an entry in the open item list
for 9.5 as well.
Regards,
--
Michael

Attachment

pgsql-bugs by date:

Previous
From: Tom Lane
Date:
Subject: Re: BUG #13514: PostgreSQL backend process crashes on jsonb_object_agg()
Next
From: sushant@indiankanoon.com
Date:
Subject: BUG #13515: Much higher disk writes after postgres upgrade 9.4.3->9.4.4