Re: BUG #12910: Memory leak with logical decoding - Mailing list pgsql-bugs

From Andres Freund
Subject Re: BUG #12910: Memory leak with logical decoding
Date
Msg-id 20150409173455.GE9764@awork2.anarazel.de
Whole thread Raw
In response to Re: BUG #12910: Memory leak with logical decoding  (Peter Slavov <pet.slavov@gmail.com>)
Responses Re: BUG #12910: Memory leak with logical decoding  (Peter Slavov <pet.slavov@gmail.com>)
Re: BUG #12910: Memory leak with logical decoding  (Peter Slavov <pet.slavov@gmail.com>)
List pgsql-bugs
Hi,

On 2015-04-09 18:11:27 +0300, Peter Slavov wrote:
> I prepared a test case, that can reproduce the problem.

Yup. I can reproduce it... I did not (yet) have the time to run the test
to completion, but i believe the attached patch should fix the problem
(and also improve performance a bit...).

Using the SQL interface for such large transactions isn't going to be
fun as all of the data, due to the nature of the set returning function
implementation in postgres, will be additionally written into a
tuplestore. The streaming interface doesn't have that behaviour.

Additionally it's probably not a good idea to stream such a large
resultset via SELECT using psql - IIRC it'll try to store all that data
in memory :). Try something like
\copy (select * from pg_logical_slot_peek_changes('testing', null, 1)) TO /tmp/f
or such.

Greetings,

Andres Freund

Attachment

pgsql-bugs by date:

Previous
From: Peter Slavov
Date:
Subject: Re: BUG #12910: Memory leak with logical decoding
Next
From: Heikki Linnakangas
Date:
Subject: Re: PQexec() hangs on OOM