Re: BUG #12910: Memory leak with logical decoding - Mailing list pgsql-bugs

Hi I tried to do a backtrace, but I have nothing useful there. Only
lines like this:

Program received signal SIGUSR1, User defined signal 1.
Detaching after fork from child process 13074.
Detaching after fork from child process 13077.
Detaching after fork from child process 13083.
Detaching after fork from child process 13087.
Detaching after fork from child process 13093.
Detaching after fork from child process 13100.
Detaching after fork from child process 13105.
Detaching after fork from child process 13109.
Detaching after fork from child process 13115.
Detaching after fork from child process 13119.

And the process fails again with:

WARNING:  terminating connection because of crash of another server process
DETAIL:  The postmaster has commanded this server process to roll back
the current transaction and exit, because another server process exited
abnormally and possibly corrupted shared memory.
HINT:  In a moment you should be able to reconnect to the database and
repeat your command.
CONTEXT:  slot "peshka", output plugin "test_decoding", in the change
callback, associated LSN D0/F66B52C8
server closed the connection unexpectedly
     This probably means the server terminated abnormally
     before or while processing the request.
connection to server was lost

Peter Slavov


На 31.03.2015 в 17:40, Andres Freund написа:
> Hi,
>
> On 2015-03-27 09:47:57 +0000, pet.slavov@gmail.com wrote:
>> I am trying to use logical decoding to replay the data modifying queries on
>> a different server, to synchronize some of the tables. The primary server
>> has some load, but not so much. I am getting the changes with
>> pg_logical_slot_get_changes limiting the changes to 50 at a time.
> That's generally not a very good idea. It's much better to use the
> streaming interface. Repeatedly starting/stopping a logical slot (as the
> SQL interface has to do every time) isn't all that efficient.
>
>> At some point pg_logical_slot_get_changes queries become slow and starts to
>> eat all the ram and swap, which eventually kills the primary database with
>> this error:
>>   FATAL:  the database system is in recovery mode.
> That indicates a crash. Please check the server log for any details?  If
> it crashes, could you perhaps get a backtrace?
>
> What output plugin are you using?
>
>> The nature of changes on the primary can have a lot of data in one
>> transaction. Which I guess is the reason of the slow
>> pg_logical_slot_get_changes.
> In that case changes should just be spooled to disk.
>
> Greetings,
>
> Andres Freund