pgsql: Fix the logical replication timeout during large transactions. - Mailing list pgsql-committers

From Amit Kapila
Subject pgsql: Fix the logical replication timeout during large transactions.
Date
Msg-id E1nofFa-0007fr-TR@gemulon.postgresql.org
Whole thread Raw
List pgsql-committers
Fix the logical replication timeout during large transactions.

The problem is that we don't send keep-alive messages for a long time
while processing large transactions during logical replication where we
don't send any data of such transactions. This can happen when the table
modified in the transaction is not published or because all the changes
got filtered. We do try to send the keep_alive if necessary at the end of
the transaction (via WalSndWriteData()) but by that time the
subscriber-side can timeout and exit.

To fix this we try to send the keepalive message if required after
processing certain threshold of changes.

Reported-by: Fabrice Chapuis
Author: Wang wei and Amit Kapila
Reviewed By: Masahiko Sawada, Euler Taveira, Hou Zhijie, Hayato Kuroda
Backpatch-through: 10
Discussion: https://postgr.es/m/CAA5-nLARN7-3SLU_QUxfy510pmrYK6JJb=bk3hcgemAM_pAv+w@mail.gmail.com

Branch
------
REL_14_STABLE

Details
-------
https://git.postgresql.org/pg/commitdiff/d6da71fa8f28faa68823e163f318ffb38a7a9a54

Modified Files
--------------
src/backend/replication/logical/logical.c   | 27 ++++++++++++++++++
src/backend/replication/pgoutput/pgoutput.c | 44 +++++++++++++++++++++++++++--
src/backend/replication/walsender.c         | 38 +++++++++++++++++++++----
src/include/replication/logical.h           |  3 ++
4 files changed, 105 insertions(+), 7 deletions(-)


pgsql-committers by date:

Previous
From: Michael Paquier
Date:
Subject: pgsql: Silence extra logging when using "postgres -C" on runtime-comput
Next
From: Thomas Munro
Date:
Subject: pgsql: Add logging for excessive ProcSignalBarrier waits.