Question about advance confirmed_flush_lsn using logic replicationslot when there is no modification. - Mailing list pgsql-general

From mo jia
Subject Question about advance confirmed_flush_lsn using logic replicationslot when there is no modification.
Date
Msg-id CAJ17HSf49rXqsjchdNZrZ4GbKv+w-nD75EudpUkfEMvp72UEFg@mail.gmail.com
Whole thread Raw
List pgsql-general
Hi:

I am not sure it is ok to ask this question in this lislt.

I met this problem using RDS Postgres. (I think the normal postgres
may have the same problem).

I have posted the question on StackOverflow here:

https://stackoverflow.com/questions/52589058/aws-rds-postgresql-transaction-logs-keep-going-when-there-is-no-data-change

I also try to got some suggestion from psycopg. (And got suggestion
asking there)
https://github.com/psycopg/psycopg2/issues/780

The problem may be described like this in simple words:
When there is no modification, the postgres wal( or transaction logs
in RDS) keep going increase.
pg_current_wal_lsn is keeping increase, however the
confirmed_flush_lsn in (select * from pg_replication_slots;) can not
be advanced.
In this case my problem using pgyscopy can't read message from the slot.

Someone have give me a answer:
RDS dms service
https://docs.aws.amazon.com/dms/latest/userguide/CHAP_ReleaseNotes.html

there is a wal-heartbeat concept:

Added a write-ahead log (WAL) heartbeat (that is, running dummy
queries) for replication from a PostgreSQL source. This feature was
added so that idle logical replication slots don't hold onto old WAL
logs, which can result in storage full situations on the source. This
heartbeat keeps restart_lsn moving and prevents storage full
scenarios.

Any more suggestion? I am more or less a application level programmer.
I think this problem need to understand more about the inner of
postgres and some protocol on replication mechanism.

Thanks.


pgsql-general by date:

Previous
From: magodo
Date:
Subject: how to cleanup archive based on datetime
Next
From: magodo
Date:
Subject: Re: how to identify the timeline of specified recovery_target_timewhen do multiple PITR