Re: wal recycling problem - Mailing list pgsql-hackers

From Fabrice Chapuis
Subject Re: wal recycling problem
Date
Msg-id CAA5-nLC4o2YLeQnV2ts=CnzAHrMsMvmOukQiP2rwwzs-Mg3c=g@mail.gmail.com
Whole thread Raw
In response to Re: wal recycling problem  (Christoph Moench-Tegeder <cmt@burggraben.net>)
Responses Re: wal recycling problem
List pgsql-hackers
Thanks Christoph for your message.
Now I understand why the wals are preserved if logical replication is configured and enabled. The problem is that when a large volume of data is loaded into a database, for example during a pg_restore, the wal sender process associated with the logical replication slot will have to decrypt all of the wals generated during this operation which will take a long time and the restart_lsn will not be modified.
From a conceptual point of view I think that specific wals per subscription should be used and stored in the pg_replslot folder in order to avoid working directly on the wals of the instance. 

What do you think about this proposal?

Regards 

Fabrice


On Mon, Oct 2, 2023 at 12:06 PM Christoph Moench-Tegeder <cmt@burggraben.net> wrote:
Hi,

## Fabrice Chapuis (fabrice636861@gmail.com):

> on the other hand there are 2 slots for logical replication which display
> status extended. I don't understand why given that the confirmed_flush_lsn
> field that is up to date. The restart_lsn remains frozen, for what reason?

There you have it - "extended" means "holding wal". And as long as the
restart_lsn does not advance, checkpointer cannot free any wal beyond
that lsn. My first idea would be some long-running (or huge) transaction
which is in process (active or still being streamed). I'd recommend
looking into what the clients on these slots are doing.

Regards,
Christoph

--
Spare Space

pgsql-hackers by date:

Previous
From: Peter Eisentraut
Date:
Subject: Re: Make --help output fit within 80 columns per line
Next
From: Peter Eisentraut
Date:
Subject: Re: Request for comment on setting binary format output per session