Re: POC: enable logical decoding when wal_level = 'replica' without a server restart - Mailing list pgsql-hackers

On Mon, Dec 8, 2025 at 1:06 PM Masahiko Sawada <sawada.mshk@gmail.com> wrote:
>
> On Fri, Dec 5, 2025 at 4:56 PM Masahiko Sawada <sawada.mshk@gmail.com> wrote:
> >
> > >
> > > Hmm, ISTM that the root cause is that we don't synchronously update
> > > the XLogLogicalInfo cache on the standby. A backend on the standby who
> > > started when logical decoding is disabled keeps holding
> > > XLogLogicalInfo = false until the promotion. So even if the startup
> > > process enables logical decoding when applying the STATUS_CHANGE
> > > record, logical decoding is enabled on the standby but the backend
> > > process would start logical decoding with XLogLogicalInfo = false,
> > > resulting in RelationIsAccessibleInLogicalDecoding() returns false. I
> > > missed the fact that we check XLogLogicalInfoActive() even read paths.
> > > Will fix it.
> > >
> >
> > I noticed that the XLogLogicalInfoXactCache doesn't work as expected
> > even on the primary; because the process keep using the
> > XLogLogicalInfoXactCache in a transaction, sending
> > PROCSIGNAL_BARRIER_UPDATE_XLOG_LOGICAL_INFO with a wait doesn't
> > actually ensure that all processes are writing logical-WAL records
> > before enabling logical decoding. XLogLogicalInfoXactCache could be
> > cached even read paths without XID assignment, for example when HOT
> > pruning happens while reading a table. So, if a process has been
> > executing only read queries while logical decoding is disabled, it has
> > XLogLogicalInfoXactCache=0 and can get an XID after logical decoding
> > is enabled, writing non-logical WAL records. Such read-only
> > transactions don't appear in xl_running_xacts so we cannot wait for
> > them to finish during logical decoding initialization.
> >
> > I think there are two possible solutions to resolve this issue:
> >
> > 1. Revert the XLogLogicalInfoXactCache idea. While we can simplify the
> > code, I'm concerned that it could be problematic if
> > XLogLogicalInfoActive() could return different results whenever it's
> > called. For instance, we should at least remove the assertion in
> > ExecuteTruncateGuts(), but we could face a similar issue in the
> > future.
>
> I'm concerned that even this idea could be problematic without waiting
> for all running transactions to finish before enabling logical
> decoding, depending on how we write WAL records. If
> XLogLogicalInfoActive() could return a different value within the same
> command, we could write a WAL record mixiting logical-information and
> non-logical information, because some functions use
> XLogLogicalInfoActive() several times before writing a one WAL record.
> Further, if there is a path where a transaction decides whether to
> include logical information based on XLogLogicalInfoActive() *before*
> getting an XID, the transaction might not appear in xl_running_xacts
> record, resulting in that logical decoding that started before the
> transaction decodes such WAL records in the CONSISTENT snapbuild
> state.
>

Are you imagining a case with following steps that could lead to a
problem: (a) backend determine whether to include logical_info in WAL
via XLogLogicalInfoActive() and the answer is false, (b) assign
transaction_id, (c) got a signal to update XLogLogicalInfo to enable
logical_info and the backend updates it, (d) backend still writes WAL
without logical_info because it uses the information determined in
(a), (e) logical decoding won't wait for such a transaction before
reaching consistent state as by that time xid was not assigned, (f)
decoding results are unpredictable because the WAL doesn't have
logical_info?

> Therefore, I'm thinking that in addition to making
> XLogLogicalInfoActive() return a consistent result within a single
> command or in a transaction, we would need to wait for all running
> transactions (including those who don't have an XID) to finish before
> enabling logical decoding.
>

So create_slot operation could wait for hours or in worst case days
especially if the system has any long running query. I don't know if
that would be a big concern for users but at least we need to provide
some monitoring capability as to what leads to large wait times for
the create slot operation.

--
With Regards,
Amit Kapila.