Re: Proposal: Conflict log history table for Logical Replication - Mailing list pgsql-hackers

From Amit Kapila
Subject Re: Proposal: Conflict log history table for Logical Replication
Date
Msg-id CAA4eK1KV3rYkaxys5fh-PtE9kq5xrFbiaRpOSPoRgQG494ek+g@mail.gmail.com
Whole thread Raw
In response to Re: Proposal: Conflict log history table for Logical Replication  (Dilip Kumar <dilipbalaut@gmail.com>)
Responses Re: Proposal: Conflict log history table for Logical Replication
List pgsql-hackers
On Mon, Dec 8, 2025 at 10:25 AM Dilip Kumar <dilipbalaut@gmail.com> wrote:
>
> On Thu, Dec 4, 2025 at 4:20 PM Amit Kapila <amit.kapila16@gmail.com> wrote:
> >
> > On Thu, Dec 4, 2025 at 10:49 AM Dilip Kumar <dilipbalaut@gmail.com> wrote:
> > >
> > > > ---
> > > > I think the conflict history table should not be transferred to the
> > > > new cluster when pg_upgrade since the table definition could be
> > > > different across major versions.
> > >
> > > Let me think more on this with respect to behaviour of other factors
> > > like subscriptions etc.
> > >
> >
> > Can we deal with different schema of tables across versions via
> > pg_dump/restore during upgrade?
> >
>
> While handling the case of conflict_log_table option during pg_dump, I
> realized that the restore is trying to create conflict log table 2
> different places 1) As part of the regular table dump 2) As part of
> the CREATE SUBSCRIPTION when conflict_log_table option is set.
>
> So one option is we can avoid dumping the conflict log tables as part
> of the regular table dump if we think that we do not need to conflict
> log table data and let it get created as part of the create
> subscription command, OTOH if we think we want to keep the conflict
> log table data,
>

We want to retain conflict_history after upgrade. This is required for
various reasons (a) after upgrade DBA user will still require to
resolved the pending unresolved conflicts, (b) Regulations often
require keeping audit trails for a longer period of time. If a
conflict occurred at time X (which is less than the regulations
requirement) regarding a financial transaction, that record must
survive the upgrade, (c)
If something breaks after the upgrade (e.g., missing rows, constraint
violations), conflict history helps trace root causes. It shows
whether issues existed before the upgrade or were introduced during
migration, (d) as users can query the conflict_history tables, it
should be treated similar to user tables.

BTW, we are also planning to migrate commit_ts data in thread [1]
which would be helpful for conflict_resolutions after upgrade.

 let it get dumped as part of the regular tables and in
> CREATE SUBSCRIPTION we will just set the option but do not create the
> table,
>

Yeah, we can turn this option during CREATE SUBSCRIPTION so that it
doesn't try to create the table again.

> although we might need to do special handling of this case
> because if we allow the existing tables to be set as conflict log
> tables then it may allow other user tables to be set, so need to think
> how to handle this if we need to go with this option.
>

Yeah, probably but it should be allowed internally only not to users.
I think we can split this upgrade handling as a top-up patch at least
for the purpose of review.

[1] - https://www.postgresql.org/message-id/182311743703924%40mail.yandex.ru

--
With Regards,
Amit Kapila.



pgsql-hackers by date:

Previous
From: Jakub Wartak
Date:
Subject: Parallel query: Use TopTransactionContext for ReinitializeParallelDSM()
Next
From: Victor Yegorov
Date:
Subject: Re: Moving _bt_readpage and _bt_checkkeys into a new .c file