Re: Missing rows after migrating from postgres 11 to 12 with logical replication - Mailing list pgsql-general

From Lars Vonk
Subject Re: Missing rows after migrating from postgres 11 to 12 with logical replication
Date
Msg-id CAMX1ThjgOoov2pdDz+FMQPRRR5OBc-V4NX+NkfFsqQX9ZtOpcQ@mail.gmail.com
Whole thread Raw
In response to Re: Missing rows after migrating from postgres 11 to 12 with logical replication  (Adrian Klaver <adrian.klaver@aklaver.com>)
Responses Re: Missing rows after migrating from postgres 11 to 12 with logical replication  (Adrian Klaver <adrian.klaver@aklaver.com>)
List pgsql-general
Did you have some other replication running on the 11 instance?

Yes the 11 instance also had another (11) replica running. (But these logs are from the 12 instance)

The new 12 instance also had a replica running.

In any case what was the command logged just before the ERROR.

There is nothing logged.

These are the only log statements just before the error message, one second later the ERROR is logged:

2020-12-10 13:26:43 UTC::@:[5537]:LOG:  checkpoints are occurring too frequently (20 seconds apart)
2020-12-10 13:26:43 UTC::@:[5537]:HINT:  Consider increasing the configuration parameter "max_wal_size".
2020-12-10 13:26:43 UTC::@:[5537]:LOG:  checkpoint starting: wal

Lars

On Mon, Dec 21, 2020 at 11:51 PM Adrian Klaver <adrian.klaver@aklaver.com> wrote:
On 12/21/20 2:42 PM, Lars Vonk wrote:
>     What was being run when the above ERROR was triggered?
>
>
> The initial copy of a table. Other than that we ran select
> pg_size_pretty(pg_relation_size('table_name')) to see the current size
> of the table being copied to get a feeling on progress.
>
> And whenever we added a new table to the publication we ran ALTER
> SUBSCRIPTION migration REFRESH PUBLICATION; to add any new table to the
> subscription. But not around that timestamp, about 50 minutes before the
> first occurence of that ERROR. (no ERRORS after prior ALTER SUBSCRIPTIONs).
>
> But after the initial copy's ended there are more ERROR's on different
> WAL segments missing. Each missing wal segment is logged as ERROR a
> couple of times and then no more. After a couple of hours no errors are
> logged.

Something was looking for the WAL segment.

Did you have some other replication running on the 11 instance?

In any case what was the command logged just before the ERROR.

>
> Lars
>


--
Adrian Klaver
adrian.klaver@aklaver.com

pgsql-general by date:

Previous
From: Tom Lane
Date:
Subject: Re: Avoid excessive inlining?
Next
From: "Joel Jacobson"
Date:
Subject: Re: Avoid excessive inlining?