Re: 001_rep_changes.pl fails due to publisher stuck on shutdown - Mailing list pgsql-hackers

From Shlok Kyal
Subject Re: 001_rep_changes.pl fails due to publisher stuck on shutdown
Date
Msg-id CANhcyEUpv-T2qYcOYaOxoCt6qfVZpUg65xtS8PwuL+DN9sBykQ@mail.gmail.com
Whole thread Raw
In response to Re: 001_rep_changes.pl fails due to publisher stuck on shutdown  (Shlok Kyal <shlok.kyal.oss@gmail.com>)
List pgsql-hackers
On Mon, 10 Jun 2024 at 15:10, Shlok Kyal <shlok.kyal.oss@gmail.com> wrote:
>
> On Thu, 6 Jun 2024 at 11:49, Kyotaro Horiguchi <horikyota.ntt@gmail.com> wrote:
> >
> > At Thu, 6 Jun 2024 12:49:45 +1000, Peter Smith <smithpb2250@gmail.com> wrote in
> > > Hi, I have reproduced this multiple times now.
> > >
> > > I confirmed the initial post/steps from Alexander. i.e. The test
> > > script provided [1] gets itself into a state where function
> > > ReadPageInternal (called by XLogDecodeNextRecord and commented "Wait
> > > for the next page to become available") constantly returns
> > > XLREAD_FAIL. Ultimately the test times out because WalSndLoop() loops
> > > forever, since it never calls WalSndDone() to exit the walsender
> > > process.
> >
> > Thanks for the repro; I believe I understand what's happening here.
> >
> > During server shutdown, the latter half of the last continuation
> > record may fail to be flushed. This is similar to what is described in
> > the commit message of commit ff9f111bce. While shutting down,
> > WalSndLoop() waits for XLogSendLogical() to consume WAL up to
> > flushPtr, but in this case, the last record cannot complete without
> > the continuation part starting from flushPtr, which is
> > missing. However, in such cases, xlogreader.missingContrecPtr is set
> > to the beginning of the missing part, but something similar to
> >
> > So, I believe the attached small patch fixes the behavior. I haven't
> > come up with a good test script for this issue. Something like
> > 026_overwrite_contrecord.pl might work, but this situation seems a bit
> > more complex than what it handles.
> >
> > Versions back to 10 should suffer from the same issue and the same
> > patch will be applicable without significant changes.
>
> I tested the changes for PG 12 to master as we do not support prior versions.
> The patch applied successfully for master and PG 16. I ran the test
> provided in [1] multiple times and it ran successfully each time.
> The patch did not apply on PG 15. I did a similar change for PG 15 and
> created a patch.  I ran the test multiple times and it was successful
> every time.
> The patch did not apply on PG 14 to PG 12. I did a similar change in
> each branch. But the tests did not pass in each branch.
I, by mistake, applied wrong changes in PG 14 to PG 12. I tested again
for all versions and the test ran successfully for all of them till
PG12.
I have also attached the patch which applies for PG14 to PG12.

Sorry for the inconvenience.

Thanks and Regards,
Shlok Kyal

Attachment

pgsql-hackers by date:

Previous
From: vignesh C
Date:
Subject: Re: confirmed flush lsn seems to be move backward in certain error cases
Next
From: Ranier Vilela
Date:
Subject: Re: list_free in addRangeTableEntryForJoin