Re: archive status ".ready" files may be created too early - Mailing list pgsql-hackers

From Bossart, Nathan
Subject Re: archive status ".ready" files may be created too early
Date
Msg-id D4ED6D6A-7999-4F21-B1E4-9C3FCE52A4B5@amazon.com
Whole thread Raw
In response to Re: archive status ".ready" files may be created too early  (Kyotaro Horiguchi <horikyota.ntt@gmail.com>)
List pgsql-hackers
On 8/4/21, 6:58 PM, "Kyotaro Horiguchi" <horikyota.ntt@gmail.com> wrote:
> Addition to that, while NotifySegmentsReadyForArchive() is notifying
> pending segments, other backends simultaneously reach there are
> blocked until the notification, incuding file creation, finishes.  I
> don't think that's great. Couldn't we set lastNotifiedSegment before
> the loop?  At the moment a backend decides to notify some segments,
> others no longer need to consider those segments.  Even if the backend
> crashes meanwhile, as you mentionied below, it's safe since the
> unnotified segments are notifed after restart.

That seems reasonable to me.  It looks like we rely on
RemoveOldXlogFiles() even today for when XLogArchiveNotify() fails.  I
updated this in v4 of the patch.

In addition to this change, I also addressed your other feedback by
changing XLogSegNoIsInvalid() to XLogSegNoIsValid() and by moving
record boundary registration to the "if" block for cross-page records.

> Does it work that RegisterRecordBoundaryEntry omits registering of the
> bounary if it finds lastNotifiedSeg have gone too far?

Yeah, there's no reason to add a record boundary if we've already
notified the prior segment.  For that to happen, another cross-segment
record would have to be flushed to disk and
NotifySegmentsReadyForArchive() would have to be called before
registering the boundary.  With that being said, I don't expect an
extra map entry here and there to impact performance enough for us to
worry about it.

Nathan


Attachment

pgsql-hackers by date:

Previous
From: Masahiko Sawada
Date:
Subject: Re: [BUG] wrong refresh when ALTER SUBSCRIPTION ADD/DROP PUBLICATION
Next
From: "Bossart, Nathan"
Date:
Subject: Re: archive status ".ready" files may be created too early