Re: prevent immature WAL streaming - Mailing list pgsql-hackers

From Kyotaro Horiguchi
Subject Re: prevent immature WAL streaming
Date
Msg-id 20210901.131541.1769424721914243648.horikyota.ntt@gmail.com
Whole thread Raw
In response to Re: prevent immature WAL streaming  (Andres Freund <andres@anarazel.de>)
List pgsql-hackers
At Tue, 31 Aug 2021 20:15:24 -0700, Andres Freund <andres@anarazel.de> wrote in 
> Hi,
> 
> On 2021-09-01 11:34:34 +0900, Fujii Masao wrote:
> > On 2021/09/01 0:53, Andres Freund wrote:
> > > Of course, we need to be careful to not weaken WAL validity checking too
> > > much. How about the following:
> > > 
> > > If we're "aborting" a continued record, we set XLP_FIRST_IS_ABORTED_PARTIAL on
> > > the page at which we do so (i.e. the page after the valid end of the WAL).
> > 
> > When do you expect that XLP_FIRST_IS_ABORTED_PARTIAL is set? It's set
> > when recovery finds a a partially-flushed segment-spanning record?
> > But maybe we cannot do that (i.e., cannot overwrite the page) because
> > the page that the flag is set in might have already been archived. No?
> 
> I was imagining that XLP_FIRST_IS_ABORTED_PARTIAL would be set in the "tail
> end" of a partial record. I.e. if there's a partial record starting in the
> successfully archived segment A, but the end of the record, in B, has not been
> written to disk before a crash, we'd set XLP_FIRST_IS_ABORTED_PARTIAL at the
> end of the valid data in B. Which could not have been archived yet, or we'd
> not have a partial record.  So we should never need to set the flag on an
> already archived page.

I agree that that makes sense.

Is that that, crash recovery remembers if the last record was an
immature record that looks like continue to the next segment, and if
so, set the flag when inserting the first record, which would be
CHECKPOINT_SHUTDOWN? (and reader deals with it)

I'll try to show how it looks like.

regards.

-- 
Kyotaro Horiguchi
NTT Open Source Software Center



pgsql-hackers by date:

Previous
From: Bharath Rupireddy
Date:
Subject: Re: improve pg_receivewal code
Next
From: Andres Freund
Date:
Subject: Re: pgstat_send_connstats() introduces unnecessary timestamp and UDP overhead