Re: Detecting File Damage & Inconsistencies - Mailing list pgsql-hackers

From Simon Riggs
Subject Re: Detecting File Damage & Inconsistencies
Date
Msg-id CANbhV-FjLYjfLDRCHqy6KBTEU2hvL4HKruj4SbzONQ1x9WXmsw@mail.gmail.com
Whole thread Raw
In response to Re: Detecting File Damage & Inconsistencies  (Craig Ringer <craig.ringer@enterprisedb.com>)
Responses Re: Detecting File Damage & Inconsistencies
List pgsql-hackers
On Thu, Mar 18, 2021 at 6:20 AM Craig Ringer
<craig.ringer@enterprisedb.com> wrote:
>
> On Mon, 15 Mar 2021 at 21:01, David Steele <david@pgmasters.net> wrote:
>>
>> On 11/18/20 5:23 AM, Simon Riggs wrote:
>> > On Wed, 18 Nov 2020 at 06:42, Craig Ringer
>> > <craig.ringer@enterprisedb.com> wrote:
>> >>
>> >> On Fri, Nov 13, 2020 at 7:24 PM Simon Riggs <simon@2ndquadrant.com> wrote:
>> >>>
>> >>>
>> >>> What I'm proposing is an option to add 16 bytes onto each COMMIT
>> >>> record
>> >>
>> >>
>> >> Would it make sense to write this at the time we write a topxid assignment to WAL instead?
>> >>
>> >> Otherwise it won't be accessible to streaming-mode logical decoding.
>> >
>> > Do you mean extend the xl_xact_assignment record? My understanding is
>> > that is not sent in all cases, so not sure what you mean by "instead".
>>
>> Craig, can you clarify?
>
>
> Right. Or write a separate WAL record when the feature is enabled. But it's probably sufficient to write it as an
optionalchunk on xl_xact_assignment records. We often defer writing them so we can optimise away xacts that never
actuallywrote anything, but IIRC we still write one before we write any WAL that references the xid. That'd be fine,
sincewe don't need the info any sooner than that during decoding. I'd have to double check that we write it in all
casesand won't get to that too soon, but I'm pretty sure we do... 

The commit record is optimized away if no xid is assigned, though is
still present if we didn't write any WAL records.

But if a commit record exists in the WAL stream, we want to know where
it came from.

A later patch will add PITR capability based on this information so
attaching it directly to the commit record is fairly important, IMHO.

--
Simon Riggs                http://www.EnterpriseDB.com/



pgsql-hackers by date:

Previous
From: Daniel Gustafsson
Date:
Subject: Re: Add version macro to libpq-fe.h
Next
From: Peter Geoghegan
Date:
Subject: Re: disfavoring unparameterized nested loops