Re: Changing WAL Header to reduce contention during ReserveXLogInsertLocation() - Mailing list pgsql-hackers

From Simon Riggs
Subject Re: Changing WAL Header to reduce contention during ReserveXLogInsertLocation()
Date
Msg-id CANP8+jKJecg5hfiDfw4vExSFAuCZWXhai_2A=BXsN6iu4e26uQ@mail.gmail.com
Whole thread Raw
In response to Re: Changing WAL Header to reduce contention duringReserveXLogInsertLocation()  (Tomas Vondra <tomas.vondra@2ndquadrant.com>)
List pgsql-hackers
On 29 March 2018 at 18:13, Tomas Vondra <tomas.vondra@2ndquadrant.com> wrote:
> On 03/29/2018 06:42 PM, Tom Lane wrote:
>> Simon Riggs <simon@2ndquadrant.com> writes:
>>> I know the approach is new and surprising but I thought about it a lot
>>> before proposing it and I couldn't see any holes; still can't. Please
>>> give this some thought so we can get comfortable with this idea and
>>> increase performance as a result. Thanks.
>>
>> The long and the short of it is that this is a very dangerous-looking
>> proposal, we are at the tail end of a development cycle, and there are
>> ~100 other patches remaining in the commitfest that also have claims
>> on our attention in the short time that's left.  If you're expecting
>> people to spend more time thinking about this now, I feel you're being
>> unreasonable.
>>
>
> I agree.

We require consensus on such things for good reason. Risk is an issue
and everyone must be comfortable.

If its a good idea now it will still be a good idea in the next cycle.


>> Also, I will say it once more: this change DOES decrease robustness.
>> It's like blockchain without the chain aspect, or git commits without
>> a parent pointer.  We are not only interested in whether individual
>> WAL records are valid, but whether they form a consistent series.
>> Cross-checking xl_prev provides some measure of confidence about that;
>> xl_curr offers none.
>>
>
> Not sure.
>
> If each WAL record has xl_curr, then we know to which position the
> record belongs (after verifying the checksum). And we do know the size
> of each WAL record, so we should be able to deduce if two records are
> immediately after each other. Which I think is enough to rebuild the
> chain of WAL records.
>
> To defeat this, this would need to happen:
>
> a) the WAL record gets written to a different location
> b) the xl_curr gets corrupted in sync with (a)
> c) the WAL checksum gets corrupted in sync with (b)
> d) the record overwrites existing record (same size/boundaries)
>
> That seems very much like xl_prev.

That's how I see it. But I also see that it feels different to what we
had before and that takes time to either come up with a concrete,
quantifiable reason not to, or accept that it is OK.


So given my observation that no technical objection remains, I'm happy
to move this to next CF to give us more time and less risk.

-- 
Simon Riggs                http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services


pgsql-hackers by date:

Previous
From: legrand legrand
Date:
Subject: Poc: pg_stat_statements with planid
Next
From: Tomas Vondra
Date:
Subject: Re: Changing WAL Header to reduce contention duringReserveXLogInsertLocation()