Re: Hot Backup with rsync fails at pg_clog if under load - Mailing list pgsql-hackers

From Florian Pflug
Subject Re: Hot Backup with rsync fails at pg_clog if under load
Date
Msg-id 1A94CE1A-C3E5-4923-B677-623D41BC273F@phlo.org
Whole thread Raw
In response to Re: Hot Backup with rsync fails at pg_clog if under load  (Simon Riggs <simon@2ndQuadrant.com>)
Responses Re: Hot Backup with rsync fails at pg_clog if under load
Re: Hot Backup with rsync fails at pg_clog if under load
List pgsql-hackers
On Oct25, 2011, at 11:13 , Simon Riggs wrote:
> On Tue, Oct 25, 2011 at 8:03 AM, Simon Riggs <simon@2ndquadrant.com> wrote:
>> We are starting recovery at the right place but we are initialising
>> the clog and subtrans incorrectly. Precisely, the oldestActiveXid is
>> being derived later than it should be, which can cause problems if
>> this then means that whole pages are unitialised in subtrans. The bug
>> only shows up if you do enough transactions (2048 is always enough) to
>> move to the next subtrans page between the redo pointer and the
>> checkpoint record while at the same time we do not have a long running
>> transaction that spans those two points. That's just enough to happen
>> reasonably frequently on busy systems and yet just enough to have
>> slipped through testing.
>>
>> We must either
>>
>> 1. During CreateCheckpoint() we should derive oldestActiveXid before
>> we derive the redo location

> (1) looks the best way forwards in all cases.

Let me see if I understand this

The probem seems to be that we currently derive oldestActiveXid end the end of
the checkpoint, just before writing the checkpoint record. Since we use
oldestActiveXid to initialize SUBTRANS, this is wrong. Records written before
that checkpoint record (but after the REDO location, of course) may very well
contain XIDs earlier than that wrongly derived oldestActiveXID, and if attempt
to touch these XID's SUBTRANS state, we error out.

Your patch seems sensible, because the checkpoint "logically" occurs at the
REDO location not the checkpoint's location, so we ought to log an oldestActiveXID
corresponding to that location.

What I don't understand is how this affects the CLOG. How does oldestActiveXID
factor into CLOG initialization?

best regards,
Florian Pflug



pgsql-hackers by date:

Previous
From: Magnus Hagander
Date:
Subject: Re: Updated version of pg_receivexlog
Next
From: Magnus Hagander
Date:
Subject: Re: Idea: Always consistent in-database cache using SSI mechanisms