Re: xlog location arithmetic - Mailing list pgsql-hackers

From Tom Lane
Subject Re: xlog location arithmetic
Date
Msg-id 4618.1331323463@sss.pgh.pa.us
Whole thread Raw
In response to Re: xlog location arithmetic  (Robert Haas <robertmhaas@gmail.com>)
Responses Re: xlog location arithmetic  (Robert Haas <robertmhaas@gmail.com>)
Re: xlog location arithmetic  (Bruce Momjian <bruce@momjian.us>)
Re: xlog location arithmetic  (Fujii Masao <masao.fujii@gmail.com>)
List pgsql-hackers
Robert Haas <robertmhaas@gmail.com> writes:
> On Fri, Mar 9, 2012 at 2:34 PM, Robert Haas <robertmhaas@gmail.com> wrote:
>> On Fri, Mar 9, 2012 at 2:26 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>>> Hm. �I think "thousands" is an overestimate, but yeah the logic could be
>>> greatly simplified. �However, I'm not sure we could avoid breaking the
>>> existing naming convention for WAL files. �How much do we care about
>>> that?

>> Probably not very much, since WAL files aren't portable across major
>> versions anyway. �But I don't see why you couldn't keep the naming
>> convention - there's nothing to prevent you from converting a 64-bit
>> integer back into two 32-bit integers if and where needed.

> On further reflection, this seems likely to break quite a few
> third-party tools.  Maybe it'd be worth it anyway, but it definitely
> seems like it would be worth going to at least some minor trouble to
> avoid it.

The main actual simplification would be in getting rid of the "hole"
at the end of each 4GB worth of WAL, cf this bit in xlog_internal.h:

/** We break each logical log file (xlogid value) into segment files of the* size indicated by XLOG_SEG_SIZE.  One
possiblesegment at the end of each* log file is wasted, to ensure that we don't have problems representing*
last-byte-position-plus-1.*/
#define XLogSegSize        ((uint32) XLOG_SEG_SIZE)
#define XLogSegsPerFile (((uint32) 0xffffffff) / XLogSegSize)
#define XLogFileSize    (XLogSegsPerFile * XLogSegSize)

If we can't get rid of that and have a continuous 64-bit WAL address
space then it's unlikely we can actually simplify any logic.

Now, doing that doesn't break the naming convention exactly; what it
changes is that there will be WAL files numbered xxxFFFF (for some
number of trailing-1-bits I'm too lazy to work out at the moment) where
before there were not.  So the question really is how much external code
there is that is aware of that specific noncontiguous numbering behavior
and would be broken if things stopped being that way.
        regards, tom lane


pgsql-hackers by date:

Previous
From: Simon Riggs
Date:
Subject: Re: Avoiding shutdown checkpoint at failover
Next
From: Robert Haas
Date:
Subject: Re: Is it time for triage on the open patches?