Re: [HACKERS] Problems with >2GB tables on Linux 2.0 - Mailing list pgsql-hackers

From Tom Lane
Subject Re: [HACKERS] Problems with >2GB tables on Linux 2.0
Date
Msg-id 5106.918410601@sss.pgh.pa.us
Whole thread Raw
In response to Re: [HACKERS] Problems with >2GB tables on Linux 2.0  (Peter T Mount <peter@retep.org.uk>)
Responses RE: [HACKERS] Problems with >2GB tables on Linux 2.0  ("Stupor Genius" <stuporg@erols.com>)
Re: [HACKERS] Problems with >2GB tables on Linux 2.0  (Peter T Mount <peter@retep.org.uk>)
Re: [HACKERS] Problems with >2GB tables on Linux 2.0  ("D'Arcy" "J.M." Cain <darcy@druid.net>)
List pgsql-hackers
Peter T Mount <peter@retep.org.uk> writes:
> As expected, the splitting of the file works fine. So the code isn't
> broken. What I think is happening is that the code extends the table, then
> tests to see if it's at the 2Gig limit, and when it is, creates the next
> file for that table.

> However, I think the OS has problems with a file exactly 2Gb in size.

Oh!  I didn't realize that we were trying to extend the file to
*exactly* 2Gb.  Indeed that's a very dangerous thing to do: the file
size in bytes will be 0x80000000, which will appear to be negative when
viewed as a signed 32-bit integer; unless your OS is very careful about
signed vs. unsigned arithmetic, it will break.

For that matter it's not impossible that our own code contains similar
problems, if it does much calculating with byte offsets into the file.
(The pushups that darrenk had to do in order to calculate RELSEG_SIZE
in the first place should have suggested to him that running right at
the overflow limit was not such a hot idea...)

I'd suggest setting the limit a good deal less than 2Gb to avoid any
risk of arithmetic overflow.  Maybe 200000 8K blocks, instead of 262144.

And change the comment while you're at it, not just the code ;-)
        regards, tom lane


pgsql-hackers by date:

Previous
From: Tom Lane
Date:
Subject: Re: [HACKERS] equal: don't know whether nodes of type 600 are equal
Next
From: Tom Lane
Date:
Subject: Re: [HACKERS] v6.4.3 ?