Thread: tables > 1 gig
Config.h has this. Does this need to be updated because we can't vacuum multi-segment relations? I have changed it to 7F000000: /** RELSEG_SIZE is the maximum number of blocks allowed in one disk file.* Thus, the maximum size of a single file is RELSEG_SIZE* BLCKSZ;* relations bigger than that are divided into multiple files.** CAUTION: RELSEG_SIZE * BLCKSZ must beless than your OS' limit on file* size. This is typically 2Gb or 4Gb in a 32-bit operating system. By* default, we makethe limit 1Gb to avoid any possible integer-overflow* problems within the OS. A limit smaller than necessary only meanswe* divide a large relation into more chunks than necessary, so it seems* best to err in the direction of a small limit. (Besides, a power-of-2* value saves a few cycles in md.c.)** CAUTION: you had best do an initdb if you change eitherBLCKSZ or* RELSEG_SIZE.*/ #define RELSEG_SIZE (0x40000000 / BLCKSZ) -- Bruce Momjian | http://www.op.net/~candle maillist@candle.pha.pa.us | (610) 853-3000+ If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup. | Drexel Hill, Pennsylvania19026
Bruce Momjian <maillist@candle.pha.pa.us> writes:
> Config.h has this. Does this need to be updated because we can't vacuum
> multi-segment relations? I have changed it to 7F000000:
Why? I thought we'd fixed the mdtruncate issue.
regards, tom lane
> Bruce Momjian <maillist@candle.pha.pa.us> writes: > > Config.h has this. Does this need to be updated because we can't vacuum > > multi-segment relations? I have changed it to 7F000000: > > Why? I thought we'd fixed the mdtruncate issue. > > regards, tom lane > I am told we did not by Hiroshi. It was news to me too. -- Bruce Momjian | http://www.op.net/~candle maillist@candle.pha.pa.us | (610) 853-3000+ If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup. | Drexel Hill, Pennsylvania19026
Bruce Momjian <maillist@candle.pha.pa.us> writes:
>>>> Config.h has this. Does this need to be updated because we can't vacuum
>>>> multi-segment relations? I have changed it to 7F000000:
>>
>> Why? I thought we'd fixed the mdtruncate issue.
> I am told we did not by Hiroshi. It was news to me too.
Then we'd better fix the underlying problem. We can't change
RELSEG_SIZE for a minor release, unless you want to give up the
principle of not forcing initdb at minor releases.
regards, tom lane
> Bruce Momjian <maillist@candle.pha.pa.us> writes: > >>>> Config.h has this. Does this need to be updated because we can't vacuum > >>>> multi-segment relations? I have changed it to 7F000000: > >> > >> Why? I thought we'd fixed the mdtruncate issue. > > > I am told we did not by Hiroshi. It was news to me too. > > Then we'd better fix the underlying problem. We can't change > RELSEG_SIZE for a minor release, unless you want to give up the > principle of not forcing initdb at minor releases. Why can't we increase it? -- Bruce Momjian | http://www.op.net/~candle maillist@candle.pha.pa.us | (610) 853-3000+ If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup. | Drexel Hill, Pennsylvania19026
Bruce Momjian <maillist@candle.pha.pa.us> writes:
>> Then we'd better fix the underlying problem. We can't change
>> RELSEG_SIZE for a minor release, unless you want to give up the
>> principle of not forcing initdb at minor releases.
> Why can't we increase it?
Consider a 1.5-gig table. 6.5 will store it as one gig in file "table",
one-half gig in file "table.1". Now recompile with larger RELSEG_SIZE.
The file manager will now expect to find all blocks of the relation in
file "table", and will never go to "table.1" at all. Presto, you lost
a bunch of data.
Bottom line is just as it says in the config.h comments: you can't
change either BLCKSZ or RELSEG_SIZE without doing initdb.
regards, tom lane
> Bruce Momjian <maillist@candle.pha.pa.us> writes: > >>>> Config.h has this. Does this need to be updated because we can't vacuum > >>>> multi-segment relations? I have changed it to 7F000000: > >> > >> Why? I thought we'd fixed the mdtruncate issue. > > > I am told we did not by Hiroshi. It was news to me too. > > Then we'd better fix the underlying problem. We can't change > RELSEG_SIZE for a minor release, unless you want to give up the > principle of not forcing initdb at minor releases. No initdb for minor releases! -- Bruce Momjian | http://www.op.net/~candle maillist@candle.pha.pa.us | (610) 853-3000+ If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup. | Drexel Hill, Pennsylvania19026
> Bruce Momjian <maillist@candle.pha.pa.us> writes:
> >> Then we'd better fix the underlying problem. We can't change
> >> RELSEG_SIZE for a minor release, unless you want to give up the
> >> principle of not forcing initdb at minor releases.
> > Why can't we increase it?
> Consider a 1.5-gig table. 6.5 will store it as one gig in file "table",
> one-half gig in file "table.1". Now recompile with larger RELSEG_SIZE.
> The file manager will now expect to find all blocks of the relation in
> file "table", and will never go to "table.1" at all. Presto, you lost
> a bunch of data.
> Bottom line is just as it says in the config.h comments: you can't
> change either BLCKSZ or RELSEG_SIZE without doing initdb.
Sorry for backing up so far on this thread...
Would it be possible to make BLCKSZ and/or RELSEG_SIZE (the latter
perhaps the most important, and perhaps the easiest?) a configurable
parameter which is read out of a global variable for each database? If
so, we could later think about moving it, along with things like
"default character set", to pg_database as per-db information, and
make it an option on CREATE DATABASE. That kind of thing might make
in-place upgrades easier too.
- Thomas
--
Thomas Lockhart lockhart@alumni.caltech.edu
South Pasadena, California
Thomas Lockhart <lockhart@alumni.caltech.edu> writes:
> Would it be possible to make BLCKSZ and/or RELSEG_SIZE (the latter
> perhaps the most important, and perhaps the easiest?) a configurable
> parameter which is read out of a global variable for each database?
Doable, perhaps, but I'm not quite sure why it's worth the trouble...
there doesn't seem to be that much value in running different DBs with
different values inside a single installation. Tweaking BLCKSZ, in
particular, will become fairly uninteresting once we solve the tuples-
bigger-than-a-block problem.
regards, tom lane