Re: Maximum table size - Mailing list pgsql-hackers

From Bruce Momjian
Subject Re: Maximum table size
Date
Msg-id 200309091825.h89IPJX09778@candle.pha.pa.us
Whole thread Raw
In response to Re: Maximum table size  (Tatsuo Ishii <t-ishii@sra.co.jp>)
Responses Re: Maximum table size
Re: Maximum table size
List pgsql-hackers
Tatsuo Ishii wrote:
> > Tom Lane wrote:
> > > Bruce Momjian <pgman@candle.pha.pa.us> writes:
> > > > Is our maximum table size limited by the maximum block number?
> > > 
> > > Certainly.
> > > 
> > > > Is the 16TB number a hold-over from when we weren't sure block number
> > > > was unsigned, though now we are pretty sure it is handled as unsigned
> > > > consistenly?
> > > 
> > > It's a holdover.  As to how certain we are that all the
> > > signed-vs-unsigned bugs are fixed, who have you heard from running a
> > > greater-than-16Tb table?  And how often have they done CLUSTER, REINDEX,
> > > or even VACUUM FULL on it?  AFAIK we have zero field experience to
> > > justify promising that it works.
> > > 
> > > We can surely fix any such bugs that get reported, but we haven't got
> > > any infrastructure that would find or prevent 'em.
> > 
> > I guess the big question is what do we report as the maximum table size?
> > Do we report 32TB and fix any bug that happen over 16TB?
> 
> That seems right direction for me. I see no reason why 16TB is more
> reliable number than 32TB, since nobody has ever tried to build 16TB
> tables.

Agreed. I think the question is how large does the design support,
rather than how large have we tested.  (In fact, the check for using
block numbers as unsigned was removed from the FAQ when I reviewed the
code.)

I know Tom is concerned because we haven't tested it, but I don't think
anyone has tested 16TB either, nor our 1600-column limit.

Also, I think people look at these numbers to determine if PostgreSQL
can handle their data needs 5-10 years down the road.

In fact, if you increase the page size, you can quadruple most of the
existing limits.  This is already mentioned in the FAQ:
  <P>The maximum table size and maximum number of columns can  be increased if the default block size is increased to
32k.</P>

I have updated the FAQ to say 32TB, and of course, larger page sizes
could make this 128TB.

--  Bruce Momjian                        |  http://candle.pha.pa.us pgman@candle.pha.pa.us               |  (610)
359-1001+  If your life is a hard drive,     |  13 Roberts Road +  Christ can be your backup.        |  Newtown Square,
Pennsylvania19073
 


pgsql-hackers by date:

Previous
From: Kurt Roeckx
Date:
Subject: Re: Stats Collector Error 7.4beta1 and 7.4beta2
Next
From: "Jeroen T. Vermeulen"
Date:
Subject: Re: libpq++