Re: Hardware estimation - Mailing list pgsql-general

On Fri, 8 Nov 2002, Vidal Salem Zebulum wrote:

>
> ----- Original Message -----
> From: "scott.marlowe" <scott.marlowe@ihs.com>
> To: "Vidal" <vzebulum@yahoo.com.br>
> Cc: <pgsql-general@postgresql.org>
> Sent: Thursday, November 07, 2002 3:08 PM
> Subject: Re: [GENERAL] Hardware estimation
>
>
> > On 5 Nov 2002, Vidal wrote:
> >
> > > Hi,
> > >
> > > I am a first timer in this forum. We have  an application  using
> > > Postgre Database with a considerable growth expected for the next
> > > year. We are facing some dificulties to estimate the equipment we´ll
> > > need  and I would like to exchange ideas about:
> > >
> > > - hardware estimation
> > > - performance concerns
> > >
> > >  with anyone that faced Postgre applications with more than 150Gb.
> > >
> > > I am talking about a database that will achieve:
> > >
> > > - 200Gb total space
> > > - 2 major space cosuming tables (400 million and 50 million tuples)
> > > - major system process - batch process reading from and writing to
> > > text files (telephony records) - aprox 2 million records processed
> > > (read or written every day)
> > >
> > > Operating System : Linux
> > >
> > > Equipment we are planning:
> > > - IBM xSeries 255 4 Intel Xeon 1.5GHz, MP, 512KB Cache, 2GB RAM,
> > > External Storage System(IBM SSA)
> >
> > Are you going to be supporting a fair number of simo users, or is this
> > gonna be mostly a single batch at a time operation?
> >
> > If you're mostly gonna be running single user batch files for all the
> > heavy lifting, save the money you'd spend on a quad Xeon and spend it on
> > more / bigger / faster drive arrays and faster individual CPUs, like a
> > dual Athlon 2800.
> >
> > But maybe the multi-user part is still very important to your uses.  If
> > so...
> >
> > If you are looking at a quad Xeon, then take a look at going all the way
> > to 64 bit architecture (Ultra Sparc, HP, P6xx IBM, Itanium 2) where you
> > can throw scads of memory at your problems and postgresql can access it.
> >
> > You might find a dual Ultra Sparc will outrun the quad xeon due to better
> > memory access, larger caches (8Meg L2 cache), and being able to put
> > ungodly amounts of memory into the machine.
> >
> > Some of the "low end" 64 bit machines are not any more expensive than a
> > quad xeon can run.
> >
> > A quad 1GHz Power4 with 8 gig ram, dual 36 Gig drives and all the fixings
> > goes for $44,000.  That's with 64 Meg L3 cache.
> >
> >
> Mr Scott,
>
>
> Thanks for the prompt response.
>
> Our processes will be mostly batch ones. On line activity will be restricted
> to a few users. And for some on-line process we will use consolidated data.
> But all batch processes will have to use the tables with 50M  and 500M
> records.
>
> We will strongly consider your option for external storage. We would rather
> relay on a Risc arquitecture due to the I/O effort of our batch processes.
> Our IBM rep then raised a problem: there was no linux driver in
> thePower3(P610) for the external storage solution we wanted. Then they
> mentioned the possibility to use the quad Xeon solution. We will still push
> them and HP for a Risc solution with external storafe. Your statement
> enforce this choice. Thanks for that.
>
> Do you think we should have problems with Postgre at that  database size
> assuming we will take double care with tunning ? Do you know anyone with
> similar volume ?

There are LOTS of postgresql installations out there with gigabytes of
data.  Take a search through the pgsql-general and pgsql-hackers list for
"large database" or "gigabyte" and you should find a few folks who are
running hundreds of gigs of data on postgresql.