Re: Hardware selection - Mailing list pgsql-general

On Mon, 2003-07-28 at 11:28, psql-mail@freeuk.com wrote:
> Ron thank you for your comments, sorry of the slow response - i
> actually replied to you on saturday but i think the list was having
> trouble again?!
>
> Your questions are answered below...
>
> > On Fri, 2003-07-25 at 07:42, psql-mail@freeuk.com wrote:
> > > As mentioned previously I have a large text database with upwards
> of
> >
> > > 40GB of data and 8 million tuples.
> > >
> > > The time has come to buy some real hardware for it.
> > >
> > > Having read around the subject online I see the general idea is to
>
> > get
> > > as much memory and the fastest I/O possible.
> > >
> > > The buget for the server is approx 20K GBP / 30K US$
> > >
> > > The data will be stored on an external raid,
> > > SCSI based 2.5TB with IDE disks. Configured as 1 large volume,
> RAID5.
> > (
> > > We already have this hardware)
> >
> > There's a SCSI-IDE bridge somewhere in there?
>
> The raid hardware takes care of it, IDE disks slot into the backplane,
> the RAID then SCSI attached to the server.
>
> If this hardware is percieved to be the bottleneck - and i think it
> might be, then whats the best thing to replace it with? (a few
> different specs/prices would be very helpful!)
>
> > 40GB of data, and 2.5TB disk space??  Even with RAID1+0, that's
> > 1.25TB usable; 40GB (even after accounting for overhead and indexes)
> > would get lost in 1250GB.  Planning for growth, or also storing lots
> > of non-DB related files (which is not a good idea)?
>
> Data is currently 40GB, maybe a bit more, it will increase steadily in
> the future. There will be no non-database files on there.
>
> I am also planning to do a full text index on the data so that will
> take up alot of room.

Besides, if you can get the disk space, take it!  Will these be
250GB drives?

What kind of DB backup scheme do you have planned?

> > > Question 1 - whats the best raid configuration for this device?
> >
> > What's your read-to-write ratio?
>
> Faily small bulk INSERTS every 30 mins.
> Continous querying within office hours.
>
> > Few complex multi-table queries, lots of simple queries, or
> > a mixture?
>
> Mostley queries of the form 'ILIKE "%chicken%"' across >= 30GB of data.
> This may change if full text indexing is sucessful but will still be
> required for phrases 'ILIKE "%chicken and chips%"
>
> There will also be lots of smaller queries going on  constantly, but
> these are already quick enough on the old hardware.
>
> > > Onto the server, I have currently priced up the following:
> > > Dell Poweredge 6650
> > > 16GB RAM
> > > Quad intel hyperthreaded arch
> > > PERC3Q controller
> > > 36GB System HDD
> > > approx 20K UKP
> >
> > Because of the limits of the x86 architecture, with 16GB RAM, I'd
> > really think about a 64 bit system.  Opteron, Alpha, Power5,
> > Sparc64, etc.
> >
> > You can put 16GB RAM in x86 systems, but 64 bit systems use it
> > more efficiently.
>
> I was under the impression that both the O/S and the application (
> postmaster) needed to be optimised   to make best use of 64 bit arch.
> Is this impression wrong? If not, is postgresql optimised for 64bit
> arch?

PG mostly uses the OS' cache, so if it's a 64 bit OS, and PG is
compiled for the target, things will be fine.

SuSE has an Opteron distro:
http://www.suse.co.uk/uk/business/products/server/sles/index.html
http://www.suse.co.uk/en/business/products/server/sles/misc/sles8_amd64.pdf

> > > Question 2 - Comments and opinions on this as the postgresql server.
>
> >
> > > Any comments on better alternatives or modification to this system
>
> > > would be very helpful!
> > >
> > > Question 3 - what would be the best way to distribute tables/swap/
> > > system disk space?
> >
> > 16GB swap would fit nicely on the system disk.  Hope it's on a
> > separate channel from the DB...
>
> Will put the swap on the local disk and the data on the external RAID -
> different channels.
>
> > What OS are you using?
> I will be using Red hat advanced server 2.1 - thats not set in stone
> yet but RH is the distro that I am most familiar with.
>
> Thanks for you help so far!
>
> PS. Just in case theres an amazing system for 60K or less point it out
> in case i can get more funding!
--
+-----------------------------------------------------------------+
| Ron Johnson, Jr.        Home: ron.l.johnson@cox.net             |
| Jefferson, LA  USA                                              |
|                                                                 |
| "I'm not a vegetarian because I love animals, I'm a vegetarian  |
|  because I hate vegetables!"                                    |
|    unknown                                                      |
+-----------------------------------------------------------------+