Thread: How to make a REALLY FAST db server?
I'm looking to make a DB server for a project I'm working on (using pgsql) and am wondering if people have suggestions? Thoughts: - Hardware: dual / quad Intel class - OS: Prolly FreeBSD (FFS is your friend (what with syncs and all) and it can do multi proc support - Disk: SCSI Raid 1+0 - Ram: Not really sure here. Is there math somewhere for ram needs for pgsql? I imagine is has something to do with # connections, db size, etc. Any people have any comments? - Brandon ---------------------------------------------------------------------------- b. palmer, bpalmer@crimelabs.net pgp:crimelabs.net/bpalmer.pgp5
On Mon, 10 Sep 2001, bpalmer wrote: > - Hardware: dual / quad Intel class Fairly easy to obtain. If all you want is a dual, you can use desktop-class motherboards from such makers as Asus, Abit, and IWill. If you're going for speed, stick to the DDR or SDRAM capable boards. > - Disk: SCSI Raid 1+0 To really eek out as much speed as possible here, you'll want 10k RPM Ultra-160 Fibre Channel SCSI drives with a dedicated hardware raid controller. If have more reads than writes, you may want to use Raid 5 instead. Postgres won't let you separate indexes from the database they represent, so you can't make separate raid clusters for indexes and data; no optimization there. Maybe in the next version that implements schemas? What you can do if you use multiple DB's in your app design, is put different DB's on different raid clusters. That'll help parallel execution times. If you do this, make sure template1 and template0 are separated from the rest of the databases, this will allow fast responses from the system tables and make sure no application database IO affects them adversely. > - Ram: Not really sure here. Is there math somewhere for ram needs for > pgsql? I imagine is has something to do with # connections, db size, > etc. No reason not to go 2GB. Ram is cheap these days, and you can always increase shared buffers and caches to actually fill the server memory up with as much quick-fetch info as possible. All in all, if you're making a DB machine, do whatever you can to get rid of hits caused by disk IO. Parallelize as much as possible between your databases, and if you have a DB capable of separating indexes from the mix, do that too. Don't run any other services on it, and make sure it has a nice wide 100MBit or 1GBit pipe so it doesn't saturate when servicing multiple hosts. Hope that helps. -- +-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-+ | Shaun M. Thomas INN Database Programmer | | Phone: (309) 743-0812 Fax : (309) 743-0830 | | Email: sthomas@townnews.com AIM : trifthen | | Web : hamster.lee.net | | | | "Most of our lives are about proving something, either to | | ourselves or to someone else." | | -- Anonymous | +-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-+
> I'm looking to make a DB server for a project I'm working on (using pgsql) > and am wondering if people have suggestions? > > Thoughts: > > - Hardware: dual / quad Intel class If you're going dual P3, go with Athlons, they have a *much* higher bandwidth across the front side and memory buses, not to mention being better processors to start with. If you're thinking a quad Xeon, you can still consider a dual Athlon. I do run a quad Xeon (SuperMicro 8050 chassis/motherboard), and it's fast, but there are tradeoffs. Dual 1.2 GHz Athlons = 2.4 GHz total, 4x700 MHz Xeons = 2.8 GHz total. Cost difference = ~$4,000 or more. Another determining factor is the usage that you expect. If you need extremely low latency on infrequent queries, the higher-frequency Athlons will likely be much better, as for a single query you're pitting a 700 MHz Xeon on a 100 MHz bus against a 1.2 MHz Athlon with a 266 MHz bus. However, for all-out, slam-the-machine work, the ServerWorks Xeon chips do shine, as they have 4-way interleaved memory and a crossbar to individual CPU buses. (I wasn't aware that they used the crossbar until recently.) If you do go with a quad Xeon, the SuperMicro 8050 is a *superb* platform to start with. Add RAM, CPU's, RAID card, drives, and video, and you're off to the races. Triple, redundant power supplies connected to an intelligent power distribution system, an unearthly number of fans (again, all in redundant push/pull pairs), 64/66 PCI, and 10 hot-swap, SCA drive bays across two channels. Tough to beat. It is a very large chassis (makes two mid-towers look puny), the rack-mountable 8060 looks like it should be equally sturdy and reliable. The only time ours ever gets rebooted is for hardware upgrades. > can do multi proc support > - Disk: SCSI Raid 1+0 > - Ram: Not really sure here. Is there math somewhere for ram needs for > pgsql? I imagine is has something to do with # connections, db size, > etc. While 1+0 is fast, you can realistically use RAID 5 for redundancy, saving disks, provided that you're not using fsync(), and have plenty of RAM. Between the write-cache on the RAID controller and from the kernel, even a large number of inserts/updates don't actually have to hit the disk at all. As for how much RAM to use, you want to have enough that your entire $PGDATA directory is held in disk cache, a good chunk for shared buffers, a good chunk for sorts, enough for the backends, and enough spare for write-caching that you won't hit the disk. Just how much that is depends on the size of your data, and what type of usage you're going to give the machine. steve
> I'm looking to make a DB server for a project I'm working on (using pgsql) > and am wondering if people have suggestions? > > Thoughts: > > - Hardware: dual / quad Intel class OK, but remember I/O is key for databases. The more spindles the better. See my performance article on techdocs. > - OS: Prolly FreeBSD (FFS is your friend (what with syncs and all) and it > can do multi proc support I would recommend soft updates be enabled. > - Disk: SCSI Raid 1+0 Not sure about that. Is that optimal for I/O? > - Ram: Not really sure here. Is there math somewhere for ram needs for > pgsql? I imagine is has something to do with # connections, db size, > etc. Again, see article. -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 853-3000 + If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup. | Drexel Hill, Pennsylvania 19026
>> - Hardware: dual / quad Intel class > >Fairly easy to obtain. If all you want is a dual, you can use >desktop-class motherboards from such makers as Asus, Abit, and >IWill. If you're going for speed, stick to the DDR or SDRAM >capable boards. I'm not sure how much a 2+ way system will help. Most of the db work will be done in one long serialized processes and those can't be spread out over multiple processors (with current day postgresql). >> - Disk: SCSI Raid 1+0 > >To really eek out as much speed as possible here, you'll want 10k RPM >Ultra-160 Fibre Channel SCSI drives with a dedicated hardware raid >controller. If have more reads than writes, you may want to use Raid 5 >instead. Why 5? 1+0 is far better and faster. I was planning on doing a hardware RAID controller (just need to find the one that FBSD likes the best). >Postgres won't let you separate indexes from the database they represent, >so you can't make separate raid clusters for indexes and data; no >optimization there. Maybe in the next version that implements >schemas? What you can do if you use multiple DB's in your app design, >is put different DB's on different raid clusters. That'll help parallel >execution times. If you do this, make sure template1 and template0 are >separated from the rest of the databases, this will allow fast responses >from the system tables and make sure no application database IO affects >them adversely. Interesting thoughts.. >No reason not to go 2GB. Ram is cheap these days, and you can always >increase shared buffers and caches to actually fill the server memory >up with as much quick-fetch info as possible. But then why not 4G? I would love some real numbers rather than 'a lot'. With oracle, you can plug in some numbers and a real extimate will be spit out. I've worked with DB servers w/ 14G of ram that were killing that, so "get a lot" isn't really good enough. >All in all, if you're making a DB machine, do whatever you can to get >rid of hits caused by disk IO. Parallelize as much as possible between >your databases, and if you have a DB capable of separating indexes from Regretfully, it's only one database and the data model won't really let me split it up much. >the mix, do that too. Don't run any other services on it, and make >sure it has a nice wide 100MBit or 1GBit pipe so it doesn't saturate when >servicing multiple hosts. - Brandon ---------------------------------------------------------------------------- b. palmer, bpalmer@crimelabs.net pgp:crimelabs.net/bpalmer.pgp5
>> - Hardware: dual / quad Intel class > >OK, but remember I/O is key for databases. The more spindles the >better. See my performance article on techdocs. >> - OS: Prolly FreeBSD (FFS is your friend (what with syncs and all) and it >> can do multi proc support >I would recommend soft updates be enabled. Good points. >> - Disk: SCSI Raid 1+0 > >Not sure about that. Is that optimal for I/O? From my experience it is. As long as you have a raid controler that can do 2 level RAID abstraction. First you need mirrored pairs and then you stripe over them. It costs a lot in disk, but is stupid fast with the right raid controller. With some Suns / FC / EMC, we were getting ~100M/s+ with that setup for our Oracle server. >> - Ram: Not really sure here. Is there math somewhere for ram needs for >> pgsql? I imagine is has something to do with # connections, db size, >> etc. > >Again, see article. Thanks. ---------------------------------------------------------------------------- b. palmer, bpalmer@crimelabs.net pgp:crimelabs.net/bpalmer.pgp5
I am currently working on a patch to 7.2 that will allow data/indexes to be in different locations. I am also looking at replacing the current LOCATION code with a table driven (pg_locations) that will allow not only the default data/indexes locations to be set but also each table/index to have its own location. The first part is finished. I will be making a patch tomorrow after some more testing. On my app (about 400G of table data and 350G of index data) it really makes a difference... Jim > >> - Hardware: dual / quad Intel class > > > >OK, but remember I/O is key for databases. The more spindles the > >better. See my performance article on techdocs. > > >> - OS: Prolly FreeBSD (FFS is your friend (what with syncs and all) and it > >> can do multi proc support > > >I would recommend soft updates be enabled. > > Good points. > > >> - Disk: SCSI Raid 1+0 > > > >Not sure about that. Is that optimal for I/O? > > >From my experience it is. As long as you have a raid controler that > can do 2 level RAID abstraction. First you need mirrored pairs and > then you stripe over them. It costs a lot in disk, but is stupid fast > with the right raid controller. With some Suns / FC / EMC, we were > getting ~100M/s+ with that setup for our Oracle server. > > >> - Ram: Not really sure here. Is there math somewhere for ram needs for > >> pgsql? I imagine is has something to do with # connections, db size, > >> etc. > > > >Again, see article. > > Thanks. > > > ---------------------------------------------------------------------------- > b. palmer, bpalmer@crimelabs.net pgp:crimelabs.net/bpalmer.pgp5 > > > > ---------------------------(end of broadcast)--------------------------- > TIP 1: subscribe and unsubscribe commands go to majordomo@postgresql.org > >
bpalmer <bpalmer@crimelabs.net> writes: > I'm looking to make a DB server for a project I'm working on (using pgsql) > and am wondering if people have suggestions? > > Thoughts: > > - Hardware: dual / quad Intel class Athlon gives more bang for the buck - the dual Athlons are _really_ nice, and have proven stable as well. > - Disk: SCSI Raid 1+0 I'd probably go for a 3ware RAID instead... IDE disks are so much cheaper nowadays than SCSI, and the premium isn't really justifiable anymore. > - OS: Prolly FreeBSD (FFS is your friend (what with syncs and all) and it > can do multi proc support I'd recommend Linux, which has more mature MP support and scales better, but I'm obviously biased :). It's of course very important to keep what you are familiar with - a good sysadmin makes a world of difference no matter what you're using. > - Ram: Not really sure here. Is there math somewhere for ram needs for > pgsql? I imagine is has something to do with # connections, db size, > etc. "More is better. RAM is cheap. Avoid RAMBUS". -- Trond Eivind Glomsrød Red Hat, Inc.
> > - Hardware: dual / quad Intel class The ultimate would be an IBM S/390 mainframe running some distribution of Linux S/390. I/O bandwidth on mainframe DASD is incredible, the memory is robust and fast, and the CPU is trememdous. The price is also trememdous. -- Lamar Owen WGCR Internet Radio 1 Peter 4:11
> I'd probably go for a 3ware RAID instead... IDE disks are so much > cheaper nowadays than SCSI, and the premium isn't really justifiable > anymore. Having used IDE and SCSI disks, when I'm serious about performance, IDE doesn't even enter my mind. Also, over on the XFS list, there are a few people that have been using 3ware cards, and it sounds like there are still some serious caveats/bugs to them. Myself, I much prefer a good SCSI RAID card that's going to work, and going to work well. (As an aside, one person was in a heated argument about how much cheaper IDE was than SCSI. I got on pricewatch, found some prices, and would have been able to put together a very fast SCSI system for the same price as his IDE array.) steve
> I'm not sure how much a 2+ way system will help. Most of the db work > will be done in one long serialized processes and those can't be spread > out over multiple processors (with current day postgresql). That's assuming that only one query will ever be executed at once. As a new backend is spawned for each connection, extra CPU's are very helpful if the database will see more than occasional use. Also, even if there's only one query, dual-CPU machines are generally much more responsive, especially under load, as one CPU can be handling interrupts, kernel code, and other system processes while the other sits there doing your task. > >To really eek out as much speed as possible here, you'll want 10k RPM > >Ultra-160 Fibre Channel SCSI drives with a dedicated hardware raid > >controller. If have more reads than writes, you may want to use Raid 5 > >instead. > > Why 5? 1+0 is far better and faster. I was planning on doing a > hardware RAID controller (just need to find the one that FBSD likes the > best). If you have enough RAM, disk speed isn't terribly important, so RAID 5 gives you the redundancy without as many disks. Throw in an extra gig of RAM for your disk cache, turn of fsync(), and you're likely to see a lot bigger speed-up than any disk upgrade will give you. There are cases where that isn't the case (such as updating every row in a multi-gigabyte table), but unless you're involved in those specialized cases, it's not as important. So, why did I say that I don't use IDE for high-performance machines? IDE has limitations. For example, say I wanted 10 drives in my array. Finding a 5-channel IDE RAID controller is probably not as easy (and not as cheap) as a dual-channel SCSI RAID controller. Also, SCSI buses are much better about sharing bandwidth than IDE, as IDE doesn't have some of the "nifty" features that SCSI does. And to go one further, hot-swappable SCA bays are pretty common in server chassis. I simply plugged the RAID controller into the SCA backplanes, and was done. Had I gone IDE, there would have been additional cost in obtaining the hot-swap IDE bays. As an aside, if you do go with a 3ware card, you might NOT want to use RAID 5. The processors on the card are not up to the computing demands of RAID 5, you might want to take a look at: http://marc.theaimsgroup.com/?l=linux-xfs&m=99970690219042&w=2 > >No reason not to go 2GB. Ram is cheap these days, and you can always > >increase shared buffers and caches to actually fill the server memory > >up with as much quick-fetch info as possible. > > But then why not 4G? I would love some real numbers rather than 'a > lot'. With oracle, you can plug in some numbers and a real extimate > will be spit out. I've worked with DB servers w/ 14G of ram that were > killing that, so "get a lot" isn't really good enough. We run 1.5 gigs, and that's plenty for us. I increased the shared buffers until it didn't help any more, then doubled it, I believe that it came out to around 128 gigs. I did the same with sort memory, that came out to around 64 megs. The machine right now uses about 860 megs of disk cache, but took a few months to ge that high. It hasn't used swap at all. If it ever hits swap, we'll add more. Luckily, with the 4-way interleaved memory, it'll take up to 16 gigs, and with 16 slots, there's a lot of room to add more. : ) steve
On Mon, 10 Sep 2001, Steve Wolfe wrote: > So, why did I say that I don't use IDE for high-performance machines? > IDE has limitations. Mainly, the fact that IDE controllers require far more CPU involvement than any SCSI controller, especially on a saturated bus. A good SCSI controller can stay below 2% under almost any circumstance. A bad IDE one can go above 20%. I don't think I should have to say any more. ^_^ > Luckily, with the 4-way interleaved memory, it'll take up to 16 gigs, > and with 16 slots, there's a lot of room to add more. : ) And I was assuming a non server-class. Stupid me. Yeah, why 2GB? Hell, I'd put in the 16GB, and let it start caching result sets in memory. Even if Postgres doesn't allow such fine-grained control of various memory-hogging caching schemes now, it may add them later for optimization's sake. Memory is cheap. More *IS* the answer these days when concerning RAM. Besides, if you have a small database (5-10GB), you could serve it from a RAM device. Can't get much faster than that. ^_^ -- +-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-+ | Shaun M. Thomas INN Database Programmer | | Phone: (309) 743-0812 Fax : (309) 743-0830 | | Email: sthomas@townnews.com AIM : trifthen | | Web : hamster.lee.net | | | | "Most of our lives are about proving something, either to | | ourselves or to someone else." | | -- Anonymous | +-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-+
I second the RAM point. Not only is more better, you also need to configure postgres to use it. You will need to take the average size of your backends, the number of concurrent connections you expect and the amount of RAM you want postgres to leave for the OS and all the other processes. This will let you back into the proper buffer setting to best use the available RAM without letting the postmaster go to swap space. Buffers are not the only variable for memory. You need to allow space in ram for sort memory as well. As far as disks go. You cannot have too many spindles. But the number of spindles you have available depends on which pieces of postgres get split onto dedicated spindles. A note on SMP. Postgres is not a threaded application. The best you can hope for is that multiple connections get spread over multiple processors. Each individual connection lives on a single CPU. Tunning is somewhat of a black art to get the right balance. If you have to make a choice, buy fewer processors, faster disks, and as much RAM as the board will handle. -- Randy Hall - Red Hat Certified Engineer - Ex-Great Bridge PostgreSQL Expert Resume: http://members.home.net/rthall3 ----- Original Message ----- From: "Trond Eivind Glomsrød" <teg@redhat.com> To: "bpalmer" <bpalmer@crimelabs.net> Cc: <pgsql-general@postgresql.org> Sent: Monday, September 10, 2001 4:54 PM Subject: Re: [GENERAL] How to make a REALLY FAST db server? > bpalmer <bpalmer@crimelabs.net> writes: > > > I'm looking to make a DB server for a project I'm working on (using pgsql) > > and am wondering if people have suggestions? > > > > Thoughts: > > > > - Hardware: dual / quad Intel class > > Athlon gives more bang for the buck - the dual Athlons are _really_ > nice, and have proven stable as well. > > > - Disk: SCSI Raid 1+0 > > I'd probably go for a 3ware RAID instead... IDE disks are so much > cheaper nowadays than SCSI, and the premium isn't really justifiable > anymore. > > > - OS: Prolly FreeBSD (FFS is your friend (what with syncs and all) and it > > can do multi proc support > > I'd recommend Linux, which has more mature MP support and scales > better, but I'm obviously biased :). It's of course very important to > keep what you are familiar with - a good sysadmin makes a world of > difference no matter what you're using. > > > - Ram: Not really sure here. Is there math somewhere for ram needs for > > pgsql? I imagine is has something to do with # connections, db size, > > etc. > > "More is better. RAM is cheap. Avoid RAMBUS". > > -- > Trond Eivind Glomsrød > Red Hat, Inc. > > ---------------------------(end of broadcast)--------------------------- > TIP 5: Have you checked our extensive FAQ? > > http://www.postgresql.org/users-lounge/docs/faq.html >
> As far as disks go. You cannot have too many spindles. But the number of > spindles you have available depends on which pieces of postgres get split > onto dedicated spindles. But if you have enough RAM that the entire DB can be held in RAM, and if the DB acess pattern is 90% read, 10% write (which is usually the case) then do you really need to worry about disk performance? It seems like for most applications, it is possible to hold all the data in RAM, and RAM is cheap these days.
"Shaun Thomas" <sthomas@townnews.com> wrote in message news:Pine.LNX.4.33L2.0109101410050.1809-100000@hamster.lee.net... > > - Disk: SCSI Raid 1+0 > > To really eek out as much speed as possible here, you'll want 10k RPM > Ultra-160 Fibre Channel SCSI drives with a dedicated hardware raid > controller. If have more reads than writes, you may want to use Raid 5 > instead. 15k RPM drives are available from Seagate and IBM. The Seagate drives are slightly faster. Cheers, Colin
Indeed, there are applications where this is true. Postgres will try to keep as much in the buffers as possible. You always have to take your application into account when tuning the database. Your case where the database may fit entirely in RAM is quite different from a server holding many gigabytes of data and associated index files. -- Randy Hall - Red Hat Certified Engineer - Ex-Great Bridge PostgreSQL Expert Resume: http://members.home.net/rthall3 ----- Original Message ----- From: "Dr. Evil" <drevil@sidereal.kz> To: <pgsql-general@postgresql.org> Sent: Tuesday, September 11, 2001 12:12 AM Subject: Re: [GENERAL] How to make a REALLY FAST db server? > > As far as disks go. You cannot have too many spindles. But the number of > > spindles you have available depends on which pieces of postgres get split > > onto dedicated spindles. > > But if you have enough RAM that the entire DB can be held in RAM, and > if the DB acess pattern is 90% read, 10% write (which is usually the > case) then do you really need to worry about disk performance? > > It seems like for most applications, it is possible to hold all the > data in RAM, and RAM is cheap these days.
> Tunning is somewhat of a black art to get the right balance. If you have to > make a choice, buy fewer processors, faster disks, and as much RAM as the > board will handle. Wow. I'd buy more RAM and processors, and maybe skimp a *little* on the disks. The RAID array in my machine is made up of some 9-gig Quantum Atlas 4's that I bought for something like $80 each, far from being state-of the art: But with all of the RAM, the disk lights only blink *occasionally*. The number of drives makes up for the lower transfer rate of each individual unit, and the disk cache (both on the controller and from the kernel) make up for the latency. steve
On Mon, 10 Sep 2001 21:14:33 +0000 (UTC), teg@redhat.com (Trond Eivind Glomsrød) wrote: >> - Hardware: dual / quad Intel class > >Athlon gives more bang for the buck - the dual Athlons are _really_ >nice, and have proven stable as well. > Agreed, single Athlons are much better than single Intels, but I still think Intel win in the SMP game. As for stability, I think it's a tad too early to make such a statement. >> - Disk: SCSI Raid 1+0 > >I'd probably go for a 3ware RAID instead... IDE disks are so much >cheaper nowadays than SCSI, and the premium isn't really justifiable >anymore. > Aargh, No! SCSI is MUCH better under load and more reliable. I've yet to see an enterprise server utilise IDE raid. >> - OS: Prolly FreeBSD (FFS is your friend (what with syncs and all) and it >> can do multi proc support > >I'd recommend Linux, which has more mature MP support and scales >better, but I'm obviously biased :). It's of course very important to >keep what you are familiar with - a good sysadmin makes a world of >difference no matter what you're using. I'd recommend Solaris, which has MUCH more mature MP support! Linux still has a way to go (although it's getting there very quickly). If you do decide to use Linux, check out ReiserFS file system.
On Mon, 10 Sep 2001 20:39:18 +0000 (UTC), bpalmer@crimelabs.net ("B.
Palmer") wrote:
>>Not sure about that. Is that optimal for I/O?
>
From my experience it is. As long as you have a raid controler that
>can do 2 level RAID abstraction. First you need mirrored pairs and
>then you stripe over them. It costs a lot in disk, but is stupid fast
>with the right raid controller. With some Suns / FC / EMC, we were
>getting ~100M/s+ with that setup for our Oracle server.
Agreed. I've seen 2-3 times speed improvement over raid5.
"Shaun Thomas" <sthomas@townnews.com> wrote in message news:Pine.LNX.4.33L2.0109101709380.1809-100000@hamster.lee.net... > On Mon, 10 Sep 2001, Steve Wolfe wrote: > > > So, why did I say that I don't use IDE for high-performance machines? > > IDE has limitations. > > Mainly, the fact that IDE controllers require far more CPU involvement > than any SCSI controller, especially on a saturated bus. A good SCSI > controller can stay below 2% under almost any circumstance. A bad IDE one > can go above 20%. I don't think I should have to say any more. ^_^ Your example is not very convincing. You want to compare a good SCSI controller with a bad IDE one? That's not what one would call equitable. Consider: let's get an antique 50 pin adaptec SCSI controller, and compare it to a 3ware escalade running RAID 0 over 4 drives. The IDE one is faster! It saturates the PCI bus! The SCSI controller pokes along at 10 MB/s. Doesn't prove anything. The business about CPU utilization did, in fact, use do be an issue, but hasn't been since the introduction of UltraDMA IDE controllers, which was a few years ago. (Admittedly, if you've been running WinNT 4 all this time, you haven't been able to take advantage of it.) Marshall
""Steve Wolfe"" <steve@iboats.com> wrote in message news:002801c13a3f$15f34660$50824e40@iboats.com... > (As an aside, one person was in a heated argument about how much cheaper > IDE was than SCSI. I got on pricewatch, found some prices, and would have > been able to put together a very fast SCSI system for the same price as > his IDE array.) That's nuts: SCSI disks cost a lot more than comparable IDE disks. Marshall
Ian Linwood <ian@dinwoodi.plus.com> writes: > On Mon, 10 Sep 2001 21:14:33 +0000 (UTC), teg@redhat.com (Trond Eivind > Glomsrød) wrote: > > >> - Hardware: dual / quad Intel class > > > >Athlon gives more bang for the buck - the dual Athlons are _really_ > >nice, and have proven stable as well. > > > Agreed, single Athlons are much better than single Intels, but I still > think Intel win in the SMP game. As for stability, I think it's a tad > too early to make such a statement. Their SMP works very well, and has less bus competition than Intel (of course, this means making motherboards gets more complicated(. > >I'd probably go for a 3ware RAID instead... IDE disks are so much > >cheaper nowadays than SCSI, and the premium isn't really justifiable > >anymore. > > > Aargh, No! SCSI is MUCH better under load and more reliable. I've yet > to see an enterprise server utilise IDE raid. Mostly prejudice - 3ware is not a lowend "software disguised as hardware" RAID. It looks like a SCSI disk to your system, and handles stress very well. Remember - everything is hardware, there is nothing to stress on your system by using IDE in the subsystem. > >> - OS: Prolly FreeBSD (FFS is your friend (what with syncs and all) and it > >> can do multi proc support > > > >I'd recommend Linux, which has more mature MP support and scales > >better, but I'm obviously biased :). It's of course very important to > >keep what you are familiar with - a good sysadmin makes a world of > >difference no matter what you're using. > > I'd recommend Solaris, which has MUCH more mature MP support! We're talking 2-4 CPUs here. Linux has no problems with that, and is much more mature on x86 than Solaris (it's not their primary platform, by a long shot) > If you do decide to use Linux, check out ReiserFS file system. ReiserFS has had plenty problems, and is still not something I'd consider safe. Also, performance for postgresql on reiserfs was a quite a bit slower than ext3 and XFS the last time I checked (which was 4-5 months ago). Avoid it. -- Trond Eivind Glomsrød Red Hat, Inc.
I agree with this - the 3ware IDE RAID controllers are really, really good. Get your hands on one if you want a top-notch RAID with IDE hard drive prices. Most of the issues people have had with IDE raid have been addressed.. and with 160GB IDE drives coming in, its possible to build 1 terabyte RAID for $2k-$3k USD. > Your example is not very convincing. You want to compare a > good SCSI controller with a bad IDE one? That's not what one > would call equitable. > > Consider: let's get an antique 50 pin adaptec SCSI > controller, and compare it to a 3ware escalade running RAID 0 > over 4 drives. The IDE one is faster! It saturates the PCI > bus! The SCSI controller pokes along at 10 MB/s. Doesn't > prove anything. > > The business about CPU utilization did, in fact, use do be an > issue, but hasn't been since the introduction of UltraDMA IDE > controllers, which was a few years ago. (Admittedly, if > you've been running WinNT 4 all this time, you haven't been > able to take advantage of it.)
> > (As an aside, one person was in a heated argument about how much cheaper > > IDE was than SCSI. I got on pricewatch, found some prices, and would have > > been able to put together a very fast SCSI system for the same price as > > his IDE array.) > > That's nuts: SCSI disks cost a lot more than comparable IDE disks. But it's true. For cutting-edge SCSI disks, the price is quite high. If you look, though, you can find places trying to get rid of "last-year's model" for very low prices, and you can sometimes find very good performers like that. I picked up some 10K IBM drives with 4 ms access times for something like $125 or $150 each. If you'd like to find "comparable" IDE drives for that price, you're out of luck, as IDE doesn't have a 160 MB/sec bus, and I don't think you'll find any IDE drives with that low of access times anywhere. (Not to mention the fact that IDE drives only do well when a single process is accessing them. SCSI, having been designed from the gound up for this sort of thing, does much better when you're hitting the disks from several places at once.) steve
* Ian Linwood <ian@dinwoodi.plus.com> wrote: | | I'd recommend Solaris, which has MUCH more mature MP support! | Linux still has a way to go (although it's getting there very | quickly). Maybe the Solaris MP support is more mature, I haven't tested. But I did some performance tests with PostgreSQL 7.0.x a little more than a year ago. The test were replay of real life usage of our web applications. The platforms were : Solaris Sparc E220r? (450 MHz cpu, 1 gig ram, 10000 rpm disks) Solaris Intel 750 Mhz, 256 MB ram, 7200 rpm disk. Linux Intel(my laptop), 466 Mhz, 128 MB ram, don't know the speed of the disks. My laptop had no problem outpeforming the others by a factor of 2. On a big batchimport, I first tested on my laptop and it took like 15-20 minutes. The same import uses approximately 4 hours on our production box(the Solaris Sparc...) -- Gunnar Rønning - gunnar@polygnosis.com Senior Consultant, Polygnosis AS, http://www.polygnosis.com/
On 15 Sep 2001, Gunnar Rønning wrote:
> * Ian Linwood <ian@dinwoodi.plus.com> wrote:
> |
> | I'd recommend Solaris, which has MUCH more mature MP support!
> | Linux still has a way to go (although it's getting there very
> | quickly).
>
> Maybe the Solaris MP support is more mature, I haven't tested.
> But I did some performance tests with PostgreSQL 7.0.x a little
> more than a year ago. The test were replay of real life usage of our
> web applications.
>
> The platforms were :
>
> Solaris Sparc E220r? (450 MHz cpu, 1 gig ram, 10000 rpm disks)
> Solaris Intel 750 Mhz, 256 MB ram, 7200 rpm disk.
> Linux Intel(my laptop), 466 Mhz, 128 MB ram, don't know the speed of the disks.
>
> My laptop had no problem outpeforming the others by a factor of 2.
>
> On a big batchimport, I first tested on my laptop and it took like 15-20
> minutes. The same import uses approximately 4 hours on our production box(the
> Solaris Sparc...)
A "me too" was posted under subject "Sparc seems very slow" some weeks
ago. I did some hd-performance tests using dd and found out that while
the plain
dd if=big-file-in of=big-file-out
shows a relation
PC/Linux : Sparc/Solaris : Sparc/Linux
51s 191s 99s
an additional bs=1024k changed this relation to
PC/Linux : Sparc/Solaris : Sparc/Linux
61s 45s 35s
(which makes the choice of Linux for this particular test of dd which
is surely no SMP-test the best). So adjusting the read/write buffer
size could be the reason for the performance problems in the beginning.
Anyway I have no idea how to tune the thing for real applications like
PostgreSQL.
Any hints?
Kind regards
Andreas.
"Ian Linwood" <ian@dinwoodi.plus.com> wrote in message news:8p1hqt8ds5ejacr7hhefb381f4nrerdkt4@4ax.com... > >> Aargh, No! SCSI is MUCH better under load and more reliable. I've yet > >> to see an enterprise server utilise IDE raid. > > > >Mostly prejudice - 3ware is not a lowend "software disguised as > >hardware" RAID. It looks like a SCSI disk to your system, and handles > >stress very well. Remember - everything is hardware, there is nothing > >to stress on your system by using IDE in the subsystem. > > No, not prejudice, mostly experience. Do you have some data to back this up? I'd be interested in any published reports, benchmarks, or even the experience you mention, if you took measurements. Marshall
""Steve Wolfe"" <steve@iboats.com> wrote in message news:002c01c13d59$528f6f00$50824e40@iboats.com... > > > (As an aside, one person was in a heated argument about how much > cheaper > > > IDE was than SCSI. I got on pricewatch, found some prices, and would > have > > > been able to put together a very fast SCSI system for the same price > as > > > his IDE array.) > > > > That's nuts: SCSI disks cost a lot more than comparable IDE disks. > > But it's true. For cutting-edge SCSI disks, the price is quite high. > If you look, though, you can find places trying to get rid of "last-year's > model" for very low prices, and you can sometimes find very good > performers like that. I picked up some 10K IBM drives with 4 ms access > times for something like $125 or $150 each. If you'd like to find > "comparable" IDE drives for that price, you're out of luck, as IDE doesn't > have a 160 MB/sec bus, and I don't think you'll find any IDE drives with > that low of access times anywhere. What I said: "SCSI disks cost a lot more than comparable IDE disks." What you said: "No, because I found some cheap SCSI disks that don't have comparable IDE models." What you say may be true, but it does not in any way refute what I said about comparing prices between comparable SCSI and IDE models. My statement only operates in the domain where there ARE comparable models. SCSI disks cost more than comparable IDE disks. An example: (prices from pricewatch) 7200rpm 36.4gb SCSI disk $191 7200rpm 40.0gb ATA100 disk $97. Here the ratio is juts below 2:1, which is cheaper than the last time I made the survey, when the ratio was about 2.5:1. These days, there aren't many SCSI drives below 10000 rpm, and there aren't any (?) IDE drives above 7200 rpm, so making the comparison is harder. I suspect that IDE is taking over the low end and SCSI is losing market share all around. I'd guess the reduced demand is responsible for the declining prices. As far as the first contention, about arrays: IDE array 3ware 7410 $265 4x WD100BB drives $254 ea total: $1281, uses 4 drive bays SCSI array (cheapest $/gb on scsi is had with smaller drives. You can run the numbers with larger drives but it's more expensive.) adaptec 131-u2 RAID $316 11x IBM Ultrastar 36XP $191ea total: $2417. Both arrays will saturate your bus. The scsi array will probably perform marginally better, if only because it has more drives. > (Not to mention the fact that IDE drives only do well when a single > process is accessing them. SCSI, having been designed from the gound up > for this sort of thing, does much better when you're hitting the disks > from several places at once.) Do you have any hard data to back this up? I'd be interested. Usually when this question is asked, the response is "well, I've run a lot of servers, and they just seem faster." But I'm only interested in hard data. Marshall
""Steve Wolfe"" <steve@iboats.com> wrote in message news:002c01c13d59$528f6f00$50824e40@iboats.com... > > > (As an aside, one person was in a heated argument about how much > cheaper > > > IDE was than SCSI. I got on pricewatch, found some prices, and would > have > > > been able to put together a very fast SCSI system for the same price > as > > > his IDE array.) > > > > That's nuts: SCSI disks cost a lot more than comparable IDE disks. > > But it's true. For cutting-edge SCSI disks, the price is quite high. > If you look, though, you can find places trying to get rid of "last-year's > model" for very low prices, and you can sometimes find very good > performers like that. I picked up some 10K IBM drives with 4 ms access > times for something like $125 or $150 each. If you'd like to find > "comparable" IDE drives for that price, you're out of luck, as IDE doesn't > have a 160 MB/sec bus, and I don't think you'll find any IDE drives with > that low of access times anywhere. What I said: "SCSI disks cost a lot more than comparable IDE disks." What you said: "No, because I found some cheap SCSI disks that don't have comparable IDE models." What you say may be true, but it does not in any way refute what I said about comparing prices between comparable SCSI and IDE models. My statement only operates in the domain where there ARE comparable models. SCSI disks cost more than comparable IDE disks. An example: (prices from pricewatch) 7200rpm 36.4gb SCSI disk $191 7200rpm 40.0gb ATA100 disk $97. Here the ratio is juts below 2:1, which is cheaper than the last time I made the survey, when the ratio was about 2.5:1. These days, there aren't many SCSI drives below 10000 rpm, and there aren't any (?) IDE drives above 7200 rpm, so making the comparison is harder. I suspect that IDE is taking over the low end and SCSI is losing market share all around. I'd guess the reduced demand is responsible for the declining prices. As far as the first contention, about arrays: IDE array 3ware 7410 $265 4x WD100BB drives $254 ea total: $1281, uses 4 drive bays SCSI array (cheapest $/gb on scsi is had with smaller drives. You can run the numbers with larger drives but it's more expensive.) adaptec 131-u2 RAID $316 11x IBM Ultrastar 36XP $191ea total: $2417. Both arrays will saturate your bus. The scsi array will probably perform marginally better, if only because it has more drives. > (Not to mention the fact that IDE drives only do well when a single > process is accessing them. SCSI, having been designed from the gound up > for this sort of thing, does much better when you're hitting the disks > from several places at once.) Do you have any hard data to back this up? I'd be interested. Usually when this question is asked, the response is "well, I've run a lot of servers, and they just seem faster." But I'm only interested in hard data. Marshall
Marshall Spight wrote: > Do you have some data to back this up? I'd be interested in any > published reports, benchmarks, or even the experience you mention, > if you took measurements. Someone could volunteer to collect pgbench results for various configurations. I'd be very interested, since we're about to purchase a new RAID system and considering the 3ware Escalade series vs. several SCSI controllers... Collected data should include: - OS/Version - File system - CPUs & RAM (including type of RAM, i.e. PC133, DDR, Rambus...) - Controller & RAID configuration - PostgreSQL version & configuration (e.g. "-F -B512 -o '-S 65536'") Useful parameters for pgbench would probably be: pgbench -c1 -t 1000 pgbench -c4 -t 1000 pgbench -c10 -t 1000 Comments? Volunteers? -mjy
> What I said: "SCSI disks cost a lot more than comparable IDE disks." > > What you said: "No, because I found some cheap SCSI disks that > don't have comparable IDE models." That's not what I said. If you're going to quote me, get it right. > My statement only operates in the domain where > there ARE comparable models. You say that SCSI disks are more expensive than comparable IDE disks. I say that it's usually impossible to find an IDE disk that truly *is* comparable to a SCSI disk - and when you do, you're generally comparing the most recent IDE disk to an older SCSI disk. If nothing but peak transfer rate is all that matters to you in a hard drive, then yes, you can find comparable IDE and SCSI disks - but if that's all that matters, you're being pretty short-sighted. And to compare nothing but drive features when talking about your disk I/O subsytem is also very short-sighted. steve
> > (Not to mention the fact that IDE drives only do well when a single > > process is accessing them. SCSI, having been designed from the gound up > > for this sort of thing, does much better when you're hitting the disks > > from several places at once.) > > > Do you have any hard data to back this up? I'd be interested. > > Usually when this question is asked, the response is "well, > I've run a lot of servers, and they just seem faster." But I'm > only interested in hard data. Well, SCSI with tagged queueing allows you to send multiple disk requests to the drive and the drive orders them to be optimal. Only the drive knows there the head it at a given moment so it seems good to push such optimizations into the disk drive. -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 853-3000 + If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup. | Drexel Hill, Pennsylvania 19026
> On Thu, Sep 27, 2001 at 03:35:09PM -0400, Bruce Momjian wrote: > > > Usually when this question is asked, the response is "well, > > > I've run a lot of servers, and they just seem faster." But I'm > > > only interested in hard data. > > > > Well, SCSI with tagged queueing allows you to send multiple disk > > requests to the drive and the drive orders them to be optimal. Only the > > drive knows there the head it at a given moment so it seems good to push > > such optimizations into the disk drive. > > >From what I gather from reading lkml, recent ATA standards do include such > capabilities. Whether it is supports by the drivers, controllers and disks > is another matter entirely. But it is there. The ATA system basically has the disk controller in the CPU while SCSI has commands that you send to the drive and the drive implements them. Not sure how you could implement tagged queueing with the ATA system. -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 853-3000 + If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup. | Drexel Hill, Pennsylvania 19026
On Thu, Sep 27, 2001 at 03:35:09PM -0400, Bruce Momjian wrote: > > Usually when this question is asked, the response is "well, > > I've run a lot of servers, and they just seem faster." But I'm > > only interested in hard data. > > Well, SCSI with tagged queueing allows you to send multiple disk > requests to the drive and the drive orders them to be optimal. Only the > drive knows there the head it at a given moment so it seems good to push > such optimizations into the disk drive. From what I gather from reading lkml, recent ATA standards do include such capabilities. Whether it is supports by the drivers, controllers and disks is another matter entirely. But it is there. -- Martijn van Oosterhout <kleptog@svana.org> http://svana.org/kleptog/ > Magnetism, electricity and motion are like a three-for-two special offer: > if you have two of them, the third one comes free.
> > >From what I gather from reading lkml, recent ATA standards do include such > > capabilities. Whether it is supports by the drivers, controllers and disks > > is another matter entirely. But it is there. > > The ATA system basically has the disk controller in the CPU while SCSI > has commands that you send to the drive and the drive implements them. > Not sure how you could implement tagged queueing with the ATA system. Or the "disconnect" feature that lets a device receive a request, then release control of the bus and do it's own thing until it's ready to respond. AFAIK, once an IDE device has received a request, it can't relinquish control of the bus until it's sent the answer back. steve
I need a scalable DB server to handle very large datasets, concurency and scalability are not that critical (yet) I'm working with a production engineer at Channel Micro (www.channel-micro.com) to test different hardware configurationsand one of the configs is using a 3Ware RAID. Can anyone provide any DB benchmaking software? I'm trying http://osdb.sourceforge.net/ as well as the MySQL suit and somegeneric testing based on my data. Thanks, -Aaron Held http://www.metrony.com "Marshall Spight" <marshall@meetstheeye.com> wrote in message news:9odit2$r49$1@news.tht.net... > "Ian Linwood" <ian@dinwoodi.plus.com> wrote in message > news:8p1hqt8ds5ejacr7hhefb381f4nrerdkt4@4ax.com... > > >> Aargh, No! SCSI is MUCH better under load and more reliable. I've yet > > >> to see an enterprise server utilise IDE raid. > > > > > >Mostly prejudice - 3ware is not a lowend "software disguised as > > >hardware" RAID. It looks like a SCSI disk to your system, and handles > > >stress very well. Remember - everything is hardware, there is nothing > > >to stress on your system by using IDE in the subsystem. > > > > No, not prejudice, mostly experience. > > > Do you have some data to back this up? I'd be interested in any > published reports, benchmarks, or even the experience you mention, > if you took measurements. > > > Marshall > > >
At 18:35 27.9.2001, Bruce Momjian wrote the following message: >Well, SCSI with tagged queueing allows you to send multiple disk >requests to the drive and the drive orders them to be optimal. Only the >drive knows there the head it at a given moment so it seems good to push >such optimizations into the disk drive. IBM Deskstar drives DPTA and DLTA and new IC35 all have tagged queuing. FreeBSD supports that and the results are impressive - lower CPU usage and faster disks. ---- Tomaz Borstnar <tomaz.borstnar@over.net> "Love is the answer to the final question you ask" - Unknown
FWIW, I have some IBM-DPTA-372050 and after enabling TCQ with FreeBSD I barely noticed any improvement, and occasionally strange error messages appeared, so I disabled it again. I believe you need more recent drives (with updated firmware, no doubt) to get any performance gain.. - Andrew On Thu, 6 Dec 2001, Tomaz Borstnar wrote: > At 18:35 27.9.2001, Bruce Momjian wrote the following message: > >Well, SCSI with tagged queueing allows you to send multiple disk > >requests to the drive and the drive orders them to be optimal. Only the > >drive knows there the head it at a given moment so it seems good to push > >such optimizations into the disk drive. > > IBM Deskstar drives DPTA and DLTA and new IC35 all have tagged queuing. > FreeBSD supports that and the results are impressive - lower CPU usage and > faster disks. > > > > ---- > Tomaz Borstnar <tomaz.borstnar@over.net> > "Love is the answer to the final question you ask" - Unknown > > > > ---------------------------(end of broadcast)--------------------------- > TIP 3: if posting/reading through Usenet, please send an appropriate > subscribe-nomail command to majordomo@postgresql.org so that your > message can get through to the mailing list cleanly >
since aceshardware didn't publish their database server in the article about "building a better webserver" i sent an email to ask. sybase. rjsjr
You have to use zero rotation delay to newfs to see an improvement. Do you have that? --------------------------------------------------------------------------- > > FWIW, I have some IBM-DPTA-372050 and after enabling TCQ with FreeBSD > I barely noticed any improvement, and occasionally strange error messages > appeared, so I disabled it again. I believe you need more recent > drives (with updated firmware, no doubt) to get any performance gain.. > > - Andrew > > > On Thu, 6 Dec 2001, Tomaz Borstnar wrote: > > > At 18:35 27.9.2001, Bruce Momjian wrote the following message: > > >Well, SCSI with tagged queueing allows you to send multiple disk > > >requests to the drive and the drive orders them to be optimal. Only the > > >drive knows there the head it at a given moment so it seems good to push > > >such optimizations into the disk drive. > > > > IBM Deskstar drives DPTA and DLTA and new IC35 all have tagged queuing. > > FreeBSD supports that and the results are impressive - lower CPU usage and > > faster disks. > > > > > > > > ---- > > Tomaz Borstnar <tomaz.borstnar@over.net> > > "Love is the answer to the final question you ask" - Unknown > > > > > > > > ---------------------------(end of broadcast)--------------------------- > > TIP 3: if posting/reading through Usenet, please send an appropriate > > subscribe-nomail command to majordomo@postgresql.org so that your > > message can get through to the mailing list cleanly > > > > > ---------------------------(end of broadcast)--------------------------- > TIP 2: you can get off all lists at once with the unregister command > (send "unregister YourEmailAddressHere" to majordomo@postgresql.org) > -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 853-3000 + If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup. | Drexel Hill, Pennsylvania 19026