Thread: Configuring System for Speed
I am in the process of speccing out a new box for a highly utilized (updates, inserts, selects) 40GB+ database. I'm trying to maximize performance on a budget, and I would appreciate any feedback on any of the following. Hardware: 2 - Intel Xeon 5160 3.0 GHz 4MB 1333MHz 8 - Kingston 4GB DDR2 ECC Fully Buffered 16 - WD Raptor 150GB 10000 RPM SATA II HD 1 - 3Ware 9550SX-16ML Serial ATA II RAID card 1 - Supermicro X7DBE+ Motherboard 1 - Supermicro SC836TQ-R800V Case The Woodcrest CPU, Raptor HD and 3Ware 9550SX RAID card have all been highly recommended on this list, so there may not be much to comment on there. The Supermicro X7DBE+ motherboard is appealing because of its 16 RAM slots and 64GB of maximum memory. The Supermicro SC836TQ- R800 case was selected because of its 16 drive bays in 3U and dual 800W power supplies (the motherboard requires 700W). RAID Configuration/File System: A lot of info in the lists, but I'm not sure of the best approach for this setup. I was thinking of using 14 drives in a RAID 10 (ext3) for the db and wals and 2 mirrored drives (ext3 or xfs) for the OS. Another option would be 12 drives in a RAID 10 for the database (ext3, maybe ext2) and 4 drives in a RAID 10 for the OS and wals (ext3) (with separate RAID cards). There are many choices here. Any suggestions? OS: The consensus in the list seems to be as long as you have the 2.6 Linux Kernel, it's really a matter of personal preference. However, it's hard to have a preference when you're new to the Linux world, like I am. Red Hat, Fedora Core, Slackware, Suse, Gentoo? I guess my primary goal is speed, stability, and ease of use. Any advice here, no matter how minimal, would be appreciated. Thanks, Brian Wipf
Brian, I like all of the HW - I just thoroughly reviewed this and came to the same HW choices you did. The new SuperMicro chassis is an improvement on one we have used for 21 servers like this. One modification: we implemented two internal 60GB laptop hard drives with an additional 3Ware 8006-2LP controller on each machine for the OS. This frees up all 16 SATA II drives for data. The laptop drives are super stable and reliable under high heat conditions and they are small, so having them inside the case is no big deal. BTW - we had ASAcomputers.com make our systems for us and they did a good job. We use CentOS 4 on our lab systems and it works fine. I recommend XFS for the DBMS data drives, along with RAID10 on the 3Ware controllers. With normal Postgres you shouldn't expect to get more than about 350MB/s on those CPUs for a single query, but with increased user count, the RAID10 should scale fine to about 1200MB/s sequential transfer using XFS and a lot slower with ext3. - Luke On 9/8/06 12:52 AM, "Brian Wipf" <brian@clickspace.com> wrote: > I am in the process of speccing out a new box for a highly utilized > (updates, inserts, selects) 40GB+ database. I'm trying to maximize > performance on a budget, and I would appreciate any feedback on any > of the following. > > Hardware: > 2 - Intel Xeon 5160 3.0 GHz 4MB 1333MHz > 8 - Kingston 4GB DDR2 ECC Fully Buffered > 16 - WD Raptor 150GB 10000 RPM SATA II HD > 1 - 3Ware 9550SX-16ML Serial ATA II RAID card > 1 - Supermicro X7DBE+ Motherboard > 1 - Supermicro SC836TQ-R800V Case > > The Woodcrest CPU, Raptor HD and 3Ware 9550SX RAID card have all been > highly recommended on this list, so there may not be much to comment > on there. The Supermicro X7DBE+ motherboard is appealing because of > its 16 RAM slots and 64GB of maximum memory. The Supermicro SC836TQ- > R800 case was selected because of its 16 drive bays in 3U and dual > 800W power supplies (the motherboard requires 700W). > > RAID Configuration/File System: > A lot of info in the lists, but I'm not sure of the best approach for > this setup. > I was thinking of using 14 drives in a RAID 10 (ext3) for the db and > wals and 2 mirrored drives (ext3 or xfs) for the OS. Another option > would be 12 drives in a RAID 10 for the database (ext3, maybe ext2) > and 4 drives in a RAID 10 for the OS and wals (ext3) (with separate > RAID cards). There are many choices here. Any suggestions? > > OS: > The consensus in the list seems to be as long as you have the 2.6 > Linux Kernel, it's really a matter of personal preference. However, > it's hard to have a preference when you're new to the Linux world, > like I am. Red Hat, Fedora Core, Slackware, Suse, Gentoo? I guess my > primary goal is speed, stability, and ease of use. Any advice here, > no matter how minimal, would be appreciated. > > Thanks, > > Brian Wipf > > > ---------------------------(end of broadcast)--------------------------- > TIP 6: explain analyze is your friend >
Brian Wipf wrote: > I am in the process of speccing out a new box for a highly utilized > (updates, inserts, selects) 40GB+ database. I'm trying to maximize > performance on a budget, and I would appreciate any feedback on any of > the following. > Perhaps this is off topic, but here is bit from my experience. Using single server for both read (select) and write (insert, update, delete) operations is the way to slow things down. Consider to split query workload into OLTP and OLAP queries . Set up Slony replication and use slave server for read operations only . Buy 1 cheap box (slave) and another more expensive one for master. Keep in mind that DB schema optimization for particular query workload is essential . You just can not get good performance for selects if schema is optimized for inserts and vice versa. > > OS: > The consensus in the list seems to be as long as you have the 2.6 > Linux Kernel, it's really a matter of personal preference. However, > it's hard to have a preference when you're new to the Linux world, > like I am. Red Hat, Fedora Core, Slackware, Suse, Gentoo? I guess my > primary goal is speed, stability, and ease of use. Any advice here, no > matter how minimal, would be appreciated. My answer is FreeBSD6.1. > > Thanks, > > Brian Wipf > > > ---------------------------(end of broadcast)--------------------------- > TIP 6: explain analyze is your friend -- Best Regards, Alvis
I agree with Luke's comments, although Luke knows I'm an unabashed Areca admirer where RAID controllers are concerned. As tweakers.net recently reaffirmed, Areca cards with 1GB of BB cache on them are faster than the AMCC/3Ware 9550 series. Where Linux kernels are concerned, make sure you are at least at 2.6.15 Most decent distros will be. 4GB DIMMs are still much more than 2x as expensive per than 2GB DIMMs are. You may want to buy 16 2GB DIMMs instead of 8 4GB DIMMs given the current price difference. Especially if you won't be upgrading your RAM for a while. Ron At 04:44 AM 9/8/2006, Luke Lonergan wrote: >Brian, > >I like all of the HW - I just thoroughly reviewed this and came to the same >HW choices you did. The new SuperMicro chassis is an improvement on one we >have used for 21 servers like this. > >One modification: we implemented two internal 60GB laptop hard drives with >an additional 3Ware 8006-2LP controller on each machine for the OS. This >frees up all 16 SATA II drives for data. The laptop drives are super stable >and reliable under high heat conditions and they are small, so having them >inside the case is no big deal. BTW - we had ASAcomputers.com make our >systems for us and they did a good job. > >We use CentOS 4 on our lab systems and it works fine. I recommend XFS for >the DBMS data drives, along with RAID10 on the 3Ware controllers. With >normal Postgres you shouldn't expect to get more than about 350MB/s on those >CPUs for a single query, but with increased user count, the RAID10 should >scale fine to about 1200MB/s sequential transfer using XFS and a lot slower >with ext3. > >- Luke > > >On 9/8/06 12:52 AM, "Brian Wipf" <brian@clickspace.com> wrote: > > > I am in the process of speccing out a new box for a highly utilized > > (updates, inserts, selects) 40GB+ database. I'm trying to maximize > > performance on a budget, and I would appreciate any feedback on any > > of the following. > > > > Hardware: > > 2 - Intel Xeon 5160 3.0 GHz 4MB 1333MHz > > 8 - Kingston 4GB DDR2 ECC Fully Buffered > > 16 - WD Raptor 150GB 10000 RPM SATA II HD > > 1 - 3Ware 9550SX-16ML Serial ATA II RAID card > > 1 - Supermicro X7DBE+ Motherboard > > 1 - Supermicro SC836TQ-R800V Case > > > > The Woodcrest CPU, Raptor HD and 3Ware 9550SX RAID card have all been > > highly recommended on this list, so there may not be much to comment > > on there. The Supermicro X7DBE+ motherboard is appealing because of > > its 16 RAM slots and 64GB of maximum memory. The Supermicro SC836TQ- > > R800 case was selected because of its 16 drive bays in 3U and dual > > 800W power supplies (the motherboard requires 700W). > > > > RAID Configuration/File System: > > A lot of info in the lists, but I'm not sure of the best approach for > > this setup. > > I was thinking of using 14 drives in a RAID 10 (ext3) for the db and > > wals and 2 mirrored drives (ext3 or xfs) for the OS. Another option > > would be 12 drives in a RAID 10 for the database (ext3, maybe ext2) > > and 4 drives in a RAID 10 for the OS and wals (ext3) (with separate > > RAID cards). There are many choices here. Any suggestions? > > > > OS: > > The consensus in the list seems to be as long as you have the 2.6 > > Linux Kernel, it's really a matter of personal preference. However, > > it's hard to have a preference when you're new to the Linux world, > > like I am. Red Hat, Fedora Core, Slackware, Suse, Gentoo? I guess my > > primary goal is speed, stability, and ease of use. Any advice here, > > no matter how minimal, would be appreciated. > > > > Thanks, > > > > Brian Wipf > > > > > > ---------------------------(end of broadcast)--------------------------- > > TIP 6: explain analyze is your friend > > > > > >---------------------------(end of broadcast)--------------------------- >TIP 5: don't forget to increase your free space map settings
On 8-Sep-06, at 2:44 AM, Luke Lonergan wrote: > One modification: we implemented two internal 60GB laptop hard > drives with > an additional 3Ware 8006-2LP controller on each machine for the OS. > This > frees up all 16 SATA II drives for data. That's a great idea. One question though. If I put all 16 drives in a RAID 10 for the database, where should I put the logs? On that large RAID set? If I use a RAID controller with a BB cache for the mirrored laptop drives, might I be able to use that for the logs and OS? Brian Wipf
Brian, On 9/11/06 8:50 AM, "Brian Wipf" <brian@clickspace.com> wrote: > That's a great idea. One question though. If I put all 16 drives in a > RAID 10 for the database, where should I put the logs? On that large > RAID set? If I use a RAID controller with a BB cache for the mirrored > laptop drives, might I be able to use that for the logs and OS? I think I'd probably reserve a couple of the data drives for the WAL, at 16 you have far oversubscribed non-MPP postgres' ability to use them for bandwidth and you'd only lose about 5% of your seek performance by dropping a couple of drives. The laptop drives are really slow, so putting your WAL on them might be a net loss. Alternately you could go with dual SFF SAS drives, which are the same physical size as the laptop drives but are 10,000 RPM SCSI equivalent. Heat removal will be more of an issue with these potentially, especially if you have a lot of activity on them. - Luke