Thread: Hardware suggestions for high performance 8.3
Hi list, We have a database with lots of small simultaneous writes and reads (millions every day) and are looking at buying a good hardware for this. What are your suggestions. What we are currently looking at is. Dual Quad Core Intel 8 - 12 GB RAM 10 disks total. 4 x 146 GB SAS disk in RAID 1+0 for database 6 x 750 GB SATA disks in RAID 1+0 or RAID 5 for OS and transactions logs. Good RAID controller with lots of memory and BBU. Any hints, recommendations would be greatly appreciated. Cheers, Henke
> We have a database with lots of small simultaneous writes and reads > (millions every day) and are looking at buying a good hardware for this. > > What are your suggestions. What we are currently looking at is. > > Dual Quad Core Intel > 8 - 12 GB RAM > > 10 disks total. > > 4 x 146 GB SAS disk in RAID 1+0 for database > 6 x 750 GB SATA disks in RAID 1+0 or RAID 5 for OS and transactions logs. > > Good RAID controller with lots of memory and BBU. I have very positive experiences with HP's DL360 and DL380. The latter slightly more expandable (2U vs. 1U). I have used the internal p400i-controller with 512 MB cache on the DL380 and bought an external p800-controller (512 MB cache as well) and a MSA-70-cabinet. I've have 11 disks in raid-6 (one hotspare). I don't see any reason to mix sas- and sata-disks with different sizes. I'd go for sas-disks, smaller and faster, less power and heat. Raid 1+0 or raid-6 does not seem to make much of a difference today as it used to if you have more than 6-7 disks. The DL380 is a 4-way woodcrest at 3 GHz and 16 GB ram and the DL360 is a two-way woodcrest at 2.66 GHz with 16 GB. My personal preference is FreeBSD and the DL3x0-servers all run without problems on this platform. But choose your OS depending on what you're most comfortable with. And choose hardware according to what your OS supports. Areca-controllers may also be worth looking into but I haven't tried these myself. Our largest table has 85 mill. entries. -- regards Claus When lenity and cruelty play for a kingdom, the gentlest gamester is the soonest winner. Shakespeare
On Wed, 25 Jun 2008, Henrik wrote:
> What are your suggestions. What we are currently looking at is.
>
> Dual Quad Core Intel
> 8 - 12 GB RAM
More RAM would be helpful. It's not that expensive, compared to the rest
of your system.
> 10 disks total.
>
> 4 x 146 GB SAS disk in RAID 1+0 for database
> 6 x 750 GB SATA disks in RAID 1+0 or RAID 5 for OS and transactions logs.
>
> Good RAID controller with lots of memory and BBU.
If you have a good RAID controller with BBU cache, then there's no point
splitting the discs into two sets. You're only creating an opportunity to
under-utilise the system. I'd get ten identical discs and put them in a
single array, probably RAID 10.
Also, do you really need 6*750GB for OS and transaction logs? How big can
they be?
However, the most important factor is that you get a good BBU cache.
Matthew
--
I don't want the truth. I want something I can tell parliament!
-- Rt. Hon. Jim Hacker MP
25 jun 2008 kl. 12.56 skrev Claus Guttesen: >> We have a database with lots of small simultaneous writes and reads >> (millions every day) and are looking at buying a good hardware for >> this. >> >> What are your suggestions. What we are currently looking at is. >> >> Dual Quad Core Intel >> 8 - 12 GB RAM >> >> 10 disks total. >> >> 4 x 146 GB SAS disk in RAID 1+0 for database >> 6 x 750 GB SATA disks in RAID 1+0 or RAID 5 for OS and transactions >> logs. >> >> Good RAID controller with lots of memory and BBU. > > I have very positive experiences with HP's DL360 and DL380. The latter > slightly more expandable (2U vs. 1U). I have used the internal > p400i-controller with 512 MB cache on the DL380 and bought an external > p800-controller (512 MB cache as well) and a MSA-70-cabinet. I've have > 11 disks in raid-6 (one hotspare). Mmm I've used DL380 and I also have had good experience with them. I guess that the nees of splitting up the transactions logs are not that important if you have enought disks in a raid 10 or raid 6. > > My personal preference is FreeBSD and the DL3x0-servers all run > without problems on this platform. But choose your OS depending on > what you're most comfortable with. And choose hardware according to > what your OS supports. > I like BDS also but this time its a 64bit Linux system which wil be used. > > > Our largest table has 85 mill. entries. > I believe we will be running in the 200 mill. area. Thanks for your input! //Henke
25 jun 2008 kl. 13.15 skrev Matthew Wakeling: > On Wed, 25 Jun 2008, Henrik wrote: >> What are your suggestions. What we are currently looking at is. >> >> Dual Quad Core Intel >> 8 - 12 GB RAM > > More RAM would be helpful. It's not that expensive, compared to the > rest of your system. True, as long as I can build the system on 2G or 4G modules I can max out the banks. > > >> 10 disks total. >> >> 4 x 146 GB SAS disk in RAID 1+0 for database >> 6 x 750 GB SATA disks in RAID 1+0 or RAID 5 for OS and transactions >> logs. >> >> Good RAID controller with lots of memory and BBU. > > If you have a good RAID controller with BBU cache, then there's no > point splitting the discs into two sets. You're only creating an > opportunity to under-utilise the system. I'd get ten identical discs > and put them in a single array, probably RAID 10. OK, thats good to know. Really want to keep it as simple as possible. Would you turn off fsync if you had a controller with BBU? =) > > > Also, do you really need 6*750GB for OS and transaction logs? How > big can they be? Ahh, we are going to save a lot of other datafiles on those also but maybe i'll just get a cabinett. > > > However, the most important factor is that you get a good BBU cache. Here that! Thanks for your input! //Henke
>> If you have a good RAID controller with BBU cache, then there's no point >> splitting the discs into two sets. You're only creating an opportunity to >> under-utilise the system. I'd get ten identical discs and put them in a >> single array, probably RAID 10. > > OK, thats good to know. Really want to keep it as simple as possible. Would > you turn off fsync if you had a controller with BBU? =) No, don't do that. Leaving this setting on is *highly* recommended unless you have data which can easily be reproduced. :-) -- regards Claus When lenity and cruelty play for a kingdom, the gentlest gamester is the soonest winner. Shakespeare
On Wed, 25 Jun 2008, Henrik wrote:
> Would you turn off fsync if you had a controller with BBU? =)
No, certainly not. Fsync is what makes the data move from the volatile OS
cache to the non-volatile disc system. It'll just be a lot quicker on a
controller with a BBU cache, because it won't need to actually wait for
the discs. But you still need the fsync to move the data from main OS
cache to BBU cache.
>> Also, do you really need 6*750GB for OS and transaction logs? How big can
>> they be?
>
> Ahh, we are going to save a lot of other datafiles on those also but maybe
> i'll just get a cabinett.
Or you could just get 10 large SATA drives. To be honest, the performance
difference is not large, especially if you ensure the database data is
held compactly on the discs, so the seeks are small.
Matthew
--
It's one of those irregular verbs - "I have an independent mind," "You are
an eccentric," "He is round the twist."
-- Bernard Woolly, Yes Prime Minister
On Wed, 25 Jun 2008, Henrik wrote: > 4 x 146 GB SAS disk in RAID 1+0 for database > 6 x 750 GB SATA disks in RAID 1+0 or RAID 5 for OS and transactions logs. The transaction logs are not that big, and there's very little value to striping them across even two disks. You should just get more SAS disks instead and make them available to the database, adding more spindles for random I/O is much more important. Separating out a single RAID-1 pair from that set to hold the logs is a reasonable practice, with a good battery-backed controller even that might not buy you anything useful. -- * Greg Smith gsmith@gregsmith.com http://www.gregsmith.com Baltimore, MD
On Wed, 25 Jun 2008, Henrik wrote: > Would you turn off fsync if you had a controller with BBU? =) Turning off fsync has some potential to introduce problems even in that environment, so better not to do that. The issue is that you might have, say, 1GB of OS-level cache but 256MB of BBU cache, and if you turn fsync off it won't force the OS cache out to the controller when it's supposed to and that can cause corruption. Also, if you've got a controller with BBU, the overhead of fsync for regular writes is low enough that you don't really need to turn it off. If writes are cached the fsync is almost free. -- * Greg Smith gsmith@gregsmith.com http://www.gregsmith.com Baltimore, MD
25 jun 2008 kl. 17.45 skrev Greg Smith: > On Wed, 25 Jun 2008, Henrik wrote: > >> Would you turn off fsync if you had a controller with BBU? =) > > Turning off fsync has some potential to introduce problems even in > that environment, so better not to do that. The issue is that you > might have, say, 1GB of OS-level cache but 256MB of BBU cache, and > if you turn fsync off it won't force the OS cache out to the > controller when it's supposed to and that can cause corruption. > > Also, if you've got a controller with BBU, the overhead of fsync for > regular writes is low enough that you don't really need to turn it > off. If writes are cached the fsync is almost free. Thanks for a thoroughly answer. I guess I wont be turning of fsync. :) Thanks Greg! > > > -- > * Greg Smith gsmith@gregsmith.com http://www.gregsmith.com > Baltimore, MD > > -- > Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org > ) > To make changes to your subscription: > http://www.postgresql.org/mailpref/pgsql-performance
I've seen some concerns about buying database performance hardware from DELL. Are there at least some of the RAID cards that work well with Linux or should I stay clear of DELL permanently? Thanks! //Henke 25 jun 2008 kl. 17.45 skrev Greg Smith: > On Wed, 25 Jun 2008, Henrik wrote: > >> Would you turn off fsync if you had a controller with BBU? =) > > Turning off fsync has some potential to introduce problems even in > that environment, so better not to do that. The issue is that you > might have, say, 1GB of OS-level cache but 256MB of BBU cache, and > if you turn fsync off it won't force the OS cache out to the > controller when it's supposed to and that can cause corruption. > > Also, if you've got a controller with BBU, the overhead of fsync for > regular writes is low enough that you don't really need to turn it > off. If writes are cached the fsync is almost free. > > -- > * Greg Smith gsmith@gregsmith.com http://www.gregsmith.com > Baltimore, MD > > -- > Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org > ) > To make changes to your subscription: > http://www.postgresql.org/mailpref/pgsql-performance
On Thu, 26 Jun 2008, Henrik wrote: > I've seen some concerns about buying database performance hardware from DELL. > Are there at least some of the RAID cards that work well with Linux or should > I stay clear of DELL permanently? People seem to be doing OK if the RAID card is their Perc/6i, which has an LSI Logic MegaRAID SAS 1078 chipset under the hood. There's some helpful benchmark results and follow-up meesages related to one of those at http://archives.postgresql.org/message-id/47D8B3B6.3010606@emolecules.com That said, I consider the rebranded LSI cards a pain and hate the quality of Dell's hardware. Seems like everybody I talk to lately is buying HP's DL380 instead of Dells for this level of Linux installs nowadays, I haven't gotten one of those HP boxes myself yet to comment. -- * Greg Smith gsmith@gregsmith.com http://www.gregsmith.com Baltimore, MD
On Thu, Jun 26, 2008 at 10:47 PM, Greg Smith <gsmith@gregsmith.com> wrote: > On Thu, 26 Jun 2008, Henrik wrote: > >> I've seen some concerns about buying database performance hardware from >> DELL. Are there at least some of the RAID cards that work well with Linux or >> should I stay clear of DELL permanently? > > People seem to be doing OK if the RAID card is their Perc/6i, which has an > LSI Logic MegaRAID SAS 1078 chipset under the hood. There's some helpful > benchmark results and follow-up meesages related to one of those at > http://archives.postgresql.org/message-id/47D8B3B6.3010606@emolecules.com Yeah, the problems I've had have been with the internal RAID (perc 5???) lsi based controllers. They kick their drives offline. Dell has a firmware update but we haven't had a chance to install it just yet to see if it fixes the problem with that one. > That said, I consider the rebranded LSI cards a pain and hate the quality of > Dell's hardware. Yeah, I'd just as soon get a regular LSI bios as the remade one Dell seems intent on pushing. Also, we just discovered the broadcom chipsets we have in our Dell 1950s and 1850s will not negotiate to gigabit with our Nortel switches. Everything else I've plugged in just worked. Went looking at Dell's site, and for the 1950 they recommend buying a dual port Intel NIC for it. Why couldn't they just build in better NICS to start?
Greg Smith wrote: > On Thu, 26 Jun 2008, Henrik wrote: > >> I've seen some concerns about buying database performance hardware >> from DELL. Are there at least some of the RAID cards that work well >> with Linux or should I stay clear of DELL permanently? > > People seem to be doing OK if the RAID card is their Perc/6i, which has > an LSI Logic MegaRAID SAS 1078 chipset under the hood. There's some > helpful benchmark results and follow-up meesages related to one of those > at > http://archives.postgresql.org/message-id/47D8B3B6.3010606@emolecules.com > > That said, I consider the rebranded LSI cards a pain and hate the > quality of Dell's hardware. Seems like everybody I talk to lately is > buying HP's DL380 instead of Dells for this level of Linux installs > nowadays, I haven't gotten one of those HP boxes myself yet to comment. > The HP P800 controller is a top notch performer. Joshua D. Drake P.S. The DL360-380 series is very nice as well > -- > * Greg Smith gsmith@gregsmith.com http://www.gregsmith.com Baltimore, MD >