Thread: RAID Configuration Sugestion
Hello, We are about to install a new PostgreSQL server, and despite of being a very humble configuration compared to the ones we see in the list, it's the biggest one we've got till now. The server is a Dual Xeon 3.0 with 2 GB RAM and two SCSI disks. Our main doubt is what is the best configuration for the disks. We are thinking about use them in a RAID-0 array. Is this the best option? What do you suggest on partitioning? Separate partitions for the OS, data and pg_xlog? We'll have some time to work on performance tests, and if someone is interested we can provide our results. Thanks in advance, Alvaro
On Tue, Aug 30, 2005 at 09:37:17 -0300, Alvaro Nunes Melo <al_nunes@atua.com.br> wrote: > > The server is a Dual Xeon 3.0 with 2 GB RAM and two SCSI disks. Our main > doubt is what is the best configuration for the disks. We are thinking > about use them in a RAID-0 array. Is this the best option? What do you > suggest on partitioning? Separate partitions for the OS, data and pg_xlog? You don't have a lot of options with just two disks. What are you trying to accomplish with raid? Raid 0 will possibly give you some speed up, while raid 1 will give you some fault tolerance, some speed of of reads, but cost you half your disk space.
Please keep replies copied to the list so that others may contribute to and learn from the discussion. On Tue, Aug 30, 2005 at 10:15:13 -0300, Alvaro Nunes Melo <al_nunes@atua.com.br> wrote: > Hello Bruno, > > Bruno Wolff III wrote: > > >On Tue, Aug 30, 2005 at 09:37:17 -0300, > > Alvaro Nunes Melo <al_nunes@atua.com.br> wrote: > > > > > >>The server is a Dual Xeon 3.0 with 2 GB RAM and two SCSI disks. Our main > >>doubt is what is the best configuration for the disks. We are thinking > >>about use them in a RAID-0 array. Is this the best option? What do you > >>suggest on partitioning? Separate partitions for the OS, data and pg_xlog? > > > Our main goal is performance speedup. Disk space might not be a problem. > I've read a lot here about movig pg_xlog to different partitions, and > we'll surely make tests to see what configuration might be better. This isn't a very good mix of hardware for running postgres. Xeons have some context switching issues for which you will probably see some speed up in 8.1. (So if you aren't going into production for sevral months you might want to be using 8.1beta.) Having only two disk drives is also not a good idea. With what you have you either want to use raid 0 and not worry too much about how the disks are partitioned or use one disk for wal logging and the other for other stuff. There are other people on the list who can probably give you a better idea of which of these options is likely to be better in your case. However, they may need to know more about your raid controller. In particular how much battery backed memory does it have and its model.
At 08:37 AM 8/30/2005, Alvaro Nunes Melo wrote: >Hello, > >We are about to install a new PostgreSQL server, and despite of >being a very humble configuration compared to the ones we see in the >list, it's the biggest one we've got till now. > >The server is a Dual Xeon 3.0 with 2 GB RAM and two SCSI disks. Our >main doubt is what is the best configuration for the disks. We are >thinking about use them in a RAID-0 array. Is this the best option? >What do you suggest on partitioning? Separate partitions for the OS, >data and pg_xlog? This is _very_ modest HW. Unless your DB and/or DB load is similarly modest, you are not going to be happy with the performance of your DBMS. At a minimum, for safety reasons you want 4 HDs: 2 for a RAID 1 set for the DB, and 2 for a RAID 1 set for the OS + pg_xlog. 2 extra HDs, even SCSI HDs, is cheap. Especially when compared to the cost of corrupted or lost data. HD's and RAM are cheap enough that you should be able to upgrade in more ways, but do at least that "upgrade"! Beyond that, the best ways to spend you limited $ are highly dependent on your exact DB and its usage pattern. Ron Peacetree
> > >On Tue, Aug 30, 2005 at 09:37:17 -0300, > > > Alvaro Nunes Melo <al_nunes@atua.com.br> wrote: > > >>The server is a Dual Xeon 3.0 with 2 GB RAM and two SCSI disks. Our main > > >>doubt is what is the best configuration for the disks. We are thinking > > >>about use them in a RAID-0 array. Is this the best option? What do you > > >>suggest on partitioning? Separate partitions for the OS, data and pg_xlog? > > > > > Our main goal is performance speedup. Disk space might not be a problem. > > I've read a lot here about movig pg_xlog to different partitions, and > > we'll surely make tests to see what configuration might be better. > I've set up several servers with a config like this. Its not ideal, but there's no reason you can't enjoy the benefits of a snappy application. The best results I've had involve dedicating one drive to OS, swap, logs, tmp and everything and dedicate one drive to postgres. If you use *nix you can mount the second drive as /var/lib/pgsql (or where ever postgres lives on your server) with noatime as a mount option. In retrospect, you might have saved the money on the second CPU and gotten two more hard drives, but if you're running a dual task server (i.e. LAMP) you may appreciate the second CPU. The beauty of a server like this is that it puts more of the wizardry of creating a fast application into the hands of the app developer, which results in a better db schema, optimized queries and generally *thinking* about the performance of the code. I personally feel that to be a very rewarding aspect of my job. (As a hobby I program microntrollers that run at 4MHz and have only 256 bytes of RAM, so that could just be me.;-) -- Matthew Nuzum www.bearfruit.org
Ron wrote: > At 08:37 AM 8/30/2005, Alvaro Nunes Melo wrote: > >> Hello, >> >> We are about to install a new PostgreSQL server, and despite of being >> a very humble configuration compared to the ones we see in the list, >> it's the biggest one we've got till now. >> >> The server is a Dual Xeon 3.0 with 2 GB RAM and two SCSI disks. Our >> main doubt is what is the best configuration for the disks. We are >> thinking about use them in a RAID-0 array. Is this the best option? >> What do you suggest on partitioning? Separate partitions for the OS, >> data and pg_xlog? > > > This is _very_ modest HW. Unless your DB and/or DB load is similarly > modest, you are not going to be happy with the performance of your DBMS. Well that is a pretty blanket statement. I have many customers who happily run in less hardware that what is mentioned above. It all depends on the application itself and how the database is utilized. > At a minimum, for safety reasons you want 4 HDs: 2 for a RAID 1 set > for the DB, and 2 for a RAID 1 set for the OS + pg_xlog. > 2 extra HDs, even SCSI HDs, is cheap. Especially when compared to the > cost of corrupted or lost data. Your real test is going to be prototyping the performance you need. A single RAID 1 mirror (don't use RAID 0) may be more than enough. However based on the fact that you speced Xeons my guess is you spent money on CPUs when you should have spent money on hard drives. If you still have the budget, I would suggest considering either what Ron suggested or possibly using a 4 drive RAID 10 instead. If you can't afford to put a couple more SCSI disks it may be worth while to put a software RAID 1 with ATA disks for the OS and swap and then use straight SCSI hardware RAID 1 for the DB. That will allow you to push any swap operations off to the OS disks without sacrificing the performance and reliability of the database itself. Sincerely, Joshua D. Drake > > HD's and RAM are cheap enough that you should be able to upgrade in > more ways, but do at least that "upgrade"! > > Beyond that, the best ways to spend you limited $ are highly dependent > on your exact DB and its usage pattern. > > Ron Peacetree > > > > ---------------------------(end of broadcast)--------------------------- > TIP 1: if posting/reading through Usenet, please send an appropriate > subscribe-nomail command to majordomo@postgresql.org so that your > message can get through to the mailing list cleanly
At 12:56 PM 8/30/2005, Joshua D. Drake wrote: >Ron wrote: > >>At 08:37 AM 8/30/2005, Alvaro Nunes Melo wrote: >> >>>Hello, >>> >>>We are about to install a new PostgreSQL server, and despite of >>>being a very humble configuration compared to the ones we see in >>>the list, it's the biggest one we've got till now. >>> >>>The server is a Dual Xeon 3.0 with 2 GB RAM and two SCSI disks. >>>Our main doubt is what is the best configuration for the disks. We >>>are thinking about use them in a RAID-0 array. Is this the best >>>option? What do you suggest on partitioning? Separate partitions >>>for the OS, data and pg_xlog? >> >> >>This is _very_ modest HW. Unless your DB and/or DB load is >>similarly modest, you are not going to be happy with the >>performance of your DBMS. > >Well that is a pretty blanket statement. I have many customers who >happily run in less hardware that what is mentioned above. >It all depends on the application itself and how the database is utilized. If your customers "run happily" on 2 HD's, then IME they have very modest DB storage and/or DB performance needs. For safety reasons, the best thing to do if you only have 2 HD's is to run them as a RAID 1 with everything on them. The slightly better performing but considerably less safe alternative is to put the OS + logs on 1 HD and the DB on the other. Any resemblance to a semi-serious OLTP load will reduce either such system to an HD IO bound one with poor IO rates. If, as above, your DBMS is bounded by the performance of one HD, then you are AT BEST getting the raw IO rate of such a device: say ~70-80MB/s in average sustained raw sequential IO. Files system overhead and any seeking behavior will rapidly reduce that number to considerably less. Consider that the CPU <-> memory IO subsystem is easily capable of ~3.2GBps. So you are talking about slowing the DB server to at most ~1/40, maybe even as little as ~1/200, its potential under such circumstances. If your DB can fit completely in RAM and/or does light duty write IO, this may not be a serious issue. OTOH, once you start using those HD's to any reasonable extent, most of the rest of the investment you've made in server HW is wasted. As I keep saying, the highest priority in purchasing a DBMS is to make sure you have enough HD IO bandwidth. RAM comes second, and CPU is a distant third. >>At a minimum, for safety reasons you want 4 HDs: 2 for a RAID 1 set >>for the DB, and 2 for a RAID 1 set for the OS + pg_xlog. >>2 extra HDs, even SCSI HDs, is cheap. Especially when compared to >>the cost of corrupted or lost data. > >Your real test is going to be prototyping the performance you need. >A single RAID 1 mirror (don't use RAID 0) may be more >than enough. However based on the fact that you speced Xeons my >guess is you spent money on CPUs when you should have >spent money on hard drives. I agree with Josh on both points. Don't use RAID 0 for persistent data unless you like losing data. Spend more on HDs and RAM and less on CPU's (fast FSB is far more important than high clock rate. In general buy the highest FSB with the slowest clock rate.). If fact, if you are that strapped for cash, exchange those 2 SCSI HD's for their $ equivalent in SATA HD's. The extra spindles will be well worth it. >If you still have the budget, I would suggest considering either >what Ron suggested or possibly using a 4 drive RAID 10 instead. IME, with only 4 HDs, it's usually better to split them them into two RAID 1's (one for the db, one for everything else including the logs) than it is to put everything on one RAID 10. YMMV. Ron Peacetree
On 8/30/05, Ron <rjpeace@earthlink.net> wrote: > >If you still have the budget, I would suggest considering either > >what Ron suggested or possibly using a 4 drive RAID 10 instead. > > IME, with only 4 HDs, it's usually better to split them them into two > RAID 1's (one for the db, one for everything else including the logs) > than it is to put everything on one RAID 10. YMMV. This coresponds to what I have observed as well. Of course, we all know that work loads varry. Just a note for the OP who has only two drives, there are tools for a variety of OSs that monitor the S.M.A.R.T. features of the drive and give an early warning in case it senses impending failure. I've caught two drives before failure with these types of tools. Also note that when reading discussions of this nature you must take into consideration the value of your data. For some people, restoring from a nightly backup is inconvienent, but not life-or-death. Some people even do twice-daily backups so that in case of a failure they can recover with little loss of data. This might be a good way to mitigate the cost of expensive server hardware. If you cannot afford to lose any data then you need to consider it imperitive to use some type of RAID setup (not RAID 0) and to achieve great performance you'll want more than 2 drives. -- Matthew Nuzum www.bearfruit.org
> >> If you still have the budget, I would suggest considering either what >> Ron suggested or possibly using a 4 drive RAID 10 instead. > > > IME, with only 4 HDs, it's usually better to split them them into two > RAID 1's (one for the db, one for everything else including the logs) > than it is to put everything on one RAID 10. YMMV. Really? That's interesting. My experience is different, I assume SCSI? Software/Hardware Raid? Sincerely, Joshua D. Drake > > > Ron Peacetree >
At 03:27 PM 8/30/2005, Joshua D. Drake wrote: >>>If you still have the budget, I would suggest considering either >>>what Ron suggested or possibly using a 4 drive RAID 10 instead. >> >> >>IME, with only 4 HDs, it's usually better to split them them into >>two RAID 1's (one for the db, one for everything else including the >>logs) than it is to put everything on one RAID 10. YMMV. > >Really? That's interesting. My experience is different, I assume >SCSI? Software/Hardware Raid? The issue exists regardless of technologies used, although the technology used does affect when things become an irritation or serious problem. The issue with "everything on the same HD set" seems to be that under light loads anything works reasonably well, but as load increases contention between DB table access, OS access, and xlog writes can cause performance problems. In particular, _everything_ else hangs while logs are being written with "everything on the same HD set". Thus leaving you with the nasty choices of small log writes that cause more seeking behavior, and the resultant poor overall HD IO performance, or large log writes that basically freeze the server until they are done. Having the logs on a different HD, and if possible different IO bus, reduces this effect to a minimum and seems to be a better choice than the "shared everything" approach. Although this effect seems largest when there are fewest HDs, the general pattern is that one should use as many spindles as one can make use of and that they should be as dedicated as possible in their purpose(s). That's why the TPC bench marked systems tend to have literally 100's of HD's and they tend to be split into very focused purposes. Ron Peacetree
On Tue, Aug 30, 2005 at 07:02:28PM -0400, Ron wrote: >purpose(s). That's why the TPC bench marked systems tend to have >literally 100's of HD's and they tend to be split into very focused >purposes. Of course, TPC benchmark systems are constructed such that cost and storage capacity are irrelevant--in the real world things tend to be more complicated. Mike Stone
At 08:04 PM 8/30/2005, Michael Stone wrote: >On Tue, Aug 30, 2005 at 07:02:28PM -0400, Ron wrote: >>purpose(s). That's why the TPC bench marked systems tend to have >>literally 100's of HD's and they tend to be split into very focused purposes. > >Of course, TPC benchmark systems are constructed such that cost and >storage capacity are irrelevant--in the real world things tend to be >more complicated. The scary thing is that I've worked on RW production systems that bore a striking resemblance to a TPC benchmark system. As you can imagine, they uniformly belonged to BIG organizations (read: lot's 'o $$$) who were using the systems for mission critical stuff where either it was company existence threatening for the system to be done, or they would lose much $$$ per min of down time, or both. Financial institutions, insurance companies, central data mines for Fortune 2000 companies, etc _all_ build systems that push the state of the art in how much storage can be managed and how many HDs, CPUs, RAM DIMMs, etc are usable. Historically, this has been the sole province of Oracle and DB2 on the SW side and equally outrageously priced custom HW. Clearly, I'd like to see PostgreSQL change that ;-) Ron Peacetree
On Tue, Aug 30, 2005 at 08:41:40PM -0400, Ron wrote: >The scary thing is that I've worked on RW production systems that >bore a striking resemblance to a TPC benchmark system. As you can >imagine, they uniformly belonged to BIG organizations (read: lot's 'o >$$$) who were using the systems for mission critical stuff where >either it was company existence threatening for the system to be >done, or they would lose much $$$ per min of down time, or both. Yeah, and that market is relevant to someone with one dell server and 2 hard disks how? Mike Stone
At 08:43 PM 8/30/2005, Michael Stone wrote: >On Tue, Aug 30, 2005 at 08:41:40PM -0400, Ron wrote: >>The scary thing is that I've worked on RW production systems that >>bore a striking resemblance to a TPC benchmark system. As you can >>imagine, they uniformly belonged to BIG organizations (read: lot's >>'o $$$) who were using the systems for mission critical stuff where >>either it was company existence threatening for the system to be >>done, or they would lose much $$$ per min of down time, or both. > >Yeah, and that market is relevant to someone with one dell server >and 2 hard disks how? Because successful small companies that _start_ with one small server and 2 HDs grow to _become_ companies that need far more HW; ...and in the perfect world their SW scales to their increased needs... _Without_ exponentially increasing their costs or overhead (as Oracle and DB2 currently do) THIS is the real long term promise of OS DBMS. Ron Peacetree