Thread: RAID Configuration Sugestion

RAID Configuration Sugestion

From
Alvaro Nunes Melo
Date:
Hello,

We are about to install a new PostgreSQL server, and despite of being a
very humble configuration compared to the ones we see in the list, it's
the biggest one we've got till now.

The server is a Dual Xeon 3.0 with 2 GB RAM and two SCSI disks. Our main
doubt is what is the best configuration for the disks. We are thinking
about use them in a RAID-0 array. Is this the best option? What do you
suggest on partitioning? Separate partitions for the OS, data and pg_xlog?

We'll have some time to work on performance tests, and if someone is
interested we can provide our results.

Thanks in advance,
Alvaro

Re: RAID Configuration Sugestion

From
Bruno Wolff III
Date:
On Tue, Aug 30, 2005 at 09:37:17 -0300,
  Alvaro Nunes Melo <al_nunes@atua.com.br> wrote:
>
> The server is a Dual Xeon 3.0 with 2 GB RAM and two SCSI disks. Our main
> doubt is what is the best configuration for the disks. We are thinking
> about use them in a RAID-0 array. Is this the best option? What do you
> suggest on partitioning? Separate partitions for the OS, data and pg_xlog?

You don't have a lot of options with just two disks. What are you trying
to accomplish with raid?

Raid 0 will possibly give you some speed up, while raid 1 will give you some
fault tolerance, some speed of of reads, but cost you half your disk space.

Re: RAID Configuration Sugestion

From
Bruno Wolff III
Date:
Please keep replies copied to the list so that others may contribute to
and learn from the discussion.

On Tue, Aug 30, 2005 at 10:15:13 -0300,
  Alvaro Nunes Melo <al_nunes@atua.com.br> wrote:
> Hello Bruno,
>
> Bruno Wolff III wrote:
>
> >On Tue, Aug 30, 2005 at 09:37:17 -0300,
> > Alvaro Nunes Melo <al_nunes@atua.com.br> wrote:
> >
> >
> >>The server is a Dual Xeon 3.0 with 2 GB RAM and two SCSI disks. Our main
> >>doubt is what is the best configuration for the disks. We are thinking
> >>about use them in a RAID-0 array. Is this the best option? What do you
> >>suggest on partitioning? Separate partitions for the OS, data and pg_xlog?
> >
> Our main goal is performance speedup. Disk space might not be a problem.
> I've read a lot here about movig pg_xlog to different partitions, and
> we'll surely make tests to see what configuration might be better.

This isn't a very good mix of hardware for running postgres. Xeons have
some context switching issues for which you will probably see some
speed up in 8.1. (So if you aren't going into production for sevral
months you might want to be using 8.1beta.) Having only two disk drives
is also not a good idea.

With what you have you either want to use raid 0 and not worry too much
about how the disks are partitioned or use one disk for wal logging
and the other for other stuff. There are other people on the list who
can probably give you a better idea of which of these options is likely
to be better in your case. However, they may need to know more about
your raid controller. In particular how much battery backed memory does
it have and its model.

Re: RAID Configuration Sugestion

From
Ron
Date:
At 08:37 AM 8/30/2005, Alvaro Nunes Melo wrote:
>Hello,
>
>We are about to install a new PostgreSQL server, and despite of
>being a very humble configuration compared to the ones we see in the
>list, it's the biggest one we've got till now.
>
>The server is a Dual Xeon 3.0 with 2 GB RAM and two SCSI disks. Our
>main doubt is what is the best configuration for the disks. We are
>thinking about use them in a RAID-0 array. Is this the best option?
>What do you suggest on partitioning? Separate partitions for the OS,
>data and pg_xlog?

This is _very_ modest HW.  Unless your DB and/or DB load is similarly
modest, you are not going to be happy with the performance of your DBMS.

At a minimum, for safety reasons you want 4 HDs: 2 for a RAID 1 set
for the DB, and 2 for a RAID 1 set for the OS + pg_xlog.
2 extra HDs, even SCSI HDs, is cheap.  Especially when compared to
the cost of corrupted or lost data.

HD's and RAM are cheap enough that you should be able to upgrade in
more ways, but do at least that "upgrade"!

Beyond that, the best ways to spend you limited $ are highly
dependent on your exact DB and its usage pattern.

Ron Peacetree



Re: RAID Configuration Sugestion

From
Matthew Nuzum
Date:
> > >On Tue, Aug 30, 2005 at 09:37:17 -0300,
> > > Alvaro Nunes Melo <al_nunes@atua.com.br> wrote:
> > >>The server is a Dual Xeon 3.0 with 2 GB RAM and two SCSI disks. Our main
> > >>doubt is what is the best configuration for the disks. We are thinking
> > >>about use them in a RAID-0 array. Is this the best option? What do you
> > >>suggest on partitioning? Separate partitions for the OS, data and pg_xlog?
> > >
> > Our main goal is performance speedup. Disk space might not be a problem.
> > I've read a lot here about movig pg_xlog to different partitions, and
> > we'll surely make tests to see what configuration might be better.
>

I've set up several servers with a config like this. Its not ideal,
but there's no reason you can't enjoy the benefits of a snappy
application.

The best results I've had involve dedicating one drive to OS, swap,
logs, tmp and everything and dedicate one drive to postgres. If you
use *nix you can mount the second drive as /var/lib/pgsql (or where
ever postgres lives on your server) with noatime as a mount option.

In retrospect, you might have saved the money on the second CPU and
gotten two more hard drives, but if you're running a dual task server
(i.e. LAMP) you may appreciate the second CPU.

The beauty of a server like this is that it puts more of the wizardry
of creating a fast application into the hands of the app developer,
which results in a better db schema, optimized queries and generally
*thinking* about the performance of the code. I personally feel that
to be a very rewarding aspect of my job. (As a hobby I program
microntrollers that run at 4MHz and have only 256 bytes of RAM, so
that could just be me.;-)

--
Matthew Nuzum
www.bearfruit.org

Re: RAID Configuration Sugestion

From
"Joshua D. Drake"
Date:
Ron wrote:

> At 08:37 AM 8/30/2005, Alvaro Nunes Melo wrote:
>
>> Hello,
>>
>> We are about to install a new PostgreSQL server, and despite of being
>> a very humble configuration compared to the ones we see in the list,
>> it's the biggest one we've got till now.
>>
>> The server is a Dual Xeon 3.0 with 2 GB RAM and two SCSI disks. Our
>> main doubt is what is the best configuration for the disks. We are
>> thinking about use them in a RAID-0 array. Is this the best option?
>> What do you suggest on partitioning? Separate partitions for the OS,
>> data and pg_xlog?
>
>
> This is _very_ modest HW.  Unless your DB and/or DB load is similarly
> modest, you are not going to be happy with the performance of your DBMS.

Well that is a pretty blanket statement. I have many customers who
happily run in less hardware that what is mentioned above.
It all depends on the application itself and how the database is utilized.

> At a minimum, for safety reasons you want 4 HDs: 2 for a RAID 1 set
> for the DB, and 2 for a RAID 1 set for the OS + pg_xlog.
> 2 extra HDs, even SCSI HDs, is cheap.  Especially when compared to the
> cost of corrupted or lost data.

Your real test is going to be prototyping the performance you need. A
single RAID 1 mirror (don't use RAID 0) may be more
than enough. However based on the fact that you speced Xeons my guess is
you spent money on CPUs when you should have
spent money on hard drives.

If you still have the budget, I would suggest considering either what
Ron suggested or possibly using a 4 drive RAID 10 instead.

If you can't afford to put a couple more SCSI disks it may be worth
while to put a software RAID 1 with ATA disks for the OS and
swap and then use straight SCSI hardware RAID 1 for the DB. That will
allow you to push any swap operations off to the OS disks
without sacrificing the performance and reliability of the database itself.

Sincerely,

Joshua D. Drake


>
> HD's and RAM are cheap enough that you should be able to upgrade in
> more ways, but do at least that "upgrade"!
>
> Beyond that, the best ways to spend you limited $ are highly dependent
> on your exact DB and its usage pattern.
>
> Ron Peacetree
>
>
>
> ---------------------------(end of broadcast)---------------------------
> TIP 1: if posting/reading through Usenet, please send an appropriate
>       subscribe-nomail command to majordomo@postgresql.org so that your
>       message can get through to the mailing list cleanly



Re: RAID Configuration Sugestion

From
Ron
Date:
At 12:56 PM 8/30/2005, Joshua D. Drake wrote:
>Ron wrote:
>
>>At 08:37 AM 8/30/2005, Alvaro Nunes Melo wrote:
>>
>>>Hello,
>>>
>>>We are about to install a new PostgreSQL server, and despite of
>>>being a very humble configuration compared to the ones we see in
>>>the list, it's the biggest one we've got till now.
>>>
>>>The server is a Dual Xeon 3.0 with 2 GB RAM and two SCSI disks.
>>>Our main doubt is what is the best configuration for the disks. We
>>>are thinking about use them in a RAID-0 array. Is this the best
>>>option? What do you suggest on partitioning? Separate partitions
>>>for the OS, data and pg_xlog?
>>
>>
>>This is _very_ modest HW.  Unless your DB and/or DB load is
>>similarly modest, you are not going to be happy with the
>>performance of your DBMS.
>
>Well that is a pretty blanket statement. I have many customers who
>happily run in less hardware that what is mentioned above.
>It all depends on the application itself and how the database is utilized.

If your customers "run happily" on 2 HD's, then IME they have very
modest DB storage and/or DB performance needs.  For safety reasons,
the best thing to do if you only have 2 HD's is to run them as a RAID
1 with everything on them.  The slightly better performing but
considerably less safe alternative is to put the OS + logs on 1 HD
and the DB on the other.  Any resemblance to a semi-serious OLTP load
will reduce either such system to an HD IO bound one with poor IO rates.

If, as above, your DBMS is bounded by the performance of one HD, then
you are AT BEST getting the raw IO rate of such a device: say
~70-80MB/s in average sustained raw sequential IO.  Files system
overhead and any seeking behavior will rapidly reduce that number to
considerably less.  Consider that the CPU <-> memory IO subsystem is
easily capable of ~3.2GBps.  So you are talking about slowing the  DB
server to at most ~1/40, maybe even as little as ~1/200, its
potential under such circumstances.

If your DB can fit completely in RAM and/or does light duty write IO,
this may not be a serious issue.  OTOH, once you start using those
HD's to any reasonable extent, most of the rest of the investment
you've made in server HW is wasted.

As I keep saying, the highest priority in purchasing a DBMS is to
make sure you have enough HD IO bandwidth.  RAM comes second, and CPU
is a distant third.


>>At a minimum, for safety reasons you want 4 HDs: 2 for a RAID 1 set
>>for the DB, and 2 for a RAID 1 set for the OS + pg_xlog.
>>2 extra HDs, even SCSI HDs, is cheap.  Especially when compared to
>>the cost of corrupted or lost data.
>
>Your real test is going to be prototyping the performance you need.
>A single RAID 1 mirror (don't use RAID 0) may be more
>than enough. However based on the fact that you speced Xeons my
>guess is you spent money on CPUs when you should have
>spent money on hard drives.

I agree with Josh on both points.  Don't use RAID 0 for persistent
data unless you like losing data.  Spend more on HDs and RAM and less
on CPU's (fast FSB is far more important than high clock rate.  In
general buy the highest FSB with the slowest clock rate.).  If fact,
if you are that strapped for cash, exchange those 2 SCSI HD's for
their $ equivalent in SATA HD's.  The extra spindles will be well worth it.


>If you still have the budget, I would suggest considering either
>what Ron suggested or possibly using a 4 drive RAID 10 instead.

IME, with only 4 HDs, it's usually better to split them them into two
RAID 1's (one for the db, one for everything else including the logs)
than it is to put everything on one RAID 10.  YMMV.


Ron Peacetree



Re: RAID Configuration Sugestion

From
Matthew Nuzum
Date:
On 8/30/05, Ron <rjpeace@earthlink.net> wrote:
> >If you still have the budget, I would suggest considering either
> >what Ron suggested or possibly using a 4 drive RAID 10 instead.
>
> IME, with only 4 HDs, it's usually better to split them them into two
> RAID 1's (one for the db, one for everything else including the logs)
> than it is to put everything on one RAID 10.  YMMV.

This coresponds to what I have observed as well. Of course, we all
know that work loads varry.

Just a note for the OP who has only two drives, there are tools for a
variety of OSs that monitor the S.M.A.R.T. features of the drive and
give an early warning in case it senses impending failure. I've caught
two drives before failure with these types of tools.

Also note that when reading discussions of this nature you must take
into consideration the value of your data. For some people, restoring
from a nightly backup is inconvienent, but not life-or-death. Some
people even do twice-daily backups so that in case of a failure they
can recover with little loss of data. This might be a good way to
mitigate the cost of expensive server hardware. If you cannot afford
to lose any data then you need to consider it imperitive to use some
type of RAID setup (not RAID 0) and to achieve great performance
you'll want more than 2 drives.
--
Matthew Nuzum
www.bearfruit.org

Re: RAID Configuration Sugestion

From
"Joshua D. Drake"
Date:
>
>> If you still have the budget, I would suggest considering either what
>> Ron suggested or possibly using a 4 drive RAID 10 instead.
>
>
> IME, with only 4 HDs, it's usually better to split them them into two
> RAID 1's (one for the db, one for everything else including the logs)
> than it is to put everything on one RAID 10.  YMMV.

Really? That's interesting.  My experience is different, I assume SCSI?
Software/Hardware Raid?

Sincerely,

Joshua D. Drake


>
>
> Ron Peacetree
>


Re: RAID Configuration Sugestion

From
Ron
Date:
At 03:27 PM 8/30/2005, Joshua D. Drake wrote:


>>>If you still have the budget, I would suggest considering either
>>>what Ron suggested or possibly using a 4 drive RAID 10 instead.
>>
>>
>>IME, with only 4 HDs, it's usually better to split them them into
>>two RAID 1's (one for the db, one for everything else including the
>>logs) than it is to put everything on one RAID 10.  YMMV.
>
>Really? That's interesting.  My experience is different, I assume
>SCSI? Software/Hardware Raid?

The issue exists regardless of technologies used, although the
technology used does affect when things become an irritation or
serious problem.

The issue with "everything on the same HD set" seems to be that under
light loads anything works reasonably well, but as load increases
contention between DB table access, OS access, and xlog writes can
cause performance problems.

In particular, _everything_ else hangs while logs are being written
with "everything on the same HD set".  Thus leaving you with the
nasty choices of small log writes that cause more seeking behavior,
and the resultant poor overall HD IO performance, or large log writes
that basically freeze the server until they are done.

Having the logs on a different HD, and if possible different IO bus,
reduces this effect to a minimum and seems to be a better choice than
the "shared everything" approach.

Although this effect seems largest when there are fewest HDs, the
general pattern  is that one should use as many spindles as one can
make use of and that they should be as dedicated as possible in their
purpose(s).  That's why the TPC bench marked systems tend to have
literally 100's of HD's and they tend to be split into very focused purposes.

Ron Peacetree



Re: RAID Configuration Sugestion

From
Michael Stone
Date:
On Tue, Aug 30, 2005 at 07:02:28PM -0400, Ron wrote:
>purpose(s).  That's why the TPC bench marked systems tend to have
>literally 100's of HD's and they tend to be split into very focused
>purposes.

Of course, TPC benchmark systems are constructed such that cost and
storage capacity are irrelevant--in the real world things tend to be
more complicated.

Mike Stone

Re: RAID Configuration Sugestion

From
Ron
Date:
At 08:04 PM 8/30/2005, Michael Stone wrote:
>On Tue, Aug 30, 2005 at 07:02:28PM -0400, Ron wrote:
>>purpose(s).  That's why the TPC bench marked systems tend to have
>>literally 100's of HD's and they tend to be split into very focused purposes.
>
>Of course, TPC benchmark systems are constructed such that cost and
>storage capacity are irrelevant--in the real world things tend to be
>more complicated.

The scary thing is that I've worked on RW production systems that
bore a striking resemblance to a TPC benchmark system.  As you can
imagine, they uniformly belonged to BIG organizations (read: lot's 'o
$$$) who were using the systems for mission critical stuff where
either it was company existence threatening for the system to be
done, or they would lose much $$$ per min of down time, or both.

Financial institutions, insurance companies, central data mines for
Fortune 2000 companies, etc _all_ build systems that push the state
of the art in how much storage can be managed and how many HDs, CPUs,
RAM DIMMs, etc are usable.

Historically, this has been the sole province of Oracle and DB2 on
the SW side and equally outrageously priced custom HW.  Clearly, I'd
like to see PostgreSQL change that ;-)

Ron Peacetree



Re: RAID Configuration Sugestion

From
Michael Stone
Date:
On Tue, Aug 30, 2005 at 08:41:40PM -0400, Ron wrote:
>The scary thing is that I've worked on RW production systems that
>bore a striking resemblance to a TPC benchmark system.  As you can
>imagine, they uniformly belonged to BIG organizations (read: lot's 'o
>$$$) who were using the systems for mission critical stuff where
>either it was company existence threatening for the system to be
>done, or they would lose much $$$ per min of down time, or both.

Yeah, and that market is relevant to someone with one dell server and 2
hard disks how?

Mike Stone

Re: RAID Configuration Sugestion

From
Ron
Date:
At 08:43 PM 8/30/2005, Michael Stone wrote:
>On Tue, Aug 30, 2005 at 08:41:40PM -0400, Ron wrote:
>>The scary thing is that I've worked on RW production systems that
>>bore a striking resemblance to a TPC benchmark system.  As you can
>>imagine, they uniformly belonged to BIG organizations (read: lot's
>>'o $$$) who were using the systems for mission critical stuff where
>>either it was company existence threatening for the system to be
>>done, or they would lose much $$$ per min of down time, or both.
>
>Yeah, and that market is relevant to someone with one dell server
>and 2 hard disks how?
Because successful small companies that _start_ with one small server
and 2 HDs grow to _become_ companies that need far more HW; ...and in
the perfect world their SW scales to their increased needs...

_Without_ exponentially increasing their costs or overhead (as Oracle
and DB2 currently do)

THIS is the real long term promise of OS DBMS.

Ron Peacetree