Re: Hardware vs Software RAID - Mailing list pgsql-performance

"Also sprach Greg Smith:"
> On Wed, 25 Jun 2008, Peter T. Breuer wrote:
>
> > You can put the log/bitmap wherever you want in software raid, including
> > on a battery-backed local ram disk if you feel so inclined.  So there is
> > no intrinsic advantage to be gained there at all.
>
> You are technically correct but this is irrelevant.  There are zero
> mainstream battery-backed local RAM disk setups appropriate for database
> use that don't cost substantially more than the upgrade cost to just

I refrained from saying in my reply that I would set up a firewire-based
link to ram in a spare old portable (which comes with a battery) if I
wanted to do this cheaply.

One reason I refrained was because I did not want to enter into a
discussion of transport speeds vs latency vs block request size.  GE,
for example, would have horrendous performance at 1KB i/o blocks. Mind
you, it still would be over 20MB/s (I measure 70MB/s to a real scsi
remote disk across GE at 64KB blocksize).

> getting a good hardware RAID controller with cache integrated and using
> regular disks.
>
> What I often do is get a hardware RAID controller, just to accelerate disk
> writes, but configure it in JBOD mode and use Linux or other software RAID
> on that platform.

I wonder what "JBOD mode" is ... :) Journaled block over destiny? Oh ..
"Just a Bunch of Disks". So you use the linux software raid driver
instead of the hardware or firmware driver on the raid assembly. Fair
enough.

> Advantages of using software RAID, in general and in some cases even with
> a hardware disk controller:
>
> -Your CPU is inevitably faster than the one on the controller, so this can
> give better performance than having RAID calcuations done on the
> controller itself.

It's not clear. You take i/o bandwidth out of the rest of your system,
and cpu time too.  In a standard dual core machine which is not a
workstation, it's OK.  On my poor ol' 1GHz P3 TP x24 laptop, doing two
things at once is definitely a horrible strain on my X responsiveness.
On a risc machine (ARM, 250MHz) I have seen horrible cpu loads from
software raid.

> -If the RAID controllers dies, you can move everything to another machine
> and know that the RAID setup will transfer.  Usually hardware RAID

Oh, I agree with that. You're talking about the proprietary formatting
in hw raid assemblies, I take it? Yah.

> -There is a transparency to having the disks directly attached to the OS

Agreed. "It's alright until it goes wrong".

> Disadvantages:
>
> -Maintenance like disk replacement rebuilds will be using up your main CPU

Agreed (above).

>
> -It's harder to setup a redundant boot volume with software RAID that

Yeah. I don't bother. A small boot volume in readonly mode with a copy
on another disk is what I use.

> works right with a typical PC BIOS.  If you use hardware RAID it tends to
> insulate you from the BIOS quirks.

Until the machine dies? (and fries a disk or two on the way down ..
happens, has happend to me).

> -If a disk fails, I've found a full hardware RAID setup is less likely to
> result in an OS crash than a software RAID is.  The same transparency and

Not sure.

> >> However, not all hardware RAID will have such a battery-backed-up cache,
> >> and those that do tend to have a hefty price tag.
> >
> > Whereas software raid and a firewire-attached log device does not.
>
> A firewire-attached log device is an extremely bad idea.  First off,
> you're at the mercy of the firewire bridge's write guarantees, which may
> or may not be sensible.

The log is sync. Therefore it doesn't matter what the guarantees are,
or at least I assume you are worrying about acks coming back before the
write has been sent, etc.  Only an actual net write will be acked by the
firewire transport as far as I know.  If OTOH you are thinking of "a
firewire attached disk" as a complete black box, then yes, I agree, you
are at the mercy of the driver writer for that black box.  But I was not
thinking of that.  I was only choosing firewire as a transport because
of its relatively good behaviour with small requests, as opposed to GE
as a transport, or 100BT as a transport, or whatever else as a
transport...


> It's not hard to find reports of people whose
> disks were corrupted when the disk was accidentally disconnected, or of
> buggy drive controller firmware causing problems.  I stopped using
> Firewire years ago because it seems you need to do some serious QA to
> figure out which combinations are reliable and which aren't, and I don't
> use external disks enough to spend that kind of time with them.

Sync operation of the disk should make you immune to any quirks, even
if you are thinking of "firewire plus disk" as a black-box unit.

> Second, there's few if any Firewire setups where the host gets to read
> SMART error data from the disk.

An interesting point, but I really was considering firewire only as the
transport (I'm the author of the ENBD - enhanced network block device -
driver, which makes any remote block device available over any
transport, so I guess that accounts for the different assumption).

Peter