SAN/NAS options - Mailing list pgsql-performance

Hello all,

It seems that I'm starting to outgrow our current Postgres setup.  We've
been running a handful of machines as standalone db servers.  This is all
in a colocation environment, so everything is stuffed into 1U Supermicro
boxes.  Our standard build looks like this:

Supermicro 1U w/SCA backplane and 4 bays
2x2.8 GHz Xeons
Adaptec 2015S "zero channel" RAID card
2 or 4 x 73GB Seagate 10K Ultra 320 drives (mirrored+striped)
2GB RAM
FreeBSD 4.11
PGSQL data from 5-10GB per box

Recently I started studying what we were running up against in our nightly
runs that do a ton of updates/inserts to prep things for the tasks the db
does during the business day (light mix of selects/inserts/updates).
While we have plenty of disk bandwidth (according to bonnie), we are
really dying on IOPS.  I'm guessing this is a mix of a rather anemic RAID
controller (ever notice how adaptec doesn't publish any real
performance specs on raid cards?) and having only two or four spindles
(effectively 1 or 2 on writes).

So that's where we are...

I'm new to the whole SAN thing, but did recently pick up a few used NetApp
shelves and a Fibre Channel RAID HBA (Mylex ExtremeRAID 3000, also used)
to toy with.  I started wondering if I could put something together to
both get our storage on one set of boxes and allow me to get data striped
across more drives.  Our budget is not huge and we are not adverse to
getting used gear where appropriate.

What do you folks recommend?  I'm just starting to look at what's out
there for SANs and NAS, and from what I've seen, our options are:

NetApp Filers - the pluses with these are that if we use NFS, we don't
have to worry about either large filesystem support in FreeBSD (2TB
practical limit), or limitation on "growing" partitions as the NetApp just
deals with that.  I also understand these make backups a bit simpler.  I
have a great, trusted, spare-stocking source for these.

Apple X-Serve RAID - well, it's pretty cheap.  Honestly, that's all I know
about it - they don't talk about IOPS numbers, and I have no idea what
lurks in that box as a RAID controller.

SAN box w/integrated RAID - it seems like this might not be a good choice
since the RAID hardware in the box may be where I hit any limits.  I also
imagine I'm probably overpaying for some OEM RAID controller integrated
into the box.  No idea where to look for used gear.

SAN box, JBOD - this seems like it might be affordable as well.  A few big
shelves full of drives a SAN "switch" to plug all the shelves and hosts
into and a FC RAID card in each host.  No idea where to look for used gear
here either.

You'll note that I'm being somewhat driven by my OS of choice, FreeBSD.
Unlike Solaris or other commercial offerings, there is no nice volume
management available.  While I'd love to keep managing a dozen or so
FreeBSD boxes, I could be persuaded to go to Solaris x86 if the volume
management really shines and Postgres performs well on it.

Lastly, one thing that I'm not yet finding in trying to educate myself on
SANs is a good overview of what's come out in the past few years that's
more affordable than the old big-iron stuff.  For example I saw some brief
info on this list's archives about the Dell/EMC offerings.  Anything else
in that vein to look at?

I hope this isn't too far off topic for this list.  Postgres is the
main application that I'm looking to accomodate.  Anything else I can do
with whatever solution we find is just gravy...

Thanks!

Charles