Re: SAN/NAS options - Mailing list pgsql-performance

Charles Sprickman wrote:
> Hello all,
>
> It seems that I'm starting to outgrow our current Postgres setup.  We've
> been running a handful of machines as standalone db servers.  This is
> all in a colocation environment, so everything is stuffed into 1U
> Supermicro boxes.  Our standard build looks like this:
>
> Supermicro 1U w/SCA backplane and 4 bays
> 2x2.8 GHz Xeons
> Adaptec 2015S "zero channel" RAID card
> 2 or 4 x 73GB Seagate 10K Ultra 320 drives (mirrored+striped)
> 2GB RAM
> FreeBSD 4.11
> PGSQL data from 5-10GB per box
>
> Recently I started studying what we were running up against in our
> nightly runs that do a ton of updates/inserts to prep things for the
> tasks the db does during the business day (light mix of
> selects/inserts/updates). While we have plenty of disk bandwidth
> (according to bonnie), we are really dying on IOPS. I'm guessing this is
> a mix of a rather anemic RAID controller (ever notice how adaptec
> doesn't publish any real performance specs on raid cards?) and having
> only two or four spindles (effectively 1 or 2 on writes).
>
> So that's where we are...
>
> I'm new to the whole SAN thing, but did recently pick up a few used
> NetApp shelves and a Fibre Channel RAID HBA (Mylex ExtremeRAID 3000,
> also used) to toy with.  I started wondering if I could put something
> together to both get our storage on one set of boxes and allow me to get
> data striped across more drives.  Our budget is not huge and we are not
> adverse to getting used gear where appropriate.
>
> What do you folks recommend?  I'm just starting to look at what's out
> there for SANs and NAS, and from what I've seen, our options are:
>

Leaving the whole SAN issue for a moment:

It would be interesting to see if moving to FreeBSD 6.0 would help you -
the vfs layer is no longer throttled by the (SMP) GIANT lock in this
version, and that may make quite a difference (given you have SMP boxes).

Another interesting thing to try is rebuilding the database ufs
filesystem(s) with 32K blocks and 4K frags (as opposed to 8K/1K or
16K/2K - can't recall the default on 4.x). I found this to give a factor
of 2 speedup on random disk access (specifically queries doing indexed
joins).

Is it mainly your 2 disk machines that are IOPS bound? if so, a cheap
option may be to buy 2 more cheetahs for them! If it's the 4's, well how
about a 2U U320 diskpack from whomever supplies you the Supermicro boxes?

I have just noticed Luke's posting - I would second the advice to avoid
SAN - in my experience it's an expensive way to buy storage.

best wishes

Mark