Re: Postgres over Linux NBD or NFS - Mailing list pgsql-performance

In an attempt to throw the authorities off his trail, ronz@ravensfield.com (Andrew Rawnsley) transmitted:
> On Jun 21, 2004, at 2:02 PM, Andrew Hammond wrote:
>> We're looking for an alternative to fiber-channel disk arrays for mass
>> storage. One of the ideas that we're exploring would involve having the
>> cluster on an NFS mounted filesystem. Another technology we're looking
>> at is the Linux NBD (Network Block Device).
>
> No idea about NBDs, but its generally accepted that running over NFS
> would significantly decrease reliability and performance, i.e. it
> would be a Bad Move (tm). Not sure what you think to gain. I sure
> wouldn't trust NFS with a production database.
>
> What exactly are you trying to gain, avoid, or do?

The point of the exercise is to try to come up with something that is
a near-substitute for a SAN.

With a SAN, you have a box with a whole lot of disk in it, and then
your database servers connect to that box, typically via something
like fibrechannel.

One of the goals is for this to allow trying out Opterons at low risk.
Should performance turn out to suck or there be some other
disqualification, it's simple to hook the disk up to something else
instead.

The other goal is to be able to stick LOTS of disk into one box, and
dole it out to multiple servers.  It's more expensive to set up and
manage 3 RAID arrays than it is to set up and manage just 1, because
you have to manage 3 sets of disk hardware rather than 1.

But I'm getting convinced that the attempt to get this clever about it
is counterproductive unless you have outrageous amounts of money to
throw at it.

- NFS may well be acceptable if you buy into something with potent FS
  semantics, as with NetApp boxes.  But they're REALLY expensive.

- FibreChannel offers interesting options in conjunction with a fairly
  smart SAN box and Veritas, where you could have 5TB of storage in
  one box, and then assign 2TB apiece to two servers, and the other
  1TB to a third.  But the pricing premium again leaps out at ya.

The "poor man's approach" involves trying to fake this by building a
"disk box" running Linux that exports the storage either as a
filesystem (using NFS) or as disk blocks (NBD).  NFS clearly doesn't
provide the filesystem semantics needed to get decent reliability;
with NBD, it's not clear what happens :-(.

Barring that, it means building a separate RAID array for each server,
and living with the limitation that a full set of disk hardware has to
be devoted to each DB server.
--
wm(X,Y):-write(X),write('@'),write(Y). wm('cbbrowne','acm.org').
http://www3.sympatico.ca/cbbrowne/
Rules of the  Evil Overlord #46. "If an advisor says  to me "My liege,
he  is but  one man.  What can  one man  possibly do?",  I  will reply
"This." and kill the advisor." <http://www.eviloverlord.com/>