Thread: Linux hard drive/device nodes for a Postgres RAID array

Hi all, I apologize for asking a Linux question here in the postgres
list, but I figure somebody here must know the answer, or a better place
to ask it.

I am building an x86_64 postgres server (see, it isn't JUST about Linux
:-) with SATA drives and I want to use software RAID for my postgres
storage.

When Linux boots, it finds all the drives in the system and assigns
device numbers to them, apparently in the order found.  So, the first
drive (on SATA port 0 for example) become SDA, the second becomes SDB,
and so on.  If you have say, four drives, on ports 0,1,2,3, everything
works fine and you get SDA,SDB,SDC,SDD.

But now, pull the drive from port 2 and boot the system.  You will now
have SDA,SDB,SDC.  The kernel will now fail BOTH of the last two drives
from the RAID array.  The one that was SDC is gone, and obviously fails.
  The one that was SDD is now SDC, so its ID doesn't match what the
kernel thought it should be, so it fails it too.  If you kill the FIRST
drive in the array, I believe the entire array becomes inoperable
because of the resulting shift and ID mismatch.

So the question is, is there some way to "pin" a drive to a device
mapping?  In other words, is there a way to force the drive on port 0 to
always be SDA, and the drive on port 2 to always be SDC, even if the
drive on port 1 fails or is pulled?

-Glen

On Thu, Nov 16, 2006 at 12:40:41PM -0800, Glen Parker wrote:
> But now, pull the drive from port 2 and boot the system.  You will now
> have SDA,SDB,SDC.  The kernel will now fail BOTH of the last two drives
> from the RAID array.  The one that was SDC is gone, and obviously fails.
>  The one that was SDD is now SDC, so its ID doesn't match what the
> kernel thought it should be, so it fails it too.  If you kill the FIRST
> drive in the array, I believe the entire array becomes inoperable
> because of the resulting shift and ID mismatch.

Is that really so? AIUI the position of the disk in the array is stored
on the disk itself, so it should be able to handle disks moving around
no problem, have you tried it?

> So the question is, is there some way to "pin" a drive to a device
> mapping?  In other words, is there a way to force the drive on port 0 to
> always be SDA, and the drive on port 2 to always be SDC, even if the
> drive on port 1 fails or is pulled?

I thought you could do this with options on the command-line, or using
udev. But I don't think it's actually necessary.

Have a nice day,
--
Martijn van Oosterhout   <kleptog@svana.org>   http://svana.org/kleptog/
> From each according to his ability. To each according to his ability to litigate.

On Thu, 2006-11-16 at 14:56, Martijn van Oosterhout wrote:
> On Thu, Nov 16, 2006 at 12:40:41PM -0800, Glen Parker wrote:
> > But now, pull the drive from port 2 and boot the system.  You will now
> > have SDA,SDB,SDC.  The kernel will now fail BOTH of the last two drives
> > from the RAID array.  The one that was SDC is gone, and obviously fails.
> >  The one that was SDD is now SDC, so its ID doesn't match what the
> > kernel thought it should be, so it fails it too.  If you kill the FIRST
> > drive in the array, I believe the entire array becomes inoperable
> > because of the resulting shift and ID mismatch.
>
> Is that really so? AIUI the position of the disk in the array is stored
> on the disk itself, so it should be able to handle disks moving around
> no problem, have you tried it?

Just FYI, I've tried this before.  yes, linux software RAID, knowing
that the linux scsi numbering system is non-deterministic, is designed
to handle this.

In fact, you can build a RAID5 or RAID0 array of as many disks as you
like, shut down the machine, change every single drive ID, and the
machine will still find the RAID arrays.

Last I tested this was on something like RH 7.2 by the way.  Times may
have changed, but I can't imagine someone being stupid enough to break
the RAID array handling that worked so well back then.

>
> > So the question is, is there some way to "pin" a drive to a device
> > mapping?  In other words, is there a way to force the drive on port 0 to
> > always be SDA, and the drive on port 2 to always be SDC, even if the
> > drive on port 1 fails or is pulled?
>
> I thought you could do this with options on the command-line, or using
> udev. But I don't think it's actually necessary.

You can, it's generally not.