Translation(s): none


This page describes how to achieve persistent disk names according to disk bay position, generally - according to physical device path. It is tested on Debian Lenny but it should work on Debian Etch as well

The problem

When using Linux SW RAID made of several SATA hotswap disks, I have found a big problem: when a disk in the array is physically removed and then inserted again, the kernel does not assign it the same name, but a first free one.

Example:

I had a RAID made of 2 disks – sda and sdb.

When I physically remove the sda disk (without first executing mdadm –fail /dev/md* /dev/sda*; mdadm –remove /dev/md* /dev/sda*) and then insert this disk back, the disk never appears as /dev/sda. Instead, it is named as /dev/sdc. It seems that the kernel uses first free drive letter, because it thinks that /dev/sda is still used.

Another problem:

If /dev/sda fails completely such a way, that the server will not detect it. Then, if the server is rebooted, the second physical disk, named previously /dev/sdb, will become /dev/sda! This is very confusing, because how will you know which physical disk should be replaced?

The behaviour I would like to see is: disk in disk bay 1 will be named /dev/sda disk in disk bay 2 will be named /dev/sdb – even if there is no disk in disk bay 1 etc.

Then, if /dev/sda fails and the server is restarted, the /dev/sda will be missing and the second disk is still /dev/sdb. And I am able to tell the operator „please go and exchange the disk in disk bay 1“

How to do this?

After 4 days of trial&error experiments, I have developed this solution:

The solution

Although it may sound simple, Google have not found any solution for this problem. First of all, you have to find the UDEV physical device path of your disks. Disk bays are connected to the mainboard in a fixed way, which corresponds to the UDEV physical device path.

Now the fun begins: the udev developers probably did not suppose anyone would need physical device path, so they are changing the location of this information quite frequently.

* Historically, physical device path was stored in ENV{PHYSDEVPATH} udev key. This was deprecated and is no longer supported by newest kernels. But in Lenny, I was forced to use this old key, because all the other keys with physical device path (e.g., ENV{ID_PATH}) did not work for me * In Squeeze kernel (2.6.30), physical device path is available just as DEVPATH

Therefore, the resulting udev rules are different for each kernel version:

Squeeze (2.6.30)

For each disk, the path is printed by udevadm command. Example for /dev/sda and /dev/sda2:

 udevadm info --query=path --name=/dev/sda
 /devices/pci0000:00/0000:00:1f.2/host0/target0:0:0/0:0:0:0/block/sda

For discovering the fixed part of the path, check the output for the other disks in your system. E.g., this is a USB stick put in the USB port 1:

 udevadm info --query=path --name=/dev/sdb
 /devices/pci0000:00/0000:00:1d.7/usb1/1-1/1-1:1.0/host10/target10:0:0/10:0:0:0/block/sdb

And this is the same USB stick in USB port 2:

 udevadm info --query=path --name=/dev/sdb
 /devices/pci0000:00/0000:00:1d.7/usb1/1-2/1-2:1.0/host9/target9:0:0/9:0:0:0/block/sdb

Now let's say we want to name any disk inserted into USB port 1 as /dev/sdu. So we create a file (e.g.) /etc/udev/rules.d/20-disk-bays.rules with this content:

KERNEL=="sd?", SUBSYSTEM=="block", DEVPATH=="*usb1/1-1*", NAME="sdu", RUN+="/usr/bin/logger My disk ATTR{partition}=$ATTR{partition}, DEVPATH=$devpath, ID_PATH=$ENV{ID_PATH}, ID_SERIAL=$ENV{ID_SERIAL}", GOTO="END_20_PERSISTENT_DISK" 

KERNEL=="sd?*", ATTR{partition}=="1", SUBSYSTEM=="block", DEVPATH=="*usb1/1-1*", NAME="sdu%n" RUN+="/usr/bin/logger My partition parent=%p number=%n, ATTR{partition}=$ATTR{partition}" 
LABEL="END_20_PERSISTENT_DISK"

Lenny (2.6.26)

KERNEL=="sd?", SUBSYSTEM=="block", ENV{PHYSDEVPATH}=="*1f.2/host0/target0:0:0/0:0:0:0*", NAME="sda", RUN+="/usr/bin/logger Muj disk ATTR{partition}=$ATTR{partition}, DEVPATH=$devpath, ID_PATH=$ENV{ID_PATH}, ID_SERIAL=$ENV{ID_SERIAL}", GOTO="END_10_DISK_BAY" 

KERNEL=="sd?*", ENV{DEVTYPE}=="partition", SUBSYSTEM=="block", ENV{PHYSDEVPATH}=="*1f.2/host0/target0:0:0/0:0:0:0*", NAME="sda%n" RUN+="/usr/bin/logger Moje partition parent=%p number=%n, ATTR{partition}=$ATTR{partition}" 
LABEL="END_10_DISK_BAY"

Testing the disk failures

To test the result of a complete disk failure, you can use this script:

#place this script to /usr/local/bin/stop-disk
if [ "" = "$1" ]; then
        echo "Usage: `basename $0` device"
else
        #extract the SCSI ID numbers from the output of lsscsi:
        read -d ] A B C D < <(IFS=':'; echo $(lsscsi | grep $1))
        #remove the "[" from begining of A:
        A=${A##*[}   #quicker version: A=${A:1}
        #stop the disk spinning
        sg_start -i -v --stop $1
        echo "Host adapter ID=$A, SCSI channel=$B, ID=$C, LUN=$D"
        #and remove it from the scsi bus
        echo "scsi remove-single-device $A $B $C $D" > /proc/scsi/scsi
fi

If you name this script /usr/local/bin/stop-disk, then you can stop a disk (e.g., sda) by issuing the command  stop-disk /dev/sda .

For spinning down the disk, this script uses the sg_start command, which is part of sg3-utils, if you don't have it installed, then  apt-get install sg3-utils  will do the job.

Now you can start the disk again using another script:

if [ "" = "$1" ]; then
        echo "Usage: `basename $0` HostAdapterID Channel ID LUN"
else
        echo "Host adapter ID=$1, SCSI channel=$2, ID=$3, LUN=$4"
        echo "scsi add-single-device $1 $2 $3 $4" > /proc/scsi/scsi
        #sg_start -i -v --start $1  
fi

See also