Differences between revisions 1 and 8 (spanning 7 versions)
Revision 1 as of 2007-04-05 16:17:54
Size: 1711
Editor: ?Eduard Bloch
Comment:
Revision 8 as of 2007-04-05 17:59:01
Size: 7554
Editor: ?Eduard Bloch
Comment:
Deletions are marked like this. Additions are marked like this.
Line 1: Line 1:
= On Locking schemes on device files and device drivers = = On Locking Schemes on Linux Device Drivers =
Line 17: Line 17:
either willingly (e.g. liblkid) or unwillingly (e.g. hald, opening with either willingly (e.g. liblkid) or accidentally (e.g. hald, opening with
Line 27: Line 27:
State of the practice
---------------------
== State of the practice ==
Line 30: Line 29:
Currently, following mechanisms can be considered: There are various locking techniques used in other areas which are more or less applicable in our case.
Line 32: Line 31:
1: O_EXCL locking === General inter-process locking mechanisms ===

In general, all the mechanisms listed below are not optimally appropriate for our purpose. They lack on two places which make then not reliable when used alone:

 * they do not cope with multiple device file which imply the access to the same driver through different files
 * they do not automatically cope with multiple device '''drivers''' accessible through '''different''' user space interfaces, like with sg vs. sr drivers on Linux. No matter how many excuses some kernel developers do present to paper over this obvious shortcomings. Automatic use of /dev/sr instead of /dev/sg is not always possible or may not be wanted by the user.

Finally, they may be sufficient to lower the risk on inappropriate operation. Which exactly are available in the wild?

 * Lock files associated with target file

 Principle: an additional file is created during the action on the real target file.

 Pros: regular filesystem operation, no additional infrastructure required
   
 Cons:
  * Possible races unless OS mechanisms are used for exclusive operation on the lock file, see below
  * The location and name of the lock file need to be known and discussed upfront among all application developers, or be documented excessively
  * Permission problems may disallow the creation of lock files (security issues), especially for self-compiled applications and having no root permissions to install them in a required way

 * fcntl(2) exclusive file locking

 Principle: lock applied on open file handles. Internally associated with a path, see fcntl(2) for details.
 
 Pros:

  * known (POSIX.1-2001), usually reliable mechanism

 Cons:

  * diverges from flock() implementation on Linux, see below. Results in independent locking.
  * possible problems on network file systems

 * flock(2) exclusive file locking

 Principle: similar to fcntl locks, applied with a different system function.

 Pros: see fcntl(2) locking above

 Cons: like flock(2), but less portable, not working over network file systems
  
==== Advanced Linux-specific locking mechanisms ====

 * O_EXCL locking
Line 36: Line 79:
   Pros:
    - reliable for a device accessible through one driver
   Cons:

 Pros:
  * reliable for a device accessible through one driver
 
 Cons:
  * requires kernel 2.6.x (x>=7 or so)
  * does not automagicaly make the device inaccessible, only applications using O_EXCL will know about the locked state when getting negative result with EBUSY errno value.

=== Applicability on CD/(HD)DVD/BD drives ===

As explained in the introduction, the locking is important on optical media recording due to the delicate operation mode during the recording. Ideally, no application should touch them, even reading from the media is an evil task. But how does the state of the practice look like?

 * mount: the block device is mounted with the O_EXCL flag '''BUT''' the mount executable also uses '''libblkid''' which opens the devices without locking and read magic data from it. This also provides no solution for operation through the sg driver.
 
 * hald (HAL daemon): periodically opens the cdrom block devices with O_EXCL flag. Clashes with operation on sg is possible.

 * wodim: opens the devices with O_EXCL flag. Opening /dev/sg is possible and happens more likely with versions prior to 1.1.4.

 * cdrskin: opens the devices with O_EXCL flag. Opening of /dev/sg is prevented by creation of bus/target/lun table and mapping the request to /dev/sr with it, even when dev=/dev/sgX is specified.

 * cdrecord: no locking. Author recommends to get rid of applications which may touch the device somehow.


=== Proposed general locking algorithm ===

The following method is proposed to create a midway between the limitations of kernel and the requirements of others, also unifying the way of dealing with the device locks.

The locking methods with additional lock files are identified as inappropriate because of inconsistent security settings on different applications, see above. Instead, we propose a two-step locking method:

 1. Open the device. For applications that operate in a delicate way (burning tools), O_EXCL shall be set. For others, it may be omited.
 2. Set or check the additional fcntl lock on the device file. It must be exclusive! Sample code (by Thomas Schmidt)

{{{
        struct flock lockthing;
        ...
        f = open(device, mode|O_EXCL);
        if (f != -1) {
             memset(&lockthing, 0, sizeof(lockthing));
             lockthing.l_type = F_WRLCK;
             lockthing.l_whence = SEEK_SET;
             lockthing.l_start = 0;
             lockthing.l_len = 0;
             if (fcntl(f, F_SETLK, &lockthing)) {
                close(f);
                /* user feedback, report error, etc... */
                f = -1;
             }
        }
}}}

 3. (For burning applications only)
 Since there is still the problem with Linux' inability to share locks across the device driver borders, we need to make sure that the access happens through a unique device file. For kernel 2.6, it is possible to retrieve physical address information from /dev/sgX which can be used to scan "/dev/sr%d" nodes quickly in this order unless the first node matching the physical address of /dev/sgX device in question is found. Thereafter this device file should be used for locking and further communication. Since no additional physical information is maintained by the kernel for /dev/hdX and maybe other types of devices, the locking shall happen on the device chosen by user. If some kind of fake SCSI hardware address emulation needs to be performed (for example for old cdrecord-using applications) then the scanning mechanism should always select the first appropriate device when scanning device files matching the "/dev/hd%c" pattern beginning "/dev/hda".

, we are simply out of luck. When access through the old ATAPI driver

On Locking Schemes on Linux Device Drivers

Hello fellow application developer or maintainer,

recently we (cdrkit and cdrskin developers) came accross increasing problems with reliable and safe device locking. This paper enlightens the issues behind the scenes and presents possible future solutions.

Introduction

Our original concern is the influence of even read-only operations on optical media drives (recorders) during their duty as recorders -- depending on the device model such read-only work may interrupt the process badly practically destroying the medium.

Since many programs already do act on such devices in an unsafe manner, either willingly (e.g. liblkid) or accidentally (e.g. hald, opening with O_EXCL but still clashing with cdr applications working on the competing sg driver), we see the need for reliable communication in order to ensure proper device locking where appropriate, in a way which is appropriate for the particular application. In the following document, first the currently possible mechanisms are itemized with their advantages and their problems, followed by a draft of a locking scheme which shall cope with the particular requirements and which may be implemented in a library shared by our applications later.

State of the practice

There are various locking techniques used in other areas which are more or less applicable in our case.

General inter-process locking mechanisms

In general, all the mechanisms listed below are not optimally appropriate for our purpose. They lack on two places which make then not reliable when used alone:

  • they do not cope with multiple device file which imply the access to the same driver through different files
  • they do not automatically cope with multiple device drivers accessible through different user space interfaces, like with sg vs. sr drivers on Linux. No matter how many excuses some kernel developers do present to paper over this obvious shortcomings. Automatic use of /dev/sr instead of /dev/sg is not always possible or may not be wanted by the user.

Finally, they may be sufficient to lower the risk on inappropriate operation. Which exactly are available in the wild?

  • Lock files associated with target file Principle: an additional file is created during the action on the real target file. Pros: regular filesystem operation, no additional infrastructure required Cons:
    • Possible races unless OS mechanisms are used for exclusive operation on the lock file, see below
    • The location and name of the lock file need to be known and discussed upfront among all application developers, or be documented excessively
    • Permission problems may disallow the creation of lock files (security issues), especially for self-compiled applications and having no root permissions to install them in a required way
  • fcntl(2) exclusive file locking Principle: lock applied on open file handles. Internally associated with a path, see fcntl(2) for details. Pros:
    • known (POSIX.1-2001), usually reliable mechanism
    Cons:
    • diverges from flock() implementation on Linux, see below. Results in independent locking.
    • possible problems on network file systems
  • flock(2) exclusive file locking Principle: similar to fcntl locks, applied with a different system function. Pros: see fcntl(2) locking above Cons: like flock(2), but less portable, not working over network file systems

Advanced Linux-specific locking mechanisms

  • O_EXCL locking
    • Principle: passing of the O_EXCL flag to the open call. The device is locked exclusively for the calling PID, the lock is maintained in the device driver to the particular major/minor combination.
    Pros:
    • reliable for a device accessible through one driver
    Cons:
    • requires kernel 2.6.x (x>=7 or so)

    • does not automagicaly make the device inaccessible, only applications using O_EXCL will know about the locked state when getting negative result with EBUSY errno value.

Applicability on CD/(HD)DVD/BD drives

As explained in the introduction, the locking is important on optical media recording due to the delicate operation mode during the recording. Ideally, no application should touch them, even reading from the media is an evil task. But how does the state of the practice look like?

  • mount: the block device is mounted with the O_EXCL flag BUT the mount executable also uses libblkid which opens the devices without locking and read magic data from it. This also provides no solution for operation through the sg driver.

  • hald (HAL daemon): periodically opens the cdrom block devices with O_EXCL flag. Clashes with operation on sg is possible.
  • wodim: opens the devices with O_EXCL flag. Opening /dev/sg is possible and happens more likely with versions prior to 1.1.4.
  • cdrskin: opens the devices with O_EXCL flag. Opening of /dev/sg is prevented by creation of bus/target/lun table and mapping the request to /dev/sr with it, even when dev=/dev/sgX is specified.
  • cdrecord: no locking. Author recommends to get rid of applications which may touch the device somehow.

Proposed general locking algorithm

The following method is proposed to create a midway between the limitations of kernel and the requirements of others, also unifying the way of dealing with the device locks.

The locking methods with additional lock files are identified as inappropriate because of inconsistent security settings on different applications, see above. Instead, we propose a two-step locking method:

  1. Open the device. For applications that operate in a delicate way (burning tools), O_EXCL shall be set. For others, it may be omited.
  2. Set or check the additional fcntl lock on the device file. It must be exclusive! Sample code (by Thomas Schmidt)

        struct flock lockthing;
        ...
        f = open(device, mode|O_EXCL);
        if (f != -1) {
             memset(&lockthing, 0, sizeof(lockthing));
             lockthing.l_type = F_WRLCK;
             lockthing.l_whence = SEEK_SET;
             lockthing.l_start = 0;
             lockthing.l_len = 0;
             if (fcntl(f, F_SETLK, &lockthing)) {
                close(f);
                /* user feedback, report error, etc... */
                f = -1;
             }
        }
  1. (For burning applications only) Since there is still the problem with Linux' inability to share locks across the device driver borders, we need to make sure that the access happens through a unique device file. For kernel 2.6, it is possible to retrieve physical address information from /dev/sgX which can be used to scan "/dev/sr%d" nodes quickly in this order unless the first node matching the physical address of /dev/sgX device in question is found. Thereafter this device file should be used for locking and further communication. Since no additional physical information is maintained by the kernel for /dev/hdX and maybe other types of devices, the locking shall happen on the device chosen by user. If some kind of fake SCSI hardware address emulation needs to be performed (for example for old cdrecord-using applications) then the scanning mechanism should always select the first appropriate device when scanning device files matching the "/dev/hd%c" pattern beginning "/dev/hda".

, we are simply out of luck. When access through the old ATAPI driver