Migrating /etc Flat File Databases to LDAP

Authentication and Name Services, Debian 3.1 (Sarge)

i. Install the LDAP migration tools

Do 'apt-get install migrationtools' (at least for 'Sarge'). This will install a collection of scripts in /usr/share/migrationtools

ii. Edit migrationtools configuration file

Edit /etc/migrationtools/migrate_common.ph by changing the lines show below. The DEFAULT_ changes are customisations for your site, while the UID/GID lines ignore system users (that is those users created and modified by debian package scripts), and the nobody user and group (65534:65534). You can find your system settings by looking in /etc/adduser.conf.

Skeleton

 # Default DNS domain
 $DEFAULT_MAIL_DOMAIN = "your.domain";
 # Default base
 $DEFAULT_BASE = "[["BaseDN"]]";
 ...
 # Uncomment these to exclude Debian-managed system users and groups
 $IGNORE_UID_BELOW = 1000;
 # Don't uncomment this if you want to be able to add users to system groups
 # $IGNORE_GID_BELOW = 1000;
 # And here's the opposite for completeness
 $IGNORE_UID_ABOVE = 29999;
 $IGNORE_GID_ABOVE = 29999;

Example

 # Default DNS domain
 $DEFAULT_MAIL_DOMAIN = "example.com";
 # Default base
 $DEFAULT_BASE = "dc=example,dc=com";
 ...
 # Uncomment these to exclude Debian-managed system users and groups
 $IGNORE_UID_BELOW = 1000;
 # Don't uncomment this if you want to be able to add users to system groups
 # $IGNORE_GID_BELOW = 1000;
 # And here's the opposite for completeness
 $IGNORE_UID_ABOVE = 29999;
 $IGNORE_GID_ABOVE = 29999;

For Samba LDAP Users

The default subtrees used for user and group information are not compatible with the ?smbldap-tools package which is recommended when using LDAP for Samba authentication and mapping. For that reason, if you are using Samba with LDAP you should make the following additional changes to /etc/migrationtools/migrate_common.ph.

 $NAMINGCONTEXT{'passwd'}            = "ou=Users";
 $NAMINGCONTEXT{'group'}             = "ou=Groups";

Note: smbldap-tools can be configured to use the migrationtool naming context defaults. Many things default to using the migrationtool naming context, such as pam_ldap and libnss_ldap. IMHO it is easier to change the smbldap-tools config than change everything else to conform to it. Note the "ldapscripts" package is an alternative to smbldap-tools that defaults to the migrationtool naming context.

Optional: Use different subtrees based on function

If you are doing more than LDAP Authentication with your LDAP server you may wish to divide the various functions of the LDAP server into different subtrees. This can also be important if you are using different LDAP servers for different functions while still having the tree look like it is coming from a single source (it can be done but is not discussed here).

In my examples, I have 'ou=dns,[["BaseDN"]] for the DNS server, 'ou=auth,[["BaseDN"]]' for users and groups (authentication/authorization), 'ou=mail,[["BaseDN"]]' for email related information, 'ou=syscfg,[["BaseDN"]]' for system configuration information (like /etc/fstab), and 'ou=net,[["BaseDN"]]' for networking configuration info handled by NSS.

The following assumes you also need to make the changes above for ?smbldap-tools.

 } else {
         $NAMINGCONTEXT{'aliases'}           = "ou=Aliases,ou=mail";
         $NAMINGCONTEXT{'fstab'}             = "ou=Mounts,ou=syscfg";
         $NAMINGCONTEXT{'passwd'}            = "ou=Users,ou=auth";
         $NAMINGCONTEXT{'netgroup_byuser'}   = "nisMapName=netgroup.byuser,ou=auth";
         $NAMINGCONTEXT{'netgroup_byhost'}   = "nisMapName=netgroup.byhost,ou=auth";
         $NAMINGCONTEXT{'group'}             = "ou=Groups,ou=auth";
         $NAMINGCONTEXT{'netgroup'}          = "ou=Netgroup,ou=auth";
         $NAMINGCONTEXT{'hosts'}             = "ou=Hosts,ou=net";
         $NAMINGCONTEXT{'networks'}          = "ou=Networks,ou=net";
         $NAMINGCONTEXT{'protocols'}         = "ou=Protocols,ou=net";
         $NAMINGCONTEXT{'rpc'}               = "ou=Rpc,ou=net";
         $NAMINGCONTEXT{'services'}          = "ou=Services,ou=net";
 }

You will also need to makes the following changes to the  sub ldif_entry function in the same file /etc/migrationtools/migrate_common.ph:

 sub ldif_entry
 {
 # remove leading, trailing whitespace
         local ($HANDLE, $lhs, $rhs) = @_;
         local ($type, $val) = split(/\=/, $lhs);
         local ($dn);
         local (@newval);
         if ($val =~ /\,/) {
                 @newval = split(/\,/, $val);
                 $val = $newval["0"];
         }

Note on EXTENDED_SCHEMA = 0

Apparently EXTENDED_SCHEMA is set to '1' in many other documents. This probably will not work without modification under Debian 3.1 'Sarge'. I haven't tried going all the way, however I have looked at the ldif that would be used and appears the following note applies.

Note: Since Debian Lenny there is a kerberos.schema within 'krb5-kdc-ldap' (although the package 'heimdal-kdc' contains hdb.schema, which you may investigate using as an alternative to kerberos.schema. WFM), so one must manually edit passwd.ldif to remove the two lines refering to kerberos for every user. That is, the following two lines:

 objectClass: kerberosSecurityObject
 krbName: user@YOUR.DOMAIN

Where user@YOUR.DOMAIN is the username with @YOUR.DOMAIN appended.

2007-03-22: note, EXTENDED_SCHEMA = 1 is useful for adding more fields like 'mail' to ldap People records.

iii. Perform the Migration

If you just want ?LDAPAuthentication you probably want option 3, migrating only the /etc/passwd and /etc/group databases (but first using migrate_base.pl).

Notes

Skeleton

 # node, [["BaseDN"]]
 dn: ou=node,[["BaseDN"]]
 objectClass: top
 objectClass: organizationalUnit
 objectClass: domainRelatedObject
 associatedDomain: your.domain
 ou: node

Example

 # auth, example, com
 dn: ou=auth,dc=example,dc=com
 objectClass: top
 objectClass: organizationalUnit
 objectClass: domainRelatedObject
 associatedDomain: example.com
 ou: auth

In all cases (following Options 1..3) you will need to migrate some base settings

  1. Execute './migrate_base.pl >base.ldif'

  2. Execute 'ldapadd -h localhost -x -W -D "cn=admin,{["BaseDN"]}" -c -f base.ldif'

  3. Example commands:
     ./migrate_base.pl >base.ldif
     ldapadd -h localhost -x -W -D "cn=admin,dc=example,dc=com" -c -f base.ldif

Hint: You can migrate in a single step

You can pipe the output of migrate into ldapadd instead of redirecting to a file and using ldapadd -f filename. For example,

 ./migrate_base.pl >base.ldif
 ldapadd -h localhost -x -W -D "cn=admin,dc=example,dc=com" -c -f base.ldif

would become

 ./migrate_base.pl | ldapadd -h localhost -x -W -D "cn=admin,dc=example,dc=com" -c

Option 1: Just migrate everything

  1. Make sure the slapd service is started

  2. Confirm that /etc/ldap/ldap.conf is correctly configured.

  3. cd /usr/share/migrationtools

  4. Execute './migrate_all_online', answering the following questions: normally you can just accept the defaults

    1. Enter the X.500 naming context you wish to import into: [dc=your,dc=domain]

      1. Here you should enter the ?BaseDN for your LDAP tree.

    2. Enter the hostname of your LDAP server ["ldap":]

    3. Enter the manager DN: [cn=admin,[["BaseDN"]]:]

    4. (e.g. cn=admin,dc=example,dc=com)

      1. And this should the ?LdapDn for the LDAP administrative user.

    5. Enter the credentials to bind with:

      1. This should be the password for the LDAP administrative user.

    6. Do you wish to generate a ["DUAConfigProfile" [yes||no]?]

      1. For most the safe answer is no as DUAConfigProfile depends on schema not included in stock Debian. The profile speeds up searching for the various flat file databases imported into LDAP in the instructions you are now reading.

      2. If you are using a single LDAP server you don't really need this.

      3. If you are using multiple servers for different subtrees this can help the resolution process by directing the query to the appropriate server. Describing how that works is beyond the scope of this document.
  5. If it worked, great, otherwise you may need to generate an LDIF and modify as described below.

Option 2: Migrate everything by way of a single LDIF

As Option 1, above, except,

 ./migrate_all_online.sh: line 196: badword: command not found
 badword : returned non-zero exit status: saving failed LDIF to /tmp/nis.5746.ldif

Synchronize the password encoding

From the original LDAPMigrationTools page

Make sure the password encoding as stored in the LDAP is something glibc can understand. I'm using '{md5}', but you can use '{crypt}'. Anything else will not work.

The migration script doesn't appear to detect the coding; if it's reading from a shadow file it simply tacks '(crypt}' on ahead of the encrypted password it found. This can be a problem if your system has actually stored the passwords in MD5 format. I don't know how to get around this; editing the migrate_passwd.pl script or creating and then editing an ldif (see below) might work. -Rj

New

I started fresh with LDAP so I've never had to convert old /etc/passwd users, however you may be interested in the notes on LDAP/PAM which provides a way to convert passwords to md5 format when the password is changed (so root could elect to change all the passwords, thus getting md5 on all passwords). DanielDickinson

Option 3: Migrate by way of an LDIF for each /etc database you want

Migrateable Settings and How To Import Them

I don't use the automounter daemon so I don't know what is needed for this DD

I don't use netgroups so I don't know what is needed to migrate netgroup_byhost and netgroup_byuser DD

iv. Verify Data Import

Execute

 ldapsearch -x -h localhost

This should return everything in your LDAP tree except hashed passwords.

If you want to see the hashed passwords as well, try

 ldapsearch -x -W -D "cn=admin,[["BaseDN"]]" -h localhost

E.g:

 ldapsearch -x -W -D "cn=admin,dc=example,dc=com" -h localhost

v. Configure System to Use LDAP

See: LDAP/NSS, and LDAP/PAM.



Old: Configuring LDAP Authentication

Debian 3.0 (Woody), LDAPMigrationTools

Importing Data

1. Install the LDAP migration tools

Do 'apt-get install migrationtools' (for stable (a.k.a. 'woody'), although I would not be surprised if this package gets renamed).

2. Edit a migration tool configuration file

Edit /etc/migrationtools/migratecommon.ph by changing the following lines.

Wherever you see the line 'objectClass = account', comment it out with a '#'.

  # Default DNS domain
  $DEFAULT_MAIL_DOMAIN = "<your.domain>";
  #
  # Default base
  $DEFAULT_BASE = "dc=<your>,dc=<domain>";
  #
  # turn this on to support more general object classes
  # such as person.
  $EXTENDED_SCHEMA = 0;
  #
  # Uncomment these to avoid Debian-managed system users and groups
  $IGNORE_UID_BELOW = 1000;
  $IGNORE_GID_BELOW = 100;


NOTE

Note that EXTENDED_SCHEMA is set to '1' in many other documents.

I found that this caused problems for me as I don't know if Debian includes all the additional schema definitions in /etc/ldap/schema/.

This problem has to do with the version of slapd that you are running. If you are running slapd 2.0.x then you may use EXTENDED_SCHEMA=1 freely. If you are using slapd 2.1.x then you can set the value to '1', but you must make a small change to the ldif file.

(Debian doesn't include a kerberos.schema (although the package 'heimdal-kdc' contains hdb.schema, which you may investigate using as an alternative to kerberos.schema. WFM), so one must manually edit passwd.ldif to remove the two lines refering to kerberos. slapd 2.1 also has a problem with having both account and inetOrgPeson as object classes. I believe inetOrgPerson is a superset of account, and removed account, and was able to import the data. TRL)


3. Import the data

Do you want to import everything, or are you just wanting to do user/password lookups? If you are only doing this as an NIS replacement, you can probably skip through to step 5 or 6 below.

To import all the data into LDAP, do the following as 'root' and answering questions as required. Note that if you don't do this as root, the shadow passwords will not be imported, and you will have all sorts of strange problems: see 6 to adjust migrate_groups (do this before you do migrate_all)

  # cd /usr/share/migrationtools
  # LDAPADD="/usr/bin/ldapadd -c" ./migrate_all_online.sh

4. Verify the data was imported

Check that the data is in there using 'ldapsearch' as above under Tools.

5. Synchronize the password encoding

Make sure the password encoding as stored in the LDAP is something glibc can understand. I'm using '{["MD5"}]', but you can use '{CRYPT}'. Anything else will not work.

The migration script doesn't appear to detect the coding; if it's reading from a shadow file it simply tacks '(CRYPT}' on ahead of the encrypted password it found. This can be a problem if your system has actually stored the passwords in MD5 format. I don't know how to get around this; editing the migrate_passwd.pl script or creating and then editing an ldif (see below) might work. -Rj

6. If you do not want to move all of your data

If you don't want to do it all (migrate_all_online.sh also moves a lot of IP-related stuff into the LDAP tree ... (yep this makes yor system a whole lot slower).

First we need to fix a bug in 'migrate_group' There is a bug in the migrationtools package (version 40-1) 'migrate_group' script around lines 55 and 56.

Change them into:

 next if (int($IGNORE_GID_BELOW) and int($gid) < int ($IGNORE_GID_BELOW));

 next if (int($IGNORE_GID_ABOVE) and int($gid) > int($IGNORE_GID_ABOVE));

Do not forget to change 'int($uid)' to 'int($gid)' on both lines...

Move all the data in three steps:

Step 1:

 # cd /usr/share/migrationtools

Step 2:

 # ./migrate_base.pl > base.ldif
 # ./migrate_group.pl /etc/group > group.ldif
 # ./migrate_passwd.pl /etc/passwd >passwd.ldif

Step 3:

 # ldapadd -x -W -h localhost -D "cn=admin,dc=<your>,dc=<domain>" -c -f base.ldif
 # ldapadd -x -W -h localhost -D "cn=admin,dc=<your>,dc=<domain>" -c -f group.ldif
 # ldapadd -x -W -h localhost -D "cn=admin,dc=<your>,dc=<domain>" -c -f passwd.ldif

And you are done.

During the import of the base.ldif files you get two (2) warnings -- ignore them. The Debian package creates 'base.ldif' with 'ou=People' already populated.

7. Verify things are running correctly

Be sure to set your defaults in /etc/ldap/ldap.conf to point at your LDAP server, then run ldapsearch and see if it returns anything:

 # ldapsearch -x -D'dc=<your>,dc=<domain>'

03-12-26: --- This search line did not work for me, but this one does:

 ldapsearch -b "ou=People,dc=<your>,dc=<domain>" -LLL -D "cn=admin,dc=<your>,dc=<domain>" \
 -H "ldap://127.0.0.1:389/" -W -x "(uid=loginnameoftestuser)"

Enter det password for the cn=admin,dc=<your>,dc=<domain> when prompted. Then you should get a few lines if account info. If you return to prompt directly the uid was not found.

04-01-06: The original search line didn't work for me either but simply appending -x does make it work. I don't know what the implications of this are yet.

2004-07-28: The -x switches to "simple authentication" instead of "SASL" -- Balu

04-04-11

  # ldapsearch -x

This works for me. It shows an unauthenticated view of the database. I.E. it doesn't show password lines. This works only if one has edited ldap.conf correctly. TRL