Running the LSB test suites on Debian

These instructions are for the LSB 3.x test suites, but need some editing.

The official LSB test suites are available from http://www.linuxbase.org/download/

The tests are not packaged for Debian. These are some issues in the upstream source. While these could be cleaned up and packaged and available in the archive, it hasn't been a priority yet. For now the recommended method is to use the official rpms. The official rpms are used for the certification program so Debian needs to be able to support testing with them anyway.

In order to run the official test suite rpms on Debian use the alien(1) tool. Since the lsb-runtime-test rpm uses package scripts in order to create users and groups for testing, YOU NEED TO USE THE "-c" OPTION so that the install scripts get converted too. Also there was a change in coreutils that trigger a bug in alien that caused setuid files to not have their setuid'ness properly restored when alien converted the package. So when you convert the official rpms, MAKE SURE TO USE ALIEN 8.39 OR NEWER.

Make sure to install the lsb package before testing too. If you're going to test an older Debian branch you may want to grab backports from backports.org or backport newer version from unstable yourself.

lsb-runtime-test

To run the LSB test suite,

  1. grab the right lsb-runtime-test rpm for your architecture
  2. as root, run alien -c lsb-runtime-test* on it

  3. as root, install the resulting deb
  4. as root, set the passwd for the test user vsx0
  5. ssh into the test system as the test user ssh testsystem -l vsx0

  6. run the tests with ./run_tests

  7. answer the questions and the tests begin
  8. wait many hours (if a single test has been running for more than 4 hours, it is probably hung, ctrl-c it to continue the rest of the tests)
  9. once the tests finish, copy the report and journal files from the "results" directory to some place safe
  10. use the /home/tet/bin/tjreport tool on the journal to generate a nice summary report
  11. put your results on the LsbTestResults page or send them to Matt Taggart to post and then send a pointer to the debian-lsb list

Running individual test(s) To run a single test

To run multiple tests

lsb-libchk

To run the the lsb-libchk test

  1. grab the right lsb-libchk rpm for your architecture
  2. as root, run alien -c lsb-libchk* on it

  3. as root, install the resulting deb
  4. in a scratch directory run /opt/lsb/bin/lsblibchk, a journal file will be produced
  5. use the tjreport tool from lsb-runtime-test package to generate a tjreport from the journal
  6. put your results on a webpage or send them to Matt Taggart to post and then send a pointer to the debian-lsb list

lsb-test-pam

To run the the lsb-test-pam test

  1. grab the right lsb-test-pam rpm for your architecture
  2. as root, run alien -c lsb-test-pam* on it

  3. as root, install the resulting deb, it conflicts with lsb-runtime-test so you might need to remove that
  4. as root, set the passwd for the test users vsx0-vsx2
  5. ssh in as vsx0
  6. Start the tests with ./run_tests, answer the questions, the test run

  7. a journal and report will be generated in the results/ directory, save them
  8. use the tjreport tool from lsb-runtime-test package to generate a tjreport from the journal
  9. put your results on a webpage or send them to Matt Taggart to post and then send a pointer to the debian-lsb list

qmtest_libstdcpp

To run the the qmtest_libstdcpp tests

  1. grab the right qmtest_libstdcpp and qm rpms for your architecture
  2. grab the LSB appbat python package from the same site
  3. run alien -c on them

  4. install the resulting debs
  5. TEMPORARY WORKAROUND: the tests require certain locales to be installed. Eventually the test suite is supposed to provide them, but for now you can run dpkg-reconfigure -plow locales and then select something like the following:

    • de_DE@euro ISO-8859-15, de_DE ISO-8859-1, de_DE.UTF-8@euro UTF-8, de_DE.UTF-8 UTF-8

    • en_HK ISO-8859-1, en_HK.UTF-8 UTF-8, en_PH ISO-8859-1, en_PH.UTF-8 UTF-8, en_US ISO-8859-1, en_US.ISO-8859-15 ISO-8859-15, en_US.UTF-8 UTF-8

    • es_MX ISO-8859-1, es_MX.UTF-8 UTF-8

    • fr_FR@euro ISO-8859-15, fr_FR ISO-8859-1, fr_FR.UTF-8@euro UTF-8, fr_FR.UTF-8 UTF-8

    • it_IT@euro ISO-8859-15, it_IT ISO-8859-1, it_IT.UTF-8@euro UTF-8, it_IT.UTF-8 UTF-8 

  6. run export PATH=/usr/opt/lsb/appbat/bin:$PATH

  7. run the tests /opt/lsb/test/share/qmtest_libstdcpp_3.3.4/run_tests

  8. answer the questions, the tests run
  9. when the tests complete, it will give a pointer to the journal file
  10. use the tjreport tool from lsb-runtime-test package to generate a tjreport from the journal
  11. put your results on a webpage or send them to Matt Taggart to post and then send a pointer to the debian-lsb list