Differences between revisions 3 and 4
Revision 3 as of 2011-07-28 15:31:42
Size: 2349
Editor: NielsThykier
Comment: Putting it under the team namespace
Revision 4 as of 2011-07-28 15:41:04
Size: 3190
Editor: NielsThykier
Comment: Added "How to write a test for the test suite"
Deletions are marked like this. Additions are marked like this.
Line 49: Line 49:

== Writing tests (test suite) ==

The test-suite generally consists of building a number of packages, running Lintian on them and comparing the output to the expected results. There are 5 "sub" test suites named "scripts", "debs", "changes", "source" and "tests". The scripts sub suite consists of perl tests used to unit test parts of lintian and will not be covered here. The rest are used to test if Lintian can diagnose tags correctly.

Generally the "tests" subsuite (in t/tests) should be used. It has more features and uses templates to fill in the "blanks" (which means you have to write less to make it work). The "source", "changes" and "debs" suites are used to hand-make special packages to test edge-cases (like missing required fields). Please consult t/$suite/README for how to write a test in each of the suites.

Hacking Lintian 101

This document aims to give a quick guide and a few hints to understanding Lintian.

Using Lintian from git

Lintian can be used directly from git; this allows developers to skip the boring build/install phase when testing the development version on real packages. To use lintian directly from git you need to set LINTIAN_ROOT (alternatively --root can be used) to the directory where the git clone is. The following script can be used for this:

   1 #!/bin/sh
   2 LINTIAN_ROOT="<INSERT PATH TO LINTIAN GIT DIR>"
   3 export LINTIAN_ROOT
   4 exec "$LINTIAN_ROOT/frontend/lintian" "$@"

Save that as run-lintian, make it executable, put it in your PATH and you are good to go.

Caveat 1: If LINTIAN_ROOT is not set, lintian will try to use the installed version of lintian, which most likely does not have the changes you would like to test.

Caveat 2: Lintian expects to have a standard locale, currently the version from git just uses the installed locale. This works fine as long as you also have Lintian installed (or if you already have the needed locale). In chroots or on minimum installs you may have to generate such a locale and set LOCPATH. Note, lintian will auto-generate this locale for running its test suite.

Running tests

There are a number of ways to run (parts of) the Lintian test suite. To run the entire test suite, simply use:

$ debian/rules runtests

It is also possible to run only a subset of the tests by using the "onlyrun" variable.

$ debian/rules runtests onlyrun=$test

Here $test can be one of:

  • The name of a test, in which case that specific test is run. This works regardless if the test is in the new or in the legacy test suite.
  • The name of a check, in which case all tests in t/tests that starts with $name will be run. If a legacy test happens to have the same name as a check, then this test is also run.

Finally it is also possible to only check for tests that are designed to trigger (or not trigger) a specific tag. For this use:

$ debian/rules check-tag tag=$tag

In this case, the test suite will run all tests that has $tag in its Test-For or Test-Against field. This cannot be used to run a test in the legacy test suite.

Writing tests (test suite)

The test-suite generally consists of building a number of packages, running Lintian on them and comparing the output to the expected results. There are 5 "sub" test suites named "scripts", "debs", "changes", "source" and "tests". The scripts sub suite consists of perl tests used to unit test parts of lintian and will not be covered here. The rest are used to test if Lintian can diagnose tags correctly.

Generally the "tests" subsuite (in t/tests) should be used. It has more features and uses templates to fill in the "blanks" (which means you have to write less to make it work). The "source", "changes" and "debs" suites are used to hand-make special packages to test edge-cases (like missing required fields). Please consult t/$suite/README for how to write a test in each of the suites.