Differences between revisions 159 and 160
Revision 159 as of 2018-01-25 15:14:48
Size: 24561
Comment: Use the public list to find volunteers for front desk
Revision 160 as of 2018-01-25 15:27:29
Size: 25116
Editor: TheAnarcat
Comment: add summary of frontdesk tasks
Deletions are marked like this. Additions are marked like this.
Line 261: Line 261:
=== Checklist ===

Here's a quick summary checklist of the above tasks:

 * [[LTS/Development#Triage_new_security_issues|CVE triage]], which includes:
   * add package to `dla-needed.txt` or mark as `fixed/postponed/ignored/not-affected/end-of-life/unimportant/...`
   * reporting bugs (unfixed in unstable) in the BTS
   * contact maintainers
 * followup on sponsorship requests
 * respond to unanswered questions on the mailing list
 * review claimed packages without activity for > 24h
 * ensure next 4 weeks frontdesk duties are claimed

Translation(s): English - Español - Français - Русский

Contributing to Debian Long Term Support

The Debian LTS team is always looking for more volunteers to do a better job. With more resources, we could for example:

  • support packages that are currently unsupported
  • take care of all Debian packages and not only the most popular ones
  • fix lower priority issues that are currently ignored

If you want to get involved in the LTS team and help keep Debian packages secure for 5 years, have a look at this page. We are assuming that you are familiar with the repository layout described in LTS/Using and that you are subscribed to the mailing list: http://lists.debian.org/debian-lts/

You can help in many ways:

Test updated packages and report regressions

As a simple user, you can test packages that have been updated (or that are in the process of being updated). If you find a regression (i.e. something that used to work and that no longer works), then please report it to debian-lts@lists.debian.org and put the person who prepared the update in copy (in case they are not subscribed to the list).

Many LTS contributors are looking for testers of their updated packages. They send call for tests on the mailing list, so please subscribe to it and test the packages they provide when you can, and report back whether they worked for you.

Debian Security Tracker

The Debian LTS team makes extensive use of the Debian Security Tracker. This is a database of all known security issues in Debian.

Developers check out the Git source (see instructions), commit changes, and the website automatically updates to reflect the changes.

LTS Developers are strongly encouraged to keep the data in the security tracker up-to-date. This includes updating the following files:

  • data/CVE/list: Should be updated with broader information that affects more than just LTS (i.e., stable, testing). For example, consider adding to this file links to bug reports related to the security issue, links to upstream commits or new releases which address the vulnerability, links to public mailing list posts, upstream reports, upstream exploits, information regarding if the exploits do or do not work, etc. Any information that could be useful for the security team as well as the LTS team.
  • data/dla-needed.txt: Should be updated to reflect LTS specific information, such as who is working on a package, possible issues that are specific to LTS, test packages available, requests for help, problems encountered, etc.
  • data/DLA/list: In most cases there is no need to manually change this file, as it is maintained automatically by bin/gen-DLA which is described in following sections.

Make sure to read its documentation on the Debian Security Team website.

Prepare security updates for Wheezy LTS

Claim the issue in the security tracker (in dla-needed.txt)

In order to prevent duplication of effort, make sure the issue is listed in the data/dla-needed.txt file from the security tracker and add your name to it. Normally, issues are triaged by the frontdesk in dla-needed.txt at first, if not, look at how that is done below before adding an issue or adding your name to an issue.

We give package maintainers 24 to 72 hours delay before taking on updates ourselves, depending on the severity of the security issue.

As mentioned in the previous section, make sure to also review and update the relevant information in data/CVE/list when claiming a package for LTS work.

You shall only claim it for the time you do active work. You are expected to actively handle the problem within 2-3 days and then you are expected to announce your intention to upload so others can test it for a few (3-4) more days. If you can not solve the problem within the 2-3 days, then please mention why it takes more time, how far you have gone, the preliminary results and/or free it for someone else to work on it. We do not want people to work on a package for a few hours, wait a few days, work again on it, wait more and so on.

Checkout the git repository of the security tracker:

$ git clone https://salsa.debian.org/security-tracker-team/security-tracker
$ cd security-tracker
$ sudo apt install python-apt
$ bin/setup-repo

*setup/repo* installs a pre-commit hook to check data file syntax.

Then edit the data/dla-needed.txt file:

$ editor data/dla-needed.txt
$ git commit -m "Claim foo in dla-needed.txt" data/dla-needed.txt
$ git push

Write access to the security-tracker repository

Debian Developers automatically have commit access to the security-tracker repository. Otherwise you need to be a member of the security-tracker GitLab project, please request membership through the security tracker gitlab project, security-tracker mailinglist or debian-lts mailinglist.

Build the update

Backport the fix to the version in wheezy (in case there's already been an earlier update). If there is no fix, it is fine to work on a new one: be a good citizen and contribute the patch back upstream and make sure the maintainer and regular security teams will be aware of them, by adding it to a bug report against the package and linking that bug report in the security tracker. (If the issue being addressed requires AddressSanitizer in order to reproduce the vulnerability and confirm the fix, then this page may be helpful).

In the changelog, you need to set the target distribution in debian/changelog to "wheezy-security". The versioning follows the conventions already used in security.debian.org. Historically codenames have been used as version numbers, but this was changed some time ago as version numbers are more deterministic.

  • If you use dch to make the changelog entry, not that it may include the text Non-maintainer upload by the Security Team. It is important to either remove the line, remove by the Security Team, or replace it with something referring to the LTS Team, to avoid giving the appearance of support by the Security Team. Cf 762715.

  • If a package already e.g. had a +wheezy1 update, use +wheezy2 for the next update.
  • If a package hasn't seen an update, use +deb7u1 for the next update.
  • for the rare cases where new upstream versions are introduced, make sure to use a lower version than in jessie. The safe bet is ~deb7u1. Just don't use +wheezyX as this will cause problems!
  • for the rare case where backports of the entire package from newer distributions is required (as opposed to just the fix), it is expected that the changelog will not be in incrementing version order.

To actually build the binary package, you may want to use a porter box, especially if you need to fix a regression that concerns a specific architecture you may not have access to, see PorterBoxHowToUse for instructions on how to use porterboxes.

Test the update

Now build the package and run tests. You may do this by making sure a built-in test suite runs at build-time in the LTS environment. It is acceptable to enable or even backport test suites from later releases if they are available.

If there are no test suites available, you will need to run the package directly. The complexity of this will vary with the size of the package: smaller packages can just be ran in a chroot, others will need a full virtualization environment, install procedures and everything. It may be difficult for you to test the whole package, especially if you are not familiar with the package first hand.

Depending on the urgency of the security fix, and your confidence on your own work, you might want to seek reviews from interested parties (such as upstream authors, Debian maintainers, security team, LTS team, etc.). If you ask reviews, you should supply the following information:

  1. A generated debdiff against the previous version in wheezy.
  2. A link to a test package that can be downloaded and tested.

It is common for LTS workers to upload their packages to a personal repository in people.debian.org, but any public repository or even simple webserver will do. A few guidelines for those temporary uploads:

  • sign the packages (using debsign, for example) so that users can verify package integrity

  • make sure to have the target distribution set to UNRELEASED so that nobody else can upload them before they are ready.

Upload the update

Make sure you used a clean chroot for building the binary packages because the binary packages that you upload end up in the repository (they are not discarded and rebuilt). Using sbuild/pbuilder is recommended.

If you're satisfied, upload to security-master, using dput or dput-ng:

dput security-master hello_1.0-1+deb7u1_amd64.changes

Once uploaded the package will be auto-built for the architectures not covered by your upload (if it's an arch:any package). The architectures currently supported are amd64, i386, armel and armhf.

If you use dput-ng, you need to apply the patch from 745806. After that "dput CHANGES file" is sufficient.

Claim an DLA ID in DLA/list

Run bin/gen-DLA in the top directory of the Git repository. It automatically generates an entry in data/DLA/list and asks you to commit the changes to ensure that no IDs are used twice. The following command would add an entry for src:hello fixing CVE-2014-0666 and creates an advisory template in the top directory of secure-testing for you.

$ bin/gen-DLA --save hello CVE-2014-0666

That list of CVE will make sure that the security tracker is aware of the fix and mark the issues as resolved for that specific package.

Announce the update

Once the upload is completed, an email will be sent to you and to the debian-lts-changes mailing list, confirming the update has been released into the Debian archive. The BTS bot will also announce it in the #debian-devel-changes channel.

Send mail to the debian-lts-announce mailing list. The mail needs to be signed by a GPG key in the debian or debian-maintainers keyring. Inline signatures always work (PGP/MIME works with mutt but fails with Enigmail).

The advisory template has been created by bin/gen-DLA (see before) and generally looks like this:

Subject: [SECURITY] [DLA $DLA-1] $SOURCEPACKAGENAME security update

Package        : $SOURCEPACKAGENAME
Version        : $wheezy_VERSION
CVE ID         : CVE-2014-0001 CVE-2014-0002
Debian Bug     : 12345

DLA text goes here
[...]

If you use mutt, a simple way to send it is to use mutt -H DLA-123.

Only when you have confirmed that the package was processed after upload (once you get the accept email) should you send the DLA to the mailing list.

When all this is done, you should remove the package from other locations it was uploaded to in the test section, if any.

Prepare regression updates for Wheezy LTS

If an upload introduces a regression, it needs to be fixed as soon as possible.

Steps for a regression update

  1. Fix the package.
  2. Do a test build and test installation. Verify the regression issue and also the original issue are really resolved now.
  3. Make sure you used a clean chroot for building the binary packages because the binary packages you upload end up in the repository (they are not discarded and rebuilt). Using sbuild/pbuilder is recommended.

  4. Upload the fixed package to wheezy-security.
  5. Use gen-DLA script to add a DLA entry to data/DLA/list and to provide you with an announcement email template. Two possible options:
    1. The regression fixes the previous upload of the package:

      $ bin/gen-DLA  --save $SOURCEPACKAGENAME regression
    2. The regression fixes an earlier than the previous upload of the package:

      $ bin/gen-DLA  --save $PREVIOUS_DLA_ID $SOURCEPACKAGENAME regression
  6. Commit the updated data/DLA/list to the security-tracker Git repository.

  7. Wait for the uploaded package to arrive in the wheezy-security repository.
  8. In the base folder of the security-tracker Git repository, you can find an auto-generated LTS announcement template file with name DLA-$DLA-{2,3,4...}. Use this file as the template for your regression fix announcement. Don't commit this file to Git(!). Copy+paste the announcement template to your email application, complete/edit it manually and finally send the regression fix DLA email to debian-lts-announce@lists.debian.org.

Triage new security issues

The security team and the LTS team cooperate to process the flow of new security issues that are reported. Since each security issue is assigned a CVE number, we often call this "CVE triage".

If you want to help with this, you must first get familiar with the security tracker. It's the infrastructure we use to track how security issues are affecting Debian packages. Please read carefully this page: http://security-team.debian.org/security_tracker.html

You will need write access to the secure-testing repository.

CVE triaging in the LTS release

Now let's go into detail for the work concerning LTS in particular. We start from the list of open issues in the LTS release.

You can start with the list on the web at this url: https://security-tracker.debian.org/tracker/status/release/oldstable Or you can run bin/lts-cve-triage.py (script in the secure-testing repo) to get a pre-filtered list (packages in dla-needed.txt are excluded, and the issues are grouped by "status").

For each CVE/package listed on this page, there are only 2 desirable outcomes:

  • the package is added to dla-needed.txt so that someone else will have to prepare an updated package and close the security issue(s)
  • the CVE is ignored for some reason:
    • we tag it "postponed" because it's a minor issue that is not urgent to fix but might be fixed in a later update
    • we tag it "ignored" because the issue minor and doesn't warrant an update at all (e.g. the security flaw is only in not compiled and not shipped sources).
    • we tag it "not-affected" because the version in the LTS release is not affected (usually because the vulnerable code is not present in the version)
    • we tag it "end-of-life" because the package is not supported in the LTS release (the list of unsupported package for Wheezy is currently available here: http://anonscm.debian.org/cgit/collab-maint/debian-security-support.git/tree/security-support-ended.deb7. This file and the debian-security-support package should be updated if needed.)

    • we tag it as fixed in a specific version (assuming that the version in the LTS release is higher or equal, it will be automatically assumed to be fixed too)
    • the severity is set to "unimportant" (that should be seldom used and is generally done by the security team anyway)

Finally, part of the triage work includes filing undocumented bugs for security issues in the Debian BTS, using the bin/report-vuln script. Bugs should be filed unless the issue is already fixed in unstable, avoiding duplicates of course.

Guidelines for CVE triaging

Now the tricky part. How do you decide whether an issue should be added to dla-needed.txt or if it should be tagged in some other way? There's no perfect answer, you should use your best judgment after having *carefully* analyzed the issue, the risks involved, the trouble caused by admins to apply the update, etc. If in doubt check with the other lts maintainers, the package maintainer or the security team.

That said the security team is taking similar decisions all the time and we usually follow their decisions. If the security team has added a vulnerable package to "data/dsa-needed.txt", you can usually safely add it to "data/dla-needed.txt" (if the bug is present in the LTS version of the package as well). If the security team has tagged the issue "postponed" with "Minor issue" description, you can safely do the same. An exception is that when the reason picked by the security team is "Maintainer will update via stable-proposed-updates", in this case we must prepare an update for LTS since there is no lts-proposed-updates (yet).

If you triage an issue that was not researched by the security team yet and it's unclear whether the issue warrants a DLA/DSA it's good to check with the security team and with the package maintainer before adding it to dla-needed.txt. It's usually pointless to fix bugs in LTS that will not be fixed in stable or even unstable. Granted, exceptional cases exist, such as when the issue has a greater impact on LTS release.

If it's not clear whether the issue affects sid you should triage that too and open a corresponding bug in the BTS to inform the maintainer (except for embargoed issues). Maintainers often know best how severe an issue is and if an immediate update is actually needed.

Note that some packages have special instructions. Checkout packages/*.txt in the secure-testing repository. This might influence your decision and possibly prompt you to behave differently (for example by not contacting the maintainer since some pre-determined agreement is already in place).

Contact the maintainer

Whenever you take a decision for a new issue (that is real and affecting the LTS release), you should inform the corresponding package maintainers to offer them a chance to help us. We use the bin/contact-maintainers script in the secure-testing repo to this end. Only do this if the bug is already recorded in the BTS. If it's not there yet you can use *bin/report-vuln* to open it. Discussions about severity and urgency are best done in the bugreport since all involved parties (LTS team, security team and package maintainers) can get involved.

When we add a package to dla-needed.txt, you can use it this way for example:

$ bin/contact-maintainers --lts sudo CVE-2014-9680 CVE-2014-0106

When we tag issues as "no-dsa", and don't plan to take care of the updates by ourselves, then we use it in this way:

$ bin/contact-maintainers --lts --no-dsa sudo CVE-2014-9680 CVE-2014-0106

Note that the list of CVE is optional and that this script assumes that you use mutt to send mails. If not, you should consider using the --mailer option to use something else (see bin/contact-maintainer --help for details). You must customize the mail sent (at least to drop the list of uploaders and possibly to adjust the recipients list).

In order to avoid maintainer to feel pushed there are some exceptions to consider:

  • If the package was recently added to dsa-needed.txt you probably want to re-formulate the email to make it clear that this is long term security and not the usual security team email.
  • Make sure a bug report regarding the problem is also in place (if the security issue is public) before this email is sent.
  • Check who was involved in making the security team aware of the issue. If it was the maintainer you need to re-formulate the email as well.

Frontdesk duties

As described in this email, there is an assigned person for the "LTS frontdesk". The allocations are managed in the lts-frontdesk.2018.txt file.

Frontdesk work currently involves CVE triage and making sure queries on the mailing list get an answer. In particular, make sure to followup on package upload sponsorship requests and questions from new contributors to make sure things flow smoothly. Following up on stale and stalled issues is also a good idea. Contributors should feel free to contact frontdesk for any questions regarding LTS.

Frontdesk is also responsible for making sure packages claimed in data/dla-needed.txt are taken care of. When a package has been claimed for too long without activity, notify the claimant and, if there is no response within 24 hours, remove the lock. Claim's history can be inspected with:

bin/review-update-needed --lts

Finally, there should always be someone available in any given week on the frontdesk duty. When starting frontdesk, make sure the next 4 weeks are claimed by someone in the frontdesk roster. If not, send an email to the mailing list to ask for volunteers.

Checklist

Here's a quick summary checklist of the above tasks:

  • CVE triage, which includes:

    • add package to dla-needed.txt or mark as fixed/postponed/ignored/not-affected/end-of-life/unimportant/...

    • reporting bugs (unfixed in unstable) in the BTS
    • contact maintainers
  • followup on sponsorship requests
  • respond to unanswered questions on the mailing list
  • review claimed packages without activity for > 24h

  • ensure next 4 weeks frontdesk duties are claimed

We're tracking LTS related bugs in the Debian bug tracker using two usertags:

  • ease-lts: Bugs that ease long term support like enabling autopkgtests or internal test suites

  • infrastructure: LTS related issues in Debian infrastructure.

To add a certain usertag to a bug you can use the bts command:

bts user debian-lts@lists.debian.org , usertags <bugnumber> ease-lts

Switching to the next LTS release

  • Send a reminder to debian-lts-announce@lists.debian.org two months before the official support for the last LTS ends.

  • Contact the Teams/Publicity and draft an announcement that the old LTS release cycle has come to an end and a new one has started. Example

  • Ensure that the infrastructure is ready for the new LTS release.
  • Contact the FTP team and ask them to wait four weeks with archiving the old LTS release to give users a "grace period" to update their systems.
  • Contact the Stable Release Managers for coordinating the last point release (and so that they can clean up the proposed updates queues).
  • Update all relevant wiki pages especially LTS/Using and this LTS/Development itself.

Housekeeping Tasks

There are several tasks where we base our work on the work of the security team and where we can support them - especially when there are no open issues in dla-needed.txt. This will lead to more packages to fix:

  • Easy: Add bug references for issues unfixed in unstable. Check unreported issues in unstable, recheck them and file a bug if not already there. This makes sure we don't regress in unstable with bugs fixed in stable, oldstable. You can use the bin/report-vuln script to generate a template report to be further edited with reportbug or a MUA. In case you don't use reportbug to further submit the bugreport, please X-Debbugs-CC team@security.debian.org and secure-testing-team@lists.alioth.debian.org. This is done automatically if using reportbug.

  • Hard: Check TODO items from the tracker. Be sure to only mark items as NOT-FOR-US when you're sure the software is not contained in Debian in any form.

Tips & tools


CategoryLts CategoryXWindowSystem