- Old info, this is integrated and running.
This wiki page is meant to track ideas and what needs to be done in order for debci results to be used by britney as gating for package migration from unstable to testing.
Release team requirements/preferences:
migration policy is the domain of britney, so the logic must be there. (requirement)
communication of britney2 (the logic) with the outside world is handled via britney1. (strong preference)
2013-08 Bits from release team about their vision at the time
Discussion thread on debian-release@l.d.o & autopkgtest-devel@l.a.d.o
Discussion thread on debian-devel@l.d.o
autopkgtest adaptations
[First working version locally done] Debian does not have "pockets" like -proposed, but will migrate from unstable. Generalize --apt-pocket=proposed=src:mytrigger to --apt-release=unstable=src:mytrigger and make --apt-pocket use the same code with detecting the suite name from sources.list (as it already does).
Alternative second version with tests, but not 100% tested Instead of --apt-release, implement --add-apt-source="deb http.... SUITE main", --pin-package=pkg1,pkg2 and --default-release=SUITE.
debci adaptations
implement test request parameters, e.g. as in Test_request_format
Existing implementation from Ubuntu autopkgtest worker, where it dissects the json and transforms it into an autopkgtest CLI.
This line is the essential bit for recording the test trigger in result.tar so that britney can match a result to a request:
argv.append('--env=ADT_TEST_TRIGGERS=%s' % ' '.join(params['triggers']))The worker also needs to tell autopkgtest to install only a selected package (the trigger) from unstable with --apt-suite (see above).
debci must move container testbed images from unstable to testing for this to be meaningful.
Discussion @Debconf between terceiro and elbrus on debci <-> happened. See mail thread
britney adaptations
- Tell debci which package to take from unstable
Process results from e.g. Debian dbus (compare to Ubuntu's results)
Nearly done implementation: https://github.com/Debian/britney2/tree/autopkgtest
Pitti proposed cherry-picks
0d43a20471 15aab3348: these two are prerequisites
96df4080b: the main "add autopkgtest"
Ubuntu's design
describes the infra including the britney integration
raw IRC log
Until the IRC conversation between pitti and elbrus is processed and structured in this page, I'll leave most of the raw log included below:
[09:06:14] <elbrus> anybody knows where I can find the Ubuntu code for debci <-> britney interaction? I'd like to use it for inspiration. I am writing a proposal to the mailing-list, but trivially I shouldn't reinvent the wheel.
[09:06:43] <elbrus> pitti: ^^, I assume you wrote that Ubuntu code...
[09:15:38] <pitti> elbrus: yes, indeed I did; we don't actually use debci, but our infra is moderately compatible
[09:16:21] <elbrus> pitti: I am still interested about the logic, so either some documentation or the code I guess....
[09:16:27] <pitti> elbrus: e. g. the AMQP format is the same, but we put results into swift, debci sends it through AMQP (which is not a very good design IMHO, and doesn't work for us at all); but the format of the results are again the same
[09:17:42] <pitti> elbrus: i. e. you define a couple of queues like debci-unstable-amd64, send test requests in (from the britney side, e. g.) and the autopkgtest workers pick out the next job from them
[09:18:03] <pitti> elbrus: and AMQP ensures transactionality, distribution to multiple workers, and that requests never get lost
[09:18:10] <pitti> elbrus: https://wiki.ubuntu.com/ProposedMigration is the top-level doc entry point
[09:18:42] <elbrus> thanks
[09:18:48] <pitti> elbrus: https://wiki.ubuntu.com/ProposedMigration/AutopkgtestInfrastructure describes the infra, https://wiki.ubuntu.com/ProposedMigration/AutopkgtestInfrastructure#Integration_with_proposed-migration_.28britney.29 the britney integratino
[09:19:38] -*- elbrus saw that infra page yesterday, indeed
[09:20:48] -*- elbrus now spots the link to the code... \o/
[09:25:23] <elbrus> pitti: do you know why this isn't implemented in Debian? just nobody did it yet?
[09:26:21] <elbrus> pitti: I assume mostly because of the delta in architecture and then ENOTIME?
[09:27:22] <pitti> elbrus: I talked to Niels and the release team at debconf and had a lightning talk about it too -- it seems there's general consensus that we want to start gating on those
[09:27:53] <pitti> elbrus: but the Ubuntu solution is structurally fairly involved, so I think if Debian's britney would just start to slurp in the debci generated hints file, that'd be a good start
[09:28:02] <elbrus> pitti: I am in discussion with the release team and can now commit to the code on GitHub
[09:28:12] <pitti> it's not really a complete solution for gating, as you often block the wrong package
[09:28:23] <pitti> but it'd be a lot better
[09:28:35] <elbrus> pitti: the hint file is not what the release team wants, they want control in britney
[09:28:36] <pitti> and I'm sure we won't get it right at the first shot anyway -- there will be tuning
[09:28:45] <pitti> ah, they do? nice
[09:29:05] <pitti> but in Heidelberg our britney branch was an utter mess still
[09:29:15] <pitti> and the autopkgtest integration rather intrusive
[09:29:22] <elbrus> pitti: sure, that is why I want something alpha right after the release, such that we have a full release cycle to tune
[09:29:37] <pitti> Niels and I worked a lot on making britney's policies powerful enough to support autopkgtest, and I did lots of cleanup and reengineering
[09:30:03] <pitti> so now our branch is reasonably easy to understand (just ~ 15 commits on top of Debian master), and the autopkgtest.py policy nicely separated
[09:30:12] <pitti> so maybe that's a better fit now
[09:30:29] <elbrus> pitti: so your proposal is to start with Ubuntu's britney and "strip out" stuff unwanted for Debian?
[09:30:37] <pitti> but I'm fairly sure that debian's debci worker needs to get some minor features
[09:31:07] <pitti> elbrus: let's have a quick walk through https://git.launchpad.net/~ubuntu-release/britney/+git/britney2-ubuntu/log/
[09:31:22] <elbrus> pitti: sure, that is why I am discussing here as well, at least the "blame" part needs to improve to single out one package instead of all the delta between now and the last run
[09:31:59] <pitti> most commits aren't needed for Debian, so I'd rather cherry-pick
[09:32:05] <pitti> https://git.launchpad.net/~ubuntu-release/britney/+git/britney2-ubuntu/commit/?id=0d43a20471
[09:32:17] <pitti> elbrus: right, that's what the per-"trigger" tests in our britney do
[09:32:28] <pitti> https://git.launchpad.net/~ubuntu-release/britney/+git/britney2-ubuntu/commit/?id=15aab3348
[09:32:54] <pitti> ^ these two are prerequisites
[09:32:59] <pitti> https://git.launchpad.net/~ubuntu-release/britney/+git/britney2-ubuntu/commit/?id=96df4080b is the main "add autopkgtest"
[09:33:05] <pitti> those three should suffice
[09:33:40] <pitti> elbrus: so, as I said this currently reads results from swift
[09:33:51] <elbrus> ack
[09:33:58] <pitti> elbrus: so either debci needs to move to that, or autopkgtest.py needs to be changed to read them from someplace else
[09:34:11] <elbrus> on Debian it will read from some file from ci.d.n
[09:34:36] <elbrus> parsing json, just like for piuparts
[09:34:45] <pitti> elbrus: ah right, https://ci.debian.net/data/autopkgtest/ is structurally fairly similar to our swift
[09:34:50] <elbrus> I guess is the easiest way forward now
[09:35:14] <pitti> compare https://ci.debian.net/data/autopkgtest/unstable/amd64/d/dbus/ to https://objectstorage.prodstack4-5.canonical.com/v1/AUTH_77e2ada1e7a84929a74ba3b87153c0ac/autopkgtest-xenial?format=plain&prefix=xenial/amd64/d/dbus
[09:36:28] <pitti> elbrus: I'm not sure whether debci already implements test request parameters, as in https://wiki.ubuntu.com/ProposedMigration/AutopkgtestInfrastructure#Test_request_format
[09:37:02] <pitti> elbrus: i. e. in our britney you don't just run "foo's tests", you run "foo's tests to test bar/1.2.-3" (where bar is the "test trigger")
[09:37:26] <pitti> i. e. whenever we upload a new bar 1.2-3, we run both its own and all of its reverse dependencies, and record bar/1.2-3 as the test trigger
[09:37:52] <pitti> so e. g. if a new gtk breaks gnome-calculator, then we hold back gtk, not gnome-calculator
[09:37:53] <elbrus> pitti: do you think this is required on the britney side, or should debci be smarter to figure out "who to blame"?
[09:38:04] <pitti> (which is what the current debci setup can't do)
[09:38:14] <pitti> i. e. with the currently generated britney hints file you'd blame g-calc, not gtk
[09:38:40] <pitti> elbrus: no, IMHO the infra should just run a test and put the results somewhere, britney as the entity which does policy decisions should do the interpretation
[09:38:41] <elbrus> pitti: right, that really needs to be fixed someway before anything
[09:38:48] <pitti> elbrus: right, and our britney does that
[09:38:57] <pitti> elbrus: gedit {"triggers": ["glib2.0/2.20-1"]}
[09:39:09] <pitti> and the result.tar contains that trigger list
[09:39:16] <pitti> so that britney can match it to the original request
[09:39:37] <elbrus> do you think that needs major changes on debci side?
[09:39:46] <pitti> the debci worker just needs to pass that through autopkgtest as --env
[09:40:09] <pitti> elbrus: no, it's fairly trivial -- read a json dict as second part of the message, and pass it to autopkgtest as options
[09:40:43] <elbrus> but where should debci/autopktest get the packages from, both testing and unstable I assume?
[09:41:03] <elbrus> currently debci tests only unstable
[09:41:04] <pitti> elbrus: https://git.launchpad.net/~ubuntu-release/+git/autopkgtest-cloud/tree/worker/worker#n189 is the part where it dissects the json and transforms it into an autopkgtest CLI
[09:41:21] <pitti> https://git.launchpad.net/~ubuntu-release/+git/autopkgtest-cloud/tree/worker/worker#n327
[09:41:28] <pitti> these two lines are the essential bit for this
[09:41:42] <pitti> argv.append('--env=ADT_TEST_TRIGGERS=%s' % ' '.join(params['triggers']))
[09:42:10] <pitti> the bit below is just a special case for our kernel testing, but one step at a time :)
[09:43:04] <pitti> elbrus: right, debci must grow some testing container testbeds for this to be meaningful
[09:43:39] <pitti> elbrus: i. e. for us, packages get uploaded into devel-proposed, our equivalent of "unstable"
[09:43:52] <pitti> (except that no human being runs devel-proposed)
[09:44:05] <pitti> and then our britney tries to isolate as much as possible
[09:44:13] <elbrus> understood
[09:44:23] <pitti> i. e. take devel, install only that new package from -proposed (plus its dependencies from -proposed, if necessary -- we let apt figure that out)
[09:44:35] <pitti> so that you can blame the new gtk if gnome-calculator starts to fail
[09:44:41] <pitti> does that make sense?
[09:44:57] <pitti> so in fact debci would *only* need to have testbeds for testing
[09:45:08] <elbrus> so basically debci should be more of a testing tester with additional packages from unstable as opposed to what it does now...
[09:45:10] <pitti> and partially upgrade them to unstable bits (i. e. the test trigger and dependencies)
[09:45:16] <pitti> correct
[09:45:43] <pitti> at least that's what Ubuntu has done, and that approach has some two years of field experience now
[09:46:07] <pitti> we can't hold back stuff *from* unstable, any damage is already done there
[09:46:28] <pitti> but I think if we do this right, then packages can propagate to testing much faster
[09:46:32] <pitti> and broken packages not at all
[09:46:40] <pitti> so that hopefully a lot more people will run testing instead
[09:47:01] <pitti> so that provides both a carrot and a stick
[09:47:13] <pitti> stick == keep your tests working, d****it
[09:47:25] <pitti> carrot == the better my tests are, the harder it is for other people to break my package
[09:57:57] <elbrus> pitti: and indeed, the release team requires that the policy is in britney
[09:58:14] <pitti> elbrus: so that's the "pass through trigger as env" line I showed you above
[09:58:35] <pitti> elbrus: plus https://git.launchpad.net/~ubuntu-release/+git/autopkgtest-cloud/tree/worker/worker#n289 to tell autopkgtest to install only a selected package from -proposed
[09:58:43] <pitti> elbrus: that part probably needs a change in autopkgtest too
[09:58:56] <pitti> elbrus: right now it supports --apt-pocket=proposed=src:mytriggerpackage
[09:59:07] <pitti> elbrus: but in Debian, unstable is a suite, not a pocket
[09:59:13] <pitti> so we need to teach it that
[09:59:41] -*- elbrus doesn't know the delta between a pocket and a suite
[10:00:26] <pitti> elbrus: it's not too different really in sources.list syntax
[10:00:36] <elbrus> pitti: that is what I thought
[10:00:43] <elbrus> so what is the delta really?
[10:00:46] <pitti> elbrus: so "testing" and "unstable" are suites, "-updates" is a "pocket", i. e. a kind of overlay to a suite
[10:01:02] <pitti> elbrus: so you e. g. have jessie-updates or stretch-security
[10:01:28] <pitti> so --apt-pocket takes the current release and appends "-$VALUE"
[10:02:04] <pitti> but anyway, that syntax needs to be taught to autopkgtset
[10:02:17] <pitti> --apt-suite=unstable=src:mytriggerpackage
[10:02:19] <pitti> somethign like that
[10:02:22] <elbrus> indeed
[10:04:38] <pitti> --apt-pocket takes the current release name and *appends* -$value
[10:04:49] <pitti> --apt-suite would replace the entire current suite name
[10:04:59] <pitti> --apt-pocket has the advantage that it works on any release
[10:05:52] <pitti> elbrus: this is a minor detail and can be generalized easily
[10:06:23] <pitti> i. e. generalize it to --apt-suite and make --apt-pocket some syntactic sugar for it which detects the current series in the testbed
[10:06:37] <pitti> to avoid breaking the CLI