Differences between revisions 121 and 122
Revision 121 as of 2020-04-17 19:05:29
Size: 13218
Editor: BastianBlank
Comment:
Revision 122 as of 2020-04-23 11:24:44
Size: 13607
Editor: ?ŁukaszStelmach
Comment: How to verify SSH host keys
Deletions are marked like this. Additions are marked like this.
Line 244: Line 244:

= SSH Host Keys =

When connecting to Salsa to fetch or push a Git repo for the first time, it is essential to verify host's `ssh` keys. The keys for Salsa have been published as SSHFP DNS records as well as in the Debian [[https://db.debian.org/debian_known_hosts|known_hosts]] file. This is a one time operation. From now on ssh will trust the keys in the local `known_hosts` file.

Salsa Documentation

Salsa is a collaborative development platform within Debian.

Support

In case you encounter any problems with Salsa, to get support you may want to join us:

... they may help you.

Users: Login and Registration

Register an account at https://salsa.debian.org/users/sign_in#register-pane

Namespace concepts (Users, Teams)

Debian Developers

Debian Developers get synced every 6 hours from LDAP and retain their Debian login as salsa username.

External Users

To avoid clash with the Debian LDAP Usernames, external users get a suffix of -guest to their username.

Groups

Users and Group share the same namespace. To prevent clashes with usernames we enforce groups to a '-team' suffix, with the exception being the 'Debian' group, of which all Debian Developers are members.

To create a group, log in and go to the team registration page. There is also a link to it from the registration page: if you're not logged in yet, you will be asked to do so and be redirected afterwards.

Collaborative Maintenance: "Debian" group

The debian group is for CollaborativeMaintenance (the old collab-maint on Alioth). The group is accessible to all Debian developers by default, who are automatically added with Maintainer access levels. Direct commits to repositories in the Debian group by any Debian developer are implicitly welcome. No pre-commit coordination (e.g. merge-request or mail) is expected.

External users (non-Debian Developers) need to request write access to repositories inside debian group from a Debian developer they know, or their sponsor. Access should be granted to single projects and not the whole Debian group.

Projects under debian group cannot be transferred or deleted by anyone except Salsa administrators. In case you need to delete a project or have it transferred out into other namespaces, please contact Alioth administrators via support channel. See #Support section for contact information.

Canonical URLS

The canonical URLs for use in debian/control are:

Vcs-Browser: https://salsa.debian.org/<user-or-team>/<package>
Vcs-Git: https://salsa.debian.org/<user-or-team>/<package>.git

where <user-or-team> is

  • alice for DD Alice Developer <alice@debian.org>

  • bob-guest for non-DD Bob Coder <bobc@example.com>

  • debian for the Debian/ namespace (the equivalent to collab-maint on alioth)

  • foobar-team for the Foobar Packaging Team

You can instruct git to rewrite URLs into pushable ssh URLs:

git config --global url."git@salsa.debian.org:".pushInsteadOf "https://salsa.debian.org/"

This will work for all salsa repositories checked out via https:// URLs in the present, past or future.

You can also use a shortcut for all Salsa repositories:

git config --global url."git@salsa.debian.org:".insteadOf salsa:

This way you can use a shorter commandline like this:

git clone salsa:debian/htop

Projects and Repositories

In GitLab, a project is one Git repository, and each Git repository needs a project. You can create several projects in the same namespace (user or group).

Email notifications

Every project owner can enable "email on push". To do so, go the project settings → integrations → project services → emails on push and configure the list of recipients you want to send emails to.

In particular, to forward emails to tracker.debian.org, you should add dispatch@tracker.debian.org to the recipients (or, if for some not good reason the project name is not the name of the source package, dispatch+${package}_vcs@tracker.debian.org (where ${package} is the source package name)).

Take into account that the current implementation sends a single mail per push with all commits lumped together, which makes it rather useless for any post-review workflow. This is tracked upstream at https://gitlab.com/gitlab-org/gitlab-ce/issues/19901.

Information on manipulating bugs by email

GitLab has quite a lot of text commands aka "quick actions" which can be used when interacting with GitLab via email. Most things can be done via email by replying to the email notifications. There are special email addresses for creating new merge requests and issues via email.

IRC notifications

Irker

Alexander Wirt is sponsoring an Irker instance. It can be enabled with the irker integration available under Settings/Integrations/Irker. Please use the following settings:

Under recipients add a newline separated list of recipients/channels. If your channel is protected by a key, use the syntax channel-name?key=whatever omitting the leading # sign (failing to omit the # sign will result in Irker joining a channel literally named #channel-name?key=whatever and doing so making your channel key public as it is visible in the bot's /whois.
Currently only Push events are supported.

KGB

KGB supports gitlab webhooks. To use the kgb instances provided by dam, tincho, and gregoa from salsa, set a webhook in your project:

http://kgb.debian.net:9418/webhook/?channel=<irc-channel-name-without-#>

For details, additional parameters, and helper scripts see the KGB documentation at https://salsa.debian.org/kgb-team/kgb/wikis/usage

Dealing with Debian BTS from commit messages

We run a webhook receiver that can modify the Debian BTS based on commit messages. If you want to use it, go to your project, "Settings -> integrations" and add a URL (see below), then click save. No secret token is needed, and currently it only deals with push events.

Possible URLs:

https://webhook.salsa.debian.org/close/SOURCENAME
https://webhook.salsa.debian.org/tagpending/SOURCENAME

Replace SOURCENAME with the name of your source package and chose either close or tag pending, depending on the action you want to get.

You can ignore a branch or pattern, say wip/*, by providing the ignored-namespaces parameter. See the README in code for more details.

Code: https://salsa.debian.org/salsa/salsa-webhook.

Deployment keys

For automating task FIXME

Runners

Salsa provides shared runners for all projects to use. All jobs without more specific tags run within a privileged Docker container on one-time-use VM.

You may also add group runners for your group or specific runners and configure them for your project.

Running Continuous Integration (CI) tests

Gitlab provides very flexible and full featured CI functionality built in. Using simple YAML files, the Gitlab CI setup will run the scripts in .gitlab-ci.yml in the specified Docker image, and report the results with a full log. This can also be used to build and deploy static websites using "Gitlab Pages", and more. The salsa-ci-team provides standard Docker images to automate the whole process as much as possible.

  1. In your project's "CI/CD Settings" (e.g. https://salsa.debian.org/debian/fdroidserver/settings/ci_cd), set "Custom CI config path" to debian/gitlab-ci.yml

  2. Create debian/gitlab-ci.yml and commit it to your project's master branch

  3. One approach is using the following pipeline https://salsa.debian.org/salsa-ci-team/pipeline/ covers git-buildpackage, autopkgtest, reprotest, piuparts lintian and Buildd log scanner.

  4. The ci-image-git-buildpackage image is configured by manually editing debian/gitlab-ci.yml. It runs git-buildpackage, lintian, autopkgtest, and aptly, with more tools on the way:

   1 image: registry.salsa.debian.org/salsa-ci-team/ci-image-git-buildpackage:latest
   2 
   3 build:
   4   artifacts:
   5     paths:
   6     - "*.deb"
   7     expire_in: 1 day
   8   script:
   9     - gitlab-ci-git-buildpackage-all

If you want the build result to be posted to an apt repo, then use the aptly script. It'll be posted to an unsigned repo on pages.debian.net. For example, https://salsa.debian.org/foo/bar will be posted to https://foo.pages.debian.net/bar.

   1 image: registry.salsa.debian.org/salsa-ci-team/ci-image-git-buildpackage:latest
   2 
   3 pages:
   4   stage: deploy
   5   artifacts:
   6     paths:
   7       - public
   8   script:
   9     - gitlab-ci-git-buildpackage-all
  10     - gitlab-ci-aptly

It is also possible to run each step manually or even split them into separate GitLab CI "Jobs":

   1 image: registry.salsa.debian.org/salsa-ci-team/ci-image-git-buildpackage:latest
   2 
   3 pages:
   4   stage: deploy
   5   artifacts:
   6     paths:
   7       - public
   8   script:
   9     - gitlab-ci-enable-experimental
  10     - gitlab-ci-git-buildpackage
  11     - gitlab-ci-lintian
  12     - gitlab-ci-autopkgtest
  13     - gitlab-ci-aptly

There is a more basic setup based on building with dpkg-buildpackage as documented on the Gitlab blog: Automated Debian Package Build with GitLab CI

Web page hosting

Gitlab offer the "Gitlab Pages" feature, and it is enabled on Salsa as https://<namespace>.pages.debian.net/<project>

This feature makes use of Gitlab-CI to generate static pages in a public directory, on every push.

See the official documentation for details. Note that hosting pages on arbitrary domains — whilst supported by upstream — is not supported on Salsa due to lack of bandwidth within DSA to support that feature (see RT #7045).

ChrisLamb has created a number of https://lamby.pages.debian.net/salsa-ribbons/ that you can add to your site.

https://<namespace>.pages.debian.net should work, thanks to Let's Encrypt new wildcard certificate support.

Quick start

  1. On your project Home, use Set up CI/CD button. (If your project is empty, select New file instead.)

  2. Choose a Gitlab CI Yaml template (Pages templates are at the end)

  3. Edit the template to suit your needs and save it
  4. Push something to the repository. You will see there is a CI Job pending
  5. Wait a few minutes for the job to run. When it's Passed you can see your pages at https://<namespace>.pages.debian.net/<project>/)

Even though we plan to support simple page generators like Jekyll or Hugo in the future, in most cases, you should content yourself with the HTML template, and generate the pages locally to push them afterward, in order to save the resources on the runner. Some templates might require commands not available on the server anyway.

We mean that. Really. Be nice to the server. ;)

important: (at least for static pages deployment) your artifacts must be stored in a directory named public/; if they are currently in a different location, use the script section in .gitlab-ci.yml to create that dir and copy the content there.

Getting Help

See the Salsa maintenance description.

Hints for previous users of Alioth

See Salsa/AliothMigration.

API Usage Best practises

  • if you want to know if a project exists, access the project by name, authenticated, if you get a 404 then it doesn't exists.
  • do not search for getting an id. If you need the id, access the project by name and use path-encoding https://docs.gitlab.com/ee/api/#namespaced-path-encoding

  • do not request all projects in a group unless you really have. If you really have to get the list, for i.e. looping, use simple=true (https://docs.gitlab.com/ee/api/groups.html#list-a-group-s-projects).

  • Implement proper pagination, please do not just requests a few hundreds elements per page
  • set an User-Agent header with information about the project; don't make requests with generic user agent headers

  • if you use a lib, ensure the lib does implement the api properly
  • do not run extensive jobs too often
  • please consider to use vcswatch or other data gathering projects
  • do not regularly poll things
  • if in doubt, talk to us before you code and talk to us before you put your code into production

SSH Host Keys

When connecting to Salsa to fetch or push a Git repo for the first time, it is essential to verify host's ssh keys. The keys for Salsa have been published as SSHFP DNS records as well as in the Debian known_hosts file. This is a one time operation. From now on ssh will trust the keys in the local known_hosts file.