Differences between revisions 3 and 9 (spanning 6 versions)
Revision 3 as of 2011-05-07 19:40:09
Size: 1899
Comment:
Revision 9 as of 2021-04-15 20:39:53
Size: 3212
Comment:
Deletions are marked like this. Additions are marked like this.
Line 1: Line 1:
This is the documentation on the ''expo.debian.net'' deployment of [[Debexpo]]. This is the documentation on the ''mentors.debian.net'' deployment of [[Debexpo]].
Line 7: Line 7:
 * ''expo.debian.net'' is a CNAME pointer, pointing at debdev.xvm.mit.edu. We have access to this hosting thanks to spang@debian!
 * The machine is mostly maintained by AsheeshLaroia (paulproteus@debian).
 * ''mentors.debian.net'' is Xen VM hosted by [[https://www.wavecon.de/|wavecon]].
 * The machine is mostly maintained by Baptiste Beauplat (lyknode), Mattia Rizzolo (mattia) and Paul Wise (pabs).
Line 13: Line 13:
 * On the machine, there is one important account: ''expo''. Here is how its home directory looks:
 * /home/expo/
  * code/ # Inside here is where git clones of the
   * dev/ # is used by expo-dev.debian.net
   * live/ # is used by expo.debian.net
  * data/ # This is where we store things like package files
   * dev/ # for expo-dev.debian.net
   * live/ # for expo.debian.net
 * On the machine, there is one important account: ''expo''. Here are the important directories:
 * /home/expo/debexpo/ # The application repository, tracks the live branch
 * /home/expo/mentors/ # The maintenance repository
 * /var/spool/debexpo/ # The upload queues (ftp and http)
 * /var/lib/debexpo/ # The git storage and debian repository
Line 22: Line 19:
If you want to make changes to the live site, please first change the ''dev'' site and make sure the changes work. That way, we can avoid disrupting service. Do not make local changes. To deploy a new version, follow those steps:
Line 24: Line 21:
In the future, we could do something where changes go into ''dev'' and automatically migrate to ''live'' after 10 days of no terrible bug reports.  * Test your changes locally
 * Make sure to cover your code with tests
 * Merge and peer-review your work into live
 * Run the deployement script as the ''expo'' user:
    - /home/expo/mentors/bin/update-live.sh
Line 26: Line 27:
If you have access to the expo account, I ''think'' that will be all you need to maintain the service. If you need sudo for something, do ping me (paulproteus). If you have access to the expo account, I ''think'' that will be all you need to maintain the service. If you need sudo for something, do ping the current maintainers.

== configuration files ==

Configuration files are found into `/home/expo/debexpo/debexpo/settings`. The following files are loaded:

  * common.py (included in debexpo.py)
  * prod.py (linked as debexpo.py)
  * debexpo.py

Make all the changes to prod.py.

Additionally, two private files are used to store credentials and secret tokens. Those files are not tracked in the repository (only available locally).

  * redis-password
  * secretkey

== postgresql ==

We use Postgres as the database for storing debexpo data.

== backups ==

Backup are run daily via /etc/cron.d/backup, invoking /root/scripts/backup.sh.
They are stored in /root/backup. Retention period is 5 days.

The backup include a copy of /var/lib/debexpo/ and the database.

A copy is sent to a personal server (contact Baptiste Beauplat to restore the copy).

== Dependencies ==

We use dependencies packaged in Debian to run the application. The meta-package [[https://salsa.debian.org/mentors.debian.net-team/mentors.debian.net|mentors.debian.net]] is kept up-to date with the application requirements.

A copy of the package is retained in /root/m.d.n-pkg.
Line 33: Line 68:
 * Ask Asheesh to give you shell access.  * Ask Mattia to give you shell access.
Line 37: Line 72:
 * paulproteus
 * wRAR
 * lyknode
 * mapreri
 * pabs
Line 42: Line 78:
== Automatic test suite running through Jenkins == == Automatic test suite running with Salsa CI ==
Line 44: Line 80:
Every few minutes, an automated builder checks for updates in git. If there are any updates, it runs the test suite. On creating or merging a merge request, the Salsa CI is automatically triggered. It will check the coding style (flake8), the database migrations, the translations and run the actual test suite. Make sure everything passes before merging or deploying the code.
Line 46: Line 82:
You can see that at http://expo-jenkins.debian.net/. You can see pipelines results at https://salsa.debian.org/mentors.debian.net-team/debexpo/-/pipelines.

This is the documentation on the mentors.debian.net deployment of Debexpo.

Basics

Here are the basic facts:

  • mentors.debian.net is Xen VM hosted by wavecon.

  • The machine is mostly maintained by Baptiste Beauplat (lyknode), Mattia Rizzolo (mattia) and Paul Wise (pabs).
  • We run the Debexpo app through Apache2 and mod_wsgi.

Shell account(s)

  • On the machine, there is one important account: expo. Here are the important directories:

  • /home/expo/debexpo/ # The application repository, tracks the live branch
  • /home/expo/mentors/ # The maintenance repository
  • /var/spool/debexpo/ # The upload queues (ftp and http)
  • /var/lib/debexpo/ # The git storage and debian repository

Do not make local changes. To deploy a new version, follow those steps:

  • Test your changes locally
  • Make sure to cover your code with tests
  • Merge and peer-review your work into live
  • Run the deployement script as the expo user:

    • - /home/expo/mentors/bin/update-live.sh

If you have access to the expo account, I think that will be all you need to maintain the service. If you need sudo for something, do ping the current maintainers.

configuration files

Configuration files are found into /home/expo/debexpo/debexpo/settings. The following files are loaded:

  • common.py (included in debexpo.py)
  • prod.py (linked as debexpo.py)
  • debexpo.py

Make all the changes to prod.py.

Additionally, two private files are used to store credentials and secret tokens. Those files are not tracked in the repository (only available locally).

  • redis-password
  • secretkey

postgresql

We use Postgres as the database for storing debexpo data.

backups

Backup are run daily via /etc/cron.d/backup, invoking /root/scripts/backup.sh. They are stored in /root/backup. Retention period is 5 days.

The backup include a copy of /var/lib/debexpo/ and the database.

A copy is sent to a personal server (contact Baptiste Beauplat to restore the copy).

Dependencies

We use dependencies packaged in Debian to run the application. The meta-package mentors.debian.net is kept up-to date with the application requirements.

A copy of the package is retained in /root/m.d.n-pkg.

Getting access

If you want access to deploy code, the process is pretty simple:

  • Contribute a patch to debexpo, or otherwise demonstrate that you're interested in debexpo and going to be helpful.
  • Ask Mattia to give you shell access.

Right now, the following people have shell access:

  • lyknode
  • mapreri
  • pabs

We also hang out on #debexpo on irc.debian.org.

Automatic test suite running with Salsa CI

On creating or merging a merge request, the Salsa CI is automatically triggered. It will check the coding style (flake8), the database migrations, the translations and run the actual test suite. Make sure everything passes before merging or deploying the code.

You can see pipelines results at https://salsa.debian.org/mentors.debian.net-team/debexpo/-/pipelines.