|
Size: 2574
Comment:
|
← Revision 9 as of 2021-04-15 20:39:53 ⇥
Size: 3212
Comment:
|
| Deletions are marked like this. | Additions are marked like this. |
| Line 1: | Line 1: |
| This is the documentation on the ''expo.debian.net'' deployment of [[Debexpo]]. | This is the documentation on the ''mentors.debian.net'' deployment of [[Debexpo]]. |
| Line 7: | Line 7: |
| * ''expo.debian.net'' is a CNAME pointer, pointing at debdev.xvm.mit.edu. We have access to this hosting thanks to spang@debian! * The machine is mostly maintained by AsheeshLaroia (paulproteus@debian). |
* ''mentors.debian.net'' is Xen VM hosted by [[https://www.wavecon.de/|wavecon]]. * The machine is mostly maintained by Baptiste Beauplat (lyknode), Mattia Rizzolo (mattia) and Paul Wise (pabs). |
| Line 13: | Line 13: |
| * On the machine, there is one important account: ''expo''. Here is how its home directory looks: * /home/expo/ * code/ # Inside here is where git clones of the * dev/ # is used by expo-dev.debian.net * live/ # is used by expo.debian.net * data/ # This is where we store things like package files * dev/ # for expo-dev.debian.net * live/ # for expo.debian.net |
* On the machine, there is one important account: ''expo''. Here are the important directories: * /home/expo/debexpo/ # The application repository, tracks the live branch * /home/expo/mentors/ # The maintenance repository * /var/spool/debexpo/ # The upload queues (ftp and http) * /var/lib/debexpo/ # The git storage and debian repository |
| Line 22: | Line 19: |
| If you want to make changes to the live site, please first change the ''dev'' site and make sure the changes work. That way, we can avoid disrupting service. | Do not make local changes. To deploy a new version, follow those steps: |
| Line 24: | Line 21: |
| In the future, we could do something where changes go into ''dev'' and automatically migrate to ''live'' after 10 days of no terrible bug reports. | * Test your changes locally * Make sure to cover your code with tests * Merge and peer-review your work into live * Run the deployement script as the ''expo'' user: - /home/expo/mentors/bin/update-live.sh |
| Line 26: | Line 27: |
| If you have access to the expo account, I ''think'' that will be all you need to maintain the service. If you need sudo for something, do ping me (paulproteus). | If you have access to the expo account, I ''think'' that will be all you need to maintain the service. If you need sudo for something, do ping the current maintainers. |
| Line 28: | Line 29: |
| == configuration .ini files == | == configuration files == |
| Line 30: | Line 31: |
| The live ... um I'll finish this in a sec | Configuration files are found into `/home/expo/debexpo/debexpo/settings`. The following files are loaded: * common.py (included in debexpo.py) * prod.py (linked as debexpo.py) * debexpo.py Make all the changes to prod.py. Additionally, two private files are used to store credentials and secret tokens. Those files are not tracked in the repository (only available locally). * redis-password * secretkey |
| Line 34: | Line 46: |
| We use Postgres as the database for storing debexpo data. You c | We use Postgres as the database for storing debexpo data. |
| Line 36: | Line 48: |
| == virtualenvs == | == backups == |
| Line 38: | Line 50: |
| We use ''virtualenvs'', which are a way to install Python packages without root and with isolation. The ''dev'' and ''live'' code directories have corresponding virtualenvs. You can access them with: | Backup are run daily via /etc/cron.d/backup, invoking /root/scripts/backup.sh. They are stored in /root/backup. Retention period is 5 days. |
| Line 40: | Line 53: |
| * $ workon live | The backup include a copy of /var/lib/debexpo/ and the database. |
| Line 42: | Line 55: |
| * $ workon dev | A copy is sent to a personal server (contact Baptiste Beauplat to restore the copy). |
| Line 44: | Line 57: |
| Your shell's $PS1 will change to indicate which virtualenv is active. | == Dependencies == |
| Line 46: | Line 59: |
| These are shell snippets provided by [[http://www.doughellmann.com/articles/pythonmagazine/completely-different/2008-05-virtualenvwrapper/index.html|virtualenvwrapper]]. | We use dependencies packaged in Debian to run the application. The meta-package [[https://salsa.debian.org/mentors.debian.net-team/mentors.debian.net|mentors.debian.net]] is kept up-to date with the application requirements. A copy of the package is retained in /root/m.d.n-pkg. |
| Line 53: | Line 68: |
| * Ask Asheesh to give you shell access. | * Ask Mattia to give you shell access. |
| Line 57: | Line 72: |
| * paulproteus * wRAR |
* lyknode * mapreri * pabs |
| Line 62: | Line 78: |
| == Automatic test suite running through Jenkins == | == Automatic test suite running with Salsa CI == |
| Line 64: | Line 80: |
| Every few minutes, an automated builder checks for updates in git. If there are any updates, it runs the test suite. | On creating or merging a merge request, the Salsa CI is automatically triggered. It will check the coding style (flake8), the database migrations, the translations and run the actual test suite. Make sure everything passes before merging or deploying the code. |
| Line 66: | Line 82: |
| You can see that at http://expo-jenkins.debian.net/. | You can see pipelines results at https://salsa.debian.org/mentors.debian.net-team/debexpo/-/pipelines. |
This is the documentation on the mentors.debian.net deployment of Debexpo.
Basics
Here are the basic facts:
mentors.debian.net is Xen VM hosted by wavecon.
- The machine is mostly maintained by Baptiste Beauplat (lyknode), Mattia Rizzolo (mattia) and Paul Wise (pabs).
- We run the Debexpo app through Apache2 and mod_wsgi.
Shell account(s)
On the machine, there is one important account: expo. Here are the important directories:
- /home/expo/debexpo/ # The application repository, tracks the live branch
- /home/expo/mentors/ # The maintenance repository
- /var/spool/debexpo/ # The upload queues (ftp and http)
- /var/lib/debexpo/ # The git storage and debian repository
Do not make local changes. To deploy a new version, follow those steps:
- Test your changes locally
- Make sure to cover your code with tests
- Merge and peer-review your work into live
Run the deployement script as the expo user:
- - /home/expo/mentors/bin/update-live.sh
If you have access to the expo account, I think that will be all you need to maintain the service. If you need sudo for something, do ping the current maintainers.
configuration files
Configuration files are found into /home/expo/debexpo/debexpo/settings. The following files are loaded:
- common.py (included in debexpo.py)
- prod.py (linked as debexpo.py)
- debexpo.py
Make all the changes to prod.py.
Additionally, two private files are used to store credentials and secret tokens. Those files are not tracked in the repository (only available locally).
- redis-password
- secretkey
postgresql
We use Postgres as the database for storing debexpo data.
backups
Backup are run daily via /etc/cron.d/backup, invoking /root/scripts/backup.sh. They are stored in /root/backup. Retention period is 5 days.
The backup include a copy of /var/lib/debexpo/ and the database.
A copy is sent to a personal server (contact Baptiste Beauplat to restore the copy).
Dependencies
We use dependencies packaged in Debian to run the application. The meta-package mentors.debian.net is kept up-to date with the application requirements.
A copy of the package is retained in /root/m.d.n-pkg.
Getting access
If you want access to deploy code, the process is pretty simple:
- Contribute a patch to debexpo, or otherwise demonstrate that you're interested in debexpo and going to be helpful.
- Ask Mattia to give you shell access.
Right now, the following people have shell access:
- lyknode
- mapreri
- pabs
We also hang out on #debexpo on irc.debian.org.
Automatic test suite running with Salsa CI
On creating or merging a merge request, the Salsa CI is automatically triggered. It will check the coding style (flake8), the database migrations, the translations and run the actual test suite. Make sure everything passes before merging or deploying the code.
You can see pipelines results at https://salsa.debian.org/mentors.debian.net-team/debexpo/-/pipelines.