Website transition to Git

Debian's website structure is currently still kept in CVS. We have discussed several times about migrating to another VCS.

Links to several discussions:

https://lists.debian.org/debian-www/2016/11/msg00051.html

Bug report: #845297: [www.debian.org] Website transition from CVS to Git

Git repo: https://anonscm.debian.org/cgit/webwml/webwml2git.git (see the bug report for details about how the repo is organized)

See also WebsiteVCSEvaluation, WebsiteSVNTransition and CvsusingGit.

Ideas, pros/cons, discarded approaches

Migrate to Subversion (discarded)

The migration is not easier if we decide to migrate to Subversion instead of Git.

Publicity team (which also involves translators in a similar fashion as the website) migrated from Subversion to Git and it was well received, in general.

So it's probably better to migrate to Git and not consider SVN.

Migrate the whole webwml repository to Git

Note: work in this approach will be hosted in https://anonscm.debian.org/cgit/webwml/webwml2git.git/?h=githashes (webwml2git repo, githashes branch).

Transition checklist (needs review and completion)

Create a repo with the non-translatable files

Note: work in this approach will be hosted in https://anonscm.debian.org/cgit/webwml/webwml2git.git/?h=untranslatable (webwml2git repo, untranslatable branch).

This could be done meanwhile we find a solution for migrating the whole repo.

This would allow some people to maintain certain files (for example mirrors.masterlist, organization.data, books.data, and others) in git, and also it would allow to handle the .po files with a weblate instance (in a debian.net service, for example. See below for info about weblate).

Inconveniences is that we'd had the info for the website splitted in two repos, with two different workflows on how to update it.

Transition checklist (needs review and completion)

Use Gettext (.po files) in the whole website

Note: work in this approach will be hosted in https://anonscm.debian.org/cgit/webwml/webwml2git.git/?h=po4a (webwml2git repo, po4a branch).

WML allows to use Gettext in the code, and it's used for some parts. We already have a system to generate .pot and .po files and we could think on "Gettext-ifiying" the whole pages and then get rid of the "check-translation" tags tracking revision numbers.

Translators are familiarized with .po and can use many different tools to work with those files.

We could take advantage of translation memories and reuse many strings that are the same or more or less the same across many files.

We could consider using a weblate instance to handle the translations, and then, they are committed in git directly and we wouldn't need partial checkouts, since people not able to work with the whole repo, could use weblate web interface to work on the files (and they would be committed automatically by weblate, with the author/committer info of the translator).

Two approaches:

po4a-gettextize -f wml -m index.wml -p index.pot

will read the WML and create a POT file (no need to use gettext tags in the file).

Once the PO file is created (translating the pot file), we can use:

po4a-translate -f wml -m index.wml -p index.es.po -l ../spanish/index.wml

to generate the localized wml file.

We could automate the POT updates when needed, and compare the "POT-Creation-Date" in the english POT and language PO files to know if a translation is outdated. No need of versions related to commits (CVS or git).

Some inconveniences and their possible solution:

About weblate

Weblate is a framework for translations that integrates very good with git (each translated string becomes a commit in the git repo). It assumes that we use git for handling our code, and it handles many different types of translation files, but, unfortunately, it does not handle wml files.

Weblate could be used for translations, though, only if we decide to handle all the translations with .po files. Weblate provides a nice web interface showing the status of each translation, and it integrates with git, so we wouldn't need all the scripts that create the translation coordination pages.

Weblate also allows to download the .po files to work offline but don't want to checkout the whole repository.

Using weblate does not interfere with working with the git repo for the people that prefer to work with .po file in the same fashion as they do with code, because we can configure weblate to push each commit automatically, and pull frequently. There can be merge conflicts, though, as with the case of several people working with git at the same time.

You can have a look at https://weblate.org/en/ , and https://hosted.weblate.org/

Weblate is developed by Michal Čihař who is a Debian Developer. Weblate is not packaged in Debian(RFP #745661) but can be installed in a Debian system. A test repo for the website git transition can be found in https://weblate.larjona.net (weblate 2.7 in Debian jessie). Contact LauraArjona if you want an account there.