Differences between revisions 2 and 37 (spanning 35 versions)
Revision 2 as of 2021-05-28 02:39:32
Size: 5323
Editor: PaulWise
Comment: fixes
Revision 37 as of 2022-06-11 05:12:36
Size: 12311
Editor: PaulWise
Comment: adjust some of the other work section
Deletions are marked like this. Additions are marked like this.
Line 1: Line 1:
This page documents the approximate procedure for creating a new port and getting it included into Debian. It is a work in progress and is based on the [[PortTemplate|port template]] wiki page. This is an aspirational document describing the most ideal procedure for creating a new port and getting it included into Debian. Since every port will have different circumstances, the procedure may differ for your port, but the best way to do it should be documented below. It is a work in progress and is based on the [[PortTemplate|port template]] wiki page.
Line 3: Line 3:
/!\ Do not skip any of the steps. If you encounter any unfamiliar jargon, please refer to the [[Glossary|Debian glossary]] for more information.
Line 5: Line 5:
/!\ Complete each step before moving to later steps. /!\ It is best to not skip any of the steps.
Line 7: Line 7:
Assemble a team of people working on the port and find a financial sponsor to support the team's work. /!\ It is best to complete each item before moving to later steps.
Line 9: Line 9:
Identify related ports and ISAs, decide that the port is worth creating and detail the reasons why. <<TableOfContents>>
Line 11: Line 11:
Decide on the GNU triplet, [[Multiarch/Tuples|Debian multiarch tuple]], Debian architecture name, the name that end-users will use for the architecture and other [[ArchitectureSpecificsMemo|details]] of the architecture and ABI. = Preparation =
Line 13: Line 13:
Prepare patches supporting the port for [[HelmutGrohne/rebootstrap|rebootstrap]], [[https://savannah.gnu.org/projects/config|GNU config]], dpkg, binutils, gcc, gdb, qemu, glibc, linux projects.   * Assemble a team of people working on the port and find a financial sponsor to support the team's work.
Line 15: Line 15:
Use your modified rebootstrap, the patches and manual [[DebianBootstrap|bootstrapping]] to build all of build-essential. = Upstreaming =
Line 17: Line 17:
Keep working on the bootstrap process until all of build-essential is installable with debootstrap.   * Decide on the [[ArchitectureSpecificsMemo|architecture ABI details]] and the GNU triplet. A different GNU triplet is required for each ABI, which includes bitness, endianness, soft/hard-float, calling convention, or whether object files can be inter-linked.
Line 19: Line 19:
Continue building as many packages as you can using the new build-essential packages.   * Register accounts for each team member on the bug trackers and subscribe to the mailing lists for upstream projects: [[https://savannah.gnu.org/projects/config|GNU config]], [[https://www.gnu.org/software/binutils/|binutils]], [[https://gcc.gnu.org/|gcc]], [[https://www.gnu.org/software/gdb/|gdb]], [[https://www.qemu.org/|qemu]], kernel (such as [[https://www.kernel.org/|Linux]]), and libc (such as [[https://sourceware.org/glibc/|glibc]])
Line 21: Line 21:
Finalise the architecture details, especially including the ABI and CPU baseline requirements.   * Prepare kernel and toolchain patches supporting the port and get the necessary changes included into these upstream projects: [[https://savannah.gnu.org/projects/config|GNU config]] ([[https://git.savannah.gnu.org/cgit/config.git/tree/config.guess|instructions]]), [[https://www.gnu.org/software/binutils/|binutils]], gcc ([[https://gcc.gnu.org/contribute.html|instructions]]), gdb ([[https://www.gnu.org/software/gdb/contribute/|instructions]]), the kernel (such as Linux ([[https://kernelnewbies.org/|info]])) and the libc (such as glibc ([[https://sourceware.org/glibc/wiki/Contribution%20checklist|instructions]])) (in that order)
Line 23: Line 23:
Get the necessary changes included into the upstream GNU config ([[https://git.savannah.gnu.org/cgit/config.git/tree/config.guess|instructions]]), binutils, gcc ([[https://gcc.gnu.org/contribute.html|instructions]], gdb ([[https://www.gnu.org/software/gdb/contribute/|instructions]]), qemu ([[https://www.qemu.org/contribute/|instructions]]), glibc ([[https://sourceware.org/glibc/wiki/Contribution%20checklist|instructions]]), linux ([[https://kernelnewbies.org/|info]]) projects and any other upstream projects that you have modified.   * If hardware supporting your port is not yet available or your hardware or FPGA softcore or software model is quite slow, prepare patches for the appropriate simulator/emulator upstream project (such as qemu ([[https://www.qemu.org/contribute/|instructions]])). If you do not need this, it will still be very useful to complete at some point, so consider completing it later on.
Line 25: Line 25:
Get any necessary changes included into dpkg ([[Teams/Dpkg/FAQ#new-arch|instructions]]), rebootstrap and other Debian packages involved in the bootstrap process. = Downstreaming =
Line 27: Line 27:
[[Ports|Register the port]] on the Debian wiki, replace "example" throughout the page with the chosen architecture details and fill out as many details as you have.   * Identify related ports and ISAs, decide that the port is worth creating and detail the reasons why.
Line 29: Line 29:
Summarise the details of the architecture and ABI on the [[ArchitectureSpecificsMemo#Summary|arch summary wiki page]].   * Register accounts for each team member on the [[DebianWiki|Debian wiki]] ([[DebianWiki/EditorQuickStart#Account|registration]]) and [[Salsa]] ([[https://salsa.debian.org/users/sign_up|register]]).
Line 31: Line 31:
Setup communication channels for the architecture, such as a [[https://www.debian.org/MailingLists/HOWTO_start_list|mailing list]] and [[IRC#How_to_create_a_new_.23debian-.2A_channel|IRC channel]].   * [[https://www.oftc.net/Services/#register-your-account|Register]] and [[https://www.oftc.net/Services/#verify-your-account|verify]] accounts for each team member on the [[https://www.oftc.net/|OFTC]] IRC chat network and join the #debian-bootstrap, #debian-ports and other Debian [[IRC]] channels.
Line 33: Line 33:
Add your architecture to the [[Teams/Debbugs/ArchitectureTags|bugs arch usertags page]].   * Discuss the port with the community. Please send an email to the DebianList:debian-devel mailing list. If the port is related to some other architectures, ensure you CC the mailing list for the architecture family. Also join the #debian-bootstrap and #debian-ports [[IRC]] channels.
Line 35: Line 35:
[[https://www.ports.debian.org/contacts|Get]] the port included amongst the unofficial ports.   * Finalise the architecture details, especially including the ABI and CPU baseline requirements.
Line 37: Line 37:
[[https://www.debian.org/devel/buildd/|Setup unofficial buildd servers]] for the port, either in qemu or hardware.   * Get any necessary changes included into dpkg ([[Teams/Dpkg/FAQ#new-arch|instructions]]), to get the new Debian architecture name supported (which is the name that end-users will use for the architecture), this requires having the [[ArchitectureSpecificsMemo|architecture ABI details]] clear, the [[Multiarch/Tuples|Debian multiarch tuple]] is derived from the Debian architecture tuple.
Line 39: Line 39:
Include the port on the [[https://www.debian.org/ports/|ports list on the Debian website]] and add the end-user name for the port to the [[https://salsa.debian.org/webmaster-team/webwml/-/blob/master/english/releases/arches.data|website arches data]]. = Bootstrap =
Line 41: Line 41:
Monitor build failures ([[https://buildd.debian.org/status/architecture.php|1]] [[https://udd.debian.org/cgi-bin/ftbfs.cgi|2]]) and continually fix issues.   * Prepare patches supporting the port in [[HelmutGrohne/rebootstrap|rebootstrap]] and other Debian-specific packages.
Line 43: Line 43:
Donate hardware for the [[https://gcc.gnu.org/wiki/CompileFarm|GCC compile farm]].   * Use your modified rebootstrap, the other patches and manual [[DebianBootstrap|bootstrapping]] to build all of build-essential.
Line 45: Line 45:
Setup porterboxen for package maintainers to login to and port packages.   * Keep working on the bootstrap process until all of build-essential is installable with debootstrap.
Line 47: Line 47:
Optionally, donate hardware for [[https://debomatic.github.io/|debomatic]].   * Continue building as many packages as you can using the new build-essential packages.
Line 49: Line 49:
Port the [[https://www.debian.org/devel/debian-installer/|Debian installer]] so that it works on the hardware you have available.   * Get any remaining changes included into the upstream projects that you have modified.
Line 51: Line 51:
Use the Debian installer to install Debian on the hardware you have available.   * Get any necessary changes included into rebootstrap and other projects involved in the bootstrap process.
Line 53: Line 53:
Write some [[https://wiki.debian.org/InstallingDebianOn|documentation about installing the arch on the hardware]] you have installed. = Unofficial port =
Line 55: Line 55:
Submit [[https://www.debian.org/releases/stable/amd64/ch05s04#submit-bug|installation reports]] for the hardware you have installed.   * [[Ports|Register the port]] on the Debian wiki, replace "example" throughout the page with the chosen architecture details and fill out as many details as you have.
Line 57: Line 57:
Submit some [[https://wiki.debian.org/Hardware/Database|hardware probes]] for the hardware you have installed.   * Summarise the details of the architecture and ABI on the [[ArchitectureSpecificsMemo#Summary|arch summary wiki page]].
Line 59: Line 59:
Grow the hardware ecosystem until it satisfies the [[https://dsa.debian.org/ports/hardware-requirements/|requirements for an official port]].   * Setup communication channels for the architecture. Use the architecture family as the name of any communication channels you create, just in case additional related architectures are created later. A [[https://www.debian.org/MailingLists/HOWTO_start_list|mailing list]] and [[IRC#How_to_create_a_new_.23debian-.2A_channel|IRC channel]] are recommended.
Line 61: Line 61:
Improve the port until it satisfies the [[https://ftp-master.debian.org/archive-criteria.html|criteria for being included in the main archive]].   * Add your architecture to the [[Teams/Debbugs/ArchitectureTags|bugs arch usertags page]].
Line 63: Line 63:
Donate hardware for the official buildds and co-ordinate with [[Teams/DSA|the Debian sysadmins]] to setup the hardware in Debian hosting locations.   * [[https://www.ports.debian.org/contacts|Get]] the port included amongst the unofficial ports.
Line 65: Line 65:
Add quality assurance for the port for [[https://reproducible-builds.org/|Reproducible Builds]], [[https://ci.debian.net/doc/|Debian CI]], and [[https://piuparts.debian.org/|piuparts]] (doesn't yet support non-x86 ports) and other [[qa.debian.org|Debian QA efforts]].   * [[https://www.debian.org/devel/buildd/|Setup unofficial buildd servers]] ([[PortsDocs/BuilddPorterboxSetup|automated setup]]) for the port, either in qemu or hardware.
Line 67: Line 67:
Add support for the port to the [[https://www.debian.org/devel/debian-live/|live images]] and [[Cloud|cloud]] images if appropriate hardware exists.   * Include the port on the [[https://www.debian.org/ports/|ports list on the Debian website]] and add the end-user name for the port to the [[https://salsa.debian.org/webmaster-team/webwml/-/blob/master/english/releases/arches.data|website arches data]]. If you create a page on the website, create one named after the architecture family, just in case additional related architectures are created later.
Line 69: Line 69:
Improve the port until it satisfies the [[https://release.debian.org/testing/arch_policy.html|criteria for being included in a Debian release]].   * Monitor build failures ([[https://buildd.debian.org/status/architecture.php|1]] [[https://udd.debian.org/cgi-bin/ftbfs.cgi|2]]) and continually fix issues.
Line 71: Line 71:
Make the port more performant by porting or optimising various libraries (ToDo: add examples).   * Optionally, donate hardware for the [[https://gcc.gnu.org/wiki/CompileFarm|GCC compile farm]]. It is recommended to do this so that people outside Debian also have access to hardware for testing toolchains or other software on your port.
Line 73: Line 73:
Make the bootstrap process more trustworthy through [[https://bootstrappable.org/|Bootstrappable Builds]].   * [[PortsDocs/BuilddPorterboxSetup|Setup porterboxen]] for Debian package maintainers to login to and port packages.

  * Optionally, donate hardware for [[debci]]. It is recommended to do this so Debian package maintainers have a way to automatically run as-installed package tests.

  * Optionally, donate hardware for [[Teams/SalsaCI]]. It is recommended to do this so Debian package maintainers have a way to automatically build and test changes to packages before upload.

  * Optionally, donate hardware for [[https://debomatic.github.io/|debomatic]]. It is recommended to do this so Debian package maintainers have a way to automatically trigger custom package builds.

  * Port the [[https://www.debian.org/devel/debian-installer/|Debian installer]] so that it works on the hardware you have available.

  * Use the Debian installer to install Debian on the hardware you have available.

  * Write some [[https://wiki.debian.org/InstallingDebianOn|documentation about installing the arch on the hardware]] you have installed.

  * Submit [[https://www.debian.org/releases/stable/amd64/ch05s04#submit-bug|installation reports]] for the hardware you have installed.

  * Submit some [[https://wiki.debian.org/Hardware/Database|hardware probes]] for the hardware you have installed.

= Official port =

  * Grow the hardware ecosystem until it satisfies the [[https://dsa.debian.org/ports/hardware-requirements/|requirements for an official port]].

  * Improve the port until it satisfies the [[https://ftp-master.debian.org/archive-criteria.html|criteria for being included in the main archive]].

  * File a request against the [[https://bugs.debian.org/ftp.debian.org|ftp.debian.org]] [[https://www.debian.org/Bugs/pseudo-packages|pseudo-package]] for inclusion of the port amongst the official ports.

  * Donate hardware for the official buildds and co-ordinate with [[Teams/DSA|the Debian sysadmins]] via the [[Teams/DSA/RTUsage|request-tracker]] to setup the hardware in Debian hosting locations.

  * The process for switching from an unofficial port to an official port is approximately:
    * The Debian archive admins will import enough binary packages from the unofficial port to be able to setup the official buildds.
    * The Debian hosting providers will connect the hardware to power, network and remote access.
    * The Debian sysadmins will install Debian on the hardware.
    * The Debian buildd admins will setup the hardware as official buildds and connect them to wanna-build and the main archive.
    * The initially imported packages will be rebuilt on the official buildds and imported into the official archive.
    * The official buildds will be updated using the rebuilt packages.
    * The rest of the archive will be rebuilt on the official buildds.
    * The port team will break any build cycles using [[BuildProfileSpec|build profiles]] and other bootstrap processes (see [[Ports]]).

  * Add quality assurance for the port for [[https://reproducible-builds.org/|Reproducible Builds]], [[https://ci.debian.net/doc/|Debian CI]], and [[https://piuparts.debian.org/|piuparts]] (doesn't yet support non-x86 ports) and other [[qa.debian.org|Debian QA efforts]].

  * Add support for the port to the [[https://www.debian.org/devel/debian-live/|live images]] and [[Cloud|cloud]] images if appropriate hardware exists.

= Released port =

  * Improve the port until it satisfies the [[https://release.debian.org/testing/arch_policy.html|criteria for being included in a Debian release]], updating the [[https://release.debian.org/testing/arch_qualify.html|qualification status]] as progress is made on each of the criteria.

= Other work =

  * Improve the port based on reports from users and QA services.

  * Make the port more accessible by adding support for instruction and system emulation to qemu or adding other simulators/emulators to Debian.

  * Make the port support more hardware by upstreaming support to linux-firmware, bootloaders, Linux, udev, mesa, flash-kernel and other projects then updating those projects in Debian.

  * Make the port more complete by porting packages that require porting to every new architecture, including these source packages: lintian linuxinfo

  * Make the port more performant by porting or optimising various libraries, language runtimes and signal processing algorithms and cryptography libraries, for example: openssl, gnutls, openjdk, v8, ffmpeg, golang, gccgo, ghc, ocaml, simde

  * Make the port more DFSG compatible by reverse engineering common proprietary firmware, rewriting the firmware, documenting that new [[Firmware/Open|open firmware]] and packaging that firmware for Debian.

  * Make the bootstrap process more trustworthy through [[https://bootstrappable.org/|Bootstrappable Builds]].

  * Make the port used more among Debian contributors by giving away hardware via lottery at DebConf, [[Hardware/Wanted|donating hardware to contributors]], providing [[MemberBenefits|discounts to Debian contributors]] for hardware etc.

  * Make the port, the status of it, changes to it and the usage of it more widely known through talks and BoFs at DebConf and other [[DebianEvents|events]], [[DeveloperNews|DevNews]] (and other [[Teams/Publicity|publicity team]] avenues), by filing bugs from machines running the port, submitting reports to the [[https://popcon.debian.org/|package popularity contest]] etc.

  * Make the port more solidly supported through an official [[https://www.debian.org/partners/|partnership with Debian]]. Support Debian itself through [[https://www.debconf.org/sponsors/|sponsoring DebConf]], regular [[https://www.debian.org/donations|donations]] or [[https://www.debian.org/intro/help#organizations|helping Debian more generally]].

This is an aspirational document describing the most ideal procedure for creating a new port and getting it included into Debian. Since every port will have different circumstances, the procedure may differ for your port, but the best way to do it should be documented below. It is a work in progress and is based on the port template wiki page.

If you encounter any unfamiliar jargon, please refer to the Debian glossary for more information.

/!\ It is best to not skip any of the steps.

/!\ It is best to complete each item before moving to later steps.

Preparation

  • Assemble a team of people working on the port and find a financial sponsor to support the team's work.

Upstreaming

  • Decide on the architecture ABI details and the GNU triplet. A different GNU triplet is required for each ABI, which includes bitness, endianness, soft/hard-float, calling convention, or whether object files can be inter-linked.

  • Register accounts for each team member on the bug trackers and subscribe to the mailing lists for upstream projects: GNU config, binutils, gcc, gdb, qemu, kernel (such as Linux), and libc (such as glibc)

  • Prepare kernel and toolchain patches supporting the port and get the necessary changes included into these upstream projects: GNU config (instructions), binutils, gcc (instructions), gdb (instructions), the kernel (such as Linux (info)) and the libc (such as glibc (instructions)) (in that order)

  • If hardware supporting your port is not yet available or your hardware or FPGA softcore or software model is quite slow, prepare patches for the appropriate simulator/emulator upstream project (such as qemu (instructions)). If you do not need this, it will still be very useful to complete at some point, so consider completing it later on.

Downstreaming

  • Identify related ports and ISAs, decide that the port is worth creating and detail the reasons why.
  • Register accounts for each team member on the Debian wiki (registration) and Salsa (register).

  • Register and verify accounts for each team member on the OFTC IRC chat network and join the #debian-bootstrap, #debian-ports and other Debian IRC channels.

  • Discuss the port with the community. Please send an email to the debian-devel mailing list. If the port is related to some other architectures, ensure you CC the mailing list for the architecture family. Also join the #debian-bootstrap and #debian-ports IRC channels.

  • Finalise the architecture details, especially including the ABI and CPU baseline requirements.
  • Get any necessary changes included into dpkg (instructions), to get the new Debian architecture name supported (which is the name that end-users will use for the architecture), this requires having the architecture ABI details clear, the Debian multiarch tuple is derived from the Debian architecture tuple.

Bootstrap

  • Prepare patches supporting the port in rebootstrap and other Debian-specific packages.

  • Use your modified rebootstrap, the other patches and manual bootstrapping to build all of build-essential.

  • Keep working on the bootstrap process until all of build-essential is installable with debootstrap.
  • Continue building as many packages as you can using the new build-essential packages.
  • Get any remaining changes included into the upstream projects that you have modified.
  • Get any necessary changes included into rebootstrap and other projects involved in the bootstrap process.

Unofficial port

  • Register the port on the Debian wiki, replace "example" throughout the page with the chosen architecture details and fill out as many details as you have.

  • Summarise the details of the architecture and ABI on the arch summary wiki page.

  • Setup communication channels for the architecture. Use the architecture family as the name of any communication channels you create, just in case additional related architectures are created later. A mailing list and IRC channel are recommended.

  • Add your architecture to the bugs arch usertags page.

  • Get the port included amongst the unofficial ports.

  • Setup unofficial buildd servers (automated setup) for the port, either in qemu or hardware.

  • Include the port on the ports list on the Debian website and add the end-user name for the port to the website arches data. If you create a page on the website, create one named after the architecture family, just in case additional related architectures are created later.

  • Monitor build failures (1 2) and continually fix issues.

  • Optionally, donate hardware for the GCC compile farm. It is recommended to do this so that people outside Debian also have access to hardware for testing toolchains or other software on your port.

  • Setup porterboxen for Debian package maintainers to login to and port packages.

  • Optionally, donate hardware for debci. It is recommended to do this so Debian package maintainers have a way to automatically run as-installed package tests.

  • Optionally, donate hardware for Teams/SalsaCI. It is recommended to do this so Debian package maintainers have a way to automatically build and test changes to packages before upload.

  • Optionally, donate hardware for debomatic. It is recommended to do this so Debian package maintainers have a way to automatically trigger custom package builds.

  • Port the Debian installer so that it works on the hardware you have available.

  • Use the Debian installer to install Debian on the hardware you have available.
  • Write some documentation about installing the arch on the hardware you have installed.

  • Submit installation reports for the hardware you have installed.

  • Submit some hardware probes for the hardware you have installed.

Official port

  • Grow the hardware ecosystem until it satisfies the requirements for an official port.

  • Improve the port until it satisfies the criteria for being included in the main archive.

  • File a request against the ftp.debian.org pseudo-package for inclusion of the port amongst the official ports.

  • Donate hardware for the official buildds and co-ordinate with the Debian sysadmins via the request-tracker to setup the hardware in Debian hosting locations.

  • The process for switching from an unofficial port to an official port is approximately:
    • The Debian archive admins will import enough binary packages from the unofficial port to be able to setup the official buildds.
    • The Debian hosting providers will connect the hardware to power, network and remote access.
    • The Debian sysadmins will install Debian on the hardware.
    • The Debian buildd admins will setup the hardware as official buildds and connect them to wanna-build and the main archive.
    • The initially imported packages will be rebuilt on the official buildds and imported into the official archive.
    • The official buildds will be updated using the rebuilt packages.
    • The rest of the archive will be rebuilt on the official buildds.
    • The port team will break any build cycles using build profiles and other bootstrap processes (see Ports).

  • Add quality assurance for the port for Reproducible Builds, Debian CI, and piuparts (doesn't yet support non-x86 ports) and other Debian QA efforts.

  • Add support for the port to the live images and cloud images if appropriate hardware exists.

Released port

Other work

  • Improve the port based on reports from users and QA services.
  • Make the port more accessible by adding support for instruction and system emulation to qemu or adding other simulators/emulators to Debian.
  • Make the port support more hardware by upstreaming support to linux-firmware, bootloaders, Linux, udev, mesa, flash-kernel and other projects then updating those projects in Debian.
  • Make the port more complete by porting packages that require porting to every new architecture, including these source packages: lintian linuxinfo
  • Make the port more performant by porting or optimising various libraries, language runtimes and signal processing algorithms and cryptography libraries, for example: openssl, gnutls, openjdk, v8, ffmpeg, golang, gccgo, ghc, ocaml, simde
  • Make the port more DFSG compatible by reverse engineering common proprietary firmware, rewriting the firmware, documenting that new open firmware and packaging that firmware for Debian.

  • Make the bootstrap process more trustworthy through Bootstrappable Builds.

  • Make the port used more among Debian contributors by giving away hardware via lottery at DebConf, donating hardware to contributors, providing discounts to Debian contributors for hardware etc.

  • Make the port, the status of it, changes to it and the usage of it more widely known through talks and ?BoFs at DebConf and other events, DevNews (and other publicity team avenues), by filing bugs from machines running the port, submitting reports to the package popularity contest etc.

  • Make the port more solidly supported through an official partnership with Debian. Support Debian itself through sponsoring DebConf, regular donations or helping Debian more generally.