Contents
Creating a yarn plugin to resolve node modules installed as debian packages via apt was a project proposal for Outreachy 2021. Some part was implemented in Outreachy 2021 and remaining part will be completed in Outreachy 2022.
First Meeting (2022)
https://framadate.org/2oKh3SK4XbwlcdaY - choose your preferred time if you are applying for this project - meeting link https://jitsi.debian.social/outreachy-2022 and time 18:00 GMT on 30th March 2022
Contact details
Use Javascript/Mentors for questions and support
Background information
Blog posts of previous interns (this will be very useful to new interns)
Contributor's first contribution experience (helpful for new interns)
Learn more about Javascript packaging at Javascript/Mentors
Installing yarn
In Debian yarn is provided by yarnpkg package (as yarn command is already taken by cmdtest package. See 913997 for details).
# apt install yarnpkg
Using yarn 2 with node_modules plugin
$ mkdir test-project $ cd test-project $ yarnpkg add pretty-ms $ yarnpkg set version berry $ if ! grep nodeLinker .yarnrc.yml >/dev/null; then echo "nodeLinker: \"node-modules\"" >>.yarnrc.yml; fi $ yarnpkg install
End of Scripts
This will create .yarnrc.yml file in the current direcory and will append the line nodeLinker: "node-modules" to a .yarnrc.yml file which tells yarn 2 to use node_modules plugin to have the same behaviour of yarn 1 or npm. Note that yarn 2 does not create nodes_modules directory by default.
Initial Tasks
Packaging tasks
Checkout npm2deb and create a package (pretty-ms, for example). Then look at the deb file which got created and see the folder structure, etc. npm2deb create pretty-ms will get you a .deb. Inspect the deb using an archive manager like file roller or install the deb using dpkg and see where it got installed, dpkg -L node-pretty-ms https://github.com/LeoIannacone/npm2deb
Learn how to create a deb package for simple modules following https://wiki.abrahamraji.in/simple-packaging-tutorial/ and https://wiki.debian.org/SimplePackagingTutorial. See also https://blog.packagecloud.io/eng/2015/07/14/using-dh-make-to-prepare-debian-packages/ Note: debmake is broken for node packages, so use dh_make command from dh-make package instead.
Update an existing node package to new upstream minor or patch version following https://wiki.debian.org/Javascript/Nodejs/Npm2Deb#Updating_a_package_to_new_upstream_release (Pick a node package from https://qa.debian.org/developer.php?email=praveen%40debian.org compare the versions in unstable or experimental with version in watch column. https://semver.org/ defines what is a major, minor or a patch version.)
Nodejs tasks
Create a hello-world plugin with yarn2 which adds a message before a package is being installed. https://yarnpkg.com/features/plugins https://youtu.be/bxnthwe60us
Create a git repo for your project at https://salsa.debian.org and add a README file explaining how to test your code.
Fix one of the "good first issue" bugs on yarn2: https://github.com/yarnpkg/berry/labels/good%20first%20issue (Eg: https://github.com/yarnpkg/berry/issues/1621 )
Update babel build dependency to babel 7 https://github.com/TypeStrong/ts-loader (required to package this for debian as it is a build dependency of yarn 2.x). See https://wiki.debian.org/Javascript/Nodejs/Transitions/Babel7 for some notes.
Update https://www.npmjs.com/package/typanion build dependency to @rollup/plugin-typescript instead of @wessberg/rollup-plugin-ts similar to how clipanion moved.
Documentation tasks
- Create a Debian wiki account and add notes
PS: Send a mail to wiki@debian.org explaining why you want an account created, and included in that mail should be your email address for the account creation, else the account will not be created. Account creation is only limited to this process to prevent the system from being spammed by random editors.
About Patch File
In debian, whenever we need to change a file in the upstream code, we create a patch for that file and the regarding changes are saved in that patch file. In most of the cases, the patch file is needed to be forwarded upstream and involves coordinating with other upstream developers. One of the most simple and best way is to open a pull request to the upstream repository. Keep note of the following points:
- In debian, we add build dependencies in control file but upstream uses package.json, which also needed to be updated.
- Before opening a PR always do a yarn or npm install based on the upstream preferred package manager and run test with the your changes.
Helpful resources- https://wiki.debian.org/UsingQuilt https://tools.ietf.org/doc/quilt/quilt.html
List of packages with minor/patch updates
list of easy to update packages (just confirm a package is not updated already before starting the update) https://alioth-lists.debian.net/pipermail/pkg-javascript-devel/2022-May/067257.html
- node-vue-hot-reload-api - Ajayi - RFS
- node-boom - Ajayi
- node-caniuse-db - Saakshi
- node-caniuse-lite - Sonnie
- node-enhanced-resolve - izzygala
- node-postcss-value-parser - Michael
- node-globals - Michael
- node-watchpack - izzygala
- node-vue-style-loader - Michael
- node-sshpk - izzygala
- node-dompurify - Omama
- node-bootstrap-sass - izzygala
- node-color - izzygala
- node-accepts - izzygala
- node-body-parser - izzygala
- node-getobject - izzygala
- node-toidentifier - izzygala
- node-bytes - izzygala
- node-regenerator-runtime - izzygala
(Before updating existing modules, always ensure that the packages are not updated in Debian archives)
New modules to be packaged
- canvas-confetti - Michael
- cronstrue
- echarts - AmbadyAnandS
- @gitlab/favicon-overlay - Michael
- iframe-resizer
- bundt - Michael
- meros (after bundt is packaged) - Vinay Keshava
- graphql-ws - Michael
- rollup-plugin-bundle-size - izzygala
Advanced packaging updates
If you have done 2-3 minor updates and confident about the process and tools, then try one of these tasks
Embed @types/markdown-it in node-markdown-it - refer Javascript/GroupSourcesTutorial (- izzygala)
- Update graphiql upstream build system to use @types/codemirror 5.60.5 (it currently uses a very old version of the type definitions). This will help in packaging graphiql in debian.
Embed @types/escape-html in node-escape-html - refer Javascript/GroupSourcesTutorial (- Michael)
Remove .yarn from node-typanion. See https://alioth-lists.debian.net/pipermail/pkg-javascript-devel/2021-October/056537.html (- izzygala)
Advanced nodejs tasks
Replace webpack by rollup in node-es6-promise 1011432
Update ava build dependency in is-svg https://github.com/sindresorhus/is-svg/issues/34
Update cosmiconfig dependency in webfont https://github.com/itgalaxy/webfont/issues/572
gitlab use select2 3.5 and debian has 4.0, so it'd be good to update gitlab to use select3 4.0 https://gitlab.com/gitlab-org/gitlab/-/merge_requests/87616
Outreachy Project Status
Two interns started working on this project.
First Meeting ( 21st May 2021 ) : Division of tasks
First meeting on 21st May 2021 at https://jitsi.debian.social/yarn2-plugin-apt
Agenda: Division of tasks, VCS repo for the project (salsa js-team native package), License for the project (Apache 2 or LGPL v3? https://licenseuse.org), Expanding scope of adding yarn plugin apt test in autopkgtest-pkg-nodejs (autodep8)
Update yarnpkg using corepack repo https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=980316
Build yarn plugin https://salsa.debian.org/js-team/yarn-plugin-apt
May 2022 - August 2022 Round Progress Report
Corepack packaging:
- node-nock (updated)
Blogs 2021
Blogs 2022
Helpful Q & A
To avoid repetition, some of our questions and answers from our mentors are being documented for reference
Q: im trying to add a test from package.json to debian/tests/pkg-js/test. I added this line from package.json
npx mocha --reporter spec --require ts-node/register "./test/**/*.ts"
but im getting this error https://paste.debian.net/1236377
A: you don't need npx when building Debian packages. mocha command is available directly in Debian when mocha installed via apt.
Useful Resources
Official Yarn plugin authoring tutorial => https://yarnpkg.com/advanced/plugin-tutorial#all-in-one-plugin-builder
A blog post on the plugin => https://sonnie.hashnode.dev/yarn-plugin-apt
Blog post on writing a package manager by Maƫl Nison ( the author of yarn ) => https://classic.yarnpkg.com/blog/2017/07/11/lets-dev-a-package-manager/ . This gives much clue on recursive dependency resolution. Code in this article is not optimized but gives invaluable hints on what could be done to achieve the desired functionalities. Note that while this blog uses deps from node, you should be requiring and using those from yarn packages ( found at https://github.com/yarnpkg/berry/tree/master/packages ), for instance require xfs from '@yarnpkg/fslib' instead of fs from Node fs module.
A Node module that helps install deps without modifying package.json => https://github.com/darionco/yarn-add-no-save . Looking at the source code of this may give some valuable hints too.