Differences between revisions 61 and 62
Revision 61 as of 2021-07-23 14:17:07
Size: 10446
Editor: ?Mathias Behrle
Comment: Minor typos
Revision 62 as of 2021-09-23 20:02:38
Size: 10598
Editor: Praveen A
Comment: add link to learn more about native es modules in node
Deletions are marked like this. Additions are marked like this.
Line 81: Line 81:
ES modules use import and export statements, but node (current version 12.7) only support this with --experimental-modules flag ([[https://nodejs.org/api/esm.html|ES modules documentation]]). Rollup and Webpack supports ES modules but their implementation currently differs in how to specify an ES module. Rollup and Webpack accepts "module" field in package.json as ES module entry point but node core supports only "type": " module" and "main" fields or .mjs file extension. ES modules use import and export statements, but node (current version 12.7) only support this with --experimental-modules flag ([[https://nodejs.org/api/esm.html|ES modules documentation]]). Rollup and Webpack supports ES modules but their implementation currently differs in how to specify an ES module. Rollup and Webpack accepts "module" field in package.json as ES module entry point but node core supports only "type": " module" and "main" fields or .mjs file extension. [[https://medium.com/@giltayar/native-es-modules-in-nodejs-status-and-future-directions-part-i-ee5ea3001f71|Read more about native ES module support]].

Node.js

Node.js is a platform built on Chrome's JavaScript runtime for easy building of fast, scalable network applications. Node.js uses an event-driven, non-blocking I/O model that makes it lightweight and efficient, perfect for data-intensive real-time applications that run across distributed devices.

Debian packages to install

2018-01-22: please enumerate exactly which Debian packages to install, to get the following commands to work correctly

Packaging modules

On Debian systems up to and including Stretch and Buster (Debian 9 and 10), Node.js modules are located in:

/usr/lib/nodejs/<module_name>

For Debian Bullseye (11), Node.js modules might be located (but this is yet to be decided):

/usr/share/node/<module_name>
/usr/lib/<arch triplet>/node/<module_name>
/usr/lib/node/<module_name>

For more information about packaging a node module for Debian (and the reasons for the change of location for nodejs modules) please take a look at these pages:

  • npm2deb: an automatic tool to easily and quickly deploy debian packages starting from npm information

  • manual: an overview about best practice and policies in packaging nodejs modules in debian systems

debian/watch policy

Some upstream authors don't publish the same versions in Npmjs and Github tags. To be sure to be consistent between node modules, it is recommended to use Npmjs versions. With uscan ≥ 2.18.6 (stable backport at least), you can use searchmode=plain to parse npmjs versions. Example:

version=4
opts="searchmode=plain" \
  https://registry.npmjs.org/aes-js https://registry.npmjs.com/aes-js/-/aes-js-(\d[\d\.]*)@ARCHIVE_EXT@

Complex modules

For complex modules, which have many dependencies to be satisfied, you may want to track your work in this wiki in order to keep javascript team updated about.

For more information, please take a look at Tasks page. Take also a look at How to group many modules in one package.

Generated Files

In debian we take the actual source file and build it instead of the generated file because the source file is what will be modified by the upstream developer (preferably taken from version control system instead of npmjs.com.) As most code on version control system like Github are in different languages like Typescript, coffeescript etc. So the code that people write and share in their version control system cannot be used directly in nodejs or browsers unless compiled into javascript.

If generated files exists in the source tarballs, you should remove all generated files and generate them during debian package build process using tools in debian. See Javascript/Repacking for steps to exclude generated files and Javascript/Nodejs#Using_build_tools_like_grunt for using different build tools in debian during debian package build process.

Browser vs Node

Javascript was initially used for client side scripting on the browsers. Then nodejs came, which made using javascript on the servers also possible. Most widely used version of javascript today is called ECMA Script 5 or ES5.

Minified javascript

Javascript code targeted web browsers is usually minified to reduce size (further than possible with lossless zlib or brotli compressors alone).

?UglifyJS 3 and ?terser are generally recommended among the many JavaScript mification tools,

In debian, if we ship a minified js file, we should include the corresponding non minified code also. To guarantee the non minified code is corresponding to the minified code is, to run uglifyjs during build.

uglifyjs

UglifyJS 3 is the conservative choice: Has fewer reverse dependencies so more likely to be kept up-to-date and less likely to be problematic to backport.

terser

Terser is the fancier choice: Works directly with more modern dialects of ?JavaScript (UglifyJS 3 covers only ?EcmaScript 5 so newer code needs to be "transpiled e.g. using Babel).

Module bundling or browserification or webpacking

Minification is the simplest code transformation we have. There are other transformations too. nodejs supports modules, but it is very inefficient in a browser (http/2 will change this situation which allows parallel requests). Also there are some functionality only present in nodejs but not in browser environment. So using tools like browserify, webpack or rollup, code written for nodejs can be transformed to run on the browser. We need to run this transformation also during build. How To Bring Node.js Modules to the Browser ?. See also this simple explanation of different module formats.

ES6 and transpiling

There is one more type of transformation that is common. They have created a new version of Javascript with new syntax, more functionality built in, it is called ES6 or ES2015. But since browsers and nodejs still don't support es6 fully, we have to convert ES6 to ES5. babel and buble are tools to do that. webpack and rollup has plugins to do ES6 to ES5 transformation.

ES modules vs Commonjs

ES modules use import and export statements, but node (current version 12.7) only support this with --experimental-modules flag (ES modules documentation). Rollup and Webpack supports ES modules but their implementation currently differs in how to specify an ES module. Rollup and Webpack accepts "module" field in package.json as ES module entry point but node core supports only "type": " module" and "main" fields or .mjs file extension. Read more about native ES module support.

Commonjs uses require and only support default export. Rollup and Webpack can convert ES modules to Commonjs.

Read more about different module formats from this blog post.

Using build tools like grunt

You can check in package.json under scripts section for the exact commands to be used for building.

Grunt

  • grunt package has a patch to load global modules. Just call grunt in debian/nodejs/build.

   1 grunt build

Gulp

  • gulp package has a patch to load global modules by default. Just call gulp in debian/nodejs/build.

   1 gulp build

Babel

  • babeljs command is provided by babeljs. See node-d3-color for an example of using babel. Just call babeljs in debian/nodejs/build.

   1 babeljs src -d lib
  • Note: Check ?Babel 7 transition page if the module is using an older version of Babel.

Webpack

  • webpack package needs using a webpack.config.js to resolve global modules, see gitlab and node-mocha for config files.

Use this as debian/webpack.config.js

   1 'use strict';
   2 
   3 var fs = require('fs');
   4 var path = require('path');
   5 var webpack = require('webpack');
   6 
   7 var config = {
   8   resolve: {
   9     modules: ['/usr/lib/nodejs', '/usr/share/nodejs', '.'],
  10   },
  11   resolveLoader: {
  12     modules: ['/usr/lib/nodejs', '/usr/share/nodejs'],
  13   },
  14   node: {
  15     fs: 'empty'
  16   },
  17   output: {
  18     libraryTarget: 'umd'
  19   },
  20   module: { rules: [ { use: [ 'babel-loader' ] } ] }
  21 }
  22 
  23 module.exports = config;

and call webpack in debian/nodejs/build.

   1 webpack --config debian/webpack.config.js --entry ./lib/Yaml.js --output ./dist/Yaml.js --output-library=Yaml
  • Note: Check ?Webpack 4 transition page if the module is using an older version of webpack.

Rollup

  • Rollup: call rollup -c or build script in debian/nodejs/build.

   1 rollup -c

You will need to use rollup-plugin-commonjs module if you get Unresolved dependencies in rollup output like in the example below

index → build/d3.js...
(!) Unresolved dependencies
https://github.com/rollup/rollup/wiki/Troubleshooting#treating-module-as-external-dependency
d3-array (imported by index.js)

Make sure these are added as build dependencies and you use commonjs plugin (if target module is only available as commonjs) or node-resolve plugin if target (if target module is ES module).

See node-redux as an example of adding custom path for node-resolve plugin (with minimum version of node-rollup-plugin-node-resolve 11).

Browserify

  • browserify can be replaced by webpack or rollup or browserify-lite. See node-chart.js as an example.
    • In case of browserify-lite use case, packages installed via apt will be installed in the /usr/share/nodejs directory, while browserify-lite checks only the node_modules/ directory. We create a debian/nodejs/extlinks or debian/nodejs/extcopies files respectively (if either one do not work as expected) and add the installed package-name in the file. pkg-js-tools will read extlinks or extcopies files create necessary symbolic link or copy of modules or packages listed in these files so they are available in node_modules/ directory, extlinks or extcopies will make it available in node_modules/ directory during package build which is also done by by pkg-js-tools.

    • If extlinks is used, it will add symbolic link (ln -s command) if extcopies is used it will copy (cp -r command) so adding <package-name> to debian/nodejs/extlinks is same as mkdir -p node_modules && ln -s /usr/share/nodejs/<package-name>/node_modules

Cake

  • use cake.coffeescript command if you have Cakefile. Just call cake.coffeescript in debian/nodejs/build.

   1 cake.coffeescript build

Best Practices for Upstreams

Documenting things that makes it easy for upstream who wants to support packaging /BestUpstreamPractices