diff --git a/GOVERNANCE.md b/GOVERNANCE.md index 5f95868..62a4f57 100644 --- a/GOVERNANCE.md +++ b/GOVERNANCE.md @@ -33,7 +33,8 @@ There is no specific set of requirements or qualifications for Team Membership b Changes to Team membership should be proposed with an issue and labelled `modules-agenda` to be included in the next [team meeting](#team-meetings). Decisions are made via the -[Consensus Seeking Process](#consensus-seeking-process). +[Consensus Seeking Process](#consensus-seeking-process). If there are not objections in the +issue new members may attend the meeting in which their membership is officially accepted. If a Team member is unable to attend a meeting where a planned membership decision is being made, then their consent is assumed. @@ -97,10 +98,20 @@ meeting. In both cases the decision must be seconded by a majority of the Team or else the discussion will continue. Simple majority wins. Only Active Members participate in a vote. -## Merging PRs into this Repository +## Pull Requests -This section does not apply to [the Node.js core repository](https://github.com/nodejs/node). -It only applies to [the Modules repository](https://github.com/nodejs/modules). +This section details expectations for team members involved in any pull +requests that relate to the group's [scope][modules-team-purpose] and +[implementation work][modules-team-plan]. + +The following expectations apply to all team members involved in a related pull +requests in [this Repository][nodejs-modules] and +the [nodejs/ecmascript-modules][nodejs-ecmascript-modules] fork. This section does +not apply to [the Node.js core repository](https://github.com/nodejs/node). + +These expectations are intended to ensure that all concurrent efforts align +with the overall direction of the group for delivering a cohesive and predictable +user experience for ECMAScript, CommonJS, and other modules supported by Node.js. Pull requests not included under the _special exemptions_ section below must reach consensus in a meeting in order to be merged into this repository. A pull @@ -116,6 +127,9 @@ changes to this repository: - Editorial changes. - Meeting minutes. - Updates to the team lists via the `ncu-sync` tool. +- Doc Fixes +- Tests +- Fixing Conflicts with a rebase These pull requests may be merged without being presented at a meeting if a reasonable time is given for review and there no dissent. The time period seen @@ -151,3 +165,12 @@ By making a contribution to this project, I certify that: personal information I submit with it, including my sign-off) is maintained indefinitely and may be redistributed consistent with this project or the open source license(s) involved. + + + + +[nodejs-modules]: https://github.com/nodejs/modules +[nodejs-core]: https://github.com/nodejs/node +[nodejs-ecmascript-modules]: https://github.com/nodejs/ecmascript-modules +[modules-team-purpose]: ./README.md#purpose +[modules-team-plan]: ./doc/plan-for-new-modules-implementation.md diff --git a/README.md b/README.md index 9815681..64ada30 100644 --- a/README.md +++ b/README.md @@ -2,108 +2,36 @@ ## Purpose -The Node.js Modules Team maintains and actively develops the ESModules (ESM) implementation in Node.js Core. +The original Node.js Modules Team planned and developed the ECMAScript Modules (ESM) implementation in Node.js core. -Work includes: - -* ESM Loader implementation -* ESM Module Specifier Resolution Algorithm -* Loader Hooks -* ESM and Common.js Interoperability -* Node.js and Browser interoperability -* VM Modules implementation - -## Features - -Based on [these use cases](https://docs.google.com/document/d/10BBsIqdAXB9JR2KUzQGYbCiVugYBnxE4REBakX29yyo/edit) ([#55](https://github.com/nodejs/modules/issues/55)), our implementation aims to support the following features (subject to change): - -- Spec compliance ([#132](https://github.com/nodejs/modules/issues/132)) -- Browser equivalence ([#133](https://github.com/nodejs/modules/issues/133)) -- No refactoring ([#87](https://github.com/nodejs/modules/issues/87)) -- Pluggable Loaders to support multiple use cases ([#82](https://github.com/nodejs/modules/issues/82)) -- Named exports when importing CJS ([#81](https://github.com/nodejs/modules/issues/81)) -- Dual-goal packages ([#93](https://github.com/nodejs/modules/issues/93)) -- Multi-mode packages ([#94](https://github.com/nodejs/modules/issues/94)) -- Code coverage/instrumentation ([#95](https://github.com/nodejs/modules/issues/95)) -- Runtime loaders, transpilation at import time ([#96](https://github.com/nodejs/modules/issues/96)) -- Arbitrary sources for module source text ([#97](https://github.com/nodejs/modules/issues/97)) -- Mock modules (injection) ([#98](https://github.com/nodejs/modules/issues/98)) -- Mixed module types within app/module; gradual migration from CommonJS to ESM ([#99](https://github.com/nodejs/modules/issues/99)) -- Transparent interoperability for ESM importing CommonJS ([#100](https://github.com/nodejs/modules/issues/100)) -- Polyfillability ([#101](https://github.com/nodejs/modules/issues/101)) -- Tree shaking ([#102](https://github.com/nodejs/modules/issues/102)) -- File / path / URL resolving ([#103](https://github.com/nodejs/modules/issues/103)) -- Retrievable module metadata ([#104](https://github.com/nodejs/modules/issues/104)) -- Transparent migration ([#105](https://github.com/nodejs/modules/issues/105)) -- WASM modules ([#106](https://github.com/nodejs/modules/issues/106)) -- Browser and Node compatibility without building ([#107](https://github.com/nodejs/modules/issues/107)) -- Browser-compatible build that includes CommonJS ([#108](https://github.com/nodejs/modules/issues/108)) -- Browser-compatible specifier resolution ([#109](https://github.com/nodejs/modules/issues/109)) -- Specifier resolution customization ([#110](https://github.com/nodejs/modules/issues/110)) -- Package encapsulation ([#111](https://github.com/nodejs/modules/issues/111)) -- Don’t break CommonJS ([#112](https://github.com/nodejs/modules/issues/112)) -- Conditional imports ([#113](https://github.com/nodejs/modules/issues/113)) -- Import JSON without needing asynchronous syntax ([#114](https://github.com/nodejs/modules/issues/114)) -- Importing non-JavaScript files ([#115](https://github.com/nodejs/modules/issues/115)) -- Import CommonJS without needing asynchronous syntax ([#116](https://github.com/nodejs/modules/issues/116)) -- NodeJS contextual pathing use cases ([#121](https://github.com/nodejs/modules/issues/121)) +The current team exists as subject matter experts to consult regarding the Node.js ESM implementation. ## Members - + -- [@benjamingr](https://github.com/benjamingr) - Benjamin Gruenbaum -- [@benjamn](https://github.com/benjamn) - Ben Newman -- [@bmeck](https://github.com/bmeck) - Bradley Meck -- [@ceejbot](https://github.com/ceejbot) - C J Silverio -- [@chrisdickinson](https://github.com/chrisdickinson) - Chris Dickinson +- [@bmeck](https://github.com/bmeck) - Bradley Farias - [@DanielRosenwasser](https://github.com/DanielRosenwasser) - Daniel Rosenwasser -- [@devsnek](https://github.com/devsnek) - Gus Caplan -- [@Fishrock123](https://github.com/Fishrock123) - Jeremiah Senkpiel +- [@devsnek](https://github.com/devsnek) - devsnek - [@GeoffreyBooth](https://github.com/GeoffreyBooth) - Geoffrey Booth - [@giltayar](https://github.com/giltayar) - Gil Tayar - [@guybedford](https://github.com/guybedford) - Guy Bedford -- [@iarna](https://github.com/iarna) - Rebecca Turner -- [@inidaname](https://github.com/inidaname) - Hassan Sani -- [@jdalton](https://github.com/jdalton) - John-David Dalton +- [@JakobJingleheimer](https://github.com/JakobJingleheimer) - Jacob Smith - [@jkrems](https://github.com/jkrems) - Jan Olaf Krems -- [@justinfagnani](https://github.com/justinfagnani) - Justin Fagnani -- [@linclark](https://github.com/linclark) - Lin Clark - [@ljharb](https://github.com/ljharb) - Jordan Harband -- [@manekinekko](https://github.com/manekinekko) - Wassim Chegham -- [@mcollina](https://github.com/mcollina) - Matteo Collina -- [@mduleone](https://github.com/mduleone) - Matt DuLeone -- [@mhdawson](https://github.com/mhdawson) - Michael Dawson - [@MylesBorins](https://github.com/MylesBorins) - Myles Borins +- [@qballer](https://github.com/qballer) - Doron Tsur - [@robpalme](https://github.com/robpalme) - Rob Palmer - [@targos](https://github.com/targos) - Michaël Zasso -- [@tbjers](https://github.com/tbjers) - Torgny Bjers +- [@VoltrexKeyva](https://github.com/VoltrexKeyva) - Mohammed Keyvanzadeh +- [@WebReflection](https://github.com/WebReflection) - Andrea Giammarchi - [@weswigham](https://github.com/weswigham) - Wesley Wigham -- [@XadillaX](https://github.com/XadillaX) - Khaidi Chu +- [@zackschuster](https://github.com/zackschuster) - Zack Schuster -## Observers +## Archives - +- **[Features](./doc/archive/features.md)**: As part of creating the ECMAScript Modules implementation for Node.js 12.0.0, [use cases](./doc/use-cases.md) were brainstormed, which led to a list of features. This feature list, which previously existed as a section in this repo's root README, informed what became the [plan for the implementation](./doc/archive/plan-for-new-modules-implementation.md). -- [@bmeurer](https://github.com/bmeurer) - Benedikt Meurer -- [@dduleone](https://github.com/dduleone) - Dan DuLeone -- [@devamaz](https://github.com/devamaz) - Ahmad Abdul-Aziz -- [@eugeneo](https://github.com/eugeneo) - Eugene Ostroukhov -- [@evanplaice](https://github.com/evanplaice) - Evan Plaice -- [@hiroppy](https://github.com/hiroppy) - Yuta Hiroto -- [@jasnell](https://github.com/jasnell) - James M Snell -- [@refack](https://github.com/refack) - Refael Ackermann -- [@robwormald](https://github.com/robwormald) - Rob Wormald -- [@tbranyen](https://github.com/tbranyen) - Tim Branyen -- [@TheLarkInn](https://github.com/TheLarkInn) - Sean Larkin -- [@TimothyGu](https://github.com/TimothyGu) - Timothy Gu -- [@WebReflection](https://github.com/WebReflection) - Andrea Giammarchi -- [@xtuc](https://github.com/xtuc) - Sven Sauleau -- [@yosuke-furukawa](https://github.com/yosuke-furukawa) - Yosuke Furukawa -- [@zackschuster](https://github.com/zackschuster) - Zack Schuster -- [@tzmanics](https://github.com/tzmanics) - Tara Z. Manicsic -- [@zenparsing](https://github.com/zenparsing) - Kevin Smith - - +- **[Plan for New Modules Implementation](./doc/archive/plan-for-new-modules-implementation.md)**: This document summarized the work that went into the new ECMAScript Modules implementation that shipped in Node.js 12.0.0. diff --git a/RESOURCES.md b/RESOURCES.md index 2aed9b7..df15431 100644 --- a/RESOURCES.md +++ b/RESOURCES.md @@ -14,18 +14,18 @@ Following a list of ESM and CommonJS relevant projects. -#### Transpilers - * [Babel](http://babeljs.io) transpiled ES6+ including modules downgraded to CommonJS with an `exports.default` strategy - * [TypeScript](https://www.typescriptlang.org) transpiled classes including ES6 like/inspired modules - #### Bundlers * [Webpack](https://webpack.js.org) multi purpose bundler compatible with CommonJS `exports.default` strategy. Actually diverging from npm resolution preferring a `browser` field in the `package.json` instead of actual CommonJS. * [Rollup](https://rollupjs.org) next-generation ES6 module bundler with tree shaking capability * [Parceljs](https://parceljs.org/) multi purpose bundler focused on zero config and performance * [Browserify](http://browserify.org) the CommonJS module bundler that brought NodeJS to the Web -#### Compilers - * [Reify](https://www.npmjs.com/package/reify) compiles ESM syntax to CommonJS, supporting live bindings without rewriting imported variable names +#### Transpilers + * [Babel](http://babeljs.io) converts ES2015+ to older versions, including a CommonJS interop layer with ESM + * [TypeScript](https://www.typescriptlang.org) TypeScript is a typed superset of JavaScript that compiles to plain JavaScript, including a CommonJS interop layer with ESM + +#### Loaders + * [Reify](https://www.npmjs.com/package/reify) compiles ESM syntax to CommonJS on the fly, supporting live bindings without rewriting imported variable names * [@std/esm](https://github.com/standard-things/esm) is a fork of Reify focused on following Node's ESM support roadmap while providing a bridge from the CJS/ESM usage of today to the ESM of tomorrow. #### Runtime Utilities diff --git a/doc/archive/features.md b/doc/archive/features.md new file mode 100644 index 0000000..b1a6308 --- /dev/null +++ b/doc/archive/features.md @@ -0,0 +1,55 @@ +> _This document has been archived._ +> +> As part of creating the ECMAScript Modules implementation for Node.js 12.0.0, use cases were brainstormed, which led to a list of features. This feature list, which previously existed as a section in this repo's root [README](../../README.md), informed what became the [plan for the implementation](./plan-for-new-modules-implementation.md). + +# Features + +Based on [these use cases](./doc/use-cases.md) ([#55](https://github.com/nodejs/modules/issues/55)), our implementation aims to support the following features (subject to change): + +### Baseline Modules Implementation Features: +* Spec compliance ([#132](https://github.com/nodejs/modules/issues/132)) +* Browser equivalence ([#133](https://github.com/nodejs/modules/issues/133)) +* Don’t break CommonJS ([#112](https://github.com/nodejs/modules/issues/112)) +* No refactoring ([#87](https://github.com/nodejs/modules/issues/87)) + +#### Browser Interop: +* Browser and Node compatibility without building ([#107](https://github.com/nodejs/modules/issues/107)) +* Browser-compatible specifier resolution ([#109](https://github.com/nodejs/modules/issues/109)) +* Browser-compatible builds for ESM and CommonJS ([#108](https://github.com/nodejs/modules/issues/108)) + +#### CommonJS Interop: +* Named exports when importing CJS ([#81](https://github.com/nodejs/modules/issues/81)) +* Multi-mode packages ([#94](https://github.com/nodejs/modules/issues/94)) +* Transparent interoperability for ESM importing CommonJS ([#100](https://github.com/nodejs/modules/issues/100)) +* Consumer-agnostic imports ([#105](https://github.com/nodejs/modules/issues/105)) +* Mixed module types within app/module; gradual migration from CommonJS to ESM ([#99](https://github.com/nodejs/modules/issues/99)) +* ESM in .js files ([#151](https://github.com/nodejs/modules/issues/151)) + +### Existing Node.js Utility Features: + +* Importing non-JavaScript files ([#115](https://github.com/nodejs/modules/issues/115)) +* NodeJS contextual pathing use cases ([#121](https://github.com/nodejs/modules/issues/121)) +* ESM in executable files ([#152](https://github.com/nodejs/modules/issues/152)) +* Callable resolver ([#157](https://github.com/nodejs/modules/issues/157)) + +### Loader Features: + +* Code coverage/instrumentation ([#95](https://github.com/nodejs/modules/issues/95)) +* Pluggable Loaders to support multiple use cases ([#82](https://github.com/nodejs/modules/issues/82)) +* Runtime loaders, transpilation at import time ([#96](https://github.com/nodejs/modules/issues/96)) +* Arbitrary sources for module source text ([#97](https://github.com/nodejs/modules/issues/97)) +* Mock modules (injection) ([#98](https://github.com/nodejs/modules/issues/98)) +* Specifier resolution customization ([#110](https://github.com/nodejs/modules/issues/110)) +* Package encapsulation ([#111](https://github.com/nodejs/modules/issues/111)) +* Conditional imports ([#113](https://github.com/nodejs/modules/issues/113)) + +### WASM Features: + +* WASM modules (#[106](https://github.com/nodejs/modules/issues/106)) + +### Developer and Tooling Features: + +* File / path / URL resolving (#[103](https://github.com/nodejs/modules/issues/103)) +* Import CommonJS without needing asynchronous syntax (#[116](https://github.com/nodejs/modules/issues/116)) +* Tree shaking (#[102](https://github.com/nodejs/modules/issues/102)) +* Polyfillability (#[101](https://github.com/nodejs/modules/issues/101)) diff --git a/doc/archive/plan-for-new-modules-implementation.md b/doc/archive/plan-for-new-modules-implementation.md new file mode 100644 index 0000000..20612f9 --- /dev/null +++ b/doc/archive/plan-for-new-modules-implementation.md @@ -0,0 +1,134 @@ +> _This document has been archived._ +> +> This document summarized the work that went into the new ECMAScript Modules implementation that shipped in Node.js 12.0.0. + +# Plan for New Modules Implementation + +This document outlines the plan for building a new implementation to support ECMAScript modules in Node.js. The effort is split up into phases: + +* **Phase 0** branches off of current Node but removes much of the Node 8.5.0+ `--experimental-modules` implementation so that a new implementation could be built in its place. + +* **Phase 1** adds the “minimal kernel,” features that the modules working group felt would likely appear in any potential new ES modules implementation. + +* **Phase 2** fleshes out the implementation with enough functionality that it should be useful to average users as a minimum viable product. + + - At the completion of Phase 2, the old `--experimental-modules` implementation was [replaced](https://github.com/nodejs/node/pull/26745) with this new one (still behind the `--experimental-modules` flag). It was released as part of Node 12 on 2019-04-23. + +* **Phase 3** improves user experience and extends the MVP. + + - At the completion of Phase 3, the new implementation’s experimental flag was [dropped](https://github.com/nodejs/node/pull/29866). It was released as part of Node 13.2.0 on 2019-11-21. + +* **Phase 4** are items that were under development in earlier phases but weren’t finished when the new implementation’s experimental flag was dropped; these items may continue development after unflagging and potentially ship in later versions of Node.js. + +The effort is currently in **[Phase 4](#phase-4-further-improvements-after-unflagging)** + +At every phase, the following standards must be maintained: + +* Spec compliance ([#132](https://github.com/nodejs/modules/issues/132)): We must always follow the ES spec. +* Browser equivalence ([#133](https://github.com/nodejs/modules/issues/133)): There’s room for debate in specific cases, but in general if Node is doing something that browsers also do, Node should do it in the same way. Alternatively, code that executes in both environments should produce identical results. +* Don’t break CommonJS ([#112](https://github.com/nodejs/modules/issues/112)): We cannot cause breaking changes with regards to CommonJS. + +See also the [features list in the README](https://github.com/nodejs/modules#features). + +## Phase 0: Start Fresh + +From current shipping Node, the following changes were made to strip out most of the Node 8.5.0+ `--experimental-modules` implementation so that a new implementation could be built in its place: + +* Remove support in the `import` statement of formats other than ESM: + - No CommonJS. + - No JSON. + - No native modules. + +* Remove dynamic path searching: + - No extension adding. + - No directory resolution, including no support for `index.js` or `index.mjs`. + - No support for `main` field for ESM. + +* Remove current VM implementation + +* Remove current Loader implementation + +These changes were implemented in https://github.com/nodejs/ecmascript-modules/pull/6. + +## Phase 1: The Minimal Kernel + +The “minimal kernel” consists of features that the @nodejs/modules group have agreed will be necessary for all potential iterations of our ESM implementation. Phase 1 does _not_ include features that preclude other potential features or implementation approaches; and Phase 1 also does not include some features that should naturally be built in a later phase of development, for example because those features depend on features planned for Phase 1. + +* `module.createRequireFromPath` ([nodejs/node#19360](https://github.com/nodejs/node/pull/19360)) is the only way to import CommonJS into an ES module, for now. + - `import.meta.require` fails at runtime as opposed to import time. This is not desireable to all committee members. + - Hold off on `import` statements for CommonJS until more progress is made on the dynamic modules spec. + - Landed in core in https://github.com/nodejs/node/commit/246f6332e5a5f395d1e39a3594ee5d6fe869d622 + +* `import` statements will only support files with an `.mjs` extension, and will import only ES modules, for now. + - No JSON or native modules; `createRequireFromPath` can be used to get these. + +* `import.meta.url`. + - Already in the existing implementation. + +* Dynamic `import()`. + - Already in the existing implementation. + +* Support for built-in modules with named exports + - Already in the existing implementation. + +## Phase 2: Minimum Viable Product: Required to Upstream + +Phase 2 fleshes out the implementation with enough functionality that it should be useful to average users as a minimum viable product. At the completion of Phase 2, the old `--experimental-modules` implementation was replaced with this new one (still behind the `--experimental-modules` flag). + +* Define semantics for importing a package entry point, e.g. `import _ from 'lodash'` + - Proposal: [“File Specifier Resolution” proposal](https://github.com/GeoffreyBooth/node-import-file-specifier-resolution-proposal) covers bare module specifier resolution of CommonJS packages. + - Landed in https://github.com/nodejs/ecmascript-modules/pull/28. + +* Define semantics for determining when to load sources as CommonJS or ES module for both the top-level main (`node x.js`) and dependency loading. + - Proposal: [“File Specifier Resolution” proposal](https://github.com/GeoffreyBooth/node-import-file-specifier-resolution-proposal) covers `import` statements of ESM files; and CommonJS files, package entry point and package deep imports. + - Landed in https://github.com/nodejs/ecmascript-modules/pull/28. + +* Define semantics for enabling ESM treatment of source code loaded via `--eval`, STDIN, and extensionless files (both with and without shebang lines). + - Proposal: [“Entry Points Proposal”](https://github.com/geoffreybooth/node-esm-entry-points-proposal) covers non-file forms of input as well as adding `--type` flag for controlling file-based input. + - Landed in https://github.com/nodejs/ecmascript-modules/pull/32. + - Renamed to `--entry-type` as part of upstream [PR](https://github.com/nodejs/node/pull/26745) to Node.js core. + - Renamed to `--intry-type` and limited to `--eval`, `--print` and `STDIN` as part of follow-up [PR](https://github.com/nodejs/node/pull/27184) to Node.js core. + +* File extension and directory index searching in ESM, behind its own flag, `--es-module-specifier-resolution`. + - See https://github.com/nodejs/modules/issues/268. + - Landed in https://github.com/nodejs/ecmascript-modules/pull/48. + +The work through the end of Phase 2 landed in Node.js `master` as part of https://github.com/nodejs/node/pull/26745 and was released in Node 12.0.0. + +## Phase 3: Path to Stability: Removing `--experimental-modules` Flag + +Phase 3 improves user experience and extends the MVP. Phase 3 is malleable based on how things proceed while working on this phase. At the end of this phase, the `--experimental-modules` flag is dropped. + +* Better mechanism for creating `require` function: `createRequire`. + - Landed in https://github.com/nodejs/node/pull/27405 and shipped in 12.2.0. + +* `"exports"` field: for consumers of a package, map the paths of deep imports to provide encapsulation (an explicit public API); pretty specifiers (no file exensions or paths that include things like `dist/`) and flexibility for future package versions renaming or moving files without affecting the package’s public API. Applies to both ESM and CommonJS. + - Proposal: [Package Exports Proposal](https://github.com/jkrems/proposal-pkg-exports). + - Landed in https://github.com/nodejs/node/pull/28568 and shipped in 12.7.0 behind `--experimental-exports`. Further improvements are being made as additional PRs against core. + - The separate `--experimental-exports` flag was dropped in https://github.com/nodejs/node/pull/29867, merging the feature with overall `--experimental-modules`. + +* Define behavior for builtin globals between CommonJS and ESM. Does modifying a builtin in one module system carry over into the other? If it does, we may have major performance concerns. + - Issue raised in: https://github.com/nodejs/node/pull/29426. + - Solution was to not sync bindings automatically, but provide an API to manually sync them when desired: `module.syncBuiltinESMExports()`. + - Landed in https://github.com/nodejs/node/pull/29737 and shipped in 12.12.0. + +* Shortcut to resolve to package root. + - Proposal: [Package `"name"` Resolution Proposal](https://github.com/guybedford/package-name-resolution). + - Discussion: https://github.com/nodejs/modules/issues/306. + - Landed in https://github.com/nodejs/node/pull/29327 behind the flag `--experimental-resolve-self`. + +* Finalize behavior of `import` of CommonJS files and packages. + - Overview: https://github.com/nodejs/modules/issues/264. + - At time of unflagging: `import` only the CommonJS default export, so `import _ from 'cjs-pkg'` but not `import { shuffle } from 'cjs-pkg'`. + - [Conditional exports](https://github.com/nodejs/node/pull/29978) allows creating an ESM wrapper to provide named exports of an otherwise all-CommonJS package; see [“Approach #1: Use an ES Module Wrapper.”](https://github.com/nodejs/node/blob/master/doc/api/esm.md#approach-1-use-an-es-module-wrapper) + - No further improvements are expected. + +* Dual CommonJS/ESM packages: Support packages with both CommonJS and ESM sources that can be used in either environment. + - At time of unflagging: `"main"` (or `"exports": { ".": "file.js" }` overriding `"main"`) points to exactly one file, and full filenames are required (by default), so there is no possibility of an `import` specifier pointing to different files in ESM versus CommonJS; unless `--experimental-conditional-exports` is used (see next bullet). Without that flag, dual packages must provide secondary entry point via a path, e.g. `'pkg/module'` or `'pkg/commonjs'`. + - With `--experimental-conditional-exports`, paths within the `package.json` `"exports"` block can have separate entry points per environment. + - Landed in https://github.com/nodejs/node/pull/29978 and https://github.com/nodejs/node/pull/30051 behind `--experimental-conditional-exports` flag. + - Unflagged in https://github.com/nodejs/node/pull/31001 and shipped in Node 13.7.0. + +## Phase 4: Further Improvements After Unflagging + +Now that the implementation has shipped, further efforts are listed on the [README](../README.md). diff --git a/doc/meetings/2018-07-18.md b/doc/meetings/2018-07-18.md new file mode 100644 index 0000000..28736a7 --- /dev/null +++ b/doc/meetings/2018-07-18.md @@ -0,0 +1,125 @@ +# Node.js Foundation Modules Team Meeting 2018-07-18 + +* **Recording**: https://www.youtube.com/watch?v=_k63DkTJpUI +* **GitHub Issue**: https://github.com/nodejs/modules/issues/154 +* **Minutes Google Doc**: https://docs.google.com/document/d/1zruFPTN9dIpvSEkStxo4fkPenXTdhsLEVkcmoKlVmOA/edit + +## Present + +- Gus Caplan (@devsnek) +- Jan Krems (@jkrems) +- Ben Newman (@benjamn) (took notes below) +- Michael Dawson (@mhdawson) +- Matt DuLeone (@mduleone) +- Bradley Farias (@bmeck) +- Guy Bedford +- Jordan Harband (@ljharb) +- Geoffrey Booth +- Jeremiah Senkpiel +- Kevin Smith +- Geoffrey Booth (@GeoffreyBooth) +- Saleh Abdel Motaal (@SMotaal) + +## Agenda + +Extracted from **modules-agenda** labelled issues and pull requests from the **nodejs org** prior to the meeting. + +### Update on Progress (15 minute Timebox) + +* Pull request opened for import.meta.require on core [#130](https://github.com/nodejs/modules/issues/130) + - 3 minute timebox + - Jordan: philosophical objection to implementing something we haven’t come to consensus on yet +* Thinking about deadlines [#123](https://github.com/nodejs/modules/issues/123) + - 3 minute timebox + - Jordan: this working group wasn’t convened for expediency, so we should make sure we’re shipping the right thing above all else + - Bradley: there’s been a thought going around about starting a very minimal implementation that we can all agree on, so that we can decide about future features one at a time (more so than now) +* Initiative: Terminology / Historical Decisions documents [#119](https://github.com/nodejs/modules/issues/119) + - 3 minute timebox + - Jan: Myles said probably no progress this week, but next week he may have more time + - Guy: let’s get an update on that from Myles next week then +* Developer Survey [#85](https://github.com/nodejs/modules/issues/85) + - 3 minute timebox + - Guy: goal should be to create agenda items to discuss in these meetings, and also iterate on between meetings + - Michael Dawson: a GitHub issue would make it easier for folks to add ideas asynchronously + - Bradley: survey shouldn’t just be about user expectations, since we can violate those in service of meeting user needs +* Features List Adjustments #156 + - 3 minute timebox + - Guy: removing duplicates and flagging features that weren’t on the README page + - Unfortunately don’t have a quorum to merge PRs + +### Discussion + +* Package-Name-Maps a proposal for bare imports in browsers [#51](https://github.com/nodejs/modules/issues/51) + - 5 Minute Timebox + - Jordan: summarizing: + - Static file that declaratively tells the browser how to resolve bare specifiers to final URLs + - Has nested structure (scopes) to support deep imports from packages + - Should be easy to create to adapt Node code to browsers + - Question: Should Node provide support for optional package name maps, too? + - Bradley: + - Folks are making assumptions about the limits of package name maps without fully investigating the feature set + - For example, you *can* resolve directories and missing file extensions + - Michael Dawson: + - How does this fit into overall priorities? Can we think about this later on? + - Geoffrey Booth: + - I’m trying to avoid code that behaves differently in Node and browsers, so package name maps are a compatibility concern for me + - As packages add maps, they could cause breaking changes in resolution, e.g. for code doing deep imports from those packages + - Lots of value in aligning with the browser on this, though + - Bradley: I don’t think this is the same level of compatibility concern + - Always going to be some mismatch between how Node and browsers work, so we should be considering the subset of functionality that works the same in both + - Bigger compatibility problems with caching, import.meta.url, importing JSON, HTML modules + - Node is not a subset of the web + - Using tools to generate package name maps solves some of the problems here + - Guy: + - Package name maps can only map bare specifiers, so ./ specifiers aren’t addressed by the proposal + - Bradley: + - Shouldn’t make Node dependent on what browsers do here + - Other approaches to solve the same problems, such as loader hooks + - Michael Dawson: + - Node probably would not want to allow mapping bare specifiers to external URLs + - Undermines Node’s security model, though it works in browsers + - Guy: Relative import specifiers are one of the big unresolved problems + - Bradley: that’s something the designers of package name maps should think about + - Michael Dawson: + - Maybe we should be providing input to the design of package name maps rather than discussing whether to implement them as-is + - Guy: + - If anybody believes there is something we should prioritize on this front please raise, otherwise might be left until we figure out some of the other issues. +* Have presentation on loaders. [#135](https://github.com/nodejs/modules/issues/135) + - 20 Minute Timebox + - Purpose: get folks to stop thinking of loaders as an amorphous API that allows anything + - Instead, talk about what loaders actually do, and must be able to do + - Discussion: + - MD: Are facades not able to change / add things to the module? + - BF: when we say code is ESM, we don’t touch it, we just load it directly + - MD: Why would we do otherwise? + - BF: Changing the ordering of imports would be an example of not directly importing an ESM module + - BF: We’re following the spec as long as resolution returns an ESM module record + - Proposal: + - Move loaders off the main thread for isolation and performance + - Command line --loader option + - Environment variables + - Per-package loader hooks configured in package.json? (BF in favor) + - Works in user-land currently, though difficult to keep rebasing it against Node master + - See slides + - Differences between this proposal and existing ideas + - See slides + - Issues + - Caching is tricky, since it works so differently in the browser than other tools + - Would prefer to remove caching from the proposal + - Raw amount of time spent doing transformations is significant + - Caching compilation results should therefore be easy/simple + - Can’t solve this problem in this proposal, but should show it can be solved +* transparent-or-not interop [#90](https://github.com/nodejs/modules/issues/90) + - 20 Minute Timebox + - Skipped this due to running out of time + +Saleh Abdel Motaal on terminology document: + +* Not much to say about terminology document progress +* Two documents + - One document went “way overboard” (first draft) + - Another took a different direction with input from other people +* Guy: can we add one of those documents as a link to the list of resources that we maintain? +* Saleh: yes, will create a markdown document in the repo +* Sounds good to everyone + diff --git a/doc/meetings/2018-08-01.md b/doc/meetings/2018-08-01.md new file mode 100644 index 0000000..d69d6f7 --- /dev/null +++ b/doc/meetings/2018-08-01.md @@ -0,0 +1,128 @@ +# Node.js Foundation Modules Team Meeting 2018-08-01 + +* **Recording**: https://www.youtube.com/watch?v=VakyGVeMUpg +* **GitHub Issue**: https://github.com/nodejs/modules/issues/161 +* **Minutes Google Doc**: https://docs.google.com/document/d/11oDyvFSnPp3mkyvr8SwRQTKo4aEu9Z0khIiQKPdT1Yk/edit + +## Present + +- @MylesBorins (Myles Borins) +- @jdalton (John-David Dalton) +- @mduleone (Matt DuLeone) +- @giltayar (Gil Tayar) +- @GeoffreyBooth (Geoffrey Booth) +- @devsnek (Gus Caplan) +- @weswigham (Wesley Wigham) +- @DanielRosenwasser (Daniel Rosenwasser) +- @SMotaal (Saleh Abdel Motaal) +- @mhdawson (Michael Dawson) +- @targos (Michael Zasso) +- @benjamingr (Benjamin Gruenbaum) +- @robpalme (Rob Palmer) + +## Agenda + +Extracted from **modules-agenda** labelled issues and pull requests from the **nodejs org** prior to the meeting. + +### Approving PRs (2 Minute Timebox) + +* Add Sendil Kumar as an observer [#159](https://github.com/nodejs/modules/pull/159) + * No objections + +### Update on Progress (10 Minute Timebox) + +* Have presentation on loaders. [#135](https://github.com/nodejs/modules/issues/135) + - 3 minute timebox + * Need to remove modules agenda +* Create Terminology.md [#158](https://github.com/nodejs/modules/pull/158) + - 3 minute timebox + * Last week at a point where it was just Google docs - switched to markdown going forward. + * Anyone who has any different ideas should weigh in. +* Developer Survey [#85](https://github.com/nodejs/modules/issues/85) + - 3 minute timebox + * Not had time to dig into this + * Anyone else have bandwidth to take over this? + * Mostly silence :) + * Conclusion: Let's kick it back to the discussion thread. + +### Discussion (45 Minute Timebox) + +* ESM in .js Files: + - 15 minute timebox + - Proposals for configuring parse goals of files in --experimental-modules [#160](https://github.com/nodejs/modules/pull/160) + - ESM in .js files proposals [#150](https://github.com/nodejs/modules/pull/150) + + * Issues around use-cases where Node can parse a .js file with an ESM parse goal. + * Have come to some consensus that Node needs to support this somehow + * Doesn't mean that we're getting rid of ESM, just that we want to tackle this use-case + * Have some ideas to tackle this. + * Jordan: Agree with Geoffrey's stance; really sounds like this is a mechanism for overriding the default parse goal for a file. + * Bradley: Happy that this is a cooperative approach and would love to see more of this in the group. Whatever we design needs to be well-thought-out, and whatever features we produce shouldn't step on other features' toes. + * Michael: Is there any chance that some of the existing use-cases we have can form the basis of what we want? + * Need to determine which use-cases are satisfied by a given solution. + * Geoffrey: These use-cases started as "these are the things I can't do in `--experimental-modules`". + * Bradley: any implementation won't be flawless, which is why we have so much to discuss. But perhaps we can have some agreed-upon "base" implementation to base off of. + * Like a kernel implementation with bare-bones functionality. + * Myles: perhaps we can spin up an issue that discusses what this is, then adding the other features that we want later. + * Perhaps one discussion would be around making a more restrictive loader. + * May mean that this is a version of the module system nobody would want to use, and would discourage people from using it in production, but would elicit feedback + * Gil: think it's a really good idea; starts up from ES modules and builds consensus slowly. + * Saleh: two points + * Always consider what would happen if we started off with ESM and implemented a CJS loader within ESM. + * Started off thinking this way, but initially wasn't enough bandwidth when first implementing. + * We would need to be careful as to which features are bottlenecks and which features are the essentials + * Myles: have we found consensus? should we table the discussion if so? + * Jordan: we should make sure that we have consensus; doesn't look like we have consensus + * Daniel: sounds good on a surface level, but if you solve each of the other use-cases, unsolved use-cases might be "left behind" and unsolved + * Jordan: Don't care what we ship flagged, but the first unflagged thing we have should *not* be minimal. + * But need to be careful because big userland features that go flagged for a long time gain momentum + * Jeremiah: agree with Daniel and Jordan + * also, unsure how minimal you can get with a big feature like this + * Bradley: idea was that this minimal kernel doesn't ship anywhere + * Gil: Agree with Brad - probably don't even have to have an actual implementation + * Jeremiah: Okay with last few discussion points + * Michael: want to have some place to land code changes - maybe a fork of the repo. + * Myles: can incubate that kernel in the fork. + * Any objections? + * Geoffrey: still want to be able to improve it + * Should probably talk about ideas of deprecation warnings at the next meeting. + +* Features adjustments and grouping [#156](https://github.com/nodejs/modules/pull/156) + - 10 minute timebox + * Think this was just around fine-tuning certain things around the repo. + * Conclusion: consensus around approval + + +* transparent-or-not interop [#90](https://github.com/nodejs/modules/issues/90) + - 10 minute Timebox + * Two ideas: use of `import` interop and `require` interop + * `import` interop = importing CommonJS modules + * `require` interop = requiring ESM modules + * Some people felt that interop is required for a minimal implementation. + * Wesley: if we're talking strong feelings for interop + * First expectation is the ability to require or import things, and that regardless of whether the required/imported module is authored in ESM and CJS. + * Second expectation is the expectation that ESM consumers should be able to get a reasonably similar API regardless of a dependency's format + * Bradley: what do you specifically mean by that? + * Wesley: authors need to be able to upgrade to ESM; ESM consumers need to be able to consume the same way regardless of the library format. + * Jeremiah: some level of interop a requirement, static import of cjs important, note on require-ing esm: considered bad when I had previously consulted JS teacher folks for the TSC in early 2017 (See https://github.com/nodejs/CTC/blob/master/meetings/2017-02-01.md - https://youtu.be/8Snc5lh0VZQ?t=23m29s) + * Myles: some level of interop is definitely expected + * import() from CJS is likely the recommended mechanism for + * Importing Common.js can be supported but shouldn’t be default + * Likely to fracture the ecosystem if we do otherwise. + * Important to specify a MIME type for .mjs + * Brad: Clarify: do you want .js to be reserved for CJS? + * \[\[Fill this in]] + * Gil: require() ESM is a no-go for me: just use await import() -. + * The current implementation enables you to import CJS from ESM, and is called “‘transparent” but it only (correctly!) allows it to come up as a default import. That’s a no go for me too—most transpiled code today assumes “real” transparent interop (i.e. the ability to use named imports), and so this is not very transparent for it. So if we can’t have it real, then let’s not have it at all. + * Jordan: TC39 enthusiastically said they would strive to make sure CJS can interoperate with ESM. + * Bradley: sounds like this isn't something we want in a minimal implementation + * Gil: or any implementation + * Saleh: I don't consider built-in modules to be CJS. + * "static import" vs "dynamic import" is something we should be careful of because it implies a completely different story. + * Bradley: you want to treat static/dynamic imports as separate mechanisms? + * Saleh: It is more that if I am writing cjs and opt to import(…) then it could be a good way to future proof my codebase as I always expect a promise. If I statically import cjs into esm, then I can rely on module.exports.default = with sync or async in play. + * Wesley: If TC39 is willing to grant us the ability to have named imports, we should take advantage of that. + * Daniel: what community fracture did we have in mind before? + * Jeremiah: care about things actually working well for Node, so being able to statically import CJS is very important. + * Myles: not against interop, just not sure if it should be the default + diff --git a/doc/meetings/2018-08-15.md b/doc/meetings/2018-08-15.md new file mode 100644 index 0000000..0b8da43 --- /dev/null +++ b/doc/meetings/2018-08-15.md @@ -0,0 +1,107 @@ +# Node.js Foundation Modules Team Meeting 2018-08-15 + +* **Recording**: https://www.youtube.com/watch?v=YiXrnHu8BSI +* **GitHub Issue**: https://github.com/nodejs/modules/issues/167 +* **Minutes Google Doc**: https://docs.google.com/document/d/11SQ4Ib5jdyusHPJfGG_aJR2lsTAX1oAiiGklL2EHxhw/edit + +## Present + +- @devsnek (Gus Caplan) +- @mduleone (Matt DuLeone) +- @giltayar (Gil Tayar) +- @GeoffreyBooth (Geoffrey Booth) +- @guybedford (Guy Bedford) +- @MylesBorins (Myles Borins) +- @LJHarb (Jordan Harband) +- @zenparsing (Kevin Smith) +- @jdalton (John-David Dalton) +- @jkrems (Jan Krems) +- @inidaname (Hassan Sani) +- @fishrock123 (Jeremiah Senkpiel) +- @SMotaal (Saleh Abdel Motaal) + +## Agenda + +Extracted from **modules-agenda** labelled issues and pull requests from the **nodejs org** prior to the meeting. + +### approving PRs (2 minute timebox) + +* Nominate @SMotaal as an active member [#162](https://github.com/nodejs/modules/issues/162) +* No quorum to nominate... + +### Update on Progress (4 minute timebox) + +* Create Terminology.md [#158](https://github.com/nodejs/modules/pull/158) + - 2 minute timebox + * Saleh is not in the meeting, so we will move to next meeting + +* Developer Survey [#85](https://github.com/nodejs/modules/issues/85) + - 2 minute timebox + * the person responsible is not in the meeting, so we will move to next meeting + +### Discussion (50 minute timebox) + +* Thinking about deadlines [#123](https://github.com/nodejs/modules/issues/123) + - 5 minute Timebox + * Myles: The previous sentiment that we do not want deadlines just so that there will be deadlines. We need to reach consensus without the pressure of deadlines. With that being said, we have been on this for a while, and yet do not have any insight on when we may have something to release without a flag. There are concern from the steering committee that we need to ship. Large organizations want the Node project to ship something so that they will have guidance on how it will work. I believe we should have a goal of shipping without a flag before Node 12, which is about 14 months, and maybe backport it if possible. Anybody has any ideas on this. + * Geoffrey: I don’t have a specific suggestion. + * Saleh: checkpoints, like having a prototype, or having the terminology document. A structural flow of how the checkpoints interdependent, more than the actual dates. +* Jan: I agree with the checkpoints. For me personally, it’s not the dates, but any way of expressing progress would be good. A list of decisions we can make, so that we can see that there is progress. + +* Managing fork and agreed upon minimal kernel [#166](https://github.com/nodejs/modules/issues/166) + - 10 minute timebox +* Myles: this goes well with the next agenda meeting. The decision last meeting was to strip apart the current implementation. Coming to a consensus around what a minimal implementation is that is inclusive to all future implementations. And then create PRs that add to that. There was a discussion on that: fork? Branch? And the other discussion would be what would be part of this. Perhaps milestones around that minimal implementation and additions to it. + * Gus: I want to keep it as close as possible to other core development. There are a lot of concerns when developing other parts of node, so we need to communicate the changes. So I don’t want it as another repo, ideally, but at least communicate the things outside of ourselves. + * Myles: the http/2 implementation was a fork, that was currently rebased. And maybe we should explore that. + * Geoffrey: I’m OK with that. If it would have been my project, I would take the current implementation, move that out to a PR against the core, and then the minimal implementation would be another PR. + * Myles: removing things from master is a non-starter, and it would be a huge disruption to the ecosystem. + * Geoffrey: why? + * Myles: we need to reach consensus, and we can’t create lots of PRs and changes to master, as it will create lots of noise. + * Saleh: whenever I saw the module wrap I thought it could be a native module. If we create a native module that is a separate implementation. If this module would have the same hooks, then we could also port it to other stuff like Electron. We could stay as close to module wrap or as far away as we need. I prefer being close to module wrap. + * jdalton: a repo within the org is fine. We can create multiple branches within it to scope each discussion. That’s what we did with worker, and doing it with REPL. + * Kevin: I also like the fork idea. We need to figure out who’s in charge of rebasing upstream. + * Jan: to the idea of making it into a library that we can install into the existing node, because then people can use it and try it out without building a fork. Huge advantage, but the problem is that it works for the regular import, tricky for dynamic import, and almost impossible for import.meta. We need changes to the v8 api to make this possible. + * Myles: how can we improve that? + * Jan: if we fork, to try out, people will need to build, unless we do nightly. + * Myles: the build people can create nightlies from the fork. + * Jan: yes, but there is a reason why people don’t try out nightly builds. Getting nightly builds will address some of the pain. If we want is as a native module, we need to ???. + * Gus: for the idea of external module, there are things we need to do ???. It would be really cool to do, but it’s not worth pursuing at this moment. + * Saleh: I see this as an opportunity to house clean. When we write isolated modules, we tend to uncover tight couplings that are hidden because it’s all in one repo. Yes, there will be technical challenges, I think it’s high time to clean up things that have happened over the implementations. There are certain things that might be closed. I hope that people buy into the idea of the native module. + * Myles: we can kick off an issue for that, and given that forking seems like the most buyin, we can do that in the fork repo. + * Myles: let’s continue on the discussion of what will be in the minimal implementation. First Gil’s [reading Gil’s issue] + * Gil: it’s basically Myles’ option, but including the regular node module resolution. + * Myles: [reading Myles’ issue]. Do people have any thoughts on this? + * Jan: it would be important what we are trying to prove or try out. For me, a foundation that everyone can agree on, and then start ??? For me, createRequireFunction is ??? * Myles: ??? The idea with createRequireFunction. Import does not have transparent interop, so we need some kind of basic thing to support interop. + * Saleh: ??? I think anything that is node specific for the minimal implementation, might in most cases, ??? + * LJHarb: the hard part is deciding what ships. If the group decides this is important, then fine, but I feel that this is delaying the final implementation. + * Myles: I’ll create the fork and we can iterate on it. + +* ESM in .js files: + - ESM in .js files proposals [#150](https://github.com/nodejs/modules/pull/150) + - [#160](https://github.com/nodejs/modules/pull/160) + - 10 minute timebox + - Proposals for configuring parse goals of files in --experimental-modules + * Myles: Can Geoff update us on this issue? + * Geoffrey: at least for CoffeeScript (and others), they only support .js as output. So there needs some way to treat .js files as ESM. This issue is—should we enable this, and #160 is some suggestions on this. + * Myles: are you implying that every folder with package.json will have a different resolution algorithm. + * Geoffrey: yes. Not sure about resolution, but definitely mapping. All the others assume that determining a package boundary is what is in the PR that enables this. Then the rest of the issue is that the merged final proposal is that there is a new “mimes” field which maps extensions to mime types. This follows the browser method of thinking about it. The final proposal takes that as a starting point, and what some people also wanted was a way to define these through external files as well. So either an object, inline, or an array of strings that are an array of json files. If it starts with “null”, then we erase the default mappings, otherwise we start with the default node mappings. I was kind of hoping that this is something we want to sign on. + * LJHarb: I like the concept. It’s generic and can be per-package. The only concern is that using mime types is confusing. The web does have mime types, but others don’t necessarily understand this. So my preference is mapping of extension to extensions. But I am all for the idea. + * Myles: question: where do the list of available mimes come from? Is there a list of definitive mime types? Can you do application/typescript? + * Geoffrey: you know the code better than I do. I assume given a mime type, it knows how to treat each file extension. This would be making that more explicit. Instead of saying “for .json do this”, it would say “for application/json do this”. And there would be a mapping. So for all file types that node supports, there would be mime types. And mime types that are not default, maybe the loaders will be able to support this. + * Myles: I believe there is an abstraction that seems a bit ???. To me the mental model that makes more sense is the connection to the loader hooks. I would love to see how that would work and how it fleshes out. + * Saleh: I love the declarative aspect of it. I’m against mime types outside of the browser, but I believe we should address the idea of declaring things in a certain folder. package.json makes sense to use as for this. + * Jan: My problem with the proposal, for me it’s either everyone agrees with the defaults, or everybody adds the same two scaffolding lines everywhere. It feels like something that is opting into complexity and support problems. + +* transparent-or-not interop [#90](https://github.com/nodejs/modules/issues/90) + - 10 minute timebox + * Myles: one of the reason that we are talking about this. Benjamin? + * Benjamin: listening to Geoffrey talking about CoffeeScript, makes me realize that generating new modules and ???. If it’s ESM, we don’t need anything, but if we want to enable using import declarations to import ESM, and for other stuff to get wrapped as ESM modules, then we can just import mymodule.coffee. But if we need import.meta.require, then, I guess I believe there is a future where we don’t need to worry about file extensions and such. + * LJHarb: browsers are definitely going to be using mime types to parse what you import. In browsers it’s either parsed as script or as a module. + * Myles: But the file extension is usually how the server determines mime type. + * Jan: I think it’s problematic to say that everything imported is only ESM. But for most practical purposes the browser has a whitelist of extensions that is allowed to be imported as ESM. + * Jeremiah: I’m not OK that the only way to import is using “import”. Making it so that if we import CJS, it would import CJS and wrap it with an ESM. + * Benjamin: yes, CJS is the exception because it uses the same .js extension as ESM should. But the others, like TypeScript, there is less of a question there and we just need to figure out how to configure this. + * Jan: the problem with enabling importing things like html is that this will be a future backward compatible problem, because we will not in core be able to use that file extension in the future. And it can be really confusing if half of the ecosystem is using one file extension and another is using another file extension for the same purpose. + + [Oops. Meeting got cut off.] + diff --git a/doc/meetings/2018-08-29.md b/doc/meetings/2018-08-29.md new file mode 100644 index 0000000..6d07365 --- /dev/null +++ b/doc/meetings/2018-08-29.md @@ -0,0 +1,89 @@ +# Node.js Foundation Modules Team Meeting 2018-08-29 + +* **Recording**: https://www.youtube.com/watch?v=u9P-nN-FDgA +* **GitHub Issue**: https://github.com/nodejs/modules/issues/172 +* **Minutes Google Doc**: https://docs.google.com/document/d/1DhUZYo8Cw-CNZCnnsV8qVPA0WeK6SH2kt971Rt0DqR0/edit?usp=sharing + +## Present +* Wesley Wigham (@weswigham) +* Daniel Rosenwasser (@DanielRosenwasser) +* Geoffrey Booth (@GeoffreyBooth) +* Bradley Farias (@bmeck) +* Guy Bedford (@guybedford) +* Gus Caplan (@devsnek) +* Saleh Abdel Motaal (@SMotaal) +* John-David Dalton (@jdalton) +* Kevin Smith (@zenparsing) +* Jeremiah Senkpiel (@fishrock123) +* Jordan Harband (@LJHarb) +* Rob Palmer (@robpalme) + +## Agenda + +Extracted from **modules-agenda** labelled issues and pull requests from the **nodejs org** prior to the meeting. + +### approving PRs (2 minute timebox) + +* Nominate @SMotaal as an active member [#162](https://github.com/nodejs/modules/issues/162) + +### Update on Progress (5 minute timebox) + +* Create Terminology.md [#158](https://github.com/nodejs/modules/pull/158) + - 2 minute timebox +* Developer Survey [#85](https://github.com/nodejs/modules/issues/85) - proposal - https://github.com/nodejs/modules/issues/173 + - 2 minute timebox + - Don't have any specifics in mind on what we want to know with this survey, but a systematic process could help us get started. + - Someone who comes from this forum will already have information about the survey contents. + - Don't want to write this survey in one person's frame of reference. + - When you ask questions in a particular way, this can sway how people perceive the question and how they'll answer. +- next step: get 4-8 people involved in iterations for the survey + +### ECMAScript Modules PRs (25 minute timebox) + +Extracted from PRs against https://github.com/nodejs/ecmascript-modules prior to the meeting. + +* import.meta.require [#1](https://github.com/nodejs/ecmascript-modules/pull/1) + + * Brad: This fork is something we're putting PRs on, and we explicitly stated the fork might not be pleasant to use. + * We're weighing on whether this fork is something that is "pretty and minimal" or which has more controversial features. + * Geoffrey: Figured initial features would be the least controversial features, and yet this (CJS interop) is the first PR. + * Do we really need this in the first iteration? + * No, but it means that npm is unusable so it's not totally useful. + * Jordan: What's the goal of the implementation? Is it a playground? Is the hard part the implementation? I think the hard stuff is agreeing on stuff, not the implementation. + * Also, what does minimal mean? If it can't work with npm, is it minimal? + * \[\[Someone else has to fill this in, we were having video driver and/or connection issues]] + * Brad: Currently with this implementation, it seems like we're saying the web doesn't matter. + +* remove file extension and directory resolving [#2](https://github.com/nodejs/ecmascript-modules/pull/2) +* default extensions and directory resolving across package boundaries [#4](https://github.com/nodejs/ecmascript-modules/pull/4) + * There's a desire to treat bare names as packages. + * Brad: this is being done in the same name as package name maps, and people are saying "this can be done with ahead-of-time tools". + * Brad: I object to this, not just because the spec is so alpha, but you'd also have to convince me that they're worthwhile. +* remove ".json" default loading [#3](https://github.com/nodejs/ecmascript-modules/pull/3) + * Idea: you can still import from CJS with require, but not with ES modules + * Geoffrey: I recalled that we'd add in importing JSON later on. + * G: instead of removing things, do we need to rethink what we're even going for? + +### Discussion (30 minute timebox) + +* Browser / bundler compatibility [#170](https://github.com/nodejs/modules/issues/170) + * How much should we be tailoring the implementation to the needs of browsers and bundlers? + * Brad: We've seen a miniature version of this feature in past meetings: different people prioritizing different things like existing tools, browsers, and new design space. + * Stripping everything out is the most neutral view of the situation. We need to come to an agreement about what the level of support we care about is. + * Geoffrey: we already came to an agreement on all of this when we came up with a prioritization list. + * Saleh: if we try to turn these questions into something we can ask in the survey, it'd be easier to define the goals. + * Kevin: there will be a lot of programs that *won't* care about programs that work in the browser; we should be considering programs that are written for the the intersection as well as the exclusion as first-class. + * Brad: I want to explicitly state that I think that's a bad idea. Prioritizing one paradigm makes the design space easier. Have to ask how much value is added in this intersection when you consider how it impacts other targets? Choosing one won't lead to all scenarios being ideal, but choosing a prioritized set of paradigms will narrow down the design space, and we have a *very* large design space. + * Guy: If you think about how bundlers work nowadays, they're all based on conventions set by Browserify. Node resolution, importing json, polyfilling process environment variables, etc. - all continued past Browserify. And that was all based on Node. So what we're doing here is setting a precedent for other tools. + * Saleh: While it's good to keep those things in mind, people struggle based on the precedent based on Browserify. + * Wesley: coming at the module design with the mindset that "what people've done with modules is wrong". If I can't continue to do the things + * Bradley: we can't just *do* what Webpack does, or what TypeScript does. We've discussed this for months. There are issues with how the spec is + * Wesley: If the ecosystem is so divorced from the runtime, why are we starting with a different runtime entirely? The idea is we want to capture the existing userbase of Node. + * Going with an intersection is probably the worst thing you can possibly do. I can see two stances: either make things as smooth as possible for Node, or prioritize compatibility with the browser. In either case, you're going to have friction, so the middle ground is going to cause frustration there. +* Brad: what if we can't provide the compatibility compilers have? +* \[\[Fill this in]] +* Jeremiah: the problem with modules the entire time is that we've been stuck in this middle ground. +* Wesley: existing module systems are present and have the goal of emulating modules systems in the ecosystem +* Managing fork and agreed upon minimal kernel [#166](https://github.com/nodejs/modules/issues/166) +* Proposals for configuring parse goals of files in --experimental-modules [#160](https://github.com/nodejs/modules/pull/160) + diff --git a/doc/meetings/2018-09-12.md b/doc/meetings/2018-09-12.md new file mode 100644 index 0000000..8f8b9a5 --- /dev/null +++ b/doc/meetings/2018-09-12.md @@ -0,0 +1,103 @@ +# Node.js Foundation Modules Team Meeting 2018-09-12 + +* **Recording**: https://www.youtube.com/watch?v=y2Ce_0z5qK4 +* **GitHub Issue**: https://github.com/nodejs/modules/issues/179 +* **Minutes Google Doc**: https://docs.google.com/document/d/1G6siAcmcpza6wyxoMv0cPWlCLpoIoS7duqPSXBu3AIU/edit + +## Present + +- Daniel Rosenwasser @DanielRosenwasser +- MylesBorins, @mylesborins +- Matt DuLeone, @mduleone +- Kevin Smith, @zenparsing +- Michael Zasso, @targos +- Jan Krems, @jkrems +- Sendil Kumar @sendilkumarn +- Geoffrey Booth @GeoffreyBooth +- Wesley Wigham, @weswigham +- Jeremiah Senkpiel @fishrock123 +- Saleh Abdel Motaal @SMotaal +- Guy Bedford @guybedford +- Gus Caplan @devsnek + +## Agenda + +Extracted from **modules-agenda** labelled issues and pull requests from the **nodejs org** prior to the meeting. + +## Agenda + +Extracted from **modules-agenda** labelled issues and pull requests from the **nodejs org** prior to the meeting. + +### Update on Progress (6 minute timebox) + +* Session at Collab Summit Vancouver [#177](https://github.com/nodejs/modules/issues/177) + * Have time on the agenda specifically to talk about modules in Node + * If you're interested in travel fund/reduced fee for entry, reach out to Myles. +* Create Terminology.md [#158](https://github.com/nodejs/modules/pull/158) + * Would like to introduce a process for coming to consensus of well-understood terms among the group. + +### approving PRs (2 minute timebox) + +* @rubys requesting membership [#176](https://github.com/nodejs/modules/issues/176) + * Welcome to the team! + +### Discussion (40 minute timebox) + +* Thinking about deadlines [#123](https://github.com/nodejs/modules/issues/123) + - 5 minute timebox + - Saleh: Before moving forward with a particular idea, we want to be able to gather input from the community. + - Myles: Growing concerned around our inability to reach consensus. + - If we find ourselves unable to do so, I will step down. + - Context: we've had this working group for ~6 months and had difficulties here. + - Saleh: we have actually found consensus (e.g. the survey) + - Some part of this is due to lack of participation + - But keep in mind everyone has good intentions in this group and we can turn this around. + - Jeremiah: agree with Myles; if we don't see any progress for modules in Node 12, doesn't seem like there's much of a point here. + - Geoffrey: The tools in place for collaboration (e.g. GitHub) doesn't seem fit for what we're doing. + - Also, feel like we need some direction on responsibilities. Who is focusing on what? What are the expectations? + - Also, not sure where to contribute - to the minimal kernel? To the current implementation? And how so? + +* Managing fork and agreed upon minimal kernel [#166](https://github.com/nodejs/modules/issues/166) + - 25 minute timebox + - Had a hard time figuring out what a minimal kernel actually should be. + - Myles: we believe the following should be in a minimal implementation (https://github.com/nodejs/modules/issues/166#issuecomment-420767222) + - a minimal implementation needs to support bare paths (implementation TBD) + - cannot have dynamic path searching (i.e. explicit paths) + - static analysis is important here. + - With respect to CommonJS compatibility, we want to hold off on importing CommonJS modules. + - .js will eventually be able to support modules, but not + - Only support importing ESM + - Jordan: I don't necessarily disagree that this is feasible, but I would feel uncomfortable shipping without dynamic path searching. + - Saleh: Could we add a 3rd bullet list - things we're committed to actually reaching. + - Myles: I don't think we could reach consensus on that. + - Saleh: What about a list of things we acknowledge aren't actually going into this implementation so that people have a good understanding of what's missing. + - Gus: What if we flagged things outside of the minimal codebase? + - Saleh: +1 + - Myles: Only difficult thing is that it makes things messier to send PRs given that you have to account for these different paths. + - Geoffrey: We're conflating "technologically minimal" and "minimal in terms of what we can all agree upon" + - For the former: there are things that we can eventually get to even if we agree it should be done. + - Maybe we could come up with lists here to distinguish + - Saleh: part of the idea with the survey is to get consensus across the group on what we want without having to delegate to a single meeting every 2 weeks. + - Jan: problem is that the survey will consist of individual questions that may not capture the whole picture, or may affect answers to other questions. + - Wesley: are we saying we want to ship options for things we're not sure we want to ship? + - Myles: Sounds like there's a misconception. We're talking entirely about the fork. + - Guy: want to summarize the process this would create going forward and get some checks on assumptions + - My understanding was that we would build a minimal implementation based on an intersection of our consensus + - The process of adding features is to build something that could actually ship, but we are not shipping the minimal kernel itself. + - Myles: do we have any objections on bringing createRequireFunction to Node? + - Someone: I don't know enough about it to be confident. + - Geoffrey: why do we need this in Node if we can land it in our fork? + - Myles: it's generally useful functionality and we'd like to bring something useful to the project. + +* Survey: Initial Draft [#173](https://github.com/nodejs/modules/issues/173) & Developer Survey [#85](https://github.com/nodejs/modules/issues/85) + - 10 minute timebox to Talk about B1 (internal track) + - 5 minute timebox to Talk about C1 (developers track) + - Idea: these surveys are meant to capture consensus given that it is often difficult to participate for reasons such as noise over GitHub. + - Most important thing is that we need tools to reach the consensus we all need. + - Two tracks: B track (internal to this group) and C track (external devs) + - Wesley: what are the action items that these responses are going to drive? + - Saleh: we're getting there, but we need to have more people on the group help out here. + - Myles: if we talk about every feature individually, they may be "good" or "bad", but it's problematic to ask about each feature if they don't have the whole picture/context. + - Don't use the survey as a means of consensus. It's a lens to understand different groups. + - Saleh: hope is that each person who has a set of specific interests will contribute to the survey + - Myles: want to discourage too much process given that we need to ship before JS Interactive diff --git a/doc/meetings/2018-10-03.md b/doc/meetings/2018-10-03.md new file mode 100644 index 0000000..6d666b4 --- /dev/null +++ b/doc/meetings/2018-10-03.md @@ -0,0 +1,110 @@ +# Node.js Foundation Modules Team Meeting 2018-10-03 + +* **Recording**: https://www.youtube.com/watch?v=KJ5sQq9X_e4 +* **GitHub Issue**: https://github.com/nodejs/modules/issues/190 +* **Minutes Google Doc**: https://docs.google.com/document/d/1lJ94nnBEBdwadOXTay-IwgxGzOTg2WFRuHl5dWFJZKA/edit + +## Present + +- Myles Borins (@MylesBorins) +- John-David Dalton (@jdalton) +- Matt DuLeone (@mduleone) +- Wesley Wigham (@weswigham) +- Daniel Rosenwasser (@DanielRosenwasser) +- Rob Palmer (@robpalme) +- Saleh Abdel Motaal (@SMotaal) +- Geoffrey Booth (@GeoffreyBooth) +- Gus Caplan (@devsnek) +- Michael Zasso (@targos) +- Guy Bedford (@GuyBedford) +- Jordan Harband (@ljharb) + +## Agenda + +Extracted from **modules-agenda** labelled issues and pull requests from the **nodejs org** prior to the meeting. + +### Update on Progress (5 Minute Timebox) + +* Session at Collab Summit Vancouver [#177](https://github.com/nodejs/modules/issues/177) + - 2 minute timebox + - We'll have a Zoom chat during this. + - No question. +* Terminology + - 2 minute timebox + - Refs: + - Create Terminology.md [#158](https://github.com/nodejs/modules/pull/158) + - Terminology: Stipulative Terms [#184](https://github.com/nodejs/modules/issues/184) + - We were really just dealing with normative terms, but we might want to go deeper in our definitions + - Want to define stipulative terms - a more specialized way to define these ideas. + - Really specific ideas of what we mean when we say something like "browser interop" + +### Discussion (50 Minute Timebox) + +* Surveys + - 15 minute timebox + - Refs: + - Developer Survey [#85](https://github.com/nodejs/modules/issues/85) + - Surveys: Initial Drafts [#173](https://github.com/nodejs/modules/issues/173) + - Surveys: Purpose and Scope [#182](https://github.com/nodejs/modules/issues/182) + - Have time between Tuesday and Thursday to work on this. + - Didn't have a great way to get input on the survey. + - Idea: ask 3-4 people at the summit for their perception of the survey to get an idea of what we could do better. (e.g. asking questions differently, clarifying meaning of questions). + - Have a unique group who we can check with. + +* Minimal Kernel + - 30 minute timebox + - Refs: + - doc: first pass at minimal-kernel [#180](https://github.com/nodejs/modules/pull/180) + - Minimal kernel [ecmascript-modules#6](https://github.com/nodejs/ecmascript-modules/pull/6) + - Any objections to the current functionality in the minimal kernel? + +* Lack of quorum + - Do we need some people to step down to observer status? + - Also, want people to move from observer to members! + - Ironically, need to have quorum to make some changes to get easier quorum. + +* APIs around require + - Has anyone had any thoughts around the `createRequireFunction` APIs? + - There's a set of common scenarios that require composing several other existing APIs + - Wesley: Someone else is going to do this, so maybe it'd make sense to build it into core. + - Gus: import() also should subsume these scenarios. + - Myles: there will be scenarios where people want a dynamic CJS module to come in as something that's not a module. + - Gus: yes, but that's not the common scenario. + - Geoffrey: I just don't see what the use-case is here for modules. + - Gus: This was originally to solve a use-case around tooling - it can be used in an all CJS environment. + - Rob: a lot of people use `require(...)` for lazy synchronous loading, so it definitely has a use-case. + +Guy Bedford arrived! We have quorum! + +* Proposals + - Moving Kevin Smith to an active member + - Unanimous! + - Landing PR [#180](https://github.com/nodejs/modules/pull/180) + - Unanimous! + - [ecmascript-modules#6](https://github.com/nodejs/ecmascript-modules/pull/6) + - Unanimous! + +* Phases + - Myles: Sounds like we can start moving into phase 2 + - Wesley: phase 2 is when we do things that could potentially block things off in the future, right? + - Myles: potentially! But it could just be a phase where we make things more ergonomic, but not necessarily blocking. + - Rob: a document would be the most efficient way of aggregating feedback from the group. + - Geoffrey: \[\[return to this]] + - Myles: do you think writing inline MIME types is the most ergonomic option? + - Geoffrey: this is the current design; if there are any other ideas, provide feedback + - Myles: there are several people who are focused around loaders. Probably a good idea to get those people discussing their vision around loaders in the next meeting. + - Saleh: one problem with --experimental-modules is that without module graph optimizations, things might not be good enough. + - Jordan: what's your criteria for "good enough"? We already have people using babel-node and ts-node, and while they're relatively slow, many people consider them good enough. + - Is this a question of native vs. JS? + - Really just want performance to be a key concern. + - Myles: everything that goes into Node core has a lot of scrutiny on performance. + - Saleh: as we consider loaders, we should envision how they might work in the browser and how they could come with a standard. + - Jan: Thinking about package name maps might be better to think of sooner rather than later. + - Myles: direct support could be done via a loader, so not necessarily mutually exclusive + - Jordan: it's insanely premature to do work around announcements for yarn pnp or tink/crux + - Kevin: It's an exciting time! We should experiment with these different ideas. + - Myles: Wesley said that pnp/tink are like incremental caches, but they're also hijacking the resolution process to do this. This kind of gives a strong signal that there's a need for experimentation. +- Proposal: + - Phase 2: Uncontroversial things not related to loaders + - Phase 3: Flexible loaders that allow experimentation/learning to inform Phase 4 + - Phase 4: Hard decisions (things that may exclude certain use-cases) diff --git a/doc/meetings/2018-10-24.md b/doc/meetings/2018-10-24.md new file mode 100644 index 0000000..bc733e9 --- /dev/null +++ b/doc/meetings/2018-10-24.md @@ -0,0 +1,120 @@ +# Node.js Foundation Modules Team Meeting 2018-10-24 + +* **Recording**: https://www.youtube.com/watch?v=WES0lGeneNs +* **GitHub Issue**: https://github.com/nodejs/modules/issues/206 +* **Minutes Google Doc**: https://docs.google.com/document/d/10HOj79n61bSGXEh6CKfaeB4mc0KW4wwB_EsTh6MVs1k/edit + +## Present + +- Myles Borins (@MylesBorins) +- Matt DuLeone (@mduleone) +- Geoffrey Booth (@GeoffreyBooth) +- John-David Dalton (@jdalton) +- Daniel Rosenwasser (@DanielRosenwasser) +- Gil Tayar (@giltayar) +- Saleh Abdel Motaal (@smotaal) +- Gus Caplan (@devsnek) +- Michael Zasso (@targos) +- Rob Palmer (@robpalme) +- Jordan Harband (@ljharb) + +## Agenda + +Extracted from **modules-agenda** labelled issues and pull requests from the **nodejs org** prior to the meeting. + +### Approving PRs (7 Minute Timebox) + +* add @soldair to observers [#200](https://github.com/nodejs/modules/pull/200) + - consensus +* Create Terminology.md [#158](https://github.com/nodejs/modules/pull/158) + +### Update on Progress (8 Minute Timebox) + +* Terminology: Stipulative Terms [#184](https://github.com/nodejs/modules/issues/184) + - 4 minute timebox +* Surveys: + - 4 minute timebox + - Refs: + - Purpose and Scope [#182](https://github.com/nodejs/modules/issues/182) + - Initial Drafts [#173](https://github.com/nodejs/modules/issues/173) + - Developer Survey [#85](https://github.com/nodejs/modules/issues/85) + +### Discussion (40 Minute Timebox) + +* Doc: Phase 2 for new modules implementation [#196](https://github.com/nodejs/modules/pull/196) + - 30 minute timebox + +* keeping ecmascript-modules up to date [#197](https://github.com/nodejs/modules/issues/197) + - 5 minute timebox +* Discussion about an implicit (or explicit) "node" scheme [#169](https://github.com/nodejs/modules/issues/169) + - 5 minute timebox + +* Update on Dynamic Modules [#188](https://github.com/nodejs/modules/issues/188) + - Guy Bedford needed to drop, punted to next meeting + +## Invited + +* Modules team: @nodejs/modules + +## Notes + +The agenda comes from issues labelled with `modules-agenda` across **all of the repositories in the nodejs org**. Please label any additional issues that should be on the agenda before the meeting starts. + +* Terminology: Stipulative Terms [#184](https://github.com/nodejs/modules/issues/184) + - Saleh: We created a branch that's been out for a bit now to try to clarify specific terms we want to use within this group. + - Has been enough time for people to object to take a look, so... do that! + - Myles: What's the deal with the deprecated terms? Does that mean we can't use those terms? For example, "transparent interop" is listed there. + - Saleh: Everyone has a different interpretation of transparent interop, so it didn't seem like we were productive using these terms and talking past each other. Maybe "deprecated" is not the right term? + - Myles: as an ask, can we eventually separate out the deprecated section, create a policy around that, and as we make the changes please squash the commits +* Surveys: + - 4 minute timebox + - Refs: + - Purpose and Scope [#182](https://github.com/nodejs/modules/issues/182) + - Initial Drafts [#173](https://github.com/nodejs/modules/issues/173) + - Developer Survey [#85](https://github.com/nodejs/modules/issues/85) + - Saleh: I feel pretty uncomfortable putting questions on the survey that aren't representative of what the group is interested in + - Overall the survey was well-received (in terms of willingness to answer the survey), but the questions themselves need feedback from the group + - Saleh's dog: *barking* (okay, they did) + - Saleh: it would be great if we could iterate more with Google Docs + - Myles: it might be productive to set up a meeting outside of this biweekly discussion + +* Doc: Phase 2 for new modules implementation [#196](https://github.com/nodejs/modules/pull/196) + - 20 minute timebox + - Geoffrey: Idea of phases is "what's the next lowest-hanging fruit that we could start implementing?" + - Finishing any given phase doesn't mean we're done. + - Also, things aren't set in stone. + - Saleh: [fill in here] (see new issue [#208](https://github.com/nodejs/modules/issues/208) + - Bradley: I am against that point, but we can chat offline about that. + - Geoffrey: wanted not to discuss specific technical details here + - Jan: Not sure how I feel about first bullet (virtual module integration), third bullet (importing from package names) has a lot of implied semantics + - `import "lodash/index.mjs"` doesn't work today? + - Myles: think it does, but not totally clear + - Jan: anyway, I don't think any of them should work; otherwise npm and yarn will end up hacking around whatever we do + - Myles: regardless of what happens with the packagers, node_modules, package name maps, etc., throwing away node_modules would cause an ecosystem fracture + - Jan: but how is this different from any other loader we'd want people to use? node_modules resolution seems like it could be part of a custom loader. + - Geoffrey: the idea is just to make bare path importing possible in the first place. + - Bradley: I don't know how we're going to make this possible without *some* default. If we can't come to consensus on that default, then we need to push out of stage 2 + - Kevin: what if we swapped phase 2 with phase 3? That might give us some clarity over the points in stage 2. + - Are we ready to talk about loaders? Do we have an implementation? + - Bradley: they exist ephemerally but we really need people to raise loaders as a discussion point - not enough people are talking about them. + - Myles: maybe stage 2 could be a stage of things that are completely unrelated to loaders. + - Gil: Clarification: what are "virtual modules"? + - Saleh: a module that doesn't exist on disk, but that can be created from some source text. For example, want to create a "virtual" module from a service worker. + - Jan: Overall, ideally we'd have some sort of default loader that builds on Node loader API + - Bradley: Loader spec started in TC39 and was discontinued; Realms proposal has evolved from that and it's a string-based API. If you're interested in why it's bad to introduce arbitrary objects into the module graph, you too can join this group which spends 4 hours a week discussing the matter. + - Objections? + - Only if we're intent on shoving things into the module graph. + - Myles: can we at least say we want to explore it in phase 2, without implying an implementation? + - Can we look at both implementations? + - Bradley: no, don't want to take the decisions here that lightly. People have spent years on this, and we're not even in touch with with Realms people. + - Saleh: I object to something like createDynamicModule (if loaders can’t evaluate ESM) + - Myles: this is probably more implementation-specific than the point implies. Can we at least just say we're exploring? Perhaps engaging with the Realms team as well? + - Do we have any objections? For point 3 (`import _ from "lodash"`) can we just retitle this to something like "exploring supporting" rather than just "supporting"? + - "Explore design space for virtual module from source" + - Point 4 (out-of-band command line disambiguation) doesn't have agreement; changing to "exploring" design space might be too broad. + - Could emulate anything with a bash script. + - Consensus! + +* keeping ecmascript-modules up to date [#197](https://github.com/nodejs/modules/issues/197) + - 5 minute timebox + - Not enough time! diff --git a/doc/meetings/2018-11-07.md b/doc/meetings/2018-11-07.md new file mode 100644 index 0000000..fbfa87f --- /dev/null +++ b/doc/meetings/2018-11-07.md @@ -0,0 +1,112 @@ +# Node.js Foundation Modules Team Meeting 2018-11-07 + +* **Recording**: https://www.youtube.com/watch?v=b8wLboT_A_0 +* **GitHub Issue**: https://github.com/nodejs/modules/issues/217 +* **Minutes Google Doc**: https://docs.google.com/document/d/1PdkHSzthrjBHx3OzUaEL5Vqa2L4WtAs8WdJl_L6HT6E/edit + +## Present + +- Matt DuLeone (@mduleone) +- Gus Caplan (@devsnek) +- Daniel Rosenwasser (@DanielRosenwasser) +- Wesley Wigham (@weswigham) +- Geoffrey Booth (@GeoffreyBooth) +- Myles Borins (@MylesBorins) +- Guy Bedford (@guybedford) +- Ujjwal Sharma (@ryzokuken) +- Michael Zasso (@targos) +- Jordan Harband (@ljharb) +- Jan Krems (@jkrems) +- Gil Tayar (@giltayar) + +## Agenda + +Extracted from **modules-agenda** labelled issues and pull requests from the **nodejs org** prior to the meeting. + +### Approving PRs (12 minute timebox) + +* Add @ryzokuken as Observer [#213](https://github.com/nodejs/modules/issues/213) + - 1 minute timbox + - No objections. +* @bmacnaughton requesting membership/observer [#211](https://github.com/nodejs/modules/issues/211) + - 1 minute timbox + - No objections. +* esm: refactor dynamic modules [#9](https://github.com/nodejs/ecmascript-modules/pull/9) + - 5 minute timebox + - Gus: Was able to avoid a double module wrap - no behavioral change, just an optimization. + - Awkward silence? + - Myles has dropped out of the call - Brad is the host now! + - Any objections? + - Nope. + - Action item: Gus, can you land this and upstream it? +* Add format distinction to phase 2 [#215](https://github.com/nodejs/modules/pull/215) + - 5 minute timebox + - Guy: at the moment we only have support for top-level execution of ES Modules and you can only import ES modules. + - Trying to provide a distinction for formats in which a module can be imported - want to move this into the phase 2 discussions. + - Action item: Guy can you land the PR and lead the discussions? + +### Update on Progress (1 minute timebox) + +* Surveys: Members formally taking ownership (leadership) is the next step [#209](https://github.com/nodejs/modules/issues/209) + - 1 minute timebox + - Saleh: As a group we must decide how we want to proceed with the survey. + - Must find a way to distribute work in the group for this. + - Jan: is there a list of things that still need to be done? + - Saleh: just simply need people to drive questions they want answered. + +### Discussion (40 Minute Timebox) + +* keeping ecmascript-modules up to date [#197](https://github.com/nodejs/modules/issues/197) + - 10 minute timebox + - Myles: this repository hasn't been actively rebased against `master` + - Silence + - Myles: as a suggestion, let's rebase and allow force rebases on the fork + - Guy: force rebase is kind of hard to review; can we do a 2-branch master for reviews? + - Jordan: ideally these things should be fast-forward rebased so that we can stay as close as possible to the original history. + - Action item: Ryzo will write up a proposal for the process in mind. +* Dynamic Modules Status [#188](https://github.com/nodejs/modules/issues/188) + - 10 minute timebox + - Guy: Gus originally wrote a PR to support importing from CJS + - It was blocked because in the spec it appeared that you need to know the named exports ahead of time. + - But CJS modules don't have statically analyzable imports; you need to run the module to know what's available. + - TC39 is open to allowing exports to be specified during the execution phase. + - Technical changes involve applying TDZ to uninitialized exports. + - Implementation seems to be going well in V8. + - Jordan: got the chance to raise it with TC39 editors group + - Gil: what were the changes in the spec that were needed? + - Guy: Two changes: (1) need to be able to throw during the instantiation phase (in the GetExportedNames function). (2) being able to track namespace records from dynamic modules +* esm resolver spec and implementation refinements [#12](https://github.com/nodejs/ecmascript-modules/pull/12) + - 15 minute timebox + - Guy: want a public resolver spec as a "source of truth" for ESM resolution + - Today + - Error messages aren't that great + - no context on which module caused the error + - no single quotes around the specifier + - doesn't tell you when you try to use an import like a `require()` + - Proposal: provide that information! + - Scoped package resolution: + - Gil: Seems like a potential problem that we'd now be parsing our paths. + - Package resolution keeps walking up the spine, doesn't tell you where you imported from + - Proposal: keep the first node_module path you hit for a deep import, then stop. + - Brad: when we looked through who was using it, we found exactly one hard dependency. +* Explicit package exports proposal (https://github.com/jkrems/proposal-pkg-exports) + - 5 minute timebox (if there is time) + - Jan + - Bare imports behavior is very, well, bare. + - People don't typically use extensions. + - Multiple independent APIs in a single package. + - First draft of a proposal. + - Declare what the public interface is via an `"exports"` field in `package.json`. + + ```json5 + "exports": { + "": "./src/moment.mjs", + "/": "./src/util/", + "/timezones/": "./data/timezones/", + "/timezones/utc": "./data/timezones/utc/index.mjs" + } + ``` + + - Brad: I would like to see how to deal with multiple "modes" (e.g. GraphQL). + - GraphQL introduced multiple modes and ended up eventually splitting into two different packages. + diff --git a/doc/meetings/2018-11-21.md b/doc/meetings/2018-11-21.md new file mode 100644 index 0000000..32b3b72 --- /dev/null +++ b/doc/meetings/2018-11-21.md @@ -0,0 +1,89 @@ +# Node.js Foundation Modules Team Meeting 2018-11-21 + +* **Recording**: https://www.youtube.com/watch?v=NvJgv0-rm1A +* **GitHub Issue**: https://github.com/nodejs/modules/issues/226 +* **Minutes Google Doc**: https://docs.google.com/document/d/1f9orlGCs0q6nLw6qtUpS3-GhdKcuxfYw9uc8Abfk-OA/edit + +## Present + +- Geoffrey Booth (@GeoffreyBooth) +- Myles Borins (@MylesBorins) +- Guy Bedford (@guybedford) +- Michael Zasso (@targos) +- Gus Caplan (@devsnek) +- Rob Palmer (@robpalme) +- Saleh Abdel Motaal (@SMotaal) +- Michael Dawson (@mhdawson) +- Jeremiah Senkpiel (@fishrock123) +- Jan Krems (@jkrems) +- Brad Farias (@bmeck) + +## Agenda + +Extracted from **modules-agenda** labelled issues and pull requests from the **nodejs org** prior to the meeting. + +### Approving PRs (7 minute timebox) + +* governance: be explicit about meetings new members can join [#218](https://github.com/nodejs/modules/pull/218) +2 minute timebox +Asking for LTGMs +No objections, landed + +* esm resolver spec and implementation refinements [#12](https://github.com/nodejs/ecmascript-modules/pull/12) + - 5 minute timebox +A number of improvements to error handling +Goal is to make the spec and implementation evolve together. +Asking for LGTMs +No objections, landed + +### Update on Progress (3 minute timebox) + +* Surveys: Members formally taking ownership (leadership) is the next step [#209](https://github.com/nodejs/modules/issues/209) + - 3 minute timebox +Geoffrey: Could use a survey to help settle UX questions especially in Phase 2 +Myles will connect us with NPM survey + +### Discussion (45 Minute Timebox) + +* Thinking about deadlines [#123](https://github.com/nodejs/modules/issues/123) + - 5 minute timebox + +Myles: we should start to think about timelines for phases 2 and 3. +Bradley: potentially move Phase 2-1 into 3 +Saleh: objects to moving Phase 2-1 into 3 +Myles: would love to see Phase 2 finished before Dec 31, big Node 12 milestone in April +Jeremiah: need time for Node team to review, so needs to be before April +Guy: how much time before? + +* When should we drop the moratorium? [#225](https://github.com/nodejs/modules/issues/225) + - 10 minute timebox +Myles: wants to upstream as soon as possible, especially to remove/change things that we know will change in the new implementation +Myles: would people object to upstreaming minimal kernel as it is now? Guy objects +Saleh: wants loaders in +Guy is for upstreaming under certain conditions +Geoffrey: consider the PR aspects, what would we tell users when/if we upstream? +Bradley: doesn’t feel comfortable keeping --loader if we upstream +Guy: could find ways of packaging this, like upstream phases 2 and 3 separately +Myles: what about specifically upstreaming Guy’s PR? No objections + +* keeping ecmascript-modules up to date [#197](https://github.com/nodejs/modules/issues/197) + - 5 minute timebox +Myles: Anyone have any issues with current process that involves opening a PR to trigger a CI build, and then Myles rebases? No objections +Anyone have time helping to automate this? Myles and Michael and Guy offer; Gus: also @ryzokuken from last call offered +Myles: now have a master branch kept current with upstream, “lkgr” is last known good release kept current + +* re-enable issues tab on ecmascript-modules [#212](https://github.com/nodejs/modules/issues/212) + - 5 minute timebox +Gus: The repo is an implementation where code things are happening, when issues pop up they should be with the code. I’ve seen issues opened on other repos, those seem like poor places to track that. +Myles: Consensus to enable issues but move issues that are off-scope to other repos? No objections, issues re-enabled. + +* Resolution Proposal [#228](https://github.com/nodejs/modules/issues/228) +20 minute timebox +Geoffrey gave overview of proposal +Rob: Dual-mode packages: import(“lodash”) and require(“lodash”) *will* return separate instances, assuming “lodash” is a dual-mode package. +Bradley: top concern when he evaluates all proposals now is the potential for separate import specifiers to return different modules for the same specifier, e.g. imported as ESM or CJS, and this proposal addresses that as far as we can tell. +Myles asked for executive summary to be added, Geoffrey agreed to work on that +Rob asked if the “searching up the path for a package.json” logic would happen for relative paths as well as absolute URLs, Guy and Bradley and Geoffrey discussed and agreed that it would, Geoffrey will update text to make that clear. +Geoffrey to Myles: way forward on this and package exports proposals? Vote in next meeting? +Myles and Guy suggest wrapping this into Guy’s resolution spec PR, and approving that PR would implicitly approve the proposals that went into making it, and would clear the way for implementation. + diff --git a/doc/meetings/2018-12-05.md b/doc/meetings/2018-12-05.md new file mode 100644 index 0000000..6a5a50d --- /dev/null +++ b/doc/meetings/2018-12-05.md @@ -0,0 +1,75 @@ +# Node.js Foundation Modules Team Meeting 2018-12-05 + +* **Recording**: https://www.youtube.com/watch?v=OyLCPXym7bE +* **GitHub Issue**: https://github.com/nodejs/modules/issues/236 +* **Minutes Google Doc**: https://docs.google.com/document/d/1b5aA1jiYRSjjvjIwU2pt1FK7OiaR5EMLmM72E9oR9pI/edit + +## Present + +- Saleh Abdel Motaal (@SMotaal) +- Michael Zasso (@targos) +- Bradley Farias (@bmeck) +- Gus Caplan (@devsnek) +- Matt DuLeone (@mduleone) +- Geoffrey Booth (@GeoffreyBooth) +- Kevin Smith (@zenparsing) +- Myles Borins (@MylesBorins) +- Wesley Wigham (@weswigham) +- Guy Bedford (@guybedford) +- Jordan Harband (@ljharb) + +## Agenda + +Extracted from **modules-agenda** labelled issues and pull requests from the **nodejs org** prior to the meeting. + +### nodejs/modules + +* Locking down the "process" and "Buffer" globals [#235](https://github.com/nodejs/modules/issues/235) + +- Guy: tried to prev remove Buffer/process globals. Was untenable due to performance. Lots of capabilities that might not be desirable to give to all modules. As we move towards security we want to lock down some of those capabilities. If we can lock these down, the time to do it is before we ship if we apply it to ESM. e.g. we could lock down `fs` by removing it from a 3rd party package using a resolver. Approach idea is to make global.process/global.Buffer warnings and add them to the CJS wrapper. +- Jordan: Buffer is a weird thing to be a global. Like the idea of limiting the privilege granting things from process as being a global. Platform detection uses the fact that `process` is a global so we might look at removing dangerous parts rather than the entire global. ESM is a reason to restrict module capabilities. Any feature of that nature should work for CJS. We should not leverage the ESM initiative to directly change features outside of ESM. These changes are not the scope of the ESM working group. +- Myles: CJS interop means that we could export these privileged capabilities. We should record what privileged bits of the process people are using, and we might be able to deprecate the `process` global. Don't think we can ever truly remove `process` from CJS. +- Jordan: If things are possible in CJS but not ESM it is a barrier to migrate to ESM. +- Jeremiah: Buffer was tried to be deprecated in the past (citation?) but it didn't work out. process and Buffer shouldn't be globals but soo much stuff relies on it that removing things is extremely difficult to do. If we removed things in both environments that seems possible (?), but if it only affects ESM that seems confusing. Has anyone looked at how this affects existing solution like webpack? +- Wes: Webpack/Typescript/Babel migration is extra work, if you want to do big breaking migrations you can do it. Making changing during ESM implementation is being argued as a possible time. Most people don't want to do big breaking changes to existing code. +- Guy: the upgrade path is adding 2 imports to your module. I am optimistic about this path. When doing environment detection, need a replacement workflow. How do we want users of modules to determine the environment/host they are running on. +- Kevin: Expanding scope is something we should be careful of. Expanding scope makes it hard to understand the problems we are trying to solve. +- Myles: PR around namespaces. If we have a @nodejs namespace inside of core we could be putting things there. Streams are a good example of this. Larger platform question about if this could be a pattern amongst the greater JS ecosystem. Could shim @nodejs in the browser with something like import maps. +- Jordan: Built-in language modules is desired and people are excited about. Recently there was a concrete proposal https://github.com/tc39/proposal-javascript-standard-library . No one has figured out how to make this work w/ polyfilling/shimming and deleting or denying access to them. We might not want to put our hopes on that. +- Myles: Node Collab summit and TC39 might want to spend time around cross platform concerns. +- Guy: Was hoping to get some thinking about module security here, and can go to node core with it. + +* Dynamic Modules Status [#188](https://github.com/nodejs/modules/issues/188) + +- Guy: Slides from last TC39 https://docs.google.com/presentation/d/1--k5tJmhXJeLlUOmBdHGK3my9VCbZVL0MF8OrTeqS0I/edit#slide=id.p +- Guy: Some changes related to namespaces and cycles since the last meeting. Consensus at the TC39 meeting was some questions about behaviors and we had a breakout. Main question was re-exporting using export * from a dynamic module could cause unknown export names during linking, causing errors. TC39 felt that changes to make this not work so that dynamic behavior does not cascade into ESM was desirable. Open to extending that restriction over time, we could revisit use cases in the future. Next meeting will present the changed spec. +- Gus: Accessing unfinished namespaces is already possible +- Guy: You can access namespaces in cycle before execution, but they have all the exported names already. You cannot know the names of CJS prior to execution, so there is a pending phase for dynamic modules, that is not part of ESM. +- Myles: Order of operation concern. TC39 said they would do it if V8 does it which was waiting on signal from TC39 that it is ok and is now waiting on signal from Node. +- Guy: Correct, TC39 has provisional approval, V8 is pending Node approval. We can then do this step by step to get it to TC39. Could start bringing the impl into our work already. +- Myles: we could float that patch in our fork. +- Jordan: Some reflection methods work but some throw on ESM namespaces that have not evaluated. Haven't found a way to make it work without terrible workarounds. Someone asked why these namespaces are reflectable anyway. If we make all ESM namespaces non-reflectable during cycles this problem might go away. + +Conclusion: we need to have some follow ups. Will make an issue/statement from Node core about endorsing this. + +### nodejs/ecmascript-modules + +* \[Do not merge\] doc: Add pkg-exports proposal to resolve spec [#14](https://github.com/nodejs/ecmascript-modules/pull/14) + - Jan: This is a PR about the "exports" in package.json. Planned on rewriting this based upon Resource loading based upon other doc if/when we put things into Resource based loading. The algorithm in terms of name resolution should not be changed due to any such things and it would just require a refactoring without logic changes. +- Geoffrey: Should we set of goal by next meeting to have the spec document to be done. This gives us homework/time to open any issues on that repo. That would let us have final consensus. +- Bradley: Would like to see write up on thrashing when 2 packages produce a cycle. +- Jan: Behavior doesn't allow for chaining, relative semantics are not allowed to be aliased/go through this other resolution mechanism. +- Myles: Would like to see something other than the empty string for the "main", (leading slash??). How do people feel about extending what the root means. This might be meta-data that could be used to generate import maps. We might want Domenic's input on this. +- Bradley: Uncomfortable with leading slash being rewritten. +- Jan: This is based upon the import map proposal. Leading slash Myles pointed out was after bare name in a specifier. We can revisit empty string. + +* Specify import file specifier resolution proposal [#19](https://github.com/nodejs/ecmascript-modules/pull/19) + +- Guy: this is a compliment proposal to what Jan was presenting. The implementation matches Geoffrey's repo. Takes that and turns it into spec language. Hope is that we can discuss specifics w/ the proposals. +- Geoffrey: Lets try not to put every possible concern about the spec on that PR, please open issues on the proposal repo. Leading slash can be done in another PR if we want it, this is not the last PR we will have and we can add it on as desired. Empty string vs … , all are up for debate and it is bikesheddy at this point. Behavior is locked down it seems. +- Myles: How does this change the loader? +- Geoffrey: This introduces "package scope" also called "package boundary". 2 parts of resolution: location and interpretation. Locating file for an import specifier like `import "lodash"` -> file:///…/lodash/… . This is about interpretation, find the package scope for any located file and read the package.json to know what to do with the file. If it has something flagging the package as ESM you could understand that files are ESM. +- Myles: this is package.json data on how .js files are interpretted? How far does this go? +- Geoffrey: this currently only discusses .js but for any file with a parse goal we may want to deal with this +- Jordan: for packages it seems good to have import-map style rewriting of where files are. Why aren't we doing this for both ESM and CJS. For parsing goals we could benefit from having this in CJS. +- Bradley: We can apply this to a CJS loader that we have discussed in the past. We need the collision-less intention of a format that this proposal supplied if we are to apply it to CJS as well. I will voice strong objection if I am unable to find a way that proposals could not *theoretically* work in CJS, that doesn't cover actual implementation though. Modules work shouldn't be held up by the CJS implementation of this feature. diff --git a/doc/meetings/2018-12-19.md b/doc/meetings/2018-12-19.md new file mode 100644 index 0000000..b4d61bf --- /dev/null +++ b/doc/meetings/2018-12-19.md @@ -0,0 +1,98 @@ +# Node.js Foundation Modules Team Meeting 2018-12-19 + +* **Recording**: https://www.youtube.com/watch?v=T2i-dkaKklE +* **GitHub Issue**: https://github.com/nodejs/modules/issues/240 +* **Minutes Google Doc**: https://docs.google.com/document/d/1Kj6Ja4DM_x8Ehis-nahgttCFLMupsmWiZUhdf-WPl6U/edit + +## Present + +- Matt DuLeone (@mduleone) +- Wesley Wigham (@weswigham) +- Daniel Rosenwasser (@DanielRosenwasser) +- Guy Bedford (@guybedford) +- Myles Borins (@MylesBorins) +- Rob Palmer (@robpalme) +- Jan Krems (@jkrems) +- Gus Caplan (@devsnek) +- Bradley Farias (@bmeck) +- Hassan Sani (@inidaname) +- Geoffrey Booth (@GeoffreyBooth) +- John-David Dalton (@jdalton) +- Jordan Harband (@ljharb) + +## Agenda + +Extracted from **modules-agenda** labelled issues and pull requests from the **nodejs org** prior to the meeting. + +### Discussion (55 minute timebox) + +* Discussion: Organizing work [#230](https://github.com/nodejs/modules/issues/230) + - 10 minute timebox + - How we can organize & structure work + - If we had some sort of page to say "who's working on what", that'd be great. + - Looking to expand this outside of the existing groups + - Myles: Node as a whole is a bit of a "do-ocracy" - it's just a matter of who does the work. + - Not sure whether a designated responsibilities model is something that will work or scale. + - What do you imagine the commitment of this document? + - Geoffrey: we have something like that with the phase documents; but it's more of a statement of who's interested in what, and who to include on PRs and the like. + - Kevin: Hard to get clarity on goals of various groups, and how we want to do work, but I don't have a specific recommendation. + - Myles: Saleh has recommended a "project board"; could imagine a column for each topic + - Rob: Focus groups can definitely lead to progress; really would love an index to point back to this work. + - Geoffrey: can we get consensus on a project board? + - No objections! + - Myles: Great, who will do this? + - Seems like both Geoffrey and Saleh + +* Dynamic Modules Development in Node.js [#24894](https://github.com/nodejs/node/issues/24894) + - 15 minute timebox + - Refs: + - Update spec to throw on export * from dynamic modules [#11](https://github.com/nodejs/dynamic-modules/pull/11) + - Bradley: we encountered something controversial about the behavior here, so this PR is a response to that. + - so by removing this behavior, we are open to new design space + - for clarity: nobody is saying "get rid of `export *`", just saying it isn't clear given that there's lack of consensus around it + - we may come back and add this, but we don't want to lose momentum on this feature + - Jordan: I've made similar objections in the past (i.e. "shipping without X is unacceptable"), but we're not even shipping this unflagged, so it's reasonable. Nobody should be relying on this experimental behavior. + - Kevin: don't have to do a "line in the sand" approach + - Bradley: blocking on a forward path because of a controversial feature is going to block whole proposals + - we're refusing to let ourselves agree on anything because of absolutes + - we should feel comfortable reverting features + - also, we have deadlines here + - JDD: this feature is kind of the point of transparent interop + - if we believe this is something that adds complexity to the spec; it's going to be easier to work this in from the start than later on + - we should go to TC39 first and get clarity on what we can do here + - Myles: is this something that can wait 2 weeks? + - JDD: saying "this work can be reverted" or making an explicit goal to support this somehow would be reasonable. + - Myles: so what if we added this work to phase 2? + - Bradley: I'm not okay with a commitment to not shipping without `export *` interop, but am okay with a heavy investigation here. + - JDD: If we aren't shipping this feature, then we are shipping non-transparent interop. + - Myles: Guy, do you have anything to add? + - Guy: Nothing I can verbalize. + - Myles: can we agree that we're adding exploring the `export *` space to our roadmap? + - Guy: I've spent so much time on this. Nobody has wanted to contribute, nor has anyone provided a use-case for `export *` with CJS. I've worked with TC39 to rewrite parts of the spec for transparent interop. And then one person coming to this thread has jumped into the thread and derailed the process. This is incredibly demoralizing. If someone can provide *one* use-case, that would be minimally helpful here; along with someone willing to talk about the minimal details + - JDD: have tons of users who would be impacted by this; took a survey of code in the wild. + - Bradley: Actually didn't see that many occurrences; numbers don't entirely back that up. Many of those are occurrences are compiled code anyway. + - Jeremiah: don't think compiled code is the audience for dynamic module interop. + - Myles: Perhaps we'll have to revisit the idea of whether dynamic modules are the right solution. + - JDD, what would be your feeling if this whole thing went to a vote? + - JDD: I'd feel pretty lousy if the goal is transparent interop; I can do this today with a compiler. + - Myles: how about we land the current work, and work on `export *`, revisiting the work itself if we're unable to reach a conclusion. + - Guy: can we resolve this discussion before the next TC39 meeting? + - Do we have any objections around merging in the current PR? + - No objections! + - Do we have any objections around merging in the documentation over the next week? + - No objections! + +* Specify import file specifier resolution proposal [#19](https://github.com/nodejs/ecmascript-modules/pull/19) + - ~~15 minute timebox~~ + - 2 minute update + +\[Out of time] + +Next meeting will be January 2nd. + +\[Zoom issues - abrupt meeting cutoff 2 minutes early :D] + +* \[Do not merge\] doc: Add pkg-exports proposal to resolve spec [#14](https://github.com/nodejs/ecmascript-modules/pull/14) + - 15 minute timebox + + diff --git a/doc/meetings/2019-01-16.md b/doc/meetings/2019-01-16.md new file mode 100644 index 0000000..d6bf29b --- /dev/null +++ b/doc/meetings/2019-01-16.md @@ -0,0 +1,85 @@ +# Node.js Foundation Modules Team Meeting 2019-01-16 + +* **Recording**: https://www.youtube.com/watch?v=miv2i1h5CM0 +* **GitHub Issue**: https://github.com/nodejs/modules/issues/248 +* **Minutes Google Doc**: https://docs.google.com/document/d/1JTNIz5XMoKKAxyKF4pC-GoK9Wy3ZUIF0EwsUezZ87L8/edit + +## Present + +- Myles Borins (@MylesBorins) +- Saleh Abdel Motaal (@SMotaal) +- Matt DuLeone (@mduleone) +- Jeremiah Senkpiel (@fishrock123) +- Gus Caplan (@devsnek) +- Rob Palmer (@robpalme) +- Guy Bedford (@guybedford) +- Wesley Wigham (@weswigham) +- Daniel Rosenwasser (@DanielRosenwasser) +- Geoffrey Booth (@GeoffreyBooth) +- Jordan Harband (@LJHarb) + +## Agenda + +Extracted from **modules-agenda** labelled issues and pull requests from the **nodejs org** prior to the meeting. + +### Updates (5 minute timebox) + +* Structuring a "Breakout Teams" project board [#244](https://github.com/nodejs/modules/issues/244) + - 2 minute timebox + +* Discussion: Organizing work [#230](https://github.com/nodejs/modules/issues/230) + - 3 minute timebox + +Selah: it is up and ready to start being used. Needs groups to start using it. + +### Approving PRs (5 minute timebox) + +* doc: update stage 2 regarding dynamic modules [#242](https://github.com/nodejs/modules/pull/242) + - 5 minute timebox + - No objections! + +### Discussion (40 minute timebox) + +* Mode: esm proposal [#247](https://github.com/nodejs/modules/issues/247) + - 10 minute timebox + - Demo time! + - Started off with a minimal implementation with several proposals. + - Have a few edge cases with how they interact which we'll go over. + - This minimal implementation supports `.mjs`, and Node can run with that. + - Can also use CommonJS from a `.js` file from plain Node. + - And can also `import` from that CommonJS `.js` file! + - Note that + - the `.js` comes in with a `default`. + - have to use the fully-qualified name. + - Setting the `main` field in a `package.json` still resolves correctly when directing to a CommonJS, but has to be explicit with relative and extensionful paths when pointing to ESM. + - "Deep" imports from packages don't work - each package is totally encapsulated. + - New `exports` field - can forward deep paths within a package. + - We discussed treating all `.js` as ESM - here, to accomplish that, users have to select `mode: esm`. + - All this means you can use the `package.json` `main` field for CJS and `exports` for ESM, and you can have them both work simultaneously. + - You can also turn off the encapsulation behavior (unclear - how @guybedford?) + - Jordan: You have a really good case for the ergonomics of `"mode": "esm"` + - Guy: I don't care what it is as long as it takes 20 keystrokes or fewer + - In terms of the `exports` functionality, it might be a little difficult to make it apply to CommonJS + - Gus: Why is encapsulation the default in `"mode": "esm"`? + - Guy: `"exports"` is an encapsulation of what you get in the package. + - Gus: I have go to but I have concerns about this not including JavaScript inputs. We need something that gives people more freedom. + - Rob: feels like the "desired" way of using it and I wonder - \[\[background noise from siren]] + - Guy: Have to ensure that people who don't like encapsulation have a way out. + - Geoffrey: Would love to have people try this out to see what's confusing and get user feedback. Specifically, how the main, mode, and exports fields work together. + +* Dynamic Modules Development in Node.js [#24894](https://github.com/nodejs/node/issues/24894) + - 10 minute timebox + - Without spec text there's really nothing we can do here. + +* Specify import file specifier resolution proposal [#19](https://github.com/nodejs/ecmascript-modules/pull/19) + - 10 min + + - If `"mode"` is the one thing that turns things on/off, then maybe `"exports"` shouldn't be the signifier for whether a package is ESM. + - Jordan: A map from files to parsing modes would probably be more general. + - Bradley: treating any one given circularity problem as "fatal" means you have to treat the whole system as fatal. We should avoid treating all circularities as fatal. Goto example is loader.js being loaded as CJS but making all .js files treated as ESM + - Myles: even if you just use strings for `mode`, you can always expand to mapping objects later on. + - Guy: if this is something we want to pursue, this is worth raising with npm given that it's focused around UX. + +* \[Do not merge\] doc: Add pkg-exports proposal to resolve spec [#14](https://github.com/nodejs/ecmascript-modules/pull/14) + - 10 minute timebox + - Action item: create an issue for updating the exports spec to be applicable to the CJS goals diff --git a/doc/meetings/2019-01-30.md b/doc/meetings/2019-01-30.md new file mode 100644 index 0000000..fea7348 --- /dev/null +++ b/doc/meetings/2019-01-30.md @@ -0,0 +1,106 @@ +# Node.js Foundation Modules Team Meeting 2019-01-30 + +* **Recording**: https://www.youtube.com/watch?v=a1YraH8GoHk +* **GitHub Issue**: https://github.com/nodejs/modules/issues/258 +* **Minutes Google Doc**: https://docs.google.com/document/d/1yx4k1syLuMU_98sa2ASu3mgXSJ4-9Lac8uWzqIlEoYc/edit + +## Present + +- Saleh Abdel Motaal (@smotaal) +- Myles Borins (@MylesBorins) +- Geoffrey Booth (@GeoffreyBooth) +- Matt DuLeone (@mduleone) +- Wesley Wigham (@weswigham) +- Jan Krems (@jkrems) +- Guy Bedford (@guybedford) +- Michael Zasso (@targos) +- John-David Dalton (@jdalton) +- Gus Caplan (@devsnek) +- Hassan Sani (@inidaname) +- Daniel Rosenwasser (@DanielRosenwasser) +- Ahmad Abdul-Aziz (@devamaz) +- Jordan Harband (@ljharb) + +## Agenda + +Extracted from **modules-agenda** labelled issues and pull requests from the **nodejs org** prior to the meeting. + +### Announcements + +* --eval, STDIN, and extensionless files [#251](https://github.com/nodejs/modules/pull/251) + +Please express interest in the thread + +### Discussion + +* Slack channel [#249](https://github.com/nodejs/modules/pull/249) + - 5 minute timebox +* Dynamic Modules Development in Node.js [#24894](https://github.com/nodejs/node/issues/24894) + - 15 minute timebox + - Guy: briefly went through the change + - Was some resistance from Till + - Some concern about whether dynamic modules loosen certain invariants + - Have been promises from TC39 over the years that these issues would be worked on, but there's not necessarily any implementation feedback except that there could potentially be a spec exception to "here's how this is allowed to work" + - Brad + - Domenic Denicola and AWB were not able to attend, so they weren't present to object, but they have raised concerns over GitHub. + - The "forbids" clause over other implementations seems like it could alleviate certain concerns. + - AWB is concerned around existing invariants, Domenic wants to have a separate web standard and a specific behavior for Node. + - Myles: This probably won't happen today. It probably won't happen quickly if it does. + It feels like it may not happen. + - JDD: I have a sunnier disposition. Sounds like there might be a path here. + - Daniel: Same. Really sounded like Till's concern was about browsers taking on the "exceptional' behavior. + - Brad: Ben Newman from Meteor sounded like the current design was incomplete. + - Myles: May need to come up with a come up with something by the March meeting + - Saleh: need to have a parallel approach that can defer to a JS polyfill. + - Myles: I don't think the concern is implementation, just about consensus and staying conformant with the spec + - Guy: Node doesn't generally add its own functionality to V8, and keeping any sort of implementation synced with V8 might be a pain. + - Jeff: Can't have interop without planning for it. + - Brad: Saying "no ESM interop" could keep things open and work towards it, and use a loader otherwise. + - JDD: What's the future look like if it doesn't happen? + - Myles: fill this in + - Jordan: Trying to lock down a design in the next 2 months out of a sense of urgency is probably not the right way to go about this. +* Minimum to release? [#253](https://github.com/nodejs/modules/issues/253) + - 15 minute timebox + - Myles: Cut Node 12 in April - any semantically versioned changes need to go in by mid-March + - the only thing that would need to go in is the new "package fields" proposals + - but the longer it sits and stuff touches it, the harder it is to modify/remove + - Myles: We need to ship something. We have decision paralysis. + - Joke - we wanted to avoid a Python 2/Python 3 decision, but in the time it took to decide all this, Python has fixed their ecosystem + - Provide named exports in April if we can; if not, don't provide transparent interop. + - Figuring out file extensions and IRP proposals seem like they could be done, but must be done. + - Saleh: shouldn't take away experimental features that would otherwise be useful due to issues around risk + - JDD: We give people explicit warranties not to use experimental stuff! + - Jordan: people have to learn their lesson with breaking changes - they shouldn't be relied on. + - Guy: Having the capability with an experimental loader hook, you'd be able to generally address the broader concerns including interop. + - Myles: things we need to tackle + - \[\[Fill this in]] + - file specifier resolution + - Missing features + - loaders & VM + - If we copied over the minimal kernel as it exists today, we'd lose functionality for putting ES modules in VM. Is that a blocker? + - Gus: they're a separate feature, it should be fine to merge in both + - Jans: At least from what I remember, the VM module leaked a lot of information about how modules work. Keeping the VM module in the minimal kernel would actually restrict our design space. + - Jeff: adding a list to the required features, there's things like entry points. + - Myles: Realms are probably coming eventually - the truly perfect solution to VM modules would be realms. + - Brad: Realms don't let you create new modules. The APIs are *very* different as well. + - Myles: but can the same things achievable in Realms? + - Brad: largely, yes + - Myles: so what if we ship vm as deprecated the same way domains are? Would that be fine? + - Jans: it's always a known unknown. + - Myles: can you work on this between now and when we want to ship? + - Jans: sure + - Myles: can we just flag vm module support? seems like it's not blocking + - Action item: Myles to come up with a list of blocking and non-blocking issues + - Saleh: could imagine an experimental module loader that tries to achieve the same things as vm. + +* WIP \[Do not merge\] - Irp type modules [#29](https://github.com/nodejs/ecmascript-modules/pull/28) + - 15 minute timebox + - Refs: + - Mode: esm proposal [#247](https://github.com/nodejs/modules/issues/247) + + + +* Import file specifier proposal implementation [#256](https://github.com/nodejs/modules/issues/256) + - 5 minute timebox + +Can land if we have consensus by next wednesday diff --git a/doc/meetings/2019-02-13.md b/doc/meetings/2019-02-13.md new file mode 100644 index 0000000..9593642 --- /dev/null +++ b/doc/meetings/2019-02-13.md @@ -0,0 +1,123 @@ +# Node.js Foundation Modules Team Meeting 2019-02-13 + +* **Recording**: https://www.youtube.com/watch?v=W25K1c2e2y8 +* **GitHub Issue**: https://github.com/nodejs/modules/issues/266 +* **Minutes Google Doc**: https://docs.google.com/document/d/14pkjW9fnsgjImVabQQLLgtrJn2x_FCpMGUmoDJej_kQ/edit + +## Present + +- @DanielRosenwasser +- @jdalton (John-David Dalton) +- @SMotaal (Saleh Abdel Motaal) +- @robpalme (Rob Palmer) +- @inidaname (Hassan Sani) +- @weswigham (Wesley Wigham) +- @devamaz (Ahmad Abdul-Aziz) +- @devsnek (Gus Caplan) +- @guybedford (Guy Bedford) +- @Fishrock123 (Jeremiah Senkpiel) +- @MylesBorins (Myles Borins) +- @ljharb (Jordan Harband) + +## Agenda + +Extracted from **modules-agenda** labelled issues and pull requests from the **nodejs org** prior to the meeting. + +### Note + +The majority of this week’s discussion will be based on the following doc + +https://docs.google.com/document/d/1DSWrdV1fzXvlOdTZ5MngDX7v6CU4ZUheJ7ysOZ2uK0w/edit?usp=sharing + +Discussion in this issue + +https://github.com/nodejs/modules/issues/261 + +We will walk through contentious subjects, attempt to reach consensus quickly, otherwise move towards a vote. We will then review the resulting implementation and attempt to reach consensus that this is what we will move forward with. + +### Discussion + +* Agree on timeline + - 5 minute timebox + - Basically replacing the flagged implementation by April 2019, potentially unflag by October 2019 + - Any objections here? + - Jordan: Want to ensure that we don't try to rush anything based on deadlines. + - But no objections! +* What features are out of scope for timeline + - 5 minute timebox + - Proposal to omit `package.json` exports and VM modules + - Can put VM modules behind a flag + - JDD: does that mean VM modules can become stable before modules? + - Myles: technically the project can do anything since we're not a chartered working group, but we're assuming good faith decisions. Node core participants are aware that not everyone in the group is necessarily comfortable with VM modules going in before ES modules. + - Geoffrey: Doesn't feel like VM modules should be blocked, but definitely not with old module semantics + - Gus: Being the person who works on VM modules, I think I can keep track of that, and we don't need to worry about changes there. + - Objections to removing scope +* CommonJS interop + - 20 minute timebox + - Refs: + - CommonJS import interoperability decisions +[#264](https://github.com/nodejs/modules/issues/264) +- What is considered acceptable iteration can we do addon proposals? + - Breaking vs additive. +- What can be done that intersects existing tooling/support? + -Subset of behavior? +- What can be done that allows iterative migration? + - Subset acceptable? + - Make an update to Dynamic Modules Development in Node.js [#24894](https://github.com/nodejs/node/issues/24894) + - Import named vs default from CommonJS packages [#260](https://github.com/nodejs/modules/issues/260) + - Moving forward with Dynamic Modules? [#252](https://github.com/nodejs/modules/issues/252) + - CJS named exports via two-phase execution [#31](https://github.com/nodejs/ecmascript-modules/pull/31) + - Clarification: we mean `import`-ing a CJS, right? + - Right. + - should we continue to pursue dynamic modules in TC39? + - Brad: The "dynamic modules" behavior can be seen as an extension of "standard" behavior with ESM. + - Jordan: After shipping unflagged, it is much harder to take something back. It still feels like it's worth pursuing and something we could address. Much of the work would just be rewriting parts of the spec operating over Source Text Module Record (STMR) as operating over Abstract Module Records instead. \[\[please review]] + - Myles: Lin Clark is currently working on something called Cyclical Module Records, which is a new record type that sits between STMRs and Abstract Module Records + - Robert: Would Google be willing to ship that change? + - Domenic Denicola from Chrome suggested the change + - Myles: if we decide to move forward with out-of-order execution, doesn't seem like we'd need dynamic modules + - Guy: only reason I bring it up is we might be blocking work on dynamic modules + - Geoffrey: can always replace one with the other if it's compatible + - should we rely on out-of-order execution? + - Wesley: would be okay with OOO-execution; most code that relies on execution order can be rewritten with workarounds. + - Brad: we have run into security issues based on execution ordering exploits + - Myles: \[\[something about top-level `await` - please review]] + - Brad: top level await is definitely unrelated + - Guy: I believe that out of order execution is a spec violation - CJS doesn't have a corresponding "module type" in the ES spec. + - Brad: Out of order execution means that you can't safely refactor without sometimes different behavior. + - Guy: Instantiation is an asynchronous phase - any new top level module starts at an instantiation phase. Execution is synchronous and well-defined. + - Jordan: Idea is that this goes against the "spirit/intent" of the spec. TC39 says they would forbid OOO-exec if they could + - Myles: we're focusing a lot on whether we violate the spec and are almost out of time. Guy has two implementations, one with named exports and an OOO-exec implementation. Can we bring this forward to TC39 in March and + - Brad: Just to add, OOO-exec isn't something we can do as an additive phase. + - Myles: but do we have consensus to bring this to TC39? + - Brad: I won't be presenting OOO execution again + - Myles: I will be happy to present. + - Consensus to solving named exports using OOO execution? + - Against: Myles, Gus, Bradley, Jeremiah, Jordan, Rob, Kevin, Hassan (8) + - For: Daniel, JDD, Wesley, Jan, Geoffrey (5) + - is it even possible to ship named exports in interop? + - Geoffrey: If this was the only way to get named exports, would we still pursue it? + - Gotta move on. + - is it okay to only ship CJS as default exports? + - Daniel: no if we think we can do better, you give people a blessing/mandate to write in that way + - Jordan: to specifically add to that, I agree, but would be fine with this only if named is impossible + - WIP \[Do not merge\] - Irp type dynamic modules [#29](https://github.com/nodejs/ecmascript-modules/pull/29) +* File Extension Resolution + - 10 minute timebox +* Loaders + - 5 minute timebox +* Requirements for different phases + - 5 minute timebox + - Refs: + - * Minimum to release? [#253](https://github.com/nodejs/modules/issues/253) +* Requirements to remove flag + - 5 minute timebox + - Refs: + - Entry points proposal spec and implementation [#32](https://github.com/nodejs/ecmascript-modules/pull/32) + - Jordan: no objection if it's flagged + - Import file specifier proposal implementation [#256](https://github.com/nodejs/modules/issues/256) + - Mode: esm proposal [#247](https://github.com/nodejs/modules/issues/247) + + + + diff --git a/doc/meetings/2019-02-20.md b/doc/meetings/2019-02-20.md new file mode 100644 index 0000000..83369f0 --- /dev/null +++ b/doc/meetings/2019-02-20.md @@ -0,0 +1,122 @@ +# Node.js Foundation Modules Team Meeting 2019-02-20 + +* **Recording**: https://www.youtube.com/watch?v=H34VieUOGu4 +* **GitHub Issue**: https://github.com/nodejs/modules/issues/270 +* **Minutes Google Doc**: https://docs.google.com/document/d/1AFnHc85SzFZFUqL6xL7xTDOmFB9FM6q9yapjYGQiXi0/edit + +## Present + +- John-David Danger Dalton (@jdalton) +- Myles Borins (@MylesBorins) +- Bradley Farias (@bmeck) +- Wesley Wigham (@weswigham) +- Daniel Rosenwasser (@DanielRosenwasser) +- Gus Caplan (@devsnek) +- Jeremiah Senkpiel (@Fishrock123) +- Jordan Harband (@ljharb) + +## Agenda + +Extracted from **modules-agenda** labelled issues and pull requests from the **nodejs org** prior to the meeting. + +### Note + +This is an out of band follow up to last week’s meeting. The majority of this week’s discussion will be based on the following doc + +https://docs.google.com/document/d/1DSWrdV1fzXvlOdTZ5MngDX7v6CU4ZUheJ7ysOZ2uK0w/edit?usp=sharing + +Discussion in this issue + +https://github.com/nodejs/modules/issues/261 + +We will walk through contentious subjects, attempt to reach consensus quickly, otherwise move towards a vote. We will then review the resulting implementation and attempt to reach consensus that this is what we will move forward with. + +### Discussion +* Review last weeks discussion + * 5 minute timebox +* CommonJS interop + * 10 minute timebox + * https://github.com/nodejs/ecmascript-modules/pull/261 + * Out of order execution (https://github.com/nodejs/ecmascript-modules/pull/31) + * Leaned against out-of-order execution + * Shipping just default exports + * Leaning towards not doing this if named exports are a possibility. + * Removed some of the functionality here, but the "plumbing" is there + * Myles: could we put that behind a flag? + * Daniel: Seems reasonable, but just concerned with the name. call out that it's an interop strategy. + * Jeremiah: Shipping without interop seems like not an option. + * Myles: Just as an FYI, pushing back means we'll need to go to a vote + * Myles: So do we have objections? + * Geoffrey: wouldn't feel comfortable shipping because messaging might be confusing if that strategy isn't what we want to go with in the first place. Also, if we take it back, people could say we're "removing interop". + * Myles: Don't think that's true, we're within our right to remove anything flagged. + * Jordan: meta-point: certain choices have to be made for different use-cases. Making different choices shut down certain use-cases. Want us to think about our use-cases in the final product. + * Wesley: we're generally in agreement that we and TC39 want to support named exports. Why can't we ship one of the proposals behind a flag? + * Brad: which one? OOO-exec? JDD's? Guy's? + * Wesley: not the OOO-exec one + * Brad: `export *` has come up in the committee again. + * Wesley: these changes aren't observable for code people tend to write + * Brad: These changes are observable. + * Wesley: Minor in comparison to the change of not having any interop + * Gus: these are pretty observable + * Wesley: for people writing code, are they observable? + * Gus: Not necessarily, but what about bundlers? Linters? Those need static imports + * Wesley: all of these tools already analyze just fine despite dynamic behavior. + * Myles: we need true static analyzability for the sort of cold start improvements that we hope to achieve in Node core. + * Jordan: this is the sort of thing that can't be handled in TC39. \[\[fill in later]] + * Wesley: going with a non-final version of named imports is the more reasonable thing to do than shipping a default only + * Brad: would you be okay throwing that implementation out if we can't make it work? + * Daniel: we'd potentially have to throw some implementation out no matter what we ship + * Wesley: and shipping named exports is more aligned with what this group wants to ship + * Brad: counter-point: should we ship the thing we're more likely to be able to ship? + * Myles: There are more people who want no interop than no named interop (with just a default import available) + * Named exports + * Have a path towards supporting it with dynamic modules. + * Some people might feel discouraged from the work, but it's an open path. + * Geoffrey: can we just ship this unflagged? + * \[\[objections]] + * JDD: would be against even a flagged implementation shipping + * Myles: is this because `export *`? + * JDD: not clear that dynamic modules will solve this exactly; the + * Brad: strong mischaracterization of Guy's work, calling it a failed proposal is unreasonable! + * JDD: Just because one person has worked on it doesn't mean we need to rubber-stamp it. I am a delegate and I would block the current proposal as-is. + * Brad: work being from just one person doesn't mean it's good or bad, it's something that we can iterate on + * JDD: if we're trying to come to the best design, I don't believe this is the right direction. + * Jordan: why do you find default acceptable, but the rest of it not? + * JDD: `default` doesn't cause any spec changes, has a reasonable precedent. Would rather start from low-to-no-effort spec work. Otherwise you have to throw on syntax. + * Wesley: No named exports means you *do* have errors on syntax. + * JDD: no, that's erroring on bindings that don't exist + * Wesley: there's no difference as a user!!!! + * Refs: + * CommonJS import interoperability decisions [#264](https://github.com/nodejs/modules/issues/264) + * Make an update to Dynamic Modules Development in Node.js [#24894](https://github.com/nodejs/node/issues/24894) + * Import named vs default from CommonJS packages [#260](https://github.com/nodejs/modules/issues/260) + * Moving forward with Dynamic Modules? [#252](https://github.com/nodejs/modules/issues/252) + * CJS named exports via two-phase execution [#31](https://github.com/nodejs/ecmascript-modules/pull/31) + * WIP [Do not merge] - Irp type dynamic modules [#29](https://github.com/nodejs/ecmascript-modules/pull/29) +* File Extension Resolution + * 15 minute timebox + * Refs: + * File extension/directory index resolution in ESM [#268](https://github.com/nodejs/modules/issues/268) + * Existing resolution algorithm for CJS searches through a handful of extensions to search. + * Currently the minimal kernel doesn't support this. Should we? People on the committee don't want this to begin with. + * Brad: we should be okay shipping without it initially and then shipping. That way we have two paths later on. + * Geoffrey: if we make this opt-in, that's fine. users become aware of the behavior. + * Jordan: I would object to shipping without it unflagged. I find extension lookup so critical that I don't think we can ship without it. + * Gus: to people who don't want to ship by default, I'm confused. If you don't want this feature, don't use it and just be explicit specifiers. + * Myles: we want an ecosystem of modules that can be shared across all ecosystems/runtimes. If you allow people mixing, your dependencies can throw a wrench in the mix if their paths don't work. + * Could we use something like import maps to make this easier. + * Myles: that doesn't work, import maps would blow up exponentially in size to model the entire dependency tree. They're meant for modeling only a few bare imports. + * Gus: I could totally see Yarn or npm shipping something that just generates these files anyway. +* Loaders + * 5 minute timebox +* Requirements to replace upstream + * 10 minute timebox + * Refs: + * Minimum to release? [#253](https://github.com/nodejs/modules/issues/253) +* Requirements to remove flag + * 5 minute timebox + * Refs: + * Entry points proposal spec and implementation [#32](https://github.com/nodejs/ecmascript-modules/pull/32) + * Import file specifier proposal implementation [#256](https://github.com/nodejs/modules/issues/256) + * Mode: esm proposal [#247](https://github.com/nodejs/modules/issues/247) + diff --git a/doc/meetings/2019-02-27.md b/doc/meetings/2019-02-27.md new file mode 100644 index 0000000..c43ac6f --- /dev/null +++ b/doc/meetings/2019-02-27.md @@ -0,0 +1,107 @@ +# Node.js Foundation Modules Team Meeting 2019-02-27 + +* **Recording**: https://youtu.be/wYKeySvX8EI +* **GitHub Issue**: https://github.com/nodejs/modules/issues/276 +* **Minutes Google Doc**: https://docs.google.com/document/d/1gXpUwJGPAgHNLqbOdqKEoRQsbdQwqgXAC2kC_tyQ6Tk/edit + +## Present + +- Myles Borins (@MylesBorins) +- Gus Caplan (@devsnek) +- Wesley Wigham (@weswigham) +- Geoffrey Booth (@GeoffreyBooth) +- Jan Krems (@jkrems) +- Saleh Abdel Motaal (@smotaal) +- Jeremiah Senkpiel (@Fishrock123) +- Guy Bedford (@guybedford) +- Matt DuLeone (@mduleone) +- Jordan Harband (@ljharb) + +## Agenda + +Extracted from **modules-agenda** labelled issues and pull requests from the **nodejs org** prior to the meeting. + +### Note + +This is an out of band follow up to last week’s meeting. The majority of this week’s discussion will be based on the following doc + +https://docs.google.com/document/d/1DSWrdV1fzXvlOdTZ5MngDX7v6CU4ZUheJ7ysOZ2uK0w/edit?usp=sharing + +Discussion in this issue + +https://github.com/nodejs/modules/issues/261 + +We will walk through contentious subjects, attempt to reach consensus quickly, otherwise move towards a vote. We will then review the resulting implementation and attempt to reach consensus that this is what we will move forward with. + +### Review Open PRs (15 minute timebox) + +All discussion are limited to 5 minutes and will be ended early if it is obvious there is no consensus + +* -m alias for --type=module [#39](https://github.com/nodejs/ecmascript-modules/pull/39) + +Consensus to land and discuss further if necessary. +Also discussing [#45](https://github.com/nodejs/ecmascript-modules/pull/45). + +Can land in 48 hours based on review + +* Exports main [#41](https://github.com/nodejs/ecmascript-modules/pull/41) + +To be revisited after extension resolution; ask to review extension resolution in this meeting in order to unblock + +* esm: add experimental .json support to loader [#43](https://github.com/nodejs/ecmascript-modules/pull/43) + +Skip for now + +* Warn about `--type` with shebang [#37](https://github.com/nodejs/ecmascript-modules/pull/37) + +* ESM_FORMAT: Avoid reading package.json when extension is unambiguous [#33](https://github.com/nodejs/ecmascript-modules/pull/33) + +#44 adds tests to be merged into other PR #45 + +### Discussion (40 minutes) + +* Review last weeks discussion + * 5 minute timebox + +* What is an MVP to upstream? + +Reviewing https://github.com/nodejs/modules/pull/280, the revised road map + * No objections to overall structure or phases 0 or 1 + + + * 10 minute timebox + * Refs: + * Minimum to release? [#253](https://github.com/nodejs/modules/issues/253) + * Entry points proposal spec and implementation [#32](https://github.com/nodejs/ecmascript-modules/pull/32) + * Import file specifier proposal implementation [#256](https://github.com/nodejs/modules/issues/256) + * Mode: esm proposal [#247](https://github.com/nodejs/modules/issues/247) + +* Loaders + * 5 minute timebox + +* File Extension Resolution + * 10 minute timebox + * Refs: + * File extension/directory index resolution in ESM [#268](https://github.com/nodejs/modules/issues/268) + +* CommonJS interop + * 10 minute timebox + * Refs: + * Proposal for dual ESM/CommonJS packages [#273](https://github.com/nodejs/modules/issues/273) + * CommonJS import interoperability decisions [#264](https://github.com/nodejs/modules/issues/264) + * Make an update to Dynamic Modules Development in Node.js [#24894](https://github.com/nodejs/node/issues/24894) + * Import named vs default from CommonJS packages [#260](https://github.com/nodejs/modules/issues/260) + * Moving forward with Dynamic Modules? [#252](https://github.com/nodejs/modules/issues/252) + * CJS named exports via two-phase execution [#31](https://github.com/nodejs/ecmascript-modules/pull/31) + * WIP [Do not merge] - Irp type dynamic modules [#29](https://github.com/nodejs/ecmascript-modules/pull/29) + + + + + + + + + + + diff --git a/doc/meetings/2019-03-04.md b/doc/meetings/2019-03-04.md new file mode 100644 index 0000000..9b7964a --- /dev/null +++ b/doc/meetings/2019-03-04.md @@ -0,0 +1,96 @@ +# Node.js Foundation Modules Team Out of Band Meeting 2019-03-04 + +* **Recording**: https://www.youtube.com/watch?v=5TGRRlFbbEA +* **GitHub Issue**: https://github.com/nodejs/modules/issues/282 + +## Present + +- John-David Dalton (@jdalton) +- Myles Borins (@MylesBorins) +- Jan Krems (@jkrems) +- Geoffrey Booth (@GeoffreyBooth) +- Kevin Smith (@zenparsing) +- Wesley Wigham (@weswigham) +- Jeremiah Senkpiel (@Fishrock123) +- Jordan Harband (@ljharb) + +## Agenda + +### Automatic File Extension/Directory Index Resolution + +- Myles summarized https://github.com/nodejs/modules/blob/master/doc/plan-for-new-modules-implementation.md +- Myles offers proposal: build automatic file extension/directory index resolution behind an additional flag +- He feels without it included we leave open design space, while with it we exclude space +- He wants to allow the ecosystem to experiment with this, get responses from the community to see how much or how little they want this feature +- Concern is that without having it at all, without flagged, there’s no opportunity for experimentation; whereas without the flag, no one will likely experiment with not using searching + +- Jordan: most controversy is around defaults rather than capabilities. If the flag doesn’t alter likelihood of whether or not it ships as default, we would be fine with flagged; but this likely does alter that calculus. +- If we ship extension resolution off by default and on by a flag, versus on by default and off by a flag, and you think that people will experiment more with the former, doesn’t that speak to user desire? +- Myles: People already have code assuming searching, so we don’t really know what the desire is. Also is design space we haven’t figured out yet, such as “main” field for dual packages. To the point on defaults and capabilities, turning on later is something we will likely offer later even as an app-level loader but we don’t have consensus as to what should be the default. + +- Guy: Follow up as to polarity on this: people already expect extension searching to work, and what we’re doing by removing it is that we’re saying that this is under consideration, and we should make that clear. If we ship it in the same, and we change it later it will seem like taking the candy away from the baby. Changing it now sets expectations. These arguments come down to managing users’ expectations. + +- Gus: Have a few things to say. I’ve actually made this enormous JS engine written in JS, and as an experiment I’ve not used any extension/index resolution, and my experience with that has been that it results in maybe three times I’ve had to run a find/replace regex to fix things resulting in that. Myles: Is that unacceptable or reasonable? Gus: I was able to do it, but it was terrible. +- Myles: Gus, to your point, my feeling is that the intuition that you have and Jordan has that we have to ship this whether we like it or not because the community expects it may be correct. If we ship it behind a flag, we will get a signal from the community extremely quickly. If we don’t ship even with a flag, we might get pushback even before merging the upstream. Shipping with a flag lets people opt in to the experience they want and give us the feedback. If we ship with it on, we won’t get that feedback. Doing it this way (with a flag) gives us an opportunity to do what we were hoping to do with the polls. Doing our upstream implementation gives us one shot to do some ecosystem advocacy to get awareness and feedback to help guide us moving forward. A flag may create friction but that would be good, to generate feedback. + +- JDD: Does Node have any precedent in opting into telemetry for flag use? Gus: Not that I know of. Geoffrey: We could query GitHub to look for package.json run scripts to look for flags used in projects. Myles: Can help with scanning work. + +- Kevin: Would caution that design space is not as open as we’d like it to be. Lot of choices you can make for resolution, but once you start making choices you find yourself following choices similar to current CJS implementation. Another choice would be going in a completely different direction like export maps. If we did have a flag that might lead the process in one direction rather than another. + +- Jordan: What are we trying to achieve with user feedback here? It should be taken as a given that there will be nonzero people that will find their workflow seriously impeded without extension resolution. What are you expecting to get? +- Myles: My intention would be to see how strong the reaction from the community is. Similar to .mjs, we’ve found ways to appease the complaints that people brought forward. It’s possible that the feedback we get may not be entirely actionable, but it will give us more information than we have today. We may get positive feedback. +- Geoffrey: Other users might want this disabled by default in order to get packages to be browser-safe by default. +- Gus: Those people have failed to persuade packages to change. +- Jordan: Why should that argument influence us? + +- Kevin: I think we can predict that people will take a look at the flag and compare it against the experience they get with “esm” and Babel and feel like things are missing based on that comparison. To try to make some attempt to get away from that comparison was my drive to create the “web modules” for Node implementation. If we present it without a new story we can predict the feedback we’ll get. + +- Guy: On the user feedback point, it’s important to note that the user feedback we get doesn’t make the decision for us, just as with .mjs it’s still our decision. There are many other factors that play into this decision, and to state a strong argument like Gus does that it’s impossible to make node_modules work in a browser is a strong argument without evidence. +- Gus objects, crosstalk, discussion of unpkg.com. Gus claims that there’s still bundling happening somewhere. +- Guy: It’s not an obvious point and we should be careful in such a design space, there are new things happening on the Web that affect how the whole ecosystem is going to work. And to deny that the ecosystem is shifting is also a mistake. Where we sit in the middle it’s a difficult thing to predict. We will have more information in 3-6 months’ time to more fully investigate this space. There are things coming out like pika, we should hold back and see what happens before we make a decision. +- Myles: Building on what Guy said, turning it on by default makes decisions that influence other things like dual packages. + +- Saleh: My personal experience: requiring .mjs in experimental, the way resolution worked with a loader you could bring in extension matching etc. Just the fact that I could not resolve without .mjs early on changed all the tools that I was relying on forced me to explore different workflows than I was used to. The insights I got shocked me, I went into directions I didn’t think I was going to explore. A year and a half later, I think we have to be careful saying that because it worked it should continue to work and that since people didn’t show favor in that direction it should continue to. Since Myles proposes that this is an opt-in conscious decision, whoever’s experimenting knows which path they would like to see this move forward with. Forcing them to experiment give us a more complete picture as to what people think works for them. + +- Gus: We can enable extension searching without dual-mode by changing around the loader, I have a POC. +- Myles: Based on the conversation that we’ve had, do we have consensus on moving forward with this behind a flag? (silence) +- Jordan: To clarify, this is still a question to be settled in Phase 3? +- Myles: Yes. As we upstream this, we will need to reevaluate how we work as a group as to how we tackle these problems. Re “exports”, extensions map, there’s a wave of other things that making a decision on this will inform. +- Jordan: Can we have either option be achievable via flag? E.g. turning it off is also a flag option, even if it’s a noop for now. +- Myles/Geoffrey: Yes. Like --type=commonjs right now. + +### Other topics + +https://github.com/nodejs/ecmascript-modules/pull/47 + +- Reverts commit that we had removing the loader API +- Myles and Guy discussed and felt that there wasn’t a way to move forward with the new loaders implementation before mid-March without making decisions that Brad should be a part of. +- Will add back in old loaders implementation with extra warning that screams that this loaders implementation is deprecated and will be replaced. So people know, and we’re not breaking current workflows, but users are warned. +- Any objections? None. +- Will document this as deprecated in the docs. +- Any objections to landing this by Wed or Thu? None. + +https://github.com/nodejs/modules/issues/286 + +- Steps to upstream document +- Myles explains that dates are based on Node 12 release schedule, the dates prior are backdated from that. +- We should try to get remaining work done by the 13th so that we can open the PR by March 20th. +- Any concerns/thoughts around these dates? +- Jordan: Wants extensions map in next few weeks, but okay with dates. +- Jan: No special meaning about what can go in or can’t go in? +- Myles: Right. Once we land things upstream we need to think about what it means to keep working on this. Keep moratorium upstream? Keep fork? Charter our group? There’s a whole slew of other governance-related questions we need to work on, but there’s no reason we can’t upstream our monolith and iterate when we get there. +- Myles: To your point about extension maps, our current consensus on phase 2 did not include them, but yet we could add them quickly afterward. +- Jordan: We may want to reevaluate that we have a sugar for a specialized subset for what my implementation achieves. I’m less confident that we’ll be able to rip out that sugar that lives in published packages. +- Myles: I’m certain that we will need to change things that will affect packages that people have published if they did so based on our experimental branch. I’m hoping that those breaking changes are at a faster pace. And I’m hoping future iterations will happen far faster. Does that appease your concern? +- Jordan: Hopefully. + +- Next step is to update planning document: move loaders to phase 3, update to reflect this meeting re extension searching behind a flag. +- Geoffrey: I’ll make a PR to do that, can I get consensus that such a PR can land? No objections. + +https://github.com/nodejs/modules/issues/286 + +- Any objections to PRs that specifically concern the points in the TODO list for upstreaming being merged between meetings if the PR has no objections for 48 hours? +- Discussion: Just related to this list? Yes, this is just the stuff in 286, docs, old loaders implementation, extension searching behind a flag. No new features slipping in, no extension maps. +- No objections. + +- Myles congratulates the group on finding consensus and is miraculously not cut off mid-word. diff --git a/doc/meetings/2019-03-13.md b/doc/meetings/2019-03-13.md new file mode 100644 index 0000000..a33dc85 --- /dev/null +++ b/doc/meetings/2019-03-13.md @@ -0,0 +1,228 @@ +# Node.js Foundation Modules Team Meeting 2019-03-13 + +* **Recording**: https://www.youtube.com/watch?v=0vB55lvVqFc +* **GitHub Issue**: https://github.com/nodejs/modules/issues/292 +* **Minutes Google Doc**: https://docs.google.com/document/d/1N2pz8grs6WpOymgo862T3al3EwIwL7bo7ovwLxyPqtQ/edit + +## Present + +- Myles Borins (@MylesBorins) +- Michael Zasso (@targos) +- Jordan Harband (@LJHarb) +- John David-Dalton (@jdalton) +- Wesley Wigham (@weswigham) +- Guy Bedford (@guybedford) +- Saleh Abdel Motaal (@smotaal) +- Gus Caplan (@devsnek) +- Rob Palmer (@robpalme) +- Daniel Rosenwasser (@DanielRosenwasser) +- Geoffrey Booth (@GeoffreyBooth) +- Jeremiah Senkpiel (@Fishrock123) + +## Agenda + +Extracted from **modules-agenda** labelled issues and pull requests from the **nodejs org** prior to the meeting. + +### Approving PRs with no blocks (6 minutes) + +2 minutes per item. + +* Cpp Refactoring Ensuring avoiding duplicate package cache [#58](https://github.com/nodejs/ecmascript-modules/pull/58) + * No objections + +Land away + +* esm: add experimental .json support to loader [#43](https://github.com/nodejs/ecmascript-modules/pull/43) + * Functionality: No named exports, default only + * No objections + +* Warn about `--type` with shebang [#37](https://github.com/nodejs/ecmascript-modules/pull/37) + * Punting on this +### Discuss PRs that have blocks (10 minutes) + +* ESM_FORMAT: Avoid reading package.json when extension is unambiguous [#33](https://github.com/nodejs/ecmascript-modules/pull/33) + * Jordan: If it's premature optimization, don't do it because it could be a distraction + * No objections to not landing...so… + * Not doing it yet! +* esm: scoped `--type`, cpp refactoring [#57](https://github.com/nodejs/ecmascript-modules/pull/57) + * Need to clarify whether `--type` is a mode. + * When set to esm, all `.js `files are ESM, commonjs implies `.js` is CJS. + * Alternative viewpoint is that it's the parsing goal for the entrypoint. + * Explicit signal is only needed, if no package.json, or no extension (and no package.json) + * Jordan: Objects to a mode. `.mjs` is ESM. `.js` is CommonJS. The defaults are there for a reason. `.js` should never default to anything but CommonJS (absent an overriding explicit per-file signal). + * Guy: use-cases are good starting point for file that has no package.json; problem is when a file has a relative import to a sibling module with a `.js` extension + * If another file imports your entry-point as a dependency, that means your code can behave differently. + * Should we instead think of the type as the same as putting a `package.json` in the folder that contains that type. It’s like a virtual package.json. + * Current behaviour it is only an entrypoint. + * Gus: --type is a really blunt tool to use. File has been affected by another file importing it. At that point you should be using package.json. + * Myles: Field for CLI and package.json need to have the same name or we'd confuse people. I like the idea of flipping modes without package.json. Thinking of it as flagging the entrypoint is useful but more confusing. That Developer Experience should not call the command line flag “type”. +Salah: Consider usefulness. If type is module, it’s really about semantics of file execution. Type flag is for toggling the mode of things that load other things? Or is it just setting semantics? +Jordan: Importing means nothing about the target. It could be WASM - more than just CJS/ESM. +JDD: I assumed --type was another way to do package.json type field - discovered differently in the PR! Creating the virtual package.json is fine by me. +Geoffrey: If we don’t have it like the fields, type=auto is useless. Not sure why we didn’t do it like the field - it still solves the one-off case and other cases. Can experiment with “let’s just try running it as ESM”. +Jordan: I object to this mode. +JDD: Vote? +Myles: We wanted to open a PR this week. Consensus is ifi it’s called types, it should match package.json but Jordan doesn’t like this. +Jordan: We can bikeshed the name outside. +JDD: Unless it’s like “type”, we should not do it. +Guy: Could call it --format? +JDD: I have experience with API design. I disagree with it. +Geoffrey: People are opposed to a flag over editing files? +Jordan: I am concerned about a world where people pave over the reasons .mjs exists. +Salah: If you like running with the flag on, you get the right results, or you fix your scripts. People have to make good choices. +Myles: I believe keeping it as --type and same as package.json is better - it is easier to learn. If we introduce a whole new thing to learn, it’s not a relevant or useful feature. The intent is what Guy is providing. +Jordan: This is not what I was told. We should revert it until we have consensus, or we slip schedule, or rename flag. Need more time to talk about it. +Myles: so for now, change the flag to be mode and only apply to entrypoint. +Geoffrey: I object - ship it as is and users can complain. +JDD: Let’s revert if we don't have consensus. +Jordan: We can revert if changes are fully discussed first. +Myles: This allows unwinding any consensus. I understood the implemented behaviour. +Jordan: Discussion on PR was misunderstood. +Myles: Majority here agree on behaviour. Multiple people object to your alternative. +Guy: This has always been unclear to me. The elegant behaviour is to operate at a scope level, but it’s non-obvious. I have no opinion either way. So what is the harm of renaming? +Geoffrey: It will never be the user’s intent. +Guy: We can use bug reports from users based on feedback. And therefore it needs to be renamed. What’s the risk of shipping? +Salah: You have to make people assume the right thing to get data. +Myles: Objection for renaming when we upstream? +Geoffrey. Yes +Object to landing the PR? +Jordan: Yes. +Myles: Leave it how it is. Current situation is the worst for everyone. + +### Upstreaming (25 minutes) + +* discuss ‘.cjs’ [#293](https://github.com/nodejs/modules/issues/293) +Myles: In module mode, there is no way to import CJS without it. +Geofrrey: This is why is was added: +Jordan: Agree, when you have a mode. In absence of extension map, you’re right. But I don’t like modes. With ext map, you can make any extension you want be CommonJS. Until we have strong consensus on modes. +Yles: We have consensus on modes. It landed. Stuff relies on this. Removing it now makes the DX unusable. +Jordan: You’re saying the concession is something I should not have done. I said “this should not exist when we have extension map”. +Myles: We can revive it when we have ext maps. We can’t remove it now. +Jordan: Worried about ecosystem relying on it. I intend to remove it in future. +Myles: Today you can require an extension that doesn’t exist it falls back to CJS. +JDD:It’s super cool. +Jordan: Requiqr with explicit extension falls back to cJs? +Myles: Yes +Jordan: So we don’t need special handling of CJS. +Myles: This is specific to our loader. Can be overridden by export maps later. In CJS it will fallback to CJS. In ESM it will fallback to ESM. +Jordan: We should think more about this. Good to hear it works today! It’s necessary to have new extensions for new parsing goals. Concerned about setting expectation that CJS should be in CJS files. +Myles: But we need ESM mode where we make it easy to import CJS without that extension. People shouldn’t have to opt in to import maps just because they want to use CJS from ESM. +Salah: Will it be more problematic to use CJS from ESM mode if we remove the mode? +Jordan: I want to talk about Wes’ comment. +Gus?: If you want to use .js but you need to rename CJS, why not just rename your .js to .mjs. The use-case here seems to be to avoid .mjs. +Geoffrey: CoffeeScript needs .js! Tools, transpilers need it and allow mixing. +Gus: Does your build tool understand .cjs? +JDD: I like the symmetry of .cjs. Future is ESM. So CJS is legacy. So the wart should decrease over time. +Myles: Does anyone object to .cjs? +Jordan: Not a fan. Would like another solution. It is hostile to the ecosystem that needs .mjs +Myles: I disagree. We have no other code with another approach. You’re saying we remove the current reference to .cjs - does anyone object? +Geoffrey: Yes, I object. There’s value in what we have. We can flip the default in future. +Jordan: We could add cjs later. +Myles: We are in the same as --type. We don’t have an approach. Does anyone object to continuing forward? +Jordan: CJS snuck in. I do not view it as having consensus. +Myles: You are forcing a deadlock for the second time. +Jordan: Any alternative needs to remove CJS, but there’s no consensus to remove. +Gus: There’s a lot of work to implement a loader. IT’s different to having something built in that works. +Myles: a package.extensions that has a mapping with translaters could exists, but there’s no implementation. +Jordan: I filed an issue a week ago. +Jeremiah: It unclear to me when loaders will happen. I don’t see this solving itself. +Myles: Jordan, you object to shipping .cjs and is disingenuous and you wouldn’t have agreed to it if you had known. But you have not brought an alternative. +Jordan: You don’t need to write code to object to decisions. I filed an issue for the extensions map. IT’s straight forward. Mentioned 18 months ago. The responsibility is not only on the objector. The shipper must do the work. If folks want to ship the type flag, and I want import map, they can implement that. Writing code is not a requirement. +Myles: But no implementers means it’s not happening. +Jeremiah: Our governance is different. +Geoffrey: shall we vote? +JDD: If deadlocked, that’s the next step. +Myles: I apologise for the approach. We do not have consensus on type, or modes, or landed stuff - the entire implementation. +Jordan: Not everything. Type is an enhancement - we could remove it. Altering the defaults is something people should have. +Myles: Go back to package type? +Jordan: we might want to do it, after we get extension maps. Yes it’s still on the table. Just because it has landed does not make it permanent. +Myles: We have minimal kernel, package types. It feels like you are unwinding consensus. +Jordan: Not proposing to remove … Its presence in the implementation does not imply consensus. IT should not accelerate other ideas unless we say it doesn’t have consensus. +Myles: How should we move forward? +Jordan: It’s frustrating if I suggest things but can’t do it. Sorry. I want to see extensions map go in. So ergonomics of CJS no longer applies. Then, having type:esm be sugar for an extensions map - needs thinking time - might be less problematic. It does not force file extension opinions. Makes me feel better about shipping type sugar. Then we’re saying, this is common, but you can do any pattern. Want extensions map, then we have pleasant discussions because we’re talking about refinements. +Myles: Objections on this path forward? +Guy/Geoffrey: We object. It’s not an argument. It’s about UX. It doesn’t matter if there is a more foundational way to define extensions. We’re designing UX in Node. +Jordan: No - we’re doing both. +Salah: We’re throwing experimental modules, the default mjs does not exist except in the old. We started fresh and there’s no default. So mjs goes out the window. +Jordan: We reacquired consensus for MJS being default. +Salah: I never voted on that. +Jordan: The needs are still there. +Myles: We have not achieved much today. +Jordan: [to Guy] we are trying to design the most perfect system we can. We can’t know how this will be used in future. Usability comes from ability. In future, people may want to use arbitrary extensions for arbitrary things. You can’t say people don’t want this. +Guy: I believe those cases will be fulfilled by loaders. Loaders are the more powerful way to deal with arbitrary extensions. I don’t object to extension maps, but don’t see where it gets us. How does it move us forwards? +Wes: Text editors. Arbitrary extensions are great. TypeScript and IDEs don’t support arbitrary extensions. There’s a reason we’re not fixing it. There are way more people who are happy with defaults. I don’t think the ability to have arbitrary extensions should block shipping. We have regular traffic on .mjs. Unless you tell lots of people to use it, they won’t use it. Don’t see why it needs to be a requirement. +Myles: Jordan, you said type could be sugar for an export map. It’s reasonable that in mode module, where .js is ESM, it’s likely we’ll want to map common js from one. There’s not much userland difference. Seems to me that waiting for exports should not block type stuff right now if it eventually turns into sugar. +Jordan: Agree, technically. My concern is about messaging and best-practice. Modules is a place where we create implicit recommendations. +Myles: When we get extension map, we’ll want a sugary default. So we end up in the same place. +Jordan: It enables very different forms of feedback. +Myles: Are you blocking upstreaming until we have extension maps? +Jordan: Needs talking about. +Myles: Do we need sugary defaults or not? I think it makes sense for loaders. There’s value in having ecosystem defaults, instead of a 100 different things. Do you assume extension maps will reach consensus? +Jordan: I am not saying we must ship extension maps. If we got consensus on an implementation, I would probably remove my objections so we could implement them after upstreaming. +Salah: You can’t offer time to implement, so assuming we say yes, there needs to be some willingness to implement in the group. Do we have capacity to implement this? +Jordan: I may have some time in a couple of months. Would be willing after that. +Guy: We hoped to move forward with upstreaming soon. We can change things later before unflagging. We have two blocks from Jordan: type flag and CJS extension. Should we vote or revert? +Rob: I’d be happy to vote. +Guy: How about we put in what we have, and we put in items to re-evaluate later? E.g. we can remove the documentation. Is there a middle ground? +Jordan: Yes, I would prefer to remove the documentation. There are already going to be massive renames. Tool configs will change. Renaming to .mjs is safe. Renaming to .cjs is not safe. We risk ecosystem churn whenever you blog. We have talked about minimal kernel to avoid bad signals. +Guy: Can we mark these explicitly as phase 3? Would it alleviate concerns? +Jordan: That would be an improvement, regardless. I would feel better if we got consensus on extension mapping. The loader we have does an extension map. It is minimal work. It won’t be time-consuming. We just need to decide what it needs to do. I don’t think it’s useful to have .mjs and.cjs files. +Myles: I am concerned about the patterns triggered by extension maps. The eventual implementation of loaders would be the extension maps. I am concerned about introducing extension maps too soon - it adds extra package lookups. It is additional work on a per-scope basis. I would like us to have as little in package.json as possible. “Type” is good - parallel with script tag. Loaders will have package.json impact. +Jordan: Extensions map alleviates most use of loaders. +Salah: That is not accurate. I experimented with that. +JDD: Should we vote? +Myles: I will not participate in a vote. +JDD: How do we call it? +Jordan: confirming that only allowing CJS is blocking upstreaming +JDD: I am calling the vote. Who is in favour of CJS moving forwards? ++1 means move forwards with .cjs +Results: + 4 in favour + 6 neutral + 1 against +Jordan: For anyone raising PRs, please actively seek objections to surface objections sooner rather than alter. +Geoffrey: I have seen a lot of objections recently of “I don’t like this” rather than constructive comments. + + +* Steps required to upstream implementation [#286](https://github.com/nodejs/modules/issues/286) + - 15 minute timebox +Myles: We need to add docs for phase 3 around type flag. Re-adding folder resolution. Have updated docs to reflect latest implementation. Geoffrey, is the docs ready? +Geoffrey: What are we doing about --type? Anything else that lands needs docs. It’s basically done. +Salah: Can we put a gist on the reliable way to use it? +Myles: I’ll put a test build there. +Geoffrey: I’ll add readme. +Myles: We have a blog post - 291. +Geoffrey: We need to vote if we capitalize the M in ES Module. It’s lowercase throughout Node docs. +(will talk about capitalization outside the meeting) +Salah: If you refer to a file, it’s ESM. +Myles: We now have LKGR. The only thing that will land is Guy refactoring, doc fixes, I will open PR containing this after the meeting. Objections on opening the PR to Node this week? NO OBJECTIONS! +Will open it today or tomorrow. +Myles: Jordan, I’m willing to work on the exports proposal. It’s coupled with loaders. + + + +* Announcement blog post [#291](https://github.com/nodejs/modules/pull/291) + - 10 minute timebox + +### Next Steps (15 minutes) + +Split time across the following topics if time permits. + +* Locking down the “process” and “Buffer” globals [#235](https://github.com/nodejs/modules/issues/235) + +Myles: Any objections? +Jordan: Yes, I object (to process, not Buffer) +Guy: It’s reserving design space. I don’t think it should go into LKGR. I just want support, or support that there won’t be objections. +Myles: We should talk about this at the next meeting. + +* CommonJS import interoperability decisions [#264](https://github.com/nodejs/modules/issues/264) + - Refs: + - Moving forward with Dynamic Modules? [#252](https://github.com/nodejs/modules/issues/252) + - Import named vs default from CommonJS packages [#260](https://github.com/nodejs/modules/issues/260) +* Proposal for dual ESM/CommonJS packages [#273](https://github.com/nodejs/modules/issues/273) + - Refs: + - Exports main [#41](https://github.com/nodejs/ecmascript-modules/pull/41) + + + + diff --git a/doc/meetings/2019-03-27.md b/doc/meetings/2019-03-27.md new file mode 100644 index 0000000..ae49d78 --- /dev/null +++ b/doc/meetings/2019-03-27.md @@ -0,0 +1,100 @@ +# Node.js Foundation Modules Team Meeting 2019-03-27 + +* **Recording**: https://www.youtube.com/watch?v=rz6KhC9t_pQ +* **GitHub Issue**: https://github.com/nodejs/modules/issues/302 +* **Minutes Google Doc**: https://docs.google.com/document/d/1D4Wf27NoMoFuL02wYcYO1Iy1k0pOT0XeTa2OZAkbPQo/edit + +## Present + +- Myles Borins (@MylesBorins) +- Jordan Harband (@ljharb) +- Daniel Rosenwasser (@DanielRosenwasser) +- Rob Palmer (@robpalme) +- Michael Zasso (@targos) +- Wes Wigham (@weswigham) +- Saleh Abdel Motaal (@SMotaal) +- Geoffrey Booth +- Hassan Sani +- Ryan Day (@soldair) +- Guy Bedford +- Jeremiah Senkpiel (@Fishrock123) +- Gus Caplan (@devsnek) + +## Agenda + +Extracted from **modules-agenda** labelled issues and pull requests from the **nodejs org** prior to the meeting. + +### Review Upstream Changes + +* upstream objection to --type [#296](https://github.com/nodejs/modules/issues/296) + - changed to --entry-type (no recursive changes) + - CONCLUSION: No objections +* upstream objection to -m [#295](https://github.com/nodejs/modules/issues/295) + - Removed from current implementation + - CONCLUSION: No objections + +* Rename errType mismatch + - Jordan: We can make this change in future + +### Landing Upstream Implementation + +* Tracking upstream PR [#64](https://github.com/nodejs/ecmascript-modules/issues/64) + - Myles: Folded 40 commits to 10 commits to ease reviewing, today folded into 1 commit. Refers to input PRs. Includes co-authors: Guy, Myles, JDD, Evan, Geoffrey, MichaelZ. Has 7 approvals. + - CONCLUSION: No objections to landing on master! + * extreme whooping and applause * + * Later: Myles prepares to land the PR * + - Michael: Should we backport to Node 11? + - Myles: Let's wait one release. + - Jordan: It's good to wait. + * Myles observes lots of green tick checks * + * All 4 people in the room press the merge button: PR is now landed! * + * Even greater whoops of joy * + + +### Next Steps + +* What's next for the team? + - Moratorium: Propose we drop it. Myles pledges not to allow big changes without consulting the modules team. + - Jordan: Maybe have a policy for major/minor changes? + - Myles: The team has had friction. What is our role/expectations? + - Gus: It would be more efficient if we just moved to core and iterate there. Should be working more closely with the collaborators there. + - Jordan: That's just the github repo choice. It would be ideal for the modules team had to approve of changes. + - Myles: We need to be chartered to do that. We are not chartered. + - Jordan: Can we pretend? We represent various constituents. That usually represents a larger group, so the feedback is useful. It would be a loss if this group did not continue to contribute those perspectives, regardless of charter. Last meeting suggested we need adjustments. We should try that before altering the venue. + - Geoffrey: This group adds value debating design decisions. Throwing PRs at core might not consider our design parameters. How can we limit the scope of PRs allowed in? + - Myles: Looking in modules repo today. Our governance says "consensus seeking process, merging PRs here, not NodeJS core". Not bug fixes. Not error message changes. I suggest removing the NodeJS bit, pare down the exemptions, what we work on. The request could be a moratorium to not stray from the phases doc. Could be more explicit: please don't land things relating to the needs-consensus things in our doc. Members with commit bits can put red crosses on PRs that violate this. + - Jeremiah: Node core can't own a subsystem in that way. WGs can do this kinda. People will respect people that come with PRs and talk about things. + - Salah: I second Geoffrey's idea. Having the group so far has allowed insight into relevant activities in core. This group brought them to our attention. That distillation is good to keep. Concerned about not having this. + - Myles: Core works by triaging and then adding the teams that are experts. Highly unlikely for something to go through on ESM that would not @ modules. Controversial things would put a clear red x for items we need to discuss. This applies to things that are not working groups, e.g. inspector that has 5 members. The difference is that things can be landed without a meeting. + - Saleh: What do we need to do to be chartered? + - Myles: To be blunt, I need to be convinced and am not currently convinced. Apologies. Maybe re-examining our governance. One idea is "rough consensus" instead of "full consensus". All groups have this problem. Could even take it to core or other venues if the model works. I want to be so proud of our work that we offer it to other groups. I can have a go modifying our governance document. Modules is a hard domain - lots of things trigger relitigation of other areas. Maybe we are stewards of the roadmap. + - Jordan: Low level C++ implementation is the Node core group. But user-observable things is the modules group should steward. + - Myles: I am comfortable with this. Would anyone like to update the governance doc? (updating to reflect non-moratorium, change of scope). I will do it! And I feel positive about it. + - Saleh: I will help iterate with you. + + + +* Locking down the "process" and "Buffer" globals [#235](https://github.com/nodejs/modules/issues/235) + - PR on Node core to deprecate these inside ESM only. Made them getters first. Introduced it at SES. Mark Miller verified it. Will update the PR over the next few months. + - Myles: Canary in the Gold Mine looks ok with this. Breakages are the same as Node 12. We have made major change before that turned out to be mistakes. They can be reverted in a patch if the change is found to be destructive. 100+ modules say this change is safe. So we can only find out the impact by releasing. + - Jordan: I think Buffer deprecation is fine. Very concerned about "process". It's fine to remove the unsafe parts, but it's hard to do them individually, so we remove it fully. process is used for environment detection. I once broke Yahoo.com. So worried about this. Keeping process out of ESM for now, would be fine. It would be wiser to keep process global, but eliminate the dangerous parts. The PR now is fine. Please can everyone consider the impacts. + - Guy: The getter PR is already merged to master. Adding a global is a very different thing. Websites have strong backwards compatibility concerns. No code is depending on this change. The new process can just have bare minimum. Or import.meta.platform. We accept that some things will hit people, like the package.json module field - this might be one of them. + - Salah: This group needs syntax detection of node without relying on process. typeof(process). ESM code relying on this must be behind a flag, if it exists today. + +* Moving forward with Dynamic Modules? [#252](https://github.com/nodejs/modules/issues/252) + + +* Exports main [#41](https://github.com/nodejs/ecmascript-modules/pull/41) + + +* esm: scoped --type, cpp refactoring (This is package vs entry) [#57](https://github.com/nodejs/ecmascript-modules/pull/57) + + +* Flags functionality options [#300](https://github.com/nodejs/modules/issues/300) + + +* Proposal for dual ESM/CommonJS packages [#273](https://github.com/nodejs/modules/issues/273) + + +* Warn about `--type` with shebang [#37](https://github.com/nodejs/ecmascript-modules/pull/37) + diff --git a/doc/meetings/2019-04-10.md b/doc/meetings/2019-04-10.md new file mode 100644 index 0000000..6c55ed8 --- /dev/null +++ b/doc/meetings/2019-04-10.md @@ -0,0 +1,90 @@ +# Node.js Foundation Modules Team Meeting 2019-04-10 + +* **Recording**: https://www.youtube.com/watch?v=eCAcQ-r0CfY +* **GitHub Issue**: https://github.com/nodejs/modules/issues/310 +* **Minutes Google Doc**: https://docs.google.com/document/d/1t9sYWYkDEAZKMhj-OGNlCeVyPjxOVYpscyOEury3Vhw/edit + +## Present + +- Geoffrey Booth (@GeoffreyBooth) +- Rob Palmer (@robpalme) +- Jordan Harband (@ljharb) +- Saleh Abdel Motaal +- Wes Wigham +- Guy Bedford +- Gus Caplan (@devsnek) +- Jeremiah Senkpiel +- Jan Krems + + +## Agenda + +* Locking down the "process" and "Buffer" globals [#235](https://github.com/nodejs/modules/issues/235) +- GB: Only proposing to deprecate in ESM. Want to find out if we can keep the door open to this possibility. Once exposed, can never take back. Exposes root level security access. Do we want every package to have access to this? We have already made them getters on Node 12. Matteo has blocked this feature - so I need to work with him next. +- JH: Agree - once unflagged they must remain, and we can remove them up to that point. The roadmap to a new security model is questionable - a long discussion. Eliminating `Buffer` is fine. `Process` is problematic. Maybe just strip some properties? +- SAM: Platform detection / removing `process`. Worry about copy-paste. A frozen proxy could be sanitised in module scope. +- GB: Security/usability are important and need fleshing out another time. Saleh, this work is a path to the restricted process. Common detection cases so need to workout those workflows. Jordan provoked me to talk to Deno. Response was "global access is not a good pattern". React uses require() to allow dev vs production switching. Build tools can deal with this. +- JH: Deno already has a global. Ryan think many things are bad - there is not broad agreement on this list +- Jan: You could import `process`. +- JH: `typeof process` matters +- SAM: You may not know you are in a module. Does not satisfy `import` dynamically failing. +- WW: Removing unsafe globals is good. However, compatibility. ESM introduction does not make this ok. We must present the same APIs. It's unlike `__dirname` which are module-related. process and Buffer are not module-related. Removing them is not ok. +GB: We put process/Buffer into a scope wrapper for performance optimizations. When you access it, we do a stack-based check to see if you are in a module. It is not a good pattern. We could not do this for `import "fs"` or property access. Global access is syntactic. +JS: Is the global object accessible via `global`? + + +* Flags functionality options [#300](https://github.com/nodejs/modules/issues/300) + - refs: + - Replace --entry-type with --input-type, plus --input-type=auto [#68](https://github.com/nodejs/ecmascript-modules/pull/68) + - GB: I have heard objections. For string input, the user may not know the type. coffee/babel do not use eval. As we know, the syntax is ambiguous. Unambiguous case with import/export is handleable. For the ambiguous case, maybe look for CJS globals? Consensus was that this was too messy. So we redefine the goal, and ambiguous files fall back to CJS. Would like it to be added to `VM.createScript` - the one that Babel uses so that tooling becomes consistent. + - JH: It's fine to ask the VM "is this string unambiguous?". Some of TC39 from Browsers objected to auto-detection is that mid-way through typing a file, the mode would switch and the users would be surprised. "auto detection is a footgun" + - GB: This behaviour is different because you must opt-in to the behaviour. + - GC: So this is for something where you don't know the type so the smartest you can be is detecting import/export means you are NOT sure and have NOT solved the problem. + - GB: At least it mostly works for the entry file. + - GC: I don't imagine I will be in this situation. + - JS: Want to avoid any form of auto-detection. Maybe have it as a function to ensure consistency? Though I think anyone using this is laying a landmine. + - GB: The use-case is REPL-like. Users don't want to specify. + - JS: The REPL is special. + - GB: I will refactor the PR into a function on `module` that tools can use. It's a starting point. Any objections? + - GC: I am not a fan. But keep it behind a flag and I'm ok with this. + - GB: e.g. --detect-input-type + + - Replace --entry-type with --input-type [#66](https://github.com/nodejs/ecmascript-modules/pull/66) + - GB: Current implementation is a footgun IMO. People expected it to be package-scoped, not just apply to the initial file. Initially we proposed `--package-type` but there were objections. Then people suggested eliminating the flag. So maybe we only have it for eval/STDIN not for file, i.e. scale it back. + - JH: Strictly critical case is string input - otherwise parse goal is uncontrollable. Standalone file case can be handled by `.mjs` vs `.cjs`. Remaining case is extension-less file. + - GB: Symlinks can handle that. It's a small price to pay. Good to ditch the footgun. + - JH: We just need to explain ourselves. Seems reasonable to tell people to add a `package.json` with a field. We should be clear: extensionless file will be annoying. + - SAM: npm installation gets symlinked into .bin + - GB: This only happens if you already have a `package.json`. Users of --preserve-symlinks get burnt. You can't run npm with this flag. Let's go ahead and see if users speak up. It's a rare use-case. Consensus to merge the PR? (does not include `--input-type=auto`) + - CONCLUSION: We have consensus to merge! + +* Dual Mode Packages + - refs: + - Proposal for single-mode packages with optional fallbacks for older versions of node [#299](https://github.com/nodejs/modules/issues/299) + - WW: Concept: Some people said importing must be async, preventing substitutability - you can't `require(ESM)`. We have used this as an assumption for a while. I have been unhappy. Gus did work syncifying arbitrary promises. Spec doesn't force loaders to be async. Today, Node implementation is async. Syncifying permits deadlock. But this doesn't matter for loaders in Node. So we have a choice: sync vs async. So we can go back on the long-held assumption. Enabling substitutability. Eliminates multiple versions of the same module. So we can side-step this problem. + - GB: These two proposals aren't really counter-proposals. Wes is encouraging require(ESM). That should be a standalone PR. The other part is about reusing the "main" fields, which I am not a fan of. + - WW: Once you can require(ESM), things become simple. + - GB: People like me want to publish things usable in old versions of Node. Overloading main doesn't work with source vs dist packages. + - JH: Source vs dist is already a pain point. We should solve it for both CJS and ESM - or not solve it at all. Want some way to do dual mode packages. I like extension resolution. source vs dist in CJS is solved via top-level modules that resolve on disk. I want to avoid two instances of the same module in memory for the same module on disk. + - WW: Agree. Exports is a good thing to apply to CJS and ESM. A prettier solution that is limited to ESM is limiting. Old versions of Node already have a single entrypoint. Offering this for new versions of Node is fine. + - SAM: Syncifying. If we can do this, we really need to explore it. Can we work towards a model and see if this works? + - WW: This exists. + - JS: Dual-mode packages is invalidated if you have substitutability (by eliminating sync vs async). Top-level await may arrive. + - WW: This doesn't matter. + - GB: Let's have a dedicated dual-mode meeting after Wes gets the PR up. + - RP: But TLA does matter. + - WW: `require("ESM")` will return the namespace object immediately, even if the module's evaluation has not yet completed and therefore the bindings may not be populated at the point it returns. + + +**END OF MEETING** + +**NOT DISCUSSED** + +- esm: scoped --type, cpp refactoring [#57](https://github.com/nodejs/ecmascript-modules/pull/57) +- Warn about `--type` with shebang [#37](https://github.com/nodejs/ecmascript-modules/pull/37) +- Proposal for dual ESM/CommonJS packages [#273](https://github.com/nodejs/modules/issues/273) + +* Proposal: Support loading package by own "name" [#306](https://github.com/nodejs/modules/issues/306) +* Exports main [#41](https://github.com/nodejs/ecmascript-modules/pull/41) +* Moving forward with Dynamic Modules? [#252](https://github.com/nodejs/modules/issues/252) + diff --git a/doc/meetings/2019-04-24.md b/doc/meetings/2019-04-24.md new file mode 100644 index 0000000..9ca1a99 --- /dev/null +++ b/doc/meetings/2019-04-24.md @@ -0,0 +1,74 @@ +# Node.js Foundation Modules Team Meeting 2019-04-24 + +* **Recording**: N/A due to technical problems 😭 +* **GitHub Issue**: https://github.com/nodejs/modules/issues/317 +* **Minutes Google Doc**: https://docs.google.com/document/d/12hKVk4C9PNkHHZoOQA6I2Y_oUlAPteDTnEIyyp9Odqc/edit + +## Present + +- Myles Borins (@MylesBorins) +- Wesley Wigham (@weswigham) +- Daniel Rosenwasser (@DanielRosenwasser) +- Jeremiah Senkpiel (@Fishrock123) +- Rob Palmer (@robpalme) +- Saleh Abdel Motaal (@SMotaal) +- Gus Caplan (@devsnek) +- Michael Zasso (@targos) +- Guy Bedford (@guybedford) +- Jan Krems (@jkrems) +- Jordan Harband (@ljharb) +- Hassan Sani (@inidaname) +- John-David (@jdalton) +- Geoffrey Booth (@GeoffreyBooth) + +## Agenda + +Extracted from **modules-agenda** labelled issues and pull requests from the **nodejs org** prior to the meeting. + +### Approve PR (5 Minutes) + +* Governance: "Pull Requests" post Phase 2 upstream [#311](https://github.com/nodejs/modules/pull/311) + - CONCLUSION: NO OBJECTIONS + +### Discussion (50 minutes) + +* Review Phase 3 [#316](https://github.com/nodejs/modules/issues/316) + - 10 minute timebox + - MB: Want to identify objections to break out into separate issues. + - Loader solution: no objections. Guy, Saleh, Gus volunteer as co-champions. Maybe form a group? + - Dual-mode packages: to be (two approaches) or not to be. Geoffrey, Jordan as co-champions. + - createRequire: + - MB: discovered `makeRequireFunction(".")`. Maybe make dot the default. + - JH: Maybe this is not necessary if we get dual-mode packages + - SM: Needs to be detectable in non-Node, e.g. prefer global over `import.meta` + - JS: If no dual-mode, this needs to be very simple. Current is not good enough. + - Champion: Myles +- import map equivalent + - JH: This is important. Want to have it for both CJS and ESM. Depends on outcome of dual-mode packages. + - WW: No reason for this to be format-specific. Agree with Jordan. + - GB: This is itself a dual-mode proposal, due to lack of full backporting. + - WW: Dual-mode to allow packages to work in old Node is not "dual mode" that we are talking about here. + - Champion: Jan Krems + - Participants: Saleh +- Auto-detection of entrypoints + - Champion: Geoffrey (if we even do this) + - Participants: Saleh + +* To Dual Mode or Not to Dual Mode + - 30 minute timebox + - Refs: + - Exports main [#41](https://github.com/nodejs/ecmascript-modules/pull/41) + - Proposal for dual ESM/CommonJS packages [#273](https://github.com/nodejs/modules/issues/273) + - Proposal for single-mode packages with optional fallbacks for older versions of node [#299](https://github.com/nodejs/modules/issues/299) + +- Jordan: Gist is that starting to use ESM as soon as it's unflagged is that it's a breaking change for all consumers and that they'll have to swap from `require` to `import`. That's pretty unnecessary and it can cause a fork in the ecosystem. + - At one time there was a hope that you could just `require` an ESM, there was an idea about extension resolution (cjs/mjs), there were lots of things here. + - Two singletons in graph. + - First thing we need to decide on is whether switching to ESM is the "right thing" or if we do want to improve the incremental migration story. +- Geoffrey: Talking about bare specifiers (when require/import "package") + - One question to allow bare specifier to be mapped to another + +* Proposal: Support loading package by own "name" [#306](https://github.com/nodejs/modules/issues/306) + - 5 minute timebox +* Moving forward with Dynamic Modules? [#252](https://github.com/nodejs/modules/issues/252) + - 5 minute timebox diff --git a/doc/meetings/2019-05-08.md b/doc/meetings/2019-05-08.md new file mode 100644 index 0000000..14628a6 --- /dev/null +++ b/doc/meetings/2019-05-08.md @@ -0,0 +1,83 @@ +# Node.js Foundation Modules Team Meeting 2019-05-08 + +* **Recording**: https://www.youtube.com/watch?v=NCcIRQN7PHs +* **GitHub Issue**: https://github.com/nodejs/modules/issues/326 +* **Minutes Google Doc**: https://docs.google.com/document/d/18ZUWtRYq-H5JwcFfGdUFsHi5Q5QB5mCbCngAM_N-MUE/edit + +## Present + +- Geoffrey Booth (@GeoffreyBooth) +- Rob Palmer (@robpalme) +- Saleh Abdel Motaal (@SMotaal) +- Jordan Harband (@ljharb) +- Wes Wigham (@weswigham) +- Guy Bedford (@guybedford) +- John-David Dalton (@jdalton) +- Jan Krems (@jkrems) + +## Agenda + +Extracted from **modules-agenda** labelled issues and pull requests from the **nodejs org** prior to the meeting. + +### `createRequire` + +- `createRequire` is now available in Node 12 unflagged. + +### nodejs/modules + +* Proposal: CommonJS named exports defined in package.json [#324](https://github.com/nodejs/modules/issues/324) + - Orthogonal to dual-packages + - `import { something } from "underscore"` + - *out-of-order* and *dynamic modules* solutions are not making much progress - this is an alternative that would be replaced if we get either of the previous approaches - it's better than nothing + - Jordan: If we have dual-mode packages, it becomes trivial for them to roll this themselves without needing package.json config + - Saleh: If we can be diligent and add config to package.json, you could similarly wrap a module. + - Jordan: Package.json config and ESM wrappers are equivalent in terms of being static and therefore impacting circularity. + - Wes: There are non-main entrypoints to consider, so it's odd to be asymmetric. + - Geoffrey: Do we want to pursue this? + - Guy: Solutions can be pursued in parallel - not confined to one path. Shooting down outright is not beneficial. + - Wes: I don't think this is worth pursuing. + - Geoffrey: We want to improve the general migration story. + - CONCLUSION: Can be pursued in parallel. Contact Geoffrey if you're interested. + +- Saleh: Can we synchronously await loading an ESM into a CommonJS module? + - Wes: `std/esm` effectively does this. Guy Bedford's SystemJS did something like this. + - Guy: All my projects were async and did not flatten to synchronous. + - Saleh: Has the Regenerator approach been explored? + - Wes: The C++ resolver for ESM is separate to the JS CJS resolver. Maybe we could migrate the CJS one to use C++? + - Guy: Would this still allow use cases like std/esm + - CONCLUSION: Deferred for now. + +* Proposal for dual ESM/CommonJS packages [#273](https://github.com/nodejs/modules/issues/273) + - Specifies the new pkg.json field to point to the ESM entrypoint + - Geoffrey: This does not conflict with Wes' require(ESM) proposal. Are there objections? + - Jordan: It's weird that the same specifier can resolve to separate things. (idempotency) If we have extension searching, we don't need the field. + - Geoffrey: But extension searching is not moving forwards now. We can change our mind later. + - Jordan: Want a holistic approach - not piecemeal. If we're going to solve this only for non-deep entrypoints, we shouldn't solve it at all. + - Jan: In the past, we disagreed about whether extension searching would solve this. Having sibling files with .js and .mjs is a weird hybrid. More junk files. Would prefer not to go in this direction. + - Saleh: Extension searching should be convention-based. + - Geoffrey: People want everything. It's hard to get massive PRs through. Trying to be iterative. Could we start a branch on ES repo and put pieces in and build it early. + - Geoffrey: Asking for consensus: The proposal that adds a field to pkg.json (e.g. "exports") for the ESM entrypoint - can we merge a PR on the basis that it can go away and the branch won't be upstreamed until consensus is reached? + - Wes & Jordan state they would veto + - Geoffrey: We had meetings on extension searching. More than half opposed extension searching by default. + - Jordan: We made it available behind a flag to get feedback. I think it will land. I think it's a much better way to do this. + - Geoffrey: I don't think extension searching will land. + - Jan: Can we put the pkg.json field behind a flag as we have done for extension searching? And how do we quantify the feedback - because users satisfied with the default will not speak up. + - Jordan: We should ship pkg.json field behind a flag - it's a good way to collect feedback. + - CONCLUSION: We implement behind a flag + + +* Feedback on extension resolution [#323](https://github.com/nodejs/modules/issues/323) +* Proposal: Support loading package by own "name" [#306](https://github.com/nodejs/modules/issues/306) +* Proposal for single-mode packages with optional fallbacks for older versions of node [#299](https://github.com/nodejs/modules/issues/299) +* Moving forward with Dynamic Modules? [#252](https://github.com/nodejs/modules/issues/252) + +### nodejs/ecmascript-modules + +* esm: experimental wasm modules [#46](https://github.com/nodejs/ecmascript-modules/pull/46) + - Jordan: Why can't WASM import live bindings? + - Guy: WASM is a static format. Updating a binding's type is not compatible with this. Exports may need to go through a translation too. + +* esm: scoped --type, cpp refactoring [#57](https://github.com/nodejs/ecmascript-modules/pull/57) +* Exports main [#41](https://github.com/nodejs/ecmascript-modules/pull/41) + + diff --git a/doc/meetings/2019-05-22.md b/doc/meetings/2019-05-22.md new file mode 100644 index 0000000..88ca047 --- /dev/null +++ b/doc/meetings/2019-05-22.md @@ -0,0 +1,62 @@ +# Node.js Foundation Modules Team Meeting 2019-05-22 + +* **Recording**: https://www.youtube.com/watch?v=nOCaOUWIlaQ +* **GitHub Issue**: https://github.com/nodejs/modules/issues/332 +* **Minutes Google Doc**: https://docs.google.com/document/d/1jYEGcbCsjoOU3GrnhkidurMtzzm_rCS3B7iBMCe3XxA/edit + +## Present + +- Wesley Wigham (@weswigham) +- Michael Zasso (@targos) +- Myles Borins (@MylesBorins) +- Geoffrey Booth (@GeoffreyBooth) +- Daniel Rosenwasser (@DanielRosenwasser) +- Guy Bedford (@guybedford) +- Saleh Abdel Motaal (@SMotaal) +- Ryan Day (@soldair) +- Jordan Harband (@ljharb) + +## Agenda + +Extracted from **modules-agenda** labelled issues and pull requests from the **nodejs org** prior to the meeting. + +### nodejs/modules + +* Phase 3 update [#331](https://github.com/nodejs/modules/pull/331) + - needs to be iterated based on the categories + - geoff will update: + - top-level phase 3 categories of In Progress, Done, Abandoned (nothing in Abandoned yet) + - for each item, add a Status field e.g. “Stalled because of lack of consensus” or “Stalled because of upstream TC39 concerns” or “Implementation in progress” etc. + +* Proposal: Support loading package by own "name" [#306](https://github.com/nodejs/modules/issues/306) +The name a package could use to refer to itself as an import or require. +@@name@@:P +~name +Must be semver minor if we want it to get adoption in the next 3 years +Should not be a symbol that couldn’t possibly be on the file system +Import maps today would have an entry per “scope” for root. This would bloat the import map quite a bit. +Could be specced so roots dont need to be created per scope. +No rush as of yet to seek consensus + +* An update about WASM modules and Env [29:50] +Cant import emscripten compiled wasm directly because it build wasm that expects symbols passed in from js (Env) + Something might be missing in the modules system bigger than just node to pass in global like vars to modules that may or may not be shared as the loader wishes. + realms proposal was designed to expose compartmentalized globals to specific modules. +WASI will likely be useful + +* Moving forward with Dynamic Modules? [#252](https://github.com/nodejs/modules/issues/252) + + +## Invited + +* Modules team: @nodejs/modules + +## Notes + +The agenda comes from issues labelled with `modules-agenda` across **all of the repositories in the nodejs org**. Please label any additional issues that should be on the agenda before the meeting starts. + +## Joining the meeting + +* link for participants: https://zoom.us/j/656987750 +* For those who just want to watch: https://www.youtube.com/c/nodejs+foundation/live + diff --git a/doc/meetings/2019-06-19.md b/doc/meetings/2019-06-19.md new file mode 100644 index 0000000..099e104 --- /dev/null +++ b/doc/meetings/2019-06-19.md @@ -0,0 +1,49 @@ +# Node.js Foundation Modules Team Meeting 2019-05-22 + +* **Recording**: https://www.youtube.com/watch?v=I45_tXRxoLQ +* **GitHub Issue**: https://github.com/nodejs/modules/issues/342 +* **Minutes Google Doc**: https://docs.google.com/document/d/1xFezS-2fWtASYg_QJn-koNfrHBD0-LRG-EQXKntQuVY/edit + +## Present + +- Wesley Wigham (@weswigham) +- Jordan Harband (@ljharb) +- Saleh Abdel Motaal (@SMotaal) +- Myles Borins (@MylesBorins) +- Guy Bedford (@guybedford) + +## Agenda + +Extracted from **modules-agenda** labelled issues and pull requests from the **nodejs org** prior to the meeting. + +### nodejs/modules + +* Proposal: --default-type [#335](https://github.com/nodejs/modules/issues/335) + +Geoff explains the proposal + +Jordan: Sounds similar to the --type flag we decided against previously. Why now are we going to reevaluate that decision? + +Geoffrey: A couple of users have been asking for this + +Myles: As a clarifying question, all node_modules would get this type applied right? + +Geoffrey: That is the naive implementation, yes. + +Myles: Then I would object to that, as it seems likely to be ecosystem breaking. The flag seems inherently broken in that it would require all modules in the tree to already assume that default type. If we want to do this we have to ensure all modules have type commonjs in their tree otherwise we are setting people up to break stuff. + +Guy: We’re trying to look at approaches for removing “type”: “module” in future too. Do we want this? +Jordan: I see no problem with this. + +Myles: I don’t see this as a huge burden. + +Guy: If we don’t see this as a problem, then we don’t need to worry about it. We could also possibly consider adding some guidance here to encourage users to set “type”. + + +Myles: We could be more explicit and even encourage package managers to set this on init. I think we can bring this offline and someone can create a PR for this in the docs, and if someone wanted a timeline for the switch we could note the doc deprecation and warning deprecation. As well as warnings that only get included if you add a special flag. + +Geoffrey: It is broader than just one paragraph, it is also about how you define a module or esm module. That we can maybe discuss in an issue or something as I’m sure people will have opinions. + +* Flags functionality options [#300](https://github.com/nodejs/modules/issues/300) + +* Package Exports [#341](https://github.com/nodejs/modules/issues/341) diff --git a/doc/meetings/2019-07-03.md b/doc/meetings/2019-07-03.md new file mode 100644 index 0000000..6d2a40a --- /dev/null +++ b/doc/meetings/2019-07-03.md @@ -0,0 +1,38 @@ +# Node.js Foundation Modules Team Meeting 2019-07-03 + +## Links + +* **Recording**: https://www.youtube.com/watch?v=2vSohstr-5M +* **GitHub Issue**: https://github.com/nodejs/modules/issues/346 +* **Minutes Google Doc**: https://docs.google.com/document/d/11KCURwlDyTGebqAkOaImjISrauuwXS1DXpPXKpmp3GM/edit + +## Present + +* Modules team: @nodejs/modules +* Jan Krems (@jkrems) +* Myles Borins (@Mylesborins) +* Wesley Wigham (@weswigham) +* Sendil Kumar (@sendil +kumarn) +* Rob Palmer (@robpalme) +* Saleh Abdel Motaal (@smotaal) +* Geoffrey Booth (@GeoffreyBooth) +* John-David Dalton (@jdalton) +* Jordan Harband (@ljharb) + + +## Agenda + +## Announcements + +*Extracted from **modules-agenda** labelled issues and pull requests from the **nodejs org** prior to the meeting. + +### nodejs/modules + +* Package Exports [#341](https://github.com/nodejs/modules/issues/341) + +PR is open https://github.com/nodejs/ecmascript-modules/pull/72 + + +* All Modules go into a namespace [nodejs#21551](https://github.com/nodejs/node/pull/21551) + - Refs: https://github.com/nodejs/node/pull/28466 diff --git a/doc/meetings/2019-07-17.md b/doc/meetings/2019-07-17.md new file mode 100644 index 0000000..5af7002 --- /dev/null +++ b/doc/meetings/2019-07-17.md @@ -0,0 +1,80 @@ +# Node.js Foundation Modules Team Meeting 2019-07-17 + +## Links + +* **Recording**: https://www.youtube.com/watch?v=ZByiwEcrSEo +* **GitHub Issue**: https://github.com/nodejs/modules/issues/349 +* **Minutes Google Doc**: https://docs.google.com/document/d/1vjst5ibYykhzdEcFCMhUHSLDY3N84gKr8bEr-JXpVfQ + +## Present + +* Myles Borins (@mylesborins) +* Jeremiah Senkpiel (@Fishrock123) +* Jan Krems (@jkrems) +* Alex Aubuchon (@A-lxe) +* Michael Dawson (@mhdawson) +* Gus Caplan (@devsnek) +* Wesley Wigham (@weswigham) +* Guy Bedford (@guybedford) +* Bryan Clark (@clarkbw) +* Michael Zasso (@targos) +* Alex Mullans (@infin8x) +* Saleh Abdel Motaal (@smotaal) +* Geoffrey Booth (@GeoffreyBooth) +* Jordan Harband (@ljharb) + +## Agenda + +## Announcements + +*Extracted from **modules-agenda** labelled issues and pull requests from the **nodejs org** prior to the meeting. + +### nodejs/modules + +* Namespaces + +BC: Tried to summarize the issue in the GitHub issue for the meeting. For the GitHub package registry, they want to be good citizens and co-exist with npm. There is a problem with namespacing - figuring out what the best resolution is, for GitHub to be effective for developer ergonomics and in customer aspects, how to fit in with the ecosystems and the recommended route. Using URLs to more completely describe where packages come from (npm registry uses a single global namespace eg `react` without any domain / destination). Trying to figure out how to integrate with that. Posted earlier on the TSC meeting - so some of the discussion is also happening there. + +MB: Two different bits here. Proposal for TC39 looking to add builtin JavaScript modules into the JS VM. UUID is one being explored right now. std: and js: have been proposed as these approaches. Then also questions about if there should be namespaces for different types of modules - web:, etc. Do DOM-specific features belong in a JS namespace. That is one place things are being explored. The Web Assembly working group is also working with wasi: namespaces. Both of these standards organizations are looking at kind of like a URI scheme for how we do the namespaces. nodejs: is what we’re talking about in Node.js, or whether it should be @nodejs/. Personally have a bias towards schemes for consistency across environments. From GitHub’s perspective, would a registry itself be a namespace? Some concern from the Node TSC about making registries part of the specifier versus being something that exists in metadata such as the package.json. + +JK: Entropic are in early stages and are also trying to figure out a way to distinguish different origins for packages as an additional namespace. For every registry to get its own protocol can be problematic. Intrigued in general by the idea of npm: for npm registry. + +MD: Namespacing in the language seems different from namespacing for the registry. + +BF: You can take some experience from what happened with mobile. Deep-linking into apps was done with protocols, which is now frowned upon. What they moved on to is something that is generally called universal links, where an app can occupy a scope within a protocol. For mobile operating systems, that is generally https. So they have a similar problem where they have to delegate to the proper app within these scopes. So maybe we should look into something like that and why they now frown upon things being third party to the operating system itself. + +JS: Should also be think about where you are pulling the file from. Because otherwise we are effectively thinking about deconflicting node_modules between ‘registry’ specifiers. + +JK: Prior work with Go and originally Java ecosystem com.github.* gives a URL/DNS bases ownership concept. Seems to be scaling well in that ecosystem. + +JH: Are we talking about loading modules from anywhere except on disk? Assuming package manager handles installation, things like npm can already install from different registries per scope. It would be odd to not try and get package installers to support having installations support installation from all the different kinds/locations of packages in package.json . It would not be good for node to become involved in controlling where packages come from. + +BC: npm always assumes that a GitHub URL is doing certain things against git. This is why we liked the idea of an npm: protocol. Also fear extra protocols as well. Tying the package back to a location we look at this in the long run as an IPFS style system helping out here. Because there may be a lot more locations in involved in packages, and we can mitigate this by having metadata that more concretely describes what the package is. When we hold the source we think we can provide a lot more information about how the package was built. Build receipts can provide information about what is in the package. Including handling the same package from a different source location. + +If whatever we are doing requires client stages, including through npm, then that will be difficult for our customers to ensure they are on the latest npm client if we do have changes. Want to have a long-term and then possibly short-term approach. + +* Loaders [#351](https://github.com/nodejs/modules/issues/351) + +AA: It would be nice to have a way of more generally providing hooks into dependency loading. At a more general level than either the CJS or ES module loading systems, more at the resource loading level. Opened an issue on the modules working group about this. There has been a lot of prior thought into loader hooks, like the --loader flag, per-dependency loaders, composing multiple loaders, as well as a lot of other things. I am hoping to kick off discussion again along that front and hopefully get things to a place where it is actionable again in terms of creating prototypes. + +BF: If you’re mostly interested in CommonJS, that hasn’t been discussed at great length in this group. There were proof of concepts created last year about how to use atomics to properly call back into the main thread. I would be curious as to what you are wanting to do. A lot of the latest loader designs we’ve talked about this year have been based upon old designs from last year, and I was wondering if you have looked at those designs or if you think you need more APIs than just the loader hooks themselves. I did see you linked to also a generalized resource API, so general thoughts along that is what I am asking. + +AA: Right now I am working in CommonJS because that is where a lot of the ecosystem is. From what I’ve seen of the current --loader API it doesn’t seem like there is the ability to easily compose loaders. If I create a loader and have it as the last loader on a project, how do I know I’m not overriding the functionality of previous loaders on some kind of stack. That is what was most noticeable to me. On the CommonJS side, hopefully we would be able to use that on the Node.js side to extend that to both module systems. + +JK: To amend observations on that, I think in the past year or so… For example, when Entropic came out, they said they would not be loading from node_modules because users shouldn’t be doing that. Now it’s obviously not here, but Tink is not too far away. Plug and Play is at least not to far off. If we can provide loader hooks that can do transforms and loading from archives and that sort of stuff, then if we do not support CommonJS then we get to the weird place where if you want to write your code in CoffeeScript or TypeScript and transform your code on the fly, as people will want to do, then we need to make sure that if people have some stuff that is CommonJS that you can still use require.extensions for that piece. Whereas if CommonJS would also use the same loader infrastructure then we could actually phase out some users of require.extensions and people could use one loader hook to solve it. + +SM: You’re really applying this to packages in your immediate control, but if we are going to try and break that barrier and say that this is a more universal thing, then we may have a module author saying they want their module loaded in a certain way. It might be too unpredictable to let every package decide how it wants to be loaded. + +GC: One of the things that is somewhat taken for granted in ESM is the ability to have module resources that are not necessarily local disk resources - like you mentioned loading things out of an archive. And these things really do not play well with CommonJS, not that I am saying we shouldn’t try to pursue these goals. Even the keys in the require cache. There was a PR recently that changed the specifier that the code runs under in the attempts to improve the stack trace and this broke packages in the ecosystem that assumed the structure. I’m just concerned that trying to move forward with this we will find these sorts of things that can’t really be fixed in CommonJS. If a call comes from ESM it makes complete sense to give it some kind of RAM location, but if it comes from CommonJS it might completely break everything. That might break everything. + +RD: That is a great segway in the kinds of problems we’ve had here for a while. We needed a way to load from disk that could be done at a platform level. To do that we had to know so much about CommonJS - we had to reimplement CommonJS so that it could work in memory. We created this bundle format with all of the source text and set it up. We have all of the files on disk, but none of the source text actually comes from disk, because hitting the file system cache on a container just takes too long. + +JK: This would not be the default behaviour, and 100% coverage of CommonJS is not possible. But as a feature people can opt-into so that these tools is possible. + +RD: The ability I wanted was just to provide source text without any knowledge of how CommonJS was supposed to be loaded. I’m much more interested in a low-level API I can provide source text, and have that decoupled from how modules are resolved. And that’s the API I see CJS and ESM can share. + +BF: We have multiple high-level runtimes. People are monkey-patching the file system API etc. to do stuff like this. They are not going to stop doing that unless we provide an alternative. If we want to actually lock down and provide better guarantees around module loading, we have to provide them an API. It’s not really an option. If you all think it’s fine just with a resource loading API, I would try to pass that. + +* Package Exports [#341](https://github.com/nodejs/modules/issues/341) + + diff --git a/doc/meetings/2019-07-31.md b/doc/meetings/2019-07-31.md new file mode 100644 index 0000000..5fb4ad4 --- /dev/null +++ b/doc/meetings/2019-07-31.md @@ -0,0 +1,102 @@ +# Node.js Foundation Modules Team Meeting 2019-07-31 + +## Links + +* **Recording**: Unavailable due to no account access. +* **GitHub Issue**: https://github.com/nodejs/modules/issues/362 +* **Minutes Google Doc**: https://docs.google.com/document/d/1nGdMbuJ4hyNJ8QABUwaOiPg8MFfygeBythMc4WL-1mY + +## Present + +* Modules team: @nodejs/modules +* Gus Caplan (@devsnek) +* Rob Palmer (@robpalme) +* Guy Bedford (@guybedford) +* Jordan Harband (@ljharb) +* Bradley Farias (@bmeck) +* Wesley Wigham (@weswigham) +* Hassan Sani (@inidaname) +* Geoffrey Booth (@geoffreybooth) +* John-David Dalton (@jdalton) + +* Observers +* Maël Nison (@arcanis) +* Jamie Kyle (@jamiebuilds) +* Devon Govett (@devongovett) +* Alex Aubuchon (@a-lxe) +* Darcy Clarke (@darcyclarke) + + +## Agenda + +## Announcements + +*Extracted from **modules-agenda** labelled issues and pull requests from the **nodejs org** prior to the meeting. + +* No YouTube LiveStream :-( - we couldn't figure out how to start it + +* New joiner: Mael - interested in the loader hooks +* New joiner: Devon - spoken to Guy about exports/entries +* New joiner: Darcy Clarke - joined npm to handle OSS/community ("woo" from JDD & ❤️ from Darcy) + +### nodejs/node + +* esm: support loading data: URLs [#28614](https://github.com/nodejs/node/pull/28614) +BF: ESM has a url-based module map now - it's historical. Found use for data-urls in the past. PR ready to load them via import. PR is incomplete - due to lack of MIME parser. Only handles certain MIMEs. Allows in-memory resources representable via data urls. Does not support CJS or C++ modules. Does support JSON, ESM, WASM. Concern: is it useful? Concern: what is a data url? +Having content-addressable things as a URL. String has the entire resource. Attach a MIME to express format. It's an absolute URL - not hierarchical. Previously used on the web to embed content in another format. Now used on the web for minor use-cases. Convenient for working around oddities. If Node does not support, it's a detriment. Not considered harmful. Security WG invited to look it over - reviews so far same as the browser. +Use-cases: +Why use absolute URL for an in-memory resource? You can put records into the main module map. No other way except if you write to file system. Data URLs may have performance issues. The ability to create singleton module addressable by content is the biggest use-case. You can have a thenable module that lets you get the namespace - no other way to do this with dynamic import(). +JH: Data URLs do not have a current location, so they can't access relative things. This is a historical decision in browsers. Gus and I worked on Symbol.thenable to solve the thenable module issue. The data URL trick was mentioned but it seems like it's only possible to do it on things available absolutely. How do I solve the thenable case? +BF: You need a way to resolve to an absolute specifier. +JH: You need import.meta.resolve to be available to handle this use-case. +BF: Agreed. Should be a separate PR. Browsers only agreed to a sync resolve. Not agreed to async resolve. Something that is not idempotent. Non-trivial for Node to perform sync resolve. Would be a hefty performance cost. We'd need to move the resolve off-main-thread. +Gus: Disable code generation from strings? Would data URLs get around this? +BF: That doesn't stop VM methods. Disabling code gen is better talked about in security WG. Not in scope of Modules WG. You could block it in the loader. +JH: Exports - static strings - but data URIs are content addressable - there is no way to rewrite them all. +BF: Dynamic strings are not supported in export maps. +GB: USe-cases. Given loader API is in flux, could we not change it to handle the use-cases tackled by data URLs? +BF: Loader could do it if they implement data URLs. Creating content addressable use-cases. Thenable trick might be possible with loader alone. +GB: You could have hashes of the source in the loader if you wanted. +BF: Don't think the result will be very different from data URLs at the end. Custom loaders can do this, but data URLs are more compatible. + +### nodejs/modules + +* Loader Hooks [#351](https://github.com/nodejs/modules/issues/351) +BF: We took a long break on this from last year. Recently, this issue was opened. Now working with Issue creator Alex to record historical progress/intent. Alex is willing to do a lot of work on this. We talked about direct exposure of language hooks (HostImportModuleDynamically) take referring module + specifier string. The specifier may become an object in the future - due to TC39 proposals (assets + trusted types). Must return a Module Record - not necessarily a SourceText Module - could be WASM, JSON, HTML modules. Need a way to differentiate results. +Other type of hooks: decompose into specialized hooks. Want hooks to not need to implement entire resolution algorithm, e.g. CoffeeScript transform only modifies the text body and does not need to change the resolve. +Contention over whether to expose low level language hooks or higher-level specialized hooks. +Jan had a proposal to decompose into phases. +BF is a fan of the language hooks (the JS-specced hooks). +There is contention between exposing language and being comprehensive whilst being understandable. +History suggests we will do both, e.g. a default at the language levels that most high level loaders will not override. We have not discussed this. +Yarn has expressed desire for some hooks - Node has not had an answer for this. +Yarn would like a resolver that can do partial resolution - you stop at a point in time such as when you have a bare-name, rather than resolving all the way to what "main" is point to. +Gus: +Mael: We couled re-execute but it might not be what we want to do. +BF: This is also a problem for virtualized file-systems because the default resolver could not handle it. Cloud providers are considering these. Yarn/Tink/Entropic are also looking at virtualization. +GB: Does this intercept all fs.read ops? +BF: Yes, for embedded things. +Mael: We patch many of the filesystem methods - becomes indistinguishable. Yarn tried to do all of them. A Loader API would mean we could stop this. +GB: Do you want to know where a package is without looking inside? +Mael: Yes - we have part that is synchronous, but reading pkg.json needs file system access so might need to be async. We only care about the first one. +GB: If Node exposed pkg-only resolution is that useful? +Mael: Yes. +GB: You're not using node_modules resolution. So I guess that's not useful for you. We've talked about exposing ESM resolve API. But if you're not resolving into there, I don't know if it's useful. +Mael: node_modules bit we shortcut. Other part is resolving into index.js. Assume Node uses "modules" field, in which case Yarn overrides have it. We must match Node. If we could just resolve pkg.json and index.js that's fine by us. +GBB: Use-cases fell into two cases: 1. coffeescript transform. 2. resolve specifiers that may not be on disk, e.g. @ might refer to package root, or add a file extension. +BF: We haven't changed loaders in many months. Current in master are a subset of the language hooks. You can only return locations of module records. +GBB: Does this match your expectations? +BF: I expect both. Power users like Yarn want to take over the runtime. Then specialized hooks are for people who don't want to do that. If we improve the default resolve algorithm (e.g. policies) then you might break expectations, e.g. coffeescript will want those policies to be retained. So compatibility concerns. +Next steps: Alex and BF are working on a design doc we'll push the constraints back to the loader thread. Contentious topics are language vs specialized hooks. Then data design questions. Big desire for loader hooks to support in-memory resources. +GBB: Could someone write up language hooks vs specializer hooks? +BF: Imagine a constructor in a class. If you don't provide it, you get the default. It calls super() properly. We're talking about what that constructor does. With specialized hooks, we break it down inside that constructor, e.g. describing how super() is called. One is the overall operation. Other is a sub-operation. Specialized hooks may allow certain optimizations. People could think about which hooks they want. Please write up your desires/constraints and send to Bradley/Alex. + + +* Package Exports [#341](https://github.com/nodejs/modules/issues/341) +DEFERRED + +### Observer PRs + +No objections to adding Mael/Devon/Darcy + + diff --git a/doc/meetings/2019-08-14.md b/doc/meetings/2019-08-14.md new file mode 100644 index 0000000..53d8406 --- /dev/null +++ b/doc/meetings/2019-08-14.md @@ -0,0 +1,140 @@ +# Node.js Foundation Modules Team Meeting 2019-08-14 + +## Links + +* **Recording**: https://www.youtube.com/watch?v=OEFVWjpkbao +* **GitHub Issue**: https://github.com/nodejs/modules/issues/370 +* **Minutes Google Doc**: https://docs.google.com/document/d/1Ng0-0Mp34z2gAqkDsyWsZkFK6XUIrgsyA3lv8i4917U/edit + +## Present + +* Myles Borins (@MylesBorins) +* Gus Caplan (@devsnek) +* Jordan Harband (@ljharb) +* Jan Krems (@jankrems) +* Saleh Abdel Motaal (@SMotaal) +* Bradley Farias (@bmeck) +* Alex Aubuchon(@A-lxe) +* Geoffrey Booth (@GeoffreyBooth) +* Guy Bedford (@guybedford) +* Daniel (@???) +* Darcy Clarke (@darcyclarke) + +## Agenda + +## Announcements + +*Extracted from **modules-agenda** labelled issues and pull requests from the **nodejs org** prior to the meeting. + +### nodejs/node + +* esm: support loading data: URLs [#28614](https://github.com/nodejs/node/pull/28614) + +Bradley: We talked about data URLs last week. No objections on the PR right now, no quorum, no moratorium. Can we go ahead with this? + +Myles: I think we're good to move forward + +Jordan: This feels like something that will be abused more than used, and the benefits are minimal compared to the risks. Justification seems to be "the web did it so we should too" which doesn't sound sufficient to me. I'm not clear on what the use cases are when you have a filesystem, unlike the web where you don't have a place to make a temp file on the fly. + +Bradley: We have a few use cases in the PR. Particularly when you don't have write access to the file system. + +Myles: On GCP we don't offer file system access -- none at all after build. + +Saleh: It's useful to see the use cases, and so as long as it's flagged, we can see use cases. Also, cross node/web functionality allows for more consistency between solutions (ie loaders). Node currently forces you to write virtual modules to disk. + +Jan: I talked with somebody yesterday about on-the-fly content generation, and they cannot rely on disk (security concerned, can't trust file system writes). At groupon the temp directory was not guaranteeable across environments. + +Jordan: Ok so even if all these use cases *can* be done with the filesystem, the filesystem isn't always available, so data urls become necessary. + +Geoffrey: For security concerns, you might want to avoid writing to the file system as that exposes it to disk issues, other processes looking at it. + +Bradley: I think we should avoid security concerns as they're difficult to puzzle around given Specter. + +Jordan: @jan commonjs requires the filesystem? + +Jan: The require system is open to all things, so you can generate a module from string. + +Jordan: Are there plans to add data URIs to cjs? + +Jan+bradley: No because you can do it anyways, and also the cjs specifier system doesn't work well with URIs. + +Myles: Taking a step back, we have an upstream PR, but no upstream objections. Are there any? + +Bradley: It sounds like we should keep data uris behind a flag + +Jordan: I think it should be behind the experimental modules flag -- it would be a waste of effort to make a new flag right now. + +Myles: I don't think we need a flag, but it's not much work to add one. It would be weird to not have a flag and add one later, so if that's the plan start with it. I prefer no flag though. + +Jordan: I'm not fully in favor of data uris, but I do not want to obstruct the PR. + +Myles: How about we merge it, and then Jordan opens a revert PR arguing why. + +Jordan: Give me a week or two to consider if the revert is needed, but SGTM + +### nodejs/modules + +* Validate exports keys and values [#359](https://github.com/nodejs/modules/issues/359) + +Jan: Landed already + +* Package Exports [#341](https://github.com/nodejs/modules/issues/341) + +Jan: made a lot of progress implementing exports, behind an additional flag automatically enabled if using --experimental-modules. Progress board exists. Implementation supports most of the proposal. `.` "main" discussion ongoing. Import map names still being discussed as well. Would like to gather consensus on those before unflagging. + +Jordan: Would love to see this land in CJS, tons of uses. + +Myles: would this be moved out of experimental if unflagged or stay experimental. Prior art allows us to unflag while keeping experimental. + +Jordan: If we are unflagging it, it should be considered stable. That is when people can use it reliably without opting in. Stable enough to risk no further changes possible. + +Saleh: exports written for CJS. even if ESM is flagged would this be available in CJS without a flag? + +Jordan: yes + +Jan: After unflagging exports should remain experimental potentially because it has ecosystem effects. Important to get feedback from other environments like bundlers as ecosystem support is picked up/considered. As ecosystem adopts, changes may need to occur with care. + +Myles: we can keep the console warning even without the flag when it is used. I am comfortable calling things stable even with these warnings. + +Jordan: I was mostly talking about unflagging ESM, unflagging exports is not as much a concern. When unflagged ESM will be risking things being kept as-is. We can prepare the ecosystem for unflagging by adding lint rules and support to 3rd party resolvers. + +Myles: the stability being experimental allows smaller edge cases like error codes to be changed without a semver major. + +Jordan: since exports affects 3rd party code, the warning would be coming from 3rd party module installation. + +Jan: minor issue with exports is that it cannot be unit tested by the package itself. This should be fixed. + +Jordan: this is a problem in general of "testing your package as if it were published". + +Myles: loading via relative specifiers inside the module is inconsistent vs "exports" + +Jan: See "imports" proposal from guy for internal aliasing. +… talking about using ~ to get current package ... + +Jordan: might want to enforce ~ going through exports. E.g. package.json#"files" is dangerous because forgetting to add files to the whitelist. + +Myles: seems like we are ok with "exports", flag removed separately from ESM. We should identify a new feature/propose something for ~ which we talked about. Should we sync/move package name proposal from seeking consensus to implementing? + +* Loader Hooks [#351](https://github.com/nodejs/modules/issues/351) + +Brad: Lots of comments in design doc + github. Adding success criteria to examples. Need to sync up on changing the hooks themselves. Lots of suggestions and different directions. General feeling that we should split up the hooks somewhat. A few ideas on what the divide to be. Fairly consistent feeling that resolve return location and a seperate fetch hook that turns a module record or resource for module record is an approach being talked about. Figuring out how to split that and start moving. + +Still questions about how loaders should defined their format. Design document has a mime for determine format. Current loader is using a string enum. Contention on which approach should be used. Likely something we need to reach consensus within the committee. Don’t have quorum, so perhaps we need to talk about it again. Good thing to start talking about. + +Guy: Thread on loader discussion looks [?]. Lots of agreement on things, not much contention. Biggest thing for a usability perspective is separating the hooks. Happy to see we can look at the design. Mostly usability concern regarding mimes, but it might be superficial. A little bit concerned we don’t have a mime type for a node builtin, or what a .node file would be. So we need to come up with something for those cases. + +Brad; we have “application/node” for cjs. We tried to standardize something for C++ modules but it was rejected as they don’t have a well defined format. Could look at vendor track rather than standards… just need to agree with what to call them. + +Guy: I’m completely happy to go with mimes, no problem moving forward with that. + +Brad: Working on the way to have composed instrumentation of builtins. Added example but it may not cover all examples of what people can do. Basically the contrived example is adding to instrumentations to fs.readFileSync. Adding a console.time and console.endtime to it. If you have two loaders doing this it is hard to build right now, need more design work on this. Guy showed an idea of using import.meta to pre-inject other module namespaces so they can be used without a lot of redirection or source code rewriting. This is one of the harder solves + +Another built in experiment we have is the null loader. A loader that doesn’t load something. Example preventing loading fs. Synthetic module loader is another example, something created from memory. A transpiler loader, success criteria includes having a custom format sent through the pipeline. You need to implement regular expression modules. + +Are there specific use cases we need to have? + +Geoff: glancing at design doc for first time and I see the new examples at the end. When ready would love to see an example of DX. I see code but where do I put it? How do I let node know to use the loader? How do I create / use a loader? + +Alex: @Geoff the --loader api is the same as the way cjs loaders are currently loaded with --require, so we can look towards them for usability. One difference is that with cjs, loaders can often also be loaded in the middle of runtime with require(...), which we are not allowing for esm loaders due to security, predictability concerns. + +A use case I haven't seen considered is that of loading a v8 compile cache (https://github.com/zertosh/v8-compile-cache). The current proposal does not support dynamic instantiation of modules (only returning their content so that node can do the instantiation). Given that, the only way I can see supporting compile cache is by adding a compile cache content format that node supports. This is outside the realm of loader hooks, but should be kept in mind. diff --git a/doc/meetings/2019-08-28.md b/doc/meetings/2019-08-28.md new file mode 100644 index 0000000..d117e3a --- /dev/null +++ b/doc/meetings/2019-08-28.md @@ -0,0 +1,183 @@ +# Node.js Foundation Modules Team Meeting 2019-08-28 + +## Links + +* **Recording**: https://www.youtube.com/watch?v=OBoGjhmz7jk +* **GitHub Issue**: https://github.com/nodejs/modules/issues/376 +* **Minutes Google Doc**: https://docs.google.com/document/d/1wN0J6PuUUjNli3BzzroJ2goljYtfvXOmKuYL_Xf1SAQ + +## Present + +* Modules team: @nodejs/modules +* Jan Krems (@jkrems) +* Wesley Wigham (@weswigham) +* Myles Borins (@Mylesborins) +* Saleh Abdel Motaal (@SMotaal) +* Guy Bedford (@guybedford) +* Darcy Clarke (@darcyclarke) +* Rob Palmer (@robpalme) +* Jordan Harband (@ljharb) +* Gus Caplan (@devsnek) +* Hassan Sani (@inidaname) +* Geoffrey Booth (@GeoffreyBooth) +* Jeremiah Senkpiel (@Fishrock123) +* John-David Dalton (@jdalton) + +## Agenda + +## Announcements + +*Extracted from **modules-agenda** labelled issues and pull requests from the **nodejs org** prior to the meeting. + +### nodejs/modules + +* Divergent specifier hazard [#371](https://github.com/nodejs/modules/issues/371) + +GCB: When I was working on the attempt to try to get dual packages (the same package that contains both CJS and ESM sources being able to do require(pkg) as well as import(pkg) and it works in both cases). The user has to write require(pkg/cjs) import(pkg/esm). So we didn’t want to have two specifiers. If we have one specifier mean two different things in the two module systems this can cause chaos. And this has already come up in the GraphQL project where they adopted mjs under the previous --experimental-modules. I didn’t fully understand the issue so I dug into it a bit and tried to come up with a minimal repo of what the issue was. My sense was that this isn’t just a dual package thing but would also apply to extension searching, file.js and file.mjs having import ‘./file’ and require(‘./file’) resolving to two different things is basically the same issue. We basically have three options - (1) what we have now, (2) allow a hazard and document it properly or (3) come up with some clever solution to avoid the problem. The best one we have at the moment that came up in the thread is to have require support requiring esm, so that both would get esm sources resolving the issue. There’s a lot of issues separately with getting require(esm) to work. + +JHB: The hazard you are describing is one that has always existed in npm, not just because of extension searching but also because of package duplication which is a common thing in package ecosystems. The solution to this is peer dependencies which ensure there is only one copy in the tree. This is not something that can be fixed by leaving things as they are. If you use that solution then extension searching and dual packages offer no additional hazards. For me I have not seen any examples of a case that differs from that, so this hazard should not be relevant to discussing dual packages or extension searching. My understanding of the graphQL issue was having one in CJS and one in ESM. + +JK: I think it is skipping over some finer details to say that this is nothing new. Peer dependencies cannot save you from this assuming people are publishing packages that have both which is the only scenario that is actually a thing. Even if you only have a single copy of the package, because dynamic resolution applies you may find that you have two copies of the same module implementation within that same package. So there are ways to handle it by having specific files not specifically duplicated within the package. If you only transpile parts of the package and then have to manually maintain those parts of the package is definitely not something that the ecosystem is used to. + +JHB: That is fair thank you Jan. + +WW: I agree that the hazard exists, however the current tooling and recommended practices for working with CJS and ESM involve shipping both versions side-by-side. And changing peoples views on that is a monumental task. + +Going back to what Geoffrey was saying. We don’t actually need full require of ESM. All you need to say is that the ESM resolver is a strict subset of the CJS resolver. So that CJS would do everything that the ESM would do. If the result is something that should have been loaded as an ESM file, the CJS resolver can simply say no and throw. That’s all you need to do. You just need to be able to know that you should not load the thing. Actually being able to load it would be a nice enhancement on top of that but that’s not actually the thing that resolves the problem. + +JK: I assume that this implies ESM would never apply .mjs, or if it would it would do it with a lower priority than existing things. Otherwise it could never actually do it without stopping to be a subset of the CJS resolution. + +WW: The idea is that the ESM resolver today sees mjs as needing to have the exact extensions. Yes, if there is extension searching it must be the same. But the priority can be changed in both at the same time with something like the type: module field. + +MB: I want to step back a bit. Why Geoff brought this up was to look at why we don’t want to have extension resolution on by default, which does protect this hazard. To Jordan’s point, this hazard does exist to some extent within the ecosystem and is why npm started flattening dependencies specifically to minimize when this hazard would hit. And in my own personal experience when this hazard does hit it hits in ways that aren’t very intuitive. In order to understand what exactly is happening requires a somewhat sophisticated understanding of the Node.js module system. I don’t find that a compelling reason for us not to try and minimize the hazard in our own runtime. We’ve pointed this out as one reason for not having default file extension resolution, compatibility with import maps being another. Stepping back, the intention of opening this issue was to maybe close the discussion as to turning off extension resolution by default. If we want to discuss major changes for how we do things maybe that should be another discussion. + +GCB: To clarify it was actually to close the discussion around dual packages, but that was more of a side effect. + +MB: So you would like to remove extension searching as one of the cases that we are trying to solve for with our implementation? + +GCB: It would be moved down into the tabled area of phase 3 where we are not going to pursue this further. The only way I see we can avoid that happening is the require(esm) solution. If that PR comes up in a month, two months, even after we unflag it could be unflagged. + +WW: First of all, extension resolution, while it increases the surface area of this hazard, is not the only way this hazard surfaces. Anywhere the resolvers do things differently - ie in the interpretation of package.json flags. They should not be different resolvers, and the fact that they are different is the core of the problem. We just need to be able to guarantee that the one resolver always behaves as a subset of the other that is all. We’ve made this harder implementing the ESM resolver in C++ thinking that was a good idea, but if we were willing to just go back on that for a moment, then that would actually entirely solve this problem. Because both resolvers know about extensions, type: module etc. So the ESM resolutions can throw in CJS. And ESM can load CJS fine. + +JHB: Although I have many feelings about dual packages and extension searching, the thing I think is really critical is solving the problem of how I can publish a module that can work at least across Node versions - that is across the barrier between supports ESM and does not. If dual packages or extension searching isn’t the answer then fine let's find another answer. + +Anything that requires to write prose to explain it is just a massive failure in my opinion in providing compatibility and interoperability. + + +JK: ESM resolver is currently a subset of CJS. Anything that is not is a bug. The intention is for it to be a subset. A shared implementation might help guarantee that. But my attempts at sharing were not maintainable. Changing the CJS resolver is hard. I favour using a test suite to achieve compatibility. + +WW: We added exports to both. The CJS resolve doesn't understand type:module - that's the big bug in the current implementation. + +JK: After fixing this bug, we no longer face divergent specifiers. + +WW: If we start with the subsetting, we can add extension searching, then add require(ESM). That is incremental. + +Guy: But extension searching does not change the argument. + +WW: Divergent specifiers does not imply no extension searching. + +JK: This is about a specific form of searching - prioritizing mjs. + +WW: type:module lets you opt in to the change in priority. + +MB: The order of extension searching for CJS and ESM should be the same, and should switch based on type:module, means you can't just drop the extension to customize resolution behavior for each system. + +WW: ESM would need to respect require.extensions. + +JK: We are trying to get rid of that. + +WW: We could try to remove require.extensions from both systems. + + + +Guy: If you try to require(ESM) you would get an error. + +MB: If your "main" is extensionless, it resolves based on "type":"module". This solves a case. + +Guy: ESM files loaded by the CJS system must throw. + +MB: CONCLUSION: We will make the CJS resolver respect "type":"module", and extension searching order must be the same between both implementations. + +WW: The upgrade path is currently garbage. + +MB: Maybe we can target synchronous require in Node 14 as an incremental enhancement. Node 12 would cause breaking changes. + +WW: We just have to make things throw now. + +GB: We're not going to to implement dual packages before unflagging. + +JHB: I don't think we should do that. + +MB: We're at an impasse. Do you have an alternative proposal, Jordan? + +JHB: No. I would like an explanation of how to write a package in Node 12 LTS that works in 8, 12 non-LTS, and 12 LTS without forcing "await import()". + +GB: The only solution we have is the as-is. + +JHB: But this requires reading documentation. I rarely read readmes. + +JK: We know specifiers won't be modified, and tooling does not rewrite these for intermediate packages. We need to write that guidance. + +GB: Shall we put the migration guide on the roadmap? Do we defer dual-mode? + +JHB: Let's do the migration guide first. + +MB: It's not reasonable to block unflagging without proposing an alternative solution. + +JHB: Lots of people have blocked things. + +MB: This is different. + +ACTION: TLA in CJS? + +ACTION: Migration guide + +ACTION: Respect `type: module` in the cjs resolver + +* Revisit dot-main in exports field [#368](https://github.com/nodejs/modules/issues/368) + +JK: Exports originally had a dot main as a way to drop using main as one field for both. In the very first version there was even a shorthand where if you set exports to a string that would be the main. We dropped that (a) because of the dual package thing, and (b) dot main happened to be somewhat contentious as a way of having a key that was just a dot. But now that we have additional syntax in exports such as the array fallback syntax, we basically come to a choice between two semi-bad options. (1) bring it back - exports can override main completely both for CJS and ESM. or (2) we could support that same array syntax in main but that would mean that if you want to use fallbacks for your main export you would need to drop support for all bundlers before it was introduced. + +My thinking is that it is best to go with dot main. + +MB: Is the fallbacks support already supported in the implementation? + +JK: It is already supported - the fallback takes the first thing it understands, the first URL. So the current implementation does support it. And if we want to support it at anytime in future we do need to support it right now as the point of fallbacks is being able to have support for older versions. + +MB: If we have an array, and the first is ESM and the second is a CJS, and you have two parts in your tree. One imports the ESM then CJS and the CJS uses the same specifier? + +JK: Valid URLs will be taken. If it does not exist, it throws. + +MB: It fails based on invalid URLs. + +Guy: We could say only standard modules go through existence checks. + +MB: So main cannot be used for dual modules. It will just fail in the require loader. + +JK: Resolution is 100% repeatable. + +MB: I like this idea. We're not deprecating, but it shadows it. In a Node that supports exports, it's used. And "main" is the backwards compatibility hook. So for Wes/Jordan, this could be the solution we talk about for older version of Node. So you can transpile ES5/ES3 and use it as main. + +JK: The danger is that you have a package working on Node 10, then you upgrade and suddenly your app breaks because you were using require(). Having both fields makes this dangerous. + +MB: I will chew on it. Are there concerns if we open a PR? + +JS: This is a complicated discussion. + +GB: This would be a great feature. The upgrade process is tough. It would be a huge win to have environment-specific fields. + +MB: Hazard could be solved by best practice, e.g. console.log("you are using a legacy interface") + +GB: Downgrading the dependency would solve this. + +MB: We could have a mode:legacy that ignores exports and just uses "main". An opt-out could be sufficient to solve this. + +Guy: I think it could be worthwhile bikeshedding a name for this. + +JK: Yes. For the key name and the key location. I like the string|array possibility. + +MB: Agreed. Enables simplicity. + +* Loader Hooks [#351](https://github.com/nodejs/modules/issues/351) +* Package Exports [#341](https://github.com/nodejs/modules/issues/341) +* Proposal: Support loading package by own "name" [#306](https://github.com/nodejs/modules/issues/306) diff --git a/doc/meetings/2019-09-11.md b/doc/meetings/2019-09-11.md new file mode 100644 index 0000000..c7f1b4f --- /dev/null +++ b/doc/meetings/2019-09-11.md @@ -0,0 +1,96 @@ +# Node.js Foundation Modules Team Meeting 2019-09-11 + +## Links + +* **Recording**: https://www.youtube.com/watch?v=YyuP1hAdWOk +* **GitHub Issue**: https://github.com/nodejs/modules/issues/383 +* **Minutes Google Doc**: https://docs.google.com/document/d/1pvFPml9LWjaCvCs6wBFitC_Hm3EorLrqOoP9FmZ2rV8 + +## Present + +* Modules team: @nodejs/modules +* Wesley Wigham (@weswigham) +* Jan Krems (@jkrems) +* Jordan Harband (@ljharb) +* Rob Palmer (@robpalme) +* Darcy Clarke (@darcyclarke) +* Geoffrey Booth +* Guy Bedford +* Bradley Farias +* Hassan Sani (@inidaname) + +## Agenda + +## Announcements + +*Extracted from **modules-agenda** labelled issues and pull requests from the **nodejs org** prior to the meeting. + +### nodejs/modules + +* Update Phase 3 per current status [#382](https://github.com/nodejs/modules/pull/382) + +GB: Status update, not altering any designs. +JH: WRT LTS would unflagging of things for CJS like @ as a root be able to be pushed in a minor or would it require a major. +GBD: It is expected to not land CJS features after LTS. +JH: Even if ESM is not unflagged for 12 LTS, I would like to ensure exports and package-root for CJS get through before 12 LTS. +JK: Having/expecting more discussion around CJS to unflag features before ESM it should be done to help alleviate concerns about breaking changes. CJS features are altering of behavior. +GB: Ok to land? +-- no objections -- +SM: these features could tie our hands because people start using them, and it limits the design space when we do go for unflagging ESM. It could be either helpful or hurtful. +JH: We want to tie our hands, this means the features in CJS ensure they are in ESM. + +* Divergent specifier hazard [#371](https://github.com/nodejs/modules/issues/371) + +GBD: Added a PR about loading ESM under .js causing errors is now on core. https://github.com/nodejs/node/pull/29492 +GB: A different PR with updated info on how to publish a package with both CJS and ESM sources. https://github.com/nodejs/node/pull/29497 "." main PR is also coming and amend that PR. +GBD: We should get a few more reviews of those PRs. Would these PRs resolve this issue? +JH: PR answers the question, even if it is not ergonomic. This underscores one of the use cases that is not being met / we lack a migration strategy. This seems to be an unpleasant thing that hinders adoption. I have serious concerns if we unflag without an ergonomic solution to the use case of shipping a single package that is both CJS and ESM, it may be better not to ship ESM. +GB: Encouraging people to use the "module" field provides a standardized place to document entry point for the module +JH: We are not standardizing that field. It is structured; but if node is not using it, we should not recommend it. This may poison the package.json ecosystem in the same way that altering builtin prototypes has done in the past. +JK: Agree partially with JH, migration strategy is not perfect/ergonomic. Trade off was made with if a package can force dependents to use a certain form of your package [CJS or ESM]. Discussion around if implicitly switching is dangerous or not. We have been erring on the side of avoiding breaking usage at the cost of ergonomics. This will be an issue with migration. I don't see an alternative without risks of accidental breakage. I would prefer to be conservative over ergonomics. +JH: Some large and popular packages do fall into the singleton problem being discussed. People will switch to ESM in a non-major version and this will break people. Packages maintaining singletons are aware of their problem and can use peerDependencies and/or manage their current solutions. It seems a safer approach to allow for the interop and not the current direction of things. +GBD: There are many ways of approaching this problem. JS community has experience with solving problems in creative ways. Approaching this hazard in a sensible way is good, but userland will also likely find ways to solve this. This leads into the next topic. +JK: Not just singletons, but also relying on Babel/TS interop are affected. An ESM package relying on CJS package moved to ESM could also blow up due to the difference in interop. People doing things in de-facto minor versions is more of a concern than the singleton hazard. +JH: How is that different from creating a second package/entry point? +JK: Their dependent would need to explicitly opt-in to the different version. Could alter the default, in which case it would still be implicit. +JH: As a package author adding something to package.json to declare multiple formats is sensible, but I do not want package consumers to need to know and/or opt-in to a specific format. + +* Revisit dot-main in exports field [#368](https://github.com/nodejs/modules/issues/368) + +GBD: Dual environment use case and all features of "exports" are available to the "main" entry point. PR up on node core https://github.com/nodejs/node/pull/29494 . Also allows a string rather than an object form. Affects both CJS and ESM, singleton package effectively with only 1 exported module. +SM: Is this always going to be a "bare" package? import(pkgName) vs import(pkgName/sub). Does this prevent searching nested packages? +JK: Only look at a single "exports" entry during a resolve, won't look at nested packages. import(foo/node_modules/bar) would only look through the top package.json in foo +SM: Does this search up for the package? +JK: It preparses the base name and doesn't search upward. Only looks at /((@scope)/)?name/ it doesn't allow arbitrary depth. + +* Loader Hooks [#351](https://github.com/nodejs/modules/issues/351) + +BF: we split the hooks, altered some things for nested instrumentation, and need to add more success criteria. +GB: what is the status / who could give a way of building our own hooks. +BF: we don't have an implementation, this is design work. Our last implementation is from last July. I don't have hours to work on this. +GB: wasn't someone working on this? +JK / AA: ?? could start work on this ?? +SM: perhaps we could recreate the new API by reimplementing it in the old hooks? +AA: We have a more modern multi-loader implementation +SM: Does this need any C++ work? +AA: Will take offline. +GBD: since loaders have become monumental, could we progressively land incremental loader evolution? What happens if we expect breaking changes for LTS? +BF: There is a history due to http2 shipping a breaking change even after being unflagged. Go through LTS WG -> TSC . Ping myles +GBD: we could upgrade our existing API to help ease forward compatibility. +SM: could change the existing API a different name to keep backwards compatible if we want to promise interop. I have concerns about having a true backwards compat story with the current APIs. +JK: It is worthwhile to make breaking changes as quick as possible for backward/forwards compat. Will reduce the pain either way. We should try and have something for LTS even if it called `--experimental-loader`. +GBD: Don't think we need to call it `experimental` +JK: As long as we mark things as experimental, the actual consumers of the existing API could be helped even with a breaking change due to only having a handful of usages in the wild currently. +GBD: It would be good to ensure that any breaking change we ship still allows writing a loader that runs in the older node versions / can ship a loader than works in the new API and the old API. + +* Package Exports [#341](https://github.com/nodejs/modules/issues/341) + +SKIPPED + +* Proposal: Support loading package by own "name" [#306](https://github.com/nodejs/modules/issues/306) + +JK: Discussion about options/specifiers. Compromise of using the `@` seems to lack outstanding usage / objections. Can think about it as the "non-scope". Last blocker for "exports", useful for some applications seeking to test their "exports". Could be used as a "require from root". Can move from Do Not Merge to a real PR. Right now only has CJS implementation and needs tests/docs. + +-- Conclusion: Move forward with PR -- + +JK: Hope to merge next week. Might make sense to wait on approvals until the tests/docs are finished since right now it is ½ a PR. diff --git a/doc/meetings/2019-09-25.md b/doc/meetings/2019-09-25.md new file mode 100644 index 0000000..f0e7f37 --- /dev/null +++ b/doc/meetings/2019-09-25.md @@ -0,0 +1,107 @@ +# Node.js Foundation Modules Team Meeting 2019-09-25 + +## Links + +* **Recording**: http://www.youtube.com/watch?v=7c2gWo-g_Zs +* **GitHub Issue**: https://github.com/nodejs/modules/issues/386 +* **Minutes Google Doc**: https://docs.google.com/document/d/1JjL2yo8Mm3La_1-2P51PmJp2q4efE49YKyR62YxB6GA/edit + +## Present + +* Myles Borins (@MylesBorins) +* Jan Krems (@jkrems) +* Ryan Day (@soldair) +* Alex Aubuchon (@A-lxe) +* Saleh Abdel Motaal (@SMotaal) +* Guy Bedford (@guybedford) +* Geoffrey Booth (@geoffreybooth) +* Hassan Sani (@inidaname) +* Rob Palmer (@robpalme) + +## Agenda + +## Announcements + +*Extracted from **modules-agenda** labelled issues and pull requests from the **nodejs org** prior to the meeting. + +### nodejs/node + +* esm: globals should be wrapped to stay in sync with ESM [#29426](https://github.com/nodejs/node/pull/29426) + +I'm lacking context, but here's some rough notes - Alex + +Myles: I played around with removing the flag, but found that it changed the way initialization worked -- involving proxies. Matteo gave a -1 on Bradley's PR for performance concerns. + +Bradley: There's also an issue with getters. +A few approaches: disable importing the process module; allow the process module to go out of sync with the named exports; remove named exports from the process builtin; remove named exports from everything +We have to accept that we ban getters that throw. + +Guy: There's a possibility of deprecation control by catching thrown errors. + +Jan: At some point we will have to disallow naming certain properties. API design-wise, process and buffer were always focused around a default export, so restricting named exports isn't the worst thing. + +Myles: It seems unergonomic to only have two builtins without named exports. Most people use the global process object, though that's not true for buffer. + +Saleh: It might be a better path to have people import particular things from process. Selectively export particular things from process via a wrapper module around it. + +Guy: I'd be very much for keeping the ability to import process and buffer. In an environment without globals, it's nice to be able to rely on that. + +Bradley: Keep things importable, no objections for removing names. Not sure what Saleh means by having only particular names. We would need a proxy around the entire object, which is the problem currently. + +Saleh: I am thinking less about the primitive types, and more about the object types from the process object. Primitive values needing to be synced complicates things. ie for process.next, export a wrapper function. + +Bradley: What about `process.next`? + +Saleh: That would be a wrapper function `(...args) => actualProcess.next(...args)` or a proxy of it if necessary. + +Bradley: Upstream is open to anything that doesn't impact the performance of process.next. + +Bradley: We can't freeze builtins; too much of the ecosystem relies on monkeypatching them etc. + +... + +Guy: We could stop syncing between the esm and cjs exports. Loaders eventually would let us work around this (?). + +Gus: Back when I added this proxy, it just copied the proxies. While I was adding this, I asked as a policy that we don't add any more deprecations that fire on access. + +Jan: We need to focus on the core choice that Bradley mentioned earlier, rather than getting into the weeds. It might be easiest to just make it impossible to import the builtins now. + +Myles: This specific issue is absolutely blocking unflag. Any changes that we need to make outside esm need to land asap for this. If we remove the flag and that changes performance characteristics of cjs, then that could be considered a breaking change and prevent unflag. We should find a proposal for solving this and bring it upstream needs to happen in the next week and a half. Does anyone want to make a specific proposal? + +Bradley: If we want to unflag, adding the proxy would be problematic, but leaving it out wouldn’t. I think letting esm+cjs go out of sync is fine. I can't see a clear solution that fixes all the edge cases. + +Gus: How often do APMs patch things more than once after starting? + +Bradley: An audit last year says not often, only one group was *interested* in doing it. + +Saleh: Given going out of sync, can we determine where process/buffer will be snapshotted + +Bradley: It has to be when first accessed (imported) but we could expose an API down the road to update the snapshot. + +Gus: Just to clarify, first accessed, ie imported or required. + +... + +Myles: I'm plus one on removing the proxies. We could unflag, but keep things experimental -- allowing us to solve ecosystem issues that emerge in later versions. + +... + +Myles: Action item for Bradley to make these changes, using Guy's api proposal. (:+1: from Bradley) + +### nodejs/modules + +* Divergent specifier hazard [#371](https://github.com/nodejs/modules/issues/371) + +Geoffrey: There's not much going on here. A doc change was made, and there's not much more to be done for this version. + +* Loader Hooks [#351](https://github.com/nodejs/modules/issues/351) + +Action item for Alex to alter the flag scheme for esm loaders. Either add a flag to enable the --loader flag, or change the --loader flag to something like --experimental-loader. + +* Proposal: Support loading package by own "name" [#306](https://github.com/nodejs/modules/issues/306) + +https://github.com/nodejs/node/pull/29327 + +* Fallback Support Removed in Import Maps [#385](https://github.com/nodejs/modules/issues/385) + + diff --git a/doc/meetings/2019-10-09.md b/doc/meetings/2019-10-09.md new file mode 100644 index 0000000..1365019 --- /dev/null +++ b/doc/meetings/2019-10-09.md @@ -0,0 +1,116 @@ +# Node.js Foundation Modules Team Meeting 2019-10-09 + +## Links + +* **Recording**: https://www.youtube.com/watch?v=l4lb-mURzyw&v=-6XBx7yinrI + > [Part 1](https://www.youtube.com/watch?v=l4lb-mURzyw) | [Part 2](https://www.youtube.com/watch?v=-6XBx7yinrI) +* **GitHub Issue**: https://github.com/nodejs/modules/issues/396 +* **Minutes Google Doc**: https://docs.google.com/document/d/1ZkgPmwULxJz2kLEHRZyPqbJnlXcWUgB1ouphPvxKlGE/edit + +## Present + +* Modules team: @nodejs/modules +* Jan Krems (@jkrems) +* Alex Aubuchon (@A-lxe) +* Gus Caplan (@devsnek) +* Geoffrey Booth (@GeoffreyBooth) +* Guy Bedford (@guybedford) +* Jordan Harband (@ljharb) +* Saleh Abdel Motaal (@SMotaal) +* Wesley Wigham (@weswigham) +* Bryan Clark (@clarkbw) +* Ruy Adorno (@ruyadorno) (observer/guest) +* Rob Palmer (@robpalme) + +## Agenda + +## Announcements + +* Extracted from **modules-agenda** labelled issues and pull requests from the **nodejs org** prior to the meeting. + +### nodejs/modules + +* Roadmap update [#387](https://github.com/nodejs/modules/pull/387) + + * Geoffrey: Based on discussions in latest meeting. Shouldn’t be surprising. + * Guy: Just looking for approval? + * Geoffrey: Assuming that we want to unflag for v12 LTS, anything not done yet was moved to stage 4 (post-unflag). This includes loaders, self reference (`@/`), … - all should be okay to ship after unflagging. Only blocker is PR to fix tests. + * Guy: Not as simple, update on unflagging PR? + * Geoffrey: May be documentation update, may be able to land without quorum? + * Jordan: As long as this doesn’t imply final consensus on unflagging, I’m fine with landing. We need a proper decision on that. If this is just a "guess" on what’s likely to be in the unflaged implementation, it may be misleading. + * Guy: So far focussed on technical blockers to unflagging, not yet ready to make decisions on features to go in. Biggest change is async node bootstrap which may change timing, promise/task queues, emitting events, async hooks, etc.. + * Guy: Going to talk to Matteo about async hook design decisions. Rough estimate at ~50% before we can talk about consensus / make calls on unflagging. + * Saleh: What are the critical choices that force us to do an async bootstrap? Are there other places than modules where async bootstrap would have helped / would help in the future? + * Guy: In theory, it should be historical that it is sync. This is just the first time core is forced to do something async. TLA may force us to be async sooner or later. We may be able to make it sync but not clear if it’ll pay off. + * Wes: We may be able to spin event loop without microtasks. + * Jan: Don’t we need microtasks for the modules-related Promises? + * Guy: Happy to have other people look into it, running out of cycles to do it myself. If we don’t get this figured out soon, it doesn’t look good for unflagging this month. + * Geoffrey: Don’t need to get it in this month (technically) for unflagging in LTS according to Myles. There may be wiggle room but there’s no precedent for unflagging a big feature within LTS so we can’t depend on it being possible. + * Geoffrey: Once tests pass - can we unflag? + * Jan: We should have an actual explicit decision on unflagging. + * Geoffrey: What does that mean in practice? Can we know already if we’ll have consensus to unflag in a few weeks? + * Jordan: Not convinced that loaders block unflagging. Hang-up to me is - beyond extension searching etc. - compatibility of a published package with pre- and post-module node. Achievable with exports-dot if it’s included in unflagged version. Second kind of compat is require and import possible using same specifier, in some form. The use case needs to be covered somehow. Don’t have time myself but time shouldn’t completely decide what is possible with a "complete" modules implementation. Don’t want to be blocker but we should be aware of the effect we’ll have. If we unflag right now, package authors may not know how to properly deal with ESM migration, bumping versions correctly, handle migration, etc.. + * Jan: Need to make choice between one-resolution-per-specifier-and-source vs. maximum-adoption (import may resolve to module while require may resolve the same specifier to another URL). + * Jordan: Singleton should be dealt with by package authors, better to optimize for maximizing ESM code running. + * Geoffrey: Others have already objected to "dual mode specifiers". There’s no ideal outcome most likely. + * Jordan: Right now pkg.exports is ~import map. Maybe there’s a way that can remap a specifier somehow, not by default, to opt into divergent specifiers. + * Geoffrey: Similar to declaring named exports in package.json. + * Jordan: Difference between making it impossible and being annoying but potentially temporary. + * Geoffrey: People will have tools and keep using it (module field etc.) and may use them to work around pain points. Should have another discussion around this at next meeting. + + + +* JSON modules are being reverted on web [#391](https://github.com/nodejs/modules/issues/391) + + * Guy: JSON modules have been backed out of HTML based on security concerns. Other syntax for importing data formats (like JSON) is being discussed. Suggestion: Reflag in node. + * Jan: Flagging right now seems cheap, let’s do it and potentially undo once the dust has settled. +(no objections) + + + +* Script-based configuration files in "type": "module" packages [#389](https://github.com/nodejs/modules/issues/389) + + * Guy: Found regressions on packages that eagerly adopted type:module where the packages weren’t consistently using .js for ESM only. Babel transforms & tool configs may break. For now it’s disabled but we have to decide how to go forward. Opposite direction suggested in https://github.com/nodejs/node/pull/29862 - do not restrict CJS loader for type:module, just not inject into ESM. That way it will generally fail when used from the wrong loader (because of syntax errors). Wesley had concerns? + * Wes: Allowing CJS to require JS inside of type:module is walking back the clear answer on how to interpret a file. Tools can no longer tell from the file how it should be interpreted. Heuristics used today *could* be dropped but backing out of type:module would force them back. + * Saleh: Maybe `createLegacyRequire(path)(…)` || `require.legacy(…)` that works (but is discouraged) and ignores type. + * Wes: They can already do that. ESLint already switched to another implementation. It can already be done. But code exists that "works" because it assumes the wrong semantics. + * Geoffrey: I can understand why tool authors are annoyed that node "broke" them. There’s not that many of them, they can fix it on their side. Determining how to interpret a file requires a heuristic today. + * Wes: But we’re trying to move away from those heuristics. .js is always CJS in a node package unless it’s mjs or type:module. + * Geoffrey: Suggested having a method to determine how node would interpret a file. + * Jan: I don’t think there’s disagreement that we want all .js in type:module to be enforced to be ESM. We could start with a deprecation warning right now. + * Wes: That’s fine as long as type:module hasn’t been shipped. We shouldn’t keep the deprecation warning after type:module has shipped (unflagged). + * Guy: Let’s try to add the warning now, aiming to make it throw when modules ship (are unflagged). + * Wes: That’s fine as long as the warning doesn’t survive unflagging. + * Jordan: (As maintainer of the resolve npm impl) +1 to fewer variants being live. + + + +* Loader Hooks [#351](https://github.com/nodejs/modules/issues/351) + * Guy: Skipping over + * Alex: Changed `--loader` option to `--experimental-loader` (With an undocumented alias to `--loader`) — [#29752](https://github.com/nodejs/node/pull/29752) + + * Proposal: Support loading package by own "name" [#306](https://github.com/nodejs/modules/issues/306) + + * Guy: Uncertainty about the symbol being used, parcel is pushing for ~. + * Jordan: Originally was ~, Gus objected (or echoed) because of posix home directory. Settled on @. If nobody has a strong objection, we could land. + * Geoffrey: Matteo did bring up ~ as well (TSC member), cc’d TSC member. + * Jordan: Can we agree on either being okay, as long as TSC picks one? + * Geoffrey: Think we have consensus? + + + +* Unflagging experimental exports (https://github.com/nodejs/node/pull/29867) + + * Guy: Posted PR, currently behind own flag. May be important to unflag modules itself. We don’t need to land right now. Order is important here. + * Jan: Change of position: Fine to land right now and make self-reference a follow-up. + * Saleh: At some point there may be dual mode. If we do unflag exports - is a package having multiple modes an issue? + * Guy: Right now exports are independent of mode/interpretation. They are orthogonal. You can use exports to map to different files for different implementations - using different paths. + * Saleh: Clarifying - meant one specifier pointing to different files. + * Guy: That’d be conditional mappings. Not implemented yet. + * Jan: But design space for "forking" of specifiers is open using arrays. + * Saleh: Yes, perfect, so dual mode leads to enhancement that does not break export maps in the ecosystem on various versions of Node.js (thanks) + * Jordan: Would be happy to unflag right now. But we should unflag around the same time as modules. + * Guy: We could also unflag as a single PR? + * Goeffrey: Should we just remove the additional flag and move exports to `--experimental-modules`? + * Jordan: If we come up with pre-modules/post-modules outside of exports, we could also ship exports immediately. + * (consensus) diff --git a/doc/meetings/2019-10-23.md b/doc/meetings/2019-10-23.md new file mode 100644 index 0000000..5d15492 --- /dev/null +++ b/doc/meetings/2019-10-23.md @@ -0,0 +1,90 @@ +# Node.js Foundation Modules Team Meeting 2019-10-23 + +## Links + +* **Recording**: http://www.youtube.com/watch?v=JrAPSQygySg +* **GitHub Issue**: https://github.com/nodejs/modules/issues/404 +* **Minutes Google Doc**: https://docs.google.com/document/d/1RlqUYYCFLnK9_YNMia2CSRMEGUUlbms3bZmh4XrVJ7Y/ + +## Present + +* Myles Borins (@MylesBorins) +* Jan Krems (@jkrems) +* Wesley Wigham (@weswigham) +* Geoffrey Booth (@GeoffreyBooth) +* Alex Aubuchon (@A-lxe) +* Jordan Harband (@ljharb) +* Rob Palmer (@robpalme) +* Ruy Adorno (@ruyadorno) (Observer/Guest) +* Saleh Abdel Motaal (@SMotaal) +* Guy Bedford (@guybedford) + +## Agenda + +## Announcements + +*Extracted from **modules-agenda** labelled issues and pull requests from the **nodejs org** prior to the meeting. + +### nodejs/node + +#### module: resolve self-references [#29327](https://github.com/nodejs/node/pull/29327) + + +* Jan: New issue for open questions modules 403. https://github.com/nodejs/modules/issues/403 +Is this in scope for shipping modules? +Slim use case - slightly nicer unit testing for package with exports. +Everything else is unrelated. +Looking for disagreements - otherwise it’s sunk cost and out of scope. +* Myles: Engineering is now done behind a flag. What are we waiting on? +* Jan: Current version works in theory Disagreements on things needed for unflagging - so uncomfortable landing now. Many things are more important for landing modules than this. It’s lost time on an unimportant feature. Should focus on core module experience. +* Myles: Your advice - table and revisit later? +* Jan: Yep. +* JH: Agree. I hoped unflagging would have all resolution changes at that time. For example, resolve() needs updating - would be nice to handle self referencing at that time. Not a reason to hold back modules. +* Myles: Apart from the sigil - is there a case for not moving forwards? +* JH: Matteo’s running inside and outside package. Demands package name be a self-reference. We decided that we needed a sigil - package-name is insufficient. Sigil bikeshedding is not worth it. +* GB: I thought we would ship package name then do the sigil later. +* JH: Shipping sigil later is fine. I’d prefer both go unflagged. +* MB: Objections to landing self-refs imports behind a flag using package name as the reference? +* **CONSENSUS ACHIEVED** +* Jan: As long as… +* MB: This is separate to unflagging modules. It bitrots - landing it is to allow it to stay good so people can play with it. +* WW: This name is the “name” in the package.json? package.json#name +* JH: So resolving self-ref will not block modules AND will not block exports AND we’ll land the current PR behind an additional flag and land it in future. + + +### nodejs/modules + +#### Lets document the current state of Modules in Node.js [#402](https://github.com/nodejs/modules/issues/402) + +* MB: We shared it with TSC. TSC discussed. Hazard was discussed. Existing hazards + new hazards. Equal good and bad responses. TSC want us to figure it out. TSC cannot give advice. Some people had more ideas - Myles will follow-up. + +#### Divergent specifier hazard (DiSH) [#371](https://github.com/nodejs/modules/issues/371) + +#### Conditional exports proposal [#401](https://github.com/nodejs/modules/issues/401) + +* GBD: This is a middle-ground - could exists separately from divergent specifier hazard issue. +* MB: We talk in circles about DiSH. Conditional exports is the first thing that made me think it’s worth introducing the hazard in order to get the DX of conditional exports. There’s non-trivial work to do on conditional exports. +* WW: I don’t get conditional exports. We started talking about DiSH - multiple things refer to the same thing via the same name. Feels bad a wrong to do the same thing another way. +* GB: Default extensions was another way to allow import/require of the sme specifier to lead to different modules. We avoided because author can fall into the trap accidentally - want to minimize a footgun. Explicit syntax - needs to be taught at which point the hazards are taught too. It’s a power tool with a safety latch. +* MB: To Wes, 1. Extension resolution led to DiSH and incompatibility with browser implementation. I know that's not perfect. But if we had arbitrary file extension resolution, import maps grow unbounded. It wasn’t the only reason. When we fixed the order of resolution, we removed overloaded “main” being used. So these are separate spaces to discuss. With DiSH, we already have a hazard today, creating another one is a problem. From consumer PoV, separate specifiers introduces a variant of this today (GraphQL plugins might bring in two versions of GraphQL). Could be transparent from a graph of different things. It’s on the authors to eliminate the hazard. We could block it. But people still need to solve i entrypoints can end up in both caches. The benefit of having it via single-specifier is more explicit. Education either way. With conditional exports, CJS packages can lead to auto generated ESM wrappers. Whilst I am uncomfortable, I am -0 on this - not an objector. We’ve spent a long time on this. Conditional exports allow us to hit almost every use-case we wanted. Still unsure if ready to land unflagged. Not sure if we should block unflagging on this being done. We should consider being flexible. +* Jan: You stole my thunder. Wes brought up two points. 1 hey wait, I thought we cared about DiSH. Why bring it back? 2. Hey if we want this possibility, why is exports different from file resolution solution? The reason why is because ext resolution works for exactly 2. Does not scale beyond. IF you want Node solution and a generic one for the browser, it does not help you. Extension searching would never have provided that. For the hazard, I used to be “we should not allow specifier to resolve to different things”. Currently convinced that package and package#esm just mask the problem and ignore dual instance issue - but they cannot! The message is dangerous - “it’s safe, you can be safe from dual instances” but nobody is writing import and require in the same file. So if a single file imports… In practice a user doesn’t import the same thing using import and require. In a realistic app, the hazard does not appear in a file, or a project. Solution is to port the project to ESM. Hazard exists in a package tree. Can’t fork all dependencies if different specifiers used. It’s a dangerous path. We should be clear it should not be possible two ship two versions. If you want to ship a require in an ESM package, then you are responsible for managing the situation of both versions being loaded. Conditional exports is a good way to tell that story! +* WW: Ok. If you started believing the hazard was a problem, then it’s still a problem with multi-entrypoints, it should still be a problem with mutli-entrypoint and package.json referencing. I like the current version because you can require(ESM) in future. This feature precludes that. I do not like allowing two copies to be loaded. The overhead is non-trivial. People talk about huge node trees. Doubling might not be the best idea. +* JH: I agree with Wes’s idealised goal. require(ESM) sidesteps the issue. Export allows new and old node. If we don’t ship conditional exports, the browser field does not handle ESM. People are still targeting pre and post module browsers. ConExps give us a solution for bundlers. Handles many use-cases. Does not eliminate the hazard. People can always write dumb code. Node should not police this. ConExp avoids a tragedy. require(ESM) does not eliminate the need for ConExps. So I want this feature to get to unflagging. +* GC: *long muted speech* +* GBD: Wes’ points are valid. Except we should be decisive about require(ESM). Wes did good work. Now that work has stalled, we can’t say it will appear in 6 months. But it’s not spec compliant with ESM. Promises are specced to suspend main thread. Flattening promises would be non-spec compliant. So problem is lack of work and spec adherence. Top-level await makes this observable. Additionally, we should realize strong arguments on both sides. The problem is to find consensus and to avoid stalling. We only have 1 or 2 meetings. People are already shipping packages with multiple versions for different target. So what should Node do to reduce fragmentation? If we don’t give users ESM in Node, people will dual publish and . It’s a short transition rather than something we live with forever. +* GC: *still muted* Gus says he mostly agrees with Wes and Jordan but will not suggest require(ESM). He warns to not get stuck in sunk cost fallacy. +* BF: I want us to be more careful about framing. Some people actively want to use the workflows we might state as “dumb”. This is such a primal issue, we lose focus on if the issue will be solved by us. If you install two versions of React, this issue already exists in CJS. Discussion might be overblown. Tone it back. I firmly believe this multi-copy issue will be a natural fallout of many workflows and is already the case in the wild today. +* Jan: To BF, I agree that it’s an existing issue. I agree that the framing of “we cna’t stop the user being dumb” is dangerous. Middle ground, people keep using a workflow and something just happens. Explicitly calling out some of these patterns: Babel transforms don’t rewrite specifiers. A reasonable package author will cause this if we have two specifiers. Comes from reasonable users and they get a bad outcome. This is different for trying to stop everyone from hurting themselves. For React, the new bit is the tools to manage the hazard. Npm ls shows you the multiple copies - and npm7 will surface errors during install for peer deps - but for the new hazard there is no tooling. If we use magic conventions, no tool will ever be able to understand what is going on. Those conflicts will appear and there is no way to debug. Declarative method lets you know why it breaks. I agree it’s bad for the world to be duplicated. If people want to ship both, and they cannot use require(ESM), there are two copies. As long as that’s true, we can’t prevent that. The issue is, how can we create patterns that have the right mental model. And how to track the issue without knowing all the implementations of your dependencies. +* BF: I want to pushback. Sure we don’t have tooling. React case is not the only way that leads to duplication. Vendoring - shipped in multiple bundles. Today’s tooling cannot prevent duplication. It’s not a solved problem. We cannot point to a tool and say the problem is prevented. I question how important this whole issue is. We could write tools. Or we could educate. One or the other. So maybe we should turn our attention elsewhere. +* Jan: Agree with Brad. There’s already code duplication. I pushback on the fact we could create future tooling. In one situation . In the other it needs to know about all the packages. So it’s the difference between having metadata ready for tools, or not having metdata. +* GBD: To Jan, there’s a big diff between dupe code in the same runtime and different runtimes. On the point of shipping, it’s really about authors. Everyone is waiting. A long time. If the only way they can ship it is to break CJS consumers, it gives them a hard choice. +* MB: Can we land ConExp behind a flag? +* Wes objects. +* MB: Can we land ESM as it exists? +* JH - I object. We need ConExp. (but Wes blocks ConExp) +* JH - Are we lone objectors? +* *silence* +* BF: I object to ConExp but not if it’s behind a dedicated flag. +* JH: If someone has to run Node with a flag to get a feature in a dependency, it’s like it does not exist. People don’t run Node with flags. So defaults matter. If defaults do not support dual packages, you can’t publish it. So my objection remains. +* MB: We can discuss a vote, but not now - time is up. + diff --git a/doc/meetings/2019-10-30.md b/doc/meetings/2019-10-30.md new file mode 100644 index 0000000..17d5d89 --- /dev/null +++ b/doc/meetings/2019-10-30.md @@ -0,0 +1,135 @@ +# Node.js Foundation Modules Team Out of Band Meeting 2019-10-30 + +## Links + +* **Recording**: http://www.youtube.com/watch?v=vaCSI7drtw8 +* **GitHub Issue**: https://github.com/nodejs/modules/issues/408 +* **Minutes Google Doc**: https://docs.google.com/document/d/1YZP6Sa5_qhXFaKN_rqO823AhN6PyvXRNYYstO_Cq21U/ + +## Present + +* Michael Zasso (@targos) +* Jan Krems (@jkrems) +* Geoffrey Booth (@GeoffreyBooth) +* Guy Bedford (@guybedford) +* Myles Borins (@MylesBorins) +* Saleh Abdel Motaal (@SMotaal) +* Jordan (@ljharb) +* Sven Sauleau (@xtuc) +* Wesley Wigham (@weswigham) + +## Agenda + +* Myles: Summary of the background for this meeting. We ran out of time last regular meeting and stopped short of making decisions around unflagging. + +### Windows path separators in specifiers [nodejs/node#30169](https://github.com/nodejs/node/issues/30169) + +* Reflag exports (1 open question: https://github.com/nodejs/node/issues/30169)? + +TL;DR: No clear outcome yet, throwing back to issue. + +#### Divergent specifier hazard (DiSH) [#371](https://github.com/nodejs/modules/issues/371) + +* Geoffrey: Shared gist, summed up hazard still being a thing with pkg and pkg/module +* Jan: Amend - even pkg and pkg-es are affected. Afaik only completely dropping support for CJS in new major removes the hazard +* Myles: Was very bullish on preventing hazard. Looking at developer experience, it’s more subtle. For consumers of plugins etc. it can still be invisible if that plugin is using the cjs or esm specifier. Solution is to educate about the issues around module singletons. No matter what we choose, we need to educate people. Dev experience difference between conditional exports vs. non-conditional exports is huge. Feels spiteful / baby out with the bath water to not have the feature. +* Guy: Follow-up on education. People want the superior experience of modules, want to ship them. Largest libs will do dual-mode safely as long as we can clearly communicate how to do it. If they don’t, they’ll quickly adjust based on bug reports. Issue is libraries on the edge / long tail that may not be aware. Communication is better than letting the ecosystem loose. People will also follow precedent of large libs. Hopefully also a temporary problem as CJS becomes less prevalent. +* Myles: Past argument was to ask big open source about their issues. Held it up in the past as examples for how bad the hazard is. This could now be a feature, we know people will find it quickly and will push for fixing them. +* Wes: Having dual entrypoints leads to a pit of failure, people will fall into the singleton issues by default. +* Guy: That’s why we should encourage wrapper modules as a pattern. +* Myles: What is an API to manually register the module in another cache? +* Jordan: How to make the two modules the same, at runtime? You can do it right now by getting the require cache from ESM. +* Geoffrey: That’s sounds like you want to do require(esm)..? +* Myles: Thinking about it from the other side, unidirectional. Create require function, access cache, in esm module before exporting set it in the CJS cache. If it’s later required, it would be loader from that object. +* Guy: That can be done using current APIs. +* Myles: That’s another possible way to avoid the hazard. +* Geoffrey: Have PR with more recommended patterns, first is wrapper that should be safe. Also mentions strategies to reduce singleton issues if not using a wrapper. +* Jan: My current theory is that for most of the long-tail, they will only appear once in an app. So in practice they will only appear as either CJS or ESM per project. So compiling using babel is completely fine. +* Myles: Want to continue with discussion around issues before seeking consensus. + +#### Conditional exports proposal [#401](https://github.com/nodejs /modules/issues/401) + +* Guy: Goal of PR is to extend the exports field to also allow objects as opposed to strings that express conditionals/branches in the resolution based on environment. It evolved out of rollup/parcel discussions around a userland/bundler-level “targets” field. Hope was that we could join forces and use the same kind of solution in node, browsers, electron, etc.. It’s up to runtime/environment to determine priority of resolutions. Can be analyzed by tooling. +* Wes: CJS/ESM branch is special because it varies within a runtime. Other conditions branch based on global runtime. +* Guy: If ecosystem tools do not accept it, there may be a path to deprecate it. We should put it out there as an experimental feature (unflagged) so we can get feedback. +* Myles: One thing that was brought up was the question around “default”. We’re talking about “module”, “default”, “node”. Find the exact distinction confusing. +* Guy: I’d be very tempted to not document the node condition. I’m of the opinion that the node mono-culture won’t last and there’s a future of many different runtimes. While it is the mono-culture, default should be generally used. I wouldn’t remove the node condition. +* Jan: I think modules using node APIs should generally call themselves out, to guard their behavior. Default should generally be for code that’s portable. +* Jordan: If I want to import(pkg/feature), how do I designate what to use in the conditional expression? Believe node should exist. +* Myles: For sake of time, we don’t have time to work through final list of conditions that should be supported by node. When we try to reach consensus, we could reach consensus on behaviors without locking down these details. Perhaps for the consensus better to focus on high level requirements and capabilities. +* Jan: Should consider switching back to require guard. Default should be portable and CJS isn’t portable. +* Guy: We can investigate, don’t need to lock this down (to Myles’ point above). + +### Reaching Consensus + +### Topics to reach consensus on + +* Allow divergent specifier hazard +* Allow dual mode modules +* Landing Conditional Exports Proposal + - Behind a flag? + - Blocks unflagging? + - Conditional Mapping + - Alternative entry points a single specifier within node.js runtime +* Unflagging + - What features need to be unflagged first? + - Dual mode modules blocking unflagging? + - Timeline +- Ways to move forward +- Continuing activity of the group + +### Consensus reached with quorum of voting members present + +* Landing conditionals behind a flag +* Unflag ESM with current implementation +* Document Conditionals / Hazards / Patterns +* Deadline End of January to move forward with dual modules +* Unflag conditionals w/o dual mode asap + +* Wes: Don’t understand why we’re back to letting dual instance packages be a thing. Nothing seems to have changed fundamentally. +* Jan: The alternative changed, for me. +* Geoffrey: There’s strong ecosystem demand (proven by modules field). We should have strong reasons to not support it. +* Wes: I can get behind the use case. But I’m not fine with the current solution. I’m more happy to accept a preload field that opts people into it. +* Geoffrey: It doesn’t exist, should we consider it if it maybe works out but also maybe not works? +* Wes: I think we shouldn’t ship something that’s either sub-standard or doesn’t allow all important use cases. +* Guy: I don’t think preload is better, may require globbing node_modules tree. Also could lead to out-of-order execution. +* Wes: There may be differences, depending on how preload works. +* Guy: This seems to introduce non-determinism into execution. +* Wes: Isn’t that also true for conditional exports? +* Guy: Execution is deterministic. +* Myles: It’s currently possible to insert things into the require cache. We should try to create APIs that make it safer but it shouldn’t be implicit / magic. We shouldn’t underestimate how much node cares about startup time. Just because we think of something we’d like to do, if something slows down startup or runtime, it will be rejected upstream. Proxies got rejected for that same reason. It led to a footgun: Updating CJS built-ins without calling sync function gets ESM bindings out of sync. Other example was bootstrap refactor needed to make node work with promises during startup. We can reach consensus but we have to be aware that core needs to accept our decisions. +* Wes: Cost of multiple instances is bad performance. I’m dissatisfied with any solution that supports multiple copies (CJS and ESM). +* Geoffrey: Node supports both CJS and ESM. People will access both from the same package. +* Guy: Most users will have the naive expectation that node will support the module field. Either we provide a solution or they will find a solution that we may not like. +* Myles: Any objections? +* Wes: I object. +* Myles: Leading into - does dual mode block unflagging? Does anybody object to unflagging without dua mode? Even if we think that it may be solved in a future release? +* Jordan: I object. +* Myles: We have a deadlock. We need to resolve to move forward. JSON, WASM, etc. are on the horizon. Import maps, loaders may also be something in the future. We’re now at an impasse. How do we move forward? Should we have a vote? We can’t do it today, we should have a true majority. Should we introduce rough consensus? Asking generically: If we talked about this to its conclusion, can somebody say “I believe most people think X” and acknowledge that the rough group consensus is X. This would prevent a vote. Three options: Vote, rough consensus, or end of group. +* Geoffrey: Attempt at compromise. Are you objecting to the current master? +* Wes: If we unflag today, I would feel confident that we could move towards more complete and good. +* Geoffrey: We could agree to merge, with flagged conditional exports. Wes talked about preload. If those solutions come to fruition within a set timeframe, we could replace conditional exports with something like preload. If that doesn’t work, we’d default to conditional exports. +* Wes: Could we merge flagged conditional exports and kick the can down the road? Sure. Also, having conditional exports w/o require/module guard is valuable. Clarifying: Supporting the schema but not inconsistent resolution within the same runtime. Even okay merging it without a flag. +* Jordan: We can establish rough consensus, even if we don’t move forward with it right now. To Wes: Flagging a resolution feature will be problematic once ecosystem adoption picks up. +* Geoffrey: Is this a question of timeline? +* Jordan: Actively maintained packages will get pushed to adopt it quickly. +* Myles: Independent of usage or issues: The contract is that as long as it’s behind a flag, we can make major changes. Especially if it’s only on stable but not LTS node. +* Guy: Possible restrict flag to require/module, the rest being part of the baseline implementation. +* Myles: Suggesting consensus. Land conditional exports behind additional flag. Unflag ESM with current implementation. Document conditionals, hazard, and how to address the hazard. Set timeline to end of January until we have to find an alternative. +* Wes: How about we make the whole decision in three months? +* Jan: Would it be worse if we spread pkg/esm? +* Myles: Those packages won’t break, so it may be just an incremental improvement. +* Jordan: Suggest adding docs asking people not to publish ESM packages. Questioning value of unflagged state without a solution for dual mode. Is it better to unflag all at once a few months down the line? Maybe shortening the timeline to end of year? +* Guy: The longer we don’t unflag, the more people are trying to adjust but can’t get proper feedback on how well their setups work. With 12 LTS missed, there’s questions about our progress. Would like to unflag as experimental to get more feedback. +* Myles: Worth considering that node 14 is heading for LTS, cut in April. If we want to stabilize in 14, we should have everything unflagged ahead of that branch cut. +* Myles: Any objections to: + - Land conditional exports behind additional flag. + - Document hazard around singleton. + - Unflag modules with that state in node 13. This preserves the experimental warning. + - At the end of January, if there’s no better alternative for dual packages, we will remove the conditional exports flag and turn the feature on by default. +* Wes: Clarifying that we’ll still discuss in a meeting before removing the flag. +Myles: Clarifying that end of January is a deadline. We can make decisions earlier with consensus. +* *no objections* +* Guy: Want to propose addendum - only leave the require/import switch behind the additional flag. Assuming that we can get upstream approval around naming etc. +* Myles: Clarifying that we’ll default to having the entire implementation behind flag if details cannot be worked out. +* *no objections* diff --git a/doc/meetings/2019-11-06.md b/doc/meetings/2019-11-06.md new file mode 100644 index 0000000..54b4748 --- /dev/null +++ b/doc/meetings/2019-11-06.md @@ -0,0 +1,38 @@ +# Node.js Foundation Modules Team Meeting 2019-11-06 + +## Links + +* **Recording**: http://www.youtube.com/watch?v=0uQOL7j2Fz0 +* **GitHub Issue**: https://github.com/nodejs/modules/issues/423 +* **Minutes Google Doc**: https://docs.google.com/document/d/1jmV36-0S8ZH4wXWpXTMfpsc83OsuYh4KCCEzJ2Gu9TM/ + +## Present + +* Myles Borins (@MylesBorins) +* Alex Aubuchon (@A-lxe) +* Wesley Wigham (@weswigham) +* Jordan Harband (@ljharb) +* Guy Bedford (@guybedford) +* Geoffrey Booth (@GeoffreyBooth) + +## Agenda + +## Announcements + +*Extracted from **modules-agenda** labelled issues and pull requests from the **nodejs org** prior to the meeting. + +### nodejs/modules + +* Conditional exports proposal [#401](https://github.com/nodejs/modules/issues/401) + +- Guy Bedford reviews the proposal and explains benefits to ecosystem +- Discussion about semantics and way to move forward +- Conclusion: + - Add back ‘node’ field + - Change flag to `--experimental-conditional-exports`, `--experimental-conditions` or something similar + - Land 🎉 + +* Index field proposal discussion [#422](https://github.com/nodejs/modules/issues/422) + +- Guy introduces the new proposal + - primarily about aesthetics diff --git a/doc/meetings/2019-11-20.md b/doc/meetings/2019-11-20.md new file mode 100644 index 0000000..e366e9b --- /dev/null +++ b/doc/meetings/2019-11-20.md @@ -0,0 +1,104 @@ +# Node.js Foundation Modules Team Meeting 2019-11-20 + +## Links + +* **Recording**: http://www.youtube.com/watch?v=6l55SFZWiQw +* **GitHub Issue**: https://github.com/nodejs/modules/issues/437 +* **Minutes Google Doc**: https://docs.google.com/document/d/1zK7R1qwmAtdf1GzF3EvoUDCX_nI7Eoc_My6HweBdGTs/edit + +## Present + +* Myles Borins (@MylesBorins) +* Rob Palmer (@robpalme) +* Bradley Farias (@bmeck) +* Saleh Abdel Motaal (@SMotaal) +* Guy Bedford (@guybedford) +* Jordan Harband (@ljharb) +* Jan Krems (@jkrems) +* Corey Farrell (@coreyfarrell) +* Evan Plaice (@evanplaice) + +## Agenda + +## Announcements + +* Extracted from **modules-agenda** labelled issues and pull requests from the **nodejs org** prior to the meeting. + +### Group Members + +- Adding Corey Farrell as Observer + - **CONCLUSION:** Approved + +### nodejs/node + +#### esm: The `getPackageType` utility function is not exposed [#30514](https://github.com/nodejs/node/issues/30514) + +- BF: Build tools want to know the format of files inside a package boundary. We could export the `"type"` field via a user-land API queryable by Node. Opinions differ on the request and the API design considerations. CoffeeScript wants `.coffee` files to match whatever the format of a `.js` file is. Want to determine what a `.js` file would be, if it were to exist, so that coffee files can compile to that format. History: coffeescript does not control its build artifacts - asset pipelines do this. Pipeline generates `.js` files - they just want to stay true to that. Comments from TypeScript suggest they like an API design. Consideration: expose the type field or the database mappings in the runtime. I think either works, but have concerns about misuse and forwards compaibility if we do not expose enough. Will lead to disparate systems Example: an issue neglected that `.cjs` doens't populate in the internal ext->format database mapping. Repeated in the thread, when TypeScript want to add file extensions specific to TypeScript, to retain today's experience. This means tools need to cooperate with TypeScript somehow. so this is about coordinating tools. Real world example: ESLint did not have this API, so it attempts to load files in odd ways to get the RC files to load with `--experimental-modules`. We have an old issue in this WG, about exposing configuring the database from package.json files - the issue mentions this. +- BF: Do people think it's ok to expose this machinery? + - MB: An API that would be useful would be one that takes a path and tells you which format Node will interpret that format as. It doesn't necessarily parse the file contents. Is this inline with the proposed API? + - BF: It is discussed in the issue in two forms. Expose the database at a location. Other is going from location to a format. + - JH: Does it have to exist? + - BF: Presumably no. In coffeescript case, the file will not exist. + - JH: Could be tricky if you query for a path that does not exist. Maybe it could return an array of answers. + - MB: Extensionless fallsback to CJS today. So that's the answer. + - CF: CJS will resolve index.js + - JH: API would need to be able to return multiples. + - BF: Does it do searching? + - JH: Which loader does it use? (no need for real file or extension searching - just picks a loader). Or you say "what module format is this?" it's trickier. One world: speculative. Other world: Based on filesystem, but that does not help coffeescript. + - BF: Thread has not discussed resolving to a different location. Only looks at the import loader. + - JK: More important thing is that Guy Bedford is correct - it must be a userland thing. TS and webpack want this. They do not want to hit the filesystem because they have their own abstraction. Gut feeling is we could expose it and get locked into the implementation without helping any of the users, because they need to work with a virtual filesystem. Officially supporting an npm package that does this... maybe? But not useful as a core package. TypeScript would only work on a version of Node that provided it. + - JH: If I build this userland library today, do I need to replicate resolution logic? Could Node expose building blocks for this. + - JK: I assume this is not about resolution. Your path is already resolved. Queries are already fully resolved, even if it did not exist. The thread, Guy's example, it's just encoding the logic of how we interpret the metadata. It's a very small amount of code, returning a small bit of information. The complexity is low - no need for a core API. + - SM: I don't like the speculation when dealing with the filesystem. It will require a lot of coordination and will be biased. Node does two things. Discovers package.json. Then it know resolution approach for extensions. So two APIs. 1. Get the mode. 2. Then resolve using mode. These two parts are stable. I believe this covers most tools' use-cases. + - CF: I'm not sure this is the API that would be useful to me. I want to know the ultimate result of the resolve. Which includes the result of loader hooks. It's more important than what package.json says. + - MB: We have the mechanisms and infrastructure in Node, to look at a file, and decide which transform it goes through. It is consistent and internal. Exposing it seems reasonable. Many people will try to emulate this and may get it wrong. The API "what would we do if we tried to load this path?" sounds reasonable. + - BF: We've talked a lot about "should we expose it?". What about "should we customise the database?" + - GB: I want to stress that the resolve implementation is expected to lead to many userland implementations of the resolve. Hence we have a clear spec. The spec is the source of truth, not the code. So implementation drift is mitigated by this spec. Maybe we need to make the spec more prominent. + - MB: Implementation and Spec are good - but it need tests, just like TC39 does it. We need a test suite. + - GB: Agreed. + - BF: Just because we have a spec, people may not follow it to the letter. Userland extensions happen. New formats TS, JSX, are going to break our spec, because they have no way to coordinate. Implementations will go out of date. You could be running an old version of TypeScript that does not handle the full set today's Node has. HTTP exists as a spec, but it does not mean people want to run multiple uncoordinated implementations. If we do not provide loaders with a way to get the format, we have not met our goal. A spec alone is insufficient. We need a way of coordinating. + - JK: I agree with Bradley. Loaders want to know about currently active extensions. But pre-compilation needs it too. Having implementation in Node, means you need to run it with the same flags as the app. So TypeScript needs it via a different channel. It would not use the API even if it existed. + - SM: We want the logic available ambiently. + - BF: Counterpoint to Jan. If TS does precomp to a supported format, those things are not needed at runtime. + - JK: But is needed at compile-time. + - BF: That is true for loaders. + - JK: *will take this offline* +- What considerations do we have about exposing the raw field? +- How do people feel about exposing the ability for users to configure these databases of extension->format (not necessarily via package.json). Then we can figure out how to configure it. +- **CONCLUSION:** Keep talking on the Github issue + + +### nodejs/modules + +#### Module Attributes RFC [#427](https://github.com/nodejs/modules/issues/427) + +- MB: Was raised at TC39 in response to WHATWG/W3C feedback at TPAC where JSON modules were rejected due to security concerns. Importing a JSON file Node will parse during fetch and will return a default value. Security concern was that you relied on the server declaring the correct MIME type. Server could maliciously return JS instead of JSON - turns data into executable code. Issue was kicked back to committees. This is an in-band solution. Using extra syntax to assert the type of the module being loaded. +- JH: It's premature to discuss here prior to TC39. I do not think syntax change will get consensus there. I do not buy the threat model - if you don't trust it, don't import it. Same thing as pulling JS from a different domain - good JS can be swapped to bad JS. I think Node and Browser concerns are different here. Node should not degrade UX. +- BF: Minor correction - this is not specifying the type of the import, it's restricting the allowable types. I am uncomfortable saying we should match the web's security model. I am unclear how this affects Node and why we should do it. The unspoken concern: if the web requires this, Node must support it (or ignore them) to retain web compatibility. I think both outcomes are problematic. I have stated concerns on the repo. I do not think the threats are realistic. Only minor games - one level deep import from an untrusted origin. Other things can expand the dependency graph. Wasm and CSS have issues. These seem to be an odd solution to a problem that Node does not have. Maybe even web is not solving the problem they want to solve. We must decide: can we implement it? and should we implement this? I think we can but should not. The threat model does not apply to Node. +- MB: Jordan, you said it's premature. I disagree - we need feedback from this group to take to TC39. Brad says, this could cause Node/Browser incompatibility. Don' agree with Brad that this has nothing to do with Node - we may start loading from HTTP. Then the security becomes similar. If we want JSON modules on the web, we need a palpable solution for the browsers. We must come up with something. Otherwise we agree to drift, or just live with no JSON modules on the web. +- JK: I am concerned about the current direction of the proposal. Type assertions are bad, even on the web. The use-cases outlined are questionable. Protecting everything *except* JavaScript seems weird. A basic assertion to say "this is just static data" seems better. Listing allowable file types does not seem like it will get much take-up. +- BF: Be careful about talking about HTTPS on Node. Deno did this but they have a different sandbox model. Node could not use the Web's model. Security Working Group has discussed this. +- JK: It's not about origins. Just about saying file in non-executable. +- BF: Trusted origins is a big part of this proposal. +- JK: This proposal says you need to annotate the untrusted ones. It doesn't even rely on the web security model. +- BF: Do we have implementation concerns? +- SM: Where would we implement this? +- BF: A format check at the end of the load operation. Parse time ingestion, passed through host hooks. +- JK: A place where this is problematic is the CJS bridge. We cannot forward this info easily. We'd treat all CJS as modules. That's the only implementation concern. +- MB: Jordan, you do not believe the proposal has legs? +- JH: I do not expect to be a lone objector on this. The entire design of the import syntax and the security model of the web is just not going to fit together here. We can talk about it more independently. I would not block it as a lone objector. +- MB: As a group, we should think about out-of-band alternative solutions. +- **CONCLUSION:** It's technically implementable. Open question as to whether we should. + +**END OF MEETING** + +--- + +#### Chartering the Modules team [#412](https://github.com/nodejs/modules/issues/412) + +- Not discussed + +#### Loader Hooks [#351](https://github.com/nodejs/modules/issues/351) + +- Not discussed + diff --git a/doc/meetings/2019-12-18.md b/doc/meetings/2019-12-18.md new file mode 100644 index 0000000..f68a976 --- /dev/null +++ b/doc/meetings/2019-12-18.md @@ -0,0 +1,116 @@ +# Node.js Foundation Modules Team Meeting 2019-12-18 + +## Links + +* **Recording**: http://www.youtube.com/watch?v=3g5Og5w1iV0 +* **GitHub Issue**: https://github.com/nodejs/modules/issues/455 +* **Minutes Google Doc**: https://docs.google.com/document/d/107-EWDmaUN5XCHNzme6h5AYfl1L3J-ZbgZSq7bqWC8E/edit + +## Present + +* Myles Borins (@MylesBorins) +* Jan Krems (@jkrems) +* Bradley Farias (@bmeck) +* Rob Palmer (@robpalme) +* Guy Bedford (@guybedford) +* Saleh Abdel Motaal (@SMotaal) +* Wes Wigham +* Geoffrey Booth (@GeoffreyBooth) +* Corey Farrell + +## Agenda + +## Announcements + +*Extracted from **modules-agenda** labelled issues and pull requests from the **nodejs org** prior to the meeting. + +### Conditional Exports + +* Refs: + * Conditional exports naming usability discussion [#452](https://github.com/nodejs/modules/issues/452) + * Implement logical conditional exports ordering [#31008](https://github.com/nodejs/node/pull/31008) + * Stabilizing the resolver [#451](https://github.com/nodejs/modules/issues/451) + +* GB: Ironing out usability. It’s best to merge with the next two agenda items (Implement logical conditional exports ordering + Stabilizing the resolver) +* GB: Resolver has hidden priority list - it will match the default condition last. If default is at the top, it doesn’t matter. Order did not matter. now `require` and `import` are on the same footing and mutually exclusive. Scaling exports (production conditions etc), as this list grows, the order becomes confusing. Potentially your environment matches multiple. So instead of relying on the resolver, we can give the user direct control by using object-order (defined in ES2015). Allows falling back if target is invalid, e.g. value is an empty array. Tobias had a nice example where you think of it as if-statements. As a programmer who writes if-statements, this naturally matches - seems nice. +* BF: JSON doesn’t assert an ordering. There are package utils that sort your package.json keys alphabetically. We need to be aware of this. Other languages - C# marshalling libraries do ordering based on target class, not the order inside of the JSON. Not sure if this is blocking. We should probably see if anybody big is doing this. +* MB: On ordering, keys with numbers could cause issues. So don’t allow keys that start with numbers. Seems reasonable-ish. +* JK: In Java, key-value structures, Jepson preserves key order. +* MB: Any objections to following ordering? +* BF: I want to do some research - no objection to the behaviour. +* GBO: Agree with Brad. Even if some things re-order them, is this preferable to current magic in the resolver? It would be no different to how it is now where you have to reimplement logic. It doesn’t make anything worse. +* BF: My concern with sorting… if we find something problematic… we could just say it’s alphabetically sorted (not package.json source text order). That would make it reliable. +* SAM: Json is a string. So a small patch to determine the order after parsing using a library, there are easy ways to use this. +* GB: These are the two things I want to do before shipping. #31009 is a bug. Self-resolve hits compatibility. If we add package name resolution to CJS, that is theoretically a breaking change. If we just ship self-resolve that intercepts the node_modules lookup and redirects to your own package, it’s breaking. So Jan made sure this applies after node_modules lookup so self-resolve only succeeds if it would have failed otherwise. My concern for implementing in ESM is it takes legacy into the future. I prefer better encapsulation, e.g. always load the local package. This PR says CJS will have the node_modules behaviour, ESM will have the encapsulated behaviour. +* JK: Import map bloat. We discussed opt-in in package.json. “I do want self-resolve”. Makes it safe for both +* MB: We did work to sync the resolver algorithms. This feels not aligned. Sigil seems good. We could add it later. I am personally uncomfortable with - deviating between CJS and ESM. +* GB: A major benefit of using the public name is that in a monorepo, others will use the same name, allowing import map to be de-duped. Sigil does not have - this advantage. +* MB: We should keep self-reference by name, but do it as last resolution. Sigil could be used as first resolution. +* CF: What happens if a transitive dep installs another version of the self package. Npm’s dedupe could cause another copy to land there. +* GBO: I like Jan’s suggestion of package.json opt-in, e.g. “exports”. +* CF: Could it work only if you have a direct dependency? +* MB: Self-resolve doesn’t even hit node_modules at all - it sticks within your package. +* GBO: That could have been a compromise? +* GB: I want to say self-resolve is enabled by “exports”. I need to think about it more, especially considering backporting. Might not be semver major. +* GB:Any naming concerns on conditional exports? +* *silence* +* GB: Ordering - we must ban keys that begin with numbers. Still need to talk more about self-referencing. +* GB: Stabilizing the resolver. My priorities: once we’ve got the above, we’re pretty good. Later, maybe refine error messages and validations. The imports feature is in Jan’s original spec - we could consider adding later. Benefit of self-resolve: allows internal conditions (private conditions). + +### unflagging in 12.x LTS [#450](https://github.com/nodejs/modules/issues/450) + +* MB: I want to propose: assume conditional exports and named exports lands in January. We backport everything to Node 12. V8 7.8 is not on 12. So just wait. Goes out in Jan behind a flag. Remove the flag in April. We can update it monthly. Concerns? 13 is EOL in April. We can land breaking changes to an experimental API at anytime. HTTP2 is prior art. Just must avoid semver major breaking changes to stable APIs. I will take on the first pass on the backports. + +### Unflagging conditional exports [#31001](https://github.com/nodejs/node/pull/31001) + +* MB: If we don’t have consensus on require(ESM) by January, I think we should unflag conditional exports so we have time to backport it. + + +### Unflagging experimental resolve self [#31002](https://github.com/nodejs/node/pull/31002) + +* MB: Similarly, for unflagging this the same timeline of January - potentially do this sooner. I will follow up with release time to push out the minor. Deadline is two weeks before the minor. Release plan for 12.13.2 should go out yesterday, but it did not. My guess is they slip by a week or two. So 7 Jan or 14 Jan is the cutoff - release team decide. +* GBO: require(ESM) is unlikely to land by Jan deadline - but maybe by April. +* MB: Jan 8 is the prudent date to meet. Can we do a call then? +* *silence* +* GBO: Anyone not available? +* *silence* +* MB: Will send an invite for an out-of-band meeting for Jan 8. + +### Loader Hooks [#351](https://github.com/nodejs/modules/issues/351) + +* BF: Geoffrey introduced two hooks we haven’t talked about much. Framed about it being additive. I don’t think we are aiming to do bigger refactors currently. It’s not blocking a refactor in future. That’s it. + +### `require` of ESM [#30891](https://github.com/nodejs/node/pull/30891) + +* WW: The first PR is the maximal one that could be produced. Highest compatibility. Allowed all ESM to execute as intended. So that we only walk back the parts people had issue with. On the PR, people don’t like event loop turns during what looks like a function call. Fair enough. But we don’t have to do that. We don’t have to allow sync execution of an async function. Browsers are considering banned Top Level Await - most modules don’t need it. So simpler solution is to synchronously unwrap resolved modules. It breaks the Promise spec, but TLA already does this. The way modules use it breaks it - you assume event loop turns between every module. It does not because the promises are synchronously unwrapped. A compromise so synchronous graphs stay synchronous. +* GB: The TLA spec executes modules in a synchronous way. It’s not promise based. It’s fully sync and only becomes async and promise-using if something needs it. If CommonJS were to load an async module, it would only find out at runtime. You’re suggesting it would throw here? +* WW: Two options. WHATWG spec machinery can premark a graph as containing async to make it an early error. Or just cheat a little and unwrap the promise. Then either return the promise or throw. +* GB: For consistency, you don’t want ESM loads to trigger re-processing. +* WW: You still continue the execution but the CJS require throws. Or you just return a promise from require(). +* BF: The PR intent was to allow things that look like ESM (but with different semantics) to work. We could do an early error without shenanigans. +* WW: I claim no breakage. The runtime can go beyond the spec. For ease of working, we just define it to always return a promise. +* BF: An early error seems non-objectionable. +* WW: WHATWG variant flags the graph as containing async. +* MB: I like the idea of failing early. This would preclude requiring WASM modules too. +* WW: The error message can be precise here. +* MB: Is there still multiple event loops? +* WW: No - I removed the inner event loop. +* GB: The other concern: loader hooks are async. Unwrapping promises includes unwrapping loader hook promises. +* WW: We wanted to move loaders off the main realm. By shuffling objects in a shared buffer with atomics.wait these are synchronous. +* GB: In those APIs, every resolve/source/translate must be fully synchronous. If we want HTTP loaders, the whole thread would be blocked whilst fetching. +* WW: We could promise.all it. Requires some re-design. Loaders must execute before the graph load. +* GB: With dynamic import this happens all the time and introduces pausing. +* WW: Or you just throw an error when you hit an async custom loader. Loaders interleaving with your program is scary to me. The third option: make all the loaders async with a sync adapter in front of it. +* MB: Could do an out of band meeting? Or discuss on the issue. + +**END OF MEETING** + +## NOT DISCUSSED + +### Patch/Instrument a module [#339](https://github.com/nodejs/modules/issues/339) + +* esm: The `getPackageType` utility function is not exposed [#30514](https://github.com/nodejs/node/issues/30514) + +### Chartering the Modules team [#412](https://github.com/nodejs/modules/issues/412) + +* 5 minute timebox diff --git a/doc/meetings/2020-01-08.md b/doc/meetings/2020-01-08.md new file mode 100644 index 0000000..8273613 --- /dev/null +++ b/doc/meetings/2020-01-08.md @@ -0,0 +1,126 @@ +# Node.js Foundation Modules Team Meeting 2020-01-08 + +## Links + +* **Recording**: +* **GitHub Issue**: https://github.com/nodejs/modules/issues/461 +* **Minutes Google Doc**: https://docs.google.com/document/d/1RwdeqHV9oJBBafcAajWZQ0eCnzcMNSo-dQGN1WeEszY/ + +## Present + +* Myles Borins (@MylesBorins) +* Bradley Farias(@bmeck) +* Guy Bedford (@guybedford) _GB_ +* Rob Palmer (@robpalme) +* Jan Krems (@jkrems) +* Saleh Abdel Motaal (@SMotaal) +* Corey Farrell (@coreyfarrell) +* Geoffrey Booth (@GeoffreyBooth) _GBB_ + + +## Agenda + +## Announcements + +* Extracted from **modules-agenda** labelled issues and pull requests from the **nodejs org** prior to the meeting. + +### nodejs/node + +#### module: implement logical conditional exports ordering [#31008](https://github.com/nodejs/node/pull/31008) + +* GB: We have a PR up for a big change to the way exports are matched. Current implementation is based on a hidden list of export conditions in the resolver. When resolve looks at conditions, it would match them in its internal order. Concern was: if user is using many conditions, the priority order is hidden. So resolvers could behave in different ways. Hard to determine what will match. PR switches to "object order" as defined by package author. Also, for nested conditions it will fallback - directly think of them as acting like `if` statements. Hopefully this is more predictable. Control is back in the hands of the package author. This PR has been merged to Node master. +* BF: Please confirm the previous questions about JSON order and relation to ecosystem tooling. It's a well-defined ordering. We have found no ecosystem issues. `OrdinaryOwnPropertyKeys` is the official algorithm. +* GB: Gus requested array indexes do not follow the same ordering. Numbers come first, then insertion ordering. So userland tooling could get this wrong. The PR addresses this by adding validation, to ensure there are NO array indexes in the object, otherwise throw an error. It's fully well-defined and aligned with the spec. Node achieved consensus on this. I feel `exports` is now feature complete. It's really nice. The only outstanding PR is unflagging. + + +#### module: unflag resolve self [#31002](https://github.com/nodejs/node/pull/31002) + +* MB: Landed. Released in 13.6. +* JK: Congrats to Guy Bedford for making this happen. +* GB: It's backwards compatible with CJS. We compromised by saying the feature is only active when `exports` exists in `package.json`. It's another justification for preferring `exports`. + + +#### module: unflag conditional exports [#31001](https://github.com/nodejs/node/pull/31001) + +* MB: Logical Conditional Exports landed. So this follows on. We considered require(ESM) as an alternative. We think one does not preclude the other. I am in support of landing this unflagging. +* GB: I am happy with where we have got to. +* MB: About to have 3 upstream LGTMs. Just need to run CI. Any objections to landing this today subject to green CI? +* JK: I think yes. My memory is that we were supposed to discuss... I feel bad making this call when Wes is not here. Doesn't feel great to call it now. Would prefer his awareness. +* MB: We're only talking about unflagging, so it seems reasonable to wait. +* BF: How about we reach consensus here and then wait on Wes for a reply? I would like to get consensus here to avoid back and forth meetings. +* MB: Any concerns with the Wes reach out approach? +* GB: Hope we don't wait indefinitely. Waiting one week seems sensible. +* MB: Ok, let's wait until next meeting for his response. +* GB: Sure. +* MB: I don't think this affects release timeline for backports to 12. It won't change behaviour - just unflagging. +* GB: That still affects people's ability to publish dual-mode packages. +* MB: Ok. + +#### esm: implement import.meta.resolve [#31032](https://github.com/nodejs/node/pull/31032) + +* MB: We are trying to align with WHATWG implmentation. +* GB: Browsers have discussed for a while. In Node, this is the equivalent of require.resolve. This is an important convenience feature. The PR calls the resolver, including hooks. You can provide many types of specifier. Relative, package name, etc. Useful for asset workflows. It is **async**. We chatted about this. Async feels safer. Allows us to syncify in future. A hazy argument for this being a slightly safer path. In Browser, it must be async for a few reasons, e.g. dynamic import map injection. I sent the PR to Hyrishi and Domenic. They agreed async could be alignable. Hard to get their final position because browsers may change. Do we want to take the leap first? Or be conservative and follow the browser. I would like to land. But will understand if others do not want to land. +* BF: This is our first meeting on this topic. I am not comfy landing this unflagged for a while probably. There are things happening in the Modules Group that may need to hook into this. We don't have a way to pass in conditions, but we are using them for logical ordering. There's also TC39 module attributes happening. Lots is happening. This here is too simple. In future we wil need to replace it to have the utility of a resolver. We need a much more feature rich API later no matter what. +* MB: I think this is premature to land unflagged. Before implementation, we can work with WHATWG and HTML on spec text. I am concerned about unflagging before we have spec text and clear consensus across groups. +* GB: On interactions with conditions, conditions are environment-specific constants of the environment. A generic resolver would need many things - a super resolver. That would be very different to this. One is a library. The other is the local environment resolver. It's simpler. This import feature matches the current environment. On module attributes relating to resolution... +* BF: They affect allowable forms. +* GB: I think it's dangerous to have more metadata to affect resolution. This is one of the few situations where it can just ship things and then follow up with breaking changes later. This is one of the only cases where Node.js is in this kind of position. It may even steer browsers and accelerate things. It makes it easy for Browser to follow us. Otherwise we risk losing a year. +* MB: Breaks in a flagged or unflagged? +* GB: Unflagged. +* MB: I don't think that's acceptable. +* BF: We're discussing interop and breakage. Maybe just don't put it on import.meta. We made createRequire. We could make createResolver. This PR allows you to specify another URL to resolve against. You can impersonate URLs. So it may be ok to say "you have to import this". +* MB: If we make this an experimental API on Module, that appeases my concern. If we make it a slightly lower API, it could become part of the official import.meta later. +* GB: I have needed to use asset references. import.meta is statically analyzable. Most build tools don't do the work to analyze imported stuff. The kind of breaks we might expect are "we'll support the API, but not the second argument, and it's sync" - this would only break 10-20% of users. +* MB: We could do both. Keep it flagged on import and unflagged on the lower-level. +* GB: Agreed. +* MB: I want to encourage experimentation and reaching out to WHATWG. Want to avoid mass breaks. +* GB: I can flag the PR. + +#### WIP: Move ESM loaders to worker thread[#31229](https://github.com/nodejs/node/pull/31229) + +* BF: We have previously discussed moving loaders to other realms and threads. This PR has a long way to go. We can base discuss around this. Hooks are moved into a separate thread - but no API change except it removes dynamicInstantiate because that is tied to the Realm. This PR is a way for us to discuss having them on a separate thread. Instead of each thread getting their own Loader instance, instead the thread is shared amongst all works + the main thread. This is a new type of thread - an internal work. So it is shared and can receive requests from all threads. We are probably going to need add hooks. We can't intercept the --require flag. We don't have a clean way to instrument globals. We'll need to do that. I want this PR to be blocked on all those other landable things. I want to say "in order to land this PR, we need xyz feature" then we raise a PR for the feature and block this PR on that PR. +* CF: If this PR is open for a while, does it make sense to separate out dynamicInstantiate? So that feature can get removed before it gets popular. +* BF: It's possible, but I do not see large benefit. For GetSource we have an awkward API signature. The resolve can still return a separate format. We can mark it as "soon to be removed" but I don't see removing it helping users right now. +* CF: Then let's document it that way to let people know. +* BF: It's already in use. People use it to avoid weird source code transform. When they get the GetSource hook we'll see if they transition over. +* JK: I agree with Corey. Would prefer to remove it sooner or at least add a deprecation warning. Also, do we have quorum on moving to an isolated context? We could make a call on this now. +* GBB: Putting each loader in a separate thread... I assume we can still sequence loaders. Babel->CoffeeScript. What does this PR preclude? +* BF: The big things are the isolation. Today, if you create a global property, you can see it in user code too - and can interact between app and loader. With this PR, that interaction is not possible. Leaving you to perform message passing. Sharing globals is probably not a good thing to do. Today we spin up 5 coffeescript loaders if you have 5 threads, and they may need to coordinate. +* GBB: Test runners? +* BF: We do not have a good solution to test runners. The bigger problem for utilities is the module cache - we do not expose it in any form. +* JK: Do we have consensus on moving loaders to a separate context from user code? This is the highest order decision. +* BF: You can only share the loader if the loader is isolated from the app. +* GB: Worker threads? +* BF: It creates a new message channel to the same internal worker that hosts the loader. The browser has a similar concept. + +### nodejs/modules + +#### Chartering the Modules team [#412](https://github.com/nodejs/modules/issues/412) + +* MB: We have never discussed this in a meeting. My intuition is that we have everyone we need to charter, aside from small bits. And if we do charter, what do we want authority over. +* GBB: For the big releases, we had to break our own rules or thought about dissolving the group. If we charter we can't do that. +* MB: We could decharter in that case. Right now, we are acting as though we are chartered. Suggesting we should charter. Maybe we don't need to dig into this. Would be good for people to chime in. +* GB: What are the benefits of chartering? If there are no risks, go for it. +* MB: Being officially recognised from a process perspective would allow members to block things upstream. Could stop the tsc unilaterally making changes without us chiming in. Chartering process is about setting expectation. My previous experience has been with the Release WG which had external work not inside core. Matteo Collina might have thoughts for us. + +#### Loader Hooks [#351](https://github.com/nodejs/modules/issues/351) + +* GBB: I worked on this with Brad. There's more to be done. Check out latest master. There are four hooks. Resolve. GetFormat. GetSource. TransformSource. I have not reviewed the long list of loaders. There were concerns from Guy and Bradley. The biggest outstanding thing is test runners and stubs. If people have ideas, please share. +* BF: Previously people did not want multiple loaders. +* GBB: Does the thread change impact things? +* BF: Just make sure the interchange between the hooks is JSON-serializable. You don't have to worry too much. +* CF: On multiple loaders, I do think there needs to be a way to accomplish this. I am concerned that solving this in userspace might be problematic. So I would like to see more discussion on this. +* JK: I am skeptical about multiple loaders. Because the more complex scenarios, where loaders compete to affect the same things, generates weird scenarios where the order of running the hooks becomes too complex. I am nervous about this. +* GBB: Think of Gulp. gulp.parallel. gulp.series. Having Node do this... Either it's simplistic, or userland will do something to orchestrate multiple loaders. So how sophisticated should Node be? Don't want to compete with Gulp. + +--- + +**END OF MEETING** + +--- + +## NOT DISCUSSED + +* Stabilizing the resolver [#451](https://github.com/nodejs/modules/issues/451) +* unflagging in 12.x LTS [#450](https://github.com/nodejs/modules/issues/450) +* Patch/Instrument a module [#339](https://github.com/nodejs/modules/issues/339) diff --git a/doc/meetings/2020-01-15.md b/doc/meetings/2020-01-15.md new file mode 100644 index 0000000..dd67b62 --- /dev/null +++ b/doc/meetings/2020-01-15.md @@ -0,0 +1,135 @@ +# Node.js Foundation Modules Team Meeting 2020-01-15 + +## Links + +* **Recording**: https://www.youtube.com/watch?v=SuFC_yzc51A +* **GitHub Issue**: https://github.com/nodejs/modules/issues/466 +* **Minutes Google Doc**: https://docs.google.com/document/d/1odxHwuFIuZsbaQ9SgH23riGiwvKg0y2pHlTwdaIOIbE/edit + +## Present + +* Jan Krems (@jkrems) +* Jordan Harband (@ljharb) +* Myles Borins (@MylesBorins) +* Guy Bedford (@guybedford) _GB_ +* Geoffrey Booth (@GeoffreyBooth) _GFB_ +* Wesley Wigham (@weswigham) +* Corey Farrell (@coreyfarrell) +* Evan Plaice (@evanplaice) + +## Agenda + +## Announcements + +*Extracted from **modules-agenda** labelled issues and pull requests from the **nodejs org** prior to the meeting. + +### nodejs/node + +#### WIP: Move ESM loaders to worker thread [#31229](https://github.com/nodejs/node/pull/31229) + +* MB: Anything important to add? Not time sensitive to discuss. +* CF: Did we decide on this direction? If so, should we prepare / remove dynamicInstantiate? +* MB: My understanding: Was always meant to move to worker threads. If there’s things not compatible with that, we should act on it. +* WW: Not at odds with having synchronous module load, if anything it helps. + +#### esm: implement import.meta.resolve [#31032](https://github.com/nodejs/node/pull/31032) + +* GB: Last was people were open to shipping it flagged. +* MB: Chatted with Domenic about this. Brought up `import(await resolve(specifier))` as a potential concern. Import maps are 1:1, maps URL on the left to URL on the right. As specified it’s not recursive, I think it should be. Adds complexities around cycles. Need to resolve those concerns. Overall Domenic was interested in node going forward with this, possibly start a spec in WICG. No concerns about a flagged implementation landing, can act as reference implementation. +* GB: Second argument allows customizing parent, async. Any API concerns? +* JK: Some concerns about how this relates to loaders and idempotency vs. isolation. +* MB: Objections to moving remaining discussion to Github? +* *(No objections)* + +#### module: unflag conditional exports [#31001](https://github.com/nodejs/node/pull/31001) + +* GB: Discussed conditional ordering in the past. Right now on 13 exports are shipped. Objects always pick the default key. +* MB: Want to make sure we can unflag conditional exports. +* WW: Voiced concerns in the past. Encourage shipping two copies in packages duplicates dependency trees. Bad transition path during migrations. We can do require(esm) independently of this change. +* MB: Dual specifier hazard definitely still less than ideal. +* WW: Not looking forward to people accidentally including react twice and using a mix. +* JH: This assumes bad setup in react that doesn’t take conditional exports into account? +* WW: Applies to any project that deals with current ESM tooling (rollp, webpack). +* JH: Projects will have to adjust their approach now that native ESM is a thing. +* MB: Same kind of dual instancing can appear without this feature. Also, all of this is still experimental. If require(esm) lands in the future, this could relieve the need for using this pattern. Many reports of issues with this, we could still remove the behavior even though very painful. +* WW: Node compat usually means after unflag nothing changes. +* MB: While experimental, both workers and http/2 support had dramatic changes after unflagging. +* WW: Resolver and module system have stronger ecosystem ties. +* MB: Does anybody object to unflagging? +* *(No objection)* +* MB: We can still make adjustments before April. This includes changes to the resolver. +* WW: How closely are we working with npm, e.g. around their plans for peer deps? +* MB: We have a channel with them. +* JH: Also interested in discussing it? +* WW: Skipping to charter - are we custodians of code in core? Or also including ecosystem questions? +* MB: Let’s punt for chartering discussion. There’s a team focussed on package maintenance. Scope could be as large as we want it to be, as long as TSC signs off on it. + +### nodejs/modules + +#### unflagging in 12.x LTS [#450](https://github.com/nodejs/modules/issues/450) + +* MB: We can drop from agenda if people don’t see problems. Targos got V8 7.8 against node 12 tested and landed. There’s now a [PR for next 12.x minor](https://github.com/nodejs/node/pull/31368). Got module loader on there over the weekend. Primary difference is internals, external behavior should be the same. Exception is unflagging. Everything (exports, resolve self) behind --experimental-modules. People should test that it doesn’t do new-modules-stuff without the flag, does all the things with the flag. We can update implementation until April, then flip the flag. +* JH: If we add new features, e.g. sigil for self-reference. If it lands on 13, can it be backported? +* MB: Yes, unless they depend on V8 features. Synthetic modules in V8 were primary reason that backporting of features was hard. We can make changes, including breaking, for flagged implementation in patch releases. What we have in the roadmap, should be backportable. And we really want to backport asap to de-risk April unflagging. +* JH: Is backporting `exports` for CommonJS to 10 something we want? +* MB: Likely not, would take a lot of resources. +* JH: Even if showed up with a magical PR? +* MB: If it introduces instability, may be better to not risk it. +* JH: Package maintainers hesitant to adopt exports if they want to support 10.x until EOL. People need to use rollup etc to hide internal files if they want to get encapsulation. +* MB: Good point, April maintenance makes it awkward to actually get it in. +* WW: Could a polyfill be used? +* JH: Already looking into polyfill, alternative for getting it backported. +* MB: Gut feeling is changing resolver this late into +* CF: Worked on `import()` fixes, are they going to be in 12.x backport? +* MB: Will verify. +* GB: Is loader work part of a backport? +* MB: Will run branchdiff, everything not labelled to prevent backport should have made it as long as they have made it into a 13.x release. +* JK: Will it be okay to change loader flag name during unflag in April? +* MB: Should be fine. Need to make sure that we leave silent aliases for flags we remove. We did this for `--experimental-specifier-resolution`. + +*Some time later.* + +* MB: Follow-up, branch diff: https://gist.github.com/MylesBorins/2157cd56e9cf17068c4d77d539edb6f2 +* GB: Last few PRs are landing. +* MB: List only includes 13.x commits, expected. + +#### ERR_REQUIRE_ESM LTS backport [#468](https://github.com/nodejs/modules/issues/468) + +* GB: In the CJS resolver, we have all new behavior flagged. We wanted to make `require(.mjs)` or `require(.js)` in `type:module` both error. Should we make these warnings? Only time when this comes up is when an early adopter is mixing file types. Might be worth to accept bug reports coming out of this. +* MB: Any action item? +* GB: Originally thought it was breaking a valid use case. But can’t repro now. +* WW: They opted into an experimental feature if they run into this error. + +#### Chartering the Modules team [#412](https://github.com/nodejs/modules/issues/412) + +* MB: 8 minutes are not a lot of time. Future meeting should maybe be 30 minutes of just discussing this? +* GBB: Yes, may be good after some of the more urgent items are sorted out. +* JK: Should we make the proposal more concrete before the discussion? Move conversation about scope on Github? +* MB: Can somebody pair on a high-level items to be done for chartering? +* JK: Volunteered. +* JH: Volunteered. +* GBB: Volunteered. +* MB: Will send out Doodle with times that work on calendars I can see (Jan & myself). + +#### Table source type detection [#465](https://github.com/nodejs/modules/pull/465) + +* GBB: Seems to reflect status quo? +* MB: May still want API to tell package type. +* GBB: Different API, this is about source type-based detection. +* GBB: This and conditional exports are last items before retiring the roadmap. Can more to other channels. +* MB: Any objections to landing this roadmap change? +* (No objections) +* MB: There’s a list of things we want to keep working on (package type). We should track those somewhere. +* GBB: Also includes feature gaps in loader hooks. + +#### Stabilizing the resolver [#451](https://github.com/nodejs/modules/issues/451) + +*Tabled.* + +#### Loader Hooks [#351](https://github.com/nodejs/modules/issues/351) + +*Tabled.* + +#### Patch/Instrument a module [#339](https://github.com/nodejs/modules/issues/339) + +*Tabled.* diff --git a/doc/meetings/2020-02-12.md b/doc/meetings/2020-02-12.md new file mode 100644 index 0000000..3086215 --- /dev/null +++ b/doc/meetings/2020-02-12.md @@ -0,0 +1,102 @@ +# Node.js Foundation Modules Team Meeting 2020-02-12 + +## Links + +* **Recording**: http://www.youtube.com/watch?v=n9CpwXe5nHE +* **GitHub Issue**: https://github.com/nodejs/modules/issues/483 +* **Minutes Google Doc**: https://docs.google.com/document/d/1HgbR0o38ALVCxAoNdv2IRpIfTgxP_X1n3lwCBdEBupI/edit + +## Present + +* Modules team: @nodejs/modules +* Jan Krems (@jkrems) +* Saleh Abdel Motaal (@SMotaal) +* Wesley Wigham (@weswigham) +* Geoffrey Booth (@GeoffreyBooth) +* Corey Farrell (@coreyfarrell) + +## Agenda + +## Announcements + +*Extracted from **modules-agenda** labelled issues and pull requests from the **nodejs org** prior to the meeting. + +### nodejs/node + +#### Self ref bug by Jordan + +* self import works by accident in 12.16 without flag, need fix PR +* Guy is working on a fix + +#### module: disable exports encapsulation for CJS [#31718] + + +* [GB] Based on an incident with Preact. They enabled exports for conditional branching but they broke existing (CJS) consumers so they disabled encapsulation via ./ export. +* [GB] Right now opt-out is for both require and import. Risk is that all people will disable because of CJS consumers. Opt-out is `"./": { "require": "./" }`. +* [MB] Document opt-out and make sure we warn about downsides of opt-out. +* [WW] Adding explicit exports should be documented as being preferred. +* [JK] Medium-term tools may add exports support and only opting out for require will stop being a solution. It will bring existing ESM consumers that will hit the whitelisting. +* [CF] Conditional opt-out as above will trigger the experimental warning. +* [GB] Should we remove the conditional exports warning? +* [MB] I’m fine with warning, not a huge problem in practice. Unconditional opt-out is likely preferred. Conditional opt-out may be confusing to consumers +* [WW] People should accept that exports encapsulation is potentially breaking +* [GB] What Preact ships is conditional exports for import vs require which may trigger warnings +* [MB] We should encourage them to switch to default. +* [GB] What do people think about removing the warning? +* [JK] It feels weird to ask people to work around the warning by choosing a very specific subset of possible conditional exports forms that happens to trick the warning code. +* [GBB] Could we make the warning depend on published vs not? +* [MB] Issuing the warning to authors only may be possible. We could set up a timing to remove or reduce both flags and warnings. For example in a month or two. +* [GB] Will open issue to discuss handling of warnings going forward. + +#### module: Refine and unify exports resolution error handling [#31625](https://github.com/nodejs/node/pull/31625) + +* [GB] Main problem tackled here: Conditional fallback behavior only applies to generally valid values, not for “obviously” bad configuration like invalid URLs. +* [WW] From a ideal/purity perspective: Does path not exported leak the existence of a guarded entry? +* [GB] Both not configured at all and configured but conditional not matched is the same error (path not exported). Does not leak file system existence for example. Any thoughts on splitting `MODULE_NOT_FOUND` based on likely cause? +* [WW] Would prefer not to expose error codes but node makes that impossible. +* [MB] Error messages are flexible. Important that shared error codes are semantically close enough. +* [GB] We could try to keep `MODULE_NOT_FOUND` but we already have other forked error codes. +* [WW] Do we expect people to switch on it? +* [GB] In node generally, the error code and message go together. We could use module not found and split with different messages. +* [WW] Mostly a matter of intent: Do we want people to switch on error codes? +* [JK] People are already catching module not found, it’s a feature that existing catch blocks don’t catch these new conditions. +* [GBB] Mostly care that error messages are helpful. +* [GB] Agreed that optional dependency catch blocks shouldn’t accidentally catch encapsulation errors. A lot of this is also based on use in `import.meta.resolve` where having a more fine-grained feedback channel can be valuable. +* [GB] Any objections? Would love more feedback. +* *No objections.* + +### nodejs/modules + +#### Docs/review of various affects of type field [#475](https://github.com/nodejs/modules/issues/475) + +* [MB] Removing from agenda, adding help wanted label + +#### unflagging in 12.x LTS [#450](https://github.com/nodejs/modules/issues/450) + +* [MB] Dropping from agenda +* [GBB] How do warnings work in 14 around that time? +* [MB] Separate topic, 14 may be a good time to do it. +* [GBB] Doesn’t have to correlate with 14.0.0? +* [MB] Correct. + +#### Chartering the Modules team [#412](https://github.com/nodejs/modules/issues/412) + +* [MB] Fine without doing it. Have a majority in place if we want to ask for chartering. +* [GBB] Maybe just move forward with this? Most things are unflagged. +* [MB] We can make a request and unless somebody objects in a week, we can ask the TSC. Any objections? +* *No objection* + +#### Loader Hooks [#351](https://github.com/nodejs/modules/issues/351) + +* [GBB] There were ideas for a redesign by Guy and Bradley but not movement yet. One open question was chaining loaders. If we want dramatic changes, we should do it soon. +* [GB] Worth considering if the API that we want is breaking. If somebody wants to drive, it could be done. Should go back to use cases and look at requirements. Do we know what the status is? +* [GBB] Move off-thread was another big issue. WASM and extension-less files are hard to support with current order. +* [JK] HTTP loaders are another example where combining format+source is very useful. +* [GBB] Should start with use cases and make sure our APIs support them properly. +* [JK] Likely need a new hook for global init that runs code on the main thread. + +#### Patch/Instrument a module [#339](https://github.com/nodejs/modules/issues/339) + +* [MB] Should we remove this and roll this into loader hooks? +* [WW] Would like to keep, it seems distinct enough. +* [GBB] Need to do research on how to change the active code of modules. Maybe something in the debug protocol? diff --git a/doc/meetings/2020-02-26.md b/doc/meetings/2020-02-26.md new file mode 100644 index 0000000..b558313 --- /dev/null +++ b/doc/meetings/2020-02-26.md @@ -0,0 +1,127 @@ +# Node.js Modules Team Meeting 2020-02-26 + +## Links + +* **Recording**: http://www.youtube.com/watch?v=4d0g40sSV-g +* **GitHub Issue**: https://github.com/nodejs/modules/issues/489 +* **Minutes Google Doc**: https://docs.google.com/document/d/1Evh1h_SAq7VnfXJlt5V8-HX5an6JstVdRE9IkowxU5E/edit + +## Present + +- Myles Borins (@MylesBorins) +- Bradley Farias (@bmeck) +- Corey Farrell (@coreyfarrell) +- Geoffrey Booth (@GeoffreyBooth) +- Guy Bedford (@guybedford) +- Jan Krems (@jkrems) +- Jordan Harband (@ljharb) +- Rob Palmer (@robpalme) +- Saleh Abdel Motaal (@SMotaal) +- Wes Todd (@wesleytodd) + +## Agenda + +## Announcements + +Extracted from **modules-agenda** labelled issues and pull requests from the **nodejs org** prior to the meeting. + +## nodejs/node + +### module: disable conditional exports, self resolve warnings [#31845](https://github.com/nodejs/node/pull/31845) + +- GB: We had issues with project like Preact that use `exports`. Node 13 users are getting warnings. An ES module lexer was asking the main to point to the ES module file - Guy corrected them to use `module`. They used `exports` and people did not like the warnings. Snowpack was unable to use it until the warnings are gone. If we want to backport to Node 12 in April, we need real world usage feedback. We will not get it. Packages are resisting. We should remove `experimental modules` warnings as well. Let's discuss timelines. When are people comfortable removing them? +- MB: The balance is ensuring folks using this are aware it's experimental. Sometimes people jump into issues complaining about breaking changes, and we point to the warnings as justification. I don't think we are making big changes to conditional exports. It is conditional exports and self-resolve we are thinking of removing warnings for - not the warning for ESM as a whole. I am comfortable removing those warnings on Node 13. Less clear on 12. So let's remove on 13, wait a bit before backporint. +- GBB: On 12, they wouldn't see the warning unless they opt in by the flag. +- JH: There was a brief bug on 12.6 that is fixed now. +- MB: It's good to keep consistency between 12 and 13. +- JK: +1 on removing warnings. The effect was supposed to be to advertise its experimental. In practice, people find magical ways to avoid triggering the warning. They use the feature but actively hide the warning. If removing the warning matches the world... it is what is happening. People are using the feature. Right now it's painful they have to use a specific way to avoid the warning. +- MB: The warning achieved part of its goal - people got annoyed and told us, which told us people were adopting it. The risk is that we will stop receiving feedback. +- JK: I raised this because I found an issue on a smaller project that was copying a bigger project. +- MB: They are writing things such that in CJS they are not using experimental things. I am being pedantic. I think we should remove the warning. +- JH: The point of the warning is to let people know. The author is doing this. The user is not. The warning is mis-targeted. We could make exports a non-breaking change. Preact 3.1 is breaking changes vs 3.0. Despite our warnings, people use this feature in minor bumps. We here about it from consumers. JK and GBB have a point - users know how to subvert the warning so it's not achieving much. Features like this leak. They are not encapsulated. At this point, things are using them. The ability to break them is kinda irrelevant. So let's remove the warnings. +- MB: The warnings about unsafe buffers was more scoped but is more effort. Is it worth it? +- JH: Unless they are using self reference, they are not using the feature in their own code, so their own tests won't trigger the warning. So a targeted warning would not help. +- MB: Let's get consensus. Any objection to removing warnings on 13 for conditional exports and self-referencing? +- *silence* => consensus is reached +- GB: I will update the PR to only remove these warnings. +- MB: Start a new issue thread for removing the overall ES modules warning. I don't agree with removing it yet. +- JK: Having a PR open for it would be good. Not many weeks left until we can remove the warnings in Node 13. +- MB: I do not think we will remove the warning in April when we unflag in Node 12. +- GB: This is news. +- JH: Talk about it next meeting? +- GBB: Are we going to backport the conditional exports and self-references warning removal to Node 12? +- MB: That's a separate issue from what we discussed today. +- JK: No one is accidentally triggering this on 12 because it's already behind a flag. +- GBB: That's why I thought it would be uncontroversial. I just want them to be similar. +- MB: Agree on principle. On warnings, and stability index, it's different for LTS branches. +- GBB: Ok. We can make a separate issue. + + +### Timeline Discussion + +- MB: I think removing warning for experimental modules is not something we should do until we are close to stabilising the API. I was unsure if we wanted to remove it when 14 is being cut. We probably want it gone by 14 LTS. I want to see more usage of the feature without the flag first. fs.promises had a warning for *much* longer. We have a history of warnings on other APIs. There is a history of removing them during LTS. May even be premature to remove in 13. That is where I am coming from. +- GB: We have to think about the utility of the warnings. It discourages adoption. Users do not like warnings. They are part of the UI of Node. Users cannot control it. We're discussing backporting to 12 LTS. If we want to discourage adoption, then I don't see how backporting is not as important as removing the warning. This process of getting modules into Node.js has taken a long time. Tendency towards taking longer is not good. We want to prove we can move forward. +- JK: The impact. Removing would be that if I want to use modules in a CLI tool I publish to npm, I cannot publish until the warnings are removed. There are good reasons to remove the warning sooner rather than later. The process has been long. We would lose momentum by not unflagging. +- MB: Slowing down adoption is potentially a feature not a bug. If we unflag on LTS we will get an exponential curve of more adoption, bugs, usage, issues. If we got something wrong, we have far less room to fix it. We're talking about months not years of difference. These features have only been worked on for less than a year. They are not as mature as we would like them to be. Node 10 is still maintained so people can't ship things until next April so folks can ship modules without breaking other folks. If we sketch out timelines, worst vs best, when does Node 10 go out of scope. I will create an issue and raise this in the next meeting. + + +### Add support for extensionless workers and ESM eval [#31760](https://github.com/nodejs/node/pull/31760) + +- MB: Upstream PR. The work is happening. +- BF: We need to be aware of it. The PR goes against one of the design patterns we use. They are having the consumer define the format of the target resource. The thread has alternatives. Nothing needs to happen here. Please look at the issue. +- MB: I will remove "Modules agenda" label from it. If you see stuff, label it. + + +### WIP: Move ESM loaders to worker thread [#31229](https://github.com/nodejs/node/pull/31229) + +- BF: PR has been open for a while. Global attentuation. We have to split the work. Move forward or block it. We've tried past approaches. One was adding a hook for preloads. Was very hard to wrangle. then I tried allowing loader hooks to give us source-text to be run on bootstrap - but async work is very tough because of how async hooks work. Could be synchronous if we run in the script goal. Then we could figure out interleaving later. You could "use strict" in there yourself. You need access to `require`. This is not for APMs. +- GB: Falls under the same bracket. +- BF: People are using dynamicInstantiate to punch globals around. Not to mutate existing modules. +- GB: Also, a loader should be able to be self-contained. Use of loader should not invoke a require flag. APMs need to hook in. +- BF: Separate feature. +- GB: This is critical. It makes loaders less useful. +- BF: It's not supported yet. +- GB: But today's it's possible. Loaders can hook into built-ins. +- BF: They can do it via bootstrapping. If that's the requirement, this PR will remain blocked for a long time. +- GB: As long as attenuation has access to require. +- JK: Agree with Guy. Some way of running code globally before app code runs ought to be able to load builtin modules. That captures many use-cases. Could use --require. One with bootstrap. One with loader hooks. That also applies to any attenuation code, so this is not a critical argument. We need to decide. Block this PR on bootstrap/attentuation code on stuff that could be passed into require, or can we move forward even though loaders cannot do attentuation and need a second flag. +- BF: We're not saying it's like that forever, even if we go forward. + + +## nodejs/modules + +### Chartering the Modules team [#412](https://github.com/nodejs/modules/issues/412) + + +- MB: Next step is a PR to the tsc repo. A doc is there full of all the working groups, including a scope statement. Streams scope seems similar to what we should be asking for - it says "we will take responsibility" rather than asking for sign-off on every change. +- JH: It would be nice to be chartered. Recognition is one part. Seems like the right thing to do. More decisions will be needed in future. +- MB: TSC can decharter if the group disbanded. +- GBB: The issue about wasm modules. We've expanding package.json. There may be lots of groups wanting to drive things into that crack. So maybe it's worth being chartered. +- MB: Stream scope is really good. Specifically, nothing here says they own all decisions. I would be uncomfortable saying we have total ownership of part of a codebase. +- WT: With package maintenance group, we added to package.json. npm just added it without process. Whilst I think it's good to have a group that is not npm, it should not matter for other groups. +- MB: Does anyone want to take the time to draft a charter? +- GBB: Open to doing it. Not sure when. +- JH: Same. +- MB: Two options. Keep it open? Any objections to chartering? Any objections to closing the issue? +- JH: Keep it open. +- MB: If no one has time to do the work, it's a relatively strong signal that we're not being blocked by our status either way. Maybe at some point we just close the issue. One advantage is discoverability. + + +### Loader Hooks [#351](https://github.com/nodejs/modules/issues/351) + +- GBB: We were supposed to untag it? +- MB: Objections to unlabelling? +- *no objections* + + +### Patch/Instrument a module [#339](https://github.com/nodejs/modules/issues/339) + +- GBB: This is the last missing piece. Nothing has happened over the last two weeks. +- MB: Keep the label? Objections to removing? +- CF: For NYC, current hooks don't allow multiple hooks/transforms. Suggestion was userspace composition. I don't know how it would work. There could be only one. That's my concern. +- GBB: The reason the API passes a callback at the end was to make them composable. It might still happen. Someone just needs to do the work. Jan was skeptical about making it work in a reasonable way. The user would have to provide the ordering. Rather than a strange config file, this let's the user compose the loaders. Code over config. Can be done now. That might be the best we can do. +- CF: My expectation was by order of the arguments. So --loader three times creates a stack. For NYC, it injects a transform into child process. I don't see how we deal with someone using NYC and ts-node or babel. +- GBB: Please open an issue with the specific issue. +- CF: ok +- JK: I was more bullish on multiple loaders before I tried to write them and combine them in a practical way. It's easy to write a loader in a way that is incompatible with other loaders. For example, ts loader checks for file extension. Then I have an HTTP loader that may not even have an extension. The orchestration of multiple loaders is non-trivial. I haven't seen a good story of how the user experience could be good, and the developer experience of writing loaders could be good. It's better to have "the webpack of loader orchestration" for this use-case. You need something smart - Node is not the place to do this - it does not load complex bootstrap files. It's not a good fit. + diff --git a/doc/meetings/2020-03-11.md b/doc/meetings/2020-03-11.md new file mode 100644 index 0000000..2efe856 --- /dev/null +++ b/doc/meetings/2020-03-11.md @@ -0,0 +1,85 @@ +# Node.js Modules Team Meeting 2020-03-11 + +## Links + +* **Recording**: http://www.youtube.com/watch?v=9o90Z20fFIE +* **GitHub Issue**: https://github.com/nodejs/modules/issues/494 +* **Minutes Google Doc**: https://docs.google.com/document/d/1KZBgLnJn3ZejOIitOqTylXwKdCBZ7Byka9T3fUch3wU/edit + +## Present + +- Geoffrey Booth (@GeoffreyBooth) +- Myles Borins (@MylesBorins) +- Jan Krems (@jankrems) +- Wesley Wigham (@weswigham) +- Jordan Harband (@LJHarb) +- Corey Farrell (@coreyfarrell) + +## Agenda + +## Announcements + +*Extracted from **modules-agenda** labelled issues and pull requests from the **nodejs org** prior to the meeting. + +### nodejs/node + +* module: expose exports conditions to loaders [#31303](https://github.com/nodejs/node/pull/31303) + +- did not have this functionality exposed to userland yet +- conversation active in repo +- Geoff explains alternative proposal +- Discussion +- No action, continue discussion in repo + +* WIP: Move ESM loaders to worker thread [#31229](https://github.com/nodejs/node/pull/31229) + +- removing label +- jan mentioned https://github.com/nodejs/node/pull/32068 as an FYI about a PR that can be used to unblock this PR progress + +### nodejs/modules + +* Chartering the Modules team [#412](https://github.com/nodejs/modules/issues/412) + +- Went over high-level steps last time. No urgent need identified, decided to punt on it for now. +- Geoffrey/Jordan/Myles happy to work on this just not now. +- For standards group, used part of regular meetings to pair on this. May be possible here. + +* Patch/Instrument a module [#339](https://github.com/nodejs/modules/issues/339) + +- Removing label for now, no active work. +- Bradley is looking into loader hook composition in general. + +* [m.parent] + +- Bradley shared module.parent issues in general. +- No active work. +- Jordan: module.parent is a misfeature (“which module happens to require this first”). Determine if an entrypoint is important but module.parent isn’t it. +- Myles: WASM group discussed how “knowing if entrypoint” could look like, including magical exported function. +- Myles: Bradley’s comment reflects overall opinion in the room +- (no objection) + +### Stability and Flags + +- Myles: Currently 30+ open issues and 15+ open PRs. Seems like a signal that modules are fully stabilized and are ready to remove all warnings. +- Jordan: How many are about loaders vs. core modules? +- Myles: (List of issues) Most aren’t connected to loaders. +- Guy: None of these are bugs, mostly confusing. +- Jordan: Wouldn’t count many of these beyond documentation issues. +- Jordan: Loaders were marked as out of scope for core module features. +- Myles: Disagree about the phrasing. +- Wes: Pretty sure it doesn’t work with absolute paths on Windows. +- Jan: Neither does it work for arbitrary paths on Linux (URL paths != unix paths). +- Myles: There are usability issues and it’s confusing for people to use. Don’t feel like we could remove the experimental status before node 14. +- Geoffrey: Doesn’t matter, we can remove the warning during the lifetime of 14. +- Myles: Like to see this completely stabilized, including docs stable, before end of life of 10. +- Guy: Not going to strongly argue either way. Can accept the warning in node apps. Think it’s the wrong decision for node.js as a project. It’s pushing people to other solutions. Rather use deno and not get warnings etc.. The signal is it’ll be another year before users can use it, really. +- Corey: Not sure what we’d expect to change that would break user code. No strong opinion on 14.x. Would be unfortunate if first LTS doesn’t support warning-less ESM. Full LTS cycle before maintainers would be comfortable to use ESM widely. +- Myles: Would it be better to say “before 14 LTS”? Sometime before October. +- Wes: Is it good to pick deadline vs. picking a target stability/maturity? +- Myles: Valuable signal to users to have the warning (“there are rough edges”). Corey has made a good point around removing the warning at least for 14 LTS. Docs issues are as important as bugs. Tells us that users are failing to adopt it, wasting their time. That’s another existential risk for node (user churn). +- Geoffrey: Like to triage the issues and classify severity. +- Myles: I think there is a difference between runtime stability and readiness. Our ability to triage issues and update the repo. +- Jan: Bar should be a good funnel for users to succeed, not just a possible state where people may succeed (if they discover it). +- Guy: Agreed on higher level concerns about our ability to maintain a good experience. But from a bug/feature perspective, it’s fairly stable, incredibly stable as far as software goes. We still have to take ownership on all the supporting processes. Regrettable if that causes much more delays. +- Corey: Concern was mostly about runtime warning, doc status “experimental” not a blocker for next LTS. +- Myles: One of the sign-offs at Google is “DevRel” and “Docs” signoff. Agreed that implementation is super stable, agreed with Guy. Need to make sure the other aspects match it. Will work on a clearer bar for when it’s ready to remove warning or change doc status. diff --git a/doc/meetings/2020-04-22.md b/doc/meetings/2020-04-22.md new file mode 100644 index 0000000..ae8c4c2 --- /dev/null +++ b/doc/meetings/2020-04-22.md @@ -0,0 +1,124 @@ +# Node.js Modules Team Meeting 2020-04-22 + +## Links + +* **Recording**: http://www.youtube.com/watch?v=pYg1thE_Nz0 +* **GitHub Issue**: https://github.com/nodejs/modules/issues/504 +* **Minutes Google Doc**: https://docs.google.com/document/d/1G1coLWXsCPPWsfkgbYR0aXvmSRMoaJIz-ndpVk_2Nlc/edit + +## Present + +- Myles Borins (@MylesBorins) +- Jordan Harband (@ljharb) +- Corey Farrell (@coreyfarrell) +- Michael Zasso (@targos) +- Rob Palmer (@robpalme) +- Bradley Farias (@bmeck) +- Wesley Wigham (@weswigham) + +## Agenda + +## Announcements + +- MB: Warning removed on Node 14. Released yesterday. + +### nodejs/node + +#### module: development and production exports conditions [#32869](https://github.com/nodejs/node/pull/32869) + +- MB: React community inspired this conditional exports. Lots of discussion. Some -1s - mostly about NODE_ENV not normally relied on for Node behaviour. Those env vars don't have clear defaults. Discussed with Matteo. Might change the mental model, e.g. a new env var that is not NODE_ENV. We chose new ecosystem patterns for "type": "module". There was interest in introducing a DEBUG mode to Node - relating to log levels. +- JHD: Conceptually this is wrong. Resolution should not be affected by environment. Conditional exports is about which implementation do I want. Production and development React have never been different implementations - it's simply warning code deleted in production. Doesn't feel appropriate to moosh this into conditional exports. Not the proper place for environment switching. +- BF: For React, they are not changing names or signatures. But they do hook into DevTools and run side-effects that should not leak into production. +- JHD: Some people use those things in production. +- BF: You should never do that. +- JHD: Some developers need DevTools in production. +- BF: We do a network override instead. It's bad. There is an obvious reason to add this. The idea the environment affects the code is beneficial. +- JHD: I am specifically talking about resolution. +- BF: But the resolution determines what code is loaded. +- MB: Some premises the PR is built upon. Independent of the use-case, one thing that came up with Matteo on debuggability was the ability for this to be dynamic over the lifetime of an application. Are conditional exports set in stone at app launch - or change later? NODE_ENV can be changed over the course of the app. 2. Do we want the conditions to be static or incorporate statements? Conditions on the left could be NODE_ENV==production. +- BF: We have lots of flags for resolution. Symlinks. Home directory for CJS. These are set when Node starts up. We shouldn't make this feature the only dynamic feature. +- MB: Agree. Dynamicism would re-open closed topics. +- CF: If people published packages on development, this could surprise people using it on their package, e.g. NODE_DEBUG==my-package. I agree NODE_ENV is a bad idea here. +- BF: So instead of it being applied to the whole app, you want to apply to specific packages only - not global? +- CF: Yes - so I can opt in selected packages into debug mode. +- BF: Might cause issues. Packages can share names. We have well-defined package boundaries now, so it's possible, but may not be realistic. You start to need to know internal package structure of your whole app. +- MB: Gus brought up that loaders could help with. If ecosystem aligns on patterns of tooling on top of Node. Debug loader could know this stuff. So ecosystem could solve this. We have given people most of the tools they need. +- BF: I agree that loaders are a power tool that everything can be shoved into. But we don't necessarily need to use them for everything. We have symlink flags - this affects the default loader but custom loaders may not respect this. So it depends whether this is configurable in our loader, or whether ecosystem can do this on their own. Personally I don't think why there is much to discuss about conditions. This is basically a bitfield turned on and off by name. Maybe we should figure out why configuring condition exports is controversial. +- MB: This condition is controversial because NODE_ENV respect of production/development is not consistent across the ecosystem. +- BF: I don't know why we don't want this to be configurable. Normally that's for things that would cause security issues. It's hard to configure a loader correctly - much more dangerous. We have object identity going on in the conditions. Seems weird to shove this under loaders. I don't see why configuring this is problematic. NODE_ENV could have collisions or misuse, but if we solve that I don't see the objection. +- MB: Matteo wanted it to be dynamic. People not thrilled about production vs development. Dynamic vs not dynamic. +- WW: Not a good idea because a conditionally different API is a bad idea. Makes static analysis harder. +- BF: Tools already need to deal with process.ENV. Most bundlers support it because it's the standard way. Constant substitution. Would be easier to analyze pkg.json than parsing JS. +- MB: Next steps are likely: further scope low level assumptions. Then define the feature more. +- BF: There's not a clear path forwards to me. Because there's no reason to push it either way. +- MB: Feature request came from the ecosystem. So define a path to concensus or close it. Does it need more discussion? +- BF: Currently discussion doesn't seem to have direction. Don't close it out. Let's block it on use-case constraints being unclear. + +#### module: empty fallback and null exports as not exported [#32838](https://github.com/nodejs/node/pull/32838) + +- MB: PR from Guy. Limits throwing errors when you pass an empty array or null when you pass it on the right hand side of the exports. +- BF: Just do it. Sounds fine. + + +#### esm: allow configuring default package type [#32394](https://github.com/nodejs/node/pull/32394) + +- BF: This is a build option for people who want to make custom builds of Node. Some people want to change the default package type now or in the future. I made a PR to allow them to do this, but virtually nothing runs if you set it. So it's a playground. I firmly believe this should not be runtime configurable yet, because people will start to rely on it. +- WW: You can simulate this with a fork of npm/yarn. +- BF: Parse goal determination is affected in a non-trivial way by changing the package type. There's a package out there that had to be modified because the change in default was weird. This PR let's people test it without opening issues. +- MB: We debated whether it should use a new binary name, e.g. `node-esm`. Not a hill to die on. +- JHB: Unclear why this is needs to be made easy. +- MB: It helps shutdown the debate. +- JHB: I don't want it to be easy. +- BF: it's not easy - very few people build Node. +- JHB: It's just a matter of changing a string in a few places? Like in your PR? +- BF: There's indirection for configurability but there is logic too. We do some defaulting if the package type does not exist in multiple places. +- JHB: We could refactor to have a single source of truth. Wes pointed out people can already do this. People can float you closed PR. I question the utility of landing it. +- BF: People are asking for it so I made a PR to give it to them. Why prevent them from having it? +- JHB: It causes ecosystem issues if it spreads. +- BF: It's not easy to build Node. +- WW: It's dead code. +- JHB: It's not a thing we want to exist. +- BF: I want to give people what they want. We can make a recommendation to float this PR. +- JHB: Then let's do that. +- BF: That sounds impolite compared to just merging it. +- MB: Floating patches can go out of date and will not be tested. If we have a test... I don't know if we want to maintain this. It would require a whole other build. So it is a risk landing it. +- CF: No objections to a build time flag. Runtime flag would induce requests to package owners. +- MB: We already support `type: commonjs`. Default is an internal setting which could change. We should introduce this if we would ever want to do. +- JHB: Not sure it would get consensus. +- MB: Don't see how we could ever change the default. Not trying to bring stop energy. +- BF: Fine with either way. Could we get consensus to say we will never do this? +- RP: Feels like a big commitment. +- MB: We are definitely not going to change it in the next three years. I don't see how we could change it without such big changes that the type field itself would be removed. +- RP: Seems reasonable. +- MB: So maybe this does not need to exist, because we might not even need the mechanism. +- BF: Because 14 defaults to "commonjs", likely all LTS will support .js as being commonjs. Seems like this is a no go to change. Therefore this PR should never exist. +- JHD: That's my position. +- BF: I can close it out. +- WW: CJS is first-class forever. +- JHD: Top-level await could be an exception. +- MB: And import.meta. TLA in CJS breaks the world. +- BF: Don't agree this means CJS is first-class forever. We could allow CJS to be disabled. +- RP: This feels like we've made a leap here from saying we can't change the default to saying CJS is firs-tclass forever. +- BF: Agreed. We can close the PR and say we have no intent to change the default in the foreseeable future. +- JHD: Even folks uncomfortable committing to not changing the defaults forever would agree it won't change in the near term. + + +#### Self-reference Sigil + +- JHD: Self-reference independent of package name. If we got consensus, could we add it in semver minor? +- BF: Sounds ok. + + +#### Loaders into worker threads + +- BF: I didn't want it to land earlier because it disables a loader hook. I plan to land it - but it cannot be backported. The dynamicInstantiate would be removed. Any objections? We had an objection to breaking that API - which is why we added a hook to instrument the global env. I'd like to land in a Stable (non-LTS) branch for people to play with +- CF: In Node 14, not landing the PR, means it can't have non-experimental loaders. +- BF: Some loaders do use this hook. If we want to backport and we are ok with breaking, we can discuss it. But we haven't got field testing. I don't want to break people until we have had more testing. Then we could backport it. Even though it's experimental we still don't want to break stuff. + +--- + +**END OF MEETING** + +### nodejs/modules + +* Chartering the Modules team [#412](https://github.com/nodejs/modules/issues/412) diff --git a/doc/meetings/2020-05-06.md b/doc/meetings/2020-05-06.md new file mode 100644 index 0000000..9ca3cc2 --- /dev/null +++ b/doc/meetings/2020-05-06.md @@ -0,0 +1,97 @@ +# Node.js Modules Team Meeting 2020-05-06 + +## Links + +* **Recording**: +* **GitHub Issue**: https://github.com/nodejs/modules/issues/510 +* **Minutes Google Doc**: https://docs.google.com/document/d/1Iw0xXfc080uQoarPQvm6MuCjUdIX6Yhv83MjgiXV_Pw/edit + +## Present + +* Jordan Harband (@ljharb) +* Geoffrey Booth (@GeoffreyBooth) +* Rob Palmer (@robpalme) +* Michael Zasso (@targos) +* Bradley Farias (@bmeck) +* Corey Farrell (@coreyfarrell) +* Wesley Wigham (@weswigham) +* Jan Krems (@jankrems) + +## Agenda + +### nodejs/node + +#### cli: --dev flag for development exports resolution condition [#33171](https://github.com/nodejs/node/pull/33171) + +- GB: It's incredibly useful for bundlers. Standard pattern checking process.env. Two things happen: determining the graph + dead code elimination. Benefit of graph level is that side-effects of the whole graph get triggers. Most dependencies are not pure. So bundlers struggle to remove dev code. You can "await import()" and import dev dependencies - top-level await enables this. Two problems: 1. we don't have to-level await. 2. no bundlers support this yet. Doing this at the graph level has lots of benefits. Just have two builds you switch between. If you want to point to a development build. webpack (Tobias) are interested in this - not sure about Rollup. Do we put this in Node to set a precedent for bundlers? If it goes in Node, how should it be enabled? Right now it's a flag per-run. It triggers a "development" condition in exports. We're struggling with reflecting this information in a runtime accessible variable. Hard to get consensus in Node. Motivation: trying to be adoptable universally. We don't have a pattern for universal environment variables apart from globals - but adding a global is not an option. No one will allow us unless we put it in process. We could add it later. Happy to land this? +- WW: Node's conditional exports is not the first runtime to have this. React Native has it. It is terrible from a tooling perspective. So I can only recommend conditions as a last resort. I recommend we recommend people only use conditional exports as a last resort. +- GB: But you want widespread bundler support for conditional exports? +- WW: Yes. +- BF: .... having tooling needed to parse JS control flow graphs is simpler than any alternative. +- JH: Feels weird to put this feature in. Currently the conditional exports switch between module systems. Switching between platforms. But dev/production is like a matrix applied to both. So it doesn't seem right to conflate them in the same object. Seems overly prescriptive. It's unfortunate React behaves differently in production and development. Would prefer a world where they don't need multiples. Would like to remove the need to pivot based on the environment. +- WW: On React, the next version to swap dev vs prod, they emit entirely different things. So no need for runtime switching. +- GB: This is highly specific to React. Let's not think about this as one solution. Runtime and export switching are different. Some publishers can highly optimize their build. The common catch is that they don't realize they need to set the environment and they can get two copies. If you're a team like React, maximising optimization with Closure compiler etc is going to get you best performance - conditional exports is tailored to this. +- JK: I don't see the dev build as something that allows more optimized prod builds. It doesn't preclude you maintaining your current package with no debug features. It's very non-standard to add debug features. This would give a clear way to do so. No risk of getting blamed. Can packages become more helpful because of this feature? +- BF: Let's consider alternatives to see if this is a good direction. Currently we have a kind of standard way to do this using process.env.NODE_ENV. Some bundlers support it. But it's a pain to parse. Even if you use it you end up with a similar problem of not wanting two builds. You have code that's only executed in one mode. So with React Devtools you expose things you would never want used in production. These aren't merely for production builds. We have things we want when we are the app runner. But we also run other people's code. Running with a dev flag seems like a boon to me. You kind of have it with if/else but may not work with tooling. No clear alternative apart from saying people shouldn't want this. It would be helpful if people suggest alternatives. +- WW: Yes, we are just moving the branch from explicit in code to implicitly in a loader. There is a problem that makes this unsafe: exports only affect external references, but not to internal references. So unless you have independent trees (dev and prod folders) you're liable to call back across entrypoints. Self-referential would solve this. +- BF: Disagree with implicit vs explicit. It's always explicit. +- GB: From Tobias, seems like he will implement this in webpack. The fact that proposing this triggered interest suggests strong demand. We don't control whether it exists. We can control where it exists. +- **no conclusion - let's return to it in a month** + +#### \[v12.x\] Backport unflag --experimental-modules [#33055](https://github.com/nodejs/node/pull/33055) + +- MB: We have discussed unflagging on semver minor on 12. It got delayed a bit. I think we're waiting on consensus on tiny bits. 1. Are we cool with backporting removal of the flag? tsc needs to agree. 2. Do we backport with or without the warning? I have spoken to tsc. I think we should backport removal of the flag, but I think we should keep the "experimental modules" warning until 14 goes LTS in late October. Reason: removing the warning on 14 - that's current not yet LTS - people using it know the warnings. I don't think that same signal of support and adoption should be given to LTS. +- JH: I was taken aback by the PR. Clarification: anyone using require() will see no warnings? +- MB: Yes. +- JH: So only import() and static import would trigger the warning. I would prefer no warning at all because it hinders adoption. +- MB: That's the point - we don't want to give too strong of a signal. There are rough edges. I think tsc would pushback on removal of the warning. +- CF: My only concern is that some packages might be using import() to detect support for ESM. So unflagging with a warning might trigger warnings based on this detection. +- MB: Those packages wouldn't work today? +- CF: Correct. Today it fails or doesn't. This proposal makes it pass but side-effects is a warning. +- JH: So they are try-catching eval'd code? +- CF: Yes. I may be doing this in NYC. +- GBB: We already had this with 13/14. +- JH: That's kinda what I was concerned about. The point of the warning is to give actionable feedback. In top-level code that's fine. In code where you try to use a package, and you get it that's fine. But transitive dependencies triggering this trains you to ignore warnings. Removing the flag is a much stronger signal than removing the warning. +- MB: Removing flag makes it easier to adopt. Removing the warning makes it easy to adopt en-masse. +- CF: I double-checked NYC. I have a separate load-esm/cjs which does import() - only loaded if someone attempts loading mjs. So no warning would be produced unless someone was actually using ESM. I have seen the suggestions "why don't you use import() and fallback on require() if that fails?" +- LH: I have a package to do feature detection of exports in Node. It doesn't tell you about ESM. +- MB: I looked at self-referential, you can try-catch it. I think this is the right way to sniff it. Richard Lau brought up: anyone not opting into ESM but have dependencies that will act differently on unflagging. So there's a question about whether flag removal is semver major. +- BF: Did we remove the warning for conditional exports in 12? +- MB: We removed them. I believe "exports" work. But conditional exports and self-referential are still behind a flag. +- BF: My feeling is that without conditional exports, I don't view it as breaking. +- LH: Exports field does not work at all in Node 12. In spirit, people should write "main" and "exports" having the same API. It's possible to break. I hope no one wrote a package like this. +- BF: I have not heard of anyone doing that so far. +- LH: I wrote es-get-iterator. It does terrible things in old browsers to get an iterator interface. I used conditional exports to guide Node versions. I could have used fancy syntax because I knew which version of Node it would be run in. +- GB: This issue is something we had a long session with James Snell on. We discussed if it would be semver major. He thought it would be fine to land as a minor. There are >500 packages using "exports" now. I have a list. We could test them against unflagged Node 12. +- MB: The only packages that would have behavioural changes are those where the author has opted in. CJS reached via exports would continue to have the same interface. A benefit is that broken packages should be able to be fixed in a patch release. My gut (famous last words) is that it's not going to be a significant undertaking to fix this. These shouldn't be deep in module graphs. Adopters are likely to have a faster cadence at dealing with this. Anyone want to block landing this in 12 with the warning intact? Caveat: aiming to remove warning in October unless ecosystem outcry or technical issues. +- **silence** => consensus! + + +### nodejs/modules + +#### Rediscussion for out-of-order CJS exec? [#509](https://github.com/nodejs/modules/issues/509) + +- MB: TLDR: today importing CJS is supported but not with named exports. Is the problem of named exports from CJS something we want to solve? Reducing delta between faux modules and node modules. Solutions: stack analysis, double execution. First, is this a problem space we want to dig into. We seemed to have consensus on this. I don't think dynamic modules has legs. Do we want to revisit this problem space? +- GBB: Yes. Users are complaining. It's a big pain point. We need to show effort in saying why we can't do this. We need to try the best solution we have thought of. Then we have evidence. +- BF: We're seeing people have a problem transitioning their modules. I want to allow easier transition for some. I don't think we can reach 100% coverage. I am very against re-ordered evaluation - weird side effects and no tools do this today. +- JK: I slightly disagree with "we need to write code" because I think we understand how well the approaches will work. The objections have rarely been "hey this wouldn't work", it's more that people disagree to the degree with which they will work - "we don't like heuristics". Can't be resolved by writing code - it becomes less convincing. I strongly agree we should decide on how much we care about the level of support. No level will match today's behaviour in CJS/faux-modules. module.exports = require("other") will always say there is one export if you analyze one file. Is it really a big deal to add a wrapper mjs? So we know the trade-offs. We just have to make a decision. +- MB: I started a PR https://github.com/nodejs/node/pull/33256 when a named import fails it checks to see if the module is CJS, and if so gives a better warning including copyable code to replace your named import. The RegExps need more testing. Another approach is improving docs and error - what broke, why it broke. I lean towards Jan. I want to improve the experience. But the clear cut "you have a default" is easy to teach, easy to understand, does not require static/dynamic understanding. It's cleaner. +- JK: Parsing is viable, but not strong enough for me to implement it. +- GB: I lean towards wanting to implement this. I thought we would get away with just default. But tooling will still permit named exports. So friction will continue. So it's worthwhile to mitigate this pain.Maybe we can only do it at the boundary of ESM->CJS - overhead could be <1ms per module. I would like to ask if we have consensus to do it. +- MB: Let's kick this back to the issue to discuss consensus - no time now on this call. +- GB: If I put in dev time I would like to know if there would be objections. +- JK: People on the issue tracker made strong objections. +- GBB: Anyone interested in working on this? +- JK: My concern is exports. By definition we have a collision. If we want to be compatible with faux modules - they will treat it as a default. +- MB: I suggest closing the current issue - it's too long and off-topic. Create a new issue on named exports as an RFC for what you plan to implement so we can respect your time. +- GB: Let's return to this in the next meeting. Node 12 support is time critical. +- MB: We need to decide if this is needed before unflagging in Node 12. +- **Conclusion: Guy will write an RFC as a new issue** + +--- + +**END OF MEETING** + +* Chartering the Modules team [#412](https://github.com/nodejs/modules/issues/412) + + diff --git a/doc/meetings/2020-05-20.md b/doc/meetings/2020-05-20.md new file mode 100644 index 0000000..8a80d7c --- /dev/null +++ b/doc/meetings/2020-05-20.md @@ -0,0 +1,86 @@ +# Node.js Modules Team Meeting 2020-05-20 + +## Links + +* **Recording**: +* **GitHub Issue**: https://github.com/nodejs/modules/issues/515 +* **Minutes Google Doc**: https://docs.google.com/document/d/1XUBqEtPDwsgOgjrsoyacQhXHpTTdMyWAnEiwoZcDh4o/edit + +## Present + +- Myles Borins (@MylesBorins) +- Corey Farrell (@coreyfarrell) +- Jordan Harband (@ljharb) +- Guy Bedford (@guybedford) +- Rob Palmer (@robpalme) +- Wes Todd (@wesleytodd) +- Wesley Wigham (@weswigham) +- Geoffrey Booth (@GeoffreyBooth) + +## Agenda + +## Announcements + +### nodejs/modules + +#### CJS exports detection [#33416](https://github.com/nodejs/node/pull/33416) + +* GB: This took many years of discussion. Seemed to make progress on agreement in last meeting. Parsing approach is a candidate. Bradley made a project with Acorn parse + analysis. It became clear that parsing performance is important. Even 10ms is a long time. I put together a lexer (based on ESM detection) that detects a few patterns. +* GB: https://github.com/guybedford/cjs-module-lexer#grammar +* GB: If people webpack their code before publishing we can detect that. It does not do full scope analysis. TypeScript do something similar at top-level. Geoffrey ran it against the top-1000 modules. 62% were fully detected. Fallback is to only introduce the default export. It seems largely useful. I personally think it's useful to ship. +* JH: The PR proposes using `__esModule` that Babel produces. But Geoffrey's numbers do not use that. Searching for `exports.` won't work for all cases - `module.exports =`. I'd rather ship something that nothing. Want to maximize the patterns that work. The `module.exports =` is a perfectly valid thing to do. +* GB: We support React's pattern. And `module.exports.property`. Gus has put a block on this because if you have an existing module from `exports.` to a non-`exports.` you don't know you are breaking your consumers. Would like to decide this soon. +* MB: We landed in Node 14.3 the updated change to the errors so that CJS import of named exports gives you lots of good messages. It's a good step forwards from before. Having worked with wrapper modules, it's non-trivial, and makes it harder to test, then you need to sniff support. It's non-trivial extra work. You need to introduce package exports which opens up other things. Definitely doable but a lot of effort. So put strain on maintainers of lots of modules. With today's clear warnings, it's a lot better than it was. We can improve it even more with docs. I am not convinced the parsing is worth all the effort. I am -0 or neutral. Is the complexity and edge cases worth it? Do we think it's possible we can alleviate the existing blocks? Don't want people to waste time or get demotivated. Better to settle it before the labour goes in. +* JH: Speaking for Gus, adding exports is a potential breaking change. Adding Babel is too unless you are very careful. It would be unfortunate if adding this feature increased the set of accidental breaking changes. How can we show we're not making it worse? +* GB: The initial feedback was that people expected this to work - it works in bundlers. We can add this later and be non-breaking. But the support for the esModule flag is. Flag + default means we support Babel workflows. You don't want to `.default` from CJS. Cannot be changed once these workflows become solidified. +* MB: I think we need Gus to speak to move this forwards. Any other objections? +* JH: Depends on whether accidental changes are increased. +* WW: I appreciate we can move forwards with this and add more in future. +* GBB: I am concerned about the low numbers on detection. Over a quarter are not detected correctly. +* GB: Wonder how many of the 33% are just used by `require()`. +* WW: Lodash relies on ESM. +* GBB: There's a lot of big names in the not supported list. But they will likely get updated. So maybe don't worry about them? +* CF: I'm neutral. One comment about assigning an object to `modules.export` - that's how I roll. So maybe we can improve on this? +* GB: Yes, we can be additive. The grammar must be backwards compatible. +* GBB: I use it too. +* JK: Speaking of 33% I am less concerned about the numbers, so long as we capture 100% of the ESM-transpiled-to-CJS. I care less about hand-authored CJS. I would be in favour of scoping it to Babel/webpack output only. Gus treats this as something that doesn't fundamentally changes things, I think it does. Ordering of upgrades in a monorepo matters. +* JH: I want to know what the proposal's impact will be. "Here is the effective heuristic". +* GBB: Then we put it in the docs. I'd rather not scope this to Babel. Those are the one's most likely to be updated to real ESM. This is more useful for old packages that may never get updated. +* JK: I think the opposite is true because it affects the upgrade path. +* JH: Seems better to have a clear boundary. +* JK: End user is broken vs inconvenienced. + + + +#### Unflag import.meta.resolve [#33464](https://github.com/nodejs/node/pull/33464) + +* GB: We landed import.meta.resolve a while ago. This is the analog to require.resolve in CJS. It is async. Now we have top-level-await you can do this at top-level. We also have behaviour for a trailing slash to resolve paths. It was added to load an asset that isn't a module, e.g. a template. It's a nice useful convenience. +* JH: I am fine with the use-case. Like an API for the package root directory. I think CJS and ESM are first-class for the foreseeable future. Resolution is not unique to a module system. Whatever we add needs to be for both. Don't conflate resolution with... I expect resolve in ESM to only give me things I can import. Same rules should apply to resolve and import. +* GBB: I understand the philosophy but there are so many differences. Any unknown file can be require()'d. +* JH: But the content is a property of the file. +* GBB: This is a different system that uses URLs. These don't resolve to disk. +* GB: Resolving unknown extensions in CJS does not work. +* GBB: My point is that we don't need a 1:1 match. +* JH: People file bugs on resolve all the time. It has the same name. +* MB: There is a WHATWG proposal 3871 `import.meta.resolveUrl` +* MB: https://github.com/whatwg/html/issues/3871 +* MB: It feels a bit premature for us to unflag this whilst a standard is being discussed about `import.meta` that is shared across environments. Top-level-await is behind a flag. First there was "let's not call it resolve". Maybe not naming it resolve would be better. If we want to share it, perhaps consider putting it on util - allows us to deprecate it in future. It's host-specific, but for shared things we should try to coordinate. TC39 doesn't own this. If there is an import.meta.resolve I would want them to be the same. URL created a huge amount of legacy for Node. Putting it on meta does not accomplish all our goals. If we want to say it is node-specific, maybe call it `import.meta.node.resolve`. But maybe no advantage over util. +* GB: In terms of getting buy-in, the driving use-case exist in server engines, Node has the first mover advantage. I'll post an issue on Deno. I would be careful being conservative. We should serve our users. `import.meta` is contextual - follows self-resolution. It's statically analysable. It's hard to build Node apps to detect assets. `require()` can be reassigned. It helps uses do something they can't do today. +* JH: But only for ESM users. +* GBB: In webpack, builtins are not maintained. But static syntax is. Leads to simplicity which aids adoption. +* GBB: Could we ship this unflagged as `importResolveUrlFromModule` utility function. Then if standardized later we add it to `import.meta`. +*no conclusion* + + +#### package.json resolution and exports [#33460](https://github.com/nodejs/node/issues/33460) +* MB: People keep getting bitten by this. +* WT: The original author came to Node Tooling slack to talk about this. It is very common to import pkg.json. Now if you have a tool drilling through the tree, exports breaks this. Other solutions exist in the thread. All of them place burden on toolmakers. Because exports is new, it's odd to break everyone and now tool makers have to update their code. +* JK: We knew about this problem. The pushback is not from the wide community. It's an import for a small part of the community. We can ask those few tool writers to adopt the new resolution feature in the interest of the vast majority of users. I personally don't care as much unless we make it a clear exception - not overridable. Which means nobody can decide how that magical specifier can be resolved. Makes exports harder to explain. Might be acceptable trade off. +* MB: exports is a new thing with edge cases. Today's expectations are based on old ways that may go away. Caveats makes educating people harder. I am not a fan of turning it on for that reason. +* WT: Other people feel like that. The concern is that in order for module authors to adopt this, they must all expose package.json to work with existing tools. +* GBB: Don't think that's true. +**cut short - will return at the next meeting** + +--- + +* Chartering the Modules team [#412](https://github.com/nodejs/modules/issues/412) diff --git a/doc/meetings/2020-06-17.md b/doc/meetings/2020-06-17.md new file mode 100644 index 0000000..e37b45c --- /dev/null +++ b/doc/meetings/2020-06-17.md @@ -0,0 +1,71 @@ +# Node.js Modules Team Meeting 2020-06-17 + +## Links + +* **Recording**: http://www.youtube.com/watch?v=hFfwRkR-L-I +* **GitHub Issue**: https://github.com/nodejs/modules/issues/528 +* **Minutes Google Doc**: https://docs.google.com/document/d/1FLK3nkCpxF6D-JFuoZlGpovF0lpSQ_8TP3wini6FsB0/edit + +## Present + +* Modules team: @nodejs/modules + +- Myles Borins (@MylesBorins) +- Jordan Harband (@ljharb) +- Darcy Clarke (@darcyclarke) +- Rob Palmer (@robpalme) + +## Agenda + +## Announcements + +*Extracted from **modules-agenda** labelled issues and pull requests from the **nodejs org** prior to the meeting. + +### nodejs/node + +#### Special treatment for package.json resolution and exports? [#33460](https://github.com/nodejs/node/issues/33460) + +- MB: There are ways to get an entrypoint but it may not be the package root. So heuristics may be needed to find package.json +- JH: package.jsons can be copied so you can't know for sure. API for bare-identifier to package.json provides a way for tools to work. +- MB: We could publish an ecosystem package - find-package-json - would be siblings at the root of node_modules. This matters when I want my dependency's package.json. If that hierarchy is guaranteed, direct dependencies should be peers. +- JH: Use-case is I'm a tool used in a project I don't control. So answer won't be relative to the tool's location. +- MB: So I'm a dependency of the project. Maybe I'm bundling. Instrumented by the top-level package and I need to find other dependencies. You need to set the root. +- JH: You can put node_modules at the root of your disk. +- MB: Everything is resolved to yourself. If overlapping. +- JH: Node already has the logic. You don't know where your dependencies will come from. +- MB: Why is this different to the root of your resolve? +- JH: With exports, you can't know for sure. Will show an example offline. +- MB: Let's try the ecosystem package approach first. +- JH: If it can 100% work in userland that's fine. +- MB: Some people may want to prevent package.json leaking to users. Or maybe if people are ok with it being public, we just change our defaults. +- GB: Depends how many tools are impacted. Softly feeling we don't need to do anything. +- MB: If exports is used to generate import maps, maybe package.json shouldn't be available. It's easier to remember the rule that "only exports get exported". +- JH: README can be used a hint. +- GB: API to get package root would permit fs.readFile to grab package.json +- MB: Not 100% but the basic algorithm to resolve the file and then crawl will probably always work. +- JH: Babel has an option that duplicates package.json +- GB: You don't look for package.json - you look for node_modules +- MB: If we had module.resolve to give a path, with a changeable root, you could always go up to node_modules then back down. +- JH: It's possible. Resolve already needs to duplicate the whole algorithm. +- MB: If we expose resolve, that gives you everything you need. NODE_PATHS may confuse here. +- JH: Even if it is doable in userland, it seems silly to not put it in Node. +- MB: I think we need a meta-issue to discuss this. Bring up a series of APIs at once. Maybe we want to introduce a top-level resolve module. My gut is that we don't need a new API for this. +resolveType() resolveRoot() resolvePath(). resolveRoot() may be better because it is environment-agnostic. +- JH: I remain convinced we should have a resolvePackageRoot() somewhere. +- MB: Maybe a getPackageMetadata. It could be synchronous. There is a large cost to putting this API in core. + + +#### module: CJS exports detection [#33416](https://github.com/nodejs/node/pull/33416) + +- GB: How do you define success? Maybe not everything was intended to be exported. +- MB: Maybe's there's a field in package.json to enable the heuristic? Less work than writing a wrapper. +- GB: We need to make it as good as possible then merge it in behind a flag to get feedback. +- JH: People are going to want to use this on abandoned packages. The heuristic is more likely to break due to a change in Babel that upsets the heuristic. +- MB: User research may be good - but biased responses to "do you want exports for CommonJS if it works X% of the time?" +- GB: Let's study guy's PR. +- MB: Releasing under an experimental flag seems worth exploring. +- GB: It uses a WASM library for parsing. +- MB: Can we land it non-observable when the flag is off? If so, let's do it. +- RP: Do we have many people complaining about lack of named exports for CJS? +- GB: Not really. A few issues. +- MB: Adding this experimental flag is nice. Combined with auto-extension flags, maybe that gives you the webpack experience as an opt-in mode. diff --git a/doc/meetings/2020-07-15.md b/doc/meetings/2020-07-15.md new file mode 100644 index 0000000..f23fa1b --- /dev/null +++ b/doc/meetings/2020-07-15.md @@ -0,0 +1,49 @@ +# Node.js Modules Team Meeting 2020-07-15 + +## Links + +* **Recording**: http://www.youtube.com/watch?v=LOHDJGs_FNs +* **GitHub Issue**: https://github.com/nodejs/modules/issues/532 +* **Minutes Google Doc**: https://docs.google.com/document/d/1iEMdSLFXIpEwtAzogOOcGQZgS87fZDerdRVnCSldjUM/edit + +## Present + +* Myles Borins (@MylesBorins) +* Guy Bedford (@guybedford) +* Wesley Wigham (@weswigham) +* Darcy Clarke (@darcyclarke) +* Bruce MacNaughton (@bmacnaughton) +* Julian Londono (@julian-londono) +* Jordan Harband (@ljharb) + +## Agenda + +## Announcements + +*Extracted from **modules-agenda** labelled issues and pull requests from the **nodejs org** prior to the meeting. + +### nodejs/node + +* esm: Modify getFormat and getSource loader hooks [#34144](https://github.com/nodejs/node/pull/34144) + +Crux of the issue, example: an https loader having to make a request in `getFormat` to get the resource format, then again in `getSource` to get the resource +Having state throughout the 4 loader hooks may be able to solve this +Pushing a tricky GC issue to loader hooks that may not be able to rely on getSource always being called after getFormat (especially once chaining exists). +PR should delve more into edge cases and document exploration further, covering possible alternatives. +Make sure PR works with Gus’s compositional loader PR [#33812](https://github.com/nodejs/node/pull/33812) + + +* Special treatment for package.json resolution and exports? [#33460](https://github.com/nodejs/node/issues/33460) + - Refs: + - add API for "package dir" [#516](https://github.com/nodejs/modules/issues/516) + - up for debate on the placement (ex. `module`, `require`, `import.meta`) + - proposed interface/method name: `resolvePackageRoot` + - ex. usage, `.resolvePackageRoot(import.meta.url)` & `.resolvePackageRoot(process.cwd())`, `.resolvePackageRoot(__dirname)` (?) + +* module: CJS exports detection [#33416](https://github.com/nodejs/node/pull/33416) + PR has been updated to use a flag. Should we let the PR work stall at this point? - no active interest in pursuing any escalation for the block. +- Concerns raised based on heuristic not being fully accurate. +- Discussed alternatives briefly. +- Suggestion to use a loader to provide the feature to users for further feedback. +- Snapshotting of CJS modules does not support loader interception - agreement that a PR to disable snapshotting and allow CJS mutation to allow a loader would be preferable to not being able to support CJS named exports. +- Discussion to open meta issue related to group handling of these types of consensus issues. diff --git a/doc/meetings/2020-07-29.md b/doc/meetings/2020-07-29.md new file mode 100644 index 0000000..d34dca7 --- /dev/null +++ b/doc/meetings/2020-07-29.md @@ -0,0 +1,99 @@ +# Node.js Modules Team Meeting 2020-07-29 + +## Links + +* **Recording**: http://www.youtube.com/watch?v=ZfEoSWy3Ljg +* **GitHub Issue**: https://github.com/nodejs/modules/issues/536 +* **Minutes Google Doc**: https://docs.google.com/document/d/12g-OB9ey8tXM27kJ_m4HP5ts7gqDMuP9WEkKUHUmGu4/edit + +## Present + +* Modules team: @nodejs/modules +* Jan Krems (@jkrems) +* Geoffrey Booth (@GeoffreyBooth) +* Guy Bedford (@guybedford) +* Myles Borins (@MylesBorins) +* Bradley Farias (@bmeck) +* Jordan Harband (@ljharb) +* Wesley Wigham (@weswigham) + +## Agenda + +## Announcements + +*Extracted from **modules-agenda** labelled issues and pull requests from the **nodejs org** prior to the meeting. + +### nodejs/node + +#### Policy conditions [#34414](https://github.com/nodejs/node/pull/34414) + +* Bradley: Trying to align policy and module resolution. One example `node:` vs. `nodejs:` as the protocol for built-ins. Personal preference is simply `node:`. While testing, I realized that "exports" condition traversal isn’t shared between cjs and esm. Looked at cjs code but tied closely to cjs internals, using exceptions instead of return values. +* Myles: No strong preference. Some people do prefer fewer characters (`node:`). Could also relate to built-in modules, even if it’s early in the standards process. +* Jan: Feels risky to pick `node:`. Prior discussions about using `node:` for builtins raised concerns that it is too generic. +* Bradley: As node evolves and may allow different languages, including `js` may become awkward. +* Myles: Loaders are experimental, so breaking change may be fine. Aliasing could be good. +* Jordan: Clarification: Does PR change nodejs to node? Which direction would be align? +* Bradley: Would prefer `node:` if we align. Could support both like we did with `getFormat`. +* Jordan: `node` is the name of the binary, using `node:` seems consistent. +* Myles: `node:` could be registered, not registered yet. +* Jan: Primary concern is consensus to allow the use of the same prefix for builtins in `require` expressions and `import` statements. In that context people had preferred `nodejs:`. +* Myles: If this group is okay with protocol, don’t see a reason why it couldn’t be the direction. +* Bradley: Will create 2nd PR to align protocol. May also refactor to reuse conditional logic. +* Myles: Would like to make sure major refactorings don’t interfere with backporting to 12 and 14. +* Bradley: Likely big refactor will not land cleanly. +* Myles: General question about what we want to backport, may want to delay big refactoring until important features landed. + +#### esm: Modify getFormat and getSource loader hooks [#34144](https://github.com/nodejs/node/pull/34144) + +* Julian: Update from last meeting - dug into alternatives and implications. Question about `transformSource` relationship with `getSource` change. Linked doc with result of exploration in PR. After looking into loader chaining PR, seems conceptually compatible. `transformSource` out of scope for PR. Any further thoughts on proposed solutions and moving forward with doc updates? +* (no fundamental concerns) +* Julian will, therefore, move on with updating documentation and continue formalizing PR + +#### Special treatment for package.json resolution and exports? [#33460](https://github.com/nodejs/node/issues/33460) + +* Bradley: Suggestion is to special-case `package.json` so it’s always exposed in `exports`. +* Jordan: Last time, we were looking at a PR to expose this data to tools outside of `exports` field resolution. + +#### module: CJS exports detection [#33416](https://github.com/nodejs/node/pull/33416) + +* Guy: Ongoing discussion about how to overcome existing objections. Latest was to explore if it’s possible to have a userland solution. Precondition was a PR to disable the ESM cache interactions in the CJS loader (https://github.com/nodejs/node/pull/34467). Wrote a userland loader to allow experimentation. +* Bradley: Would like to see some follow-up PRs. Haven’t seen strong desires in the ecosystem for CJS snapshotting, theoretical correctness concerns still exist. +* Wes: Community has been using named exports using “custom loader hooks”, no further proof necessary that people expect it. +* Guy: Still valuable to get feedback that this particular approach works for people. +* Jan: Most importantly, this isn’t exactly what people have been using in the ecosystem. Finding out if this reduced version is useful would help make the case that it’s “good enough”. +* Geoffrey: Have been working on tests to get a better idea about how good the detection works. Previous numbers were comparing it to all runtime properties but that isn’t necessarily what the actual expected named exports would be. Having a high number may overcome existing objections. Really about user expectations. +* Guy: One example is that it _should_ have full coverage for babel-generated outputs. +* Wes: Should be the same coverage for TypeScript-generated outputs. +* Geoffrey: Maybe it’s okay to “declare” all TS or babel generated sources as 100% because we know it covers their patterns. +* Guy: Good place to discuss the TS side of things? +* Wes: Likely the discord server (http://discord.gg/8CjzwY). +* Bradley: Well known edge case with `export * from`. +* Geoffrey: Shouldn’t be necessary to get to 100%, Gus should be okay with realistic but high coverage. +* Jan: Would it maybe be good to put aggressive bailing on the table so that partially correct results aren’t as much of a risk? +* Wes: Not good to speculate about Gus’ success criteria without talking to Gus. +* Guy: May be possible to get “100% of targeted exports” by detecting transpiled output that can be clearly identified. Help with testing would be great. Pushed 3 proposals, it’s been a lot. +* Jordan: Memory of Gus’ concerns was specifically about old code that’s unlikely to work. He wasn’t concerned as much about transpiled code. +* Guy: May be an argument to focus on transpiled code and actively ignore hand-written code. + +### nodejs/modules + +#### Subpath extension patterns and wildcard expansions [#535](https://github.com/nodejs/modules/issues/535) + +* Guy: Noticed a lot of component libraries and SVG icon libraries that can have 100s of directly importable files. Using `exports` field requires prefix/path matching because maintaining 100s of entries isn’t practical. Usually `exports` field contains entries without file extension but for these packages it has to be a specifier with file extension. Do we want to allow users to append extensions so they can use prefix matches but still write extension-less specifiers? +* Jordan: Agreed with desire. But we explicitly choose to not have extension searching. This drove us to very expansive lists. If we can find a way to make extensionless work, would be in favor. I don’t want people to use extensionless when importing from my packages. +* Guy: We did add extension searching for prefix matching but only for CJS. +* Jordan: If we care about consistent APIs, you always have to use exact paths. Can’t use prefix matching. Don’t want users to dirty their code with file extensions. +* Guy: What is the concern about current proposal? +* Jordan: Slash on RHS normally means “do what the module system normally does”. +* Guy: Better way than “normal” path export. Current proposal makes it more complicated. +* Jan: We don’t know where the ecosystem will be heading. Browsers require full relative paths, with file extensions. It may mean that in the future people do expect file extensions, in general. So extension-less bare imports may not be as desirable, may even stick out as “weird”. +* Bradley: Like the idea of “templates” or something else that’s more expressive. Shouldn’t try to pass judgement on what the future will look like. Explicit extensions allow for surprising rewrites like `import foo/x.json` being actually `file:///path/to/x.wasm`. Relates to import assertions, may break unexpectedly. +* Guy: Reason LHS doesn’t have wildcard. With both wildcard and path allows both, unclear precedent. +* Jan: Active error for path vs wildcard? +* Guy: Also different prefix length with nested prefixes. Can have further discussions on this. +* Jan: One property that we lose: With current import map proposal, we cannot convert this to an import map without scanning file system. +* Jordan: Not sure if we want to tie ourselves down by import map spec, may choose to pick dev ergonomics and accept heavier conversion cost to import map. + +#### add API for "package dir" [#516](https://github.com/nodejs/modules/issues/516) + +(Out of time) diff --git a/doc/meetings/2020-08-12.md b/doc/meetings/2020-08-12.md new file mode 100644 index 0000000..0dae583 --- /dev/null +++ b/doc/meetings/2020-08-12.md @@ -0,0 +1,83 @@ +# Node.js Modules Team Meeting 2020-08-12 + +## Links + +* **Recording**: http://www.youtube.com/watch?v=PDFFQnbFEcM +* **GitHub Issue**: https://github.com/nodejs/modules/issues/543 +* **Minutes Google Doc**: https://docs.google.com/document/d/1zjqjv8G9fDnpGG01yBS39-tRRZPQ5E8N5aset38FKKk/edit + +## Present + +* Myles Borins (@MylesBorins) +* Bradley Farias (@bmeck) +* Guy Bedford (@guybedford) +* Doron Tsur (@qballer) +* Julian Londono (@julian-londono) +* Jordan Harband (@ljharb) +* Jan Krems (@jkrems) + +## Agenda + +## Announcements + +*Extracted from **modules-agenda** labelled issues and pull requests from the **nodejs org** prior to the meeting. + +We unflagged Top-Level Await in Node.js 14.8.0. There is an open pull-request to unflag Top-Level Await in the REPL, but it needs to be reviewed on it’s own merit as it uses a completely different implementation from TLA in ESM. There is also an open discussion about support static import in the REPL, as it could share similar infrastructure to TLA. + +### nodejs/node + +* Policy conditions [#34414](https://github.com/nodejs/node/pull/34414) + + + +* Special treatment for package.json resolution and exports? [#33460](https://github.com/nodejs/node/issues/33460) + +JHB: lots of tools need access to package.json, with "exports" there is no reliable way to get it in every situation. 3 paths forward: +1. do nothing, packages must explicitly expose package.json +2. automatically/implicitly expose package.json +3. Add API to resolve a package root + +No PR yet, but it seems like 3 is a likely route (module.getPackageRoot()?). Seems sufficient to have the ecosystem to use that without relying on require/import. + +MBS: We don't have a way of loading JSON in ESM right now. It is being worked on/standardized. I could see value in some API that loads the meta-data without all the boilerplate. This is separate from when "exports" cannot resolve a thing. I recognize people are used to having the ability to load `package.json` without opting into anything. Perhaps a opt-out via setting it to "undefined" in the JSON would work. Tools that need to get what is / is not exposed package.json is the static source of what is /is not available and they have to do this. + +JKS: If we state that tools can use X to get the package.json, it would be surprising if there is a way to opt out. It would be surprising if you were allowed to opt-out. + +MBS: A lot of these tools are expecting more than just package.json at the root and are relying on our loaders to get them. It can be a slippery slope for all the assets. + +BFS: the semantics of any API are complex / the data may not exist + +JKS: concern is around publishing a popular package, which then opts-out and then breaks the npm ecosystem. + +BFS: yes, we saw this with is-promise, but the breakage was due to implicit not explicit choice to prevent access to package.json + +JKS: Making an API to get the package root is useful one way or another. The only way for a coffeescript loader to determine the type of the package is that the loader needs to replicate behavior or make a fake filename and call the hook. Such an API would allow a loader to reuse a lot of the builtin machinery to do these tasks. + +JHB: Will proceed making PR for API. + +* module: CJS exports detection [#33416](https://github.com/nodejs/node/pull/33416) + +GBD: Geoffrey did analysis/numbers based upon specific success criteria based upon name exports intent from readmes. The block on the PR landing is from Gus on criteria must be for exact output rather than the analysis. Unclear if we can get consensus on this. Suggestion to only have criteria to detect transpiled modules to get 90+% case needed. Are we still defining the criteria about what users expect / esModules check. 81% for esModule currently 90+% for readme. +MBS: Geoffreys last comment for esModule > 99% success rate for esModule compared to docs. Accuracy seems to be an acceptable level. Any small work we can do to improve this would be good. + +GBD: *lack of clarity on if it is 99% or 81% accuracy*. If it is in the readme 99% accurate. + +MBS: maybe we can ignore all modules lacking esModule. + +JKS: numbers may be biased for a high percentage rather than package author expectation. + +### nodejs/modules + +* Subpath extension patterns and wildcard expansions [#535](https://github.com/nodejs/modules/issues/535) + +MBS: maybe we should look at stuff, dynamism is concerning. Perhaps things like an extension map in the package JSON as a separate feature. + +JHB: prioritized static in loading, we should re-evaluate usability vs static. + +* import assertions RFC [#427](https://github.com/nodejs/modules/issues/427) + +* Import Maps and Node.js [#544](https://github.com/nodejs/modules/issues/544) + +* Removing `getFormat` hook [#34114](https://github.com/nodejs/node/pull/34144/#issuecomment-666716738) + +Issue for this created [here](https://github.com/nodejs/node/issues/34753) diff --git a/doc/meetings/2020-08-26.md b/doc/meetings/2020-08-26.md new file mode 100644 index 0000000..ace44a9 --- /dev/null +++ b/doc/meetings/2020-08-26.md @@ -0,0 +1,64 @@ + +# Node.js Modules Team Meeting 2020-08-26 + +## Links + +* **Recording**: http://www.youtube.com/watch?v=9PU3XLIlzw0 +* **GitHub Issue**: https://github.com/nodejs/modules/issues/548 +* **Minutes Google Doc**: https://docs.google.com/document/d/1h-lxJGyzmBAKhiweg3hesXZ1j5PLcGFEQPZJl4rpkKc/ + +## Present + +* Myles Borins (@Myles Borins) +* Christopher Hiller (@boneskull) +* Doron Tsur (@qballer) +* Wesley Wigham (@weswigham) +* Guy Bedford (@guybedford) +* Jordan Harband (@ljharb) + +## Agenda + +## Announcements + +* Discussion about proxyquire + +- Christopher: + - Module level mocking in tests is an issue with ESM + - With common.js and requires and the cache you can do this fairly easily + - There are various modules you can use such a proxyquire + - It is nice to be able to just mock a module + - ESM makes this difficult + - I’ve seen some loader implementations out there that allow you to do this sort of thing, although I haven’t been able to play with them to figure out how sufficient they are. + - One challenge, the requirement of using the loader flag at all. We don’t necessarily know if / when we are going to start mocking things. As it’s implemented right now you can only have one loader. If you want to use a loader for something like typescript you are in a tough position + - Even with chaining loaders it puts a lot of responsibility on the user / test runner to manage this such as spawning a subprocess for every test even if the loader isn’t being used. Not very ergonomic + +### nodejs/node + +* esm: Modify ESM Experimental Loader Hooks [#34753](https://github.com/nodejs/node/issues/34753) + +Skipped + +* Special treatment for package.json resolution and exports? [#33460](https://github.com/nodejs/node/issues/33460) + +Tabled for now will come back if we have time + +* module: CJS exports detection [#33416](https://github.com/nodejs/node/pull/33416) + + + +### nodejs/modules + +* Import Maps and Node.js [#544](https://github.com/nodejs/modules/issues/544) + + + +* Add Doron Tsur as observer [#542](https://github.com/nodejs/modules/pull/542) + +Done added + +* Subpath extension patterns and wildcard expansions [#535](https://github.com/nodejs/modules/issues/535) + +* import assertions RFC [#427](https://github.com/nodejs/modules/issues/427) + + + diff --git a/doc/meetings/2020-09-09.md b/doc/meetings/2020-09-09.md new file mode 100644 index 0000000..91bb8ca --- /dev/null +++ b/doc/meetings/2020-09-09.md @@ -0,0 +1,42 @@ +# Node.js Modules Team Meeting 2020-09-09 + +## Links + +* **Recording**: http://www.youtube.com/watch?v=hh76jaYtrTE +* **GitHub Issue**: https://github.com/nodejs/modules/issues/552 +* **Minutes Google Doc**: https://docs.google.com/document/d/1EgBV_oRONelVM4nLWuzWJJEyDUqmqFF5FR0OwpqCKQU/ + +## Present + +* Modules team: @nodejs/modules +* Geoffrey Booth (@GeoffreyBooth) +* Myles Borins (@MylesBorins) +* Doron Tsur (@qballer) +* Guy Bedford (@guybedford) +* Jordan Harband (@ljharb) +* Bradley Farias (@bmeck) + +## Agenda + +## Announcements + +*Extracted from **modules-agenda** labelled issues and pull requests from the **nodejs org** prior to the meeting. + +### nodejs/node + +* module: exports patterns [#34718](https://github.com/nodejs/node/pull/34718) + +- Is the complexity worth it? Maybe! + +* Special treatment for package.json resolution and exports? [#33460](https://github.com/nodejs/node/issues/33460) + +* module: CJS exports detection [#33416](https://github.com/nodejs/node/pull/33416) + +* esm: Modify ESM Experimental Loader Hooks [#34753](https://github.com/nodejs/node/issues/34753) + +### nodejs/modules + +* Enabling import meta resolve by default? [#550](https://github.com/nodejs/modules/issues/550) +* import assertions RFC [#427](https://github.com/nodejs/modules/issues/427) +* Mocking use cases [#549](https://github.com/nodejs/modules/issues/549) +* hot reload modules w/ es6 modules [#459](https://github.com/nodejs/modules/issues/459) diff --git a/doc/meetings/2020-10-07.md b/doc/meetings/2020-10-07.md new file mode 100644 index 0000000..9a6741a --- /dev/null +++ b/doc/meetings/2020-10-07.md @@ -0,0 +1,45 @@ +# Node.js Modules Team Meeting 2020-10-07 + +## Links + +* **Recording**: https://www.youtube.com/watch?v=Kgq08dm7kkM +* **GitHub Issue**: https://github.com/nodejs/modules/issues/557 + +## Present + +* Modules team: @nodejs/modules + +## Agenda + +## Announcements + +*Extracted from **modules-agenda** labelled issues and pull requests from the **nodejs org** prior to the meeting. + +### nodejs/node + +* esm: Modify ESM Experimental Loader Hooks [#34753](https://github.com/nodejs/node/issues/34753) + +TL;DR: Geoffrey is working on a proposed redesign of loader hooks, including how they would work with different kinds of chaining use cases. + + +* Special treatment for package.json resolution and exports? [#33460](https://github.com/nodejs/node/issues/33460) + +ljharb@ is working on it. Removing the label from upstream Issue. + +### nodejs/modules + +* Empty path matching in patterns [#553](https://github.com/nodejs/modules/issues/553) + + +* Enabling import meta resolve by default? [#550](https://github.com/nodejs/modules/issues/550) + +Punted and label removed + +* Mocking use cases [#549](https://github.com/nodejs/modules/issues/549) + +Punted and label removed + + +* hot reload modules w/ es6 modules [#459](https://github.com/nodejs/modules/issues/459) + +Punted and label removed diff --git a/doc/meetings/2020-10-21.md b/doc/meetings/2020-10-21.md new file mode 100644 index 0000000..bf761ae --- /dev/null +++ b/doc/meetings/2020-10-21.md @@ -0,0 +1,38 @@ +# Node.js Modules Team Meeting 2020-10-21 + +## Links + +* **Recording**: https://www.youtube.com/watch?v=tGosRXDqdRk +* **GitHub Issue**: https://github.com/nodejs/modules/issues/563 + +## Present + +* Myles Borins (@MylesBorins) +* Geoffrey Booth (@GeoffreyBooth) +* Bradley Farias (@bmeck) +* Wesley Wigham (@weswigham) +* Jordan Harband (@ljharb) +* Doron Tsur(@qballer) + +## Agenda + +## Announcements + +*Extracted from **modules-agenda** labelled issues and pull requests from the **nodejs org** prior to the meeting. + +### nodejs/modules + +* Experimental status [#564](https://github.com/nodejs/modules/issues/564) + * Separating out all the specific features + * Missing VM? + * Incomplete: no way yet to do the equivalent of `node --input-type=module --eval ` the way we can for CommonJS via vm.Script + * No hot module reload + * Gil found a hacky way to achieve this via query strings, but since we can’t remove modules from the ESM cache, this will lead to the Node process gradually using more and more memory the longer it’s running and a developer uses HMR (so basically it’s possible now but only with a memory leak) + * Instrumentation + * Missing chaining loaders, missing events for load ops + * separate package features (affect both CJS/ESM) from ESM itself? + * +* Empty path matching in patterns [#553](https://github.com/nodejs/modules/issues/553) + + + diff --git a/doc/new-implementation-announcement.md b/doc/new-implementation-announcement.md new file mode 100644 index 0000000..92c2bc9 --- /dev/null +++ b/doc/new-implementation-announcement.md @@ -0,0 +1,99 @@ +# Announcing a new `--experimental-modules` + +Back in 2017, Node.js 8.9.0 shipped experimental support for [ECMAScript modules](https://hacks.mozilla.org/2018/03/es-modules-a-cartoon-deep-dive/), known for their `import` and `export` statements. This support was behind the flag `--experimental-modules`. + +A lot has happened since then. All major browsers [now support](https://caniuse.com/#feat=es6-module) ECMAScript modules (ES modules) via `