Repository navigation
[BUG] npm i in workspace runs prepare script concurrently #4100
Description
Is there an existing issue for this?
- I have searched the existing issues
This issue exists in the latest npm version
- I am using the latest npm
Current Behavior
npm i runs Prepare script concurrently across all workspaces.
This causes major problems when using TypeScript and having one workspace dependent on another, since the build (prepare script) runs concurrently, there is no way to ensure the prepare command is done on one workspace before running on the next.
Expected Behavior
It to actually follow what the documentation says (https://docs.npmjs.com/cli/v7/using-npm/workspaces#running-commands-in-the-context-of-workspaces):
Nothing in there says anything about commands being run concurrently, or at the same time.
Steps To Reproduce
- Create 2 workspaces
- Add a prepare command to each where the second is reliant on the first completing
- Run
npm i - Notice that everything fails since the commands are run concurrently
Environment
- npm: 8.1.0
- Node: v16.13.0
- OS: macOS 12.1 Beta (21C5039b)
- platform: MacBook Pro M1 Max
- npm config:
; "user" config from /Users/charliefish/.npmrc
//npm.pkg.github.com/:_authToken = (protected)
//registry.npmjs.com/:_authToken = (protected)
registry = "https://npm.network.charlie.fish/"
; node bin location = /Users/charliefish/.nvm/versions/node/v16.13.0/bin/node
; cwd = /Users/charliefish
; HOME = /Users/charliefish
; Run `npm config ls -l` to show all defaults.Reacted by Grigory Streltsov
Activity
fishcharlie commentedon Dec 1, 2021
If anyone is able to provide a workaround to this I'd really appreciate it.
fishcharlie commentedon Dec 6, 2021
It's been over a week since I've posted this. Can I please get a reply here? This is a major blocker for me.
fishcharlie commentedon Dec 13, 2021
@ljharb Sorry to tag you randomly here. I notice you've been pretty active in replying to other issues.
Is there any chance you could point me in the right direction here? Who on the npm team would be best to assist here?
This is a major blocker for my project currently.
Thanks in advance!!
ljharb commentedon Dec 13, 2021
I'm not sure who's owning workspaces atm, sorry.
MylesBorins commentedon Dec 13, 2021
Hey @fishcharlie thanks for your patience. Had to coordinate between a couple people being out of office to get an answer
The docs you linked are referring to running commands in workspaces, not the internal lifecycle scripts that happen during reification. So for instance if you were to run
npm run somecommand --wsthensomecommandwould be ran in every workspace in the order they appear in your package.json.As far as deterministically ordering
preparescripts, this is not possible. Here is a comment from a previous issue discussion why it can not happen in the current npm version #3034fishcharlie commentedon Dec 13, 2021
@MylesBorins Thanks for the reply!
Got it. I think the documentation can be improved in this regard to make that more clear.
I guess this leads to a question about how I should be doing this instead? I don't think my use case and what I'm trying to achieve is that out there (with having a workspace, with a primary project and dependencies, all in TypeScript).
If you look at the npm documentation for prepare, once of the use cases listed is:
Compiling CoffeeScript source code into JavaScript. Arguably compiling TypeScript code into JS is the same concept. So based on that, doing mytscbuild step within thepreparecommand is correct.However, because my primary package in my workspace has a dependency on the build step from the utilities package being complete first, that leads to the issues mentioned here.
So is there something I should change in my workspace settings to do this correctly?
Although I understand the deadlocking situation presented in #3034, it's just as valid to not have circular dependencies, and require the
preparescript to be run sequentially.It sounds like a workaround would be to use
--foreground-scriptsmaybe?? But there is no way within thepackage.jsonto set that option as the default, and as a result, contributors to the open source project will get confused when trying to runnpm installwith a bunch of errors. So I'm not sure that is a realistic workaround.Although everything you said makes sense, I'm still super confused about how to handle this for dynamoose.
I understand that this is getting out of the scope of "it's a bug" and getting into more of "how should I handle this", which might be out of scope for GitHub Issues. But I do think my situation is a common/valid use case that npm needs to support or consider.
fishcharlie commentedon Jan 4, 2022
@MylesBorins Following up on this. Is there anything I can clarify in my previous message?
lukekarrys commentedon Feb 27, 2022
Running
npm config set foreground-scripts=true --location=projectshould work to set that as a project level config in an.npmrcfile. If you commit this file, it will be applied to all commands run in the project for contributors.I'm removing the bug label but keeping this open with the following action items:
--foreground-scriptsmarijnh commentedon May 3, 2023
Are you clear on how big of a damper this is on the usefulness of workspaces? In TypeScript setups, or sets of modules that use
bincommands from each other, etc, there doesn't seem to be a way to reliably install the workspace, at all.--foreground-scriptsdoesn't appear to provide control over the order in which install scripts are run either. The order in theworkspacesfield doesn't seem to be taken into account there. So even in a completely trivial setup like below, I have not found a way to forceb's install script to run beforea's.package.json:{"workspaces": ["b", "a"]}a/package.json:{ "name": "test-a", "version": "1.0.0", "scripts": { "prepare": "echo prep A" } }b/package.json:{ "name": "test-b", "version": "1.0.0", "scripts": { "prepare": "echo prep B" } }If I run
npm i --foreground-scripts(with npm 9.6.5), A runs before B.docs(scripts): document that prepare scripts run concurrently in work…