We switched in our project from vm2 to isolated-vm recently. This worked fine, but installation slowed down a lot and I'm experiencing some troubles getting it installed on small machines (1 Core with 256 MiB of RAM). It turns out, that the kernel kills some c++(1) processes when installing with npm install isolated-vm due to an out of memory condition. I've tried to install it with --ignore-scripts and build isolated-vm manually, which worked and led to the following results:
Building with a single thread:
$ time node-gyp rebuild --release -j max
[...]
gyp info spawn args [ 'BUILDTYPE=Release', '-C', 'build', '--jobs', 1 ]
[...]
537.04 real 187.85 user 22.61 sys
But when increasing the number of threads it speed up with 2 threads and starts threshing with 3 threads:
$ JOBS=2 time node-gyp rebuild --release
[...]
256.51 real 166.68 user 10.87 sys
$ JOBS=3 time node-gyp rebuild --release
[...]
557.14 real 244.18 user 84.46 sys
Building it with more than 3 threads, did not succeeed!
Always building it manually, is not an option, because there might be other node modules, that have installation scripts which might get skipped unintentionally. What could be a solution to this? I see, that the following should be possible:
- Replace
-j 4 (parallel build with 4 threads) in package.json's "install" script with -j max. This would automatically scale the number of threads. Unfortunately I couldn't test it on a big machines, so this might be an obstacle.
- Remove the parallel
node-gyp invocation in package.json completely. This might slow down the installation on bigger machines, but it is possible to add the JOBS=4 environment variable when installing (or add JOBS=max).
There might be other solutions, that I'm no aware of. Not only small machines, but containerized builds or cross-platform builds with qemu could benefit from this, too.
Given the options above, please remove the parallelism in the install script.
We switched in our project from
vm2toisolated-vmrecently. This worked fine, but installation slowed down a lot and I'm experiencing some troubles getting it installed on small machines (1 Core with 256 MiB of RAM). It turns out, that the kernel kills somec++(1)processes when installing withnpm install isolated-vmdue to an out of memory condition. I've tried to install it with--ignore-scriptsand buildisolated-vmmanually, which worked and led to the following results:Building with a single thread:
But when increasing the number of threads it speed up with 2 threads and starts threshing with 3 threads:
Building it with more than 3 threads, did not succeeed!
Always building it manually, is not an option, because there might be other node modules, that have installation scripts which might get skipped unintentionally. What could be a solution to this? I see, that the following should be possible:
-j 4(parallel build with 4 threads) inpackage.json's "install" script with-j max. This would automatically scale the number of threads. Unfortunately I couldn't test it on a big machines, so this might be an obstacle.node-gypinvocation inpackage.jsoncompletely. This might slow down the installation on bigger machines, but it is possible to add theJOBS=4environment variable when installing (or addJOBS=max).There might be other solutions, that I'm no aware of. Not only small machines, but containerized builds or cross-platform builds with
qemucould benefit from this, too.Given the options above, please remove the parallelism in the install script.