Description
When building large purescript codebases such as https://github.com/erlscripten/erlps-stdlib it's very easy to get killed by OOM - The modules are compiled in parallel usually exceeding the amount of RAM available on the system(I will open separate issues with the results of my investigation/profiling data - for instance the lexer right now uses up a lot of memory on large files). What is really troublesome is that if the compilation gets killed by OOM after compiling a lot of heavy modules then on the next run they will get recompiled although they compiled successfully. After investivating the purescript codebase the cache-db.json file is only updated after all of the compiler tasks either succeed or failed - this is not a problem for small projects but when the compilation times become significant(https://github.com/erlscripten/erlps-stdlib) it's really troublesome :(
To Reproduce
- Compile a project with modules M_1, M_2, ..., M_N
- The compiler compiles M_1, M_2, ..., M_N-1
- While compiling M_N the compiler gets killed either by SIGTERM or OOM
- After restarting the build modules M_1, M_2, ..., M_N-1 are being recompiled again although the generated code is already present on the drive
Expected behavior
- Compile a project with modules M_1, M_2, ..., M_N
- The compiler compiles M_1, M_2, ..., M_N-1
- While compiling M_N the compiler gets killed either by SIGTERM or OOM
- After restarting the build only M_N gets build
PureScript version
Tried both on 0.13.8 and on the current master branch :(
Description
When building large purescript codebases such as https://github.com/erlscripten/erlps-stdlib it's very easy to get killed by OOM - The modules are compiled in parallel usually exceeding the amount of RAM available on the system(I will open separate issues with the results of my investigation/profiling data - for instance the lexer right now uses up a lot of memory on large files). What is really troublesome is that if the compilation gets killed by OOM after compiling a lot of heavy modules then on the next run they will get recompiled although they compiled successfully. After investivating the purescript codebase the cache-db.json file is only updated after all of the compiler tasks either succeed or failed - this is not a problem for small projects but when the compilation times become significant(https://github.com/erlscripten/erlps-stdlib) it's really troublesome :(
To Reproduce
Expected behavior
PureScript version
Tried both on 0.13.8 and on the current master branch :(