Skip to content

When the compiler get's killed by OOM/SIGTERM the cache-db.json file is not updated #3995

@gorbak25

Description

@gorbak25

Description

When building large purescript codebases such as https://github.com/erlscripten/erlps-stdlib it's very easy to get killed by OOM - The modules are compiled in parallel usually exceeding the amount of RAM available on the system(I will open separate issues with the results of my investigation/profiling data - for instance the lexer right now uses up a lot of memory on large files). What is really troublesome is that if the compilation gets killed by OOM after compiling a lot of heavy modules then on the next run they will get recompiled although they compiled successfully. After investivating the purescript codebase the cache-db.json file is only updated after all of the compiler tasks either succeed or failed - this is not a problem for small projects but when the compilation times become significant(https://github.com/erlscripten/erlps-stdlib) it's really troublesome :(

To Reproduce

  1. Compile a project with modules M_1, M_2, ..., M_N
  2. The compiler compiles M_1, M_2, ..., M_N-1
  3. While compiling M_N the compiler gets killed either by SIGTERM or OOM
  4. After restarting the build modules M_1, M_2, ..., M_N-1 are being recompiled again although the generated code is already present on the drive

Expected behavior

  1. Compile a project with modules M_1, M_2, ..., M_N
  2. The compiler compiles M_1, M_2, ..., M_N-1
  3. While compiling M_N the compiler gets killed either by SIGTERM or OOM
  4. After restarting the build only M_N gets build

PureScript version

Tried both on 0.13.8 and on the current master branch :(

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions