pipeline-manager: rust compilation cleans up target#6410
Conversation
The Rust compilation cleanup now checks whether the disk on which it takes place it about to reach a usage threshold: - If it exceeds 70%, a warning is printed. - If it exceeds 90%, the `target` directory is removed if it is 5% or more of the total disk space. The compiler server is restarted if the form factor supports it. Signed-off-by: Simon Kassing <simon.kassing@feldera.com>
| RustCompilationCleanupError::Utility(e) => { | ||
| error!("Rust worker {worker_id}: compilation cleanup failed: utility error occurred: {e}"); | ||
| } | ||
| RustCompilationCleanupError::TargetCleared => { |
There was a problem hiding this comment.
do you want to detect the case when even one compilation on a clean disk fails and you should not retry?
There was a problem hiding this comment.
Hmm, so I added a first logic that generally makes sense. I agree that the assumptions can be made tighter, will do more testing before merging this PR.
|
|
||
| // (2) For each possible compilation profile, clean up their target artifacts | ||
| // (2) If the disk that the target directory resides on has reached a certain threshold | ||
| // the entire target directory will get removed. There is not a cleaner way currently to |
| // (2) For each possible compilation profile, clean up their target artifacts | ||
| // (2) If the disk that the target directory resides on has reached a certain threshold | ||
| // the entire target directory will get removed. There is not a cleaner way currently to | ||
| // more fine-combined clean up old dependencies. |
There was a problem hiding this comment.
fine-grained? targeted?
I made a verb out of "go over it with a fine tooth comb", but the analogy actually is not correct either. Will change it
It'll need some more downstream testing before I merge this PR. Restarting is the same as it temporarily going down (it goes through continuous cycle of |
The Rust compilation cleanup now checks whether the disk on which it takes place it about to reach a usage threshold:
targetdirectory is removed if it is 5% or more of the total disk space. The compiler server is restarted if the form factor supports it.PR information