FOUR-31789: Add Docker streaming mode for script execution and ensure container cleanup on failure#8867
Open
AugustoLopezProcess wants to merge 3 commits into
Open
FOUR-31789: Add Docker streaming mode for script execution and ensure container cleanup on failure#8867AugustoLopezProcess wants to merge 3 commits into
AugustoLopezProcess wants to merge 3 commits into
Conversation
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.




Issue & Reproduction Steps
ProcessMaker scripts can run in Docker using
bindingorcopyingmodes (PROCESSMAKER_SCRIPTS_DOCKER_MODE). Copying mode creates a container, copies files in/out withdocker cp, and removes the container — which adds overhead per script run. There is also no reliable cleanup when execution fails, which can leave orphaned containers behind.To reproduce the container cleanup issue (copying mode):
PROCESSMAKER_SCRIPTS_DOCKER_MODE=copyingin.env.docker start(or otherwise throws before cleanup).docker rmis not called and the container may remain on the host.To reproduce the need for streaming mode:
/opt/executor/run-stream.sh.PROCESSMAKER_SCRIPTS_DOCKER_MODE=streaming.docker cpoperations.Solution
ScriptDockerStreamingFilesTraitto execute scripts by streaming a ustar tar archive of input files via stdin and reading output files from a tar archive on stdout.ScriptRunners\BasesoPROCESSMAKER_SCRIPTS_DOCKER_MODE=streamingselectsexecuteStreaming, with automatic fallback to copying when the image does not support/opt/executor/run-stream.sh.runStreamingContainer()(avoids shelltimeout, which is unavailable on some hosts like macOS and breaksproc_openstdin piping).ScriptDockerCopyingFilesTraitwithtry/finallyanddocker rm -fso containers are removed even when script execution fails.How to Test
Automated tests:
###Manual testing (streaming mode):
Use a script executor Docker image that includes
/opt/executor/run-stream.sh.Set in
.env:PROCESSMAKER_SCRIPTS_DOCKER_MODE=streamingRun a process with a script task (e.g. PHP/Python executor).
Confirm the script completes successfully and returns the expected output.
Check logs for executeMethod:
executeStreaming.Manual testing (fallback):
Use an older executor image without
run-stream.sh.Keep
PROCESSMAKER_SCRIPTS_DOCKER_MODE=streaming.Run a script task and confirm it still succeeds via copying fallback.
Check logs for Docker image does not support streaming, falling back to copying.
Manual testing (container cleanup):
Set
PROCESSMAKER_SCRIPTS_DOCKER_MODE=copying.Trigger a script failure during container start.
Confirm
docker ps -adoes not show a leftover container from that run.Regression:
Verify existing binding and copying modes still work with their current
.envsettings.Related Tickets & Packages
/opt/executor/run-stream.shCode Review Checklist