Skip to content

FOUR-31789: Add Docker streaming mode for script execution and ensure container cleanup on failure#8867

Open
AugustoLopezProcess wants to merge 3 commits into
feature/FOUR-30918from
feature/FOUR-31789
Open

FOUR-31789: Add Docker streaming mode for script execution and ensure container cleanup on failure#8867
AugustoLopezProcess wants to merge 3 commits into
feature/FOUR-30918from
feature/FOUR-31789

Conversation

@AugustoLopezProcess

@AugustoLopezProcess AugustoLopezProcess commented Jun 15, 2026

Copy link
Copy Markdown
Contributor

Issue & Reproduction Steps

ProcessMaker scripts can run in Docker using binding or copying modes (PROCESSMAKER_SCRIPTS_DOCKER_MODE). Copying mode creates a container, copies files in/out with docker cp, and removes the container — which adds overhead per script run. There is also no reliable cleanup when execution fails, which can leave orphaned containers behind.
To reproduce the container cleanup issue (copying mode):

  1. Set PROCESSMAKER_SCRIPTS_DOCKER_MODE=copying in .env.
  2. Run a script task whose Docker container fails during docker start (or otherwise throws before cleanup).
  3. Observe that docker rm is not called and the container may remain on the host.
    To reproduce the need for streaming mode:
  4. Use a script executor image that includes /opt/executor/run-stream.sh.
  5. Set PROCESSMAKER_SCRIPTS_DOCKER_MODE=streaming.
  6. Run a script task and confirm inputs/outputs are exchanged without per-file docker cp operations.

Solution

  • Added ScriptDockerStreamingFilesTrait to execute scripts by streaming a ustar tar archive of input files via stdin and reading output files from a tar archive on stdout.
  • Wired the new mode into ScriptRunners\Base so PROCESSMAKER_SCRIPTS_DOCKER_MODE=streaming selects executeStreaming, with automatic fallback to copying when the image does not support /opt/executor/run-stream.sh.
  • Implemented PHP-enforced timeout handling in runStreamingContainer() (avoids shell timeout, which is unavailable on some hosts like macOS and breaks proc_open stdin piping).
  • Hardened ScriptDockerCopyingFilesTrait with try/finally and docker rm -f so containers are removed even when script execution fails.
  • Added unit tests for tar build/parse/validation and feature tests with a mocked Docker binary covering streaming execution, fallback to copying, and container cleanup on failure.

How to Test

Automated tests:

./vendor/bin/phpunit tests/unit/ProcessMaker/Models/ScriptDockerStreamingFilesTraitTest.php
./vendor/bin/phpunit tests/Feature/Docker/ScriptDockerStreamingTest.php

###Manual testing (streaming mode):
Use a script executor Docker image that includes /opt/executor/run-stream.sh.
Set in .env:
PROCESSMAKER_SCRIPTS_DOCKER_MODE=streaming
Run a process with a script task (e.g. PHP/Python executor).
Confirm the script completes successfully and returns the expected output.
Check logs for executeMethod: executeStreaming.

Manual testing (fallback):

Use an older executor image without run-stream.sh.
Keep PROCESSMAKER_SCRIPTS_DOCKER_MODE=streaming.
Run a script task and confirm it still succeeds via copying fallback.
Check logs for Docker image does not support streaming, falling back to copying.

Manual testing (container cleanup):

Set PROCESSMAKER_SCRIPTS_DOCKER_MODE=copying.
Trigger a script failure during container start.
Confirm docker ps -a does not show a leftover container from that run.

Regression:

Verify existing binding and copying modes still work with their current .env settings.

Related Tickets & Packages

Code Review Checklist

  • I have pulled this code locally and tested it on my instance, along with any associated packages.
  • This code adheres to ProcessMaker Coding Guidelines.
  • This code includes a unit test or an E2E test that tests its functionality, or is covered by an existing test.
  • This solution fixes the bug reported in the original ticket.
  • This solution does not alter the expected output of a component in a way that would break existing Processes.
  • This solution does not implement any breaking changes that would invalidate documentation or cause existing Processes to fail.
  • This solution has been tested with enterprise packages that rely on its functionality and does not introduce bugs in those packages.
  • This code does not duplicate functionality that already exists in the framework or in ProcessMaker.
  • This ticket conforms to the PRD associated with this part of ProcessMaker.

@processmaker-sonarqube

Copy link
Copy Markdown

Quality Gate passed Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
0.0% Duplication on New Code

See analysis details on SonarQube

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant