container-python/sst

This branch is one test of the deploy benchmarks: the Python container API, deployed with SST. It holds only that application and the tool's configuration, so it deploys as it is.

node branches.mjs on main generates this branch from main's fixtures and tool modules. Don't edit it here: change main and generate the branches again.

SST 4.17.2, installed by npm ci. sst install adds its AWS provider during setup.

• AWS_PROFILE: a profile for the disposable AWS account.

You need Node.js 22 or newer, npm, Git, Docker with BuildKit, zip, the AWS CLI and a disposable AWS account. The AWS resources are created in eu-west-1. Run the harness from main, with the credentials above:

git clone https://github.com/stacktape/deploy-benchmarks && cd deploy-benchmarks
npm ci
node bench.mjs setup   sst container-python             # write work/bmk-sst-container-python/ and deploy it once (not measured)
node bench.mjs package sst container-python --runs 10   # record the image size
node bench.mjs deploy  sst container-python --runs 10   # deploy a code change from this machine
node report.mjs                                         # median, fastest and slowest run
node bench.mjs remove  sst container-python             # delete everything setup created

setup writes the files of this branch to work/bmk-sst-container-python/ and commits them as the same commit that this branch has.

Each run is a pair: a first push with an empty build cache, then a second push 15 minutes after the first one finished. The clock starts when git push returns.

1. Fork the repository with all branches: untick "Copy the main branch only".
2. Turn on GitHub Actions in your fork. This branch's .github/workflows/deploy.yml deploys every push to the branch.
3. Add the repository secret AWS_ROLE_ARN. AWS_ROLE_ARN is a role in the disposable account that the fork can assume through GitHub OIDC.
4. Sign gh in to an account that can manage the fork's Actions caches (gh auth login or GH_TOKEN). The first push of each pair deletes them with gh cache delete --all.
5. Run the commands below. The project in work/ has the same commit as this branch, so its pushes continue your fork's branch.

export BENCH_FORK=https://github.com/<you>/deploy-benchmarks   # setup adds it as the project's origin
node bench.mjs setup sst container-python                      # skip if you already ran it with BENCH_FORK set
node bench.mjs push sst container-python --runs 10
node report.mjs

From a checkout of this branch, without the harness:

npm ci
npx sst install
npx sst deploy --stage bmk
# delete everything:
npx sst remove --stage bmk

Each run appends one JSON line to results/package.jsonl or results/deploy.jsonl. Times are in milliseconds and sizes in bytes. Every line also records the tool version, the machine and the command log.

{ "app": "container-python", "tool": "sst", "bytes": <compressed image size>, "artifacts": [...] }
{ "app": "container-python", "tool": "sst", "journey": "cli", "run": 1, "change": "handlers", "changedTargets": 1, "readyMs": <time until the new code answers>, "commandMs": <deploy command duration>, "businessOk": true, "unchangedOk": true }
{ "app": "container-python", "tool": "sst", "journey": "git-cold", "run": 1, "change": "handlers", "changedTargets": 1, "readyMs": <time until the new code answers>, "coldReset": "scripted", "businessOk": true, "unchangedOk": true }
{ "app": "container-python", "tool": "sst", "journey": "git-warm", "run": 1, "change": "handlers", "changedTargets": 1, "readyMs": <time until the new code answers>, "businessOk": true, "unchangedOk": true }

node report.mjs prints the median with the fastest and slowest run of each series. Runs whose business check failed are listed and left out.

Pending. The results will be published in the main README after the measurement runs on a dedicated EC2 machine.