container-python/cdk

This branch is one test of the deploy benchmarks: the Python container API, deployed with AWS CDK. It holds only that application and the tool's configuration, so it deploys as it is.

node branches.mjs on main generates this branch from main's fixtures and tool modules. Don't edit it here: change main and generate the branches again.

AWS CDK CLI 2.1145.0 with aws-cdk-lib 2.273.0, constructs 10.8.1 and esbuild 0.28.2, all installed by npm ci.

• AWS_PROFILE: a profile for the disposable AWS account. cdk bootstrap runs once during setup.

You need Node.js 22 or newer, npm, Git, Docker with BuildKit, zip, the AWS CLI and a disposable AWS account. The AWS resources are created in eu-west-1. Run the harness from main, with the credentials above:

git clone https://github.com/stacktape/deploy-benchmarks && cd deploy-benchmarks
npm ci
node bench.mjs setup   cdk container-python             # write work/bmk-cdk-container-python/ and deploy it once (not measured)
node bench.mjs package cdk container-python --runs 10   # record the image size
node bench.mjs deploy  cdk container-python --runs 10   # deploy a code change from this machine
node report.mjs                                         # median, fastest and slowest run
node bench.mjs remove  cdk container-python             # delete everything setup created

setup writes the files of this branch to work/bmk-cdk-container-python/ and commits them as the same commit that this branch has.

Each run is a pair: a first push with an empty build cache, then a second push 15 minutes after the first one finished. The clock starts when git push returns.

1. Fork the repository with all branches: untick "Copy the main branch only".
2. Turn on GitHub Actions in your fork. This branch's .github/workflows/deploy.yml deploys every push to the branch.
3. Add the repository secret AWS_ROLE_ARN. AWS_ROLE_ARN is a role in the disposable account that the fork can assume through GitHub OIDC.
4. Sign gh in to an account that can manage the fork's Actions caches (gh auth login or GH_TOKEN). The first push of each pair deletes them with gh cache delete --all.
5. Run the commands below. The project in work/ has the same commit as this branch, so its pushes continue your fork's branch.

export BENCH_FORK=https://github.com/<you>/deploy-benchmarks   # setup adds it as the project's origin
node bench.mjs setup cdk container-python                      # skip if you already ran it with BENCH_FORK set
node bench.mjs push cdk container-python --runs 10
node report.mjs

From a checkout of this branch, without the harness:

npm ci
npx cdk bootstrap
npx cdk deploy --require-approval never
# delete everything:
npx cdk destroy --force

Each run appends one JSON line to results/package.jsonl or results/deploy.jsonl. Times are in milliseconds and sizes in bytes. Every line also records the tool version, the machine and the command log.

{ "app": "container-python", "tool": "cdk", "bytes": <compressed image size>, "artifacts": [...] }
{ "app": "container-python", "tool": "cdk", "journey": "cli", "run": 1, "change": "handlers", "changedTargets": 1, "readyMs": <time until the new code answers>, "commandMs": <deploy command duration>, "businessOk": true, "unchangedOk": true }
{ "app": "container-python", "tool": "cdk", "journey": "git-cold", "run": 1, "change": "handlers", "changedTargets": 1, "readyMs": <time until the new code answers>, "coldReset": "scripted", "businessOk": true, "unchangedOk": true }
{ "app": "container-python", "tool": "cdk", "journey": "git-warm", "run": 1, "change": "handlers", "changedTargets": 1, "readyMs": <time until the new code answers>, "businessOk": true, "unchangedOk": true }

node report.mjs prints the median with the fastest and slowest run of each series. Runs whose business check failed are listed and left out.

Pending. The results will be published in the main README after the measurement runs on a dedicated EC2 machine.