lambda-25/serverless

This branch is one test of the deploy benchmarks: the Lambda backend with all 25 functions, deployed with Serverless Framework. It holds only that application and the tool's configuration, so it deploys as it is.

node branches.mjs on main generates this branch from main's fixtures and tool modules. Don't edit it here: change main and generate the branches again.

Serverless Framework 4.44.0, installed by npm ci and pinned with frameworkVersion in serverless.yml.

• AWS_PROFILE: a profile for the disposable AWS account.
• SERVERLESS_ACCESS_KEY, or run npm exec --workspace=packages/functions -- serverless login once.

You need Node.js 22 or newer, npm, Git, Docker with BuildKit, zip, the AWS CLI, a disposable AWS account and a Serverless Framework account. The AWS resources are created in eu-west-1. Run the harness from main, with the credentials above:

git clone https://github.com/stacktape/deploy-benchmarks && cd deploy-benchmarks
npm ci
node bench.mjs setup   serverless lambda-25                             # write work/bmk-serverless-lambda-25/ and deploy it once (not measured)
node bench.mjs package serverless lambda-25 --runs 10                   # package from an empty build cache
node bench.mjs package serverless lambda-25 --runs 10 --warm            # package after a source change
node bench.mjs deploy  serverless lambda-25 --runs 10                   # deploy a code change from this machine
node bench.mjs deploy  serverless lambda-25 --runs 10 --change shared   # change the shared service package instead
node report.mjs                                                         # median, fastest and slowest run
node bench.mjs remove  serverless lambda-25                             # delete everything setup created

setup writes the files of this branch to work/bmk-serverless-lambda-25/ and commits them as the same commit that this branch has.

Each run is a pair: a first push with an empty build cache, then a second push 15 minutes after the first one finished. The clock starts when git push returns.

1. Fork the repository with all branches: untick "Copy the main branch only".
2. Turn on GitHub Actions in your fork. This branch's .github/workflows/deploy.yml deploys every push to the branch.
3. Add the repository secrets AWS_ROLE_ARN and SERVERLESS_ACCESS_KEY. AWS_ROLE_ARN is a role in the disposable account that the fork can assume through GitHub OIDC.
4. Sign gh in to an account that can manage the fork's Actions caches (gh auth login or GH_TOKEN). The first push of each pair deletes them with gh cache delete --all.
5. Run the commands below. The project in work/ has the same commit as this branch, so its pushes continue your fork's branch.

export BENCH_FORK=https://github.com/<you>/deploy-benchmarks   # setup adds it as the project's origin
node bench.mjs setup serverless lambda-25                      # skip if you already ran it with BENCH_FORK set
node bench.mjs push serverless lambda-25 --runs 10
node report.mjs

From a checkout of this branch, without the harness:

npm ci
npm exec --workspace=packages/functions -- serverless deploy --stage bmk --region eu-west-1
# delete everything:
npm exec --workspace=packages/functions -- serverless remove --stage bmk --region eu-west-1

Each run appends one JSON line to results/package.jsonl or results/deploy.jsonl. Times are in milliseconds and sizes in bytes. Every line also records the tool version, the machine and the command log.

{ "app": "lambda-25", "tool": "serverless", "cache": "clean", "run": 1, "ms": <packaging time>, "bytes": <upload size>, "artifacts": [{ "file": "...", "bytes": ... }] }
{ "app": "lambda-25", "tool": "serverless", "journey": "cli", "run": 1, "change": "handlers", "changedTargets": 25, "readyMs": <time until the new code answers>, "commandMs": <deploy command duration>, "businessOk": true, "unchangedOk": true }
{ "app": "lambda-25", "tool": "serverless", "journey": "git-cold", "run": 1, "change": "handlers", "changedTargets": 25, "readyMs": <time until the new code answers>, "coldReset": "scripted", "businessOk": true, "unchangedOk": true }
{ "app": "lambda-25", "tool": "serverless", "journey": "git-warm", "run": 1, "change": "handlers", "changedTargets": 25, "readyMs": <time until the new code answers>, "businessOk": true, "unchangedOk": true }

node report.mjs prints the median with the fastest and slowest run of each series. Runs whose business check failed are listed and left out.

Pending. The results will be published in the main README after the measurement runs on a dedicated EC2 machine.