This document describes the process for releasing Dagger.

Before you go ahead and produce a new release, remember that it's a team effort. The first step is to let the team know what is going to happen, preferably a few days in advance so that they can react. We do our release threads in public wherever possible, to give the community more visibility, and to help external contributors to more easily keep in sync with the release cadence.

To let the team know:

• Create a new milestone in GitHub, e.g. v0.20.0 Milestone
• Create a new post in Discord #team, e.g. v0.20.0 release

This allows others to weigh in whether:

• we should go for a patch / minor bump
• there are any PRs that people are waiting to get merged
• any big features which need to remain experimental?
• etc.

Maybe there are breaking changes which we should be aware of and message accordingly. Giving other team members a day or two to react - because timezones! - will make this entire process smoother.

• If doing a minor release, determine if there are any deprecations that can be removed.

Where possible, we try to ensure backwards compatibility between mismatched cli and engine versions. However, for technical reasons, this isn't always possible: sometime the communication protocol changes, or a bug fix or new feature requires changes on both the CLI and the engine.

Before releasing, make sure to sanity check the backwards compatibility of a release. If you encounter issues, then:

• Add a release note using changie new (or add it later manually).
• Bump the minimum version numbers in engine/version.go.

If unsure, bump both the client and engine minimum version numbers, but if the backwards compatibility is only an issue in one direction, you only need to bump that one.

In order to keep this relevant & accurate, we improve this doc during the release process. It's the best time to pause, observe how it all fits together, and improve it. We want small, constant improvements which compound. Therefore:

• Save a copy of this doc outside of this repository (e.g. ~/RELEASING.editing.md). Now open that copy in your editor and start ticking items off it as you make progress. Remember to add / remove / edit any parts which could be improved.

  If need be, modify the path of RELEASING_DOC to wherever you saved it.

  RELEASING_DOC=~/RELEASING.editing.md

• Update the date in the shields.io badge, first line in this file.

  badge_url=$(echo "https://img.shields.io/badge/Last updated on-$(date +'%B %-d, %Y')-success?style=flat-square" | sed -E 's/ /%20/g')
  sed "1s#https://.*#${badge_url})#" RELEASING.md > "$RELEASING_DOC"

Before starting the releasing process, make sure you have your local environment setup and ready to go. You'll need (at least) the following tools - though you probably already have most of them if you're regularly contributing to dagger.

• A UNIX environment and shell
• git
• dagger (the most recent release)
• changie
• gh github cli tool
• golang

• Setup some variables used throughout the release process:

  export RELEASE_BRANCH=main
  export DAGGER_REPO_REMOTE=$(git remote -v | grep -E "(github.com.dagger/dagger)" | head -n 1 | awk '{print $1}')
  export OLD_ENGINE_VERSION="$(git tag -l 'v*' | sort -V | tail -1)"

• Setup the local branch to align with the remote branch being released

  git checkout "$RELEASE_BRANCH"
  
  git pull $DAGGER_REPO_REMOTE "$RELEASE_BRANCH"

• Determine the next release version (use patch/minor/major to set the release type):

  export ENGINE_VERSION="$(dagger call -m version next-release-version)"
  
  # this is required to interpolate $ENGINE_VERSION to the SDK release notes
  export CHANGIE_ENGINE_VERSION="$ENGINE_VERSION"

• Ensure that .changes/.next contains $ENGINE_VERSION - if it doesn't, update it now!

  grep -Fx "$ENGINE_VERSION" .changes/.next || { echo "ENGINE_VERSION env var does not match what's in .changes/.next"; false }

• Create the target release notes branch for a PR.

  git checkout -b prep-$ENGINE_VERSION

• Bump internal versions (sdks + docs + helm chart) to the target version

  dagger -y call release bump --engine-version="$ENGINE_VERSION"

• Bump Go SDK package commit to the latest commit from the dagger-go-sdk repository.

  GO_SDK_COMMIT=$(git ls-remote https://github.com/dagger/dagger-go-sdk HEAD | awk '{print $1}')

  If Linux:

  sed -i -E "s#^((const |\t)goSDKLibVersion = \")[0-9a-f]{40}\" // v.*#\1${GO_SDK_COMMIT}\" // ${ENGINE_VERSION}#" core/sdk/go_sdk.go

  If MacOS (sed -i is different compared to Linux):

  sed -i '' -E "s#^((const |\t)goSDKLibVersion = \")[0-9a-f]{40}\" // v.*#\1${GO_SDK_COMMIT}\" // ${ENGINE_VERSION}#" core/sdk/go_sdk.go

• Commit

  git add docs sdk helm lib/dagger/core core/sdk
  git commit -s -m "chore: bump dependencies to $ENGINE_VERSION"

• Push to dagger/dagger - we need access to secrets that PRs coming from forks will not have. Open the PR as a draft and capture the PR number:

  git push $DAGGER_REPO_REMOTE prep-$ENGINE_VERSION
  gh pr create --draft --title "chore: prep for $ENGINE_VERSION" --body "" | tee /tmp/prep-pr.txt
  export RELEASE_PREP_PR=$(cat /tmp/prep-pr.txt | sed -r 's/^[^0-9]*([0-9]+).*/\1/')

• Generate bump changes for each SDK + the helm charts

  export GITHUB_USERNAME=$(gh api /user --jq .login)
  find sdk/go sdk/python sdk/typescript sdk/elixir sdk/php sdk/rust helm/dagger -maxdepth 1 -name .changie.yaml -execdir \
        changie new --kind "Dependencies" --body "Bump Engine to $ENGINE_VERSION" --custom PR="$RELEASE_PREP_PR" --custom Author="$GITHUB_USERNAME" \;

• Generate release notes .changes/**/v0.12.4.md for all releases:

  export CHANGIE_MAINTAINERS=$(dagger call release get-maintainers --github-org-name dagger --github-token="cmd://gh auth token" --json)
  find . sdk/go sdk/python sdk/typescript sdk/elixir sdk/php sdk/rust helm/dagger -maxdepth 1 -name .changie.yaml -execdir changie batch $ENGINE_VERSION \;

• Make any necessary edits to the newly generated file, e.g. .changes/v0.12.4.md

• Update CHANGELOG.md by running:

changie merge
find . sdk/go sdk/python sdk/typescript sdk/elixir sdk/php sdk/rust helm/dagger -maxdepth 1 -name .changie.yaml -execdir changie merge \;
find . -name .changes -type d -exec git add {} \;
find . -name CHANGELOG.md -type f -exec git add {} \;
git commit -s -m "chore: add release notes for $ENGINE_VERSION"

• Update .changes/.next with the next release number if known and commit it - otherwise, make the file empty (but don't remove it).

  git add .changes/.next
  git commit -s -m 'bump .next to next release version'

• Push changes, and bring the prep PR out of draft:

  git push $DAGGER_REPO_REMOTE prep-$ENGINE_VERSION
  gh pr ready

• Ensure that all GitHub Actions checks pass for the dagger.io PR which gets automatically created part of this PR. The PR is configured to deploy a Daggerverse preview environment with a main Dagger Engine (the one that is just about to be released). If all checks pass, close that PR & delete the branch (this will clean up the infra that gets provisioned). If checks fail, cc @marcosnils @matipan in the release thread and wait for a response before continuing with the release (this might be a blocker).

• If everything above is green, review and merge the prep PR. The merged commit is what gets tagged in the next step.

• Confirm that all checks on $RELEASE_BRANCH are green, for main you're basically checking that the merged prep commit is has a green check. Do not push tags until this is finished.

• When you have confirmed that all checks on $RELEASE_BRANCH are green, pull the latest changes, and push the tag to trigger the release workflow by running:

  git checkout "$RELEASE_BRANCH"
  git pull "$DAGGER_REPO_REMOTE" "$RELEASE_BRANCH"
  
  export ENGINE_GIT_SHA="$(git rev-parse --verify HEAD)"
  git tag "$ENGINE_VERSION" "$ENGINE_GIT_SHA"
  git push "$DAGGER_REPO_REMOTE" "$ENGINE_VERSION"

This will kick off .github/workflows/publish.yml which publishes:

• A new image to ghcr.io/dagger/engine (mirrored to registry.dagger.io/engine using https://github.com/dagger/registry-redirect).
• Go packages to 🐙 dagger.io/dagger via github.com/dagger/dagger-go-sdk.
• Python packages to 🐍 dagger-io.
• Typescript packages to ⬢ npmjs.com/package/@dagger.io/dagger.
• Elixir packages to 🧪 hex.pm/packages/dagger.
• Rust crates to ⚙️ crates.io/crate/dagger-sdk.
• PHP packages to 🐘 packagist.org/packages/dagger/dagger via github.com/dagger/dagger-php-sdk.
• Helm charts to ☸️ registry.dagger.io/dagger-helm.

This will also kick off [.github/workflows/evals.yml], which is currently broken at the moment.

• Double-check the engine+cli release:

  # install the cli to dagger-<version>, and symlink dagger to it
  export BIN_DIR=$HOME/.local/bin
  curl -fsSL https://dl.dagger.io/dagger/install.sh | DAGGER_VERSION=$ENGINE_VERSION sh
  mv $BIN_DIR/dagger{,-$ENGINE_VERSION}
  ln -s $BIN_DIR/dagger{-$ENGINE_VERSION,}
  
  dagger version
  dagger core version

• Double-check that all the above packages have been correctly published and updated to their latest versions. Note: the Go package may not be instantly updated due to caching.

• Double-check that git tags + github releases have been made for each component.

• Ask @marcosnils @matipan @sipsma on the release thread to review and merge the newly opened dagger.io PR (this is created by the publish workflow). This PR updates both the Daggerverse and Dagger Cloud. If anything fails, cc @kpenfound @matipan.

• Validate that the newly published version of the engine engine is available in Dagger Cloud by running dagger --cloud core version with the $ENGINE_VERSION binary. This upgrade occurs automatically once the OCI image is published.

• Start a post-release branch:

  git checkout -b post-release-$ENGINE_VERSION

• Update non-autogenerated .github configs to $ENGINE_VERSION

  • The version numbers (of the form <major>.<minor>.<patch>) should be updated to the new version
  • The worker runner versions (of the form dagger-v<major>-<minor>-<patch>-<worker>)
  • Update the version numbers in the commands below and run:

  old_dashed=$(echo $OLD_ENGINE_VERSION | sed -E 's/^v([0-9]+)\.([0-9]+)\.([0-9]+)$/\1-\2-\3/')
  new_dashed=$(echo $ENGINE_VERSION | sed -E 's/^v([0-9]+)\.([0-9]+)\.([0-9]+)$/\1-\2-\3/')
  old_dotted=$(echo $OLD_ENGINE_VERSION | sed -E 's/^v([0-9]+)\.([0-9]+)\.([0-9]+)$/\1\\.\2\\.\3/')
  new_dotted=$(echo $ENGINE_VERSION | sed -E 's/^v([0-9]+)\.([0-9]+)\.([0-9]+)$/\1\\.\2\\.\3/')

  If Linux:

  find .github/ -type f -exec sed -i "s/dagger-v${old_dashed}/dagger-v${new_dashed}/g; s/${old_dotted}/${new_dotted}/g" {} +

  If MacOS (sed -i is different compared to Linux):

  find .github/ -type f -exec sed -i '' -e "s/dagger-v${old_dashed}/dagger-v${new_dashed}/g" -e "s/${old_dotted}/${new_dotted}/g" {} +

• Bump the Go SDK version in our internal CI targets (these aren't actually used anywhere since we use the modularized go SDK - but it's good practice regardless).

  # Run dagger develop in docs/recorder{,2}, and everywhere else but not tests to make sure we do test backwards compat.
  find . -name dagger.json \( -path "./docs/recorder*" -o -not -path "./docs/*" \) -not -path '*/tests/*' -not -path '*/testdata/*' -not -path '*/viztest/*' -not -path './core/integration/*' -execdir dagger develop \;
  
  # update deps and run go mod tidy on all go modules that were updated
  find . -name go.mod \( -path "./docs/recorder*" -o -not -path "./docs/*" \) -not -path '*/tests/*' -not -path '*/testdata/*' -not -path '*/viztest/*' -not -path './core/integration/*' -execdir sh -c 'for dep in dagger.io/dagger github.com/dagger/dagger/engine/distconsts; do git grep -qF "$dep " go.mod && go get "${dep}@${ENGINE_VERSION}"; done; go mod tidy' \;
  
  # add, commit and push the changes to the branch
  git commit -a -s -m "chore: bump internal tooling to $ENGINE_VERSION"

• When all the above is done, review this current RELEASING_DOC file ALL THE WAY TO THE END, to find possible improvements, then come back here.

• When done editing, add the changes with ticks unmarked

  sed 's#- \[x\] #- [ ] #' "$RELEASING_DOC" > RELEASING.md
  git commit -a -s -m "chore: Improve RELEASING.md during $ENGINE_VERSION"

• Open a PR with the title chore: post-release $ENGINE_VERSION

  git push "$DAGGER_REPO_REMOTE" "post-release-$ENGINE_VERSION"
  gh pr create --title "chore: post-release during $ENGINE_VERSION" --body ""

• If the git remote url was changed to ssh, now is a good time to revert it to https.

  git remote set-url $DAGGER_REPO_REMOTE https://github.com/dagger/dagger.git

• Close the GitHub milestone you created in Let the team know
  • If there are remaining PRs/issues that were not resolved, then move them into the next milestone (or remove it from a milestone entirely)