Summary
Add a non-secret companion to withSecretVariable:
type Container {
withVolatileVariable(name: String!, value: String!): Container!
withoutVolatileVariable(name: String!): Container!
}
withVolatileVariable(name, value) sets a non-secret env var for future withExec calls on that container. Changing only its value does not invalidate cached withExec results. If the exec reruns for some other reason, it sees the latest volatile value.
Warning
withVolatileVariable is an expert-only escape hatch. Use it only if you really know what you're doing.
By using it, the caller is asserting that changes to this variable alone must not invalidate cached results of future withExec calls. If that assertion is wrong, Dagger may reuse stale or incorrect results from cache.
Problem
Today the options are:
withEnvVariable(name, value)
withSecretVariable(name, secret)
withEnvVariable is the wrong fit because changing plain env vars invalidates downstream exec cache.
withSecretVariable is closer semantically because Secret.cacheKey lets users stabilize cache identity across changing plaintext values. Users are already using this as a workaround for public CI/reporting metadata.
The problem is that these values are not secrets, so the workaround causes unwanted redaction in logs and output.
Typical examples:
- commit SHA
- branch/ref
- CI run ID
Contract
withVolatileVariable(name, value) should mean:
- visible to future
withExec calls only
- non-secret, with no secret redaction behavior
- changing only volatile values does not invalidate cached
withExec results
- if a
withExec reruns because some other input changed, it sees the latest volatile values
- not persisted into OCI image config
- not exposed by
envVariable / envVariables
- not available to Dagger-side
expand: true
- repeated
withVolatileVariable("FOO", ...) uses the last volatile value for FOO
- if both persistent env and volatile env exist for the same name, volatile wins for
withExec
withoutVolatileVariable(name) removes the volatile value for future withExec calls
Example
const ctr = dag
.container()
.from("mcr.microsoft.com/playwright:v1.52.0-noble")
.withVolatileVariable("CURRENTS_COMMIT_SHA", process.env.GITHUB_SHA!)
.withVolatileVariable("CURRENTS_GITHUB_RUN_ID", process.env.GITHUB_RUN_ID!)
.withExec(["pnpm", "playwright", "test"])
Why a separate method?
This should not be withEnvVariable(volatile: true).
withEnvVariable currently means persistent container env. Overloading it with a boolean would mix two different models in one API:
- persistent OCI env config
- exec-scoped cache-excluded env
A separate method keeps the distinction clear.
Precedent
Dagger already has precedent for this kind of cache behavior through:
withSecretVariable
Secret.cacheKey
Today users can already make different runtime values share one cache identity by routing them through a secret with a fixed cacheKey. This proposal makes that pattern available for public, non-secret metadata without unwanted redaction.
Acceptance Criteria
- Two otherwise identical
withExec calls that differ only in volatile variable values reuse cached results.
- If a non-volatile input changes and the exec reruns, the process sees the latest volatile values.
envVariable / envVariables do not expose volatile variables.
- Exported image config does not contain volatile variables.
expand: true rejects references to volatile variables.
withoutVolatileVariable(name) removes the value from future withExec calls.
Non-goals
- This is not for inputs that should invalidate
withExec results.
- This is not a replacement for
withEnvVariable.
- This is not a secret-management feature.
Summary
Add a non-secret companion to
withSecretVariable:withVolatileVariable(name, value)sets a non-secret env var for futurewithExeccalls on that container. Changing only its value does not invalidate cachedwithExecresults. If the exec reruns for some other reason, it sees the latest volatile value.Warning
withVolatileVariableis an expert-only escape hatch. Use it only if you really know what you're doing.By using it, the caller is asserting that changes to this variable alone must not invalidate cached results of future
withExeccalls. If that assertion is wrong, Dagger may reuse stale or incorrect results from cache.Problem
Today the options are:
withEnvVariable(name, value)withSecretVariable(name, secret)withEnvVariableis the wrong fit because changing plain env vars invalidates downstream exec cache.withSecretVariableis closer semantically becauseSecret.cacheKeylets users stabilize cache identity across changing plaintext values. Users are already using this as a workaround for public CI/reporting metadata.The problem is that these values are not secrets, so the workaround causes unwanted redaction in logs and output.
Typical examples:
Contract
withVolatileVariable(name, value)should mean:withExeccalls onlywithExecresultswithExecreruns because some other input changed, it sees the latest volatile valuesenvVariable/envVariablesexpand: truewithVolatileVariable("FOO", ...)uses the last volatile value forFOOwithExecwithoutVolatileVariable(name)removes the volatile value for futurewithExeccallsExample
Why a separate method?
This should not be
withEnvVariable(volatile: true).withEnvVariablecurrently means persistent container env. Overloading it with a boolean would mix two different models in one API:A separate method keeps the distinction clear.
Precedent
Dagger already has precedent for this kind of cache behavior through:
withSecretVariableSecret.cacheKeyToday users can already make different runtime values share one cache identity by routing them through a secret with a fixed
cacheKey. This proposal makes that pattern available for public, non-secret metadata without unwanted redaction.Acceptance Criteria
withExeccalls that differ only in volatile variable values reuse cached results.envVariable/envVariablesdo not expose volatile variables.expand: truerejects references to volatile variables.withoutVolatileVariable(name)removes the value from futurewithExeccalls.Non-goals
withExecresults.withEnvVariable.