Signed-off-by: Matias Pan <matias@dagger.io>

* Simplify function call return handling

Module function calls still used the legacy metadata directory to shuttle return values and returned errors out of the nested runtime container. That path existed for the older pre-Theseus cache model, but it now makes execution do extra container output work for state that is already part of the live function call.

Record return values and returned errors directly on the active FunctionCall, have runtimes return only execution errors, and read the recorded result after the runtime completes. This removes the .daggermod output/error mounts, module exec error extraction plumbing, and metadata constants while keeping the public returnValue/returnError API intact.

The simpler path also avoids the snapshot and read-back bottleneck in function call execution.

Signed-off-by: Erik Sipsma <erik@sipsma.dev>

* Fix function return lint cleanup

Signed-off-by: Erik Sipsma <erik@sipsma.dev>

---------

Signed-off-by: Erik Sipsma <erik@sipsma.dev>

Signed-off-by: Tibor Vass <teabee89@gmail.com>

Conflict resolution details with original conflict hunks and resolved hunks.

1. Deleted module mega-test vs main never-cache regression

Original conflict: 1.0-beta deleted the old mega-test as part of the split, while main added a regression to that deleted file.

```diff
diff --git a/core/integration/module_test.go b/core/integration/module_test.go
index 3926492..e86849c 100644
--- a/core/integration/module_test.go
+++ b/core/integration/module_test.go
@@ -7531,6 +7531,24 @@ func (m *Test) ObjsWithNothing() ([]*Test, error) {
 	require.JSONEq(t, `{"objsWithNothing":[null,{"dirs":[null]}]}`, out)
 }

+func (ModuleSuite) TestNeverCacheModuleObjectReturn(ctx context.Context, t *testctx.T) {
+	c := connect(ctx, t)
+
+	modGen := modInit(t, c, "go", `package main
+
+type Test struct{}
+
+// +cache="never"
+func (m *Test) Touch() *Test {
+	return m
+}
+`)
+
+	out, err := modGen.With(daggerCall("touch")).Stdout(ctx)
+	require.NoError(t, err)
+	require.Contains(t, out, "Test@")
+}
+
 func (ModuleSuite) TestFunctionCacheControl(ctx context.Context, t *testctx.T) {
```

Resolution: keep beta's deleted mega-test, port the regression to the split runtime behavior test, and put the test module function in beta's existing runtime cache-control fixture.

```diff
diff --git a/core/integration/module_runtime_behavior_test.go b/core/integration/module_runtime_behavior_test.go
@@ -378,6 +378,14 @@ func (ModuleSuite) TestReturnNil(ctx context.Context, t *testctx.T) {
 	require.JSONEq(t, `{"objsWithNothing":[null,{"dirs":[null]}]}`, out)
 }

+func (ModuleSuite) TestNeverCacheModuleObjectReturn(ctx context.Context, t *testctx.T) {
+	c := connect(ctx, t)
+	modGen := moduleFixture(t, c, "go/runtime-cache-control")
+	out, err := modGen.With(daggerCall("touch")).Stdout(ctx)
+	require.NoError(t, err)
+	require.Contains(t, out, "Test@")
+}
+
 func (ModuleSuite) TestFunctionCacheControl(ctx context.Context, t *testctx.T) {
diff --git a/core/integration/testdata/modules/go/runtime-cache-control/main.go b/core/integration/testdata/modules/go/runtime-cache-control/main.go
@@ -17,6 +17,11 @@ func (m *Test) TestNeverCache() string {
 	return rand.Text()
 }

+// +cache="never"
+func (m *Test) Touch() *Test {
+	return m
+}
+
 // My rad doc on TestAlwaysCache
```

2. Shell helper command path

Original conflict: main moved the helper to `dagger shell`, while beta added `daggerShellAt` and root-shell `-m` support.

```diff
diff --cc core/integration/shell_test.go
@@@ -32,16 -22,8 +32,20 @@@ func TestShell(t *testing.T)
  }

  func daggerShell(script string) dagger.WithContainerFunc {
 +	return daggerShellAt("", script)
 +}
 +
 +func daggerShellAt(modPath, script string) dagger.WithContainerFunc {
  	return func(c *dagger.Container) *dagger.Container {
++<<<<<<< HEAD:core/integration/shell_test.go
 +		execArgs := []string{"dagger"}
 +		if modPath != "" {
 +			execArgs = append(execArgs, "-m", modPath)
 +		}
 +		return c.WithExec(execArgs, dagger.ContainerWithExecOpts{
++=======
+		return c.WithExec([]string{"dagger", "shell"}, dagger.ContainerWithExecOpts{
++>>>>>>> origin/main:core/integration/module_shell_test.go
  			Stdin:                         script,
  			ExperimentalPrivilegedNesting: true,
  		})
@@ -50,7 +54,7 @@ func daggerShellAt(modPath, script string) dagger.WithContainerFunc {

 func daggerShellNoMod(script string) dagger.WithContainerFunc {
 	return func(c *dagger.Container) *dagger.Container {
-		return c.WithExec([]string{"dagger", "-M"}, dagger.ContainerWithExecOpts{
+		return c.WithExec([]string{"dagger", "shell", "-M"}, dagger.ContainerWithExecOpts{
 			Stdin:                         script,
 			ExperimentalPrivilegedNesting: true,
 		})
 	}
 }
```

Resolution: combine main's explicit `dagger shell` invocation with beta's `daggerShellAt` helper. `daggerShellAt` now starts from `dagger shell` and appends `-m <path>` when needed, while `daggerShellNoMod` uses `dagger shell -M`.

```diff
diff --git a/core/integration/shell_test.go b/core/integration/shell_test.go
@@ -37,18 +37,18 @@ func daggerShell(script string) dagger.WithContainerFunc {

 func daggerShellAt(modPath, script string) dagger.WithContainerFunc {
 	return func(c *dagger.Container) *dagger.Container {
-		execArgs := []string{"dagger"}
+		execArgs := []string{"dagger", "shell"}
 		if modPath != "" {
 			execArgs = append(execArgs, "-m", modPath)
 		}
 		return c.WithExec(execArgs, dagger.ContainerWithExecOpts{
 			Stdin:                         script,
 			ExperimentalPrivilegedNesting: true,
 		})
 	}
 }

 func daggerShellNoMod(script string) dagger.WithContainerFunc {
 	return func(c *dagger.Container) *dagger.Container {
-		return c.WithExec([]string{"dagger", "-M"}, dagger.ContainerWithExecOpts{
+		return c.WithExec([]string{"dagger", "shell", "-M"}, dagger.ContainerWithExecOpts{
 			Stdin:                         script,
 			ExperimentalPrivilegedNesting: true,
 		})
 	}
 }
```

3. Deleted container cookbook page vs volatile variable docs

Original conflict: beta deleted the old cookbook page in the docs restructure; main added volatile variable guidance there.

```diff
diff --git a/docs/current_docs/cookbook/containers.mdx b/docs/current_docs/cookbook/containers.mdx
index 4bf6e79..b97d036 100644
--- a/docs/current_docs/cookbook/containers.mdx
+++ b/docs/current_docs/cookbook/containers.mdx
@@ -18,3 +18,24 @@ Dagger allows you to build, publish, and export container images, also known as
 <ExportContainerImageToHost />

 <SetEnvVar />
+
+## Use volatile variables for exec-time metadata
+
+`withVolatileVariable` sets a non-secret environment variable for future `withExec` calls without invalidating exec cache when only the variable's value changes.
+
+Typical examples include CI and reporting metadata such as:
+- commit SHA
+- branch or ref
+- CI run ID
+
+:::warning
+`withVolatileVariable` is an expert-only escape hatch. Use it only when you are certain that changing the variable alone must not invalidate cached `withExec` results. If that assumption is wrong, Dagger may reuse stale or incorrect cached results.
+:::
```

Resolution: keep the cookbook deletion and move the content to the beta Container type page, which now owns Container behavior docs.

```diff
diff --git a/docs/current_docs/extending/types/container.mdx b/docs/current_docs/extending/types/container.mdx
@@ -60,3 +60,30 @@ The default address can be any valid container image reference, such as:
 - `alpine:3.19` - Docker Hub image with specific version
 - `ghcr.io/owner/image:tag` - GitHub Container Registry image
 - `gcr.io/project/image:tag` - Google Container Registry image
+
+## Volatile variables
+
+`withVolatileVariable` sets a non-secret environment variable for future
+`withExec` calls without invalidating exec cache when only the variable's value
+changes.
+
+Typical examples include CI and reporting metadata such as commit SHAs, branch
+or ref names, and CI run IDs.
+
+:::warning
+`withVolatileVariable` is an expert-only escape hatch. Use it only when you are
+certain that changing the variable alone must not invalidate cached `withExec`
+results. If that assumption is wrong, Dagger may reuse stale or incorrect
+cached results.
+:::
```

4. Deleted function-caching page vs volatile variable cache note

Original conflict: beta deleted the old function caching module page; main added a cache note there.

```diff
diff --git a/docs/current_docs/extending/modules/function-caching.mdx b/docs/current_docs/extending/modules/function-caching.mdx
index 707c1a3..6622233 100644
--- a/docs/current_docs/extending/modules/function-caching.mdx
+++ b/docs/current_docs/extending/modules/function-caching.mdx
@@ -252,6 +252,8 @@ This means:
 If you need a specific step to always re-run, add a cache-busting input to that step (for example, a
 timestamp environment variable used only for that operation).

+If you need non-secret metadata to be visible to a `withExec` step without invalidating cache when only that metadata changes, use `withVolatileVariable` on the `Container`. This is an expert-only escape hatch: cached `withExec` results may be reused even when the volatile value changes, and reruns caused by other inputs will see the latest volatile value. See the [container cookbook](../../cookbook/containers.mdx#use-volatile-variables-for-exec-time-metadata).
+
 ### Secrets and caching
```

Resolution: keep the page deletion and move the cache behavior note to beta's new cache explainer, with a link to the new Container section.

```diff
diff --git a/docs/current_docs/extending/how-dagger-works/cache.mdx b/docs/current_docs/extending/how-dagger-works/cache.mdx
@@ -14,6 +14,20 @@ Different inputs: Dagger reruns the affected work.

 This is why module API design affects speed.

+## Function Cache Hits
+
+A function cache hit skips the function entirely, so none of its internal steps
+run. A cache miss runs the function, but individual container/file steps inside
+it may still be cache hits if their inputs are unchanged.
+
+If you need a specific step to always re-run, add a cache-busting input to that
+step. If you need non-secret metadata to be visible to a `withExec` step without
+invalidating cache when only that metadata changes, use
+`withVolatileVariable` on the `Container`. Cached `withExec` results may be
+reused even when the volatile value changes, and reruns caused by other inputs
+will see the latest volatile value. See
+[Container volatile variables](../types/container.mdx#volatile-variables).
+
 ## Inputs Matter
```

5. Dang dependency version

Original conflict: beta had an earlier Dang pseudo-version; main had a newer one.

```diff
diff --cc go.mod
@@@ -152,7 -151,7 +152,11 @@@ require
  	github.com/urfave/cli v1.22.17
  	github.com/vektah/gqlparser/v2 v2.5.32
  	github.com/vito/bubbline v0.0.0-20250312195236-5f4f49d6ebcb
++<<<<<<< HEAD
 +	github.com/vito/dang v0.0.0-20260527191113-ad5bca2669a9
++=======
+	github.com/vito/dang v0.0.0-20260602175442-56ec0c007a23
++>>>>>>> origin/main
  	github.com/vito/go-interact v1.0.2
```

Resolution: take main's newer Dang because the merged `core/sdk/dang_helpers.go` uses the newer `dang.FieldDecl` API; go.sum follows the same version choice.

```diff
diff --git a/go.mod b/go.mod
@@ -152,7 +152,7 @@ require (
 	github.com/urfave/cli v1.22.17
 	github.com/vektah/gqlparser/v2 v2.5.32
 	github.com/vito/bubbline v0.0.0-20250312195236-5f4f49d6ebcb
-	github.com/vito/dang v0.0.0-20260527191113-ad5bca2669a9
+	github.com/vito/dang v0.0.0-20260602175442-56ec0c007a23
 	github.com/vito/go-interact v1.0.2
```

6. Generated ConfigSchema client comments

Original conflict: repeated in the generated engine-dev/dagger-engine clients. Beta listed the new workspace schema files; main kept the old list but shifted the source line.

```diff
diff --cc modules/dev/internal/dagger/engine-dev.gen.go
@@@ -278,8 -277,8 +277,13 @@@ func (r *EngineDev) ClientDockerConfig(
  }

  // Generate the json schema for a dagger config file
++<<<<<<< HEAD
 +// Currently supported: "dagger.json", "dagger-module.toml", "dagger.toml", "engine.json"
 +func (r *EngineDev) ConfigSchema(filename string) *File { // engine-dev (../../../../toolchains/engine-dev/main.go:359:1)
++=======
+ // Currently supported: "dagger.json", "engine.json"
+ func (r *EngineDev) ConfigSchema(filename string) *File { // engine-dev (../../../../toolchains/engine-dev/main.go:362:1)
++>>>>>>> origin/main
```

Resolution: keep beta's expanded schema filename list and main's shifted line number, matching the merged source at `toolchains/engine-dev/main.go:362`.

```diff
diff --git a/modules/dev/internal/dagger/engine-dev.gen.go b/modules/dev/internal/dagger/engine-dev.gen.go
@@ -278,8 +278,8 @@ func (r *EngineDev) ClientDockerConfig() *Secret {
 }

 // Generate the json schema for a dagger config file
-// Currently supported: "dagger.json", "dagger-module.toml", "dagger.toml", "engine.json"
-func (r *EngineDev) ConfigSchema(filename string) *File { // engine-dev (../../../../toolchains/engine-dev/main.go:359:1)
+// Currently supported: "dagger.json", "dagger-module.toml", "dagger.toml", "engine.json"
+func (r *EngineDev) ConfigSchema(filename string) *File { // engine-dev (../../../../toolchains/engine-dev/main.go:362:1)
```

7. Helper removed by beta split but referenced by merged main test

Original conflict effect: main's engine persistence test now used the helper from lockfile tests, while beta removed that helper with the lockfile split.

```diff
diff --git a/core/integration/engine_persistence_test.go b/core/integration/engine_persistence_test.go
@@ -1127,6 +1127,7 @@ git -C /root/srv/repo.git update-server-info
 		hostname := identity.NewID() + ".test"
 		gitSvc, repoBaseURL := gitSmartHTTPServiceDirAuth(ctx, t, c, hostname, gitDir, "", nil)
 		repoURL := repoBaseURL + "/repo.git"
+		gitHost := moduleResolveServiceHost(t, repoURL)
```

Resolution: add the small URL host helper beside the test that now needs it, without restoring the removed lockfile helper section.

```diff
diff --git a/core/integration/engine_persistence_test.go b/core/integration/engine_persistence_test.go
@@ -11,6 +11,7 @@ import (
 	"context"
 	"crypto/rand"
 	"fmt"
+	"net/url"
 	"os"
@@ -25,6 +26,17 @@ import (
 	"dagger.io/dagger"
 )

+func moduleResolveServiceHost(t *testctx.T, rawURL string) string {
+	t.Helper()
+	parsed, err := url.Parse(rawURL)
+	require.NoError(t, err)
+	host := parsed.Hostname()
+	require.NotEmpty(t, host)
+	return host
+}
+
 func (CachePersistenceSuite) TestDiskPersistenceAcrossRestart(ctx context.Context, t *testctx.T) {
```

CI follow-up: release publish dry-run in cache persistence now loads the synthetic release repo as a workspace with `-W "$MODULE_REF"` instead of as an extra module with `-m "$MODULE_REF"`. The synthetic root still uses workspace-only toolchains, so this matches the CLI migration path while preserving the dry-run assertions.

Signed-off-by: Tibor Vass <tiborvass@users.noreply.github.com>
Signed-off-by: Solomon Hykes <solomon@dagger.io>