docs: clarify safeoutputs deferred writes#39694
Conversation
Co-authored-by: Peli de Halleux <4175913+pelikhan@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
This PR updates the safe-outputs prompt documentation to clarify how agents should interpret successful safe-output tool calls, with the goal of preventing unnecessary and potentially misleading post-write verification attempts in the sandbox environment.
Changes:
- Added guidance to treat successful safe-output tool calls as authoritative confirmation of persistence.
- Discouraged verifying safe-output writes via
gh apior other read paths due to potential sandbox staleness.
Show a summary per file
| File | Description |
|---|---|
| actions/setup/md/safe_outputs_prompt.md | Adds a documentation paragraph clarifying trust in successful safe-output tool calls and discouraging independent verification reads. |
Copilot's findings
Tip
Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Files reviewed: 1/1 changed files
- Comments generated: 1
|
|
||
| **Tool retry limit:** if a safe-output tool (for example `push_to_pull_request_branch` or `close_pull_request`) fails, try at most 2 materially different recovery approaches. If the tool still fails, call `report_incomplete` with the error and the approaches attempted, then continue with other work. Do NOT debug underlying infrastructure after repeated failures. | ||
|
|
||
| **Trust tool return codes.** A successful (exit 0) safe-output call means the action was persisted. Do NOT independently verify writes via `gh api` or other read paths — the sandbox may return stale data that does not reflect safe-output writes. Treat a zero exit code as authoritative confirmation. |
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
|
|
||
| **Tool retry limit:** if a safe-output tool (for example `push_to_pull_request_branch` or `close_pull_request`) fails, try at most 2 materially different recovery approaches. If the tool still fails, call `report_incomplete` with the error and the approaches attempted, then continue with other work. Do NOT debug underlying infrastructure after repeated failures. | ||
|
|
||
| **Trust tool return codes.** A successful (exit 0) safe-output call means the action was persisted. Do NOT independently verify writes via `gh api` or other read paths — the sandbox may return stale data that does not reflect safe-output writes. Treat a zero exit code as authoritative confirmation. |
There was a problem hiding this comment.
the sandbox may return stale data that does not reflect safe-output writes
This doesn't feel right - shouldn't we just say that the safe outputs are actually written in a subsequent separate job?
|
@copilot describe what safeoutputs does: safeoutputs records transactions that will be applied after the agent is done. Therefore, effects of the safeoutputs are not visible when running gh or github tools. Use temporary ids to refer to "future" resources that will be created by safe outputs. |
|
✅ smoke-ci: safeoutputs CLI comment + comment-memory run (27654928777)
|
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Updated in 0ef5d6c to describe safeoutputs as recording deferred transactions that are applied after the agent completes, and to clarify that temporary IDs are for referring to those future resources. |
Summary
Two targeted documentation improvements to
safe_outputs_prompt.mdthat reduce agent confusion around deferred write semantics and the safeoutputs invocation model.Return-code trust guidance (
dc0356091): adds an explicit paragraph instructing agents not to re-verify a successful safe-output transaction viagh/gh api. Because safe-output writes are applied after the agent finishes, the current sandbox will never reflect them; polling for confirmation is both wasteful and misleading.Deferred-write &
temporary_idclarifications (0ef5d6ca8): tightens thetemporary_iddescription to read "cross-reference field for future resources created by safe outputs" — making clear that the field is meaningful only for resources that do not exist yet at call time.Changed files
actions/setup/md/safe_outputs_prompt.mdtemporary_idscoped to future resourcesDetailed changes
actions/setup/md/safe_outputs_prompt.mdAdded — trust rule (
dc0356091)Addresses a recurring agent failure mode: the agent calls a safe-output tool successfully, then immediately calls
gh apito check whether the PR/issue was created, gets a 404 (because the write has not landed yet), and either retries or callsreport_incompleteunnecessarily.Modified —
temporary_iddescription (0ef5d6ca8)optional cross-reference fieldoptional cross-reference field for future resources created by safe outputsClarifies scope: only relevant when the safe-output call will create a resource that does not exist yet.
Impact