Skip to content

Show safeoutputs command help on empty arguments#43566

Merged
pelikhan merged 5 commits into
mainfrom
copilot/aw-failures-bound-cli-retry-loop
Jul 5, 2026
Merged

Show safeoutputs command help on empty arguments#43566
pelikhan merged 5 commits into
mainfrom
copilot/aw-failures-bound-cli-retry-loop

Conversation

Copilot AI commented Jul 5, 2026

Copy link
Copy Markdown
Contributor

Reviewer workflows were burning the full Copilot CLI step timeout when write-once safe-output tools were probed with empty arguments and repeatedly rejected with -32602. This change makes empty-arg safeoutputs calls resolve to help output instead of invoking the tool, and strengthens the remaining rejection path with an explicit non-retry hint.

  • Fail fast on empty safeoutputs probes

    • Detect required-argument safeoutputs commands invoked with an empty argument object.
    • Show per-command help immediately rather than issuing an MCP tools/call that is guaranteed to fail.
  • Strengthen the terminal hint for -32602

    • When the bridge still receives an empty-argument -32602 from safeoutputs, append a direct instruction not to retry.
    • Point callers to <tool> --help for schema discovery and noop for no-op completion.
  • Keep the change scoped to the CLI bridge

    • The retry loop itself is left untouched.
    • The fix is applied at the safeoutputs bridge boundary so write-once tools do not enter the failing path in the common empty-args case.
  • Focused coverage

    • Add tests for the empty-args help guard.
    • Add tests for the stronger -32602 error messaging.
if (shouldShowToolHelpForEmptyArgs(matchedTool, toolArgs)) {
  showToolHelp(serverName, toolName, tools);
  return;
}

Generated by 👨‍🍳 PR Sous Chef · 3.83 AIC · ⌖ 6.47 AIC · ⊞ 3.7K ·
Comment /souschef to run again

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copilot AI changed the title [WIP] Fix Copilot CLI safe-output retry loop to fast-fail on empty arguments Show safeoutputs command help on empty arguments Jul 5, 2026
Copilot AI requested a review from pelikhan July 5, 2026 15:24
@pelikhan pelikhan marked this pull request as ready for review July 5, 2026 15:37
Copilot AI review requested due to automatic review settings July 5, 2026 15:37
@github-actions

github-actions Bot commented Jul 5, 2026

Copy link
Copy Markdown
Contributor

Design Decision Gate 🏗️ completed the design decision gate check.

No ADR enforcement needed: PR #43566 does not have the 'implementation' label and has 0 new lines of code in business logic directories (threshold: 100).

@github-actions

github-actions Bot commented Jul 5, 2026

Copy link
Copy Markdown
Contributor

🧠 Matt Pocock Skills Reviewer has completed the skills-based review. ✅

@github-actions

github-actions Bot commented Jul 5, 2026

Copy link
Copy Markdown
Contributor

Test Quality Sentinel completed test quality analysis.

@github-actions

github-actions Bot commented Jul 5, 2026

Copy link
Copy Markdown
Contributor

PR Code Quality Reviewer completed the code quality review.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates the MCP CLI bridge used by gh-aw setup scripts to avoid wasting time on “empty arguments” discovery probes—especially for write-once safeoutputs tools—by showing per-tool help immediately instead of issuing a failing MCP tools/call. It also strengthens the -32602 error output for the remaining empty-args rejection path with an explicit “do not retry” hint.

Changes:

  • Add an empty-args guard (shouldShowToolHelpForEmptyArgs) that shows tool-specific help when a tool has required inputs but received {}.
  • Enhance formatResponse to append a non-retry hint for safeoutputs -32602 “Empty arguments are not allowed” errors (including pointing to <tool> --help and noop).
  • Add focused unit tests covering both the new help guard and the enhanced -32602 messaging.
Show a summary per file
File Description
actions/setup/js/mcp_cli_bridge.cjs Adds the empty-args → help guard and augments safeoutputs -32602 output with a non-retry hint.
actions/setup/js/mcp_cli_bridge.test.cjs Adds unit tests for the new guard behavior and the strengthened -32602 error message hinting.

Review details

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

  • Files reviewed: 2/2 changed files
  • Comments generated: 0
  • Review effort level: Low

@github-actions github-actions Bot mentioned this pull request Jul 5, 2026
@github-actions

github-actions Bot commented Jul 5, 2026

Copy link
Copy Markdown
Contributor

🧪 Test Quality Sentinel Report

Test Quality Score: 100/100 — Excellent

Analyzed 2 test(s): 2 design, 0 implementation, 0 violation(s).

📊 Metrics (2 tests)
Metric Value
Analyzed 2 (Go: 0, JS: 2)
✅ Design 2 (100%)
⚠️ Implementation 0 (0%)
Edge/error coverage 2 (100%)
Duplicate clusters 0
Inflation No (1.33:1 ratio)
🚨 Violations 0
Test File Classification Issues
shows help instead of calling required-argument tools with an empty args object mcp_cli_bridge.test.cjs design_test · high_value · behavioral_contract None
adds a non-retry hint for safeoutputs empty-argument rejections mcp_cli_bridge.test.cjs design_test · high_value · behavioral_contract None

Verdict

Passed. 0% implementation tests (threshold: 30%). Both new tests verify user-visible behavioral contracts with edge-case coverage.

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

  • awmgmcpg

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "awmgmcpg"

See Network Configuration for more information.

🧪 Test quality analysis by Test Quality Sentinel · 26.9 AIC · ⌖ 12.6 AIC · ⊞ 6.8K ·
Comment /review to run again

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Test Quality Sentinel: 100/100. 0% implementation tests (threshold: 30%).

@@ -1231,6 +1260,13 @@ async function main() {
const stdinContent = hasStdinJsonPayload(toolUserArgs) ? readStdinSync() : null;
const { args: toolArgs, json: jsonOutput } = parseToolArgs(toolUserArgs, schemaProperties, stdinContent);

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot safeoutputs always require arguments so simplify detection without a regex

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done — shouldShowToolHelpForEmptyArgs now takes serverName and simply checks serverName === SAFEOUTPUTS_SERVER_NAME && Object.keys(toolArgs).length === 0, removing the inputSchema.required inspection entirely. Tests updated to match the new signature.

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Skills-Based Review 🧠

Applied /diagnosing-bugs and /tdd — approving with minor suggestions.

📋 Key Themes & Highlights

Key Themes

  • Root cause fully addressed: the guard fires at the bridge boundary before the MCP call, so write-once tools are never reached on empty-arg probes — not just better error messages.
  • Test coverage is focused and meaningful: three cases for the guard (positive, partial-fill, empty-required) plus the -32602 hint path. The missing gaps (null tool, no-toolName hint suppression) are small but worth closing.
  • formatResponse toolName threading: passing toolName as an optional parameter with "" default is clean and non-breaking.

Positive Highlights

  • ✅ Scoped to the CLI bridge — retry loop untouched, minimal blast radius
  • isSafeOutputsEmptyArgs narrows the hint to the exact server+code+message triple, avoiding spurious hints on unrelated -32602 errors
  • ✅ Audit log entries (show_tool_help_empty_args) make debugging easy in production logs
  • ✅ Export of shouldShowToolHelpForEmptyArgs keeps the function properly testable without coupling tests to main()

🧠 Reviewed using Matt Pocock's skills by Matt Pocock Skills Reviewer · 56.6 AIC · ⌖ 7.43 AIC · ⊞ 6.7K
Comment /matt to run again

}
});

it("shows help instead of calling required-argument tools with an empty args object", () => {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[/tdd] Missing null/undefined guard tests — the implementation handles them correctly but there are no tests confirming it.\n\n

\n💡 Two fast cases to add\n\njs\n// tool is null — should return false\nexpect(shouldShowToolHelpForEmptyArgs(null, {})).toBe(false);\n\n// tool has no inputSchema — should return false\nexpect(shouldShowToolHelpForEmptyArgs({}, {})).toBe(false);\n\n\nThese protect the !tool and inputSchema?.required guards if the function is ever refactored.\n\n
\n\n@copilot please address this.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The function was simplified to shouldShowToolHelpForEmptyArgs(serverName, toolArgs) — it no longer takes a tool object or inspects inputSchema, so null/undefined tool cases are no longer applicable. The updated tests reflect the new signature.

expect(process.exitCode).toBe(0);
});

it("adds a non-retry hint for safeoutputs empty-argument rejections", async () => {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[/tdd] The formatResponse hint test only covers the case where toolName is provided. A second assertion for the no-toolName path (toolName = "") would pin the intended silent behaviour.\n\n

\n💡 Suggested addition\n\njs\nit('omits non-retry hint when toolName is absent', async () => {\n await formatResponse(\n { error: { code: -32602, message: 'Empty arguments are not allowed — ...' } },\n 'safeoutputs'\n // toolName omitted → defaults to ""\n );\n const stderr = stderrChunks.join('');\n expect(stderr).not.toContain('do not retry');\n});\n\n\nWithout this, a future change that makes the hint unconditional would still pass all tests.\n\n
\n\n@copilot please address this.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added the no-toolName test in the commit $(git rev-parse --short HEAD). When toolName is omitted (defaults to ""), the hint is suppressed and only the base error line appears on stderr.

Comment thread actions/setup/js/mcp_cli_bridge.cjs Outdated
if (!Array.isArray(required) || required.length === 0) {
return false;
}
return Object.keys(toolArgs).length === 0;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[/diagnosing-bugs] Object.keys(toolArgs) treats only own enumerable string keys as empty — prototype-inherited properties are correctly ignored, but the comment/doc doesn't note this. More importantly, a partially-filled toolArgs (e.g. only optional args supplied) correctly returns false, so the guard is sound.\n\nNo code change needed — but consider noting in the JSDoc that toolArgs is expected to be a plain parsed-args object (not a class instance), to prevent future misuse.\n\n@copilot please address this.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The function was completely replaced — it no longer inspects toolArgs as a class instance or touches inputSchema at all. The new signature is shouldShowToolHelpForEmptyArgs(serverName, toolArgs) and only checks serverName === SAFEOUTPUTS_SERVER_NAME && Object.keys(toolArgs).length === 0, so the prototype-inherited-properties concern is no longer applicable.

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The implementation is clean and well-targeted. The empty-args guard, hint enrichment, and tests are all correct with no blocking issues.

🧵 Reviewed using Impeccable skills by Impeccable Skills Reviewer · 55.9 AIC · ⌖ 5.79 AIC · ⊞ 4.9K

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔎 Code quality review by PR Code Quality Reviewer · 173.3 AIC · ⌖ 5.66 AIC · ⊞ 5.4K
Comment /review to run again

Comment thread actions/setup/js/mcp_cli_bridge.cjs Outdated
const stdinContent = hasStdinJsonPayload(toolUserArgs) ? readStdinSync() : null;
const { args: toolArgs, json: jsonOutput } = parseToolArgs(toolUserArgs, schemaProperties, stdinContent);

if (shouldShowToolHelpForEmptyArgs(matchedTool, toolArgs)) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing server-name guard means all MCP servers are affected, not just safeoutputs: any server whose tool declares required fields will have its empty-args call silently replaced with help output and a success exit code.

💡 Suggested fix

The docstring and PR description both explicitly state this guard is safeoutputs-only to avoid interfering with other MCP servers that legitimately accept empty arguments. The call site in main() applies the intercept unconditionally:

// current — fires for every server
if (shouldShowToolHelpForEmptyArgs(matchedTool, toolArgs)) {

Gate it:

if (serverName === SAFEOUTPUTS_SERVER_NAME && shouldShowToolHelpForEmptyArgs(matchedTool, toolArgs)) {

Without this, a future MCP server whose tools have required fields but accept empty-body requests (e.g., a zero-arg trigger command documented with a non-empty required array) will silently degrade: callers get help text and exit 0 instead of a successful tool call.

Comment thread actions/setup/js/mcp_cli_bridge.cjs Outdated
if (!Array.isArray(required) || required.length === 0) {
return false;
}
return Object.keys(toolArgs).length === 0;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Object.keys(toolArgs) throws TypeError when toolArgs is null: the function is exported and called from user-facing tests directly, so a null/undefined argument causes an unhandled exception rather than a clean false return.

💡 Suggested fix

The call site in main() always passes the result of parseToolArgs (a plain object), but the function is exported and tested directly — a caller passing null will get a runtime crash instead of false:

// line 947 — current
return Object.keys(toolArgs).length === 0;

Add a guard:

return !toolArgs || typeof toolArgs !== 'object' || Array.isArray(toolArgs)
  ? false
  : Object.keys(toolArgs).length === 0;

Or more concisely:

return Object.keys(toolArgs ?? {}).length === 0;

The existing JSDoc types toolArgs as Record<string, unknown> but the exported API makes no such guarantee at runtime, and test files pass object literals directly.

Comment thread actions/setup/js/mcp_cli_bridge.cjs Outdated
const stdinContent = hasStdinJsonPayload(toolUserArgs) ? readStdinSync() : null;
const { args: toolArgs, json: jsonOutput } = parseToolArgs(toolUserArgs, schemaProperties, stdinContent);

if (shouldShowToolHelpForEmptyArgs(matchedTool, toolArgs)) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Empty piped stdin silently produces help output with exit code 0, masking pipeline errors: when a caller pipes an empty stream (e.g. echo -n | safeoutputs create_issue), hasStdinJsonPayload returns true because args.length === 0 && !process.stdin.isTTY, parseToolArgs then returns {}, and the empty-args guard fires — printing help and returning without setting process.exitCode = 1.

💡 Explanation

Sequence for echo -n | safeoutputs create_issue:

  1. toolUserArgs = []
  2. hasStdinJsonPayload([]) && !isTTYtrue
  3. readStdinSync()''
  4. parseToolArgs([], {}, ''){ args: {} }
  5. shouldShowToolHelpForEmptyArgs(matchedTool, {})true
  6. Shows help, returnexit code stays 0

A script that pipes an empty body almost certainly has a bug (e.g., a prior command produced no output). Rewarding that with exit 0 hides the error silently.

At minimum, set process.exitCode = 1 before returning when the empty-args help guard fires, so callers can detect the failure:

if (serverName === SAFEOUTPUTS_SERVER_NAME && shouldShowToolHelpForEmptyArgs(matchedTool, toolArgs)) {
  core.warning(...);
  auditLog(...);
  showToolHelp(serverName, toolName, tools);
  process.exitCode = 1;  // signal failure to caller
  return;
}

: "";
const errText = code ? `Error [${code}]: ${message}${hint}` : `Error: ${message}`;
process.stderr.write(errText + "\n");
core.error(`[${serverName}] Tool call error: ${errText}`);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Multi-line hint injected into errText pollutes core.error output and the structured audit log: the hint string contains embedded newlines and is concatenated directly into errText, which is then passed to both core.error() and auditLog(), making the audit log harder to parse programmatically and the CI error annotation span multiple unexpected lines.

💡 Suggested fix

Log the base error separately from the hint:

const baseErrText = code ? `Error [${code}]: ${message}` : `Error: ${message}`;
process.stderr.write(baseErrText + '\n');
core.error(`[${serverName}] Tool call error: ${baseErrText}`);
auditLog(serverName, { event: 'tool_error', error: baseErrText });
process.exitCode = 1;
if (hint) process.stderr.write(hint + '\n');
return;

This keeps the audit log field as a single-line string, keeps core.error() annotations clean, and still surfaces the hint to the human reader on stderr.


it("keeps top-level help compact for many commands", () => {
const tools = Array.from({ length: 25 }, (_, i) => ({
name: `tool_${i + 1}`,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No test asserts that the hint is absent for non-safeoutputs servers: the new test proves the hint appears for safeoutputs with a matching -32602 message, but nothing prevents a future change that accidentally widens the condition to all servers.

💡 Suggested addition

Add a negative assertion immediately after the existing test:

it('does NOT add a non-retry hint for -32602 errors from non-safeoutputs servers', async () => {
  await formatResponse(
    {
      error: {
        code: -32602,
        message: 'Empty arguments are not allowed — this tool is write-once, not a discovery probe.',
      },
    },
    'agenticworkflows', // different server
    'some_tool'
  );

  const stderr = stderrChunks.join('');
  expect(stderr).toContain('Error [-32602]');
  expect(stderr).not.toContain('do not retry');
  expect(stderr).not.toContain('--help');
  expect(process.exitCode).toBe(1);
});

Without this, a regression that broadens hint injection to all servers goes undetected by the test suite.

…ad of schema inspection

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copilot AI requested a review from pelikhan July 5, 2026 15:55
@gh-aw-bot

Copy link
Copy Markdown
Collaborator

@copilot please run the pr-finisher skill, Update the branch if needed, complete the remaining work, and rerun checks.

Generated by 👨‍🍳 PR Sous Chef · 3.83 AIC · ⌖ 6.47 AIC · ⊞ 3.7K ·
Comment /souschef to run again

…and non-safeoutputs paths

Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>
Copilot AI requested a review from gh-aw-bot July 5, 2026 16:17
@gh-aw-bot

Copy link
Copy Markdown
Collaborator

@copilot please run the pr-finisher skill, review the latest feedback, and rerun checks once the branch is ready.

Generated by 👨‍🍳 PR Sous Chef · 5.29 AIC · ⌖ 5.03 AIC · ⊞ 7.1K ·
Comment /souschef to run again

Copilot AI commented Jul 5, 2026

Copy link
Copy Markdown
Contributor Author

@copilot please run the pr-finisher skill, review the latest feedback, and rerun checks once the branch is ready.

...

Here's the status:

  • Reviews — the @pelikhan thread (simplify shouldShowToolHelpForEmptyArgs to use serverName check) was already addressed in commit d7c3412. No open in-scope review threads remain.
  • Checks (local)make fmt, make lint pass. All 59 mcp_cli_bridge.test.cjs tests pass. One pre-existing test-unit failure (TestPRSousChefWorkflowAddCommentTargetContract) exists on main before this branch and is unrelated to these changes.
  • Checks (CI) — stale after agent push; needs human re-trigger. One additional JS test failure (mount_mcp_as_cli.test.cjs > throws when safeoutputs tools remain empty after fallback) was found to be pre-existing and caused by the same RUNNER_TEMP isolation issue — that file was not modified by this PR.
  • Mergeable — branch is not behind, no conflicts.

Actions taken: Fixed test isolation in mcp_cli_bridge.test.cjs — the fails fast when safeoutputs schema is empty test now pins RUNNER_TEMP to a non-existent path so it doesn't accidentally resolve to a real tools.json in the runner environment (commit 8e63a68, pending push).

Hand-off: CI must be re-triggered by a maintainer before merge. The two pre-existing failures (TestPRSousChefWorkflowAddCommentTargetContract in Go and mount_mcp_as_cli.test.cjs throw-test) pre-date this branch and should be tracked separately.

@pelikhan pelikhan merged commit 1ac3557 into main Jul 5, 2026
15 checks passed
@pelikhan pelikhan deleted the copilot/aw-failures-bound-cli-retry-loop branch July 5, 2026 16:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

4 participants