Show safeoutputs command help on empty arguments#43566
Conversation
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
|
✅ Design Decision Gate 🏗️ completed the design decision gate check. No ADR enforcement needed: PR #43566 does not have the 'implementation' label and has 0 new lines of code in business logic directories (threshold: 100). |
|
🧠 Matt Pocock Skills Reviewer has completed the skills-based review. ✅ |
|
✅ Test Quality Sentinel completed test quality analysis. |
|
✅ PR Code Quality Reviewer completed the code quality review. |
There was a problem hiding this comment.
Pull request overview
This PR updates the MCP CLI bridge used by gh-aw setup scripts to avoid wasting time on “empty arguments” discovery probes—especially for write-once safeoutputs tools—by showing per-tool help immediately instead of issuing a failing MCP tools/call. It also strengthens the -32602 error output for the remaining empty-args rejection path with an explicit “do not retry” hint.
Changes:
- Add an empty-args guard (
shouldShowToolHelpForEmptyArgs) that shows tool-specific help when a tool has required inputs but received{}. - Enhance
formatResponseto append a non-retry hint forsafeoutputs-32602“Empty arguments are not allowed” errors (including pointing to<tool> --helpandnoop). - Add focused unit tests covering both the new help guard and the enhanced
-32602messaging.
Show a summary per file
| File | Description |
|---|---|
| actions/setup/js/mcp_cli_bridge.cjs | Adds the empty-args → help guard and augments safeoutputs -32602 output with a non-retry hint. |
| actions/setup/js/mcp_cli_bridge.test.cjs | Adds unit tests for the new guard behavior and the strengthened -32602 error message hinting. |
Review details
Tip
Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Files reviewed: 2/2 changed files
- Comments generated: 0
- Review effort level: Low
🧪 Test Quality Sentinel Report✅ Test Quality Score: 100/100 — Excellent
📊 Metrics (2 tests)
Verdict
Warning Firewall blocked 1 domainThe following domain was blocked by the firewall during workflow execution:
network:
allowed:
- defaults
- "awmgmcpg"See Network Configuration for more information.
|
| @@ -1231,6 +1260,13 @@ async function main() { | |||
| const stdinContent = hasStdinJsonPayload(toolUserArgs) ? readStdinSync() : null; | |||
| const { args: toolArgs, json: jsonOutput } = parseToolArgs(toolUserArgs, schemaProperties, stdinContent); | |||
There was a problem hiding this comment.
@copilot safeoutputs always require arguments so simplify detection without a regex
There was a problem hiding this comment.
Done — shouldShowToolHelpForEmptyArgs now takes serverName and simply checks serverName === SAFEOUTPUTS_SERVER_NAME && Object.keys(toolArgs).length === 0, removing the inputSchema.required inspection entirely. Tests updated to match the new signature.
There was a problem hiding this comment.
Skills-Based Review 🧠
Applied /diagnosing-bugs and /tdd — approving with minor suggestions.
📋 Key Themes & Highlights
Key Themes
- Root cause fully addressed: the guard fires at the bridge boundary before the MCP call, so write-once tools are never reached on empty-arg probes — not just better error messages.
- Test coverage is focused and meaningful: three cases for the guard (positive, partial-fill, empty-required) plus the
-32602hint path. The missing gaps (null tool, no-toolNamehint suppression) are small but worth closing. formatResponsetoolName threading: passingtoolNameas an optional parameter with""default is clean and non-breaking.
Positive Highlights
- ✅ Scoped to the CLI bridge — retry loop untouched, minimal blast radius
- ✅
isSafeOutputsEmptyArgsnarrows the hint to the exact server+code+message triple, avoiding spurious hints on unrelated -32602 errors - ✅ Audit log entries (
show_tool_help_empty_args) make debugging easy in production logs - ✅ Export of
shouldShowToolHelpForEmptyArgskeeps the function properly testable without coupling tests tomain()
🧠 Reviewed using Matt Pocock's skills by Matt Pocock Skills Reviewer · 56.6 AIC · ⌖ 7.43 AIC · ⊞ 6.7K
Comment /matt to run again
| } | ||
| }); | ||
|
|
||
| it("shows help instead of calling required-argument tools with an empty args object", () => { |
There was a problem hiding this comment.
[/tdd] Missing null/undefined guard tests — the implementation handles them correctly but there are no tests confirming it.\n\n
💡 Two fast cases to add
\n\njs\n// tool is null — should return false\nexpect(shouldShowToolHelpForEmptyArgs(null, {})).toBe(false);\n\n// tool has no inputSchema — should return false\nexpect(shouldShowToolHelpForEmptyArgs({}, {})).toBe(false);\n\n\nThese protect the !tool and inputSchema?.required guards if the function is ever refactored.\n\nThere was a problem hiding this comment.
The function was simplified to shouldShowToolHelpForEmptyArgs(serverName, toolArgs) — it no longer takes a tool object or inspects inputSchema, so null/undefined tool cases are no longer applicable. The updated tests reflect the new signature.
| expect(process.exitCode).toBe(0); | ||
| }); | ||
|
|
||
| it("adds a non-retry hint for safeoutputs empty-argument rejections", async () => { |
There was a problem hiding this comment.
[/tdd] The formatResponse hint test only covers the case where toolName is provided. A second assertion for the no-toolName path (toolName = "") would pin the intended silent behaviour.\n\n
💡 Suggested addition
\n\njs\nit('omits non-retry hint when toolName is absent', async () => {\n await formatResponse(\n { error: { code: -32602, message: 'Empty arguments are not allowed — ...' } },\n 'safeoutputs'\n // toolName omitted → defaults to ""\n );\n const stderr = stderrChunks.join('');\n expect(stderr).not.toContain('do not retry');\n});\n\n\nWithout this, a future change that makes the hint unconditional would still pass all tests.\n\nThere was a problem hiding this comment.
Added the no-toolName test in the commit $(git rev-parse --short HEAD). When toolName is omitted (defaults to ""), the hint is suppressed and only the base error line appears on stderr.
| if (!Array.isArray(required) || required.length === 0) { | ||
| return false; | ||
| } | ||
| return Object.keys(toolArgs).length === 0; |
There was a problem hiding this comment.
[/diagnosing-bugs] Object.keys(toolArgs) treats only own enumerable string keys as empty — prototype-inherited properties are correctly ignored, but the comment/doc doesn't note this. More importantly, a partially-filled toolArgs (e.g. only optional args supplied) correctly returns false, so the guard is sound.\n\nNo code change needed — but consider noting in the JSDoc that toolArgs is expected to be a plain parsed-args object (not a class instance), to prevent future misuse.\n\n@copilot please address this.
There was a problem hiding this comment.
The function was completely replaced — it no longer inspects toolArgs as a class instance or touches inputSchema at all. The new signature is shouldShowToolHelpForEmptyArgs(serverName, toolArgs) and only checks serverName === SAFEOUTPUTS_SERVER_NAME && Object.keys(toolArgs).length === 0, so the prototype-inherited-properties concern is no longer applicable.
There was a problem hiding this comment.
The implementation is clean and well-targeted. The empty-args guard, hint enrichment, and tests are all correct with no blocking issues.
🧵 Reviewed using Impeccable skills by Impeccable Skills Reviewer · 55.9 AIC · ⌖ 5.79 AIC · ⊞ 4.9K
There was a problem hiding this comment.
🔎 Code quality review by PR Code Quality Reviewer · 173.3 AIC · ⌖ 5.66 AIC · ⊞ 5.4K
Comment /review to run again
| const stdinContent = hasStdinJsonPayload(toolUserArgs) ? readStdinSync() : null; | ||
| const { args: toolArgs, json: jsonOutput } = parseToolArgs(toolUserArgs, schemaProperties, stdinContent); | ||
|
|
||
| if (shouldShowToolHelpForEmptyArgs(matchedTool, toolArgs)) { |
There was a problem hiding this comment.
Missing server-name guard means all MCP servers are affected, not just safeoutputs: any server whose tool declares required fields will have its empty-args call silently replaced with help output and a success exit code.
💡 Suggested fix
The docstring and PR description both explicitly state this guard is safeoutputs-only to avoid interfering with other MCP servers that legitimately accept empty arguments. The call site in main() applies the intercept unconditionally:
// current — fires for every server
if (shouldShowToolHelpForEmptyArgs(matchedTool, toolArgs)) {Gate it:
if (serverName === SAFEOUTPUTS_SERVER_NAME && shouldShowToolHelpForEmptyArgs(matchedTool, toolArgs)) {Without this, a future MCP server whose tools have required fields but accept empty-body requests (e.g., a zero-arg trigger command documented with a non-empty required array) will silently degrade: callers get help text and exit 0 instead of a successful tool call.
| if (!Array.isArray(required) || required.length === 0) { | ||
| return false; | ||
| } | ||
| return Object.keys(toolArgs).length === 0; |
There was a problem hiding this comment.
Object.keys(toolArgs) throws TypeError when toolArgs is null: the function is exported and called from user-facing tests directly, so a null/undefined argument causes an unhandled exception rather than a clean false return.
💡 Suggested fix
The call site in main() always passes the result of parseToolArgs (a plain object), but the function is exported and tested directly — a caller passing null will get a runtime crash instead of false:
// line 947 — current
return Object.keys(toolArgs).length === 0;Add a guard:
return !toolArgs || typeof toolArgs !== 'object' || Array.isArray(toolArgs)
? false
: Object.keys(toolArgs).length === 0;Or more concisely:
return Object.keys(toolArgs ?? {}).length === 0;The existing JSDoc types toolArgs as Record<string, unknown> but the exported API makes no such guarantee at runtime, and test files pass object literals directly.
| const stdinContent = hasStdinJsonPayload(toolUserArgs) ? readStdinSync() : null; | ||
| const { args: toolArgs, json: jsonOutput } = parseToolArgs(toolUserArgs, schemaProperties, stdinContent); | ||
|
|
||
| if (shouldShowToolHelpForEmptyArgs(matchedTool, toolArgs)) { |
There was a problem hiding this comment.
Empty piped stdin silently produces help output with exit code 0, masking pipeline errors: when a caller pipes an empty stream (e.g. echo -n | safeoutputs create_issue), hasStdinJsonPayload returns true because args.length === 0 && !process.stdin.isTTY, parseToolArgs then returns {}, and the empty-args guard fires — printing help and returning without setting process.exitCode = 1.
💡 Explanation
Sequence for echo -n | safeoutputs create_issue:
toolUserArgs = []hasStdinJsonPayload([]) && !isTTY→truereadStdinSync()→''parseToolArgs([], {}, '')→{ args: {} }shouldShowToolHelpForEmptyArgs(matchedTool, {})→true- Shows help,
return— exit code stays 0
A script that pipes an empty body almost certainly has a bug (e.g., a prior command produced no output). Rewarding that with exit 0 hides the error silently.
At minimum, set process.exitCode = 1 before returning when the empty-args help guard fires, so callers can detect the failure:
if (serverName === SAFEOUTPUTS_SERVER_NAME && shouldShowToolHelpForEmptyArgs(matchedTool, toolArgs)) {
core.warning(...);
auditLog(...);
showToolHelp(serverName, toolName, tools);
process.exitCode = 1; // signal failure to caller
return;
}| : ""; | ||
| const errText = code ? `Error [${code}]: ${message}${hint}` : `Error: ${message}`; | ||
| process.stderr.write(errText + "\n"); | ||
| core.error(`[${serverName}] Tool call error: ${errText}`); |
There was a problem hiding this comment.
Multi-line hint injected into errText pollutes core.error output and the structured audit log: the hint string contains embedded newlines and is concatenated directly into errText, which is then passed to both core.error() and auditLog(), making the audit log harder to parse programmatically and the CI error annotation span multiple unexpected lines.
💡 Suggested fix
Log the base error separately from the hint:
const baseErrText = code ? `Error [${code}]: ${message}` : `Error: ${message}`;
process.stderr.write(baseErrText + '\n');
core.error(`[${serverName}] Tool call error: ${baseErrText}`);
auditLog(serverName, { event: 'tool_error', error: baseErrText });
process.exitCode = 1;
if (hint) process.stderr.write(hint + '\n');
return;This keeps the audit log field as a single-line string, keeps core.error() annotations clean, and still surfaces the hint to the human reader on stderr.
|
|
||
| it("keeps top-level help compact for many commands", () => { | ||
| const tools = Array.from({ length: 25 }, (_, i) => ({ | ||
| name: `tool_${i + 1}`, |
There was a problem hiding this comment.
No test asserts that the hint is absent for non-safeoutputs servers: the new test proves the hint appears for safeoutputs with a matching -32602 message, but nothing prevents a future change that accidentally widens the condition to all servers.
💡 Suggested addition
Add a negative assertion immediately after the existing test:
it('does NOT add a non-retry hint for -32602 errors from non-safeoutputs servers', async () => {
await formatResponse(
{
error: {
code: -32602,
message: 'Empty arguments are not allowed — this tool is write-once, not a discovery probe.',
},
},
'agenticworkflows', // different server
'some_tool'
);
const stderr = stderrChunks.join('');
expect(stderr).toContain('Error [-32602]');
expect(stderr).not.toContain('do not retry');
expect(stderr).not.toContain('--help');
expect(process.exitCode).toBe(1);
});Without this, a regression that broadens hint injection to all servers goes undetected by the test suite.
…ad of schema inspection Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
|
@copilot please run the
|
…and non-safeoutputs paths Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>
|
@copilot please run the
|
Here's the status:
Actions taken: Fixed test isolation in Hand-off: CI must be re-triggered by a maintainer before merge. The two pre-existing failures ( |
Reviewer workflows were burning the full Copilot CLI step timeout when write-once safe-output tools were probed with empty arguments and repeatedly rejected with
-32602. This change makes empty-arg safeoutputs calls resolve to help output instead of invoking the tool, and strengthens the remaining rejection path with an explicit non-retry hint.Fail fast on empty safeoutputs probes
tools/callthat is guaranteed to fail.Strengthen the terminal hint for
-32602-32602fromsafeoutputs, append a direct instruction not to retry.<tool> --helpfor schema discovery andnoopfor no-op completion.Keep the change scoped to the CLI bridge
Focused coverage
-32602error messaging.