Should MCP standardize a retry-timing hint, or leave it to SDKs? #3188
Replies: 6 comments 3 replies
|
Standardize it, and I'd argue for going slightly beyond a bare timing hint, because timing turns out to be only half of what clients need. I've spent the past weeks building exactly this layer as a transparent proxy (mcp-fuse, MIT) and can share what the payload converged to after contact with real servers. The fragmentation you list matches what I ran into. My classifier literally has to regex retry_after out of free text because every server spells it differently. That alone makes the case for a standard shape. What I'd add to the retry-after field, based on implementation experience: A category enum plus a retryable boolean as the minimal contract (transient, rate_limit, timeout, auth, invalid_input, and a few more). A client that understands nothing else still behaves correctly; timing refines it. A defined carrier at each layer where failures surface: error.data for JSON-RPC errors, result._meta for tool results with isError, and a response header for the HTTP transport case, so the signal survives an opaque 5xx body. The timing hint should mean "not before", not "sleep this long". When a server says wait 12s and the host times the call out at 10s, sleeping is the wrong move. What worked in practice: fail fast, remember the earliest retry time, and reject repeat calls cheaply until it passes. Auto-retry needs to be gated on tool annotations. readOnlyHint and idempotentHint already exist in the spec. A client that silently replays an ambiguous failure against a tool with side effects risks double-executing it. Making that gating normative would also give server authors a concrete reason to annotate accurately. Draft schema, JSON Schema file and conformance examples are here: https://github.com/YoadElkayam/mcp-fuse/tree/main/spec. The proxy synthesizes the payload for servers that don't emit it, so I also have data on what a classifier can and can't recover from free text; happy to share if useful. There's a related thread in #2930 about the classification half of the same problem. On the title question: SDK-only conventions are how the ecosystem got three incompatible retry_after fields. Even a small normative payload, or a reserved _meta key with a defined shape, would let opencode and vercel/ai trigger the retry logic they already have. |
|
Strong +1 on standardizing, and @YoadElkayam's point about gating auto-retry on We scanned 671 published MCP servers (27,153 declared tools) looking specifically at retry safety. Two findings relevant here: 1. The annotation that would make a retry-timing hint safe is usually absent. Of the servers doing real writes (80% of the sample), 32% had no visible idempotency guard of any kind. A 2. When the annotation is present, it can be decorative. In at least one server we read closely, a create tool was marked non-idempotent in its metadata, but nothing at runtime consulted that flag before the retry loop fired. So "the annotation exists" and "the annotation gates retry" are separate claims, and a normative spec is what closes the gap between them. The piece I'd add to the payload, beyond category + timing: a reconciliation pointer for the ambiguous case. The hardest failure isn't a clean rate-limit — it's the timeout where the request may have already landed and only the response was lost. I sketched this as a per-tool "retry contract" — effect class, caller-key support (+ scope + retention), duplicate-key/different-payload behavior, ambiguous-timeout reconciliation, and whether downstream dedup is relied upon — deliberately as a superset of the existing hints rather than a competitor to them: https://github.com/aurumflux20/fencescan/blob/main/docs/RETRY-CONTRACT.md. It's a draft; I'd rather have holes poked in it than adopt it, and the honesty boundary (marking what a scanner can't verify from code alone) is the part I'm least sure about. Happy to share the scan dataset/method if the "how common is this actually" question is useful to the SEP — the aggregate is public, no server names. |
|
The scan data is exactly what this thread was missing. The 32% figure and especially your finding 2 are the strongest argument yet for normative over advisory. Finding 2 is actually why mcp-fuse exists as a proxy rather than a library: if the server won't consult its own annotation at runtime, enforcement has to live somewhere that will. Today the proxy is the thing making "the annotation gates retry" a true claim regardless of what the server does. A normative spec would make it true for clients generally, and I'd happily retire that part of the proxy's job. On the reconciliation pointer: agreed, and I want to steal it. Right now when a non-idempotent tool fails ambiguously my proxy refuses to replay and tells the model to verify before re-invoking. That's correct but it ends in a dead end the model has to figure out on its own. A machine-readable pointer to the verification read (your "did this land?" query) turns the dead end into an automatable step. I'd fold it into the error-policy payload as an optional field rather than a parallel structure. Having read your RETRY-CONTRACT draft, I think the two pieces are complementary and the seam between them is clean: your contract is what a tool declares about itself ahead of time (effect class, caller keys, dedup behavior), my payload is what just happened to this particular call (category, timing, circuit state, and potentially your reconciliation pointer). The declaration answers "may I ever replay this", the payload answers "what do I do right now". A client needs both. Your effectClass also fixes a real weakness in my gate: idempotentHint is a boolean, and "reversible" vs "irreversible" is a distinction my proxy currently can't act on. Yes please on the dataset. I maintain a corpus-driven classifier that turns raw error text into the structured payload, and I'd like to run it against what your scan collected, both for category coverage and for how often retry timing is recoverable from free text. If the maintainers signal interest in an SEP, want to draft it together? Category and timing from my side, effect declaration and reconciliation from yours, your scan as the motivation section. It feels like one proposal, not two. |
|
Yes to both — and the dataset is up rather than promised: https://github.com/aurumflux20/fencescan/tree/main/docs/scan-dataset 671 anonymized rows plus the aggregate, Apache-2.0. Per repo: declared tools, write sites, visible guards, retry loops, candidate count, and which of three structural patterns matched. No names, no owners, downloads bucketed. The 84 repos that failed to clone are recorded as failures, never folded into the zeros — that particular mistake cost me a whole prospect pool once, so it's now a column rather than a footnote. One honest limit before you point your classifier at it: it contains no error strings. The scan reads source, never runtime output, so it can tell you how often a guard is absent but nothing about how often Also worth flagging so nobody over-reads the table: the pattern counts (63 / 14 / 10) are lower bounds from a deliberately conservative matcher, not prevalence rates. The scanner has no verdict field on purpose — of the first seven candidates I hand-read, four were wrong, and an outsider usually cannot prove a double-fire because the guard often lives in a service the repo merely calls. On the seam: your split is right and cleaner than how I'd been holding it. Declaration answers "may I ever replay this", payload answers "what do I do right now". The one refinement I'd make is that the reconciliation pointer wants to live in both, doing different jobs — statically, the tool declares which read answers "did this land?"; dynamically, the payload says this particular call is in that ambiguous state, go run it. Keeping the pointer only in the payload means a server has to synthesize it at failure time, which is exactly the moment it's least able to. And yes — So: happy to draft it together. Category and timing yours, effect declaration and reconciliation mine, and I'll write the motivation section off the scan since it's my data to defend. I'd rather the maintainers signal interest before we spend the effort — @devmaha, you have the SEP draft in flight already; would you rather this fold into yours as an extension, or stand as a companion? I'd sooner ship one proposal than seed a fourth convention. |
|
Both refinements land for me. The pointer living in both places is right, and it fits the seam cleanly: the declaration names the verification read once, ahead of time, and the payload just flags that this particular call is in the ambiguous state. And the tri-state effect class is a real fix rather than a nicety; my proxy currently has to treat "reversible" and "irreversible" the same way, which makes it over-cautious about one and under-informative about the other. The joined corpus is the piece I'll pick up. mcp-fuse already logs every raw failure string it sees at runtime (that's its audit log), so the runtime half mostly exists; the work is exercising servers enough to make them fail in interesting ways. I'll start with a small slice, the reference servers plus whatever I can drive locally, join it against fencescan on the same repos, and share the method before scaling anything. @devmaha same preference as @aurumflux20 here: if the shape works for you, folding into your draft beats a companion. One proposal, not a fourth convention. Happy to hand over our pieces in whatever format merges easiest. |
|
Relevant data point for the normative-vs-advisory question, from trying to answer it a different way. Rather than asking servers, I tried to infer retry-safety from source: does a retry construct wrap a payment call with no idempotency identity in scope? Built it as a public tool: agent-money-test (MIT, free, no signup). It turned out to be a good argument for a normative field, not against one. Even with careful static analysis — matching only real retry constructs, stripping comments and string literals, scoping identity checks per-file — hand-verifying the results still found the tool wrong about a third of what it flagged: a frontend component displaying payment status read as a payment call, a status-poll loop read as a payment retry, a tutorial's "solution" file counted as a live service. If inferring retry-safety from source is this unreliable even when done carefully, that is the case for the server declaring it directly rather than leaving callers to guess. Happy to be wrong — the code and the false-positive writeup are both public, and the harness self-tests against a known-broken and a known-safe target before it trusts its own verdict on anything else. |
|
That one-in-three false positive rate from careful static analysis is the number the motivation section needs. It also matches the view from the runtime side: sitting between host and server, the proxy can observe failures but cannot infer whether an effect is safe to repeat, so the only reliable input it has is what the tool declares. Inference and observation both point at declaration. One update on process. devmaha's idempotency SEP (#3182) was closed last week under the repo's AI contribution policy, so I don't think waiting to fold into that draft is the right plan anymore. I'd rather we start the companion as a human-written draft, keep it small, put the implementation and the scan data behind it, and open it with full disclosure. I set up a shared working draft with the section split we agreed on and TODO markers per owner: https://github.com/YoadElkayam/mcp-fuse/tree/main/sep. PRs welcome from anyone here, it moves to the spec repo's seps/ directory once it has a sponsor. @aurumflux20 @johnyzaguirre-glean your sections are marked. @devmaha your retryAfter work is welcome in it if you want to bring it over, you're listed as an invited co-author. The SEP guidelines say to bring a proposal to a relevant interest or working group before a cold submission. The Interceptors WG looks like the natural home for this one, so I'll raise it there next. For the record and per AI_POLICY.md: I use Claude Code to help draft my comments and code, including this one. The design decisions, the proxy implementation and its testing are mine, and I stand behind all of it. |
|
Works for me on all three counts. A small human-written companion with the implementations and data behind it is the honest shape anyway — the argument for this SEP was always the evidence, not the prose. I've read the draft and my sections are the right cut: 3.3 and 4.3 are the two halves of one argument (inference cannot replace declaration; therefore the tool must declare, and the client must consult). I'll PR them this week in that order:
On the Interceptors WG as the venue sounds right. I'll have my sections in the working draft before that conversation so it lands with the evidence attached. Per AI_POLICY.md: I use Claude Code to help draft comments and code, including this one. The scan, the datasets, the RETRY-CONTRACT design and the implementations behind my sections are mine, and I stand behind all of it. |
|
Verdict space in 4.3, and I will keep 4.1's reconcile as a bare pointer with a cross-reference to it. Your framing is the right one: the pointer says what to run, 4.3 says how to read the answer, and "could not determine is terminal" is the sentence that closes the gate. Agree on _meta preservation over a new field; I wrote it up that way in the draft but marked it as a question for the sponsor, since a preservation rule lands on SDK maintainers and they should see it coming. Looking forward to the PRs. (drafted with Claude Code, same disclosure as above) |
Uh oh!
There was an error while loading. Please reload this page.
MCP has no standard way for a server to tell a client "this failure is retryable, and here's roughly how long to wait" — error.data is intentionally application-defined, and JSON-RPC gives it no shape for this. Three independently-built MCP servers already fill that gap today, each incompatibly:
Docdex: custom error code -32029, fields retry_after_ms/retry_at
slack-mcp: a dedicated exception class, field retry_after
mcp-time-server-node: code -32000, field retryAfter (seconds)
That's not a hypothetical concern — it has a real, documented cost. opencode (a widely-used open-source MCP client) has an issue where it aborts an entire in-progress task the moment a tool returns a rate-limit error, despite already having working retry logic for LLM-provider rate limits elsewhere in the same client — the logic exists, there's just nothing standard to trigger it on. vercel/ai and awslabs/mcp both have independent issues describing the same underlying gap with different symptoms (a hallucinated retry loop; a server wrongly marked "unavailable").
For what it's worth as outside precedent: Cloudflare's production edge network standardizes exactly this shape of signal — a retry-timing value kept in sync between an HTTP header and a JSON body field — across seven different error causes, only one of which is a rate limit. Same design choice, at real scale, for the same reason.
I've worked this into a full SEP draft with a minimal design (one field, error.data.retryAfter, an integer/null/absent) and a working reference implementation, but before opening that as a PR I want to check the more basic question: does this belong in the protocol, or is it better left as something each SDK exposes on its own? Happy to share the fuller writeup, FAQ, or reference implementation if useful context.
All reactions