fix(v2): tell a caller when to come back on every failure meant to be retried - #6625
fix(v2): tell a caller when to come back on every failure meant to be retried#6625waleedlatif1 wants to merge 2 commits into
Conversation
… retried Three related gaps in retry signalling, found auditing the v2 surface against RFC 9110/6585 and against how Stripe, GitHub and Google's AIPs handle the same problems. **No 503 carried `Retry-After`.** Every one of them — the three route builders' `unhandledErrorResponse`, the execute and resume routes, and `serviceFailureResponse` — funnels through `v2Error`, so the default lands there, keyed on the response *status*: `Retry-After` is defined against the status, and the status is the only half of the code/status pair a client sees. A caller that supplies its own value still wins. RFC 9110 §15.6.4 makes this a `MAY` rather than a `SHOULD`, so it is a deliberate improvement, not a conformance fix: without it a client's only defensible policy on a 503 is an immediate retry, and Sim raises 503 exactly when a dependency is too degraded to absorb one. **A 429 that already knew its wait threw it away.** The admission descriptors declare `retryAfterSeconds` per denial, but mapping a descriptor onto a preprocess error copied only `statusCode`, `code` and `retryable`. A concurrency denial therefore reached the client as a bare 429 with no `Retry-After` despite the policy layer having named the wait five seconds earlier. The value now travels `descriptor.retryAfterSeconds` → `PreprocessExecutionError.retryAfterMs` → `ExecuteWorkflowServiceFailure.retryAfterMs` → `serviceFailureResponse`, so the transport reads a number the policy owns instead of re-guessing one. The 503 default is now only the floor for paths with no policy signal. **One failure must not advise a retry at all.** `ASYNC_ENQUEUE_AMBIGUOUS` is a 503 whose enqueue may have succeeded — it deliberately retains its execution-ID claim because a job may already exist. Telling that caller to come back in five seconds invites a client with no `X-Run-Id` to start, and bill, a second run of the same workflow. It opts out via `omitRetryAfter` and returns the run id so the caller reconciles instead. `ADMISSION_RETRY_AFTER_SECONDS` is reused rather than restated, so the execute route's capacity 429 and every other surface's 503 cannot drift apart. Also records the audit in `.agents/skills/v2-api-conventions/SKILL.md`: the retry rule, the cursor-tampering invariants, and reasoned rejections of RFC 9457 problem+json, the `RateLimit-*` draft fields, renaming `X-RateLimit-*` under RFC 6648, 422-for-semantic-validation, `Location` on 201, ETag/`If-Match`, and `merge-patch+json` — each with the spec text and the industry evidence, so they are not re-litigated. `Deprecation`/`Sunset` on v1 is left open pending a retirement date, which is a product decision.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
PR SummaryMedium Risk Overview
Admission OpenAPI documents Tests cover Reviewed by Cursor Bugbot for commit 6de7ad9. Configure here. |
Greptile SummaryThis PR adds consistent retry guidance to transient v2 API failures while preventing blind retries when async enqueue success is ambiguous.
Confidence Score: 5/5The PR appears safe to merge. No blocking failure remains.
|
| Filename | Overview |
|---|---|
| apps/sim/app/api/v2/lib/response.ts | Centralizes the default 503 retry delay, preserves explicit headers, and supports deliberate omission for ambiguous outcomes. |
| apps/sim/app/api/v2/workflows/[id]/execute/route.ts | Propagates policy-owned retry timing and suppresses blind-retry guidance for ambiguous async enqueue failures. |
| apps/sim/lib/execution/preprocessing.ts | Carries admission descriptor retry delays into preprocessing errors without assigning delays to non-retryable denials. |
| apps/sim/lib/api/contracts/v2/openapi/shared.ts | The shared 503 response now accurately describes the normally-present Retry-After header and the ambiguous-enqueue exception. |
| apps/sim/lib/billing/calculations/usage-reservation.ts | Exposes the admission policy's infrastructure retry delay for propagation to API callers. |
Reviews (2): Last reviewed commit: "docs(v2): name the one 503 that omits Re..." | Re-trigger Greptile
The shared ServiceUnavailable description claimed every 503 carries the header, which the ASYNC_ENQUEUE_AMBIGUOUS response deliberately does not. It now says the header is normally present and names that exception, so the published contract matches the runtime behaviour for all 128 operations.
|
@greptile review |
|
@cursor review |
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit 6de7ad9. Configure here.
Three related gaps in retry signalling on the v2 surface, found auditing it against RFC 9110 / RFC 6585 and against how Stripe, GitHub and Google's AIPs handle the same problems. Builds on #6620.
What was wrong
1. No 503 carried
Retry-After. Not one of them: the three route builders'unhandledErrorResponse, the execute and resume routes, andserviceFailureResponse. A client's only defensible policy on a bare 503 is an immediate retry — which is exactly the traffic a degraded dependency cannot absorb, and Sim raises 503 precisely when the API-key store, the rollout gate, the rate-limit backend, or execution-identity allocation is briefly unavailable.2. A 429 that already knew its wait threw it away. This is the more interesting one.
ADMISSION_ERROR_DESCRIPTORdeclaresretryAfterSeconds: 5forRESERVATION_CONCURRENCY, but mapping a descriptor onto aPreprocessExecutionErrorcopied onlystatusCode,codeandretryable. A concurrency denial therefore reached the client as a bare 429 with noRetry-After— despite the policy layer having named the wait five seconds earlier, and despite the OpenAPI already documentingRateLimited.headers: ['Retry-After']. The 503 infrastructure denial had the same leak; the new default happened to mask it with the same number.3. One 503 must not advise a retry at all.
ASYNC_ENQUEUE_AMBIGUOUSmeans the enqueue may have succeeded — it deliberately retains its execution-ID claim because a job may already exist. Telling that caller to come back in 5s invites a client with noX-Run-Idto start, and bill, a second run of the same workflow.What changed
v2Errorapplies aRetry-Afterdefault keyed on the response status (not the v2 error code —Retry-Afteris defined against the status, and the status is the only half of the pair a client sees). A caller-supplied value still wins.retryAfterSecondsnow travelsdescriptor→PreprocessExecutionError.retryAfterMs→ExecuteWorkflowServiceFailure.retryAfterMs→serviceFailureResponse. The transport reads a number the policy owns instead of re-guessing it; thev2Errordefault is now only the floor for paths with no policy signal.ASYNC_ENQUEUE_AMBIGUOUSopts out viaomitRetryAfterand returns the run id so the caller reconciles rather than retries.ADMISSION_RETRY_AFTER_SECONDSrather than restating5, so the execute route's capacity 429 and every other surface's 503 cannot drift apart.Retry-Afterto the sharedServiceUnavailableresponse — all 128 operations, 7 specs.The audit
.agents/skills/v2-api-conventions/SKILL.mdrecords the standards review so it is not re-litigated: the retry rule, the cursor-tamper-evidence invariants, and reasoned rejections of RFC 9457problem+json, the IETFRateLimit-*draft fields, renamingX-RateLimit-*under RFC 6648, 422-for-semantic-validation,Locationon 201, ETag /If-Match, andmerge-patch+json— each with the spec text and the industry evidence.Highlights of the "no" cases, since the evidence was not what I expected:
X-headers. §1 item 4 "makes no recommendation as to whether existing 'X-' parameters ought to remain in use or be migrated", and Appendix B argues the migration is the interoperability harm. The rule is aSHOULD NOTbinding creators of new parameters.RateLimit-*draft is unpublished, was returned "Not ready" at HTTPDIR review, and is on its third mutually incompatible wire format. 0 of Stripe/GitHub/Google emit it.etagfield (AIP-154), deliberately notIf-Match.Retry-Afteris only aMAYon 429 and 503 — it isSHOULDon exactly one status, 413. So this PR is a deliberate improvement, not a conformance fix, and says so.Deprecation/Sunseton v1 is left open: emitting either commits Sim to a retirement date, which is a product decision, not an engineering one.Idempotency is assessed and deliberately not extended here.
X-Run-Idalready gives the money path at-most-once semantics via a durable claim, and the operation description already says it is not an idempotency key. A true Stripe-styleIdempotency-Keyneeds a request fingerprint, a retention window, an in-flight-vs-completed split, and somewhere to store a large synchronous body — a designed piece of work, not an increment. Aliasing the header name without the replay semantics would be worse than leaving it, since clients written against Stripe would read our 409 as a hard failure.Verification
bun run type-check,check:api-validation,check:openapi(7 specs, 128 operations, 225 examples), biome on all changed files, and 1299 tests across the v2 / routes / execution / billing suites — all green.