SEP-3304 - Standardizing Rate-Limiting Errors - #3304
Open
tyree731 wants to merge 3 commits into
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Abstract
This SEP standardizes on
-32023/RateLimitedfrom the reserved sub-range, with a typed response indicating when to retry after, with optional quota fields, binding the response to the HTTP 429 to additionally allow for middlewareproxies to detect this.
Motivation
There are a couple of reasons for this specification. stdio has no HTTP layer: For the stdio protocol, no header
exchange can occur at present, meaning that relying on the HTTP layer to communicate a rate limit being hit cannot work
with the stdio protocol. The Transports WG, in the current roadmap, has committed to changing the stdio protocol to
speak Streamable HTTP, but there is no timeframe for this as of now.
In addition, the current SDK implementations do not define rate limiting errors, and are inconsistent in how they
surface HTTP 429 errors:
How Each SDK Surfaces an HTTP 429
SdkHttpError(SdkErrorCode.ClientHttpNotImplemented, "Error POSTing to endpoint: ...")statuson the error objectErrorData(code=INTERNAL_ERROR, message="Server returned an error response")(-32603)HttpRequestExceptionwith the response body appended to the messageHttpRequestException.StatusCodeStreamableHttpError::UnexpectedServerResponse("HTTP 429 Too Many Requests: <body>")McpTransportException("Invalid request. Status code: 429")jsonrpc2.ErrRejected, so the connection is preservedhttp.StatusTextonlyStreamableHttpError(code = 429, message = body)codepropertysend()branches only onContent-Typeand never inspects the status code, so a 429 with a non-JSON body is silently droppedFaraday::Errorcatch-all, raisingRequestHandlerError(error_type: :internal_error)original_error.response[:status]MCPError.internalError("Too many requests")This inconsistency means that clients cannot reliably report or detect rate limits being hit across implementations.
How Has This Been Tested?
TODO.
Breaking Changes
The error code being used here is within the range of reserved codes for the MCP specification, so no existing client
should be relying on it. Older clients will see the same error shape they have previously with HTTP 429 errors, so only
clients which have implemented custom request and response handling using HTTP 429 errors will need to consider the new response shape.
Types of changes
Checklist
Additional context
Is this an error?
Yes. For much the same reason that
InvalidParamsErroris an error, namely that the client has made a mistake whichit needs to respond to,
RateLimitedErrorcommunicates that the client has requested the server too rapidly, and needsto back off appropriately.
Why milliseconds for retryAfterMs?
The Retry-After HTTP header supports both a decimal integer for its value, in addition to an HTTP date, so it's worth asking why we would use milliseconds here. The rationale is that other SEPs have standardized on milliseconds for their post-dated timings, such as SEP-2549, so we do so for consistency.