Repository navigation
fix: add _ExceptionTranslatingRoute to prevent keep-alive breakage on Linux - #4886
Merged
Merged
Conversation
iamemilio
requested review from
ashwinb,
bbrowning,
cdoern,
ehhuang,
franciscojavierarceo,
leseb,
mattf and
raghotham
as code owners
February 10, 2026 22:59
iamemilio
force-pushed
the
fix/exception-translating-route
branch
from
February 10, 2026 23:11
10cfaf9 to
5918aae
Compare
iamemilio
marked this pull request as draft
February 11, 2026 00:13
iamemilio
force-pushed
the
fix/exception-translating-route
branch
from
February 11, 2026 00:58
c17762d to
b26f73d
Compare
Add a custom APIRoute class that translates known exception types (ValueError, LlamaStackError) to HTTPException at the route level. Unknown exceptions propagate to the server's global exception handler which applies the full translate_exception pipeline. This keeps ValueError -> 400 handling centralized (removing inline try/except blocks from each endpoint) while letting the global handler deal with richer mappings (AuthenticationRequiredError -> 401, ConnectionError -> 502, etc.). Co-authored-by: Cursor <cursoragent@cursor.com>
iamemilio
force-pushed
the
fix/exception-translating-route
branch
from
February 11, 2026 01:33
b26f73d to
742536b
Compare
iamemilio
marked this pull request as ready for review
February 11, 2026 01:37
2 tasks done
iamemilio
added a commit
to iamemilio/llama-stack
that referenced
this pull request
Feb 11, 2026
The OpenAI Python SDK parses error responses as body["error"]["message"]. The global exception handler was using "detail" which the SDK could not surface in exception messages, breaking client error handling. This keeps _ExceptionTranslatingRoute in fastapi_routes.py (from ogx-ai#4886) to avoid keep-alive breakage on Linux. Route-level 400 responses use "detail"; the global handler uses {"error": {"message": ...}}. Unit tests updated: route-level ValueError->400 assert on detail; global handler path (RuntimeError) asserts on error in json(). Co-authored-by: Cursor <cursoragent@cursor.com>
raghotham
approved these changes
Feb 12, 2026
iamemilio
added a commit
to iamemilio/llama-stack
that referenced
this pull request
Feb 17, 2026
The OpenAI Python SDK parses error responses as body["error"]["message"]. The global exception handler was using "detail" which the SDK could not surface in exception messages, breaking client error handling. This keeps _ExceptionTranslatingRoute in fastapi_routes.py (from ogx-ai#4886) to avoid keep-alive breakage on Linux. Route-level 400 responses use "detail"; the global handler uses {"error": {"message": ...}}. Unit tests updated: route-level ValueError->400 assert on detail; global handler path (RuntimeError) asserts on error in json(). Co-authored-by: Cursor <cursoragent@cursor.com>
franciscojavierarceo
pushed a commit
that referenced
this pull request
Feb 17, 2026
Depends on: #4886 ## Summary - Introduce `LlamaStackError` base class with `status_code` attribute for automatic HTTP status code translation in the server error handler - Add `ClientListCommand` for consistent "not found" help messages and new named error types: `BatchNotFoundError`, `OpenAIFileObjectNotFoundError`, `ResponseNotFoundError`, `InvalidParameterError`, `ServiceNotEnabledError`, `InternalServerError` - Extract `translate_exception` into `core/exceptions` module with a shared `EXCEPTION_MAP` for single-source-of-truth status code mapping - Update file/batch providers to use specific error types instead of generic `ResourceNotFoundError` with hardcoded strings ## Stack This is PR 1/4 in the error message consistency series (split from #3913): 1. **This PR** - Error types foundation 2. Responses API error handling (targets this branch) 3. Record/replay error support (targets this branch) 4. Integration tests (targets PR 2) ## Test plan - [x] Unit tests for `ClientListCommand` and `ResourceNotFoundError` (7 tests pass) - [x] All pre-commit hooks pass (ruff, mypy, API spec codegen, etc.) --------- Co-authored-by: Cursor <cursoragent@cursor.com>
2 tasks done
github-merge-queue Bot
pushed a commit
that referenced
this pull request
Mar 6, 2026
…uter (#5057) ## Summary PR #4886 added `ExceptionTranslatingRoute` to the agents router to prevent TCP connection resets on Linux. When known exceptions (like `ValueError` or `LlamaStackError` subclasses) are thrown from a route handler, `ServerErrorMiddleware` can close the transport before the global exception handler has a chance to format a JSON response. The route class catches these exceptions earlier and converts them to `HTTPException`, which FastAPI handles gracefully without dropping the connection. The conversations router raises the same exception types (`ConversationNotFoundError`, `InvalidParameterError`, etc.) but was missing this route class, leaving it vulnerable to the same issue. This PR: - Moves `ExceptionTranslatingRoute` and `try_translate_to_http_exception` to the shared `router_utils` module so they can be reused across routers - Applies `route_class=ExceptionTranslatingRoute` to the conversations router - Adds a unit test verifying consecutive errors on the conversations router return proper JSON without breaking the connection ## Test plan - [x] All 66 existing router tests pass (no regressions from moving to shared module) - [x] New unit test verifies keep-alive behavior on the conversations router --------- Co-authored-by: Francisco Javier Arceo <farceo@redhat.com> Co-authored-by: Francisco Javier Arceo <arceofrancisco@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
_ExceptionTranslatingRoute, a customAPIRouteclass that converts all non-HTTPExceptionerrors toHTTPExceptionat the route level — before they reach Starlette'sServerErrorMiddleware._http_exception_from_value_errorand_http_exception_from_sse_errorinto a single_translate_to_http_exceptionhelper.try/except ValueErrorblocks from all agents route handlers (the route class handles this transparently).Motivation
When a non-
HTTPExceptionpropagates out of a route handler, Starlette'sServerErrorMiddlewaresends a 500 response and then re-raises the exception. The re-raise causes uvicorn to close the HTTP transport, which on Linux immediately sends a TCP RST — breaking HTTP keep-alive connections. Subsequent requests on the same connection fail withAPIConnectionError.By converting every exception to
HTTPExceptionat the route level, FastAPI's built-in handler processes it cleanly — no re-raise, no transport close, no broken connections.Test plan
test_exception_translating_route_converts_runtime_error_to_500— verifies non-ValueError exceptions get proper JSON 500 responses (not plain-text from ServerErrorMiddleware)test_consecutive_errors_keep_connection_alive— sends two error-producing requests on the same TestClient and verifies both get proper JSON error responsestest_create_response_maps_value_error_to_400,test_get_response_maps_value_error_to_400,test_delete_response_maps_value_error_to_400to use TestClient (exercises the route class end-to-end)