Skip to content

fix: add _ExceptionTranslatingRoute to prevent keep-alive breakage on Linux - #4886

Merged
raghotham merged 1 commit into
ogx-ai:mainfrom
iamemilio:fix/exception-translating-route
Feb 12, 2026
Merged

raghotham merged 1 commit into
ogx-ai:mainfrom
iamemilio:fix/exception-translating-route

Conversation

@iamemilio

@iamemilio iamemilio commented Feb 10, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • Introduces _ExceptionTranslatingRoute, a custom APIRoute class that converts all non-HTTPException errors to HTTPException at the route level — before they reach Starlette's ServerErrorMiddleware.
  • Consolidates _http_exception_from_value_error and _http_exception_from_sse_error into a single _translate_to_http_exception helper.
  • Removes redundant inline try/except ValueError blocks from all agents route handlers (the route class handles this transparently).

Motivation

When a non-HTTPException propagates out of a route handler, Starlette's ServerErrorMiddleware sends a 500 response and then re-raises the exception. The re-raise causes uvicorn to close the HTTP transport, which on Linux immediately sends a TCP RST — breaking HTTP keep-alive connections. Subsequent requests on the same connection fail with APIConnectionError.

By converting every exception to HTTPException at the route level, FastAPI's built-in handler processes it cleanly — no re-raise, no transport close, no broken connections.

Test plan

  • test_exception_translating_route_converts_runtime_error_to_500 — verifies non-ValueError exceptions get proper JSON 500 responses (not plain-text from ServerErrorMiddleware)
  • test_consecutive_errors_keep_connection_alive — sends two error-producing requests on the same TestClient and verifies both get proper JSON error responses
  • Updated test_create_response_maps_value_error_to_400, test_get_response_maps_value_error_to_400, test_delete_response_maps_value_error_to_400 to use TestClient (exercises the route class end-to-end)

@meta-cla meta-cla Bot added the CLA Signed This label is managed by the Meta Open Source bot. label Feb 10, 2026
@iamemilio
iamemilio force-pushed the fix/exception-translating-route branch from 10cfaf9 to 5918aae Compare February 10, 2026 23:11
@iamemilio
iamemilio marked this pull request as draft February 11, 2026 00:13
@iamemilio
iamemilio force-pushed the fix/exception-translating-route branch from c17762d to b26f73d Compare February 11, 2026 00:58
Add a custom APIRoute class that translates known exception types
(ValueError, LlamaStackError) to HTTPException at the route level.
Unknown exceptions propagate to the server's global exception handler
which applies the full translate_exception pipeline.

This keeps ValueError -> 400 handling centralized (removing inline
try/except blocks from each endpoint) while letting the global handler
deal with richer mappings (AuthenticationRequiredError -> 401,
ConnectionError -> 502, etc.).

Co-authored-by: Cursor <cursoragent@cursor.com>
@iamemilio
iamemilio force-pushed the fix/exception-translating-route branch from b26f73d to 742536b Compare February 11, 2026 01:33
@iamemilio
iamemilio marked this pull request as ready for review February 11, 2026 01:37

@leseb leseb left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good find

@cdoern cdoern left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

iamemilio added a commit to iamemilio/llama-stack that referenced this pull request Feb 11, 2026
The OpenAI Python SDK parses error responses as body["error"]["message"].
The global exception handler was using "detail" which the SDK could not
surface in exception messages, breaking client error handling.

This keeps _ExceptionTranslatingRoute in fastapi_routes.py (from ogx-ai#4886)
to avoid keep-alive breakage on Linux. Route-level 400 responses use
"detail"; the global handler uses {"error": {"message": ...}}.

Unit tests updated: route-level ValueError->400 assert on detail; global
handler path (RuntimeError) asserts on error in json().

Co-authored-by: Cursor <cursoragent@cursor.com>
@raghotham
raghotham merged commit 7b2c727 into ogx-ai:main Feb 12, 2026
38 checks passed
@iamemilio
iamemilio deleted the fix/exception-translating-route branch February 17, 2026 16:12
iamemilio added a commit to iamemilio/llama-stack that referenced this pull request Feb 17, 2026
The OpenAI Python SDK parses error responses as body["error"]["message"].
The global exception handler was using "detail" which the SDK could not
surface in exception messages, breaking client error handling.

This keeps _ExceptionTranslatingRoute in fastapi_routes.py (from ogx-ai#4886)
to avoid keep-alive breakage on Linux. Route-level 400 responses use
"detail"; the global handler uses {"error": {"message": ...}}.

Unit tests updated: route-level ValueError->400 assert on detail; global
handler path (RuntimeError) asserts on error in json().

Co-authored-by: Cursor <cursoragent@cursor.com>
franciscojavierarceo pushed a commit that referenced this pull request Feb 17, 2026
Depends on: #4886

## Summary
- Introduce `LlamaStackError` base class with `status_code` attribute
for automatic HTTP status code translation in the server error handler
- Add `ClientListCommand` for consistent "not found" help messages and
new named error types: `BatchNotFoundError`,
`OpenAIFileObjectNotFoundError`, `ResponseNotFoundError`,
`InvalidParameterError`, `ServiceNotEnabledError`, `InternalServerError`
- Extract `translate_exception` into `core/exceptions` module with a
shared `EXCEPTION_MAP` for single-source-of-truth status code mapping
- Update file/batch providers to use specific error types instead of
generic `ResourceNotFoundError` with hardcoded strings

## Stack
This is PR 1/4 in the error message consistency series (split from
#3913):
1. **This PR** - Error types foundation
2. Responses API error handling (targets this branch)
3. Record/replay error support (targets this branch)
4. Integration tests (targets PR 2)

## Test plan
- [x] Unit tests for `ClientListCommand` and `ResourceNotFoundError` (7
tests pass)
- [x] All pre-commit hooks pass (ruff, mypy, API spec codegen, etc.)

---------

Co-authored-by: Cursor <cursoragent@cursor.com>
github-merge-queue Bot pushed a commit that referenced this pull request Mar 6, 2026
…uter (#5057)

## Summary

PR #4886 added `ExceptionTranslatingRoute` to the agents router to
prevent TCP connection resets on Linux. When known exceptions (like
`ValueError` or `LlamaStackError` subclasses) are thrown from a route
handler, `ServerErrorMiddleware` can close the transport before the
global exception handler has a chance to format a JSON response. The
route class catches these exceptions earlier and converts them to
`HTTPException`, which FastAPI handles gracefully without dropping the
connection.

The conversations router raises the same exception types
(`ConversationNotFoundError`, `InvalidParameterError`, etc.) but was
missing this route class, leaving it vulnerable to the same issue. This
PR:

- Moves `ExceptionTranslatingRoute` and
`try_translate_to_http_exception` to the shared `router_utils` module so
they can be reused across routers
- Applies `route_class=ExceptionTranslatingRoute` to the conversations
router
- Adds a unit test verifying consecutive errors on the conversations
router return proper JSON without breaking the connection

## Test plan
- [x] All 66 existing router tests pass (no regressions from moving to
shared module)
- [x] New unit test verifies keep-alive behavior on the conversations
router

---------

Co-authored-by: Francisco Javier Arceo <farceo@redhat.com>
Co-authored-by: Francisco Javier Arceo <arceofrancisco@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Meta Open Source bot.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants