Skip to content

LiteLLMIntegration stores a non-JSON-serializable Span in request metadata, breaking outbound LLM requests #6596

@jgreer013

Description

@jgreer013

How do you use Sentry?

Sentry SaaS (sentry.io)

Version

sentry-sdk 2.52.0 (also present on current master); litellm 1.81.9; Python 3.13

Steps to Reproduce

With the LiteLLM integration enabled, any call that passes caller metadata crashes during request serialization:

import asyncio, litellm, sentry_sdk
from sentry_sdk.integrations.litellm import LiteLLMIntegration

sentry_sdk.init(
    dsn="<your-dsn>", traces_sample_rate=1.0,
    send_default_pii=True, integrations=[LiteLLMIntegration()],
)

async def main():
    await litellm.anthropic.messages.acreate(
        model="anthropic/claude-3-5-sonnet-latest",
        messages=[{"role": "user", "content": "hi"}],
        max_tokens=16, stream=True, api_key="sk-ant-...",
        metadata={"user_id": "my-org"},   # Anthropic abuse-isolation id
    )

asyncio.run(main())

Root cause: _input_callback stores the live span via _get_metadata_dict(kwargs)["_sentry_span"] = span (sentry_sdk/integrations/litellm.py). litellm's Anthropic /v1/messages passthrough forwards metadata into the request body, so the live Span lands at request_body['metadata']['_sentry_span'] and json.dumps(request_body) fails — before the request is sent.

Expected Result

Enabling the integration must not make outbound provider requests unserializable. The integration's bookkeeping span should be stashed off-band (e.g. a WeakKeyDictionary keyed on the logging object) rather than written into the caller's request metadata dict, which several providers (Anthropic here) forward into the wire payload.

Actual Result

TypeError: Object of type Span is not JSON serializable

(a Transaction when there is no active parent span). Separately, with send_default_pii=True + include_prompts=True (the default), that span's gen_ai.request.messages holds the verbatim prompt — so any sink that serializes this span object can leak prompt content.


Related (defense-in-depth, downstream): litellm forwards/serializes this injected metadata into the provider body. Companion issue filed at BerriAI/litellm — see comment below. litellm has stripped such span objects in other logging paths: BerriAI/litellm#15728, BerriAI/litellm#12354.

Metadata

Metadata

Assignees

No one assigned
    No fields configured for issues without a type.

    Projects

    Status
    Waiting for: Product Owner

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions