Skip to content

MCPToolset does not propagate trace headers if present #4768

Closed

Description

@conorhodder

🔴 Required Information

Describe the Bug:
When the ADK's MCP toolset makes outbound HTTP requests to downstream MCP servers, the httpx client used does not propagate W3C traceparent headers. This means agent → MCP service calls create disconnected traces, breaking distributed trace correlation between the agent and the services it calls tools on.

The root cause is that StreamableHTTPConnectionParams uses a default httpx_client_factory (create_mcp_http_client) that creates a plain httpx.AsyncClient without OpenTelemetry instrumentation. Even though the TracerProvider and TraceContextTextMapPropagator are correctly configured by the ADK, the MCP HTTP client is never instrumented, so the propagator is never invoked on outbound requests.

Steps to Reproduce:

  1. Deploy an ADK agent with an MCP toolset connecting to a downstream service over HTTP:
    campaign_tools = McpToolset(
        connection_params=StreamableHTTPConnectionParams(
            url="http://some-mcp-server:8080/mcp",
        ),
    )
  2. Set OTEL_EXPORTER_OTLP_ENDPOINT on both the agent and the downstream service
  3. Send a message that triggers the agent to call an MCP tool
  4. Observe in Jaeger that the agent trace and the downstream service trace are separate trace roots — the MCP HTTP request does not carry a traceparent header

Expected Behavior:
Outbound MCP HTTP requests should carry the W3C traceparent header from the current trace context, so that the downstream service creates child spans under the agent's trace, producing a single end-to-end distributed trace (e.g. web → agent → MCP service).

Observed Behavior:
The MCP service receives requests without a traceparent header and creates a new trace root. The agent's trace shows execute_tool get_order_details but the downstream service's POST /mcp span is in a separate, unrelated trace.

Workaround: provide a custom httpx_client_factory that instruments the client:

from opentelemetry.instrumentation.httpx import HTTPXClientInstrumentor
from google.adk.tools.mcp_tool.mcp_session_manager import create_mcp_http_client

def create_instrumented_mcp_client(headers=None, timeout=None, auth=None):
    client = create_mcp_http_client(headers=headers, timeout=timeout, auth=auth)
    HTTPXClientInstrumentor.instrument_client(client)
    return client

campaign_tools = McpToolset(
    connection_params=StreamableHTTPConnectionParams(
        url="http://campaign:8080/mcp/mcp",
        httpx_client_factory=create_instrumented_mcp_client,
    ),
)

Environment Details:

  • ADK Library Version: google-adk 1.25.0
  • Desktop OS: Linux 6.10.14-linuxkit (Docker)
  • Python Version: 3.14.3

Model Information:

  • Are you using LiteLLM: No
  • Which model is being used: gemini-2.5-flash

🟡 Optional Information

Regression:
N/A — create_mcp_http_client has never applied OTel instrumentation. The httpx_client_factory parameter on StreamableHTTPConnectionParams was added as an extension point but the default factory does not instrument.

Logs:

# Default MCP client factory creates a plain httpx.AsyncClient:
def create_mcp_http_client(headers=None, timeout=None, auth=None):
    kwargs = {"follow_redirects": True}
    if timeout is None:
        kwargs["timeout"] = httpx.Timeout(MCP_DEFAULT_TIMEOUT, read=MCP_DEFAULT_SSE_READ_TIMEOUT)
    else:
        kwargs["timeout"] = timeout
    if headers is not None:
        kwargs["headers"] = headers
    if auth is not None:
        kwargs["auth"] = auth
    return httpx.AsyncClient(**kwargs)
    # ^ No OTel instrumentation applied

# Meanwhile _setup_instrumentation_lib_if_installed() only instruments GoogleGenAiSdkInstrumentor,
# not httpx clients.

Suggested Fix:
In create_mcp_http_client, conditionally instrument the client when OpenTelemetry is available:

def create_mcp_http_client(headers=None, timeout=None, auth=None):
    # ... existing logic ...
    client = httpx.AsyncClient(**kwargs)

    try:
        from opentelemetry.instrumentation.httpx import HTTPXClientInstrumentor
        HTTPXClientInstrumentor.instrument_client(client)
    except ImportError:
        pass

    return client

This is consistent with the existing try/except ImportError pattern used for GoogleGenAiSdkInstrumentor in _setup_instrumentation_lib_if_installed().

How often has this issue occurred?:

  • Always (100%)

Activity

  1. added
    tracing[Component] This issue is related to OpenTelemetry tracing
    on Mar 9, 2026
  2. added theissue type on Mar 9, 2026
  3. self-assigned this
    on Mar 9, 2026
  4. klateefa commented on Mar 9, 2026

    @klateefa

    @conorhodder Thanks for reporting.The issue occurs because the default MCP HTTP client is not instrumented with OpenTelemetry, so trace headers are not included in outbound requests.
    You can try instrumenting the default httpx.AsyncClient so the trace context is propagated to downstream MCP services:

    client = httpx.AsyncClient(**kwargs)
    
    try:
        from opentelemetry.instrumentation.httpx import HTTPXClientInstrumentor
        HTTPXClientInstrumentor.instrument_client(client)
    except ImportError:
        pass
    
    return client
    
    

    This should allow the agent and MCP services to appear under a single distributed trace.
    Thank you.

  5. added
    request clarification[Status] The maintainer need clarification or more information from the author
    and removed
    request clarification[Status] The maintainer need clarification or more information from the author
    on Apr 13, 2026
  6. added
    request clarification[Status] The maintainer need clarification or more information from the author
    on Apr 17, 2026
  7. surajksharma07 commented on May 27, 2026

    @surajksharma07
    Collaborator

    @conorhodder — following up here. Since v1.26.0, ADK now propagates trace context through the MCP protocol's _meta field (using the OTel global propagator) when calling tools, so upgrading should give you trace correlation at the MCP protocol level.

    If you also need W3C traceparent headers injected at the raw HTTP layer, the custom httpx_client_factory workaround from the Mar 9 comment still applies for that.

    Since we haven't heard back since the issue was opened, going ahead and closing this out — feel free to reopen if you're still seeing disconnected traces after upgrading to v1.26.0+!

  8. added a commit that references this issue on Jun 22, 2026
    431e3c2
  9. added a commit that references this issue on Aug 11, 2026
    c62aa99
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

request clarification[Status] The maintainer need clarification or more information from the authortracing[Component] This issue is related to OpenTelemetry tracing

Type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions