[FEATURE] mcp: adopt protocol revision 2026-07-28 and add OpenTelemetry log output format #729

Summary

Two related changes to stackql mcp, delivered together in one PR:

1. Adopt MCP protocol revision 2026-07-28 (the stateless core revision), retaining compatibility with prior revisions per the spec's version negotiation rules.
2. Add an OpenTelemetry output format option for the tool/decision log (currently bespoke JSONL), mapped to the OTel GenAI/MCP semantic conventions, so the log stream can be consumed by any OTLP-compatible collector or backend without a custom parser.

Downstream sinks (ClickHouse/ClickStack, OTLP network exporters, dashboards) are explicitly out of scope - they will be built separately against the format defined here. This issue makes the log stream standards-shaped; that is all.

Part 1: protocol revision 2026-07-28

2026-07-28 is the largest revision since launch. Relevant changes (see changelog and release post):

• The initialize/notifications/initialized handshake is removed; every request now carries the protocol version and client capabilities in _meta (io.modelcontextprotocol/protocolVersion, io.modelcontextprotocol/clientCapabilities).
• Protocol-level sessions and the Mcp-Session-Id header are removed from Streamable HTTP; tools/list, resources/list and prompts/list must not vary per connection.
• Cross-call state moves to explicit server-minted handles passed as ordinary tool arguments (SEP-2567).
• Extensions are formalized (extensions field on capabilities, SEP-2133); Tasks moved out of core into an extension (we do not use Tasks - note only).
• Authorization hardening (RFC 9207 iss validation, application_type in DCR, CIMD) - audit which items apply to our HTTP transport surface, if any.

Tasks:

• Advertise 2026-07-28; keep prior revisions (2025-06-18, 2025-11-25) working per version negotiation
• stdio transport: handle new-revision clients that send no initialize handshake; keep the handshake path for old-revision clients
• Streamable HTTP (if/where enabled): remove any Mcp-Session-Id dependence; make list endpoints connection-invariant
• Audit cross-call state in the server (server mode, provider auth context, anything session-scoped) - either make it stateless or move it behind explicit handles per SEP-2567
• Verify the safe mode elicitation flow (gated writes) against 2026-07-28 request/response shapes on both transports - this is our headline feature, it must not regress
• Bump SDK dependency to a version with 2026-07-28 support
• Conformance tests against both a 2026-07-28 client and a prior-revision client

Part 2: OpenTelemetry log output format

Today the server emits one JSONL record per operation: verbatim SQL, mode, elicitation decision, timestamps. This stays the default and stays byte-compatible.

Add a log format option:

stackql mcp --log-format jsonl   # default, unchanged
stackql mcp --log-format otel    # OTLP/JSON log records to the same destination (stdout/file)

Design points for otel:

• One log record per tool invocation and one per elicitation decision, mapped to the OTel GenAI/MCP semantic conventions: gen_ai.operation.name = execute_tool, gen_ai.tool.name, gen_ai.tool.call.id, plus the mcp.* attributes (method, protocol version) where defined.
• StackQL-specific fields carried under a namespaced prefix: stackql.query (verbatim SQL), stackql.provider, stackql.mode, stackql.decision, stackql.rows_returned, operation duration.
• Trace/span correlation: honor caller-supplied trace context when present in _meta; generate ids otherwise, so records from one agent session correlate.
• The GenAI semconv is Development status, not stable: pin the semconv version we emit, record it in the scope attributes, and treat the emitted attribute set as a versioned interface with a test asserting the schema.
• Redaction guarantee is preserved identically in both formats: values marked protected are never serialized. This is a hard requirement, not best-effort.
• Output destination is unchanged (stdout/file, collector tails it). No network exporter in this PR.

Acceptance criteria

• Server negotiates 2026-07-28 with a current client and still interoperates with a 2025-06-18 client
• safe mode gated-write demo passes unchanged on both revisions
• --log-format otel output is accepted by a stock OTel Collector (filelog receiver -> OTLP pipeline) with no transform processor
• Default jsonl output is byte-compatible with current behavior
• No protected value appears in either log format (test with a RETURNING secret)
• Docs updated: protocol revision support matrix, --log-format reference, attribute schema table with pinned semconv version

Out of scope (follow-up issues)

• OTLP network exporter (push to a collector endpoint)
• ClickHouse/ClickStack sink, schemas and dashboards
• Metrics and spans (this issue covers the log signal only)