Skip to content

feat(otel): add proprietary span converter#117

Open
pradystar wants to merge 1 commit into
feat/canonical-otel-attribute-normalizationfrom
feat/proprietary-span-otlp-converter
Open

feat(otel): add proprietary span converter#117
pradystar wants to merge 1 commit into
feat/canonical-otel-attribute-normalizationfrom
feat/proprietary-span-otlp-converter

Conversation

@pradystar

Copy link
Copy Markdown
Collaborator

Summary

Add a stateless conversion layer that transforms completed proprietary SDK spans into OpenTelemetry ReadableSpan objects. This prepares handler-based integrations for the SDK's OTLP export pipeline without requiring those integrations to change how they create spans.

The converter produces the same preliminary attribute contract used by SDK-native instrumentation while preserving the OpenTelemetry identity and routing Resource assigned when the proprietary span was created.

Why

Handler-based integrations currently produce Galileo span models rather than native OpenTelemetry spans. Moving their telemetry to OTLP requires a single conversion boundary that preserves existing instrumentation behavior, avoids duplicate attribute-mapping implementations, and produces standard completed spans that can be submitted to the SDK's existing batch processor.

Keeping conversion stateless and separate from logger lifecycle and export wiring makes the behavior independently testable and prevents trace buffering or a second batching implementation.

What changed

  • Added a SpanConverter that converts one completed Galileo span into one OpenTelemetry ReadableSpan.
  • Added conversion support for LLM, tool, retriever, workflow, agent, trace, and control spans.
  • Applied operation-specific span names and kinds, including SpanKind.CLIENT for LLM calls and SpanKind.INTERNAL for the remaining supported operations.
  • Preserved preassigned trace IDs, span IDs, parent context, remote-parent state, trace flags, and trace state without reading or replacing the active OpenTelemetry context.
  • Preserved the caller-provided Resource so project, log-stream, and experiment routing remains Resource-level telemetry rather than per-span attributes.
  • Mapped start time, duration-based or completion-time end time, HTTP/error status, empty events and links, and a versioned splunk_ao instrumentation scope.
  • Reused the shared canonical attribute builder exactly once per span and left final gen_ai.* to splunk_ao.* normalization at the common export boundary.
  • Rejected session and unknown step types because sessions are correlation metadata rather than exported spans.
  • Corrected a control-span mapping test fixture to use schema-valid JSON text while preserving its structured-content assertion.
  • Added focused coverage for supported types, naming, identity, distributed parents, resources, attributes, timestamps, status, instrumentation scope, unsupported types, and scorer omission.

DTB behavior

This change produces the completed ReadableSpan objects consumed by the existing DTB BatchSpanProcessor. It does not add batching, wait for an entire trace, force a flush, or export spans itself. Logger completion will submit these spans to the existing batch sink in the subsequent wiring change (HYBIM-890), allowing the standard batch processor to export completed spans on its configured schedule.

Testing

Tests: 1,998 passed and 10 skipped; focused converter/export lifecycle suite: 84 passed; Ruff lint/format, converter mypy, Poetry lock validation, and git diff --check passed.

Compatibility and scope

  • No handler, logger lifecycle, exporter, batching, authentication, routing, or deployment behavior changes in this PR.
  • Standalone and O11y behavior remains unchanged until the converter is wired into proprietary span completion.
  • Existing local-scorer APIs and behavior remain present; dynamically generated scorer results are intentionally not synthesized as OTLP attributes.
  • No dependency or lock-file changes.
  • Final attribute normalization remains shared across proprietary, SDK-native, and user-configured OpenTelemetry/OpenInference paths.

@fercor-cisco fercor-cisco left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 This review was generated by the Astra agent (claude-opus-4-8). It may contain mistakes.

Verdict: approve — Clean, well-tested stateless converter with no correctness/security issues; only minor observations about error-description mapping.

Follow-ups

Suggested follow-up work that could be tracked as Shortcut stories:

  • tests/test_attribute_mapping.py:147-158: The fixture change from input={"text": "question"} to input='{"text":"question"}' makes the ControlSpan schema-valid (native ControlSpan.input is StepAllowedInputType, which excludes plain dicts). Correct fix, but note the test now passes a pre-serialized string that _content_value returns verbatim rather than a structured object it serializes — so it no longer really exercises 'structured content' JSON-encoding. Consider a separate case that feeds structured input through the serialization path if that behavior still needs coverage.

Comment on lines +60 to +65
def _span_status(span: BaseStep) -> Status:
exception = getattr(span, "exception", None)
status_code = getattr(span, "status_code", None)
if exception is None and (status_code is None or status_code < 400):
return Status(StatusCode.UNSET)
return Status(StatusCode.ERROR, str(exception) if exception is not None else "")

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 minor (design): _span_status reads getattr(span, "exception", None), but no Galileo span type defines an exception field — BaseStep and every subclass (LlmSpan, ToolSpan, etc.) only carry status_code. So for every real span this getattr returns None, meaning: (1) the ERROR status only ever fires via status_code >= 400, and (2) error spans always get an empty status description (Status(StatusCode.ERROR, "")) since the description would come from the non-existent exception.

Handlers do set status_code=400/500 on failures, so ERROR is set correctly — but the human-readable error message is lost. Is that intended? If the error text should propagate into the OTel status description, consider sourcing it from an existing field (e.g. the span output, which handlers populate with "Error: ..."). If the exception branch is truly speculative/future, a comment noting that would help, since as written it looks like a supported path but is dead for all current span types.

🤖 Generated by the Astra agent

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tests/test_span_converter.py:412-420 (line not in diff)

🟡 minor (testing): test_exception_sets_error_status_and_description verifies the ERROR-with-description path by defining ToolSpanWithException(ToolSpan) with an exception field that no real Galileo span type has. This test passes but exercises a type that never occurs in production, so the description-propagation behavior is not actually covered for any real span. Consider adding coverage for how error messages surface on real error spans (which today only have status_code), or align this test with the actual span contract.

🤖 Generated by the Astra agent

@fercor-cisco fercor-cisco left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please see the suggested follow-ups.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants