Skip to content

Support OTLP export over "http/json" for traces, metrics, and logs#12018

Open
mcculls wants to merge 1 commit into
masterfrom
mcculls/otlp-json
Open

Support OTLP export over "http/json" for traces, metrics, and logs#12018
mcculls wants to merge 1 commit into
masterfrom
mcculls/otlp-json

Conversation

@mcculls

@mcculls mcculls commented Jul 21, 2026

Copy link
Copy Markdown
Contributor

What Does This Do

Support OTLP export over "http/json" for traces, metrics, and logs.

Adds JSON siblings of existing hand-rolled protobuf OTLP encoders (OtlpCommonJson, OtlpResourceJson, OtlpTraceJson/Collector, OtlpMetricsJson/Collector, OtlpLogsJson/Collector) and wires the previously unused/silently-disabled HTTP_JSON protocol into OtlpWriter, OtlpMetricsService, and OtlpLogsService.

Motivation

Adds support for OTLP over http/json to the existing http/protobuf and grpc support.

Contributor Checklist

Jira ticket: [PROJ-IDENT]

@mcculls mcculls added type: feature Enhancements and improvements inst: opentelemetry OpenTelemetry instrumentation labels Jul 21, 2026
@mcculls mcculls changed the title Add OTLP/JSON export support for traces, metrics, and logs Support OTLP export over "http/json" for traces, metrics, and logs Jul 21, 2026
@datadog-datadog-prod-us1

This comment has been minimized.

@dd-octo-sts

dd-octo-sts Bot commented Jul 21, 2026

Copy link
Copy Markdown
Contributor

🟢 Java Benchmark SLOs — All performance SLOs passed

Suite Status
Startup 🟢 pass

SLO thresholds are defined here based on automatically generated metrics. A warning is raised when results are within 5% of the threshold.

PR vs. master results
Scenario Candidate master Δ (95% CI of mean)
startup:insecure-bank:iast:Agent 13.96 s 13.94 s [-0.6%; +0.8%] (no difference)
startup:insecure-bank:tracing:Agent 12.86 s 13.05 s [-2.2%; -0.8%] (maybe better)
startup:petclinic:appsec:Agent 16.93 s 16.72 s [+0.4%; +2.1%] (maybe worse)
startup:petclinic:iast:Agent 16.95 s 16.92 s [-0.6%; +1.0%] (no difference)
startup:petclinic:profiling:Agent 16.79 s 16.79 s [-1.3%; +1.3%] (no difference)
startup:petclinic:sca:Agent 17.02 s 16.61 s [+1.5%; +3.4%] (significantly worse)
startup:petclinic:tracing:Agent 16.16 s 16.07 s [-0.3%; +1.4%] (no difference)

Commit: 12ec75df · CI Pipeline · Benchmarking Platform UI


Load and DaCapo benchmarks can be triggered manually in the GitLab pipeline. Results will appear in the Benchmarking Platform UI after completion.

@mcculls
mcculls force-pushed the mcculls/otlp-json branch 2 times, most recently from 573483d to ae7ca71 Compare July 21, 2026 11:34
@mcculls
mcculls requested a review from Copilot July 21, 2026 11:35
@mcculls

mcculls commented Jul 21, 2026

Copy link
Copy Markdown
Contributor Author

@codex review

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This pull request adds first-class support for exporting OTLP signals over the http/json protocol across traces, metrics, and logs within dd-trace-core, aligning the agent’s OTLP export capabilities with the existing http/protobuf and grpc options.

Changes:

  • Added JSON OTLP encoders/collectors for traces, metrics, logs, plus shared JSON helpers.
  • Wired HTTP_JSON into OtlpWriter (traces) and into OtlpMetricsService / OtlpLogsService (metrics/logs).
  • Refactored shared OTLP constants/mappings (trace flags, span kind mapping, resource attributes, metrics temporality) to be reused across proto/JSON paths.

Reviewed changes

Copilot reviewed 35 out of 35 changed files in this pull request and generated no comments.

Show a summary per file
File Description
dd-trace-core/src/test/java/datadog/trace/core/otlp/trace/OtlpTraceProtoTest.java Updates trace-flag constant reference to shared OtlpTraceFlags.
dd-trace-core/src/test/java/datadog/trace/core/otlp/trace/OtlpTraceJsonCollectorTest.java Adds coverage for trace OTLP JSON encoding (hex ids, keys, kind, status).
dd-trace-core/src/test/java/datadog/trace/core/otlp/metrics/OtlpMetricsServiceTest.java Verifies metrics service selects JSON collector + HTTP endpoint under http/json.
dd-trace-core/src/test/java/datadog/trace/core/otlp/metrics/OtlpMetricsJsonCollectorTest.java Adds coverage for metrics OTLP JSON encoding (temporality, timestamps, histograms).
dd-trace-core/src/test/java/datadog/trace/core/otlp/logs/OtlpLogsServiceTest.java Verifies logs service selects JSON collector + HTTP endpoint under http/json.
dd-trace-core/src/test/java/datadog/trace/core/otlp/logs/OtlpLogsProtoTest.java Updates trace-flag constant reference to shared OtlpTraceFlags.
dd-trace-core/src/test/java/datadog/trace/core/otlp/logs/OtlpLogsJsonCollectorTest.java Adds coverage for logs OTLP JSON encoding (ids, flags, timestamps, eventName).
dd-trace-core/src/test/java/datadog/trace/core/otlp/common/OtlpResourceJsonTest.java Adds coverage for JSON resource fragment generation from Config.
dd-trace-core/src/test/java/datadog/trace/core/otlp/common/OtlpCommonJsonTest.java Adds coverage for common JSON encoding primitives (ids, attributes, arrays).
dd-trace-core/src/test/java/datadog/trace/common/writer/OtlpWriterCombinedTest.java Adds an end-to-end assertion that HTTP_JSON sends Content-Type: application/json.
dd-trace-core/src/main/java/datadog/trace/core/otlp/trace/OtlpTraceProto.java Refactors span kind + trace flags to shared helpers/constants.
dd-trace-core/src/main/java/datadog/trace/core/otlp/trace/OtlpTraceJsonCollector.java New trace JSON collector producing OTLP trace.proto JSON payloads.
dd-trace-core/src/main/java/datadog/trace/core/otlp/trace/OtlpTraceJson.java New trace JSON writer for spans and span links in OTLP JSON shape.
dd-trace-core/src/main/java/datadog/trace/core/otlp/trace/OtlpSpanKind.java New shared span-kind mapping for proto + JSON encoders.
dd-trace-core/src/main/java/datadog/trace/core/otlp/metrics/OtlpMetricsTemporality.java Extracts temporality selection to a shared helper used by proto/JSON.
dd-trace-core/src/main/java/datadog/trace/core/otlp/metrics/OtlpMetricsService.java Enables HTTP_JSON protocol and adds package-private accessors for testing.
dd-trace-core/src/main/java/datadog/trace/core/otlp/metrics/OtlpMetricsProto.java Switches to shared temporality constants (logic moved out).
dd-trace-core/src/main/java/datadog/trace/core/otlp/metrics/OtlpMetricsJsonCollector.java New metrics JSON collector producing OTLP metrics.proto JSON payloads.
dd-trace-core/src/main/java/datadog/trace/core/otlp/metrics/OtlpMetricsJson.java New metrics JSON writer (metric headers + data point values).
dd-trace-core/src/main/java/datadog/trace/core/otlp/logs/OtlpLogsService.java Enables HTTP_JSON protocol and adds package-private accessors for testing.
dd-trace-core/src/main/java/datadog/trace/core/otlp/logs/OtlpLogsProtoCollector.java Removes duplicated protobuf content-type constant (centralized elsewhere).
dd-trace-core/src/main/java/datadog/trace/core/otlp/logs/OtlpLogsProto.java Updates trace-flag constant reference to shared OtlpTraceFlags.
dd-trace-core/src/main/java/datadog/trace/core/otlp/logs/OtlpLogsJsonCollector.java New logs JSON collector producing OTLP logs.proto JSON payloads.
dd-trace-core/src/main/java/datadog/trace/core/otlp/logs/OtlpLogsJson.java New logs JSON writer for log-record JSON fields and flags.
dd-trace-core/src/main/java/datadog/trace/core/otlp/common/OtlpTraceFlags.java New shared trace/logs flags constants to avoid duplication.
dd-trace-core/src/main/java/datadog/trace/core/otlp/common/OtlpResourceProto.java Refactors resource attribute enumeration into shared helper.
dd-trace-core/src/main/java/datadog/trace/core/otlp/common/OtlpResourceJson.java New resource JSON fragment generator for OTLP JSON payloads.
dd-trace-core/src/main/java/datadog/trace/core/otlp/common/OtlpResourceAttributes.java New shared resource-attribute enumeration used by proto + JSON.
dd-trace-core/src/main/java/datadog/trace/core/otlp/common/OtlpProtoBuffer.java Centralizes protobuf content type via OtlpPayload.
dd-trace-core/src/main/java/datadog/trace/core/otlp/common/OtlpPayload.java Adds shared content-type constants for protobuf + JSON payloads.
dd-trace-core/src/main/java/datadog/trace/core/otlp/common/OtlpHttpSender.java Exposes url() accessor (used by tests).
dd-trace-core/src/main/java/datadog/trace/core/otlp/common/OtlpCommonJson.java New common OTLP JSON helpers (ids, scope/schema, attributes).
dd-trace-core/src/main/java/datadog/trace/core/otlp/common/LazyJsonArray.java New helper for lazily-opened JSON arrays used by JSON collectors.
dd-trace-core/src/main/java/datadog/trace/common/writer/OtlpWriter.java Chooses trace collector based on protocol (HTTP_JSON vs proto).
dd-trace-core/src/main/java/datadog/trace/common/writer/OtlpPayloadDispatcher.java Removes implicit default proto collector; collector now injected explicitly.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ae7ca71920

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread dd-trace-core/src/main/java/datadog/trace/core/otlp/common/OtlpCommonJson.java Outdated
Comment thread dd-trace-core/src/main/java/datadog/trace/core/otlp/metrics/OtlpMetricsJson.java Outdated
Comment thread dd-trace-core/src/main/java/datadog/trace/core/otlp/trace/OtlpTraceJson.java Outdated
@mcculls

mcculls commented Jul 21, 2026

Copy link
Copy Markdown
Contributor Author

@codex review

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 36 out of 36 changed files in this pull request and generated 2 comments.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. 🎉

Reviewed commit: 414de6e9eb

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

@mcculls
mcculls force-pushed the mcculls/otlp-json branch from b8387d8 to bcd2ba9 Compare July 21, 2026 14:35
@mcculls

mcculls commented Jul 21, 2026

Copy link
Copy Markdown
Contributor Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Bravo.

Reviewed commit: bcd2ba94ff

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 38 out of 38 changed files in this pull request and generated 1 comment.

Comment thread components/json/src/test/java/datadog/json/JsonWriterTest.java Outdated
@mcculls
mcculls force-pushed the mcculls/otlp-json branch from bcd2ba9 to 55d0509 Compare July 21, 2026 14:58
@mcculls
mcculls marked this pull request as ready for review July 21, 2026 14:59
@mcculls
mcculls requested review from a team as code owners July 21, 2026 14:59
@mcculls
mcculls requested review from PerfectSlayer and removed request for a team July 21, 2026 14:59
@dd-octo-sts dd-octo-sts Bot added the tag: ai generated Largely based on code generated by an AI or LLM label Jul 21, 2026
@mcculls
mcculls requested a review from mhlidd July 21, 2026 14:59
datadog-datadog-prod-us1[bot]

This comment was marked as outdated.

@mcculls
mcculls force-pushed the mcculls/otlp-json branch from 55d0509 to 4447563 Compare July 21, 2026 15:38

@mhlidd mhlidd left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Small nits from Claude

@mcculls
mcculls force-pushed the mcculls/otlp-json branch 2 times, most recently from 12ec75d to b08029a Compare July 21, 2026 23:44
Adds JSON siblings of existing hand-rolled protobuf OTLP encoders (OtlpCommonJson, OtlpResourceJson,
OtlpTraceJson/Collector, OtlpMetricsJson/Collector, OtlpLogsJson/Collector) and wires the previously
unused/silently-disabled HTTP_JSON protocol into OtlpWriter, OtlpMetricsService, and OtlpLogsService.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@mcculls
mcculls force-pushed the mcculls/otlp-json branch from b08029a to 27286d6 Compare July 21, 2026 23:50
@mcculls

mcculls commented Jul 21, 2026

Copy link
Copy Markdown
Contributor Author

@codex review

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 42 out of 42 changed files in this pull request and generated no new comments.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. What shall we delve into next?

Reviewed commit: 27286d6791

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

@mhlidd mhlidd left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@mcculls
mcculls added this pull request to the merge queue Jul 22, 2026
@dd-octo-sts

dd-octo-sts Bot commented Jul 22, 2026

Copy link
Copy Markdown
Contributor

/merge

@gh-worker-devflow-routing-ef8351

gh-worker-devflow-routing-ef8351 Bot commented Jul 22, 2026

Copy link
Copy Markdown

View all feedbacks in Devflow UI.

2026-07-22 21:17:08 UTC ℹ️ Start processing command /merge
Use /merge -c to cancel this operation!


2026-07-22 21:17:12 UTC ℹ️ MergeQueue: pull request added to the queue

The expected merge time in master is approximately 2h (p90).

Use /merge -c to cancel this operation!


⏳ Merge Commit: 4e13091
Build Pipeline: 7261441494940915927

@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Jul 22, 2026
@mcculls
mcculls added this pull request to the merge queue Jul 22, 2026
@dd-octo-sts

dd-octo-sts Bot commented Jul 22, 2026

Copy link
Copy Markdown
Contributor

/merge

@gh-worker-devflow-routing-ef8351

gh-worker-devflow-routing-ef8351 Bot commented Jul 22, 2026

Copy link
Copy Markdown

View all feedbacks in Devflow UI.

2026-07-22 21:58:17 UTC ℹ️ Start processing command /merge


2026-07-22 21:58:19 UTC ❌ MergeQueue

PR already in the queue with status in_progress

@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Jul 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

inst: opentelemetry OpenTelemetry instrumentation tag: ai generated Largely based on code generated by an AI or LLM type: feature Enhancements and improvements

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants