Remove full schema collection from Kafka, Avro, and Protobuf instrumentation by piochelepiotr · Pull Request #12038 · DataDog/dd-trace-java

piochelepiotr · 2026-07-22T17:46:16Z

Summary

Removes full Avro/Protobuf schema extraction and reporting from Data Streams Monitoring. It's a deprecated feature.
Deletes the avro-1.11.3 and protobuf-3.0 instrumentation modules, the AvroSchemaExtractor used by the Kafka producer instrumentation (kafka-clients-0.11, kafka-clients-3.8), and the supporting schema-building/sampling core (SchemaBuilder, SchemaSampler, Schema, SchemaIterator) in dd-trace-core / internal-api.
Removes the associated DDTags.SCHEMA_* span tags (SCHEMA_DEFINITION, SCHEMA_WEIGHT, SCHEMA_TYPE, SCHEMA_ID, SCHEMA_TOPIC, SCHEMA_OPERATION, SCHEMA_NAME).
Schema Registry utilization tracking (confluent-schema-registry serializer/deserializer instrumentation, reportSchemaRegistryUsage) is intentionally untouched — that is a separate feature (detecting registry usage/schema ID for DSM lineage) and continues to work as before.

Test plan

Updated/removed Groovy tests referencing removed schema-extraction behavior in kafka-clients-0.11 and kafka-clients-3.8 test suites (dropped the schema param and assertions from producerSpan() helpers; removed the standalone avro-schema-extraction test).
Removed now-obsolete SchemaBuilderTest / SchemaSamplerTest and the schemaSamplerSamplesWithCorrectWeights test in DefaultDataStreamsMonitoringTest, keeping all schema-registry-usage tests intact.
dd-trace-core datastreams tests, kafka-clients-0.11 tests, kafka-clients-3.8 tests, and confluent-schema-registry-4.1 compile all pass.
Ran /techdebt and /perf-review skills over the branch diff — no findings.

🤖 Generated with Claude Code

…ntation Data Streams Monitoring no longer extracts and reports full Avro/Protobuf schema definitions from traced messages. This removes the avro-1.11.3 and protobuf-3.0 instrumentation modules, the AvroSchemaExtractor used by the Kafka producer instrumentation, and the associated schema-building/sampling core in dd-trace-core and internal-api. Schema Registry utilization tracking (confluent-schema-registry) is untouched and continues to report schema ID/registry usage for DSM lineage. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

datadog-official · 2026-07-22T18:02:30Z

🎯 Code Coverage (details)
• Patch Coverage: 100.00%
• Overall Coverage: 57.35% (-0.02%)

_{This comment will be updated automatically if new data arrives.

🔗 Commit SHA: 9cf8108 | Docs | Datadog PR Page | Give us feedback!}

…grations Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

dd-octo-sts · 2026-07-22T19:15:16Z

🟢 Java Benchmark SLOs — All performance SLOs passed

Suite	Status
Startup	🟢 pass

SLO thresholds are defined here based on automatically generated metrics. A warning is raised when results are within 5% of the threshold.

PR vs. master results

Scenario	Candidate	master	Δ (95% CI of mean)
startup:insecure-bank:iast:Agent	13.97 s	14.02 s	[-0.9%; +0.3%] (no difference)
startup:insecure-bank:tracing:Agent	12.97 s	12.90 s	[-0.2%; +1.3%] (no difference)
startup:petclinic:appsec:Agent	16.98 s	16.83 s	[-0.1%; +2.0%] (no difference)
startup:petclinic:iast:Agent	16.87 s	16.96 s	[-1.4%; +0.4%] (no difference)
startup:petclinic:profiling:Agent	16.68 s	16.93 s	[-2.9%; -0.0%] (maybe better)
startup:petclinic:sca:Agent	16.93 s	16.68 s	[+0.5%; +2.4%] (maybe worse)
startup:petclinic:tracing:Agent	16.06 s	16.06 s	[-0.9%; +0.9%] (no difference)

Commit: 9cf8108e · CI Pipeline · Benchmarking Platform UI

Load and DaCapo benchmarks can be triggered manually in the GitLab pipeline. Results will appear in the Benchmarking Platform UI after completion.

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 9cf8108ec5

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

chatgpt-codex-connector · 2026-07-22T19:24:08Z

@@ -33,13 +33,6 @@ public class DDTags {
  public static final String THREAD_ID = "thread.id";
  public static final String DB_STATEMENT = "sql.query";
  public static final String PATHWAY_HASH = "pathway.hash";


Preserve deprecated schema tag constants

When downstream instrumentation or user code compiles against dd-trace-api and imports any of the DDTags.SCHEMA_* constants, deleting these public constants turns a deprecated data-streams feature removal into a source-compatibility break even though the agent can simply stop emitting the tags. Please keep the constants, likely marked deprecated, until a planned breaking API release.

AGENTS.md reference: AGENTS.md:L12-L16

Useful? React with 👍 / 👎.

datadog-official

More details

Every removal is complete and consistent: no dangling callers of trySampleSchema/canSampleSchema/getSchema, no imports of the deleted Schema/SchemaIterator/SchemaBuilder API types, no references to the removed DDTags.SCHEMA_* constants, and all three AgentDataStreamsMonitoring implementations are in sync with the slimmed interface. The schema-registry utilization path (reportSchemaRegistryUsage, confluent-schema-registry-4.1) is untouched and correct.

Was this helpful? React 👍 or 👎

_{📊 Validated against 8 scenarios · Open Bits AI session}

_{🤖 Datadog Autotest · Commit 9cf8108 · What is Autotest? · Any feedback? Reach out in #autotest}

sarahchen6

LGTM pending the system tests update

piochelepiotr added type: refactoring inst: kafka Kafka instrumentation comp: data streams Data Streams Monitoring inst: protobuf Protocol Buffer instrumentation tag: ai generated Largely based on code generated by an AI or LLM labels Jul 22, 2026

piochelepiotr mentioned this pull request Jul 22, 2026

Remove protobuf schema tracking test DataDog/system-tests#7362

Open

1 task

Update agent-jar-checks golden file after removing avro/protobuf inte…

9cf8108

…grations Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

piochelepiotr marked this pull request as ready for review July 22, 2026 19:19

piochelepiotr requested review from a team as code owners July 22, 2026 19:19

piochelepiotr requested review from jordan-wong, manuel-alvarez-alvarez, mtoffl01 and sarahchen6 and removed request for a team July 22, 2026 19:19

chatgpt-codex-connector Bot reviewed Jul 22, 2026

View reviewed changes

datadog-official Bot reviewed Jul 22, 2026

View reviewed changes

sarahchen6 approved these changes Jul 22, 2026

View reviewed changes

robcarlan-datadog approved these changes Jul 22, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Remove full schema collection from Kafka, Avro, and Protobuf instrumentation#12038

Remove full schema collection from Kafka, Avro, and Protobuf instrumentation#12038
piochelepiotr wants to merge 2 commits into
masterfrom
remove-schema-collection-kafka-avro-protobuf

piochelepiotr commented Jul 22, 2026 •

edited

Loading

Uh oh!

datadog-official Bot commented Jul 22, 2026 •

edited by datadog-datadog-prod-us1-2 Bot

Loading

Uh oh!

dd-octo-sts Bot commented Jul 22, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Uh oh!

chatgpt-codex-connector Bot Jul 22, 2026

Uh oh!

datadog-official Bot left a comment

Uh oh!

sarahchen6 left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

piochelepiotr commented Jul 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Test plan

Uh oh!

datadog-official Bot commented Jul 22, 2026 • edited by datadog-datadog-prod-us1-2 Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dd-octo-sts Bot commented Jul 22, 2026

🟢 Java Benchmark SLOs — All performance SLOs passed

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector Bot Jul 22, 2026

Choose a reason for hiding this comment

Uh oh!

datadog-official Bot left a comment

Choose a reason for hiding this comment

Uh oh!

sarahchen6 left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

piochelepiotr commented Jul 22, 2026 •

edited

Loading

datadog-official Bot commented Jul 22, 2026 •

edited by datadog-datadog-prod-us1-2 Bot

Loading