Skip to content

[reference] eval: blind regeneration of datastax-cassandra-4.0 (toolkit output)#11996

Draft
jordan-wong wants to merge 1 commit into
masterfrom
eval/datastax-cassandra-4.0-db-blind-regen-20260719
Draft

[reference] eval: blind regeneration of datastax-cassandra-4.0 (toolkit output)#11996
jordan-wong wants to merge 1 commit into
masterfrom
eval/datastax-cassandra-4.0-db-blind-regen-20260719

Conversation

@jordan-wong

@jordan-wong jordan-wong commented Jul 20, 2026

Copy link
Copy Markdown
Contributor

🤖 Generated with APM Instrumentation Toolkit

What this is (⚠️ reference PR — NOT intended to be merged as-is)

This is a research reference PR from the 2026-07-19 database category eval cycle. The APM Instrumentation Toolkit was asked to generate a Cassandra instrumentation from scratch, blind, against a checkout of master where dd-java-agent/instrumentation/datastax-cassandra/datastax-cassandra-4.0/ had been deleted at the blind_setup step. This PR is the toolkit's output.

Please do NOT merge. This PR would create a new module at dd-java-agent/instrumentation/cassandra/ sitting alongside master's existing datastax-cassandra/datastax-cassandra-4.0/. Both would register super(\"cassandra\") — a public-config-API name collision. See the "Known issues" section below.

The purpose is to give reviewers a concrete artifact to react to: what does the toolkit currently produce when asked to instrument Cassandra?

Summary

  • New module dd-java-agent/instrumentation/cassandra/ targeting com.datastax.oss:java-driver-core:4.0.0
  • 6 main source files + 1 test + build.gradle; 8 Testcontainers-backed tests, all passing under the toolkit's internal test loop.
  • Chose a novel hook point: com.datastax.oss.driver.internal.core.session.DefaultSession.execute(Request, GenericType) — the single concrete dispatch method through which every CqlSession.execute* variant flows (sync execute, executeAsync). One method-advice covers all variants (see class comment on CqlSessionExecuteInstrumentation.java).
  • Async covered via SpanFinishingCallback on CompletionStage.whenComplete.
  • Peer/host info from ContactPointsUtil.getContactPoints(session).
  • db.statement handles SimpleStatement + BoundStatement.getPreparedStatement().getQuery().

Known issues to look at first

  1. Module-name collisionsuper(\"cassandra\") in the new CassandraClientModule collides with master's datastax-cassandra/datastax-cassandra-4.0/CassandraClientModule which also uses super(\"cassandra\"). The toolkit had the "read the existing module's super(...) and copy it verbatim" rule loaded at generation time (post-skill(apm-integrations): additional rules from recent HTTP-category PR reviews #11927) and violated it. See the cycle report's follow-up section on R-DB-1.

  2. Reactive coverage gapCqlSession.executeReactive returns a Publisher<ReactiveRow>. The DefaultSession.execute(Request, GenericType) hook does not intercept the reactive dispatch. No test exercises it. Real coverage gap in a real driver API.

  3. Internal-class target risk — hooking DefaultSession (a driver-internal class) is more fragile than hooking the CqlSession interface. DataStax could rename it between minor versions.

What was measured (C-CASS-1 through C-CASS-4)

# Criterion Result
C-CASS-1 Wrapper (TracingSession) vs individual execute* overloads NEITHER — novel single-dispatch-point pattern
C-CASS-2 sync + async + reactive coverage PARTIAL (async yes; reactive not covered)
C-CASS-3 peer.service from contact points PASS
C-CASS-4 db.statement on prepared statements PASS

Reviewer verdict (toolkit's internal reviewer): approved=True verdict='approved' todos_fixed=9 todos_remaining=0 after 3 review-cycle iterations.

Research provenance

Test plan (for reviewers assessing the toolkit output)

The toolkit's internal test loop passed 8 tests against a Testcontainers-backed Cassandra. Full CI has not been run yet on this branch — this PR is opened as draft to trigger CI so we can capture:

  • :check :muzzle :instrumentationLatestDepTest outcome (multi-JVM matrix in CI)
  • Whether the naming collision with master's datastax-cassandra is caught by any lint/registry check

The maintainer question this PR is asking is not "should we merge this?" but "is this the shape we would want the toolkit to produce for Cassandra?"

Try it out (toolkit)

TOOLKIT_BRANCH=eval/java bash <(gh api 'repos/DataDog/apm-instrumentation-toolkit/contents/bootstrap.sh?ref=eval/java' --jq '.content | @base64d')

🤖 Generated with APM Instrumentation Toolkit

CI Triage Status

Last updated: 2026-07-21

Confirmed research findings (do not fix):

  • dd-gitlab/check-instrumentation-naming — naming linter rejects the module: "Module name 'cassandra' must end with a version (e.g., '2.0', '3.1.0') or one of: '-common', '-stubs', '-iast'". The toolkit generated a bare dd-java-agent/instrumentation/cassandra/ directory instead of a versioned pattern (e.g. cassandra/cassandra-4.0/) — a second, distinct naming-convention gap on top of the super("cassandra") collision with master's datastax-cassandra/datastax-cassandra-4.0/ documented above (R-DB-1). Both are rule-adherence gaps on rules present in the loaded skill.
  • dd-gitlab/test_inst_latest: [21, 4/6] — confirmed via shard math (abs(javaHashCode(":dd-java-agent:instrumentation:cassandra")) % 6 + 1 == 4) that shard 4/6 is our module, not a sibling. CassandraClientTest > peerServiceInputTagsSetWithKeyspace() and peerServiceCleanup() FAILED against the latest published 4.x driver with DriverTimeoutException / InvalidKeyspaceException. Genuine latestDepTest failure in the generated test suite — not yet determined whether transient (Testcontainers timing) or a real drift against newer driver releases.

Classified as flake / unrelated:

  • main / End-to-end #10 / akka-http 10 — failure is in tests/appsec/test_blocking_addresses.py::Test_Blocking_request_body_filenames::test_blocking, an AppSec test with no code path through Cassandra instrumentation.

Downstream aggregates (no independent signal):

  • Check system tests success, dd-gitlab/default-pipeline — both fail only because the above checks fail; no separate action.

Still unclassified: none.

@jordan-wong jordan-wong added tag: apm integration toolkit Changes generated by DataDog/apm-instrumentation-toolkit as part of IDM AIT experimentation ai-generated labels Jul 20, 2026
@datadog-prod-us1-3

This comment has been minimized.

@dd-octo-sts

dd-octo-sts Bot commented Jul 20, 2026

Copy link
Copy Markdown
Contributor

🟢 Java Benchmark SLOs — All performance SLOs passed

Suite Status
Startup 🟢 pass

SLO thresholds are defined here based on automatically generated metrics. A warning is raised when results are within 5% of the threshold.

PR vs. master results
Scenario Candidate master Δ (95% CI of mean)
startup:insecure-bank:iast:Agent 13.95 s 13.93 s [-0.6%; +1.0%] (no difference)
startup:insecure-bank:tracing:Agent 12.91 s 12.98 s [-1.4%; +0.3%] (no difference)
startup:petclinic:appsec:Agent 16.92 s 16.63 s [+0.8%; +2.7%] (maybe worse)
startup:petclinic:iast:Agent 16.39 s 16.91 s [-7.6%; +1.5%] (no difference)
startup:petclinic:profiling:Agent 16.58 s 16.77 s [-2.4%; +0.2%] (no difference)
startup:petclinic:sca:Agent 16.85 s 16.41 s [-2.0%; +7.4%] (no difference)
startup:petclinic:tracing:Agent 16.09 s 16.06 s [-0.7%; +1.1%] (no difference)

Commit: f8c29806 · CI Pipeline · Benchmarking Platform UI


Load and DaCapo benchmarks can be triggered manually in the GitLab pipeline. Results will appear in the Benchmarking Platform UI after completion.

@PerfectSlayer PerfectSlayer added tag: ai generated Largely based on code generated by an AI or LLM and removed ai-generated labels Jul 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

tag: ai generated Largely based on code generated by an AI or LLM tag: apm integration toolkit Changes generated by DataDog/apm-instrumentation-toolkit as part of IDM AIT experimentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants