[reference] eval: blind regeneration of datastax-cassandra-4.0 (toolkit output)#11996
Draft
jordan-wong wants to merge 1 commit into
Draft
[reference] eval: blind regeneration of datastax-cassandra-4.0 (toolkit output)#11996jordan-wong wants to merge 1 commit into
jordan-wong wants to merge 1 commit into
Conversation
This comment has been minimized.
This comment has been minimized.
Contributor
🟢 Java Benchmark SLOs — All performance SLOs passed
PR vs. master results
Commit: Load and DaCapo benchmarks can be triggered manually in the GitLab pipeline. Results will appear in the Benchmarking Platform UI after completion. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🤖 Generated with APM Instrumentation Toolkit
What this is (⚠️ reference PR — NOT intended to be merged as-is)
This is a research reference PR from the 2026-07-19 database category eval cycle. The APM Instrumentation Toolkit was asked to generate a Cassandra instrumentation from scratch, blind, against a checkout of
masterwheredd-java-agent/instrumentation/datastax-cassandra/datastax-cassandra-4.0/had been deleted at theblind_setupstep. This PR is the toolkit's output.Please do NOT merge. This PR would create a new module at
dd-java-agent/instrumentation/cassandra/sitting alongside master's existingdatastax-cassandra/datastax-cassandra-4.0/. Both would registersuper(\"cassandra\")— a public-config-API name collision. See the "Known issues" section below.The purpose is to give reviewers a concrete artifact to react to: what does the toolkit currently produce when asked to instrument Cassandra?
Summary
dd-java-agent/instrumentation/cassandra/targetingcom.datastax.oss:java-driver-core:4.0.0build.gradle; 8 Testcontainers-backed tests, all passing under the toolkit's internal test loop.com.datastax.oss.driver.internal.core.session.DefaultSession.execute(Request, GenericType)— the single concrete dispatch method through which everyCqlSession.execute*variant flows (syncexecute,executeAsync). One method-advice covers all variants (see class comment onCqlSessionExecuteInstrumentation.java).SpanFinishingCallbackonCompletionStage.whenComplete.ContactPointsUtil.getContactPoints(session).db.statementhandlesSimpleStatement+BoundStatement.getPreparedStatement().getQuery().Known issues to look at first
Module-name collision —
super(\"cassandra\")in the newCassandraClientModulecollides with master'sdatastax-cassandra/datastax-cassandra-4.0/CassandraClientModulewhich also usessuper(\"cassandra\"). The toolkit had the "read the existing module'ssuper(...)and copy it verbatim" rule loaded at generation time (post-skill(apm-integrations): additional rules from recent HTTP-category PR reviews #11927) and violated it. See the cycle report's follow-up section on R-DB-1.Reactive coverage gap —
CqlSession.executeReactivereturns aPublisher<ReactiveRow>. TheDefaultSession.execute(Request, GenericType)hook does not intercept the reactive dispatch. No test exercises it. Real coverage gap in a real driver API.Internal-class target risk — hooking
DefaultSession(a driver-internal class) is more fragile than hooking theCqlSessioninterface. DataStax could rename it between minor versions.What was measured (C-CASS-1 through C-CASS-4)
TracingSession) vs individualexecute*overloadsdb.statementon prepared statementsReviewer verdict (toolkit's internal reviewer):
approved=True verdict='approved' todos_fixed=9 todos_remaining=0after 3 review-cycle iterations.Research provenance
docs/eval-research/cycles/2026-07-19-database-cycle-report.md(toolkit repo)docs/eval-research/hypotheses/cassandra.mddocs/superpowers/plans/2026-06-28-database-category-eval.mdorigin/master@846103dfeb, includes PR skill(apm-integrations): additional rules from recent HTTP-category PR reviews #11927 (084b01b643 skill(apm-integrations): additional rules from recent HTTP-category PR reviews).blind_setupcommit deleteddatastax-cassandra-4.0/(0 files at that tree; parent had 8 subdirs). See cycle report §Blind protocol.Test plan (for reviewers assessing the toolkit output)
The toolkit's internal test loop passed 8 tests against a Testcontainers-backed Cassandra. Full CI has not been run yet on this branch — this PR is opened as draft to trigger CI so we can capture:
:check :muzzle :instrumentationLatestDepTestoutcome (multi-JVM matrix in CI)datastax-cassandrais caught by any lint/registry checkThe maintainer question this PR is asking is not "should we merge this?" but "is this the shape we would want the toolkit to produce for Cassandra?"
Try it out (toolkit)
TOOLKIT_BRANCH=eval/java bash <(gh api 'repos/DataDog/apm-instrumentation-toolkit/contents/bootstrap.sh?ref=eval/java' --jq '.content | @base64d')🤖 Generated with APM Instrumentation Toolkit
CI Triage Status
Last updated: 2026-07-21
Confirmed research findings (do not fix):
dd-gitlab/check-instrumentation-naming— naming linter rejects the module: "Module name 'cassandra' must end with a version (e.g., '2.0', '3.1.0') or one of: '-common', '-stubs', '-iast'". The toolkit generated a baredd-java-agent/instrumentation/cassandra/directory instead of a versioned pattern (e.g.cassandra/cassandra-4.0/) — a second, distinct naming-convention gap on top of thesuper("cassandra")collision with master'sdatastax-cassandra/datastax-cassandra-4.0/documented above (R-DB-1). Both are rule-adherence gaps on rules present in the loaded skill.dd-gitlab/test_inst_latest: [21, 4/6]— confirmed via shard math (abs(javaHashCode(":dd-java-agent:instrumentation:cassandra")) % 6 + 1 == 4) that shard 4/6 is our module, not a sibling.CassandraClientTest > peerServiceInputTagsSetWithKeyspace()andpeerServiceCleanup()FAILED against the latest published4.xdriver withDriverTimeoutException/InvalidKeyspaceException. GenuinelatestDepTestfailure in the generated test suite — not yet determined whether transient (Testcontainers timing) or a real drift against newer driver releases.Classified as flake / unrelated:
main / End-to-end #10 / akka-http 10— failure is intests/appsec/test_blocking_addresses.py::Test_Blocking_request_body_filenames::test_blocking, an AppSec test with no code path through Cassandra instrumentation.Downstream aggregates (no independent signal):
Check system tests success,dd-gitlab/default-pipeline— both fail only because the above checks fail; no separate action.Still unclassified: none.