Skip to content

Fix TwitterBenchmark which only measured a validation error - #4499

Open
Meemaw wants to merge 1 commit into
graphql-java:masterfrom
Meemaw:fix/twitter-benchmark-depth-limit
Open

Meemaw wants to merge 1 commit into
graphql-java:masterfrom
Meemaw:fix/twitter-benchmark-depth-limit

Conversation

@Meemaw

@Meemaw Meemaw commented Oct 17, 2026 •

Copy link
Copy Markdown

Problem

TwitterBenchmark builds a query that is 150 levels deep. Since e4c287c ("Add query depth and field count limits to Validator") validation enforces QueryComplexityLimits with a default maxDepth of 100, so every benchmark execution fails with:

MaxQueryDepthExceeded: Query depth 101 exceeds maximum allowed depth 100

and returns data == null. Because the benchmark uses a persisted query cache, it has been measuring the return of a cached validation error (~8M ops/s) rather than execution, and those numbers flow into the performance results history.

Fix

  • Pass a QueryComplexityLimits with maxDepth(DEPTH + 1) through the request's GraphQLContext (QueryComplexityLimits.KEY) rather than changing the global default, so other benchmarks in the same JVM are unaffected.
  • Execute the query once during class initialisation and fail fast if it returns errors, so the benchmark cannot silently degrade into measuring an error path again.

Verification

  • Before: query returns MaxQueryDepthExceeded, data == null; benchmark reports ~8,090,000 ops/s.
  • After: query returns no errors and fully nested data (150 levels, 150 leaves at the bottom); benchmark reports 359 ± 12 ops/s (~2.8 ms per ~22.6k field query, Apple Silicon, JDK 21).
  • Temporarily lowering the limit to DEPTH - 1 makes class initialisation throw ExceptionInInitializerError, confirming the fail-fast check works.
  • graphql.archunit.* tests pass (JMH fork rule).

Note: the historical TwitterBenchmark numbers recorded since the limits were introduced are not comparable with results after this change.

The benchmark query is 150 levels deep, but since validation started
enforcing QueryComplexityLimits (default maxDepth=100) every execution
failed with MaxQueryDepthExceeded and returned no data. The benchmark
was therefore measuring a cached validation error (~8M ops/s) rather
than execution.

Raise the max depth for the benchmark requests via the GraphQLContext
and fail fast at class initialisation if the query does not execute
cleanly, so the benchmark cannot silently regress like this again.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Meemaw
Meemaw marked this pull request as ready for review October 17, 2026 19:55

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant