Skip to content

fix(bigquery-jdbc): lazy caller inference and zero-overhead coercion logging#13365

Open
Neenu1995 wants to merge 2 commits into
mainfrom
fix-stackwalk
Open

fix(bigquery-jdbc): lazy caller inference and zero-overhead coercion logging#13365
Neenu1995 wants to merge 2 commits into
mainfrom
fix-stackwalk

Conversation

@Neenu1995
Copy link
Copy Markdown
Contributor

@Neenu1995 Neenu1995 commented Jun 4, 2026

Problem

When the BigQuery JDBC driver is run with JUL-to-SLF4J bridges (like SLF4JBridgeHandler in Spring Boot environments), JUL's logging level is configured to ALL to allow filtering to occur at the SLF4J/Logback layer. As a result, the driver's custom logger always proceeded to log, eagerly capturing and walking stack traces (new Throwable().getStackTrace()) to find the caller class and method. On hot paths (such as BigQueryTypeCoercer.coerceTo converting values for millions of rows), this caused severe CPU and memory overhead (representing up to ~99% CPU time in profile runs), even if SLF4J immediately discarded the logs.

Summary of Changes

  • Lazy Caller Inference (BigQueryJdbcLogRecord):
    • Introduced BigQueryJdbcLogRecord (a custom subclass of LogRecord) inside BigQueryJdbcCustomLogger to defer caller stack-trace walking until requested by formatters/bridges.
    • Ensures zero stack-walk overhead when logging is disabled under SLF4J bridges.
    • Added framework packages (like org.slf4j.bridge.*) to the caller skipper list to maintain inference correctness.
  • Coercion Hot Path Optimization (BigQueryTypeCoercer):
    • Switched type coercion logger to the specialized BigQueryJdbcResultSetLogger.
    • Replaced generic effectiveLog.finest with explicit context passing using effectiveLog.finestTrace("coerceTo", ...).
    • Avoids stack walks entirely on the result set coercion hot path, regardless of whether logging is enabled.
  • Unit Testing:
    • Added testLazyCallerInference in BigQueryJdbcCustomLoggerTest to verify that stack-trace walking is deferred until queried and accurately resolves the correct caller class/method.

Performance Impact

  • Benchmark results: Query execution on 100k rows × 5 cols with high-throughput enabled yielded a ~5.9% end-to-end execution time reduction (and up to 3.3× speedup on larger 2M × 34 datasets where JDBC coercion overhead is the main bottleneck).
  • Eliminated millions of garbage collection allocations of StackTraceElement[] on the JDBC data retrieval hot paths.

@Neenu1995 Neenu1995 requested review from a team as code owners June 4, 2026 18:16
Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces lazy caller inference in BigQueryJdbcCustomLogger by implementing a custom BigQueryJdbcLogRecord class, deferring expensive stack trace walks until they are actually needed. Additionally, BigQueryTypeCoercer was updated to use BigQueryJdbcResultSetLogger with lazy trace logging, and a unit test was added to verify the lazy evaluation. A review comment identified a potential thread-safety issue in BigQueryJdbcLogRecord where the lack of synchronization on getters, setters, and state-checking methods could lead to data races during asynchronous log processing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant