Skip to content

fix(bigquery-jdbc): fix time getters#13868

Draft
logachev wants to merge 1 commit into
mainfrom
kirl/timestamp_fix
Draft

fix(bigquery-jdbc): fix time getters#13868
logachev wants to merge 1 commit into
mainfrom
kirl/timestamp_fix

Conversation

@logachev

Copy link
Copy Markdown
Contributor

No description provided.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces new type coercions and optimizations in BigQueryBaseArray, BigQueryTypeCoercer, and BigQueryTypeCoercionUtility, along with corresponding integration tests. However, several critical issues were identified. The size == 1 optimization in BigQueryBaseArray can violate the JDBC contract when array elements are themselves arrays, and the array-to-array coercion in BigQueryTypeCoercer can cause runtime ClassCastExceptions. Additionally, multiple conversions in BigQueryTypeCoercionUtility (such as Timestamp to OffsetDateTime, ZonedDateTime, and Instant) incorrectly use toLocalDateTime(), introducing timezone shifts. Parsing methods like parseFieldValueToLocalDateTime and FieldValueToTimestamp are fragile and will fail on BigQuery TIMESTAMP values. Finally, localTimeToTime should be refactored to use legacy Calendar manipulation to ensure proper DST handling.

Comment on lines +150 to +160
static LocalDateTime parseFieldValueToLocalDateTime(FieldValue fv) {
String raw = fv.getStringValue();
if (raw.contains("T") || raw.contains(" ")) {
return LocalDateTime.parse(raw.replace(' ', 'T'));
}
long micros = fv.getTimestampValue();
return Instant.EPOCH
.plus(micros, ChronoUnit.MICROS)
.atOffset(ZoneOffset.UTC)
.toLocalDateTime();
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

critical

The parseFieldValueToLocalDateTime method is highly fragile and will fail when parsing BigQuery TIMESTAMP values. If the column is a TIMESTAMP, fv.getStringValue() typically returns a string with a timezone suffix and multiple spaces (e.g., 2023-07-28 12:30:00 UTC). raw.replace(' ', 'T') replaces all spaces, resulting in 2023-07-28T12:30:00TUTC, which is invalid and will cause LocalDateTime.parse to throw a DateTimeParseException. Even if there was only one space, LocalDateTime.parse does not support timezone offsets/suffixes. Instead of checking raw.contains(" ") first, you should determine if the field is a DATETIME or a TIMESTAMP. Since DATETIME strings in BigQuery do not have timezone suffixes (and typically contain T or a single space), and TIMESTAMP values should be retrieved via fv.getTimestampValue(), you should refine this logic to avoid parsing timezone-annotated strings directly with LocalDateTime.parse.

Suggested change
static LocalDateTime parseFieldValueToLocalDateTime(FieldValue fv) {
String raw = fv.getStringValue();
if (raw.contains("T") || raw.contains(" ")) {
return LocalDateTime.parse(raw.replace(' ', 'T'));
}
long micros = fv.getTimestampValue();
return Instant.EPOCH
.plus(micros, ChronoUnit.MICROS)
.atOffset(ZoneOffset.UTC)
.toLocalDateTime();
}
static LocalDateTime parseFieldValueToLocalDateTime(FieldValue fv) {
String raw = fv.getStringValue();
if (raw.contains("T") || (raw.contains(" ") && !raw.endsWith("UTC") && !raw.endsWith("Z"))) {
return LocalDateTime.parse(raw.replace(' ', 'T'));
}
long micros = fv.getTimestampValue();
return Instant.EPOCH
.plus(micros, ChronoUnit.MICROS)
.atOffset(ZoneOffset.UTC)
.toLocalDateTime();
}

Comment on lines 432 to 438
public Timestamp coerce(Long value) {
// Long value is in microseconds. All further calculations should account for the unit.
Instant instant = Instant.EPOCH.plus(value, ChronoUnit.MICROS);
// Timezone-agnostic conversion preserving exact point in time as mandated by JDBC spec
return Timestamp.from(instant);
LocalDateTime utcDateTime = instant.atOffset(ZoneOffset.UTC).toLocalDateTime();
Timestamp ts = Timestamp.valueOf(utcDateTime);
ts.setNanos((int) ((value % 1_000_000) * 1000));
return ts;
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

critical

Using Timestamp.valueOf(utcDateTime) is highly problematic because Timestamp.valueOf(LocalDateTime) interprets the given local date-time in the system default timezone, not in UTC. Since utcDateTime is obtained using ZoneOffset.UTC, converting it via Timestamp.valueOf will shift the actual point in time by the system's timezone offset. For example, if the system timezone is UTC-5, the resulting Timestamp will be shifted by 5 hours. To preserve the exact point in time (as mandated by the JDBC spec), you should use Timestamp.from(instant) or construct the Timestamp directly from the epoch milliseconds while preserving the nanosecond precision.

    @Override
    public Timestamp coerce(Long value) {
      Instant instant = Instant.EPOCH.plus(value, ChronoUnit.MICROS);
      return Timestamp.from(instant);
    }

Comment on lines +487 to +490
if (rawValue.contains("T") || rawValue.contains(" ")) {
return Timestamp.valueOf(rawValue.replace('T', ' '));
} else {
// It's a TIMESTAMP numeric string.
long microseconds = fieldValue.getTimestampValue();
Instant instant = Instant.EPOCH.plus(microseconds, ChronoUnit.MICROS);
// Timezone-agnostic conversion preserving exact point in time as mandated by JDBC spec
return Timestamp.from(instant);
}
return new LongToTimestamp().coerce(fieldValue.getTimestampValue());

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

critical

This change is highly error-prone and will break parsing for BigQuery TIMESTAMP values. BigQuery TIMESTAMP string representations typically contain a space (e.g., 2023-07-28 12:30:00 UTC). Under this new logic, because the string contains a space, it will enter the if block and call Timestamp.valueOf(rawValue). However: 1. Timestamp.valueOf does not support timezone suffixes like UTC and will throw an IllegalArgumentException. 2. Even if it did parse, Timestamp.valueOf interprets the time in the system default timezone, whereas BigQuery timestamps are in UTC, leading to incorrect timezone shifts. Please revert this to the previous logic where DATETIME (containing T) is handled separately from TIMESTAMP (which should be retrieved via fieldValue.getTimestampValue() and converted timezone-agnostically).

Suggested change
if (rawValue.contains("T") || rawValue.contains(" ")) {
return Timestamp.valueOf(rawValue.replace('T', ' '));
} else {
// It's a TIMESTAMP numeric string.
long microseconds = fieldValue.getTimestampValue();
Instant instant = Instant.EPOCH.plus(microseconds, ChronoUnit.MICROS);
// Timezone-agnostic conversion preserving exact point in time as mandated by JDBC spec
return Timestamp.from(instant);
}
return new LongToTimestamp().coerce(fieldValue.getTimestampValue());
if (rawValue.contains("T")) {
return Timestamp.valueOf(rawValue.replace('T', ' '));
}
return new LongToTimestamp().coerce(fieldValue.getTimestampValue());

Comment on lines +237 to 240
.registerTypeCoercion(
(Timestamp ts) -> ts.toLocalDateTime().atOffset(ZoneOffset.UTC),
Timestamp.class,
OffsetDateTime.class)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

critical

Converting Timestamp to OffsetDateTime using ts.toLocalDateTime().atOffset(ZoneOffset.UTC) is incorrect and introduces timezone shifts. ts.toLocalDateTime() converts the timestamp to the system default timezone. Attaching ZoneOffset.UTC to this local date-time without adjusting the offset will result in an incorrect instant. Please revert this to use ts.toInstant().atOffset(ZoneOffset.UTC) (or Instant.ofEpochMilli(ts.getTime()).atOffset(ZoneOffset.UTC)) to preserve the correct instant.

Suggested change
.registerTypeCoercion(
(Timestamp ts) -> ts.toLocalDateTime().atOffset(ZoneOffset.UTC),
Timestamp.class,
OffsetDateTime.class)
.registerTypeCoercion(
(Timestamp ts) -> ts.toInstant().atOffset(ZoneOffset.UTC),
Timestamp.class,
OffsetDateTime.class)

Comment on lines +241 to +244
.registerTypeCoercion(
(Timestamp ts) -> ts.toLocalDateTime().atZone(ZoneOffset.UTC),
Timestamp.class,
ZonedDateTime.class)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

critical

Converting Timestamp to ZonedDateTime using ts.toLocalDateTime().atZone(ZoneOffset.UTC) is incorrect and introduces timezone shifts. Please use ts.toInstant().atZone(ZoneOffset.UTC) to preserve the correct instant.

Suggested change
.registerTypeCoercion(
(Timestamp ts) -> ts.toLocalDateTime().atZone(ZoneOffset.UTC),
Timestamp.class,
ZonedDateTime.class)
.registerTypeCoercion(
(Timestamp ts) -> ts.toInstant().atZone(ZoneOffset.UTC),
Timestamp.class,
ZonedDateTime.class)

Comment on lines +245 to +248
.registerTypeCoercion(
(Timestamp ts) -> ts.toLocalDateTime().atOffset(ZoneOffset.UTC).toInstant(),
Timestamp.class,
Instant.class)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

critical

Converting Timestamp to Instant using ts.toLocalDateTime().atOffset(ZoneOffset.UTC).toInstant() is incorrect and introduces timezone shifts. Please revert this to simply ts.toInstant().

Suggested change
.registerTypeCoercion(
(Timestamp ts) -> ts.toLocalDateTime().atOffset(ZoneOffset.UTC).toInstant(),
Timestamp.class,
Instant.class)
.registerTypeCoercion(
(Timestamp ts) -> ts.toInstant(),
Timestamp.class,
Instant.class)

Comment on lines +98 to 106
if (size == 1) {
Object firstVal = getCoercedValue(fromIndex);
if (firstVal != null
&& firstVal.getClass().isArray()
&& firstVal.getClass().getComponentType().equals(targetClass)) {
return firstVal;
}
}
Object javaArray = Array.newInstance(targetClass, size);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

This optimization for size == 1 is highly problematic and introduces a bug when the array elements themselves are arrays (for example, when targetClass is Object.class and the single element is an Object[] representing a struct, or when dealing with multi-dimensional arrays). If targetClass is Object.class and firstVal is Object[], firstVal.getClass().getComponentType().equals(targetClass) evaluates to true (Object.class == Object.class). The method will then return firstVal directly (an array of length N representing the struct's fields) instead of returning a wrapped array of length 1 containing firstVal as its single element. This violates the JDBC contract for Array.getArray(). Please remove this size == 1 shortcut.

    Object javaArray = Array.newInstance(targetClass, size);

Comment on lines +119 to +122
if (targetClass.isAssignableFrom(sourceClass)
|| (sourceClass.isArray() && targetClass.isArray())) {
return (T) value;
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

The condition (sourceClass.isArray() && targetClass.isArray()) is dangerous because it allows returning value directly even when the source array type is not assignable to the target array type (e.g., trying to coerce LocalTime[] to Time[]). Since targetClass.isAssignableFrom(sourceClass) is false in such cases, casting the returned value to T will inevitably throw a ClassCastException at runtime. If the intention is to support element-wise coercion for arrays of different component types, you should implement a proper array coercion mechanism that creates a new array of the target component type and coerces each element individually. Otherwise, this condition should be removed as targetClass.isAssignableFrom(sourceClass) already safely handles assignable array types.

      if (targetClass.isAssignableFrom(sourceClass)) {
        return (T) value;
      }

Comment on lines +141 to +148
static Time localTimeToTime(LocalTime lt) {
long epochMillis =
lt.atDate(LocalDate.of(1970, 1, 1))
.atZone(ZoneId.systemDefault())
.toInstant()
.toEpochMilli();
return new Time(epochMillis);
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

According to the project's general rules, when converting or shifting timezone fields for JDBC types like java.sql.Time, legacy Calendar manipulation should be used instead of java.time APIs to ensure Daylight Saving Time (DST) is handled properly. Using lt.atDate(LocalDate.of(1970, 1, 1)).atZone(ZoneId.systemDefault()) can lead to DST or historical offset discrepancies. Please refactor this to use legacy Calendar manipulation.

  static Time localTimeToTime(LocalTime lt) {
    Calendar cal = Calendar.getInstance();
    cal.clear();
    cal.set(1970, Calendar.JANUARY, 1, lt.getHour(), lt.getMinute(), lt.getSecond());
    cal.set(Calendar.MILLISECOND, lt.getNano() / 1_000_000);
    return new Time(cal.getTimeInMillis());
  }
References
  1. When converting or shifting timezone fields for JDBC types like java.sql.Time, use legacy Calendar manipulation instead of java.time APIs to ensure Daylight Saving Time (DST) is handled properly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant