Skip to content

Commit 7fad2f6

Browse files
danieljbrucegoogle-labs-jules[bot]
danieljbruce
and
google-labs-jules[bot]
authored
fix: Unblock the releases on Node Bigquery (#7946)
* Revert "feat: Add high precision TIMESTAMP values for queries (#7147)" This reverts commit bea42b2. # Conflicts: # handwritten/bigquery/src/bigquery.ts # handwritten/bigquery/src/job.ts # handwritten/bigquery/test/bigquery.ts * Revert "feat: support high precision timestamp strings on getRows calls (#1596)" This reverts commit b3217a3. # Conflicts: # handwritten/bigquery/src/table.ts * skip a test * Remove the High Precision Query System tests * Source code changes for timestamp precision flag * Add tests back * Add test file back in * Modify tests to only run for high precision or not * listParams add * remove listParams * skip tests if picosecond support is not turned on * remove only * Don’t create separate code paths when not needed * extend the default options * Skip the before hook if picosecond support not on * Add documentation back in for methods * Remove the if block since release is done * Eliminate redundant feature flag check. * Add qs back in * Add commands to test for picoseconds in system t * Revert "Remove the if block since release is done" This reverts commit 757dfdc. * Eliminate environment variable for parsing check * Always use try block * Change type to GetRowsOptions * Remove exception * Some typescript simplifications * Eliminate default options branching * Change defaults for current requests * Don’t need extra import * Don’t gate with picosecond flag * Just add an extra test file * test(bigquery): parameterize system tests for picosecond support Wraps the BigQuery system tests in a parameterized function to run them both with and without the BIGQUERY_PICOSECOND_SUPPORT environment variable. Clients and resources are re-instantiated within a before hook to ensure the library picks up the environment changes for each run. The existing indentation is preserved to maintain a clean PR diff. Co-authored-by: danieljbruce <8935272+danieljbruce@users.noreply.github.com> * test(bigquery): add high precision timestamp wrapper system test Creates a new system test file that programmatically re-runs the entire BigQuery system test suite with BIGQUERY_PICOSECOND_SUPPORT=true. This is achieved by clearing the Node.js require cache for the package and requiring the existing bigquery.ts test file within a new describe block. This approach leaves the original test file untouched, ensuring a clean PR diff and avoiding linting issues. Co-authored-by: danieljbruce <8935272+danieljbruce@users.noreply.github.com> * linting change and copyright year * Correct package.json errors * This file is not working well with test runner * Run the timestamp output format tests as well * Add a wrapper --------- Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> Co-authored-by: danieljbruce <8935272+danieljbruce@users.noreply.github.com>
1 parent fb317f8 commit 7fad2f6

7 files changed

Lines changed: 237 additions & 56 deletions

File tree

‎handwritten/bigquery/src/bigquery.ts‎

Lines changed: 18 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -1101,10 +1101,12 @@ export class BigQuery extends Service {
11011101
}),
11021102
};
11031103
} else if ((providedType as string).toUpperCase() === 'TIMESTAMP(12)') {
1104-
return {
1105-
type: 'TIMESTAMP',
1106-
timestampPrecision: '12',
1107-
};
1104+
if (process.env.BIGQUERY_PICOSECOND_SUPPORT === 'true') {
1105+
return {
1106+
type: 'TIMESTAMP',
1107+
timestampPrecision: '12',
1108+
};
1109+
}
11081110
}
11091111

11101112
providedType = (providedType as string).toUpperCase();
@@ -2263,7 +2265,6 @@ export class BigQuery extends Service {
22632265
the callback. Instead, pass the error to the callback the user provides
22642266
so that the user can see the error.
22652267
*/
2266-
// eslint-disable-next-line @typescript-eslint/no-explicit-any
22672268
const listParams = {
22682269
'formatOptions.timestampOutputFormat':
22692270
queryReq.formatOptions?.timestampOutputFormat,
@@ -2367,11 +2368,18 @@ export class BigQuery extends Service {
23672368
const hasAnyFormatOpts =
23682369
options['formatOptions.timestampOutputFormat'] !== undefined ||
23692370
options['formatOptions.useInt64Timestamp'] !== undefined;
2370-
const defaultOpts = hasAnyFormatOpts
2371+
let defaultOpts: bigquery.IDataFormatOptions = hasAnyFormatOpts
23712372
? {}
23722373
: {
2373-
timestampOutputFormat: 'ISO8601_STRING',
2374+
useInt64Timestamp: true,
23742375
};
2376+
if (process.env.BIGQUERY_PICOSECOND_SUPPORT === 'true') {
2377+
defaultOpts = hasAnyFormatOpts
2378+
? {}
2379+
: {
2380+
timestampOutputFormat: 'ISO8601_STRING',
2381+
};
2382+
}
23752383
const formatOptions = extend(defaultOpts, {
23762384
timestampOutputFormat: options['formatOptions.timestampOutputFormat'],
23772385
useInt64Timestamp: options['formatOptions.useInt64Timestamp'],
@@ -2585,12 +2593,9 @@ function convertSchemaFieldValue(
25852593
2023-01-01T12:00:00.123456789123Z
25862594
*/
25872595
const listParams = options.listParams;
2588-
const timestampOutputFormat = listParams
2589-
? listParams['formatOptions.timestampOutputFormat']
2590-
: undefined;
2591-
const useInt64Timestamp = listParams
2592-
? listParams['formatOptions.useInt64Timestamp']
2593-
: undefined;
2596+
const timestampOutputFormat =
2597+
listParams?.['formatOptions.timestampOutputFormat'];
2598+
const useInt64Timestamp = listParams?.['formatOptions.useInt64Timestamp'];
25942599
if (timestampOutputFormat === 'ISO8601_STRING') {
25952600
// value is ISO string, create BigQueryTimestamp wrapping the string
25962601
value = BigQuery.timestamp(value);

‎handwritten/bigquery/src/table.ts‎

Lines changed: 12 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -55,7 +55,6 @@ import {JobMetadata, JobOptions} from './job';
5555
import bigquery from './types';
5656
import {IntegerTypeCastOptions} from './bigquery';
5757
import {RowQueue} from './rowQueue';
58-
import IDataFormatOptions = bigquery.IDataFormatOptions;
5958

6059
// This is supposed to be a @google-cloud/storage `File` type. The storage npm
6160
// module includes these types, but is current installed as a devDependency.
@@ -1894,15 +1893,24 @@ class Table extends ServiceObject {
18941893
}
18951894
callback!(null, rows, nextQuery, resp);
18961895
};
1896+
18971897
const hasAnyFormatOpts =
18981898
options['formatOptions.timestampOutputFormat'] !== undefined ||
18991899
options['formatOptions.useInt64Timestamp'] !== undefined;
1900-
const defaultOpts = hasAnyFormatOpts
1900+
let defaultOpts: GetRowsOptions = hasAnyFormatOpts
19011901
? {}
19021902
: {
1903-
'formatOptions.timestampOutputFormat': 'ISO8601_STRING',
1903+
'formatOptions.useInt64Timestamp': true,
19041904
};
1905-
const qs = extend(defaultOpts, options);
1905+
if (process.env.BIGQUERY_PICOSECOND_SUPPORT === 'true') {
1906+
defaultOpts = hasAnyFormatOpts
1907+
? {}
1908+
: {
1909+
'formatOptions.timestampOutputFormat': 'ISO8601_STRING',
1910+
};
1911+
}
1912+
const qs: GetRowsOptions = extend(defaultOpts, options);
1913+
19061914
this.request(
19071915
{
19081916
uri: '/data',

‎handwritten/bigquery/system-test/bigquery.ts‎

Lines changed: 41 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -1042,25 +1042,27 @@ describe('BigQuery', () => {
10421042
});
10431043

10441044
it('should create a table with timestampPrecision', async () => {
1045-
const table = dataset.table(generateName('timestamp-precision-table'));
1046-
const schema = {
1047-
fields: [
1048-
{
1049-
name: 'ts_field',
1050-
type: 'TIMESTAMP',
1051-
timestampPrecision: 12,
1052-
},
1053-
],
1054-
};
1055-
try {
1056-
await table.create({schema});
1057-
const [metadata] = await table.getMetadata();
1058-
assert.deepStrictEqual(
1059-
metadata.schema.fields[0].timestampPrecision,
1060-
'12',
1061-
);
1062-
} catch (e) {
1063-
assert.ifError(e);
1045+
if (process.env.BIGQUERY_PICOSECOND_SUPPORT === 'true') {
1046+
const table = dataset.table(generateName('timestamp-precision-table'));
1047+
const schema = {
1048+
fields: [
1049+
{
1050+
name: 'ts_field',
1051+
type: 'TIMESTAMP',
1052+
timestampPrecision: 12,
1053+
},
1054+
],
1055+
};
1056+
try {
1057+
await table.create({schema});
1058+
const [metadata] = await table.getMetadata();
1059+
assert.deepStrictEqual(
1060+
metadata.schema.fields[0].timestampPrecision,
1061+
'12',
1062+
);
1063+
} catch (e) {
1064+
assert.ifError(e);
1065+
}
10641066
}
10651067
});
10661068

@@ -1562,6 +1564,11 @@ describe('BigQuery', () => {
15621564

15631565
testCases.forEach(testCase => {
15641566
it(`should handle ${testCase.name}`, async () => {
1567+
if (process.env.BIGQUERY_PICOSECOND_SUPPORT !== 'true') {
1568+
// These tests are only important when the high precision
1569+
// timestamp support is turned on.
1570+
return;
1571+
}
15651572
/*
15661573
The users use the new TIMESTAMP(12) type to indicate they want to
15671574
opt in to using timestampPrecision=12. The reason is that some queries
@@ -1614,6 +1621,11 @@ describe('BigQuery', () => {
16141621
}
16151622
});
16161623
it(`should handle nested ${testCase.name}`, async () => {
1624+
if (process.env.BIGQUERY_PICOSECOND_SUPPORT !== 'true') {
1625+
// These tests are only important when the high precision
1626+
// timestamp support is turned on.
1627+
return;
1628+
}
16171629
/*
16181630
The users use the new TIMESTAMP(12) type to indicate they want to
16191631
opt in to using timestampPrecision=12. The reason is that some queries
@@ -2009,6 +2021,11 @@ describe('BigQuery', () => {
20092021

20102022
testCases.forEach(testCase => {
20112023
it(`should handle ${testCase.name}`, async () => {
2024+
if (process.env.BIGQUERY_PICOSECOND_SUPPORT !== 'true') {
2025+
// These tests are only important when the high precision
2026+
// timestamp support is turned on.
2027+
return;
2028+
}
20122029
/*
20132030
The users use the new TIMESTAMP(12) type to indicate they want to
20142031
opt in to using timestampPrecision=12. The reason is that some queries
@@ -2063,6 +2080,11 @@ describe('BigQuery', () => {
20632080
}
20642081
});
20652082
it(`should handle nested ${testCase.name}`, async () => {
2083+
if (process.env.BIGQUERY_PICOSECOND_SUPPORT !== 'true') {
2084+
// These tests are only important when the high precision
2085+
// timestamp support is turned on.
2086+
return;
2087+
}
20662088
/*
20672089
The users use the new TIMESTAMP(12) type to indicate they want to
20682090
opt in to using timestampPrecision=12. The reason is that some queries

‎handwritten/bigquery/system-test/high_precision_timestamp.ts‎

Lines changed: 78 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,78 @@
1+
// Copyright 2026 Google LLC
2+
//
3+
// Licensed under the Apache License, Version 2.0 (the "License");
4+
// you may not use this file except in compliance with the License.
5+
// You may obtain a copy of the License at
6+
//
7+
// http://www.apache.org/licenses/LICENSE-2.0
8+
//
9+
// Unless required by applicable law or agreed to in writing, software
10+
// distributed under the License is distributed on an "AS IS" BASIS,
11+
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
// See the License for the specific language governing permissions and
13+
// limitations under the License.
14+
15+
import {describe, before, after} from 'mocha';
16+
import * as path from 'path';
17+
18+
// This file registers a second run of the BigQuery system tests with picosecond support enabled.
19+
// It uses targeted cache clearing to ensure the library and tests are re-initialized with the
20+
// 'BIGQUERY_PICOSECOND_SUPPORT' environment variable set.
21+
22+
const originalValue = process.env.BIGQUERY_PICOSECOND_SUPPORT;
23+
24+
/**
25+
* Helper to clear the require cache for all files within the current package.
26+
*/
27+
function clearPackageCache() {
28+
// Find the root of the current package by looking for package.json
29+
const packageJsonPath = require.resolve('../../package.json');
30+
const packageDir = path.dirname(packageJsonPath);
31+
32+
Object.keys(require.cache).forEach(key => {
33+
if (key.startsWith(packageDir) && !key.includes('node_modules')) {
34+
delete require.cache[key];
35+
}
36+
});
37+
}
38+
39+
// 1. REGISTRATION PHASE:
40+
// Set the environment variable and clear the cache so that the subsequent 'require'
41+
// of the system test file (and any library files it imports) sees the updated state.
42+
process.env.BIGQUERY_PICOSECOND_SUPPORT = 'true';
43+
clearPackageCache();
44+
45+
describe('BigQuery System Tests (High Precision)', () => {
46+
// 2. EXECUTION PHASE:
47+
// Mocha runs 'before' hooks before the tests in the required file are executed.
48+
before(() => {
49+
process.env.BIGQUERY_PICOSECOND_SUPPORT = 'true';
50+
// We don't clear cache here as it's too late for Mocha's registration phase,
51+
// but we ensure the env var is set for any execution-time checks.
52+
});
53+
54+
after(() => {
55+
if (originalValue === undefined) {
56+
delete process.env.BIGQUERY_PICOSECOND_SUPPORT;
57+
} else {
58+
process.env.BIGQUERY_PICOSECOND_SUPPORT = originalValue;
59+
}
60+
});
61+
62+
describe('Run all tests', () => {
63+
// Wrap all the other tests in a describe block to ensure the entire file is
64+
// executed when the environment variable is set to true.
65+
66+
// Programmatically require the tests. Mocha will discover and register them inside this 'describe' block.
67+
require('./bigquery');
68+
require('./timestamp_output_format');
69+
});
70+
});
71+
72+
// 3. CLEANUP:
73+
// Restore the environment variable after the registration phase to minimize side effects on other test files.
74+
if (originalValue === undefined) {
75+
delete process.env.BIGQUERY_PICOSECOND_SUPPORT;
76+
} else {
77+
process.env.BIGQUERY_PICOSECOND_SUPPORT = originalValue;
78+
}

‎handwritten/bigquery/system-test/timestamp_output_format.ts‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -38,6 +38,9 @@ describe('Timestamp Output Format System Tests', () => {
3838
const expectedTsValuePicoseconds = '2023-01-01T12:00:00.123456789123Z';
3939

4040
before(async () => {
41+
if (process.env.BIGQUERY_PICOSECOND_SUPPORT !== 'true') {
42+
return;
43+
}
4144
await dataset.create();
4245
await table.create({
4346
schema: [{name: 'ts', type: 'TIMESTAMP', timestampPrecision: '12'}],
@@ -159,6 +162,9 @@ describe('Timestamp Output Format System Tests', () => {
159162
expectedTsValue,
160163
}) => {
161164
it(name, async () => {
165+
if (process.env.BIGQUERY_PICOSECOND_SUPPORT !== 'true') {
166+
return;
167+
}
162168
const options: {[key: string]: any} = {};
163169
if (timestampOutputFormat !== undefined) {
164170
options['formatOptions.timestampOutputFormat'] =
@@ -185,6 +191,10 @@ describe('Timestamp Output Format System Tests', () => {
185191
);
186192

187193
it('should make a request with ISO8601_STRING when no format options are being used', done => {
194+
if (process.env.BIGQUERY_PICOSECOND_SUPPORT !== 'true') {
195+
done();
196+
return;
197+
}
188198
void (async () => {
189199
const originalRequest = table.request;
190200
const requestPromise: Promise<RequestResponse> = new Promise(

‎handwritten/bigquery/test/bigquery.ts‎

Lines changed: 22 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -3439,6 +3439,14 @@ describe('BigQuery', () => {
34393439
delete req[key];
34403440
}
34413441
}
3442+
const formatOptions =
3443+
process.env.BIGQUERY_PICOSECOND_SUPPORT === 'true'
3444+
? {
3445+
timestampOutputFormat: 'ISO8601_STRING',
3446+
}
3447+
: {
3448+
useInt64Timestamp: true,
3449+
};
34423450
const expectedReq = {
34433451
query: QUERY_STRING,
34443452
useLegacySql: false,
@@ -3465,9 +3473,7 @@ describe('BigQuery', () => {
34653473
key: 'value',
34663474
},
34673475
jobCreationMode: 'JOB_CREATION_REQUIRED',
3468-
formatOptions: {
3469-
timestampOutputFormat: 'ISO8601_STRING',
3470-
},
3476+
formatOptions,
34713477
};
34723478
assert.deepStrictEqual(req, expectedReq);
34733479
});
@@ -3508,6 +3514,9 @@ describe('BigQuery', () => {
35083514

35093515
testCases.forEach(testCase => {
35103516
it(`should handle ${testCase.name}`, () => {
3517+
if (process.env.BIGQUERY_PICOSECOND_SUPPORT !== 'true') {
3518+
return;
3519+
}
35113520
const req = bq.buildQueryRequest_(QUERY_STRING, testCase.opts);
35123521

35133522
const expectedReq = {
@@ -3535,21 +3544,28 @@ describe('BigQuery', () => {
35353544
});
35363545
});
35373546
});
3547+
35383548
it('should create a QueryRequest from a SQL string', () => {
35393549
const req = bq.buildQueryRequest_(QUERY_STRING, {});
35403550
for (const key in req) {
35413551
if (req[key] === undefined) {
35423552
delete req[key];
35433553
}
35443554
}
3555+
const formatOptions =
3556+
process.env.BIGQUERY_PICOSECOND_SUPPORT === 'true'
3557+
? {
3558+
timestampOutputFormat: 'ISO8601_STRING',
3559+
}
3560+
: {
3561+
useInt64Timestamp: true,
3562+
};
35453563
const expectedReq = {
35463564
query: QUERY_STRING,
35473565
useLegacySql: false,
35483566
requestId: req.requestId,
35493567
jobCreationMode: 'JOB_CREATION_OPTIONAL',
3550-
formatOptions: {
3551-
timestampOutputFormat: 'ISO8601_STRING',
3552-
},
3568+
formatOptions,
35533569
};
35543570
assert.deepStrictEqual(req, expectedReq);
35553571
});

0 commit comments

Comments
 (0)