This repository was archived by the owner on Mar 6, 2026. It is now read-only.
Description recent runs of presubmit kokoro show we're taking something on the order of 37-48 minutes for typical runs, with a couple invocations significantly higher than that. It's a significant impediment to actually getting things done.
We can definitely test much faster than this.
Some things to look at:
Decompose the monolilthic kokoro job that runs all the nox sessions serially into parallel invocations, the same way we decompose the samples invocations. Fan out to get better wall time.
Look for obvious slow tests and improve them, reducing the tail of the (to be) parallelized test runs. Based on logging, system tests are unsuprisingly the longer session typically.
Look at coverage overlap between samples and library; we may be able to reduce redundancy in testing.
Here's some quick info from running locally on 3.8:
$ nox -s system-3.8 -- --durations=0
44.84s call tests/system.py::TestBigQuery::test_dbapi_w_standard_sql_types
29.28s call tests/system.py::TestBigQuery::test_dbapi_w_query_parameters
27.39s call tests/system.py::TestBigQuery::test_query_many_columns
15.91s call tests/system.py::TestBigQuery::test_query_w_query_params
15.04s call tests/system.py::TestBigQuery::test_load_table_from_file_w_explicit_location
13.98s call tests/system.py::TestBigQuery::test_load_table_from_dataframe_w_automatic_schema
13.79s call tests/system.py::TestBigQuery::test_query_w_standard_sql_types
13.19s call tests/system.py::TestBigQuery::test_load_table_from_dataframe_w_nullable_int64_datatype
11.90s call tests/system.py::TestBigQuery::test_load_table_from_local_avro_file_then_dump_table
11.19s call tests/system.py::TestBigQuery::test_load_table_from_dataframe_w_required
10.83s call tests/system.py::TestBigQuery::test_nested_table_to_dataframe
10.50s call tests/system.py::TestBigQuery::test_load_table_from_dataframe_w_nulls
10.30s call tests/system.py::TestBigQuery::test_load_table_from_json_schema_autodetect
10.11s call tests/system.py::TestBigQuery::test_copy_table
9.60s call tests/system.py::TestBigQuery::test_load_table_from_dataframe_w_explicit_schema
8.56s call tests/system.py::TestBigQuery::test_dbapi_fetchall
8.33s call tests/system.py::TestBigQuery::test_extract_table
7.61s call tests/system.py::TestBigQuery::test_nested_table_to_arrow
7.20s call tests/system.py::TestBigQuery::test_load_table_from_dataframe_w_nullable_int64_datatype_automatic_schema
7.09s call tests/system.py::TestBigQuery::test_load_table_from_uri_then_dump_table
7.03s call tests/system.py::TestBigQuery::test_list_rows_page_size
6.76s call tests/system.py::TestBigQuery::test_query_w_legacy_sql_types
6.67s call tests/system.py::TestBigQuery::test_dbapi_connection_does_not_leak_sockets
6.34s call tests/system.py::TestBigQuery::test_query_w_dml
6.23s call tests/system.py::TestBigQuery::test_dbapi_w_dml
5.71s call tests/system.py::TestBigQuery::test_load_avro_from_uri_then_dump_table
4.26s call tests/system.py::TestBigQuery::test_load_table_from_json_basic_use
3.32s call tests/system.py::TestBigQuery::test_querying_data_w_timeout
3.12s call tests/system.py::test_bigquery_magic
2.89s call tests/system.py::TestBigQuery::test_list_tables
2.87s call tests/system.py::TestBigQuery::test_query_results_to_dataframe
2.81s call tests/system.py::TestBigQuery::test_list_datasets
2.81s call tests/system.py::TestBigQuery::test_job_cancel
2.69s call tests/system.py::TestBigQuery::test_update_table
2.69s call tests/system.py::TestBigQuery::test_insert_rows_from_dataframe
2.50s call tests/system.py::TestBigQuery::test_create_routine
2.46s call tests/system.py::TestBigQuery::test_query_w_page_size
2.22s call tests/system.py::TestBigQuery::test_query_results_to_dataframe_w_bqstorage_v1beta1
2.19s call tests/system.py::TestBigQuery::test_query_results_to_dataframe_w_bqstorage
2.18s call tests/system.py::TestBigQuery::test_large_query_w_public_data
2.10s call tests/system.py::TestBigQuery::test_create_table_rows_fetch_nested_schema
2.00s call tests/system.py::TestBigQuery::test_insert_rows_then_dump_table
1.98s call tests/system.py::TestBigQuery::test_dbapi_fetch_w_bqstorage_client_large_result_set
1.90s call tests/system.py::TestBigQuery::test_update_table_schema
1.89s call tests/system.py::TestBigQuery::test_create_table
1.84s call tests/system.py::TestBigQuery::test_dbapi_fetch_w_bqstorage_client_v1beta1_large_result_set
1.60s call tests/system.py::TestBigQuery::test_insert_rows_nested_nested_dictionary
1.60s call tests/system.py::TestBigQuery::test_create_table_w_time_partitioning_w_clustering_fields
1.59s call tests/system.py::TestBigQuery::test_insert_rows_nested_nested
1.57s call tests/system.py::TestBigQuery::test_query_statistics
1.55s call tests/system.py::TestBigQuery::test_update_dataset
1.46s call tests/system.py::TestBigQuery::test_query_w_timeout
1.44s call tests/system.py::TestBigQuery::test_list_rows_empty_table
1.30s call tests/system.py::TestBigQuery::test_get_dataset
1.18s call tests/system.py::TestBigQuery::test_delete_dataset_delete_contents_true
1.11s call tests/system.py::TestBigQuery::test_create_table_with_policy
1.07s call tests/system.py::TestBigQuery::test_delete_dataset_delete_contents_false
1.00s call tests/system.py::TestBigQuery::test_create_dataset
0.99s call tests/system.py::TestBigQuery::test_delete_dataset_with_string
0.93s call tests/system.py::TestBigQuery::test_query_w_wrong_config
0.88s call tests/system.py::TestBigQuery::test_list_datasets_w_project
0.77s call tests/system.py::TestBigQuery::test_close_releases_open_sockets
0.75s call tests/system.py::TestBigQuery::test_query_w_start_index
0.75s call tests/system.py::TestBigQuery::test_query_future
0.73s call tests/system.py::TestBigQuery::test_query_iter
0.47s call tests/system.py::TestBigQuery::test_list_partitions
0.39s call tests/system.py::TestBigQuery::test_dbapi_dry_run_query
0.27s call tests/system.py::TestBigQuery::test_get_failed_job
0.26s call tests/system.py::TestBigQuery::test_query_w_failed_query
0.17s call tests/system.py::TestBigQuery::test_get_table_w_public_dataset
0.17s call tests/system.py::TestBigQuery::test_list_rows_max_results_w_bqstorage
0.16s call tests/system.py::TestBigQuery::test_get_service_account_email
0.06s setup tests/system.py::test_bigquery_magic
0.01s setup tests/system.py::TestBigQuery::test_close_releases_open_sockets
Here's the slowest unit-3.8 timings (way more tests than included here):
1.73s call tests/unit/test_job.py::TestQueryJob::test_result_invokes_begins
1.45s call tests/unit/test_client.py::TestClient::test_get_dataset
1.00s call tests/unit/test_client.py::TestClient::test_create_job_query_config_w_rateLimitExceeded_error
1.00s call tests/unit/test_client.py::TestClient::test_get_service_account_email_w_custom_retry
0.44s call tests/unit/test_table.py::TestRowIterator::test_to_dataframe_w_bqstorage_updates_progress_bar
0.37s call tests/unit/test_job.py::Test_AsyncJob::test_cancel_w_custom_retry
0.31s call tests/unit/test_table.py::TestRowIterator::test_to_dataframe_w_bqstorage_empty_streams
0.28s call tests/unit/test_table.py::TestRowIterator::test_to_dataframe_error_if_pandas_is_none
0.27s call tests/unit/test_dbapi_connection.py::TestConnection::test_does_not_keep_cursor_instances_alive
0.26s call tests/unit/test_table.py::TestRowIterator::test_to_dataframe_progress_bar_wo_pyarrow
0.26s call tests/unit/test_table.py::TestRowIterator::test_to_dataframe_progress_bar
0.25s call tests/unit/test_table.py::TestRowIterator::test_to_arrow_progress_bar
0.25s call tests/unit/test_dbapi_cursor.py::TestCursor::test_fetchmany_w_row
0.25s call tests/unit/test_dbapi_cursor.py::TestCursor::test_execute_w_query_dry_run
0.25s call tests/unit/test_table.py::TestRowIterator::test_to_dataframe_tqdm_error
0.25s call tests/unit/test_job.py::TestQueryJob::test_result_w_timeout
0.24s call tests/unit/test_table.py::TestRowIterator::test_to_dataframe_w_bqstorage_snapshot
0.22s call tests/unit/test_table.py::TestRowIterator::test_to_dataframe_tabledata_list_w_multiple_pages_return_unique_index
0.21s call tests/unit/test_table.py::TestRowIterator::test_to_arrow_max_results_w_create_bqstorage_warning
0.21s call tests/unit/test_magics.py::test_bigquery_magic_w_table_id_and_destination_var
0.21s call tests/unit/test_client.py::TestClient::test__call_api_applying_custom_retry_on_timeout
0.20s call tests/unit/test_dbapi_connection.py::TestConnection::test_connect_w_client
Reactions are currently unavailable
recent runs of presubmit kokoro show we're taking something on the order of 37-48 minutes for typical runs, with a couple invocations significantly higher than that. It's a significant impediment to actually getting things done.
We can definitely test much faster than this.
Some things to look at:
Decompose the monolilthic kokoro job that runs all the nox sessions serially into parallel invocations, the same way we decompose the samples invocations. Fan out to get better wall time.
Look for obvious slow tests and improve them, reducing the tail of the (to be) parallelized test runs. Based on logging, system tests are unsuprisingly the longer session typically.
Look at coverage overlap between samples and library; we may be able to reduce redundancy in testing.
Here's some quick info from running locally on 3.8:
Here's the slowest unit-3.8 timings (way more tests than included here):