As discussed in #8105 (comment) and implemented in pandas-gbq at googleapis/python-bigquery-pandas#218, often times the default schema detection works well enough for pandas -> BQ, and only one or two of many columns need to be overridden.
Example:
df = pandas.DataFrame({... lots of columns ..., "needs_manual_schema": [...]})
job_config = bigquery.LoadJobConfig(schema=[
bigquery.SchemaField("needs_manual_schema", "BOOLEAN"),
])
client.load_table_from_dataframe(
df, "my_dataset.my_table", job_config=job_config,
)
# All columns from df are uploaded, but only df["needs_manual_schema"] is
# coerced into a specific type (BOOLEAN).
As discussed in #8105 (comment) and implemented in pandas-gbq at googleapis/python-bigquery-pandas#218, often times the default schema detection works well enough for pandas -> BQ, and only one or two of many columns need to be overridden.
Example: