Thanks for stopping by to let us know something could be better!
PLEASE READ: If you have a support contract with Google, please create an issue in the support console instead of filing on GitHub. This will ensure a timely response.
Please run down the following list and make sure you've tried the usual "quick fixes":
If you are still having issues, please be sure to include as much information as possible:
Environment details
- OS type and version: MACOS Catalina 10.15.5
- Python version: 3.7.6
- pip version: 20.0.2
google-cloud-bigquery version: 1.28.0
Steps to reproduce
Step 1. Extract BQ tabke data to bucket
Code example
# from google.cloud import bigquery
# client = bigquery.Client()
# bucket_name = 'my-bucket'
destination_uri = "gs://{}/{}".format(bucket_name, "shakespeare.json")
dataset_ref = bigquery.DatasetReference(project, dataset_id)
table_ref = dataset_ref.table("shakespeare")
job_config = bigquery.job.ExtractJobConfig()
job_config.destinationFormat = bigquery.DestinationFormat.NEWLINE_DELIMITED_JSON
job_config.compression = bigquery.Compression.GZIP
extract_job = client.extract_table(
table_ref,
destination_uri,
# Location must match that of the source table.
location="US",
job_config=job_config,
) # API request
extract_job.result() # Waits for job to complete.
- Fetch the table from bucket and load into bq
Code example
from google.cloud import bigquery
# Construct a BigQuery client object.
client = bigquery.Client()
# TODO(developer): Set table_id to the ID of the table to create.
# table_id = "your-project.your_dataset.your_table_name"
job_config = bigquery.LoadJobConfig(
schema=[
bigquery.SchemaField("name", "STRING"),
bigquery.SchemaField("post_abbr", "STRING"),
], #This is not shakespeare schema, as just copied pasted the code provided on google docs
source_format=bigquery.SourceFormat.NEWLINE_DELIMITED_JSON,
)
uri = "gs://{}/{}".format(bucket_name, "shakespeare.json")
load_job = client.load_table_from_uri(
uri,
table_id,
location="US", # Must match the destination dataset location.
job_config=job_config,
) # Make an API request.
load_job.result() # Waits for the job to complete.
destination_table = client.get_table(table_id)
print("Loaded {} rows.".format(destination_table.num_rows))
Stack trace
Error while reading data, error message: JSON table encountered too many errors, giving up. Rows: 1; errors: 1. Please look into the errors[] collection for more details.
Error while reading data, error message: JSON processing encountered too many errors, giving up. Rows: 1; errors: 1; max bad: 0; error percent: 0
Error while reading data, error message: JSON parsing error in row starting at position 0: Parser terminated before end of string
However when I extract the table using bq command line and follow the example of:
bq extract \
--compression GZIP \
--destination_format NEWLINE_DELIMITED_JSON \
'mydataset.mytable' \
gs://example-bucket/myfile.json
And then run the second step, I don't get any error and my table is successfully imported.
Making sure to follow these steps will guarantee the quickest resolution possible.
Thanks!
Thanks for stopping by to let us know something could be better!
PLEASE READ: If you have a support contract with Google, please create an issue in the support console instead of filing on GitHub. This will ensure a timely response.
Please run down the following list and make sure you've tried the usual "quick fixes":
If you are still having issues, please be sure to include as much information as possible:
Environment details
google-cloud-bigqueryversion: 1.28.0Steps to reproduce
Step 1. Extract BQ tabke data to bucket
Code example
Code example
Stack trace
However when I extract the table using bq command line and follow the example of:
And then run the second step, I don't get any error and my table is successfully imported.
Making sure to follow these steps will guarantee the quickest resolution possible.
Thanks!