Skip to content

fix(bigquery): fix inserting missing repeated fields#10196

Merged
plamut merged 4 commits intogoogleapis:masterfrom
plamut:iss-9602
Jan 31, 2020
Merged

fix(bigquery): fix inserting missing repeated fields#10196
plamut merged 4 commits intogoogleapis:masterfrom
plamut:iss-9602

Conversation

@plamut
Copy link
Copy Markdown
Contributor

@plamut plamut commented Jan 24, 2020

Fixes #9602.

This PR fixes the issue when inserting rows with missing REPEATED fields. Omitting a REPEATED field from the request does not result in an error anymore.

How to test

Best to run the ready-made script from one of the issue comments.

MIND:
Omitting None values only happens when the code needs to convert the data to JSON by itself, i.e. in insert_rows(). The following thus still fails when insert_rows_json() is used:

rows = [{"int_col": 9, "repeated_col": None}
errors = client.insert_rows_json(TABLE_PATH, rows)

If this inconsistency is not desired, we need to sync the logic in these two methods.

PR checklist

  • Make sure to open an issue as a bug/issue before writing your code! That way we can discuss the change, evaluate designs, and agree on the general idea
  • Ensure the tests and linter pass
  • Code coverage does not decrease (if any source code was changed)
  • Appropriate docs were updated (if necessary)

@plamut plamut added the api: bigquery Issues related to the BigQuery API. label Jan 24, 2020
@plamut plamut requested review from a team and tswast January 24, 2020 12:09
@googlebot googlebot added the cla: yes This human has signed the Contributor License Agreement. label Jan 24, 2020
@plamut
Copy link
Copy Markdown
Contributor Author

plamut commented Jan 24, 2020

The failing snippets test is unrelated, #10195 will fix it,

@plamut plamut added the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Jan 29, 2020
@yoshi-kokoro yoshi-kokoro removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Jan 29, 2020
@plamut
Copy link
Copy Markdown
Contributor Author

plamut commented Jan 29, 2020

The unit tests failure will be fixed in #10222.

@plamut plamut added the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Jan 30, 2020
@yoshi-kokoro yoshi-kokoro removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Jan 30, 2020
@plamut plamut added the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Jan 30, 2020
@yoshi-kokoro yoshi-kokoro removed kokoro:force-run Add this label to force Kokoro to re-run the tests. labels Jan 30, 2020
@plamut
Copy link
Copy Markdown
Contributor Author

plamut commented Jan 30, 2020

Weird, a couple of PermissionDenied errors in system tests. Re-running the jobs.

Update: I cannot reproduce this locally, neither on the PR branch, nor on on the latest master. Hmm...

Copy link
Copy Markdown
Contributor

@tswast tswast left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One nit, otherwise LGTM

Comment thread bigquery/tests/unit/test_client.py
This was referenced Feb 2, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

api: bigquery Issues related to the BigQuery API. cla: yes This human has signed the Contributor License Agreement.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

BigQuery: insert_rows() fails when a repeated field is missing

4 participants