BigQuery: ability to set default job / table / dataset configs in client class

The location is considered a global parameter (by the the BigQuery engineering team and the bq command-line tool), and we are tracking the feature to set a default location at #5148.
There are other non-global properties where it would be useful to set a user-configurable default.

For example:

Labels on jobs. It could be useful to set default labels so that folks can track more easily which jobs are getting run by which application.
use_legacy_sql on QueryJobConfig. Right now this defaults to standard SQL syntax, but some people might prefer to use legacy SQL for existing queries.
The destination_encryption_configuration in jobs and the encryption_configuration in tables. Maybe this one actually makes sense as a global parameter, but it's not considered on in the bq command-line tool.

I'm trying to put together a pull request for this, to get us to the behavior we're looking for that was mentioned in #6042 .

However, when trying to run nox, I'm getting errors that I wouldn't expect. For example:

$ nox -f bigquery/nox.py -s "unit(py='3.6')"
nox > Running session unit(py='3.6')
nox > Creating virtualenv using python3 in /home/blaine/experiments/google-cloud-python/.nox/unit-py-3-6
nox > Session unit(py='3.6') raised exception AttributeError("'Session' object has no attribute 'interpreter'",)
Traceback (most recent call last):
  File "/home/blaine/.local/lib/python3.6/site-packages/nox/sessions.py", line 297, in execute
    self.func(session)
  File "/home/blaine/.local/lib/python3.6/site-packages/nox/_parametrize.py", line 92, in call_wrapper
    return func(*args, **kwargs)
  File "/home/blaine/experiments/google-cloud-python/bigquery/nox.py", line 76, in unit
    session.interpreter = 'python{}'.format(py)
AttributeError: 'Session' object has no attribute 'interpreter'
nox > Session unit(py='3.6') failed.

This makes sense, since it seems nox sessions don't have an interpreter attribute (nor a virtualenv_dirname for that matter, which is also set in that method). But it's surprising to me that these errors are popping up for me? Am I running in a context I shouldn't be? Why is the nox file setting non-existent attributes?

@blainehansen

@blainehansen I was just able to run the tests using the command line you provided. Note that we would normally run from inside the bigquery directory). Do you have python3.6 on your $PATH when you start?

I think the Nox error may be due to nox (new incompatible package) versus nox-automation package (what google-cloud-python still uses AFAIK)

⬆️ This sounds right. The error I'm getting is a basic python problem (can't assign to a non-existent attribute) so it suggests a misalignment between the code I'm running in the project and the version of nox I have.

I don't know if this is helpful but:

$ nox --version
2018.9.14

Also:

$ which python3.6
/usr/bin/python3.6
$ echo $PATH
...redacted
:/usr/bin:
...redacted

Yep, installing and using nox-automation fixed it :) 🎊 🎊

Navigation Menu

Repository navigation

BigQuery: ability to set default job / table / dataset configs in client class #5183

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

BigQuery: ability to set default job / table / dataset configs in client class #5183

Description

Activity

blainehansen commented on Sep 22, 2018

tseaver commented on Sep 22, 2018

tswast commented on Sep 24, 2018

blainehansen commented on Sep 24, 2018

blainehansen commented on Sep 24, 2018

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions