Skip to content

Commit e3e2a20

Browse files
jbajba
committed
bigquery: retry idempotent RPCs
Add retry logic to every RPC for which it makes sense. Following the BigQuery team, we ignore the error code and use the "reason" field of the error to determine whether to retry. Outstanding issues: - Resumable upload consists of an initial call to get a URL, followed by posts to that URL. Getting the retry right on that initial call requires modifying the ResumableUpload class. At the same time, the num_retries argument should be removed. - Users can't modify the retry behavior of Job.result(), because PollingFuture.result() does not accept a retry argument.
1 parent 9486eeb commit e3e2a20

File tree

7 files changed

+280
-65
lines changed

7 files changed

+280
-65
lines changed

‎bigquery/google/cloud/bigquery/__init__.py‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -27,6 +27,7 @@
2727
__version__ = get_distribution('google-cloud-bigquery').version
2828

2929
from google.cloud.bigquery._helpers import Row
30+
from google.cloud.bigquery._helpers import DEFAULT_RETRY
3031
from google.cloud.bigquery.client import Client
3132
from google.cloud.bigquery.dataset import AccessEntry
3233
from google.cloud.bigquery.dataset import Dataset
@@ -61,4 +62,5 @@
6162
'Table',
6263
'TableReference',
6364
'UDFResource',
65+
'DEFAULT_RETRY',
6466
]

‎bigquery/google/cloud/bigquery/_helpers.py‎

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -20,6 +20,7 @@
2020

2121
import six
2222

23+
from google.api.core import retry
2324
from google.cloud._helpers import UTC
2425
from google.cloud._helpers import _date_from_iso8601_date
2526
from google.cloud._helpers import _datetime_from_microseconds
@@ -520,3 +521,28 @@ def _rows_page_start(iterator, page, response):
520521
total_rows = int(total_rows)
521522
iterator.total_rows = total_rows
522523
# pylint: enable=unused-argument
524+
525+
526+
def _should_retry(exc):
527+
"""Predicate for determining when to retry.
528+
529+
We retry if and only if the 'reason' is 'backendError'
530+
or 'rateLimitExceeded'.
531+
"""
532+
if not hasattr(exc, 'errors'):
533+
return False
534+
if len(exc.errors) == 0:
535+
return False
536+
reason = exc.errors[0]['reason']
537+
return reason == 'backendError' or reason == 'rateLimitExceeded'
538+
539+
540+
DEFAULT_RETRY = retry.Retry(predicate=_should_retry)
541+
"""The default retry object.
542+
543+
Any method with a ``retry`` parameter will be retried automatically,
544+
with reasonable defaults. To disable retry, pass ``retry=None``.
545+
To modify the default retry behavior, call a ``with_XXX`` method
546+
on ``DEFAULT_RETRY``. For example, to change the deadline to 30 seconds,
547+
pass ``retry=bigquery.DEFAULT_RETRY.with_deadline(30)``.
548+
"""

0 commit comments

Comments
 (0)