Skip to content

Commit d9078cb

Browse files
autoerrautoerr
authored
Add BigQuery Remote Function code sample for Vision (GoogleCloudPlatform#8426)
Add BigQuery Remote Function code sample that uses Cloud Vision API to label input images
1 parent c8aa9c8 commit d9078cb

File tree

8 files changed

+252
-0
lines changed

8 files changed

+252
-0
lines changed

‎.github/CODEOWNERS‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,7 @@
2121
/appengine/standard_python3/spanner/* @GoogleCloudPlatform/api-spanner-python @GoogleCloudPlatform/python-samples-reviewers
2222
/auth/**/* @arithmetic1728 @GoogleCloudPlatform/python-samples-reviewers
2323
/bigquery/**/* @chalmerlowe @GoogleCloudPlatform/python-samples-reviewers
24+
/bigquery/remote_function/**/* @autoerr @GoogleCloudPlatform/python-samples-reviewers
2425
/billing/**/* @GoogleCloudPlatform/billing-samples-maintainers @GoogleCloudPlatform/python-samples-reviewers
2526
/blog/**/* @GoogleCloudPlatform/python-samples-reviewers
2627
/cdn/**/* @mpwarres @GoogleCloudPlatform/python-samples-reviewers

‎bigquery/remote-function/README.rst‎

Lines changed: 89 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,89 @@
1+
.. This file is automatically generated. Do not edit this file directly.
2+
3+
BigQuery Remote Function Python Samples
4+
===============================================================================
5+
6+
.. image:: https://gstatic.com/cloudssh/images/open-btn.png
7+
:target: https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/GoogleCloudPlatform/python-docs-samples&page=editor&open_in_editor=/README.rst
8+
9+
10+
This directory contains samples for BigQuery Remote Function. `BigQuery Remote Function`_ allows you to analyze structured and unstructured data by integrating services outside BigQuery including Vision, Document AI, Speech-to-Text etc.
11+
12+
13+
14+
15+
.. _BigQuery Remote Function: https://cloud.google.com/bigquery/docs/reference/standard-sql/remote-functions
16+
17+
18+
19+
20+
21+
22+
Setup
23+
-------------------------------------------------------------------------------
24+
25+
26+
Authentication
27+
++++++++++++++
28+
29+
This sample requires you to have authentication setup. Refer to the
30+
`Authentication Getting Started Guide`_ for instructions on setting up
31+
credentials for applications.
32+
33+
.. _Authentication Getting Started Guide:
34+
https://cloud.google.com/docs/authentication/getting-started
35+
36+
Install Dependencies
37+
++++++++++++++++++++
38+
39+
#. Clone python-docs-samples and change directory to the sample directory you want to use.
40+
41+
.. code-block:: bash
42+
43+
$ git clone https://github.com/GoogleCloudPlatform/python-docs-samples.git
44+
45+
#. Install `pip`_ and `virtualenv`_ if you do not already have them. You may want to refer to the `Python Development Environment Setup Guide`_ for Google Cloud Platform for instructions.
46+
47+
.. _Python Development Environment Setup Guide:
48+
https://cloud.google.com/python/setup
49+
50+
#. Create a virtualenv. Samples are compatible with Python 3.6+.
51+
52+
.. code-block:: bash
53+
54+
$ virtualenv env
55+
$ source env/bin/activate
56+
57+
#. Install the dependencies needed to run the samples.
58+
59+
.. code-block:: bash
60+
61+
$ pip install -r requirements.txt
62+
63+
.. _pip: https://pip.pypa.io/
64+
.. _virtualenv: https://virtualenv.pypa.io/
65+
66+
Samples
67+
-------------------------------------------------------------------------------
68+
69+
- `Vision`_: this sample can detect and extract objects from input images.
70+
71+
72+
.. _Vision: vision/
73+
74+
The client library
75+
-------------------------------------------------------------------------------
76+
77+
This sample uses the `Google Cloud Client Library for Python`_.
78+
You can read the documentation for more details on API usage and use GitHub
79+
to `browse the source`_ and `report issues`_.
80+
81+
.. _Google Cloud Client Library for Python:
82+
https://googlecloudplatform.github.io/google-cloud-python/
83+
.. _browse the source:
84+
https://github.com/GoogleCloudPlatform/google-cloud-python
85+
.. _report issues:
86+
https://github.com/GoogleCloudPlatform/google-cloud-python/issues
87+
88+
89+
.. _Google Cloud SDK: https://cloud.google.com/sdk/

‎bigquery/remote-function/README.rst.in‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
# This file is used to generate README.rst
2+
# python3 ../../scripts/readme-gen/readme_gen.py README.rst.in
3+
4+
product:
5+
name: BigQuery Remote Function
6+
short_name: BigQuery Remote Function
7+
url: https://cloud.google.com/bigquery/docs/reference/standard-sql/remote-functions
8+
description: >
9+
BigQuery Remote Function allows you to analyze structured and unstructured data by integrating services outside BigQuery including Vision, Document AI, Speech-to-Text etc.
10+
11+
setup:
12+
- auth
13+
- install_deps
14+
15+
samples:
16+
- name: Vision
17+
file: vision/vision_function.py
18+
19+
cloud_client_library: true

‎bigquery/remote-function/vision/noxfile_config.py‎

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
# Copyright 2022 Google LLC
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# http://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.
14+
15+
# TEST_CONFIG_OVERRIDE copied from the source of truth:
16+
# https://github.com/GoogleCloudPlatform/python-docs-samples/blob/main/noxfile_config.py
17+
18+
TEST_CONFIG_OVERRIDE = {
19+
# You can opt out from the test for specific Python versions.
20+
"ignored_versions": ["2.7"],
21+
# Old samples are opted out of enforcing Python type hints
22+
# All new samples should feature them
23+
"enforce_type_hints": True,
24+
# An envvar key for determining the project id to use. Change it
25+
# to 'BUILD_SPECIFIC_GCLOUD_PROJECT' if you want to opt in using a
26+
# build specific Cloud project. You can also use your own string
27+
# to use your own Cloud project.
28+
"gcloud_project_env": "GOOGLE_CLOUD_PROJECT",
29+
# 'gcloud_project_env': 'BUILD_SPECIFIC_GCLOUD_PROJECT',
30+
# A dictionary you want to inject into your test. Don't put any
31+
# secrets here. These values will override predefined values.
32+
"envs": {},
33+
}

‎bigquery/remote-function/vision/requirements-test.txt‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
Flask==2.2.2
2+
functions-framework==3.2.0
3+
google-cloud-vision==3.1.2
4+
pytest==7.1.3

‎bigquery/remote-function/vision/requirements.txt‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
Flask==2.2.2
2+
functions-framework==3.2.0
3+
google-cloud-vision==3.1.2

‎bigquery/remote-function/vision/vision_function.py‎

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,42 @@
1+
# Copyright 2022 Google LLC
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# http://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.
14+
15+
# [START bigquery_remote_function_vision]
16+
import flask
17+
import functions_framework
18+
from google.cloud import vision_v1
19+
20+
21+
@functions_framework.http
22+
def label_detection(request: flask.Request) -> flask.Response:
23+
"""BigQuery remote function to label input images.
24+
Args:
25+
request: HTTP request from BigQuery
26+
https://cloud.google.com/bigquery/docs/reference/standard-sql/remote-functions#input_format
27+
Returns:
28+
HTTP response to BigQuery
29+
https://cloud.google.com/bigquery/docs/reference/standard-sql/remote-functions#output_format
30+
"""
31+
try:
32+
client = vision_v1.ImageAnnotatorClient()
33+
calls = request.get_json()['calls']
34+
replies = []
35+
for call in calls:
36+
results = client.label_detection(
37+
{'source': {'image_uri': call[0]}})
38+
replies.append(vision_v1.AnnotateImageResponse.to_dict(results))
39+
return flask.make_response(flask.jsonify({'replies': replies}))
40+
except Exception as e:
41+
return flask.make_response(flask.jsonify({'errorMessage': str(e)}), 400)
42+
# [END bigquery_remote_function_vision]

‎bigquery/remote-function/vision/vision_function_test.py‎

Lines changed: 61 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,61 @@
1+
# Copyright 2022 Google LLC
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# http://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.
14+
15+
from unittest import mock
16+
17+
import flask
18+
from google.cloud import vision_v1
19+
import pytest
20+
21+
import vision_function
22+
23+
24+
# Create a fake "app" for generating test request contexts.
25+
@pytest.fixture(scope="module")
26+
def app() -> flask.Flask:
27+
return flask.Flask(__name__)
28+
29+
30+
@mock.patch('vision_function.vision_v1')
31+
def test_vision_function(mock_vision_v1: object, app: flask.Flask) -> None:
32+
label_detection_mock = mock.Mock(side_effect=[
33+
vision_v1.AnnotateImageResponse(
34+
{'label_annotations': [{'description': 'apple'}]}),
35+
vision_v1.AnnotateImageResponse(
36+
{'label_annotations': [{'description': 'banana'}]})])
37+
mock_vision_v1.ImageAnnotatorClient = mock.Mock(
38+
return_value=mock.Mock(label_detection=label_detection_mock))
39+
mock_vision_v1.AnnotateImageResponse = vision_v1.AnnotateImageResponse
40+
with app.test_request_context(
41+
json={'calls': [['https://storage.googleapis.com/bucket/apple'],
42+
['https://storage.googleapis.com/bucket/banana']]}):
43+
response = vision_function.label_detection(flask.request)
44+
assert response.status_code == 200
45+
assert len(response.get_json()['replies']) == 2
46+
assert 'apple' in str(response.get_json()['replies'][0])
47+
assert 'banana' in str(response.get_json()['replies'][1])
48+
49+
50+
@mock.patch('vision_function.vision_v1')
51+
def test_vision_function_error(
52+
mock_vision_v1: object, app: flask.Flask) -> None:
53+
label_detection_mock = mock.Mock(side_effect=Exception('API error'))
54+
mock_vision_v1.ImageAnnotatorClient = mock.Mock(
55+
return_value=mock.Mock(label_detection=label_detection_mock))
56+
with app.test_request_context(
57+
json={'calls': [['https://storage.googleapis.com/bucket/apple'],
58+
['https://storage.googleapis.com/bucket/banana']]}):
59+
response = vision_function.label_detection(flask.request)
60+
assert response.status_code == 400
61+
assert 'API error' in str(response.get_data())

0 commit comments

Comments
 (0)