Skip to content

Commit 15c3375

Browse files
dhermesdhermes
committed
Squashed commit of PR googleapis#281 in main repo.
1 parent 735d8ca commit 15c3375

10 files changed

Lines changed: 597 additions & 32 deletions

File tree

‎CONTRIBUTING.rst‎

Lines changed: 67 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@ Contributing
33

44
#. **Please sign one of the contributor license agreements below.**
55
#. Fork the repo, develop and test your code changes, add docs.
6-
#. Make sure that your commit messages clearly describe the changes.
6+
#. Make sure that your commit messages clearly describe the changes.
77
#. Send a pull request.
88

99
Here are some guidelines for hacking on gcloud-python.
@@ -16,7 +16,7 @@ using a Git checkout:
1616

1717
- While logged into your GitHub account, navigate to the gcloud-python repo on
1818
GitHub.
19-
19+
2020
https://github.com/GoogleCloudPlatform/gcloud-python
2121

2222
- Fork and clone the gcloud-python repository to your GitHub account by
@@ -130,6 +130,70 @@ Running Tests
130130
$ cd ~/hack-on-gcloud/
131131
$ /usr/bin/tox
132132

133+
Running Regression Tests
134+
------------------------
135+
136+
- To run regression tests you can execute::
137+
138+
$ tox -e regression
139+
140+
or run only regression tests for a particular package via::
141+
142+
$ python regression/run_regression.py --package {package}
143+
144+
This alone will not run the tests. You'll need to change some local
145+
auth settings and change some configuration in your project to
146+
run all the tests.
147+
148+
- Regression tests will be run against an actual project and
149+
so you'll need to provide some environment variables to facilitate
150+
authentication to your project:
151+
152+
- ``GCLOUD_TESTS_DATASET_ID``: The name of the dataset your tests connect to.
153+
- ``GCLOUD_TESTS_CLIENT_EMAIL``: The email for the service account you're
154+
authenticating with
155+
- ``GCLOUD_TESTS_KEY_FILE``: The path to an encrypted key file.
156+
See private key
157+
`docs <https://cloud.google.com/storage/docs/authentication#generating-a-private-key>`__
158+
for explanation on how to get a private key.
159+
160+
- Examples of these can be found in ``regression/local_test_setup.sample``. We
161+
recommend copying this to ``regression/local_test_setup``, editing the values
162+
and sourcing them into your environment::
163+
164+
$ source regression/local_test_setup
165+
166+
- The ``GCLOUD_TESTS_KEY_FILE`` value should point to a valid path (relative or
167+
absolute) on your system where the key file for your service account can
168+
be found.
169+
170+
- For datastore tests, you'll need to create composite
171+
`indexes <https://cloud.google.com/datastore/docs/tools/indexconfig>`__
172+
with the ``gcloud`` command line
173+
`tool <https://developers.google.com/cloud/sdk/gcloud/>`__::
174+
175+
# Install the app (App Engine Command Line Interface) component.
176+
$ gcloud components update app
177+
178+
# See https://cloud.google.com/sdk/crypto for details on PyOpenSSL and
179+
# http://stackoverflow.com/a/25067729/1068170 for why we must persist.
180+
$ export CLOUDSDK_PYTHON_SITEPACKAGES=1
181+
182+
# Authenticate the gcloud tool with your account.
183+
$ gcloud auth activate-service-account $GCLOUD_TESTS_CLIENT_EMAIL \
184+
> --key-file=$GCLOUD_TESTS_KEY_FILE
185+
186+
# Create the indexes
187+
$ gcloud preview datastore create-indexes regression/data/ \
188+
> --project=$GCLOUD_TESTS_DATASET_ID
189+
190+
# Restore your environment to its previous state.
191+
$ unset CLOUDSDK_PYTHON_SITEPACKAGES
192+
193+
- For datastore query tests, you'll need stored data in your dataset.
194+
To populate this data, run::
195+
196+
$ python regression/populate_datastore.py
133197

134198
Test Coverage
135199
-------------
@@ -184,4 +248,4 @@ Before we can accept your pull requests you'll need to sign a Contributor Licens
184248
- **If you are an individual writing original source code** and **you own the intellectual property**, then you'll need to sign an `individual CLA <https://developers.google.com/open-source/cla/individual>`__.
185249
- **If you work for a company that wants to allow you to contribute your work**, then you'll need to sign a `corporate CLA <https://developers.google.com/open-source/cla/corporate>`__.
186250

187-
You can sign these electronically (just scroll to the bottom). After that, we'll be able to accept your pull requests.
251+
You can sign these electronically (just scroll to the bottom). After that, we'll be able to accept your pull requests.

‎gcloud/datastore/key.py‎

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -251,3 +251,29 @@ def parent(self):
251251

252252
def __repr__(self):
253253
return '<Key%s>' % self.path()
254+
255+
def __eq__(self, other):
256+
if self is other:
257+
return True
258+
259+
if not isinstance(other, self.__class__):
260+
return False
261+
262+
# Check that paths match.
263+
if self.path() != other.path():
264+
return False
265+
266+
# Check that datasets match.
267+
if not (self._dataset_id == other._dataset_id or
268+
self._dataset_id is None or other._dataset_id is None):
269+
return False
270+
271+
# Check that namespaces match.
272+
if not (self._namespace == other._namespace or
273+
self._namespace is None or other._namespace is None):
274+
return False
275+
276+
return True
277+
278+
def __ne__(self, other):
279+
return not self.__eq__(other)

‎gcloud/datastore/query.py‎

Lines changed: 104 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -60,6 +60,9 @@ def __init__(self, kind=None, dataset=None, namespace=None):
6060
self._namespace = namespace
6161
self._pb = datastore_pb.Query()
6262
self._cursor = None
63+
self._projection = []
64+
self._offset = 0
65+
self._group_by = []
6366

6467
if kind:
6568
self._pb.kind.add().name = kind
@@ -411,3 +414,104 @@ def order(self, *properties):
411414
property_order.direction = property_order.ASCENDING
412415

413416
return clone
417+
418+
def projection(self, projection=None):
419+
"""Adds a projection to the query.
420+
421+
This is a hybrid getter / setter, used as::
422+
423+
>>> query = Query('Person')
424+
>>> query.projection() # Get the projection for this query.
425+
[]
426+
>>> query = query.projection(['name'])
427+
>>> query.projection() # Get the projection for this query.
428+
['name']
429+
430+
:type projection: sequence of strings
431+
:param projection: Each value is a string giving the name of a
432+
property to be included in the projection query.
433+
434+
:rtype: :class:`Query` or `list` of strings.
435+
:returns: If no arguments, returns the current projection.
436+
If a projection is provided, returns a clone of the
437+
:class:`Query` with that projection set.
438+
"""
439+
if projection is None:
440+
return self._projection
441+
442+
clone = self._clone()
443+
clone._projection = projection
444+
445+
# Reset projection values to empty.
446+
clone._pb.projection._values = []
447+
448+
# Add each name to list of projections.
449+
for projection_name in projection:
450+
clone._pb.projection.add().property.name = projection_name
451+
return clone
452+
453+
def offset(self, offset=None):
454+
"""Adds offset to the query to allow pagination.
455+
456+
NOTE: Paging with cursors should be preferred to using an offset.
457+
458+
This is a hybrid getter / setter, used as::
459+
460+
>>> query = Query('Person')
461+
>>> query.offset() # Get the offset for this query.
462+
0
463+
>>> query = query.offset(10)
464+
>>> query.offset() # Get the offset for this query.
465+
10
466+
467+
:type offset: non-negative integer.
468+
:param offset: Value representing where to start a query for
469+
a given kind.
470+
471+
:rtype: :class:`Query` or `int`.
472+
:returns: If no arguments, returns the current offset.
473+
If an offset is provided, returns a clone of the
474+
:class:`Query` with that offset set.
475+
"""
476+
if offset is None:
477+
return self._offset
478+
479+
clone = self._clone()
480+
clone._offset = offset
481+
clone._pb.offset = offset
482+
return clone
483+
484+
def group_by(self, group_by=None):
485+
"""Adds a group_by to the query.
486+
487+
This is a hybrid getter / setter, used as::
488+
489+
>>> query = Query('Person')
490+
>>> query.group_by() # Get the group_by for this query.
491+
[]
492+
>>> query = query.group_by(['name'])
493+
>>> query.group_by() # Get the group_by for this query.
494+
['name']
495+
496+
:type group_by: sequence of strings
497+
:param group_by: Each value is a string giving the name of a
498+
property to use to group results together.
499+
500+
:rtype: :class:`Query` or `list` of strings.
501+
:returns: If no arguments, returns the current group_by.
502+
If a list of group by properties is provided, returns a clone
503+
of the :class:`Query` with that list of values set.
504+
"""
505+
if group_by is None:
506+
return self._group_by
507+
508+
clone = self._clone()
509+
clone._group_by = group_by
510+
511+
# Reset group_by values to empty.
512+
clone._pb.group_by._values = []
513+
514+
# Add each name to list of group_bys.
515+
for group_by_name in group_by:
516+
clone._pb.group_by.add().name = group_by_name
517+
return clone

‎gcloud/datastore/test_key.py‎

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -322,3 +322,31 @@ def test_parent_explicit_top_level(self):
322322
def test_parent_explicit_nested(self):
323323
key = self._getTargetClass().from_path('abc', 'def', 'ghi', 123)
324324
self.assertEqual(key.parent().path(), [{'kind': 'abc', 'name': 'def'}])
325+
326+
def test_key___eq__(self):
327+
key1 = self._getTargetClass().from_path('abc', 'def')
328+
key2 = self._getTargetClass().from_path('abc', 'def')
329+
self.assertFalse(key1 is key2)
330+
self.assertEqual(key1, key2)
331+
332+
self.assertEqual(key1, key1)
333+
key3 = self._getTargetClass().from_path('abc', 'ghi')
334+
self.assertNotEqual(key1, key3)
335+
336+
def test_key___eq___wrong_type(self):
337+
key = self._getTargetClass().from_path('abc', 'def')
338+
self.assertNotEqual(key, 10)
339+
340+
def test_key___eq___dataset_id(self):
341+
key1 = self._getTargetClass().from_path('abc', 'def')
342+
key2 = self._getTargetClass().from_path('abc', 'def', dataset_id='foo')
343+
self.assertEqual(key1, key2)
344+
key3 = self._getTargetClass().from_path('abc', 'def', dataset_id='bar')
345+
self.assertNotEqual(key2, key3)
346+
347+
def test_key___eq___namespace(self):
348+
key1 = self._getTargetClass().from_path('abc', 'def')
349+
key2 = self._getTargetClass().from_path('abc', 'def', namespace='foo')
350+
self.assertEqual(key1, key2)
351+
key3 = self._getTargetClass().from_path('abc', 'def', namespace='bar')
352+
self.assertNotEqual(key2, key3)

‎gcloud/datastore/test_query.py‎

Lines changed: 64 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -415,6 +415,70 @@ def test_order_multiple(self):
415415
self.assertEqual(prop_pb.property.name, 'bar')
416416
self.assertEqual(prop_pb.direction, prop_pb.DESCENDING)
417417

418+
def test_projection_empty(self):
419+
_KIND = 'KIND'
420+
before = self._makeOne(_KIND)
421+
after = before.projection([])
422+
self.assertFalse(after is before)
423+
self.assertTrue(isinstance(after, self._getTargetClass()))
424+
self.assertEqual(before.to_protobuf(), after.to_protobuf())
425+
426+
def test_projection_non_empty(self):
427+
_KIND = 'KIND'
428+
before = self._makeOne(_KIND)
429+
after = before.projection(['field1', 'field2'])
430+
projection_pb = list(after.to_protobuf().projection)
431+
self.assertEqual(len(projection_pb), 2)
432+
prop_pb1 = projection_pb[0]
433+
self.assertEqual(prop_pb1.property.name, 'field1')
434+
prop_pb2 = projection_pb[1]
435+
self.assertEqual(prop_pb2.property.name, 'field2')
436+
437+
def test_get_projection_non_empty(self):
438+
_KIND = 'KIND'
439+
_PROJECTION = ['field1', 'field2']
440+
after = self._makeOne(_KIND).projection(_PROJECTION)
441+
self.assertEqual(after.projection(), _PROJECTION)
442+
443+
def test_set_offset(self):
444+
_KIND = 'KIND'
445+
_OFFSET = 42
446+
before = self._makeOne(_KIND)
447+
after = before.offset(_OFFSET)
448+
offset_pb = after.to_protobuf().offset
449+
self.assertEqual(offset_pb, _OFFSET)
450+
451+
def test_get_offset(self):
452+
_KIND = 'KIND'
453+
_OFFSET = 10
454+
after = self._makeOne(_KIND).offset(_OFFSET)
455+
self.assertEqual(after.offset(), _OFFSET)
456+
457+
def test_group_by_empty(self):
458+
_KIND = 'KIND'
459+
before = self._makeOne(_KIND)
460+
after = before.group_by([])
461+
self.assertFalse(after is before)
462+
self.assertTrue(isinstance(after, self._getTargetClass()))
463+
self.assertEqual(before.to_protobuf(), after.to_protobuf())
464+
465+
def test_group_by_non_empty(self):
466+
_KIND = 'KIND'
467+
before = self._makeOne(_KIND)
468+
after = before.group_by(['field1', 'field2'])
469+
group_by_pb = list(after.to_protobuf().group_by)
470+
self.assertEqual(len(group_by_pb), 2)
471+
prop_pb1 = group_by_pb[0]
472+
self.assertEqual(prop_pb1.name, 'field1')
473+
prop_pb2 = group_by_pb[1]
474+
self.assertEqual(prop_pb2.name, 'field2')
475+
476+
def test_get_group_by_non_empty(self):
477+
_KIND = 'KIND'
478+
_GROUP_BY = ['field1', 'field2']
479+
after = self._makeOne(_KIND).group_by(_GROUP_BY)
480+
self.assertEqual(after.group_by(), _GROUP_BY)
481+
418482

419483
class _Dataset(object):
420484

‎regression/data/index.yaml‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
indexes:
2+
3+
- kind: Character
4+
properties:
5+
- name: family
6+
- name: appearances
7+
8+
- kind: Character
9+
properties:
10+
- name: name
11+
- name: family

0 commit comments

Comments
 (0)