Using gcloud datastore with a local devserver

Issue #2 claims that the library uses the configuration specified for GCD - https://developers.google.com/datastore/docs/tools/devserver - but I don't believe this to be the case.

As far as I can tell from reading the code, there's actually no way to get the gcloud datastore library to talk to a local datastore devserver. The endpoint has a hardcoded value - API_BASE_URL.

Are there any plans to make this configurable?

Nothing concrete but it's something we'd like to do. We've had some private discussions about supporting GCD.

As for #2, we opted to just unit test without any actual HTTP calls and then talk to the real service in our regression/ tests.

@twumdjin

@twumdjin You are interested in GCD for integration tests for an application? Can you provide more details?

I'm interested in using gcd for development. I guess I could set up a separate project just for that. In either case having a separate datastore from prod (and ideally running locally) would provide isolation during the dev process, where everything is still very much in flux.
I assume teams with more than 1 developer would also want this.

In the interim I'm using environment specific logic to patch the base api url in development so that it talks to the gcd devserver. It's pretty ugly but it should do for now. :(

My project is still small so I'm doing unit testing and regression testing on a stub datastore I hacked together. Again environment variables and factory methods make this possible. I saw the comment on the other thread about stubbing at the network layer. I'm somewhat opposed to that because it requires me to understand too much of Google datastore's wire protocol, which frankly is something I'm trying to avoid by depending on an abstraction like the gcloud library in the first place.

It would be nice if there was an in-memory stub solution like the one for AppEngine testing but I'm comfortable depending on my own stub at my current scale.

Yes, for now it seems that something like

from gcloud import datastore

connection = datastore.get_connection()
connection.API_BASE_URL = 'http://localhost:8080'

datastore.set_defaults(connection=connection)

# Drivers start your engines

is a fine hack for testing locally.

In terms of dev and testing, this minimizes the number of if statements you need and the logic can be encapsulated in a custom module.

I don't recommend the hack from How can we cover unit testing for datastore? #2 about stubbing out the network. It's not worth the time.

An in memory stub seems like a great idea, but I'm not sure if it should be part of the library, a separate repo, or something else. Maybe setuptools extras? i.e. pip install gcloud[testing].

Removing unused _EMPTY sentinel from base Connection.

I'm not sure how setuptools extras comes into play: that command would only cause extra dependencies to be installed, i.e. those declared via:

setup(...
    extras_require={'testing': ['foo', 'bar', 'baz']},
)

You can have console scripts which depend on extras and have them added to the path when they run..

That link is exactly what I meant. Thanks for the add.

Closing this since the behavior is "tracked" in googleapis/google-cloud-datastore#21

chore(deps): update all dependencies (#639)

refactor: move generator.options into utils module (#639)

chore: Bump gapic-generator-python version to 1.3.0 (#639)

fix: mimic the pubsub test to clean up bigquery datasets (#639)

chore: add yoshi-python to CODEOWNERS (#639)

fix: mimic the pubsub test to clean up bigquery datasets (#639)

build: use pypi secret from secret manager (#639)

chore: allow releases from previous major versions (#639)

chore(deps): update dependency google-cloud-bigquery to v2.15.0 (#639)

test: duplicate table error class fix (#639)

feat: support schemas in queries and dml statements (#639)

docs: add README to samples subdirectory (#639)

chore: release 1.43.0 (#618)

docs: migrate code from googleapis/python-bigquery (#18303)

Navigation Menu

Repository navigation

Using gcloud datastore with a local devserver #639

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Using gcloud datastore with a local devserver #639

Description

Activity

dhermes commented on Feb 13, 2015

dhermes commented on Feb 13, 2015

twumdjin commented on Feb 13, 2015

dhermes commented on Feb 13, 2015

tseaver commented on Feb 14, 2015

dhermes commented on Feb 14, 2015

dhermes commented on Aug 28, 2015

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions