Skip to content

Commit 9a64e63

Browse files
committed
Merge pull request #913 from aozarov/pubsub-temp
Update pubsub-alpha branch with the changes from master
2 parents 22e5665 + 613f19d commit 9a64e63

379 files changed

Lines changed: 5342 additions & 4681 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

README.md

Lines changed: 83 additions & 74 deletions
Large diffs are not rendered by default.

RELEASING.md

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -7,10 +7,10 @@ Most of the release process is handled by the `after_success.sh` script, trigger
77
1. Run `utilities/update_pom_version.sh` from the repository's base directory.
88
This script takes an optional argument denoting the new version. By default, if the current version is X.Y.Z-SNAPSHOT, the script will update the version in all the pom.xml files to X.Y.Z. If desired, another version can be supplied via command line argument instead.
99

10-
2. Create a PR to update the pom.xml version.
11-
The PR should look something like [#225](https://github.com/GoogleCloudPlatform/gcloud-java/pull/225). After this PR is merged into GoogleCloudPlatform/gcloud-java, Travis CI will push a new website to GoogleCloudPlatform/gh-pages, push a new artifact to the Maven Central Repository, and update versions in the README files.
10+
2. Create a PR to update the pom.xml version. If releasing a new client library, this PR should also update javadoc grouping in the base directory's [pom.xml](./pom.xml).
11+
PRs that don't release new modules should look something like [#225](https://github.com/GoogleCloudPlatform/gcloud-java/pull/225). PRs that do release a new module should also add the appropriate packages to the javadoc groups "SPI" and "Test helpers", as shown in [#802](https://github.com/GoogleCloudPlatform/gcloud-java/pull/802) for `gcloud-java-dns`. After this PR is merged into GoogleCloudPlatform/gcloud-java, Travis CI will push a new website to GoogleCloudPlatform/gh-pages, push a new artifact to the Maven Central Repository, and update versions in the README files. Do not merge in any non-release-related pull requests between the start of step 2 and the end of step 6. Between these steps, the project version is a non-snapshot version, so any commits to the master branch will cause Travis to spend extra resources attempting to redeploy artifacts.
1212

13-
3. Before moving on, verify that the artifacts have successfully been pushed to the Maven Central Repository. Open Travis CI, click the ["Build History" tab](https://travis-ci.org/GoogleCloudPlatform/gcloud-java/builds), and open the second build's logs for Step 2's PR. Be sure that you are not opening the "Pull Request" build logs. When the build finishes, scroll to the end of the log and verify that the artifacts were successfully staged and deployed. You can also search for `gcloud-java` on the [Sonatype website](https://oss.sonatype.org/#nexus-search;quick~gcloud-java) and check the latest version number. If the deployment didn't succeed because of a flaky test, rerun the build.
13+
3. Before moving on, verify that the artifacts have successfully been pushed to the Maven Central Repository. Open Travis CI, click the ["Build History" tab](https://travis-ci.org/GoogleCloudPlatform/gcloud-java/builds), and open the second build's logs for Step 2's PR. Be sure that you are not opening the "Pull Request" build logs. When the build finishes, scroll to the end of the log and verify that the artifacts were successfully staged and deployed. Search for `gcloud-java` on the [Sonatype website](https://oss.sonatype.org/#nexus-search;quick~gcloud-java) and check the latest version number. In rare cases (when the Maven plugin that determines the version of the repository fails), the artifacts may not be deployed even if the version in the pom.xml files doesn't contain `SNAPSHOT`. If the artifacts weren't deployed due to invalid version parsing or a flaky test, rerun the build.
1414

1515
4. Publish a release on Github manually.
1616
Go to the [releases page](https://github.com/GoogleCloudPlatform/gcloud-java/releases) and open the appropriate release draft. Make sure the "Tag Version" is `vX.Y.Z` and the "Release Title" is `X.Y.Z`, where `X.Y.Z` is the release version as listed in the `pom.xml` files. The draft should already have all changes that impact users since the previous release. To double check this, you can use the `git log` command and look through the merged master branch pull requests. Here is an example of the log command to get non-merge commits between v0.0.12 and v0.1.0:
@@ -26,7 +26,7 @@ As mentioned before, there is an optional version argument. By default, the scr
2626

2727
6. Create and merge in another PR to reflect the updated project version. For an example of what this PR should look like, see [#227](https://github.com/GoogleCloudPlatform/gcloud-java/pull/227).
2828

29-
7. Be sure to update App Engine documentation and [java-docs-samples](https://github.com/GoogleCloudPlatform/java-docs-samples) code as necessary. See directions [here](https://docs.google.com/a/google.com/document/d/1SS3xNn2v0qW7EadGUPBUAPIQAH5VY6WSFmT17ZjjUVE/edit?usp=sharing).
29+
7. Be sure to update Google Cloud Platform docs, [java-docs-samples](https://github.com/GoogleCloudPlatform/java-docs-samples) code/docs, and [getting-started-java](https://github.com/GoogleCloudPlatform/getting-started-java) code/docs. See directions [here](https://docs.google.com/a/google.com/document/d/1SS3xNn2v0qW7EadGUPBUAPIQAH5VY6WSFmT17ZjjUVE/).
3030

3131
### To push a snapshot version
3232

TESTING.md

Lines changed: 34 additions & 27 deletions
Original file line numberDiff line numberDiff line change
@@ -11,55 +11,64 @@ This library provides tools to help write tests for code that uses the following
1111

1212
#### On your machine
1313

14-
You can test against a temporary local datastore by following these steps:
14+
You can test against a temporary local Datastore by following these steps:
1515

16-
1. Start the local datastore emulator using `LocalGcdHelper`. This can be done in two ways:
17-
- Run `LocalGcdHelper.java`'s `main` method with arguments `START` and (optionally) `--port=<port number>`. This will create a temporary folder on your computer and bind `localhost:<port number>` for communication with the local datastore. The port number is an optional argument. If no port number is specified, port 8080 will be used.
18-
- Call `LocalGcdHelper.start(<project ID>, <port number>)` before running your tests. Save the `LocalGcdHelper` object returned so that you can stop the emulator later.
16+
1. Start the local Datastore emulator before running your tests using `LocalDatastoreHelper`'s `create` and `start` methods. This will create a temporary folder on your computer and bind a port for communication with the local Datastore. There is an optional argument for `create`: consistency. The consistency setting controls the fraction of Datastore writes that are immediately visible in global queries.
17+
```java
18+
// Use the default consistency setting of 0.9
19+
LocalDatastoreHelper helper = LocalDatastoreHelper.create();
20+
// or explicitly set the consistency
21+
helper = LocalDatastoreHelper.create(0.6);
22+
23+
helper.start(); // Starts the local Datastore emulator in a separate process
24+
```
1925

20-
2. In your program, create and use a datastore whose host is set host to `localhost:<port number>`. For example,
26+
2. Create and use a `Datastore` object with the options given by the `LocalDatastoreHelper` instance. For example:
2127
```java
22-
DatastoreOptions options = DatastoreOptions.builder()
23-
.projectId(PROJECT_ID)
24-
.host("http://localhost:8080")
25-
.build();
26-
Datastore localDatastore = options.service();
28+
Datastore localDatastore = helper.options().service();
2729
```
30+
2831
3. Run your tests.
2932

30-
4. Stop the local datastore emulator.
31-
- If you ran `LocalGcdHelper.java`'s `main` function to start the emulator, run `LocalGcdHelper.java`'s `main` method with arguments `STOP` and (optionally) `--port=<port number>`. If the port is not supplied, the program will attempt to close the last port started.
32-
- If you ran `LocalGcdHelper.start()` to start the emulator, call the `stop()` method on the `LocalGcdHelper` object returned by `LocalGcdHelper.start()`.
33+
4. Stop the local datastore emulator by calling the `stop()` method, like so:
34+
```java
35+
helper.stop();
36+
```
3337

3438
#### On a remote machine
3539

36-
You can test against a remote datastore emulator as well. To do this, set the `DatastoreOptions` project endpoint to the hostname of the remote machine, like the example below.
40+
You can test against a remote Datastore emulator as well. To do this, set the `DatastoreOptions` project endpoint to the hostname of the remote machine, like the example below.
3741

3842
```java
3943
DatastoreOptions options = DatastoreOptions.builder()
40-
.projectId(PROJECT_ID)
44+
.projectId("my-project-id") // must match project ID specified on remote machine
4145
.host("http://<hostname of machine>:<port>")
46+
.authCredentials(AuthCredentials.noAuth())
4247
.build();
4348
Datastore localDatastore = options.service();
4449
```
4550

46-
Note that the remote datastore must be running before your tests are run.
51+
We recommend that you start the emulator on the remote machine using the [Google Cloud SDK](https://cloud.google.com/sdk/gcloud/reference/beta/emulators/datastore/) from command line, as shown below:
52+
53+
```
54+
gcloud beta emulators datastore start --host-port <hostname of machine>:<port>
55+
```
4756

4857
### Testing code that uses Storage
4958

50-
Currently, there isn't an emulator for Google Cloud Storage, so an alternative is to create a test project. `RemoteGcsHelper` contains convenience methods to make setting up and cleaning up the test project easier. To use this class, follow the steps below:
59+
Currently, there isn't an emulator for Google Cloud Storage, so an alternative is to create a test project. `RemoteStorageHelper` contains convenience methods to make setting up and cleaning up the test project easier. To use this class, follow the steps below:
5160

5261
1. Create a test Google Cloud project.
5362

5463
2. Download a JSON service account credentials file from the Google Developer's Console. See more about this on the [Google Cloud Platform Storage Authentication page][cloud-platform-storage-authentication].
5564

56-
3. Create a `RemoteGcsHelper` object using your project ID and JSON key.
57-
Here is an example that uses the `RemoteGcsHelper` to create a bucket.
65+
3. Create a `RemoteStorageHelper` object using your project ID and JSON key.
66+
Here is an example that uses the `RemoteStorageHelper` to create a bucket.
5867
```java
59-
RemoteGcsHelper gcsHelper =
60-
RemoteGcsHelper.create(PROJECT_ID, new FileInputStream("/path/to/my/JSON/key.json"));
61-
Storage storage = gcsHelper.options().service();
62-
String bucket = RemoteGcsHelper.generateBucketName();
68+
RemoteStorageHelper helper =
69+
RemoteStorageHelper.create(PROJECT_ID, new FileInputStream("/path/to/my/JSON/key.json"));
70+
Storage storage = helper.options().service();
71+
String bucket = RemoteStorageHelper.generateBucketName();
6372
storage.create(BucketInfo.of(bucket));
6473
```
6574

@@ -68,7 +77,7 @@ Here is an example that uses the `RemoteGcsHelper` to create a bucket.
6877
5. Clean up the test project by using `forceDelete` to clear any buckets used.
6978
Here is an example that clears the bucket created in Step 3 with a timeout of 5 seconds.
7079
```java
71-
RemoteGcsHelper.forceDelete(storage, bucket, 5, TimeUnit.SECONDS);
80+
RemoteStorageHelper.forceDelete(storage, bucket, 5, TimeUnit.SECONDS);
7281
```
7382

7483
### Testing code that uses Resource Manager
@@ -80,8 +89,6 @@ You can test against a temporary local Resource Manager by following these steps
8089
1. Before running your testing code, start the Resource Manager emulator `LocalResourceManagerHelper`. This can be done as follows:
8190

8291
```java
83-
import com.google.gcloud.resourcemanager.testing.LocalResourceManagerHelper;
84-
8592
LocalResourceManagerHelper helper = LocalResourceManagerHelper.create();
8693
helper.start();
8794
```
@@ -134,4 +141,4 @@ Here is an example that clears the dataset created in Step 3.
134141
```
135142

136143
[cloud-platform-storage-authentication]:https://cloud.google.com/storage/docs/authentication?hl=en#service_accounts
137-
[create-service-account]:https://developers.google.com/identity/protocols/OAuth2ServiceAccount#creatinganaccount
144+
[create-service-account]:https://developers.google.com/identity/protocols/OAuth2ServiceAccount#creatinganaccount

codacy-conf.json

Lines changed: 1 addition & 1 deletion
Large diffs are not rendered by default.

gcloud-java-bigquery/README.md

Lines changed: 24 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -5,12 +5,12 @@ Java idiomatic client for [Google Cloud BigQuery] (https://cloud.google.com/bigq
55

66
[![Build Status](https://travis-ci.org/GoogleCloudPlatform/gcloud-java.svg?branch=master)](https://travis-ci.org/GoogleCloudPlatform/gcloud-java)
77
[![Coverage Status](https://coveralls.io/repos/GoogleCloudPlatform/gcloud-java/badge.svg?branch=master)](https://coveralls.io/r/GoogleCloudPlatform/gcloud-java?branch=master)
8-
[![Maven](https://img.shields.io/maven-central/v/com.google.gcloud/gcloud-java-bigquery.svg)]( https://img.shields.io/maven-central/v/com.google.gcloud/gcloud-java-bigquery.svg)
8+
[![Maven](https://img.shields.io/maven-central/v/com.google.cloud/gcloud-java-bigquery.svg)]( https://img.shields.io/maven-central/v/com.google.cloud/gcloud-java-bigquery.svg)
99
[![Codacy Badge](https://api.codacy.com/project/badge/grade/9da006ad7c3a4fe1abd142e77c003917)](https://www.codacy.com/app/mziccard/gcloud-java)
1010
[![Dependency Status](https://www.versioneye.com/user/projects/56bd8ee72a29ed002d2b0969/badge.svg?style=flat)](https://www.versioneye.com/user/projects/56bd8ee72a29ed002d2b0969)
1111

1212
- [Homepage] (https://googlecloudplatform.github.io/gcloud-java/)
13-
- [API Documentation] (http://googlecloudplatform.github.io/gcloud-java/apidocs/index.html?com/google/gcloud/bigquery/package-summary.html)
13+
- [API Documentation] (http://googlecloudplatform.github.io/gcloud-java/apidocs/index.html?com/google/cloud/bigquery/package-summary.html)
1414

1515
> Note: This client is a work-in-progress, and may occasionally
1616
> make backwards-incompatible changes.
@@ -20,24 +20,24 @@ Quickstart
2020
If you are using Maven, add this to your pom.xml file
2121
```xml
2222
<dependency>
23-
<groupId>com.google.gcloud</groupId>
23+
<groupId>com.google.cloud</groupId>
2424
<artifactId>gcloud-java-bigquery</artifactId>
25-
<version>0.1.5</version>
25+
<version>0.2.0</version>
2626
</dependency>
2727
```
2828
If you are using Gradle, add this to your dependencies
2929
```Groovy
30-
compile 'com.google.gcloud:gcloud-java-bigquery:0.1.5'
30+
compile 'com.google.cloud:gcloud-java-bigquery:0.2.0'
3131
```
3232
If you are using SBT, add this to your dependencies
3333
```Scala
34-
libraryDependencies += "com.google.gcloud" % "gcloud-java-bigquery" % "0.1.5"
34+
libraryDependencies += "com.google.cloud" % "gcloud-java-bigquery" % "0.2.0"
3535
```
3636

3737
Example Application
3838
-------------------
39-
- [`BigQueryExample`](../gcloud-java-examples/src/main/java/com/google/gcloud/examples/bigquery/BigQueryExample.java) - A simple command line interface providing some of Cloud BigQuery's functionality.
40-
Read more about using this application on the [`BigQueryExample` docs page](http://googlecloudplatform.github.io/gcloud-java/apidocs/?com/google/gcloud/examples/bigquery/BigQueryExample.html).
39+
- [`BigQueryExample`](../gcloud-java-examples/src/main/java/com/google/cloud/examples/bigquery/BigQueryExample.java) - A simple command line interface providing some of Cloud BigQuery's functionality.
40+
Read more about using this application on the [`BigQueryExample` docs page](http://googlecloudplatform.github.io/gcloud-java/apidocs/?com/google/cloud/examples/bigquery/BigQueryExample.html).
4141

4242
Authentication
4343
--------------
@@ -82,8 +82,8 @@ These credentials are automatically inferred from your environment, so you only
8282
code to create your service object:
8383

8484
```java
85-
import com.google.gcloud.bigquery.BigQuery;
86-
import com.google.gcloud.bigquery.BigQueryOptions;
85+
import com.google.cloud.bigquery.BigQuery;
86+
import com.google.cloud.bigquery.BigQueryOptions;
8787

8888
BigQuery bigquery = BigQueryOptions.defaultInstance().service();
8989
```
@@ -96,7 +96,7 @@ With BigQuery you can create datasets. A dataset is a grouping mechanism that ho
9696
tables. Add the following import at the top of your file:
9797

9898
```java
99-
import com.google.gcloud.bigquery.DatasetInfo;
99+
import com.google.cloud.bigquery.DatasetInfo;
100100
```
101101
Then, to create the dataset, use the following code:
102102

@@ -113,12 +113,12 @@ are created from a BigQuery SQL query. In this code snippet we show how to creat
113113
with only one string field. Add the following imports at the top of your file:
114114

115115
```java
116-
import com.google.gcloud.bigquery.Field;
117-
import com.google.gcloud.bigquery.Schema;
118-
import com.google.gcloud.bigquery.StandardTableDefinition;
119-
import com.google.gcloud.bigquery.Table;
120-
import com.google.gcloud.bigquery.TableId;
121-
import com.google.gcloud.bigquery.TableInfo;
116+
import com.google.cloud.bigquery.Field;
117+
import com.google.cloud.bigquery.Schema;
118+
import com.google.cloud.bigquery.StandardTableDefinition;
119+
import com.google.cloud.bigquery.Table;
120+
import com.google.cloud.bigquery.TableId;
121+
import com.google.cloud.bigquery.TableInfo;
122122
```
123123
Then add the following code to create the table:
124124

@@ -139,8 +139,8 @@ Google Cloud Storage file. In this code snippet we show how to stream rows into
139139
Add the following imports at the top of your file:
140140

141141
```java
142-
import com.google.gcloud.bigquery.InsertAllRequest;
143-
import com.google.gcloud.bigquery.InsertAllResponse;
142+
import com.google.cloud.bigquery.InsertAllRequest;
143+
import com.google.cloud.bigquery.InsertAllResponse;
144144

145145
import java.util.HashMap;
146146
import java.util.Map;
@@ -171,9 +171,9 @@ directly or through a Query Job. In this code snippet we show how to run a query
171171
for the result. Add the following imports at the top of your file:
172172

173173
```java
174-
import com.google.gcloud.bigquery.FieldValue;
175-
import com.google.gcloud.bigquery.QueryRequest;
176-
import com.google.gcloud.bigquery.QueryResponse;
174+
import com.google.cloud.bigquery.FieldValue;
175+
import com.google.cloud.bigquery.QueryRequest;
176+
import com.google.cloud.bigquery.QueryResponse;
177177

178178
import java.util.Iterator;
179179
import java.util.List;
@@ -203,7 +203,7 @@ while (rowIterator.hasNext()) {
203203
#### Complete source code
204204

205205
In
206-
[InsertDataAndQueryTable.java](../gcloud-java-examples/src/main/java/com/google/gcloud/examples/bigquery/snippets/InsertDataAndQueryTable.java)
206+
[InsertDataAndQueryTable.java](../gcloud-java-examples/src/main/java/com/google/cloud/examples/bigquery/snippets/InsertDataAndQueryTable.java)
207207
we put together all the code shown above into one program. The program assumes that you are
208208
running on Compute Engine or from your own desktop. To run the example on App Engine, simply move
209209
the code from the main method to your application's servlet class and change the print statements to
@@ -255,4 +255,4 @@ Apache 2.0 - See [LICENSE] for more information.
255255

256256
[cloud-bigquery]: https://cloud.google.com/bigquery/
257257
[cloud-storage]: https://cloud.google.com/storage/
258-
[bigquery-api]: http://googlecloudplatform.github.io/gcloud-java/apidocs/index.html?com/google/gcloud/bigquery/package-summary.html
258+
[bigquery-api]: http://googlecloudplatform.github.io/gcloud-java/apidocs/index.html?com/google/cloud/bigquery/package-summary.html

gcloud-java-bigquery/pom.xml

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -4,13 +4,14 @@
44
<artifactId>gcloud-java-bigquery</artifactId>
55
<packaging>jar</packaging>
66
<name>GCloud Java bigquery</name>
7+
<url>https://github.com/GoogleCloudPlatform/gcloud-java/tree/master/gcloud-java-bigquery</url>
78
<description>
89
Java idiomatic client for Google Cloud BigQuery.
910
</description>
1011
<parent>
11-
<groupId>com.google.gcloud</groupId>
12+
<groupId>com.google.cloud</groupId>
1213
<artifactId>gcloud-java-pom</artifactId>
13-
<version>0.1.6-SNAPSHOT</version>
14+
<version>0.2.1-SNAPSHOT</version>
1415
</parent>
1516
<properties>
1617
<site.installationModule>gcloud-java-bigquery</site.installationModule>

gcloud-java-bigquery/src/main/java/com/google/gcloud/bigquery/Acl.java renamed to gcloud-java-bigquery/src/main/java/com/google/cloud/bigquery/Acl.java

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@
1414
* limitations under the License.
1515
*/
1616

17-
package com.google.gcloud.bigquery;
17+
package com.google.cloud.bigquery;
1818

1919
import static com.google.common.base.Preconditions.checkNotNull;
2020

0 commit comments

Comments
 (0)