Skip to content

Commit a17c1ff

Browse files
jartjart
authored
Merge pull request #1055 from jart/merge-master-into-gcs-nio
Merge master into gcs-nio
2 parents ed440e9 + 6c47760 commit a17c1ff

File tree

254 files changed

+45908
-1807
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

254 files changed

+45908
-1807
lines changed

‎.travis.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@ before_install:
99
- cp target/travis/settings.xml ~/.m2/settings.xml
1010
install: mvn install -DskipTests=true -Dgpg.skip=true
1111
script:
12-
- utilities/verify.sh
12+
- travis_wait 30 utilities/verify.sh
1313
after_success:
1414
- utilities/after_success.sh
1515
env:

‎README.md‎

Lines changed: 87 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,7 @@ Java idiomatic client for [Google Cloud Platform][cloud-platform] services.
1515
This client supports the following Google Cloud Platform services:
1616

1717
- [Google Cloud BigQuery] (#google-cloud-bigquery-alpha) (Alpha)
18+
- [Google Cloud Compute] (#google-cloud-compute-alpha) (Alpha)
1819
- [Google Cloud Datastore] (#google-cloud-datastore)
1920
- [Google Cloud DNS] (#google-cloud-dns-alpha) (Alpha)
2021
- [Google Cloud Resource Manager] (#google-cloud-resource-manager-alpha) (Alpha)
@@ -33,23 +34,25 @@ If you are using Maven, add this to your pom.xml file
3334
<dependency>
3435
<groupId>com.google.cloud</groupId>
3536
<artifactId>gcloud-java</artifactId>
36-
<version>0.2.0</version>
37+
<version>0.2.3</version>
3738
</dependency>
3839
```
3940
If you are using Gradle, add this to your dependencies
4041
```Groovy
41-
compile 'com.google.cloud:gcloud-java:0.2.0'
42+
compile 'com.google.cloud:gcloud-java:0.2.3'
4243
```
4344
If you are using SBT, add this to your dependencies
4445
```Scala
45-
libraryDependencies += "com.google.cloud" % "gcloud-java" % "0.2.0"
46+
libraryDependencies += "com.google.cloud" % "gcloud-java" % "0.2.3"
4647
```
4748

4849
Example Applications
4950
--------------------
5051

5152
- [`BigQueryExample`](./gcloud-java-examples/src/main/java/com/google/cloud/examples/bigquery/BigQueryExample.java) - A simple command line interface providing some of Cloud BigQuery's functionality
5253
- Read more about using this application on the [`BigQueryExample` docs page](http://googlecloudplatform.github.io/gcloud-java/apidocs/?com/google/cloud/examples/bigquery/BigQueryExample.html).
54+
- [`ComputeExample`](./gcloud-java-examples/src/main/java/com/google/cloud/examples/compute/ComputeExample.java) - A simple command line interface providing some of Cloud Compute's functionality
55+
- Read more about using this application on the [`gcloud-java-examples` docs page](http://googlecloudplatform.github.io/gcloud-java/apidocs/?com/google/cloud/examples/compute/ComputeExample.html).
5356
- [`Bookshelf`](https://github.com/GoogleCloudPlatform/getting-started-java/tree/master/bookshelf) - An App Engine app that manages a virtual bookshelf.
5457
- This app uses `gcloud-java` to interface with Cloud Datastore and Cloud Storage. It also uses Cloud SQL, another Google Cloud Platform service.
5558
- [`DatastoreExample`](./gcloud-java-examples/src/main/java/com/google/cloud/examples/datastore/DatastoreExample.java) - A simple command line interface for Cloud Datastore
@@ -104,7 +107,7 @@ First, ensure that the necessary Google Cloud APIs are enabled for your project.
104107
Next, choose a method for authenticating API requests from within your project:
105108

106109
1. When using `gcloud-java` libraries from within Compute/App Engine, no additional authentication steps are necessary.
107-
2. When using `gcloud-java` libraries elsewhere, there are two options:
110+
2. When using `gcloud-java` libraries elsewhere, there are three options:
108111
* [Generate a JSON service account key](https://cloud.google.com/storage/docs/authentication?hl=en#service_accounts). After downloading that key, you must do one of the following:
109112
* Define the environment variable GOOGLE_APPLICATION_CREDENTIALS to be the location of the key. For example:
110113
```bash
@@ -113,11 +116,18 @@ Next, choose a method for authenticating API requests from within your project:
113116
* Supply the JSON credentials file when building the service options. For example, this Storage object has the necessary permissions to interact with your Google Cloud Storage data:
114117
```java
115118
Storage storage = StorageOptions.builder()
116-
.authCredentials(AuthCredentials.createForJson(new FileInputStream("/path/to/my/key.json"))
119+
.authCredentials(AuthCredentials.createForJson(new FileInputStream("/path/to/my/key.json"))
120+
.build()
121+
.service();
122+
```
123+
* If running locally for development/testing, you can use Google Cloud SDK. Download the SDK if you haven't already, then login using the SDK (`gcloud auth login` in command line). Be sure to set your project ID as described above.
124+
* If you already have an OAuth2 access token, you can use it to authenticate (notice that in this case the access token will not be automatically refreshed):
125+
```java
126+
Storage storage = StorageOptions.builder()
127+
.authCredentials(AuthCredentials.createFor("your_access_token"))
117128
.build()
118129
.service();
119-
```
120-
* If running locally for development/testing, you can use use Google Cloud SDK. Download the SDK if you haven't already, then login using the SDK (`gcloud auth login` in command line). Be sure to set your project ID as described above.
130+
```
121131
122132
`gcloud-java` looks for credentials in the following order, stopping once it finds credentials:
123133
@@ -163,16 +173,79 @@ if (table == null) {
163173
}
164174
System.out.println("Loading data into table " + tableId);
165175
Job loadJob = table.load(FormatOptions.csv(), "gs://bucket/path");
166-
while (!loadJob.isDone()) {
167-
Thread.sleep(1000L);
168-
}
176+
loadJob = loadJob.waitFor();
169177
if (loadJob.status().error() != null) {
170178
System.out.println("Job completed with errors");
171179
} else {
172180
System.out.println("Job succeeded");
173181
}
174182
```
175183
184+
Google Cloud Compute (Alpha)
185+
----------------------
186+
187+
- [API Documentation][compute-api]
188+
- [Official Documentation][cloud-compute-docs]
189+
190+
#### Preview
191+
192+
Here are two code snippets showing simple usage examples from within Compute/App Engine. Note that
193+
you must [supply credentials](#authentication) and a project ID if running this snippet elsewhere.
194+
195+
The first snippet shows how to create a snapshot from an existing disk. Complete source code can be
196+
found at
197+
[CreateSnapshot.java](./gcloud-java-examples/src/main/java/com/google/cloud/examples/compute/snippets/CreateSnapshot.java).
198+
199+
```java
200+
import com.google.cloud.compute.Compute;
201+
import com.google.cloud.compute.ComputeOptions;
202+
import com.google.cloud.compute.Disk;
203+
import com.google.cloud.compute.DiskId;
204+
import com.google.cloud.compute.Snapshot;
205+
206+
Compute compute = ComputeOptions.defaultInstance().service();
207+
DiskId diskId = DiskId.of("us-central1-a", "disk-name");
208+
Disk disk = compute.getDisk(diskId, Compute.DiskOption.fields());
209+
if (disk != null) {
210+
String snapshotName = "disk-name-snapshot";
211+
Operation operation = disk.createSnapshot(snapshotName);
212+
operation = operation.waitFor();
213+
if (operation.errors() == null) {
214+
// use snapshot
215+
Snapshot snapshot = compute.getSnapshot(snapshotName);
216+
}
217+
}
218+
```
219+
The second snippet shows how to create a virtual machine instance. Complete source code can be found
220+
at
221+
[CreateInstance.java](./gcloud-java-examples/src/main/java/com/google/cloud/examples/compute/snippets/CreateInstance.java).
222+
```java
223+
import com.google.cloud.compute.AttachedDisk;
224+
import com.google.cloud.compute.Compute;
225+
import com.google.cloud.compute.ComputeOptions;
226+
import com.google.cloud.compute.ImageId;
227+
import com.google.cloud.compute.Instance;
228+
import com.google.cloud.compute.InstanceId;
229+
import com.google.cloud.compute.InstanceInfo;
230+
import com.google.cloud.compute.MachineTypeId;
231+
import com.google.cloud.compute.NetworkId;
232+
233+
Compute compute = ComputeOptions.defaultInstance().service();
234+
ImageId imageId = ImageId.of("debian-cloud", "debian-8-jessie-v20160329");
235+
NetworkId networkId = NetworkId.of("default");
236+
AttachedDisk attachedDisk = AttachedDisk.of(AttachedDisk.CreateDiskConfiguration.of(imageId));
237+
NetworkInterface networkInterface = NetworkInterface.of(networkId);
238+
InstanceId instanceId = InstanceId.of("us-central1-a", "instance-name");
239+
MachineTypeId machineTypeId = MachineTypeId.of("us-central1-a", "n1-standard-1");
240+
Operation operation =
241+
compute.create(InstanceInfo.of(instanceId, machineTypeId, attachedDisk, networkInterface));
242+
operation = operation.waitFor();
243+
if (operation.errors() == null) {
244+
// use instance
245+
Instance instance = compute.getInstance(instanceId);
246+
}
247+
```
248+
176249
Google Cloud Datastore
177250
----------------------
178251
@@ -455,3 +528,7 @@ Apache 2.0 - See [LICENSE] for more information.
455528
[cloud-bigquery]: https://cloud.google.com/bigquery/
456529
[cloud-bigquery-docs]: https://cloud.google.com/bigquery/docs/overview
457530
[bigquery-api]: http://googlecloudplatform.github.io/gcloud-java/apidocs/index.html?com/google/cloud/bigquery/package-summary.html
531+
532+
[cloud-compute]: https://cloud.google.com/compute/
533+
[cloud-compute-docs]: https://cloud.google.com/compute/docs/overview
534+
[compute-api]: http://googlecloudplatform.github.io/gcloud-java/apidocs/index.html?com/google/cloud/compute/package-summary.html

‎gcloud-java-bigquery/README.md‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -22,16 +22,16 @@ If you are using Maven, add this to your pom.xml file
2222
<dependency>
2323
<groupId>com.google.cloud</groupId>
2424
<artifactId>gcloud-java-bigquery</artifactId>
25-
<version>0.2.0</version>
25+
<version>0.2.3</version>
2626
</dependency>
2727
```
2828
If you are using Gradle, add this to your dependencies
2929
```Groovy
30-
compile 'com.google.cloud:gcloud-java-bigquery:0.2.0'
30+
compile 'com.google.cloud:gcloud-java-bigquery:0.2.3'
3131
```
3232
If you are using SBT, add this to your dependencies
3333
```Scala
34-
libraryDependencies += "com.google.cloud" % "gcloud-java-bigquery" % "0.2.0"
34+
libraryDependencies += "com.google.cloud" % "gcloud-java-bigquery" % "0.2.3"
3535
```
3636

3737
Example Application

‎gcloud-java-bigquery/pom.xml‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@
1111
<parent>
1212
<groupId>com.google.cloud</groupId>
1313
<artifactId>gcloud-java-pom</artifactId>
14-
<version>0.2.1-SNAPSHOT</version>
14+
<version>0.2.4-SNAPSHOT</version>
1515
</parent>
1616
<properties>
1717
<site.installationModule>gcloud-java-bigquery</site.installationModule>
@@ -31,7 +31,7 @@
3131
<dependency>
3232
<groupId>com.google.apis</groupId>
3333
<artifactId>google-api-services-bigquery</artifactId>
34-
<version>v2-rev270-1.21.0</version>
34+
<version>v2-rev303-1.22.0</version>
3535
<scope>compile</scope>
3636
<exclusions>
3737
<exclusion>

‎gcloud-java-bigquery/src/main/java/com/google/cloud/bigquery/BigQueryException.java‎

Lines changed: 8 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -31,7 +31,7 @@
3131
* @see <a href="https://cloud.google.com/bigquery/troubleshooting-errors">Google Cloud
3232
* BigQuery error codes</a>
3333
*/
34-
public class BigQueryException extends BaseServiceException {
34+
public final class BigQueryException extends BaseServiceException {
3535

3636
// see: https://cloud.google.com/bigquery/troubleshooting-errors
3737
private static final Set<Error> RETRYABLE_ERRORS = ImmutableSet.of(
@@ -44,7 +44,12 @@ public class BigQueryException extends BaseServiceException {
4444
private final BigQueryError error;
4545

4646
public BigQueryException(int code, String message) {
47-
this(code, message, null);
47+
this(code, message, (Throwable) null);
48+
}
49+
50+
public BigQueryException(int code, String message, Throwable cause) {
51+
super(code, message, null, true, cause);
52+
this.error = null;
4853
}
4954

5055
public BigQueryException(int code, String message, BigQueryError error) {
@@ -100,6 +105,6 @@ public int hashCode() {
100105
*/
101106
static BaseServiceException translateAndThrow(RetryHelperException ex) {
102107
BaseServiceException.translateAndPropagateIfPossible(ex);
103-
throw new BigQueryException(UNKNOWN_CODE, ex.getMessage());
108+
throw new BigQueryException(UNKNOWN_CODE, ex.getMessage(), ex.getCause());
104109
}
105110
}

‎gcloud-java-bigquery/src/main/java/com/google/cloud/bigquery/BigQueryImpl.java‎

Lines changed: 17 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -164,7 +164,7 @@ public Dataset create(DatasetInfo dataset, DatasetOption... options) {
164164
public com.google.api.services.bigquery.model.Dataset call() {
165165
return bigQueryRpc.create(datasetPb, optionsMap);
166166
}
167-
}, options().retryParams(), EXCEPTION_HANDLER));
167+
}, options().retryParams(), EXCEPTION_HANDLER, options().clock()));
168168
} catch (RetryHelper.RetryHelperException e) {
169169
throw BigQueryException.translateAndThrow(e);
170170
}
@@ -182,7 +182,7 @@ public Table create(TableInfo table, TableOption... options) {
182182
public com.google.api.services.bigquery.model.Table call() {
183183
return bigQueryRpc.create(tablePb, optionsMap);
184184
}
185-
}, options().retryParams(), EXCEPTION_HANDLER));
185+
}, options().retryParams(), EXCEPTION_HANDLER, options().clock()));
186186
} catch (RetryHelper.RetryHelperException e) {
187187
throw BigQueryException.translateAndThrow(e);
188188
}
@@ -200,7 +200,7 @@ public Job create(JobInfo job, JobOption... options) {
200200
public com.google.api.services.bigquery.model.Job call() {
201201
return bigQueryRpc.create(jobPb, optionsMap);
202202
}
203-
}, options().retryParams(), EXCEPTION_HANDLER));
203+
}, options().retryParams(), EXCEPTION_HANDLER, options().clock()));
204204
} catch (RetryHelper.RetryHelperException e) {
205205
throw BigQueryException.translateAndThrow(e);
206206
}
@@ -221,7 +221,7 @@ public Dataset getDataset(final DatasetId datasetId, DatasetOption... options) {
221221
public com.google.api.services.bigquery.model.Dataset call() {
222222
return bigQueryRpc.getDataset(datasetId.dataset(), optionsMap);
223223
}
224-
}, options().retryParams(), EXCEPTION_HANDLER);
224+
}, options().retryParams(), EXCEPTION_HANDLER, options().clock());
225225
return answer == null ? null : Dataset.fromPb(this, answer);
226226
} catch (RetryHelper.RetryHelperException e) {
227227
throw BigQueryException.translateAndThrow(e);
@@ -244,7 +244,7 @@ private static Page<Dataset> listDatasets(final BigQueryOptions serviceOptions,
244244
Iterable<com.google.api.services.bigquery.model.Dataset>> call() {
245245
return serviceOptions.rpc().listDatasets(optionsMap);
246246
}
247-
}, serviceOptions.retryParams(), EXCEPTION_HANDLER);
247+
}, serviceOptions.retryParams(), EXCEPTION_HANDLER, serviceOptions.clock());
248248
String cursor = result.x();
249249
return new PageImpl<>(new DatasetPageFetcher(serviceOptions, cursor, optionsMap), cursor,
250250
Iterables.transform(result.y(),
@@ -273,7 +273,7 @@ public boolean delete(final DatasetId datasetId, DatasetDeleteOption... options)
273273
public Boolean call() {
274274
return bigQueryRpc.deleteDataset(datasetId.dataset(), optionsMap);
275275
}
276-
}, options().retryParams(), EXCEPTION_HANDLER);
276+
}, options().retryParams(), EXCEPTION_HANDLER, options().clock());
277277
} catch (RetryHelper.RetryHelperException e) {
278278
throw BigQueryException.translateAndThrow(e);
279279
}
@@ -292,7 +292,7 @@ public boolean delete(final TableId tableId) {
292292
public Boolean call() {
293293
return bigQueryRpc.deleteTable(tableId.dataset(), tableId.table());
294294
}
295-
}, options().retryParams(), EXCEPTION_HANDLER);
295+
}, options().retryParams(), EXCEPTION_HANDLER, options().clock());
296296
} catch (RetryHelper.RetryHelperException e) {
297297
throw BigQueryException.translateAndThrow(e);
298298
}
@@ -310,7 +310,7 @@ public Dataset update(DatasetInfo dataset, DatasetOption... options) {
310310
public com.google.api.services.bigquery.model.Dataset call() {
311311
return bigQueryRpc.patch(datasetPb, optionsMap);
312312
}
313-
}, options().retryParams(), EXCEPTION_HANDLER));
313+
}, options().retryParams(), EXCEPTION_HANDLER, options().clock()));
314314
} catch (RetryHelper.RetryHelperException e) {
315315
throw BigQueryException.translateAndThrow(e);
316316
}
@@ -328,7 +328,7 @@ public Table update(TableInfo table, TableOption... options) {
328328
public com.google.api.services.bigquery.model.Table call() {
329329
return bigQueryRpc.patch(tablePb, optionsMap);
330330
}
331-
}, options().retryParams(), EXCEPTION_HANDLER));
331+
}, options().retryParams(), EXCEPTION_HANDLER, options().clock()));
332332
} catch (RetryHelper.RetryHelperException e) {
333333
throw BigQueryException.translateAndThrow(e);
334334
}
@@ -349,7 +349,7 @@ public Table getTable(final TableId tableId, TableOption... options) {
349349
public com.google.api.services.bigquery.model.Table call() {
350350
return bigQueryRpc.getTable(tableId.dataset(), tableId.table(), optionsMap);
351351
}
352-
}, options().retryParams(), EXCEPTION_HANDLER);
352+
}, options().retryParams(), EXCEPTION_HANDLER, options().clock());
353353
return answer == null ? null : Table.fromPb(this, answer);
354354
} catch (RetryHelper.RetryHelperException e) {
355355
throw BigQueryException.translateAndThrow(e);
@@ -377,7 +377,7 @@ private static Page<Table> listTables(final String datasetId, final BigQueryOpti
377377
call() {
378378
return serviceOptions.rpc().listTables(datasetId, optionsMap);
379379
}
380-
}, serviceOptions.retryParams(), EXCEPTION_HANDLER);
380+
}, serviceOptions.retryParams(), EXCEPTION_HANDLER, serviceOptions.clock());
381381
String cursor = result.x();
382382
Iterable<Table> tables = Iterables.transform(result.y(),
383383
new Function<com.google.api.services.bigquery.model.Table, Table>() {
@@ -432,7 +432,7 @@ public BigQueryRpc.Tuple<String, Iterable<TableRow>> call() {
432432
return serviceOptions.rpc()
433433
.listTableData(tableId.dataset(), tableId.table(), optionsMap);
434434
}
435-
}, serviceOptions.retryParams(), EXCEPTION_HANDLER);
435+
}, serviceOptions.retryParams(), EXCEPTION_HANDLER, serviceOptions.clock());
436436
String cursor = result.x();
437437
return new PageImpl<>(new TableDataPageFetcher(tableId, serviceOptions, cursor, optionsMap),
438438
cursor, transformTableData(result.y()));
@@ -467,7 +467,7 @@ public Job getJob(final JobId jobId, JobOption... options) {
467467
public com.google.api.services.bigquery.model.Job call() {
468468
return bigQueryRpc.getJob(jobId.job(), optionsMap);
469469
}
470-
}, options().retryParams(), EXCEPTION_HANDLER);
470+
}, options().retryParams(), EXCEPTION_HANDLER, options().clock());
471471
return answer == null ? null : Job.fromPb(this, answer);
472472
} catch (RetryHelper.RetryHelperException e) {
473473
throw BigQueryException.translateAndThrow(e);
@@ -489,7 +489,7 @@ private static Page<Job> listJobs(final BigQueryOptions serviceOptions,
489489
call() {
490490
return serviceOptions.rpc().listJobs(optionsMap);
491491
}
492-
}, serviceOptions.retryParams(), EXCEPTION_HANDLER);
492+
}, serviceOptions.retryParams(), EXCEPTION_HANDLER, serviceOptions.clock());
493493
String cursor = result.x();
494494
Iterable<Job> jobs = Iterables.transform(result.y(),
495495
new Function<com.google.api.services.bigquery.model.Job, Job>() {
@@ -514,7 +514,7 @@ public boolean cancel(final JobId jobId) {
514514
public Boolean call() {
515515
return bigQueryRpc.cancel(jobId.job());
516516
}
517-
}, options().retryParams(), EXCEPTION_HANDLER);
517+
}, options().retryParams(), EXCEPTION_HANDLER, options().clock());
518518
} catch (RetryHelper.RetryHelperException e) {
519519
throw BigQueryException.translateAndThrow(e);
520520
}
@@ -529,7 +529,7 @@ public QueryResponse query(final QueryRequest request) {
529529
public com.google.api.services.bigquery.model.QueryResponse call() {
530530
return bigQueryRpc.query(request.setProjectId(options().projectId()).toPb());
531531
}
532-
}, options().retryParams(), EXCEPTION_HANDLER);
532+
}, options().retryParams(), EXCEPTION_HANDLER, options().clock());
533533
QueryResponse.Builder builder = QueryResponse.builder();
534534
JobId completeJobId = JobId.fromPb(results.getJobReference());
535535
builder.jobId(completeJobId);
@@ -574,7 +574,7 @@ private static QueryResponse getQueryResults(final JobId jobId,
574574
public GetQueryResultsResponse call() {
575575
return serviceOptions.rpc().getQueryResults(jobId.job(), optionsMap);
576576
}
577-
}, serviceOptions.retryParams(), EXCEPTION_HANDLER);
577+
}, serviceOptions.retryParams(), EXCEPTION_HANDLER, serviceOptions.clock());
578578
QueryResponse.Builder builder = QueryResponse.builder();
579579
JobId completeJobId = JobId.fromPb(results.getJobReference());
580580
builder.jobId(completeJobId);

0 commit comments

Comments
 (0)