Skip to content

Commit f19440d

Browse files
pongadpongad
authored
merge master into pubsub-hp (#1591)
* Update version to 0.8.1-SNAPSHOT (#1467) Also, update versions in README to 0.8.0 * Add link to Maven Central for maven-central badge. (#1468) Used to link to the image, which wasn't super useful. * fix more races in pubsub tests Previously BlockingProcessStreamReader has a terminate() method, used to tell the Reader to stop reading from the emulator process. This causes an inter-process race. If the Reader stops before reading emulator's output, the emulator process will hang as it tries to write to stdout/stderr as there's no one to read from the other side of the pipe. Since there is no way to safely stop the Reader, this commit deletes the method and its associated test. Additionally, the timeout for LocalSystemTest is increased to 3 minutes, since the emulator, somehow, consistently takes just longer than a minute to shut down. * Regenerating SPI layer (#1501) * Converting Error Reporting and Monitoring to use resource name types * Removing formatX/parseX methods from pubsub, converting usage of the same to resource name types * New methods in Logging and PubSub * Updating grpc dependency to 1.0.3 (#1504) * Release 0.8.1 (#1512) * Fix code snippet (wrong method name) in README.md Original code snippet in _"Querying data"_ section: `..while (!queryResponse.jobComplete()) {..` This results in a compile error: _"Cannot resolve method jobComplete()"_ The correct method is `jobCompleted()` * Updating version in README files. [ci skip] * Update version to 0.8.2-alpha-SNAPSHOT * Allow path in URIs passed to newFileSystem (#1470) * This makes it easier for users who start with a URI describing a full path to get a FileSystem that can work with that path, since they no longer have to needlessly remove the path from the URI. Note that Oracle's description of newFileSystem [1] puts no restriction on the passed URI. [1] https://docs.oracle.com/javase/7/docs/api/java/nio/file/FileSystems.html#newFileSystem(java.net.URI,%20java.util.Map) * Preventing logging re-entrance at FINE level (#1523) * Preventing logging re-entrance at FINE level Also: * Reducing the scope of synchronized blocks * Removing logger exclusions except for Http2FrameLogger * Add a PathMatcher for CloudStorageFileSystem (#1469) Add a test, as well. We reuse the default PathMatcher since it does a fine job of globbing files. * Set timestamp from LogRecord (#1533) * Initialize the default MonitoredResource from a GAE environment (#1535) * BigQuery: Add support to FormatOptions for AVRO #1441 Added new constant in FormatOptions, and a corresponding factory method. Updated test cases. Confirmed that AVRO does not require special treatment (like CSV does), so no additional changes are required. * Reverting changed commited by mistake before review. * BigQuery: Add support to FormatOptions for AVRO #1441 Added new constant in FormatOptions and a corresponding factory method. Updated test cases. Confirmed that AVRO does not require special treatment (like CSV does), so no additional changes are required. * use RpcFuture and remove old BundlingSettings (#1572) * use RpcFuture and remove old BundlingSettings * Release 0.8.2 Note: This version was accidentally released to Sonatype because of experimenting with deployment commands, combined with the fact that autoReleaseAfterClose is set to true. Since releases can't be taken back, we might as well own up to the release and push the code forward. * Updating version in README files. [ci skip] * Fixing javadoc error in GaeFlexLoggingEnhancer (#1582) * get tests to compile and pass
1 parent 90d442b commit f19440d

File tree

8 files changed

+46
-17
lines changed

8 files changed

+46
-17
lines changed

‎java-storage-nio/google-cloud-nio-examples/README.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -22,12 +22,12 @@ To run this example:
2222
4. Run the sample with:
2323
2424
```
25-
java -cp google-cloud-contrib/google-cloud-nio/target/google-cloud-nio-0.7.1-SNAPSHOT-shaded.jar:google-cloud-contrib/google-cloud-nio-examples/target/google-cloud-nio-examples-0.7.1-SNAPSHOT.jar com.google.cloud.nio.examples.ListFilesystems
25+
java -cp google-cloud-contrib/google-cloud-nio/target/google-cloud-nio-0.8.3-alpha-SNAPSHOT-shaded.jar:google-cloud-contrib/google-cloud-nio-examples/target/google-cloud-nio-examples-0.8.3-alpha-SNAPSHOT.jar com.google.cloud.nio.examples.ListFilesystems
2626
```
2727
2828
Notice that it lists Google Cloud Storage, which it wouldn't if you ran it without the NIO jar:
2929
```
30-
java -cp google-cloud-contrib/google-cloud-nio-examples/target/google-cloud-nio-examples-0.7.1-SNAPSHOT.jar com.google.cloud.nio.examples.ListFilesystems
30+
java -cp google-cloud-contrib/google-cloud-nio-examples/target/google-cloud-nio-examples-0.8.3-alpha-SNAPSHOT.jar com.google.cloud.nio.examples.ListFilesystems
3131
```
3232
3333
The sample doesn't have anything about Google Cloud Storage in it. It gets that ability from the NIO

‎java-storage-nio/google-cloud-nio-examples/pom.xml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@
1111
<parent>
1212
<groupId>com.google.cloud</groupId>
1313
<artifactId>google-cloud-contrib</artifactId>
14-
<version>0.8.1-SNAPSHOT</version>
14+
<version>0.8.2-alpha</version>
1515
</parent>
1616
<properties>
1717
<site.installationModule>google-cloud-nio-examples</site.installationModule>

‎java-storage-nio/google-cloud-nio/README.md‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -26,16 +26,16 @@ If you are using Maven, add this to your pom.xml file
2626
<dependency>
2727
<groupId>com.google.cloud</groupId>
2828
<artifactId>google-cloud-nio</artifactId>
29-
<version>0.8.0</version>
29+
<version>0.8.2-alpha</version>
3030
</dependency>
3131
```
3232
If you are using Gradle, add this to your dependencies
3333
```Groovy
34-
compile 'com.google.cloud:google-cloud-nio:0.8.0'
34+
compile 'com.google.cloud:google-cloud-nio:0.8.2-alpha'
3535
```
3636
If you are using SBT, add this to your dependencies
3737
```Scala
38-
libraryDependencies += "com.google.cloud" % "google-cloud-nio" % "0.8.0"
38+
libraryDependencies += "com.google.cloud" % "google-cloud-nio" % "0.8.2-alpha"
3939
```
4040

4141
Example Applications

‎java-storage-nio/google-cloud-nio/pom.xml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@
1111
<parent>
1212
<groupId>com.google.cloud</groupId>
1313
<artifactId>google-cloud-contrib</artifactId>
14-
<version>0.8.1-SNAPSHOT</version>
14+
<version>0.8.2-alpha</version>
1515
</parent>
1616
<properties>
1717
<site.installationModule>google-cloud-nio</site.installationModule>

‎java-storage-nio/google-cloud-nio/src/main/java/com/google/cloud/storage/contrib/nio/CloudStorageFileSystem.java‎

Lines changed: 2 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -27,6 +27,7 @@
2727
import java.net.URISyntaxException;
2828
import java.nio.file.FileStore;
2929
import java.nio.file.FileSystem;
30+
import java.nio.file.FileSystems;
3031
import java.nio.file.Path;
3132
import java.nio.file.PathMatcher;
3233
import java.nio.file.WatchService;
@@ -210,13 +211,9 @@ public Set<String> supportedFileAttributeViews() {
210211
return SUPPORTED_VIEWS;
211212
}
212213

213-
/**
214-
* Throws {@link UnsupportedOperationException} because this feature hasn't been implemented yet.
215-
*/
216214
@Override
217215
public PathMatcher getPathMatcher(String syntaxAndPattern) {
218-
// TODO(#813): Implement me.
219-
throw new UnsupportedOperationException();
216+
return FileSystems.getDefault().getPathMatcher(syntaxAndPattern);
220217
}
221218

222219
/**

‎java-storage-nio/google-cloud-nio/src/main/java/com/google/cloud/storage/contrib/nio/CloudStorageFileSystemProvider.java‎

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -170,14 +170,15 @@ public CloudStorageFileSystem getFileSystem(URI uri) {
170170
}
171171

172172
/**
173-
* Returns Cloud Storage file system, provided a URI with no path, e.g. {@code gs://bucket}.
173+
* Returns Cloud Storage file system, provided a URI, e.g. {@code gs://bucket}.
174+
* The URI can include a path component (that will be ignored).
174175
*
175176
* @param uri bucket and current working directory, e.g. {@code gs://bucket}
176177
* @param env map of configuration options, whose keys correspond to the method names of
177178
* {@link CloudStorageConfiguration.Builder}. However you are not allowed to set the working
178179
* directory, as that should be provided in the {@code uri}
179-
* @throws IllegalArgumentException if {@code uri} specifies a user, query, fragment, or scheme is
180-
* not {@value CloudStorageFileSystem#URI_SCHEME}
180+
* @throws IllegalArgumentException if {@code uri} specifies a port, user, query, or fragment, or
181+
* if scheme is not {@value CloudStorageFileSystem#URI_SCHEME}
181182
*/
182183
@Override
183184
public CloudStorageFileSystem newFileSystem(URI uri, Map<String, ?> env) {
@@ -191,11 +192,10 @@ public CloudStorageFileSystem newFileSystem(URI uri, Map<String, ?> env) {
191192
CloudStorageFileSystem.URI_SCHEME, uri);
192193
checkArgument(
193194
uri.getPort() == -1
194-
&& isNullOrEmpty(uri.getPath())
195195
&& isNullOrEmpty(uri.getQuery())
196196
&& isNullOrEmpty(uri.getFragment())
197197
&& isNullOrEmpty(uri.getUserInfo()),
198-
"GCS FileSystem URIs mustn't have: port, userinfo, path, query, or fragment: %s",
198+
"GCS FileSystem URIs mustn't have: port, userinfo, query, or fragment: %s",
199199
uri);
200200
CloudStorageUtil.checkBucket(uri.getHost());
201201
initStorage();

‎java-storage-nio/google-cloud-nio/src/test/java/com/google/cloud/storage/contrib/nio/CloudStorageFileSystemProviderTest.java‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -54,7 +54,9 @@
5454
import java.nio.file.OpenOption;
5555
import java.nio.file.Path;
5656
import java.nio.file.Paths;
57+
import java.util.HashMap;
5758
import java.util.List;
59+
import java.util.Map;
5860

5961
/**
6062
* Unit tests for {@link CloudStorageFileSystemProvider}.
@@ -644,6 +646,12 @@ public void testProviderEquals() {
644646
assertThat(path1.getFileSystem().provider()).isNotEqualTo(path3.getFileSystem().provider());
645647
}
646648

649+
@Test
650+
public void testNewFileSystem() throws IOException {
651+
Map<String,String> env = new HashMap<>();
652+
FileSystems.newFileSystem(URI.create("gs://bucket/path/to/file"), env);
653+
}
654+
647655
private static CloudStorageConfiguration permitEmptyPathComponents(boolean value) {
648656
return CloudStorageConfiguration.builder().permitEmptyPathComponents(value).build();
649657
}

‎java-storage-nio/google-cloud-nio/src/test/java/com/google/cloud/storage/contrib/nio/CloudStorageFileSystemTest.java‎

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -34,6 +34,7 @@
3434
import java.nio.file.FileSystems;
3535
import java.nio.file.Files;
3636
import java.nio.file.Path;
37+
import java.nio.file.PathMatcher;
3738
import java.nio.file.Paths;
3839
import java.util.ArrayList;
3940
import java.util.List;
@@ -159,4 +160,27 @@ public void testListFiles() throws IOException {
159160
assertThat(got).containsExactlyElementsIn(goodPaths);
160161
}
161162
}
163+
164+
@Test
165+
public void testMatcher() throws IOException {
166+
try (FileSystem fs = CloudStorageFileSystem.forBucket("bucket")) {
167+
String pattern1 = "glob:*.java";
168+
PathMatcher javaFileMatcher = fs.getPathMatcher(pattern1);
169+
assertMatches(fs, javaFileMatcher, "a.java", true);
170+
assertMatches(fs, javaFileMatcher, "a.text", false);
171+
assertMatches(fs, javaFileMatcher, "folder/c.java", true);
172+
assertMatches(fs, javaFileMatcher, "d", false);
173+
174+
String pattern2 = "glob:*.{java,text}";
175+
PathMatcher javaAndTextFileMatcher = fs.getPathMatcher(pattern2);
176+
assertMatches(fs, javaAndTextFileMatcher, "a.java", true);
177+
assertMatches(fs, javaAndTextFileMatcher, "a.text", true);
178+
assertMatches(fs, javaAndTextFileMatcher, "folder/c.java", true);
179+
assertMatches(fs, javaAndTextFileMatcher, "d", false);
180+
}
181+
}
182+
183+
private void assertMatches(FileSystem fs, PathMatcher matcher, String toMatch, boolean expected) {
184+
assertThat(matcher.matches(fs.getPath(toMatch).getFileName())).isEqualTo(expected);
185+
}
162186
}

0 commit comments

Comments
 (0)