Skip to content

Commit 880269b

Browse files
chesches
authored
Enforce JDK 11 for development, but build Ingestion to target Java 8 (#518)
* Require Java 11 with Enforcer We build for Java 11 now, so the build will fail with older JDKs. Have the Enforcer plugin do that for a more lucid error message. This reverts 190e605 which was squash-merged in a larger commit. Partially addresses #517 * Build Ingestion to target Java 8, for Beam compat As well as datatypes-java since ingestion depends on it. Java 11 is desirable for the other components, but for Beam it may impose limitations on what runners Feast can support, if it is even safe to run on an 11 JRE now. https://issues.apache.org/jira/browse/BEAM-2530 References #517
1 parent de3fd2b commit 880269b

5 files changed

Lines changed: 66 additions & 13 deletions

File tree

‎datatypes/java/pom.xml‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -37,6 +37,15 @@
3737

3838
<build>
3939
<plugins>
40+
<plugin>
41+
<groupId>org.apache.maven.plugins</groupId>
42+
<artifactId>maven-compiler-plugin</artifactId>
43+
<configuration>
44+
<!-- To ensure Ingestion remains compatible for Java 8-only Beam runners -->
45+
<release>8</release>
46+
</configuration>
47+
</plugin>
48+
4049
<plugin>
4150
<groupId>org.apache.maven.plugins</groupId>
4251
<artifactId>maven-dependency-plugin</artifactId>

‎docs/contributing/development-guide.md‎

Lines changed: 33 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -21,7 +21,7 @@ The following software is required for Feast development
2121

2222
* Java SE Development Kit 11
2323
* Python version 3.6 \(or above\) and pip
24-
* [Maven ](https://maven.apache.org/install.html)version 3.6.x
24+
* [Maven](https://maven.apache.org/install.html) version 3.6.x
2525

2626
Additionally, [grpc\_cli](https://github.com/grpc/grpc/blob/master/doc/command_line_tool.md) is useful for debugging and quick testing of gRPC endpoints.
2727

@@ -444,3 +444,35 @@ If you have made it to this point successfully you should have a functioning Fea
444444

445445
It is important to note that most of the functionality demonstrated above is already available in a more abstracted form in the Python SDK \(Feast management, data ingestion, feature retrieval\) and the Java/Go SDKs \(feature retrieval\). However, it is useful to understand these internals from a development standpoint.
446446

447+
### 5 Appendix
448+
449+
#### 5.1 Java / JDK Versions
450+
451+
Feast requires a Java 11 or greater JDK for building the project. This is checked by Maven so you'll be informed if you try to use an older version.
452+
453+
Leaf application modules of the build such as the Core and Serving APIs compile with [the `javac --release` flag] set for 11, and Ingestion and shared library modules that it uses target release 8. Here's why.
454+
455+
While we want to take advantage of advancements in the (long-term supported) language and platform, and for contributors to enjoy those as well, Apache Beam forms a major part of Feast's Ingestion component and fully validating Beam on Java 11 [is an open issue][BEAM-2530]. Moreover, Beam runners other than the DirectRunner may lag behind further still—Spark does not _build or run_ on Java 11 until its version 3.0 which is still in preview. Presumably Beam's SparkRunner will have to wait for Spark, and for its implementation to update to Spark 3.
456+
457+
To have confidence in Beam stability and our users' ability to deploy Feast on a range of Beam runners, we will continue to target Java 8 bytecode and Platform version for Feast Ingestion, until the ecosystem moves forward.
458+
459+
You do _not_ need a Java 8 SDK installed for development. Newer JDKs can build for the older platform, and Feast's Maven build does this automatically.
460+
461+
See [Feast issue #517][\#517] for discussion.
462+
463+
[the `javac --release` flag]: https://stackoverflow.com/questions/43102787/what-is-the-release-flag-in-the-java-9-compiler
464+
[BEAM-2530]: https://issues.apache.org/jira/browse/BEAM-2530
465+
[\#517]: https://github.com/gojek/feast/issues/517
466+
467+
#### 5.2 IntelliJ Tips and Troubleshooting
468+
469+
For IntelliJ users, this section collects notes and setup recommendations for working comfortably on the Feast project, especially to coexist as peacefully as possible with the Maven build.
470+
471+
##### Language Level
472+
473+
IntelliJ uses a notion of "Language Level" to drive assistance features according to the target Java version of a project. It often infers this appropriately from a Maven import, but it's wise to check, especially for a multi-module project with differing target Java releases across modules, like ours. Language level [can be set per module][module lang level]—if IntelliJ is suggesting things that turn out to fail when building with `mvn`, make sure the language level of the module in question corresponds to the `<release>` set for `maven-compiler-plugin` in the module's `pom.xml` (11 is project default if the module doesn't set one).
474+
475+
Ensure that the Project _SDK_ (not language level) is set to a Java 11 JDK, and that all modules use the Project SDK (see [the Dependencies tab] in the Modules view).
476+
477+
[module lang level]: https://www.jetbrains.com/help/idea/sources-tab.html#module_language_level
478+
[the Dependencies tab]: https://www.jetbrains.com/help/idea/dependencies.html

‎ingestion/pom.xml‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -31,6 +31,15 @@
3131

3232
<build>
3333
<plugins>
34+
<plugin>
35+
<groupId>org.apache.maven.plugins</groupId>
36+
<artifactId>maven-compiler-plugin</artifactId>
37+
<configuration>
38+
<!-- To ensure Ingestion remains compatible for Java 8-only Beam runners -->
39+
<release>8</release>
40+
</configuration>
41+
</plugin>
42+
3443
<plugin>
3544
<groupId>org.apache.maven.plugins</groupId>
3645
<artifactId>maven-shade-plugin</artifactId>

‎ingestion/src/test/java/feast/ingestion/transform/metrics/WriteFeatureValueMetricsDoFnTest.java‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -142,7 +142,7 @@ private Map<String, Iterable<FeatureRow>> readTestInput(String path) throws IOEx
142142
}
143143
List<String> colNames = new ArrayList<>();
144144
for (String line : lines) {
145-
if (line.strip().length() < 1) {
145+
if (line.trim().length() < 1) {
146146
continue;
147147
}
148148
String[] splits = line.split(",");
@@ -156,7 +156,7 @@ private Map<String, Iterable<FeatureRow>> readTestInput(String path) throws IOEx
156156

157157
Builder featureRowBuilder = FeatureRow.newBuilder();
158158
for (int i = 0; i < splits.length; i++) {
159-
String colVal = splits[i].strip();
159+
String colVal = splits[i].trim();
160160
if (i == 0) {
161161
featureRowBuilder.setFeatureSet(colVal);
162162
continue;

‎pom.xml‎

Lines changed: 13 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -349,7 +349,6 @@
349349
<plugin>
350350
<groupId>org.apache.maven.plugins</groupId>
351351
<artifactId>maven-javadoc-plugin</artifactId>
352-
<version>3.1.1</version>
353352
<executions>
354353
<execution>
355354
<id>attach-javadocs</id>
@@ -408,15 +407,8 @@
408407
<plugin>
409408
<groupId>org.apache.maven.plugins</groupId>
410409
<artifactId>maven-compiler-plugin</artifactId>
411-
<version>3.8.1</version>
412410
<configuration>
413411
<release>11</release>
414-
<compilerArgs>
415-
<arg>-Xlint:all</arg>
416-
<!-- -Xdoclint:all breaks on a false positive in JDK < 9 -->
417-
<!-- https://bugs.java.com/bugdatabase/view_bug.do?bug_id=JDK-8186647 -->
418-
<arg>-Xdoclint:-syntax</arg>
419-
</compilerArgs>
420412
</configuration>
421413
</plugin>
422414
<plugin>
@@ -442,7 +434,7 @@
442434
<version>[3.6,4.0)</version>
443435
</requireMavenVersion>
444436
<requireJavaVersion>
445-
<version>[1.8,11.1)</version>
437+
<version>[11.0,)</version>
446438
</requireJavaVersion>
447439
<reactorModuleConvergence />
448440
</rules>
@@ -574,6 +566,17 @@
574566
<artifactId>docker-maven-plugin</artifactId>
575567
<version>0.20.1</version>
576568
</plugin>
569+
<plugin>
570+
<groupId>org.apache.maven.plugins</groupId>
571+
<artifactId>maven-compiler-plugin</artifactId>
572+
<version>3.8.1</version>
573+
<configuration>
574+
<compilerArgs>
575+
<arg>-Xlint:all</arg>
576+
<arg>-Xdoclint:all</arg>
577+
</compilerArgs>
578+
</configuration>
579+
</plugin>
577580
<plugin>
578581
<groupId>org.apache.maven.plugins</groupId>
579582
<artifactId>maven-dependency-plugin</artifactId>
@@ -591,7 +594,7 @@
591594
<plugin>
592595
<groupId>org.apache.maven.plugins</groupId>
593596
<artifactId>maven-javadoc-plugin</artifactId>
594-
<version>3.1.0</version>
597+
<version>3.1.1</version>
595598
</plugin>
596599
<plugin>
597600
<groupId>org.codehaus.mojo</groupId>

0 commit comments

Comments
 (0)