Skip to content

JAVA-1266 & JAVA-1411: Stop using oss-parent and normalize versions across the build#851

Merged
adutra merged 1 commit into
3.xfrom
java1411
Jun 9, 2017
Merged

JAVA-1266 & JAVA-1411: Stop using oss-parent and normalize versions across the build#851
adutra merged 1 commit into
3.xfrom
java1411

Conversation

@adutra
Copy link
Copy Markdown
Contributor

@adutra adutra commented May 30, 2017

Summary of changes:

JAVA-1266:

JAVA-1411:

  • Introduced dependencyManagement and pluginManagement sections in the parent pom
  • Declared versions for all dependencies and plugins at parent pom
  • Concentrated common plugin configuration items into the parent pom
  • Revisited bundle and shade plugin configurations
  • Updated <url> tags
  • Removed unused 'env' system property
  • Profile declaration cleanup

@adutra adutra added this to the 3.3.0 milestone May 30, 2017
@adutra adutra force-pushed the java1411 branch 4 times, most recently from d18b867 to 3ab7d58 Compare May 31, 2017 16:47
@tolbertam tolbertam self-requested a review May 31, 2017 18:03
Comment thread driver-core/pom.xml Outdated
</description>
<url>https://github.com/datastax/java-driver</url>

<url>https://github.com/datastax/java-driver/driver-core</url>
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This link looks to be broken (should be: https://github.com/datastax/java-driver/tree/3.x/driver-core although would be nice if we could avoid having the branch there)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah right, silly me :) I guess there's nothing to do other than simply remove the URL tag altogether, as it does not seem that there is a way to create a link without specifying a branch.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 on removing the URL tag, for individual modules the parent link is probably appropriate anyways.

Comment thread driver-extras/pom.xml Outdated
<test.groups>unit</test.groups>
<jdk8.excludes>none</jdk8.excludes>
</properties>
<url>https://github.com/datastax/java-driver/driver-extras</url>
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

similarly to the core pom, this link will not work.

Comment thread driver-mapping/pom.xml Outdated
<properties>
<groovy.version>2.4.7</groovy.version>
</properties>
<url>https://github.com/datastax/java-driver/driver-mapping</url>
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

similar to core, link broken

Comment thread driver-mapping/pom.xml
<plugin>
<groupId>org.codehaus.gmaven</groupId>
<artifactId>gmaven-plugin</artifactId>
<version>1.5</version>
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Was this just ultimately not needed? I see MapperGroovyTest is still running so things are still working on that front.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm I tried removing the plugin, but the groovy test did not execute.

Comment thread driver-tests/osgi/pom.xml Outdated
<test.skip>true</test.skip>
</properties>

<url>https://github.com/datastax/java-driver/driver-tests/osgi</url>
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

broken link

Comment thread pom.xml
<!-- do not upgrade until https://issues.apache.org/jira/browse/SUREFIRE-1302 is fixed -->
<version>2.18</version>
<configuration>
<properties>
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was done for #730, is it no longer needed with 2.18 or is there some other change that makes it no longer needed?

Copy link
Copy Markdown
Contributor Author

@adutra adutra May 31, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is still needed, but the configuration is inherited from the main plugin declaration under section pluginManagement.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah! I didn't catch that, I ran a profile and didn't see it open / close many files in short succession, so it is still doing what it's supposed to.

@tolbertam
Copy link
Copy Markdown
Contributor

It is really odd to me how com.datastax.driver.core.PreparedStatementTest.batchTest consistently fails in jenkins and appveyor, but isn't happening on 3.x branch. Seems unrelated but must be some weird nuanced difference that's causing this..looking into it.

@tolbertam
Copy link
Copy Markdown
Contributor

It looks like tests are not running in class order, or maybe they are running concurrently which is causing weird behavior. Looking at the surefire config to see if it's possible that we're forking.

@tolbertam
Copy link
Copy Markdown
Contributor

Example of what i was referring to about test order:

-----------------------------------------------
Starting com.datastax.driver.core.PreparedStatementTest.prepareSetTest2 [Test #895]...
SUCCESS: prepareSetTest2
Test   : 00:00:00
Elapsed: 00:22:08

    1327822 [main] INFO  com.datastax.driver.core.ClockFactory - Using native clock to generate timestamps.

-----------------------------------------------
Starting com.datastax.driver.core.QueryOptionsTest.should_not_reprepare_on_up_when_disabled [Test #896]...
SUCCESS: should_not_reprepare_on_up_when_disabled
Test   : 00:00:01
Elapsed: 00:22:09


-----------------------------------------------
Starting com.datastax.driver.core.PreparedStatementTest.prepareStatementInheritPropertiesTest [Test #897]...
SUCCESS: prepareStatementInheritPropertiesTest
Test   : 00:00:00
Elapsed: 00:22:09


-----------------------------------------------
Starting com.datastax.driver.core.PreparedStatementTest.prepareListTest2 [Test #898]...
SUCCESS: prepareListTest2
Test   : 00:00:00
Elapsed: 00:22:09


-----------------------------------------------
Starting com.datastax.driver.core.PreparedStatementTest.batchTest [Test #899]...
java.lang.AssertionError: expected [prepWithNull2] but found [one]
	at org.testng.Assert.fail(Assert.java:94)
	at org.testng.Assert.failNotEquals(Assert.java:494)
	at org.testng.Assert.assertEquals(Assert.java:123)
	at org.testng.Assert.assertEquals(Assert.java:176)
	at org.testng.Assert.assertEquals(Assert.java:186)
	at com.datastax.driver.core.PreparedStatementTest.batchTest(PreparedStatementTest.java:401)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:622)
	at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:84)
	at org.testng.internal.Invoker.invokeMethod(Invoker.java:714)
	at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:901)
	at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1231)
	at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:127)
	at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:111)
	at org.testng.TestRunner.privateRun(TestRunner.java:767)
	at org.testng.TestRunner.run(TestRunner.java:617)
	at org.testng.SuiteRunner.runTest(SuiteRunner.java:348)
	at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:343)
	at org.testng.SuiteRunner.privateRun(SuiteRunner.java:305)
	at org.testng.SuiteRunner.run(SuiteRunner.java:254)
	at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52)
	at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:86)
	at org.testng.TestNG.runSuitesSequentially(TestNG.java:1224)
	at org.testng.TestNG.runSuitesLocally(TestNG.java:1149)
	at org.testng.TestNG.run(TestNG.java:1057)
	at org.apache.maven.surefire.testng.TestNGExecutor.run(TestNGExecutor.java:115)
	at org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.executeMulti(TestNGDirectoryTestSuite.java:205)
	at org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.execute(TestNGDirectoryTestSuite.java:108)
	at org.apache.maven.surefire.testng.TestNGProvider.invoke(TestNGProvider.java:111)
	at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:203)
	at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:155)
	at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:103)
FAILED : batchTest
Test   : 00:00:00
Elapsed: 00:22:09

    1335634 [main] INFO  com.datastax.driver.core.ClockFactory - Using native clock to generate timestamps.

-----------------------------------------------
Starting com.datastax.driver.core.PreparedStatementTest.should_set_routing_key_on_case_sensitive_keyspace_and_table [Test #900]...
SUCCESS: should_set_routing_key_on_case_sensitive_keyspace_and_table
Test   : 00:00:00
Elapsed: 00:22:16

    1336350 [main] INFO  com.datastax.driver.core.ClockFactory - Using native clock to generate timestamps.

-----------------------------------------------
Starting com.datastax.driver.core.QueryOptionsTest.should_reprepare_on_up_when_enabled [Test #901]...
SUCCESS: should_reprepare_on_up_when_enabled
Test   : 00:00:01
Elapsed: 00:22:17


-----------------------------------------------
Starting com.datastax.driver.core.querybuilder.QueryBuilder21ExecutionTest.should_handle_contains_key_on_map_with_index [Test #902]...
SUCCESS: should_handle_contains_key_on_map_with_index
Test   : 00:00:00
Elapsed: 00:22:17


-----------------------------------------------
Starting com.datastax.driver.core.PreparedStatementTest.prepareListTest [Test #903]...
SUCCESS: prepareListTest
Test   : 00:00:00
Elapsed: 00:22:20

I don't seem to experience the same thing when running locally, so not quite sure what can be happening yet.

@adutra
Copy link
Copy Markdown
Contributor Author

adutra commented Jun 1, 2017

Created #853 to fix the failing tests.

Comment thread ci/appveyor.yml
install:
- ps: .\ci\appveyor.ps1
build_script:
- "set \"JAVA_HOME=%JAVA_8_HOME%\" && mvn install -DskipTests=true -D\"maven.javadoc.skip\"=true -B -V"
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not needed anymore, javadocs are only generated if the release profile is activated.

@adutra
Copy link
Copy Markdown
Contributor Author

adutra commented Jun 2, 2017

Squashed commits in anticipation of feature complete today.

@tolbertam
Copy link
Copy Markdown
Contributor

tolbertam commented Jun 2, 2017

It looks like there may be a few inconsistent test issues caused by ordering changes, but I can take care of those on a different branch 👍 . There's still a few more things i need to try, but this looks good so far. The only thing I'm not sure about is the maven release plugin changes since it's not something I'm very familiar with, but I think I can use our artifactory deploy job to verify that this works so will give that a shot today. (Note that if it takes a while for me to get to that, I think this is in a good enough state to merge as we can always work out release plugin issues as part of the release)

@adutra
Copy link
Copy Markdown
Contributor Author

adutra commented Jun 2, 2017

The only thing I'm not sure about is the maven release plugin changes since it's not something I'm very familiar with,

You can totally try out a release by issuing the following commands:

mvn release:prepare
mvn release:perform

The first command will create tags in Github and you'll have to delete them manually (I suggest using a lower release number just in case, e.g. 0.0.1).
The second will deploy to Nexus OSS but will not release the artifacts. The release to Maven Central still needs to be done manually as per our documentation. Just make sure that you don't release the staging repository (you should close and drop it).

but I think I can use our artifactory deploy job to verify that this works so will give that a shot today.

That would be awesome. I did everything bearing in mind that a regular deploy to our internal Artifactory should work just as before.

@tolbertam
Copy link
Copy Markdown
Contributor

You can totally try out a release by issuing the following commands:

mvn release:prepare
mvn release:perform

Nice call. I'll try the prepare part on a personal fork (just need to change the repo references and such).

The artifactory deploy worked by the way so that checks out.

@tolbertam
Copy link
Copy Markdown
Contributor

Question, I just noticed that the licenses/developers/scm sections in pom.xml are repeated in all the child modules. If we remove those from the child modules can they be inherited from the parent? That could reduce some more duplication as well.

@tolbertam
Copy link
Copy Markdown
Contributor

Experimenting with the release plugin on my personal fork seemed to work (tag was appropriately created and my target branch was updated with the appropriate commits for incrementing versions in pom.xml). 👍

Copy link
Copy Markdown
Contributor

@tolbertam tolbertam left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks great to me! I had a small comment about duplication of licenses/scm/developer sections in the child modules, but that can be visited later (unless you wanted to approach that now). 👍 for merge.

@adutra
Copy link
Copy Markdown
Contributor Author

adutra commented Jun 2, 2017

I had a small comment about duplication of licenses/scm/developer sections in the child modules

I hesitated myself about it. Is it legally proved that the license stuff is inherited from the parent pom?

@tolbertam
Copy link
Copy Markdown
Contributor

I hesitated myself about it. Is it legally proved that the license stuff is inherited from the parent pom?

I just tried it out and it seems to work. I deleted those sections from driver-core/pom.xml and ran in the driver-core directory:

mvn help:effective-pom

and those sections appeared to be inherited from the parent pom.

@adutra
Copy link
Copy Markdown
Contributor Author

adutra commented Jun 2, 2017

Great, thanks for trying it out! I just removed these sections from all child poms.

@tolbertam
Copy link
Copy Markdown
Contributor

Looks great! Thanks for the quick change 👍

@olim7t
Copy link
Copy Markdown
Contributor

olim7t commented Jun 2, 2017

Did we try creating a staging repository on Sonatype OSS with the latest changes? Validations rules might be the reason why some sections were repeated in child poms.

Copy link
Copy Markdown
Contributor

@olim7t olim7t left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did not review everything in detail, but the changes for 1266 look ok to me.

Once again, may I kindly request that we keep commits (and ideally pull requests as well) more targeted? We're heading towards a single commit that will aggregate 1266, 1411 + additional changes, and purely cosmetic changes (e.g. blank lines). This is not cool for reviewers, mergers, or future maintainers looking at the history.

Comment thread pom.xml Outdated
<plugin>
<groupId>org.sonatype.plugins</groupId>
<artifactId>nexus-staging-maven-plugin</artifactId>
<version>1.6.7</version>
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's an 1.6.8 version in Maven central.
It looks like the changes are minor though, the main one is NEXUS-10502 which relates to proxy settings.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Version bumped, thanks.

Comment thread pom.xml Outdated
<plugin>
<groupId>org.sonatype.plugins</groupId>
<artifactId>nexus-staging-maven-plugin</artifactId>
<version>1.6.7</version>
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No need to repeat the version here if it's in pluginManagement.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems that versions of plugins declared in a profile do not get inherited from plugins declared in pluginManagement. Here is the output of mvn versions:display-plugin-updates for driver-examples if I remove the version of this plugin declaration:

[WARNING] The following plugins do not have their version specified:
[WARNING]   org.sonatype.plugins:nexus-staging-maven-plugin ............ 1.6.7

This might be a bug in the display-plugin-updates Mojo though, but I'd rather keep the version just in case.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In hindsight, the build seems to work normally without specifying versions for plugins declared in profiles, so I went ahead and implemented your suggestion.

@adutra
Copy link
Copy Markdown
Contributor Author

adutra commented Jun 2, 2017

Once again, may I kindly request that we keep commits (and ideally pull requests as well) more targeted? We're heading towards a single commit that will aggregate 1266, 1411 + additional changes, and purely cosmetic changes (e.g. blank lines)

Removing OSS parent changes the configuration and version of several plugins (compiler, surefire, javadoc, install, deploy...). That's why it seemed simpler to do both 1511+1266 in one single commit.

@adutra
Copy link
Copy Markdown
Contributor Author

adutra commented Jun 2, 2017

Did we try creating a staging repository on Sonatype OSS with the latest changes?

Yes, I validated that a staging repository is created and is valid wrt to OSS Sonatype rules (gpg signatures etc.).

Validations rules might be the reason why some sections were repeated in child poms.

Which ones? If you are referring to the nexus-staging plugin, indeed it must be mentioned a couple of times in child modules to prevent it from deploying unwanted artifacts.

@olim7t
Copy link
Copy Markdown
Contributor

olim7t commented Jun 2, 2017

Maven central requires requires license, developer and scm sections for POMs, and my thought was that maybe it didn't take into account inheritance. But that was just a wild guess, if that's been validated then all good 👍

Copy link
Copy Markdown
Contributor

@GregBestland GregBestland left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My Pom foo is weak, but after much deciphering I think this is gtg.

…cross the build

Summary of changes:

JAVA-1266:
- Removed reference to deprecated OSS Sonatype parent poms
- Adjusted release process to use the recommended method,
  as described in http://central.sonatype.org/pages/apache-maven.html
- Created new 'release' profile to contain release-specific plugins

JAVA-1411:
- Introduced dependencyManagement and pluginManagement sections in the parent pom
- Declared versions for all dependencies and plugins at parent pom
- Concentrated common plugin configuration items into the parent pom
- Revisited bundle and shade plugin configurations
- Updated <url> tags
- Removed unused 'env' system property
- Profile declaration cleanup
@adutra adutra merged commit 24723df into 3.x Jun 9, 2017
@adutra adutra deleted the java1411 branch June 9, 2017 09:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants