You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@geode.apache.org by Kirk Lund <kl...@pivotal.io> on 2016/05/05 00:24:15 UTC
JUnit Parameterized + Category is broken (was: Unable to run single dunit)
OK, the problem seems to be specific to using Category and Parameterized
(both are part of JUnit 4) together in the same test.
If a test is annotated with @Category and @RunWith(Parameterized) then it
will be silently skipped by the gradle test test targets and this is not
noticed by our checkMissedTests task.
I rewrote GEODE-1350 as "JUnit 4 Category plus Parameterized results in
tests being skipped during build" -- please see the Jira ticket for further
details.
Also, see https://github.com/junit-team/junit4/wiki. JUnit 4.12 is the
latest release. 4.13 has been in the works for some time and may or may not
have a fix for this. I can experiment with 4.13 tomorrow, but since 4.13
remains unreleased (with no ETA that I can find) we will probably need to
convert any tests that use JUnit 4 Parameterized to instead use
JUnitParamsRunner.
We already have several tests using "@RunWith(JUnitParamsRunner)" and we
confirmed that they are being run by the build:
RegionEntryFactoryBuilderJUnitTest, LogServiceJUnitTest,
WanCommandsControllerJUnitTest, JUnitParamsExampleJUnitTest
We checked all of the other tests that have @RunWith and all the other
custom runners are being run as expected.
-Kirk
On Wed, May 4, 2016 at 4:28 PM, Dan Smith <ds...@pivotal.io> wrote:
> It also looks like this means these tests are not running as part of
> precheckin! At least, I don't see ListAndDescribeDiskStoreCommandsDUnitTest
> listed here
>
>
> https://builds.apache.org/job/Geode-nightly/458/testReport/com.gemstone.gemfire.management.internal.cli.commands/
>
> Somehow it's both not picking up the category in the distributedTest task,
> but also not flagging it as a missed test in checkMissedTests. That's not
> good.
>
> -Dan
>
> On Wed, May 4, 2016 at 4:19 PM, Kirk Lund <kl...@pivotal.io> wrote:
>
> > Apparently any dunit test that uses a custom runner fails to run. But
> dunit
> > tests using regular runners still run fine:
> >
> > 1) single dunit test with no custom runner (note this is a dunit test
> that
> > has been updated to JUnit 4)
> >
> > $ ./gradlew -DdistributedTest.single=DistributedMemberDUnitTest
> > geode-core:distributedTest
> > ...
> > :geode-core:distributedTest
> > :combineReports
> > All test reports at
> > /export/latvia1/users/klund/dev/gemfire/open/build/reports/combined
> >
> > BUILD SUCCESSFUL
> >
> > Total time: 1 mins 26.859 secs
> >
> > 2) single dunit test using the JUnit 4 Parameterized runner (this is
> also a
> > dunit test that has been updated to JUnit 4)
> >
> >
> > $ ./gradlew
> > -DdistributedTest.single=ListAndDescribeDiskStoreCommandsDUnitTest
> > geode-core:distributedTest
> > ...
> > :geode-core:distributedTest FAILED
> > :combineReports
> > All test reports at
> > /export/latvia1/users/klund/dev/gemfire/open/build/reports/combined
> >
> > FAILURE: Build failed with an exception.
> >
> > * What went wrong:
> > Execution failed for task ':geode-core:distributedTest'.
> > > Could not find matching test for pattern:
> > ListAndDescribeDiskStoreCommandsDUnitTest
> >
> > * Try:
> > Run with --stacktrace option to get the stack trace. Run with --info or
> > --debug option to get more log output.
> >
> > BUILD FAILED
> >
> > Total time: 5.918 secs
> >
> >
> > So, I would expect any dunit test using a custom runner (PowerMock,
> > Parameterized, JUnitParamsRunner) would all fail if run individually in
> > this way.
> >
> > -Kirk
> >
> >
> > On Wed, May 4, 2016 at 2:38 PM, Kirk Lund <kl...@pivotal.io> wrote:
> >
> > > This used to work quite well for me. Why am I not able to run a single
> > > dunit on the command-line now? Is this caused by the gradle upgrade?
> > >
> > > <kl...@pdx2-office-dhcp32>/Users/klund/dev/gemfire/open [520]$ find .
> > > -name ListAndDescribeDiskStoreCommandsDUnitTest.java
> > >
> > >
> > >
> >
> ./geode-core/src/test/java/com/gemstone/gemfire/management/internal/cli/commands/ListAndDescribeDiskStoreCommandsDUnitTest.java
> > >
> > > <kl...@pdx2-office-dhcp32>/Users/klund/dev/gemfire/open [521]$
> ./gradlew
> > > -DdistributedTest.single=ListAndDescribeDiskStoreCommandsDUnitTest
> > > geode-core:distributedTest
> > >
> > > :buildSrc:compileJava UP-TO-DATE
> > > :buildSrc:compileGroovy UP-TO-DATE
> > > :buildSrc:processResources UP-TO-DATE
> > > :buildSrc:classes UP-TO-DATE
> > > :buildSrc:jar UP-TO-DATE
> > > :buildSrc:assemble UP-TO-DATE
> > > :buildSrc:compileTestJava UP-TO-DATE
> > > :buildSrc:compileTestGroovy UP-TO-DATE
> > > :buildSrc:processTestResources UP-TO-DATE
> > > :buildSrc:testClasses UP-TO-DATE
> > > :buildSrc:test UP-TO-DATE
> > > :buildSrc:check UP-TO-DATE
> > > :buildSrc:build UP-TO-DATE
> > > :geode-common:compileJava UP-TO-DATE
> > > :geode-common:processResources UP-TO-DATE
> > > :geode-common:classes UP-TO-DATE
> > > :geode-common:jar UP-TO-DATE
> > > :geode-joptsimple:compileJava UP-TO-DATE
> > > :geode-joptsimple:processResources UP-TO-DATE
> > > :geode-joptsimple:classes UP-TO-DATE
> > > :geode-joptsimple:jar UP-TO-DATE
> > > :geode-json:compileJava UP-TO-DATE
> > > :geode-json:processResources UP-TO-DATE
> > > :geode-json:classes UP-TO-DATE
> > > :geode-json:jar UP-TO-DATE
> > > :geode-core:compileJava UP-TO-DATE
> > > :geode-core:createVersionPropertiesFile UP-TO-DATE
> > > :geode-core:processResources UP-TO-DATE
> > > :geode-core:classes UP-TO-DATE
> > > :geode-junit:compileJava UP-TO-DATE
> > > :geode-junit:processResources UP-TO-DATE
> > > :geode-junit:classes UP-TO-DATE
> > > :geode-junit:jar UP-TO-DATE
> > > :geode-core:compileTestJava UP-TO-DATE
> > > :geode-core:processTestResources UP-TO-DATE
> > > :geode-core:testClasses UP-TO-DATE
> > > :geode-core:distributedTest FAILED
> > > :combineReports UP-TO-DATE
> > >
> > > FAILURE: Build failed with an exception.
> > >
> > > * What went wrong:
> > > Execution failed for task ':geode-core:distributedTest'.
> > > > Could not find matching test for pattern:
> > > ListAndDescribeDiskStoreCommandsDUnitTest
> > >
> > > * Try:
> > > Run with --stacktrace option to get the stack trace. Run with --info or
> > > --debug option to get more log output.
> > >
> > > BUILD FAILED
> > >
> > > Total time: 3.364 secs
> > >
> > >
> >
>
Re: JUnit Parameterized + Category is broken (was: Unable to run
single dunit)
Posted by Kirk Lund <ki...@gmail.com>.
The good news is our only tests using Parameterized were just recently
added when feature/GEODE-17-2 merged to develop.
-Kirk
On Wednesday, May 4, 2016, Kirk Lund <kl...@pivotal.io> wrote:
> OK, the problem seems to be specific to using Category and Parameterized
> (both are part of JUnit 4) together in the same test.
>
> If a test is annotated with @Category and @RunWith(Parameterized) then it
> will be silently skipped by the gradle test test targets and this is not
> noticed by our checkMissedTests task.
>
> I rewrote GEODE-1350 as "JUnit 4 Category plus Parameterized results in
> tests being skipped during build" -- please see the Jira ticket for further
> details.
>
> Also, see https://github.com/junit-team/junit4/wiki. JUnit 4.12 is the
> latest release. 4.13 has been in the works for some time and may or may not
> have a fix for this. I can experiment with 4.13 tomorrow, but since 4.13
> remains unreleased (with no ETA that I can find) we will probably need to
> convert any tests that use JUnit 4 Parameterized to instead use
> JUnitParamsRunner.
>
> We already have several tests using "@RunWith(JUnitParamsRunner)" and we
> confirmed that they are being run by the build:
>
> RegionEntryFactoryBuilderJUnitTest, LogServiceJUnitTest,
> WanCommandsControllerJUnitTest, JUnitParamsExampleJUnitTest
>
> We checked all of the other tests that have @RunWith and all the other
> custom runners are being run as expected.
>
> -Kirk
>
> On Wed, May 4, 2016 at 4:28 PM, Dan Smith <dsmith@pivotal.io
> <javascript:;>> wrote:
>
> > It also looks like this means these tests are not running as part of
> > precheckin! At least, I don't see
> ListAndDescribeDiskStoreCommandsDUnitTest
> > listed here
> >
> >
> >
> https://builds.apache.org/job/Geode-nightly/458/testReport/com.gemstone.gemfire.management.internal.cli.commands/
> >
> > Somehow it's both not picking up the category in the distributedTest
> task,
> > but also not flagging it as a missed test in checkMissedTests. That's not
> > good.
> >
> > -Dan
> >
> > On Wed, May 4, 2016 at 4:19 PM, Kirk Lund <klund@pivotal.io
> <javascript:;>> wrote:
> >
> > > Apparently any dunit test that uses a custom runner fails to run. But
> > dunit
> > > tests using regular runners still run fine:
> > >
> > > 1) single dunit test with no custom runner (note this is a dunit test
> > that
> > > has been updated to JUnit 4)
> > >
> > > $ ./gradlew -DdistributedTest.single=DistributedMemberDUnitTest
> > > geode-core:distributedTest
> > > ...
> > > :geode-core:distributedTest
> > > :combineReports
> > > All test reports at
> > > /export/latvia1/users/klund/dev/gemfire/open/build/reports/combined
> > >
> > > BUILD SUCCESSFUL
> > >
> > > Total time: 1 mins 26.859 secs
> > >
> > > 2) single dunit test using the JUnit 4 Parameterized runner (this is
> > also a
> > > dunit test that has been updated to JUnit 4)
> > >
> > >
> > > $ ./gradlew
> > > -DdistributedTest.single=ListAndDescribeDiskStoreCommandsDUnitTest
> > > geode-core:distributedTest
> > > ...
> > > :geode-core:distributedTest FAILED
> > > :combineReports
> > > All test reports at
> > > /export/latvia1/users/klund/dev/gemfire/open/build/reports/combined
> > >
> > > FAILURE: Build failed with an exception.
> > >
> > > * What went wrong:
> > > Execution failed for task ':geode-core:distributedTest'.
> > > > Could not find matching test for pattern:
> > > ListAndDescribeDiskStoreCommandsDUnitTest
> > >
> > > * Try:
> > > Run with --stacktrace option to get the stack trace. Run with --info or
> > > --debug option to get more log output.
> > >
> > > BUILD FAILED
> > >
> > > Total time: 5.918 secs
> > >
> > >
> > > So, I would expect any dunit test using a custom runner (PowerMock,
> > > Parameterized, JUnitParamsRunner) would all fail if run individually in
> > > this way.
> > >
> > > -Kirk
> > >
> > >
> > > On Wed, May 4, 2016 at 2:38 PM, Kirk Lund <klund@pivotal.io
> <javascript:;>> wrote:
> > >
> > > > This used to work quite well for me. Why am I not able to run a
> single
> > > > dunit on the command-line now? Is this caused by the gradle upgrade?
> > > >
> > > > <kl...@pdx2-office-dhcp32>/Users/klund/dev/gemfire/open [520]$ find
> .
> > > > -name ListAndDescribeDiskStoreCommandsDUnitTest.java
> > > >
> > > >
> > > >
> > >
> >
> ./geode-core/src/test/java/com/gemstone/gemfire/management/internal/cli/commands/ListAndDescribeDiskStoreCommandsDUnitTest.java
> > > >
> > > > <kl...@pdx2-office-dhcp32>/Users/klund/dev/gemfire/open [521]$
> > ./gradlew
> > > > -DdistributedTest.single=ListAndDescribeDiskStoreCommandsDUnitTest
> > > > geode-core:distributedTest
> > > >
> > > > :buildSrc:compileJava UP-TO-DATE
> > > > :buildSrc:compileGroovy UP-TO-DATE
> > > > :buildSrc:processResources UP-TO-DATE
> > > > :buildSrc:classes UP-TO-DATE
> > > > :buildSrc:jar UP-TO-DATE
> > > > :buildSrc:assemble UP-TO-DATE
> > > > :buildSrc:compileTestJava UP-TO-DATE
> > > > :buildSrc:compileTestGroovy UP-TO-DATE
> > > > :buildSrc:processTestResources UP-TO-DATE
> > > > :buildSrc:testClasses UP-TO-DATE
> > > > :buildSrc:test UP-TO-DATE
> > > > :buildSrc:check UP-TO-DATE
> > > > :buildSrc:build UP-TO-DATE
> > > > :geode-common:compileJava UP-TO-DATE
> > > > :geode-common:processResources UP-TO-DATE
> > > > :geode-common:classes UP-TO-DATE
> > > > :geode-common:jar UP-TO-DATE
> > > > :geode-joptsimple:compileJava UP-TO-DATE
> > > > :geode-joptsimple:processResources UP-TO-DATE
> > > > :geode-joptsimple:classes UP-TO-DATE
> > > > :geode-joptsimple:jar UP-TO-DATE
> > > > :geode-json:compileJava UP-TO-DATE
> > > > :geode-json:processResources UP-TO-DATE
> > > > :geode-json:classes UP-TO-DATE
> > > > :geode-json:jar UP-TO-DATE
> > > > :geode-core:compileJava UP-TO-DATE
> > > > :geode-core:createVersionPropertiesFile UP-TO-DATE
> > > > :geode-core:processResources UP-TO-DATE
> > > > :geode-core:classes UP-TO-DATE
> > > > :geode-junit:compileJava UP-TO-DATE
> > > > :geode-junit:processResources UP-TO-DATE
> > > > :geode-junit:classes UP-TO-DATE
> > > > :geode-junit:jar UP-TO-DATE
> > > > :geode-core:compileTestJava UP-TO-DATE
> > > > :geode-core:processTestResources UP-TO-DATE
> > > > :geode-core:testClasses UP-TO-DATE
> > > > :geode-core:distributedTest FAILED
> > > > :combineReports UP-TO-DATE
> > > >
> > > > FAILURE: Build failed with an exception.
> > > >
> > > > * What went wrong:
> > > > Execution failed for task ':geode-core:distributedTest'.
> > > > > Could not find matching test for pattern:
> > > > ListAndDescribeDiskStoreCommandsDUnitTest
> > > >
> > > > * Try:
> > > > Run with --stacktrace option to get the stack trace. Run with --info
> or
> > > > --debug option to get more log output.
> > > >
> > > > BUILD FAILED
> > > >
> > > > Total time: 3.364 secs
> > > >
> > > >
> > >
> >
>