You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@maven.apache.org by Christopher <ct...@apache.org> on 2018/03/30 18:58:50 UTC

Bug in processing surefire/failsafe groups

Found a bug in maven-failsafe-plugin (
https://github.com/apache/accumulo/issues/411) when using groups (JUnit
categories):
http://maven.apache.org/surefire/maven-failsafe-plugin/integration-test-mojo.html#groups
http://maven.apache.org/surefire/maven-failsafe-plugin/integration-test-mojo.html#excludedGroups

The problem is when the class corresponding to a configured group is not on
the class path. This is particularly problematic in multi-module builds, if
a group is specified which exists in one module but not another one.

It appears the bug occurs because the configured group/category is loaded
from the class path, and then compared to the classes to find matching
tests. However, this seems backwards. If the tests were examined first,
then the name of the groups could be matched against the configuration
without loading to load the configured groups/category. Even if this is not
possible due to some limitation on how these groups are passed to JUnit or
some other restriction, a ClassNotFoundException (CNFE) should be handled
more sanely: For example, in the case of a CNFE for the "groups" parameter,
then it's clearly not possible for any tests to be included, so no tests
should run. In the case of a CNFE for "excludedGroups", then it's clearly
not possible for any tests to match, so none should be excluded.

This bug has got me thinking about a related feature I'd like to see in
maven-surefire-plugin and maven-failsafe-plugin: I'd like to see groups /
excludedGroups specified by patterns, similar to `-Dtest=MyTest` instead of
`-Dtest=org.apache.project.package.subpackage.testing.MyTest`, it would be
nice to set `-Dgroups=MyGroup` rather than
`-Dgroups=org.apache.project.package.subpackage.testing.categories.MyGroup`.
If groups could be specified as patterns, it would be more obvious that a
CNFE could not occur... if a pattern didn't match any classes, then it
wouldn't result in a CNFE... it would simply return an empty set.

Thanks,

Christopher

Re: Bug in processing surefire/failsafe groups

Posted by Tibor Digana <ti...@apache.org>.
Christopher,

We fixed this in SUREFIRE-1498 for version 3.0.0-M1.
If you want to, you can build and use SNAPSHOT version.

T

On Wed, Apr 4, 2018 at 8:58 PM, Christopher <ct...@apache.org> wrote:

> Version 2.21.0 didn't quite achieve a perfect fix. Now the plugin proceeds,
> but instead, I get the following in my build:
>
> [WARNING] Corrupted STDOUT by directly writing to native stream in forked
> JVM 1. See FAQ web page and the dump file
> /path/to/target/failsafe-reports/2018-04-04T14-48-54_
> 810-jvmRun1.dumpstream
>
> And inside that file, I get:
>
> # Created on 2018-04-04T14:49:31.119
>
> Corrupted STDOUT by directly writing to native stream in forked JVM 1.
> Stream '[WARNING] Couldn't load group class
> 'org.apache.accumulo.test.categories.SunnyDayTests' in Surefire|Failsafe
> plugin. The group class is ignored!'.
> java.lang.IllegalArgumentException: Stream stdin corrupted. Expected comma
> after third character in command '[WARNING] Couldn't load group class
> 'org.apache.accumulo.test.categories.SunnyDayTests' in Surefire|Failsafe
> plugin. The group class is ignored!'.
>   at
> org.apache.maven.plugin.surefire.booterclient.output.
> ForkClient$OperationalData.<init>(ForkClient.java:511)
>   at
> org.apache.maven.plugin.surefire.booterclient.output.
> ForkClient.processLine(ForkClient.java:209)
>   at
> org.apache.maven.plugin.surefire.booterclient.output.
> ForkClient.consumeLine(ForkClient.java:176)
>   at
> org.apache.maven.plugin.surefire.booterclient.output.
> ThreadedStreamConsumer$Pumper.run(ThreadedStreamConsumer.java:88)
>   at java.lang.Thread.run(Thread.java:748)
>
> Ignoring the class is great and the expected behavior... but "Corrupted
> STDOUT" is still concerning.
>
> On Wed, Apr 4, 2018 at 2:52 PM Christopher <ct...@apache.org> wrote:
>
> > Enrico: Thanks for the info. I'm using the version inherited from the ASF
> > parent POM 19, which turns out to be 2.20.1. Version 2.21.0 seems to fix
> > the problem. Thanks!
> >
> > Tibor: No, I didn't get previous email. The Maven lists are too spammy
> for
> > me to subscribe. That could be the reason. Please CC me if follow-up is
> > needed. Thanks.
> >
> >
> > On Sat, Mar 31, 2018 at 10:16 AM Enrico Olivelli <eo...@gmail.com>
> > wrote:
> >
> >> This problem has been fixed in 2.21.0, what version are you running?
> >>
> >> Enrico
> >>
> >>
> >> Il sab 31 mar 2018, 14:54 Tibor Digana <ti...@apache.org> ha
> >> scritto:
> >>
> >>> Christopher,
> >>>
> >>> Did you receive my previous answer?
> >>> There is some problem with email delivery.
> >>> Tibor
> >>>
> >>> On Fri, Mar 30, 2018 at 8:58 PM, Christopher <ct...@apache.org>
> >>> wrote:
> >>>
> >>> > Found a bug in maven-failsafe-plugin (
> >>> > https://github.com/apache/accumulo/issues/411) when using groups
> >>> (JUnit
> >>> > categories):
> >>> > http://maven.apache.org/surefire/maven-failsafe-
> >>> > plugin/integration-test-mojo.html#groups
> >>> > http://maven.apache.org/surefire/maven-failsafe-
> >>> > plugin/integration-test-mojo.html#excludedGroups
> >>> >
> >>> > The problem is when the class corresponding to a configured group is
> >>> not on
> >>> > the class path. This is particularly problematic in multi-module
> >>> builds, if
> >>> > a group is specified which exists in one module but not another one.
> >>> >
> >>> > It appears the bug occurs because the configured group/category is
> >>> loaded
> >>> > from the class path, and then compared to the classes to find
> matching
> >>> > tests. However, this seems backwards. If the tests were examined
> first,
> >>> > then the name of the groups could be matched against the
> configuration
> >>> > without loading to load the configured groups/category. Even if this
> >>> is not
> >>> > possible due to some limitation on how these groups are passed to
> >>> JUnit or
> >>> > some other restriction, a ClassNotFoundException (CNFE) should be
> >>> handled
> >>> > more sanely: For example, in the case of a CNFE for the "groups"
> >>> parameter,
> >>> > then it's clearly not possible for any tests to be included, so no
> >>> tests
> >>> > should run. In the case of a CNFE for "excludedGroups", then it's
> >>> clearly
> >>> > not possible for any tests to match, so none should be excluded.
> >>> >
> >>> > This bug has got me thinking about a related feature I'd like to see
> in
> >>> > maven-surefire-plugin and maven-failsafe-plugin: I'd like to see
> >>> groups /
> >>> > excludedGroups specified by patterns, similar to `-Dtest=MyTest`
> >>> instead of
> >>> > `-Dtest=org.apache.project.package.subpackage.testing.MyTest`, it
> >>> would be
> >>> > nice to set `-Dgroups=MyGroup` rather than
> >>> > `-Dgroups=org.apache.project.package.subpackage.testing.
> >>> > categories.MyGroup`.
> >>> > If groups could be specified as patterns, it would be more obvious
> >>> that a
> >>> > CNFE could not occur... if a pattern didn't match any classes, then
> it
> >>> > wouldn't result in a CNFE... it would simply return an empty set.
> >>> >
> >>> > Thanks,
> >>> >
> >>> > Christopher
> >>> >
> >>>
> >> --
> >>
> >>
> >> -- Enrico Olivelli
> >>
> >
>

Re: Bug in processing surefire/failsafe groups

Posted by Enrico Olivelli <eo...@gmail.com>.
Il mer 4 apr 2018, 20:59 Christopher <ct...@apache.org> ha scritto:

> Version 2.21.0 didn't quite achieve a perfect fix. Now the plugin proceeds,
> but instead, I get the following in my build:
>

This has been fixed on current master.
Just wait for a release, I think we will be able to cut it soon.

I am an user of the same usecase and I understand clearly your problem.
It is only a matter of time

Cheers
Enrico


> [WARNING] Corrupted STDOUT by directly writing to native stream in forked
> JVM 1. See FAQ web page and the dump file
> /path/to/target/failsafe-reports/2018-04-04T14-48-54_810-jvmRun1.dumpstream
>
> And inside that file, I get:
>
> # Created on 2018-04-04T14:49:31.119
>
> Corrupted STDOUT by directly writing to native stream in forked JVM 1.
> Stream '[WARNING] Couldn't load group class
> 'org.apache.accumulo.test.categories.SunnyDayTests' in Surefire|Failsafe
> plugin. The group class is ignored!'.
> java.lang.IllegalArgumentException: Stream stdin corrupted. Expected comma
> after third character in command '[WARNING] Couldn't load group class
> 'org.apache.accumulo.test.categories.SunnyDayTests' in Surefire|Failsafe
> plugin. The group class is ignored!'.
>   at
>
> org.apache.maven.plugin.surefire.booterclient.output.ForkClient$OperationalData.<init>(ForkClient.java:511)
>   at
>
> org.apache.maven.plugin.surefire.booterclient.output.ForkClient.processLine(ForkClient.java:209)
>   at
>
> org.apache.maven.plugin.surefire.booterclient.output.ForkClient.consumeLine(ForkClient.java:176)
>   at
>
> org.apache.maven.plugin.surefire.booterclient.output.ThreadedStreamConsumer$Pumper.run(ThreadedStreamConsumer.java:88)
>   at java.lang.Thread.run(Thread.java:748)
>
> Ignoring the class is great and the expected behavior... but "Corrupted
> STDOUT" is still concerning.
>
> On Wed, Apr 4, 2018 at 2:52 PM Christopher <ct...@apache.org> wrote:
>
> > Enrico: Thanks for the info. I'm using the version inherited from the ASF
> > parent POM 19, which turns out to be 2.20.1. Version 2.21.0 seems to fix
> > the problem. Thanks!
> >
> > Tibor: No, I didn't get previous email. The Maven lists are too spammy
> for
> > me to subscribe. That could be the reason. Please CC me if follow-up is
> > needed. Thanks.
> >
> >
> > On Sat, Mar 31, 2018 at 10:16 AM Enrico Olivelli <eo...@gmail.com>
> > wrote:
> >
> >> This problem has been fixed in 2.21.0, what version are you running?
> >>
> >> Enrico
> >>
> >>
> >> Il sab 31 mar 2018, 14:54 Tibor Digana <ti...@apache.org> ha
> >> scritto:
> >>
> >>> Christopher,
> >>>
> >>> Did you receive my previous answer?
> >>> There is some problem with email delivery.
> >>> Tibor
> >>>
> >>> On Fri, Mar 30, 2018 at 8:58 PM, Christopher <ct...@apache.org>
> >>> wrote:
> >>>
> >>> > Found a bug in maven-failsafe-plugin (
> >>> > https://github.com/apache/accumulo/issues/411) when using groups
> >>> (JUnit
> >>> > categories):
> >>> > http://maven.apache.org/surefire/maven-failsafe-
> >>> > plugin/integration-test-mojo.html#groups
> >>> > http://maven.apache.org/surefire/maven-failsafe-
> >>> > plugin/integration-test-mojo.html#excludedGroups
> >>> >
> >>> > The problem is when the class corresponding to a configured group is
> >>> not on
> >>> > the class path. This is particularly problematic in multi-module
> >>> builds, if
> >>> > a group is specified which exists in one module but not another one.
> >>> >
> >>> > It appears the bug occurs because the configured group/category is
> >>> loaded
> >>> > from the class path, and then compared to the classes to find
> matching
> >>> > tests. However, this seems backwards. If the tests were examined
> first,
> >>> > then the name of the groups could be matched against the
> configuration
> >>> > without loading to load the configured groups/category. Even if this
> >>> is not
> >>> > possible due to some limitation on how these groups are passed to
> >>> JUnit or
> >>> > some other restriction, a ClassNotFoundException (CNFE) should be
> >>> handled
> >>> > more sanely: For example, in the case of a CNFE for the "groups"
> >>> parameter,
> >>> > then it's clearly not possible for any tests to be included, so no
> >>> tests
> >>> > should run. In the case of a CNFE for "excludedGroups", then it's
> >>> clearly
> >>> > not possible for any tests to match, so none should be excluded.
> >>> >
> >>> > This bug has got me thinking about a related feature I'd like to see
> in
> >>> > maven-surefire-plugin and maven-failsafe-plugin: I'd like to see
> >>> groups /
> >>> > excludedGroups specified by patterns, similar to `-Dtest=MyTest`
> >>> instead of
> >>> > `-Dtest=org.apache.project.package.subpackage.testing.MyTest`, it
> >>> would be
> >>> > nice to set `-Dgroups=MyGroup` rather than
> >>> > `-Dgroups=org.apache.project.package.subpackage.testing.
> >>> > categories.MyGroup`.
> >>> > If groups could be specified as patterns, it would be more obvious
> >>> that a
> >>> > CNFE could not occur... if a pattern didn't match any classes, then
> it
> >>> > wouldn't result in a CNFE... it would simply return an empty set.
> >>> >
> >>> > Thanks,
> >>> >
> >>> > Christopher
> >>> >
> >>>
> >> --
> >>
> >>
> >> -- Enrico Olivelli
> >>
> >
>
-- 


-- Enrico Olivelli

Re: Bug in processing surefire/failsafe groups

Posted by Christopher <ct...@apache.org>.
Version 2.21.0 didn't quite achieve a perfect fix. Now the plugin proceeds,
but instead, I get the following in my build:

[WARNING] Corrupted STDOUT by directly writing to native stream in forked
JVM 1. See FAQ web page and the dump file
/path/to/target/failsafe-reports/2018-04-04T14-48-54_810-jvmRun1.dumpstream

And inside that file, I get:

# Created on 2018-04-04T14:49:31.119

Corrupted STDOUT by directly writing to native stream in forked JVM 1.
Stream '[WARNING] Couldn't load group class
'org.apache.accumulo.test.categories.SunnyDayTests' in Surefire|Failsafe
plugin. The group class is ignored!'.
java.lang.IllegalArgumentException: Stream stdin corrupted. Expected comma
after third character in command '[WARNING] Couldn't load group class
'org.apache.accumulo.test.categories.SunnyDayTests' in Surefire|Failsafe
plugin. The group class is ignored!'.
  at
org.apache.maven.plugin.surefire.booterclient.output.ForkClient$OperationalData.<init>(ForkClient.java:511)
  at
org.apache.maven.plugin.surefire.booterclient.output.ForkClient.processLine(ForkClient.java:209)
  at
org.apache.maven.plugin.surefire.booterclient.output.ForkClient.consumeLine(ForkClient.java:176)
  at
org.apache.maven.plugin.surefire.booterclient.output.ThreadedStreamConsumer$Pumper.run(ThreadedStreamConsumer.java:88)
  at java.lang.Thread.run(Thread.java:748)

Ignoring the class is great and the expected behavior... but "Corrupted
STDOUT" is still concerning.

On Wed, Apr 4, 2018 at 2:52 PM Christopher <ct...@apache.org> wrote:

> Enrico: Thanks for the info. I'm using the version inherited from the ASF
> parent POM 19, which turns out to be 2.20.1. Version 2.21.0 seems to fix
> the problem. Thanks!
>
> Tibor: No, I didn't get previous email. The Maven lists are too spammy for
> me to subscribe. That could be the reason. Please CC me if follow-up is
> needed. Thanks.
>
>
> On Sat, Mar 31, 2018 at 10:16 AM Enrico Olivelli <eo...@gmail.com>
> wrote:
>
>> This problem has been fixed in 2.21.0, what version are you running?
>>
>> Enrico
>>
>>
>> Il sab 31 mar 2018, 14:54 Tibor Digana <ti...@apache.org> ha
>> scritto:
>>
>>> Christopher,
>>>
>>> Did you receive my previous answer?
>>> There is some problem with email delivery.
>>> Tibor
>>>
>>> On Fri, Mar 30, 2018 at 8:58 PM, Christopher <ct...@apache.org>
>>> wrote:
>>>
>>> > Found a bug in maven-failsafe-plugin (
>>> > https://github.com/apache/accumulo/issues/411) when using groups
>>> (JUnit
>>> > categories):
>>> > http://maven.apache.org/surefire/maven-failsafe-
>>> > plugin/integration-test-mojo.html#groups
>>> > http://maven.apache.org/surefire/maven-failsafe-
>>> > plugin/integration-test-mojo.html#excludedGroups
>>> >
>>> > The problem is when the class corresponding to a configured group is
>>> not on
>>> > the class path. This is particularly problematic in multi-module
>>> builds, if
>>> > a group is specified which exists in one module but not another one.
>>> >
>>> > It appears the bug occurs because the configured group/category is
>>> loaded
>>> > from the class path, and then compared to the classes to find matching
>>> > tests. However, this seems backwards. If the tests were examined first,
>>> > then the name of the groups could be matched against the configuration
>>> > without loading to load the configured groups/category. Even if this
>>> is not
>>> > possible due to some limitation on how these groups are passed to
>>> JUnit or
>>> > some other restriction, a ClassNotFoundException (CNFE) should be
>>> handled
>>> > more sanely: For example, in the case of a CNFE for the "groups"
>>> parameter,
>>> > then it's clearly not possible for any tests to be included, so no
>>> tests
>>> > should run. In the case of a CNFE for "excludedGroups", then it's
>>> clearly
>>> > not possible for any tests to match, so none should be excluded.
>>> >
>>> > This bug has got me thinking about a related feature I'd like to see in
>>> > maven-surefire-plugin and maven-failsafe-plugin: I'd like to see
>>> groups /
>>> > excludedGroups specified by patterns, similar to `-Dtest=MyTest`
>>> instead of
>>> > `-Dtest=org.apache.project.package.subpackage.testing.MyTest`, it
>>> would be
>>> > nice to set `-Dgroups=MyGroup` rather than
>>> > `-Dgroups=org.apache.project.package.subpackage.testing.
>>> > categories.MyGroup`.
>>> > If groups could be specified as patterns, it would be more obvious
>>> that a
>>> > CNFE could not occur... if a pattern didn't match any classes, then it
>>> > wouldn't result in a CNFE... it would simply return an empty set.
>>> >
>>> > Thanks,
>>> >
>>> > Christopher
>>> >
>>>
>> --
>>
>>
>> -- Enrico Olivelli
>>
>

Re: Bug in processing surefire/failsafe groups

Posted by Christopher <ct...@apache.org>.
Enrico: Thanks for the info. I'm using the version inherited from the ASF
parent POM 19, which turns out to be 2.20.1. Version 2.21.0 seems to fix
the problem. Thanks!

Tibor: No, I didn't get previous email. The Maven lists are too spammy for
me to subscribe. That could be the reason. Please CC me if follow-up is
needed. Thanks.

On Sat, Mar 31, 2018 at 10:16 AM Enrico Olivelli <eo...@gmail.com>
wrote:

> This problem has been fixed in 2.21.0, what version are you running?
>
> Enrico
>
>
> Il sab 31 mar 2018, 14:54 Tibor Digana <ti...@apache.org> ha
> scritto:
>
>> Christopher,
>>
>> Did you receive my previous answer?
>> There is some problem with email delivery.
>> Tibor
>>
>> On Fri, Mar 30, 2018 at 8:58 PM, Christopher <ct...@apache.org> wrote:
>>
>> > Found a bug in maven-failsafe-plugin (
>> > https://github.com/apache/accumulo/issues/411) when using groups (JUnit
>> > categories):
>> > http://maven.apache.org/surefire/maven-failsafe-
>> > plugin/integration-test-mojo.html#groups
>> > http://maven.apache.org/surefire/maven-failsafe-
>> > plugin/integration-test-mojo.html#excludedGroups
>> >
>> > The problem is when the class corresponding to a configured group is
>> not on
>> > the class path. This is particularly problematic in multi-module
>> builds, if
>> > a group is specified which exists in one module but not another one.
>> >
>> > It appears the bug occurs because the configured group/category is
>> loaded
>> > from the class path, and then compared to the classes to find matching
>> > tests. However, this seems backwards. If the tests were examined first,
>> > then the name of the groups could be matched against the configuration
>> > without loading to load the configured groups/category. Even if this is
>> not
>> > possible due to some limitation on how these groups are passed to JUnit
>> or
>> > some other restriction, a ClassNotFoundException (CNFE) should be
>> handled
>> > more sanely: For example, in the case of a CNFE for the "groups"
>> parameter,
>> > then it's clearly not possible for any tests to be included, so no tests
>> > should run. In the case of a CNFE for "excludedGroups", then it's
>> clearly
>> > not possible for any tests to match, so none should be excluded.
>> >
>> > This bug has got me thinking about a related feature I'd like to see in
>> > maven-surefire-plugin and maven-failsafe-plugin: I'd like to see groups
>> /
>> > excludedGroups specified by patterns, similar to `-Dtest=MyTest`
>> instead of
>> > `-Dtest=org.apache.project.package.subpackage.testing.MyTest`, it would
>> be
>> > nice to set `-Dgroups=MyGroup` rather than
>> > `-Dgroups=org.apache.project.package.subpackage.testing.
>> > categories.MyGroup`.
>> > If groups could be specified as patterns, it would be more obvious that
>> a
>> > CNFE could not occur... if a pattern didn't match any classes, then it
>> > wouldn't result in a CNFE... it would simply return an empty set.
>> >
>> > Thanks,
>> >
>> > Christopher
>> >
>>
> --
>
>
> -- Enrico Olivelli
>

Re: Bug in processing surefire/failsafe groups

Posted by Enrico Olivelli <eo...@gmail.com>.
This problem has been fixed in 2.21.0, what version are you running?

Enrico

Il sab 31 mar 2018, 14:54 Tibor Digana <ti...@apache.org> ha scritto:

> Christopher,
>
> Did you receive my previous answer?
> There is some problem with email delivery.
> Tibor
>
> On Fri, Mar 30, 2018 at 8:58 PM, Christopher <ct...@apache.org> wrote:
>
> > Found a bug in maven-failsafe-plugin (
> > https://github.com/apache/accumulo/issues/411) when using groups (JUnit
> > categories):
> > http://maven.apache.org/surefire/maven-failsafe-
> > plugin/integration-test-mojo.html#groups
> > http://maven.apache.org/surefire/maven-failsafe-
> > plugin/integration-test-mojo.html#excludedGroups
> >
> > The problem is when the class corresponding to a configured group is not
> on
> > the class path. This is particularly problematic in multi-module builds,
> if
> > a group is specified which exists in one module but not another one.
> >
> > It appears the bug occurs because the configured group/category is loaded
> > from the class path, and then compared to the classes to find matching
> > tests. However, this seems backwards. If the tests were examined first,
> > then the name of the groups could be matched against the configuration
> > without loading to load the configured groups/category. Even if this is
> not
> > possible due to some limitation on how these groups are passed to JUnit
> or
> > some other restriction, a ClassNotFoundException (CNFE) should be handled
> > more sanely: For example, in the case of a CNFE for the "groups"
> parameter,
> > then it's clearly not possible for any tests to be included, so no tests
> > should run. In the case of a CNFE for "excludedGroups", then it's clearly
> > not possible for any tests to match, so none should be excluded.
> >
> > This bug has got me thinking about a related feature I'd like to see in
> > maven-surefire-plugin and maven-failsafe-plugin: I'd like to see groups /
> > excludedGroups specified by patterns, similar to `-Dtest=MyTest` instead
> of
> > `-Dtest=org.apache.project.package.subpackage.testing.MyTest`, it would
> be
> > nice to set `-Dgroups=MyGroup` rather than
> > `-Dgroups=org.apache.project.package.subpackage.testing.
> > categories.MyGroup`.
> > If groups could be specified as patterns, it would be more obvious that a
> > CNFE could not occur... if a pattern didn't match any classes, then it
> > wouldn't result in a CNFE... it would simply return an empty set.
> >
> > Thanks,
> >
> > Christopher
> >
>
-- 


-- Enrico Olivelli

Re: Bug in processing surefire/failsafe groups

Posted by Tibor Digana <ti...@apache.org>.
Christopher,

Did you receive my previous answer?
There is some problem with email delivery.
Tibor

On Fri, Mar 30, 2018 at 8:58 PM, Christopher <ct...@apache.org> wrote:

> Found a bug in maven-failsafe-plugin (
> https://github.com/apache/accumulo/issues/411) when using groups (JUnit
> categories):
> http://maven.apache.org/surefire/maven-failsafe-
> plugin/integration-test-mojo.html#groups
> http://maven.apache.org/surefire/maven-failsafe-
> plugin/integration-test-mojo.html#excludedGroups
>
> The problem is when the class corresponding to a configured group is not on
> the class path. This is particularly problematic in multi-module builds, if
> a group is specified which exists in one module but not another one.
>
> It appears the bug occurs because the configured group/category is loaded
> from the class path, and then compared to the classes to find matching
> tests. However, this seems backwards. If the tests were examined first,
> then the name of the groups could be matched against the configuration
> without loading to load the configured groups/category. Even if this is not
> possible due to some limitation on how these groups are passed to JUnit or
> some other restriction, a ClassNotFoundException (CNFE) should be handled
> more sanely: For example, in the case of a CNFE for the "groups" parameter,
> then it's clearly not possible for any tests to be included, so no tests
> should run. In the case of a CNFE for "excludedGroups", then it's clearly
> not possible for any tests to match, so none should be excluded.
>
> This bug has got me thinking about a related feature I'd like to see in
> maven-surefire-plugin and maven-failsafe-plugin: I'd like to see groups /
> excludedGroups specified by patterns, similar to `-Dtest=MyTest` instead of
> `-Dtest=org.apache.project.package.subpackage.testing.MyTest`, it would be
> nice to set `-Dgroups=MyGroup` rather than
> `-Dgroups=org.apache.project.package.subpackage.testing.
> categories.MyGroup`.
> If groups could be specified as patterns, it would be more obvious that a
> CNFE could not occur... if a pattern didn't match any classes, then it
> wouldn't result in a CNFE... it would simply return an empty set.
>
> Thanks,
>
> Christopher
>