You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@maven.apache.org by Christopher <ct...@apache.org> on 2016/08/31 00:57:38 UTC
JUnit categories in surefire/failsafe
tl;dr - A proposal for config independence for groups/excludeGroups
properties and some special keywords for ALL, NONE, and UNCATEGORIZED
groups.
***
In the Apache Accumulo project, we're currently in process of trying to
make use of JUnit categories to separate different classes of tests.
So, we're using the maven-surefire-plugin and maven-failsafe-plugin
configuration properties: groups, excludeGroups
One thing we noticed was that the user property is the same for both
plugins. This is a problem, because one cannot pass in a system property on
the command-line to affect one without affecting the other.
I propose that maven-surefire-plugin and maven-failsafe-plugin deprecate
the existing groups/excludeGroups properties, and replace them with
tests.groups/tests.excludeGroups, and it.groups/it.excludeGroups. (This
should probably be done for other shared properties as well.)
Users can simulate this by doing something like this:
<groups>${it.groups}</groups>
However, this may cause problems if the property is not defined.... I
haven't tested to be sure.
***
That leads me to a question and a second proposal:
Is there a way to specify uncategorized test classes? Or all test classes?
Or none?
If not, I would like to propose that some special keywords be created which
can represent:
_ALL_, _NONE_, _UNCATEGORIZED_ (or similar)
That way, users can do things like:
<groups>my.special.Category,_UNCATEGORIZED_</groups><!-- logical AND -->
<excludeGroups>_NONE_</excludeGroups>
or
<groups>_NONE_</groups>
or
<groups>_ALL_</groups>
These keywords may require some support from the underlying test framework,
like JUnit, so I can understand if these keywords cannot happen.
Even if the keywords cannot be made to work, I still think it'd be good to
deprecate-and-separate the properties for the two plugins, so they can be
controlled independently with user properties.
Thanks.
Re: JUnit categories in surefire/failsafe
Posted by Tibor Digana <ti...@apache.org>.
Hi Christopher,
Right the first test hangs but it does not call setOut(), after having
a look in your code.
UnusedWalDoesntCloseReplicationStatusIT
>>I wasn't aware it was hanging.
Running org.apache.accumulo.test.replication.UnusedWalDoesntCloseReplicationStatusIT
Build timed out (after 300 minutes). Marking the build as aborted.
Build was aborted
Archiving artifacts
No prior successful build to compare, so performing full copy of artifacts
Finished: ABORTED
--
Cheers
Tibor
--
View this message in context: http://maven.40175.n5.nabble.com/JUnit-categories-in-surefire-failsafe-tp5879500p5879668.html
Sent from the Maven Developers mailing list archive at Nabble.com.
Re: JUnit categories in surefire/failsafe
Posted by Christopher <ct...@apache.org>.
On Wed, Aug 31, 2016 at 4:23 AM Tibor Digana <ti...@apache.org> wrote:
> Hi Christopher,
>
> Some offtopic. I will answer your email but first I have a question for you
> and Accumulo project.
> I visited Accumulo cca one week ago. Why the build [1] hangs on IT test
> executed by maven-failsafe-plugin:2.19.1?
>
I wasn't aware it was hanging. That particular Jenkins job is an
experimental attempt by one of our developers to try to run all of our
tests in parallel, and it's quite spammy when there's a failure. I don't
track it. I'm tracking some of our other jobs.
Usually, though, our problem is with our tests, not necessarily failsafe,
but I can't be sure unless you point me directly to the failure, instead of
the job page.
> I asked INFRA team to display Thread Dump button. Do you see this button,
> can you grap the thread dump? I would like to see what's going on in
> Failsafe plugin. We have such issue [3] but now it most probably is user
> error because the developer overrides std/out via System.setOut() but this
> hypothesis has not been confirmed yet because it's too early.
>
>
I don't see the button, but our tests do set the surefire/maven option to
redirect test output to a file. I didn't think we're doing System.setOut()
at all, but it looks like we are doing it in one failsafe IT, and one
surefire test. I'll have to investigate these a bit more. I think there's
probably a better way to do those tests.
I'm adding our dev list to the distro. Let's continue this conversation
there, if necessary, and bring it back to the Maven list if we can figure
out what's going on.
> [1] https://builds.apache.org/job/Accumulo-master-IT/
> [2] https://wiki.jenkins-ci.org/display/JENKINS/Thread+Dump+Action+Plugin
> [3] https://issues.apache.org/jira/browse/SUREFIRE-1193
>
>
>
> On Wed, Aug 31, 2016 at 3:00 AM, Christopher [via Maven] <
> ml-node+s40175n5879500h40@n5.nabble.com> wrote:
>
> > tl;dr - A proposal for config independence for groups/excludeGroups
> > properties and some special keywords for ALL, NONE, and UNCATEGORIZED
> > groups.
> >
> > ***
> >
> > In the Apache Accumulo project, we're currently in process of trying to
> > make use of JUnit categories to separate different classes of tests.
> >
> > So, we're using the maven-surefire-plugin and maven-failsafe-plugin
> > configuration properties: groups, excludeGroups
> >
> > One thing we noticed was that the user property is the same for both
> > plugins. This is a problem, because one cannot pass in a system property
> > on
> > the command-line to affect one without affecting the other.
> >
> > I propose that maven-surefire-plugin and maven-failsafe-plugin deprecate
> > the existing groups/excludeGroups properties, and replace them with
> > tests.groups/tests.excludeGroups, and it.groups/it.excludeGroups. (This
> > should probably be done for other shared properties as well.)
> >
> > Users can simulate this by doing something like this:
> > <groups>${it.groups}</groups>
> >
> > However, this may cause problems if the property is not defined.... I
> > haven't tested to be sure.
> >
> > ***
> >
> > That leads me to a question and a second proposal:
> > Is there a way to specify uncategorized test classes? Or all test
> classes?
> > Or none?
> >
> > If not, I would like to propose that some special keywords be created
> > which
> > can represent:
> > _ALL_, _NONE_, _UNCATEGORIZED_ (or similar)
> >
> > That way, users can do things like:
> > <groups>my.special.Category,_UNCATEGORIZED_</groups><!-- logical AND -->
> > <excludeGroups>_NONE_</excludeGroups>
> > or
> > <groups>_NONE_</groups>
> > or
> > <groups>_ALL_</groups>
> >
> > These keywords may require some support from the underlying test
> > framework,
> > like JUnit, so I can understand if these keywords cannot happen.
> >
> > Even if the keywords cannot be made to work, I still think it'd be good
> to
> > deprecate-and-separate the properties for the two plugins, so they can be
> > controlled independently with user properties.
> >
> > Thanks.
> >
> >
> > ------------------------------
> > If you reply to this email, your message will be added to the discussion
> > below:
> > http://maven.40175.n5.nabble.com/JUnit-categories-in-
> > surefire-failsafe-tp5879500.html
> > To start a new topic under Maven Developers, email
> > ml-node+s40175n142166h86@n5.nabble.com
> > To unsubscribe from Maven Developers, click here
> > <
> http://maven.40175.n5.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=142166&code=dGlib3JkaWdhbmFAYXBhY2hlLm9yZ3wxNDIxNjZ8LTI4OTQ5MjEwMg==
> >
> > .
> > NAML
> > <
> http://maven.40175.n5.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml
> >
> >
>
>
>
>
> --
> View this message in context:
> http://maven.40175.n5.nabble.com/JUnit-categories-in-surefire-failsafe-tp5879500p5879510.html
> Sent from the Maven Developers mailing list archive at Nabble.com.
Re: JUnit categories in surefire/failsafe
Posted by Tibor Digana <ti...@googlemail.com>.
>>because Maven might just treat it as though it weren't set at all, and ignore it.
One of our user pointed to the issue with empty property and I said it
is bug in Maven.
Somebody should fix it in Maven.
> The most useful of the three would be UNCATEGORIZED, so it can be combined
> with other categories and I'd expect that would require JUnit support
> underneath.
No it does not need JUnit support since it is our Filters we implement
and JUnit uses our filter in some kind of preprocessing of Runners.
This is doable and you can open pull in GitHub, but the test result
might be quite unusual:
Running UncategorizedTest
Tests: 0, Errors: 0, ... Skipped:5
--
Cheers
Tibor
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: JUnit categories in surefire/failsafe
Posted by Christopher <ct...@apache.org>.
On Wed, Aug 31, 2016 at 4:43 AM Tibor Digana <ti...@apache.org> wrote:
> >>One thing we noticed was that the user property is the same for both
> plugins.
> We are aware of this and we will break the backwards compatibility.
> This change is planned in Surefire 3.0.
> I am thinking of temporary fix but better not to do it.
> Please reference and use these user/properties since they will be in 3.0:
> <groups>${surefire.groups}</groups>
> <groups>${failsafe.groups}</groups>
>
>
Thanks. We'll do that. We had created our own, but it's better to use the
property names which will appear in future versions of the plugins.
> Do you still need "_ALL_, _NONE_, _UNCATEGORIZED_"?
>
>
>
Not at the moment, and these were secondary. After thinking about it, "ALL"
and "NONE" aren't needed, so long as it's easy enough to set the
groups/excludedGroups properties to empty with an override. My main concern
about those two options were that it might be hard to set a property to
"empty" to override a default non-empty value, because Maven might just
treat it as though it weren't set at all, and ignore it. I'd have to
experiment to see if that's the case or if it properly sets it to null or
empty string or whatever when it's in the DOM without a value or if it's a
system property without a value.
The most useful of the three would be UNCATEGORIZED, so it can be combined
with other categories and I'd expect that would require JUnit support
underneath.
>
> On Wed, Aug 31, 2016 at 10:23 AM, Tibor Digana <ti...@apache.org>
> wrote:
>
> > Hi Christopher,
> >
> > Some offtopic. I will answer your email but first I have a question for
> you
> > and Accumulo project.
> > I visited Accumulo cca one week ago. Why the build [1] hangs on IT test
> > executed by maven-failsafe-plugin:2.19.1?
> > I asked INFRA team to display Thread Dump button. Do you see this button,
> > can you grap the thread dump? I would like to see what's going on in
> > Failsafe plugin. We have such issue [3] but now it most probably is user
> > error because the developer overrides std/out via System.setOut() but
> this
> > hypothesis has not been confirmed yet because it's too early.
> >
> > [1] https://builds.apache.org/job/Accumulo-master-IT/
> > [2]
> https://wiki.jenkins-ci.org/display/JENKINS/Thread+Dump+Action+Plugin
> > [3] https://issues.apache.org/jira/browse/SUREFIRE-1193
> >
> >
> >
> > On Wed, Aug 31, 2016 at 3:00 AM, Christopher [via Maven] <
> > ml-node+s40175n5879500h40@n5.nabble.com> wrote:
> >
> > > tl;dr - A proposal for config independence for groups/excludeGroups
> > > properties and some special keywords for ALL, NONE, and UNCATEGORIZED
> > > groups.
> > >
> > > ***
> > >
> > > In the Apache Accumulo project, we're currently in process of trying to
> > > make use of JUnit categories to separate different classes of tests.
> > >
> > > So, we're using the maven-surefire-plugin and maven-failsafe-plugin
> > > configuration properties: groups, excludeGroups
> > >
> > > One thing we noticed was that the user property is the same for both
> > > plugins. This is a problem, because one cannot pass in a system
> property
> > > on
> > > the command-line to affect one without affecting the other.
> > >
> > > I propose that maven-surefire-plugin and maven-failsafe-plugin
> deprecate
> > > the existing groups/excludeGroups properties, and replace them with
> > > tests.groups/tests.excludeGroups, and it.groups/it.excludeGroups. (This
> > > should probably be done for other shared properties as well.)
> > >
> > > Users can simulate this by doing something like this:
> > > <groups>${it.groups}</groups>
> > >
> > > However, this may cause problems if the property is not defined.... I
> > > haven't tested to be sure.
> > >
> > > ***
> > >
> > > That leads me to a question and a second proposal:
> > > Is there a way to specify uncategorized test classes? Or all test
> > classes?
> > > Or none?
> > >
> > > If not, I would like to propose that some special keywords be created
> > > which
> > > can represent:
> > > _ALL_, _NONE_, _UNCATEGORIZED_ (or similar)
> > >
> > > That way, users can do things like:
> > > <groups>my.special.Category,_UNCATEGORIZED_</groups><!-- logical AND
> -->
> > > <excludeGroups>_NONE_</excludeGroups>
> > > or
> > > <groups>_NONE_</groups>
> > > or
> > > <groups>_ALL_</groups>
> > >
> > > These keywords may require some support from the underlying test
> > > framework,
> > > like JUnit, so I can understand if these keywords cannot happen.
> > >
> > > Even if the keywords cannot be made to work, I still think it'd be good
> > to
> > > deprecate-and-separate the properties for the two plugins, so they can
> be
> > > controlled independently with user properties.
> > >
> > > Thanks.
> > >
> > >
> > > ------------------------------
> > > If you reply to this email, your message will be added to the
> discussion
> > > below:
> > > http://maven.40175.n5.nabble.com/JUnit-categories-in-
> > > surefire-failsafe-tp5879500.html
> > > To start a new topic under Maven Developers, email
> > > ml-node+s40175n142166h86@n5.nabble.com
> > > To unsubscribe from Maven Developers, click here
> > > <http://maven.40175.n5.nabble.com/template/NamlServlet.jtp?
> > macro=unsubscribe_by_code&node=142166&code=dGlib3JkaWdhbmFAYXBhY2hlLm9yZ3
> > wxNDIxNjZ8LTI4OTQ5MjEwMg==>
> > > .
> > > NAML
> > > <http://maven.40175.n5.nabble.com/template/NamlServlet.jtp?
> > macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&
> > base=nabble.naml.namespaces.BasicNamespace-nabble.view.
> > web.template.NabbleNamespace-nabble.view.web.template.
> > NodeNamespace&breadcrumbs=notify_subscribers%21nabble%
> > 3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_
> > instant_email%21nabble%3Aemail.naml>
> > >
> >
> >
> >
> >
> > --
> > View this message in context: http://maven.40175.n5.nabble.
> > com/JUnit-categories-in-surefire-failsafe-tp5879500p5879510.html
> > Sent from the Maven Developers mailing list archive at Nabble.com.
>
Re: JUnit categories in surefire/failsafe
Posted by Tibor Digana <ti...@apache.org>.
>>One thing we noticed was that the user property is the same for both
plugins.
We are aware of this and we will break the backwards compatibility.
This change is planned in Surefire 3.0.
I am thinking of temporary fix but better not to do it.
Please reference and use these user/properties since they will be in 3.0:
<groups>${surefire.groups}</groups>
<groups>${failsafe.groups}</groups>
Do you still need "_ALL_, _NONE_, _UNCATEGORIZED_"?
On Wed, Aug 31, 2016 at 10:23 AM, Tibor Digana <ti...@apache.org>
wrote:
> Hi Christopher,
>
> Some offtopic. I will answer your email but first I have a question for you
> and Accumulo project.
> I visited Accumulo cca one week ago. Why the build [1] hangs on IT test
> executed by maven-failsafe-plugin:2.19.1?
> I asked INFRA team to display Thread Dump button. Do you see this button,
> can you grap the thread dump? I would like to see what's going on in
> Failsafe plugin. We have such issue [3] but now it most probably is user
> error because the developer overrides std/out via System.setOut() but this
> hypothesis has not been confirmed yet because it's too early.
>
> [1] https://builds.apache.org/job/Accumulo-master-IT/
> [2] https://wiki.jenkins-ci.org/display/JENKINS/Thread+Dump+Action+Plugin
> [3] https://issues.apache.org/jira/browse/SUREFIRE-1193
>
>
>
> On Wed, Aug 31, 2016 at 3:00 AM, Christopher [via Maven] <
> ml-node+s40175n5879500h40@n5.nabble.com> wrote:
>
> > tl;dr - A proposal for config independence for groups/excludeGroups
> > properties and some special keywords for ALL, NONE, and UNCATEGORIZED
> > groups.
> >
> > ***
> >
> > In the Apache Accumulo project, we're currently in process of trying to
> > make use of JUnit categories to separate different classes of tests.
> >
> > So, we're using the maven-surefire-plugin and maven-failsafe-plugin
> > configuration properties: groups, excludeGroups
> >
> > One thing we noticed was that the user property is the same for both
> > plugins. This is a problem, because one cannot pass in a system property
> > on
> > the command-line to affect one without affecting the other.
> >
> > I propose that maven-surefire-plugin and maven-failsafe-plugin deprecate
> > the existing groups/excludeGroups properties, and replace them with
> > tests.groups/tests.excludeGroups, and it.groups/it.excludeGroups. (This
> > should probably be done for other shared properties as well.)
> >
> > Users can simulate this by doing something like this:
> > <groups>${it.groups}</groups>
> >
> > However, this may cause problems if the property is not defined.... I
> > haven't tested to be sure.
> >
> > ***
> >
> > That leads me to a question and a second proposal:
> > Is there a way to specify uncategorized test classes? Or all test
> classes?
> > Or none?
> >
> > If not, I would like to propose that some special keywords be created
> > which
> > can represent:
> > _ALL_, _NONE_, _UNCATEGORIZED_ (or similar)
> >
> > That way, users can do things like:
> > <groups>my.special.Category,_UNCATEGORIZED_</groups><!-- logical AND -->
> > <excludeGroups>_NONE_</excludeGroups>
> > or
> > <groups>_NONE_</groups>
> > or
> > <groups>_ALL_</groups>
> >
> > These keywords may require some support from the underlying test
> > framework,
> > like JUnit, so I can understand if these keywords cannot happen.
> >
> > Even if the keywords cannot be made to work, I still think it'd be good
> to
> > deprecate-and-separate the properties for the two plugins, so they can be
> > controlled independently with user properties.
> >
> > Thanks.
> >
> >
> > ------------------------------
> > If you reply to this email, your message will be added to the discussion
> > below:
> > http://maven.40175.n5.nabble.com/JUnit-categories-in-
> > surefire-failsafe-tp5879500.html
> > To start a new topic under Maven Developers, email
> > ml-node+s40175n142166h86@n5.nabble.com
> > To unsubscribe from Maven Developers, click here
> > <http://maven.40175.n5.nabble.com/template/NamlServlet.jtp?
> macro=unsubscribe_by_code&node=142166&code=dGlib3JkaWdhbmFAYXBhY2hlLm9yZ3
> wxNDIxNjZ8LTI4OTQ5MjEwMg==>
> > .
> > NAML
> > <http://maven.40175.n5.nabble.com/template/NamlServlet.jtp?
> macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&
> base=nabble.naml.namespaces.BasicNamespace-nabble.view.
> web.template.NabbleNamespace-nabble.view.web.template.
> NodeNamespace&breadcrumbs=notify_subscribers%21nabble%
> 3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_
> instant_email%21nabble%3Aemail.naml>
> >
>
>
>
>
> --
> View this message in context: http://maven.40175.n5.nabble.
> com/JUnit-categories-in-surefire-failsafe-tp5879500p5879510.html
> Sent from the Maven Developers mailing list archive at Nabble.com.
Re: JUnit categories in surefire/failsafe
Posted by Christopher <ct...@apache.org>.
On Wed, Aug 31, 2016 at 4:23 AM Tibor Digana <ti...@apache.org> wrote:
> Hi Christopher,
>
> Some offtopic. I will answer your email but first I have a question for you
> and Accumulo project.
> I visited Accumulo cca one week ago. Why the build [1] hangs on IT test
> executed by maven-failsafe-plugin:2.19.1?
>
I wasn't aware it was hanging. That particular Jenkins job is an
experimental attempt by one of our developers to try to run all of our
tests in parallel, and it's quite spammy when there's a failure. I don't
track it. I'm tracking some of our other jobs.
Usually, though, our problem is with our tests, not necessarily failsafe,
but I can't be sure unless you point me directly to the failure, instead of
the job page.
> I asked INFRA team to display Thread Dump button. Do you see this button,
> can you grap the thread dump? I would like to see what's going on in
> Failsafe plugin. We have such issue [3] but now it most probably is user
> error because the developer overrides std/out via System.setOut() but this
> hypothesis has not been confirmed yet because it's too early.
>
>
I don't see the button, but our tests do set the surefire/maven option to
redirect test output to a file. I didn't think we're doing System.setOut()
at all, but it looks like we are doing it in one failsafe IT, and one
surefire test. I'll have to investigate these a bit more. I think there's
probably a better way to do those tests.
I'm adding our dev list to the distro. Let's continue this conversation
there, if necessary, and bring it back to the Maven list if we can figure
out what's going on.
> [1] https://builds.apache.org/job/Accumulo-master-IT/
> [2] https://wiki.jenkins-ci.org/display/JENKINS/Thread+Dump+Action+Plugin
> [3] https://issues.apache.org/jira/browse/SUREFIRE-1193
>
>
>
> On Wed, Aug 31, 2016 at 3:00 AM, Christopher [via Maven] <
> ml-node+s40175n5879500h40@n5.nabble.com> wrote:
>
> > tl;dr - A proposal for config independence for groups/excludeGroups
> > properties and some special keywords for ALL, NONE, and UNCATEGORIZED
> > groups.
> >
> > ***
> >
> > In the Apache Accumulo project, we're currently in process of trying to
> > make use of JUnit categories to separate different classes of tests.
> >
> > So, we're using the maven-surefire-plugin and maven-failsafe-plugin
> > configuration properties: groups, excludeGroups
> >
> > One thing we noticed was that the user property is the same for both
> > plugins. This is a problem, because one cannot pass in a system property
> > on
> > the command-line to affect one without affecting the other.
> >
> > I propose that maven-surefire-plugin and maven-failsafe-plugin deprecate
> > the existing groups/excludeGroups properties, and replace them with
> > tests.groups/tests.excludeGroups, and it.groups/it.excludeGroups. (This
> > should probably be done for other shared properties as well.)
> >
> > Users can simulate this by doing something like this:
> > <groups>${it.groups}</groups>
> >
> > However, this may cause problems if the property is not defined.... I
> > haven't tested to be sure.
> >
> > ***
> >
> > That leads me to a question and a second proposal:
> > Is there a way to specify uncategorized test classes? Or all test
> classes?
> > Or none?
> >
> > If not, I would like to propose that some special keywords be created
> > which
> > can represent:
> > _ALL_, _NONE_, _UNCATEGORIZED_ (or similar)
> >
> > That way, users can do things like:
> > <groups>my.special.Category,_UNCATEGORIZED_</groups><!-- logical AND -->
> > <excludeGroups>_NONE_</excludeGroups>
> > or
> > <groups>_NONE_</groups>
> > or
> > <groups>_ALL_</groups>
> >
> > These keywords may require some support from the underlying test
> > framework,
> > like JUnit, so I can understand if these keywords cannot happen.
> >
> > Even if the keywords cannot be made to work, I still think it'd be good
> to
> > deprecate-and-separate the properties for the two plugins, so they can be
> > controlled independently with user properties.
> >
> > Thanks.
> >
> >
> > ------------------------------
> > If you reply to this email, your message will be added to the discussion
> > below:
> > http://maven.40175.n5.nabble.com/JUnit-categories-in-
> > surefire-failsafe-tp5879500.html
> > To start a new topic under Maven Developers, email
> > ml-node+s40175n142166h86@n5.nabble.com
> > To unsubscribe from Maven Developers, click here
> > <
> http://maven.40175.n5.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=142166&code=dGlib3JkaWdhbmFAYXBhY2hlLm9yZ3wxNDIxNjZ8LTI4OTQ5MjEwMg==
> >
> > .
> > NAML
> > <
> http://maven.40175.n5.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml
> >
> >
>
>
>
>
> --
> View this message in context:
> http://maven.40175.n5.nabble.com/JUnit-categories-in-surefire-failsafe-tp5879500p5879510.html
> Sent from the Maven Developers mailing list archive at Nabble.com.
Re: JUnit categories in surefire/failsafe
Posted by Tibor Digana <ti...@apache.org>.
Hi Christopher,
Some offtopic. I will answer your email but first I have a question for you
and Accumulo project.
I visited Accumulo cca one week ago. Why the build [1] hangs on IT test
executed by maven-failsafe-plugin:2.19.1?
I asked INFRA team to display Thread Dump button. Do you see this button,
can you grap the thread dump? I would like to see what's going on in
Failsafe plugin. We have such issue [3] but now it most probably is user
error because the developer overrides std/out via System.setOut() but this
hypothesis has not been confirmed yet because it's too early.
[1] https://builds.apache.org/job/Accumulo-master-IT/
[2] https://wiki.jenkins-ci.org/display/JENKINS/Thread+Dump+Action+Plugin
[3] https://issues.apache.org/jira/browse/SUREFIRE-1193
On Wed, Aug 31, 2016 at 3:00 AM, Christopher [via Maven] <
ml-node+s40175n5879500h40@n5.nabble.com> wrote:
> tl;dr - A proposal for config independence for groups/excludeGroups
> properties and some special keywords for ALL, NONE, and UNCATEGORIZED
> groups.
>
> ***
>
> In the Apache Accumulo project, we're currently in process of trying to
> make use of JUnit categories to separate different classes of tests.
>
> So, we're using the maven-surefire-plugin and maven-failsafe-plugin
> configuration properties: groups, excludeGroups
>
> One thing we noticed was that the user property is the same for both
> plugins. This is a problem, because one cannot pass in a system property
> on
> the command-line to affect one without affecting the other.
>
> I propose that maven-surefire-plugin and maven-failsafe-plugin deprecate
> the existing groups/excludeGroups properties, and replace them with
> tests.groups/tests.excludeGroups, and it.groups/it.excludeGroups. (This
> should probably be done for other shared properties as well.)
>
> Users can simulate this by doing something like this:
> <groups>${it.groups}</groups>
>
> However, this may cause problems if the property is not defined.... I
> haven't tested to be sure.
>
> ***
>
> That leads me to a question and a second proposal:
> Is there a way to specify uncategorized test classes? Or all test classes?
> Or none?
>
> If not, I would like to propose that some special keywords be created
> which
> can represent:
> _ALL_, _NONE_, _UNCATEGORIZED_ (or similar)
>
> That way, users can do things like:
> <groups>my.special.Category,_UNCATEGORIZED_</groups><!-- logical AND -->
> <excludeGroups>_NONE_</excludeGroups>
> or
> <groups>_NONE_</groups>
> or
> <groups>_ALL_</groups>
>
> These keywords may require some support from the underlying test
> framework,
> like JUnit, so I can understand if these keywords cannot happen.
>
> Even if the keywords cannot be made to work, I still think it'd be good to
> deprecate-and-separate the properties for the two plugins, so they can be
> controlled independently with user properties.
>
> Thanks.
>
>
> ------------------------------
> If you reply to this email, your message will be added to the discussion
> below:
> http://maven.40175.n5.nabble.com/JUnit-categories-in-
> surefire-failsafe-tp5879500.html
> To start a new topic under Maven Developers, email
> ml-node+s40175n142166h86@n5.nabble.com
> To unsubscribe from Maven Developers, click here
> <http://maven.40175.n5.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=142166&code=dGlib3JkaWdhbmFAYXBhY2hlLm9yZ3wxNDIxNjZ8LTI4OTQ5MjEwMg==>
> .
> NAML
> <http://maven.40175.n5.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>
--
View this message in context: http://maven.40175.n5.nabble.com/JUnit-categories-in-surefire-failsafe-tp5879500p5879510.html
Sent from the Maven Developers mailing list archive at Nabble.com.