You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@maven.apache.org by "Ondrej Zizka (JIRA)" <ji...@codehaus.org> on 2011/12/12 23:17:02 UTC
[jira] (SUREFIRE-809) Implement boolean expression to define test
group to be run.
Ondrej Zizka created SUREFIRE-809:
-------------------------------------
Summary: Implement boolean expression to define test group to be run.
Key: SUREFIRE-809
URL: https://jira.codehaus.org/browse/SUREFIRE-809
Project: Maven Surefire
Issue Type: Improvement
Components: Junit 4.x support
Affects Versions: 2.11
Reporter: Ondrej Zizka
This is an alternative to SUREFIRE-808.
Instead of having hard-coded filtering structure combining two lists.
an expression could be parsed and evaluated for each test.
Each test would be "tagged" using
{code}
@Categories({ MyCateg1.class, MyCateg2.class, ... })
{code}
Surefire's `group` config param would be an expression like:
{code}
<groups>( Ejb AND (CommonCriteria OR Security) ) AND NOT( Clustering )<groups>
{code}
Presence of a category of given name would be evaluated as true, absence of it as false.
Interface inheritance would be taken into account.
This mechanism would provide unlimited possibilities of grouping tests, and would be very beneficial for huge testuites counting thousands of tests.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://jira.codehaus.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] (SUREFIRE-809) Implement boolean expression to define test
group to be run.
Posted by "Kristian Rosenvold (JIRA)" <ji...@codehaus.org>.
[ https://jira.codehaus.org/browse/SUREFIRE-809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kristian Rosenvold updated SUREFIRE-809:
----------------------------------------
Issue Type: New Feature (was: Improvement)
> Implement boolean expression to define test group to be run.
> ------------------------------------------------------------
>
> Key: SUREFIRE-809
> URL: https://jira.codehaus.org/browse/SUREFIRE-809
> Project: Maven Surefire
> Issue Type: New Feature
> Components: Junit 4.x support
> Affects Versions: 2.11
> Reporter: Ondrej Zizka
> Attachments: BooleanExpression.g
>
>
> This is an alternative to SUREFIRE-808.
> Instead of having hard-coded filtering structure combining two lists.
> an expression could be parsed and evaluated for each test.
> Each test would be "tagged" using
> {code}
> @Categories({ MyCateg1.class, MyCateg2.class, ... })
> {code}
> Surefire's `group` config param would be an expression like:
> {code}
> <groups>( Ejb AND (CommonCriteria OR Security) ) AND NOT( Clustering )<groups>
> {code}
> Presence of a category of given name would be evaluated as true, absence of it as false.
> Interface inheritance would be taken into account.
> This mechanism would provide unlimited possibilities of grouping tests, and would be very beneficial for huge testuites counting thousands of tests.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://jira.codehaus.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] (SUREFIRE-809) Implement boolean expression to define test
group to be run.
Posted by "John Casey (JIRA)" <ji...@codehaus.org>.
[ https://jira.codehaus.org/browse/SUREFIRE-809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=289646#comment-289646 ]
John Casey commented on SUREFIRE-809:
-------------------------------------
In revId 1235805:
Just added tests to verify the grouping expressions work both in JUnit and TestNG. The implementation for TestNG offends my design sensibilities, but I didn't have many options for injecting the group-expression matcher logic.
In the end, I had to split out the group-expression support into a separate module, and handle it much the same way the testng-utils artifact is handled...injected directly into the test classpath for testng tests. There was a tad more fine tuning related to the new integration tests too, but it's pretty minor.
This feature should be complete now.
> Implement boolean expression to define test group to be run.
> ------------------------------------------------------------
>
> Key: SUREFIRE-809
> URL: https://jira.codehaus.org/browse/SUREFIRE-809
> Project: Maven Surefire
> Issue Type: New Feature
> Components: Junit 4.x support, TestNG support
> Affects Versions: 2.11
> Reporter: Ondrej Zizka
> Fix For: 2.12
>
> Attachments: BooleanExpression.g, category-expression.jj
>
>
> This is an alternative to SUREFIRE-808.
> Instead of having hard-coded filtering structure combining two lists.
> an expression could be parsed and evaluated for each test.
> Each test would be "tagged" using
> {code}
> @Categories({ MyCateg1.class, MyCateg2.class, ... })
> {code}
> Surefire's `group` config param would be an expression like:
> {code}
> <groups>( Ejb AND (CommonCriteria OR Security) ) AND NOT( Clustering )<groups>
> {code}
> Presence of a category of given name would be evaluated as true, absence of it as false.
> Interface inheritance would be taken into account.
> This mechanism would provide unlimited possibilities of grouping tests, and would be very beneficial for huge testuites counting thousands of tests.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://jira.codehaus.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] (SUREFIRE-809) Implement boolean expression to define test
group to be run.
Posted by "Ondrej Zizka (JIRA)" <ji...@codehaus.org>.
[ https://jira.codehaus.org/browse/SUREFIRE-809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=291378#comment-291378 ]
Ondrej Zizka commented on SUREFIRE-809:
---------------------------------------
Hi John, could you please add a link to docs and/or the test cases? Thx.
> Implement boolean expression to define test group to be run.
> ------------------------------------------------------------
>
> Key: SUREFIRE-809
> URL: https://jira.codehaus.org/browse/SUREFIRE-809
> Project: Maven Surefire
> Issue Type: New Feature
> Components: Junit 4.x support, TestNG support
> Affects Versions: 2.11
> Reporter: Ondrej Zizka
> Assignee: John Casey
> Fix For: 2.12
>
> Attachments: BooleanExpression.g, category-expression.jj
>
>
> This is an alternative to SUREFIRE-808.
> Instead of having hard-coded filtering structure combining two lists.
> an expression could be parsed and evaluated for each test.
> Each test would be "tagged" using
> {code}
> @Categories({ MyCateg1.class, MyCateg2.class, ... })
> {code}
> Surefire's `group` config param would be an expression like:
> {code}
> <groups>( Ejb AND (CommonCriteria OR Security) ) AND NOT( Clustering )<groups>
> {code}
> Presence of a category of given name would be evaluated as true, absence of it as false.
> Interface inheritance would be taken into account.
> This mechanism would provide unlimited possibilities of grouping tests, and would be very beneficial for huge testuites counting thousands of tests.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://jira.codehaus.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] (SUREFIRE-809) Implement boolean expression to define test
group to be run.
Posted by "John Casey (JIRA)" <ji...@codehaus.org>.
[ https://jira.codehaus.org/browse/SUREFIRE-809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=289558#comment-289558 ]
John Casey commented on SUREFIRE-809:
-------------------------------------
Applied in revIds: 1235412-1235416
All existing tests are passing; now I need to add a few new tests to verify the extra expression combination features.
> Implement boolean expression to define test group to be run.
> ------------------------------------------------------------
>
> Key: SUREFIRE-809
> URL: https://jira.codehaus.org/browse/SUREFIRE-809
> Project: Maven Surefire
> Issue Type: New Feature
> Components: Junit 4.x support
> Affects Versions: 2.11
> Reporter: Ondrej Zizka
> Attachments: BooleanExpression.g, category-expression.jj
>
>
> This is an alternative to SUREFIRE-808.
> Instead of having hard-coded filtering structure combining two lists.
> an expression could be parsed and evaluated for each test.
> Each test would be "tagged" using
> {code}
> @Categories({ MyCateg1.class, MyCateg2.class, ... })
> {code}
> Surefire's `group` config param would be an expression like:
> {code}
> <groups>( Ejb AND (CommonCriteria OR Security) ) AND NOT( Clustering )<groups>
> {code}
> Presence of a category of given name would be evaluated as true, absence of it as false.
> Interface inheritance would be taken into account.
> This mechanism would provide unlimited possibilities of grouping tests, and would be very beneficial for huge testuites counting thousands of tests.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://jira.codehaus.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] (SUREFIRE-809) Implement boolean expression to define test
group to be run.
Posted by "John Casey (JIRA)" <ji...@codehaus.org>.
[ https://jira.codehaus.org/browse/SUREFIRE-809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
John Casey updated SUREFIRE-809:
--------------------------------
Attachment: category-expression.jj
javacc grammar for parsing category statements.
> Implement boolean expression to define test group to be run.
> ------------------------------------------------------------
>
> Key: SUREFIRE-809
> URL: https://jira.codehaus.org/browse/SUREFIRE-809
> Project: Maven Surefire
> Issue Type: New Feature
> Components: Junit 4.x support
> Affects Versions: 2.11
> Reporter: Ondrej Zizka
> Attachments: BooleanExpression.g, category-expression.jj
>
>
> This is an alternative to SUREFIRE-808.
> Instead of having hard-coded filtering structure combining two lists.
> an expression could be parsed and evaluated for each test.
> Each test would be "tagged" using
> {code}
> @Categories({ MyCateg1.class, MyCateg2.class, ... })
> {code}
> Surefire's `group` config param would be an expression like:
> {code}
> <groups>( Ejb AND (CommonCriteria OR Security) ) AND NOT( Clustering )<groups>
> {code}
> Presence of a category of given name would be evaluated as true, absence of it as false.
> Interface inheritance would be taken into account.
> This mechanism would provide unlimited possibilities of grouping tests, and would be very beneficial for huge testuites counting thousands of tests.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://jira.codehaus.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] (SUREFIRE-809) Implement boolean expression to define test
group to be run.
Posted by "Kristian Rosenvold (JIRA)" <ji...@codehaus.org>.
[ https://jira.codehaus.org/browse/SUREFIRE-809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=289569#comment-289569 ]
Kristian Rosenvold commented on SUREFIRE-809:
---------------------------------------------
I think the filter approach sounds excellent, at least for the JUnit part. I don't really know about the testng details
> Implement boolean expression to define test group to be run.
> ------------------------------------------------------------
>
> Key: SUREFIRE-809
> URL: https://jira.codehaus.org/browse/SUREFIRE-809
> Project: Maven Surefire
> Issue Type: New Feature
> Components: Junit 4.x support
> Affects Versions: 2.11
> Reporter: Ondrej Zizka
> Attachments: BooleanExpression.g, category-expression.jj
>
>
> This is an alternative to SUREFIRE-808.
> Instead of having hard-coded filtering structure combining two lists.
> an expression could be parsed and evaluated for each test.
> Each test would be "tagged" using
> {code}
> @Categories({ MyCateg1.class, MyCateg2.class, ... })
> {code}
> Surefire's `group` config param would be an expression like:
> {code}
> <groups>( Ejb AND (CommonCriteria OR Security) ) AND NOT( Clustering )<groups>
> {code}
> Presence of a category of given name would be evaluated as true, absence of it as false.
> Interface inheritance would be taken into account.
> This mechanism would provide unlimited possibilities of grouping tests, and would be very beneficial for huge testuites counting thousands of tests.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://jira.codehaus.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] (SUREFIRE-809) Implement boolean expression to define test
group to be run.
Posted by "Ondrej Zizka (JIRA)" <ji...@codehaus.org>.
[ https://jira.codehaus.org/browse/SUREFIRE-809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ondrej Zizka updated SUREFIRE-809:
----------------------------------
Attachment: BooleanExpression.g
> Implement boolean expression to define test group to be run.
> ------------------------------------------------------------
>
> Key: SUREFIRE-809
> URL: https://jira.codehaus.org/browse/SUREFIRE-809
> Project: Maven Surefire
> Issue Type: Improvement
> Components: Junit 4.x support
> Affects Versions: 2.11
> Reporter: Ondrej Zizka
> Attachments: BooleanExpression.g
>
>
> This is an alternative to SUREFIRE-808.
> Instead of having hard-coded filtering structure combining two lists.
> an expression could be parsed and evaluated for each test.
> Each test would be "tagged" using
> {code}
> @Categories({ MyCateg1.class, MyCateg2.class, ... })
> {code}
> Surefire's `group` config param would be an expression like:
> {code}
> <groups>( Ejb AND (CommonCriteria OR Security) ) AND NOT( Clustering )<groups>
> {code}
> Presence of a category of given name would be evaluated as true, absence of it as false.
> Interface inheritance would be taken into account.
> This mechanism would provide unlimited possibilities of grouping tests, and would be very beneficial for huge testuites counting thousands of tests.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://jira.codehaus.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] (SUREFIRE-809) Implement boolean expression to define test
group to be run.
Posted by "John Casey (JIRA)" <ji...@codehaus.org>.
[ https://jira.codehaus.org/browse/SUREFIRE-809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
John Casey closed SUREFIRE-809.
-------------------------------
Resolution: Fixed
Ready to go into the next release attempt for 2.12.
> Implement boolean expression to define test group to be run.
> ------------------------------------------------------------
>
> Key: SUREFIRE-809
> URL: https://jira.codehaus.org/browse/SUREFIRE-809
> Project: Maven Surefire
> Issue Type: New Feature
> Components: Junit 4.x support, TestNG support
> Affects Versions: 2.11
> Reporter: Ondrej Zizka
> Assignee: John Casey
> Fix For: 2.12
>
> Attachments: BooleanExpression.g, category-expression.jj
>
>
> This is an alternative to SUREFIRE-808.
> Instead of having hard-coded filtering structure combining two lists.
> an expression could be parsed and evaluated for each test.
> Each test would be "tagged" using
> {code}
> @Categories({ MyCateg1.class, MyCateg2.class, ... })
> {code}
> Surefire's `group` config param would be an expression like:
> {code}
> <groups>( Ejb AND (CommonCriteria OR Security) ) AND NOT( Clustering )<groups>
> {code}
> Presence of a category of given name would be evaluated as true, absence of it as false.
> Interface inheritance would be taken into account.
> This mechanism would provide unlimited possibilities of grouping tests, and would be very beneficial for huge testuites counting thousands of tests.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://jira.codehaus.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] (SUREFIRE-809) Implement boolean expression to define test
group to be run.
Posted by "John Casey (JIRA)" <ji...@codehaus.org>.
[ https://jira.codehaus.org/browse/SUREFIRE-809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=288974#comment-288974 ]
John Casey commented on SUREFIRE-809:
-------------------------------------
I've (approximately) rewritten the attached grammar so it'll work with javacc, and created a series of matcher classes that will help me evaluate a category statement. I'm currently working on integrating this work into the junit and testng providers. The advantage of javacc here is that it generates a self-contained set classes, so we're not adding any new dependencies.
I haven't committed the grammar to trunk yet, so I'll attach it here. It's my first javacc grammar, so it could use some peer review!
> Implement boolean expression to define test group to be run.
> ------------------------------------------------------------
>
> Key: SUREFIRE-809
> URL: https://jira.codehaus.org/browse/SUREFIRE-809
> Project: Maven Surefire
> Issue Type: New Feature
> Components: Junit 4.x support
> Affects Versions: 2.11
> Reporter: Ondrej Zizka
> Attachments: BooleanExpression.g, category-expression.jj
>
>
> This is an alternative to SUREFIRE-808.
> Instead of having hard-coded filtering structure combining two lists.
> an expression could be parsed and evaluated for each test.
> Each test would be "tagged" using
> {code}
> @Categories({ MyCateg1.class, MyCateg2.class, ... })
> {code}
> Surefire's `group` config param would be an expression like:
> {code}
> <groups>( Ejb AND (CommonCriteria OR Security) ) AND NOT( Clustering )<groups>
> {code}
> Presence of a category of given name would be evaluated as true, absence of it as false.
> Interface inheritance would be taken into account.
> This mechanism would provide unlimited possibilities of grouping tests, and would be very beneficial for huge testuites counting thousands of tests.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://jira.codehaus.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] (SUREFIRE-809) Implement boolean expression to define test
group to be run.
Posted by "John Casey (JIRA)" <ji...@codehaus.org>.
[ https://jira.codehaus.org/browse/SUREFIRE-809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=289558#comment-289558 ]
John Casey edited comment on SUREFIRE-809 at 1/24/12 1:35 PM:
--------------------------------------------------------------
Applied in revIds: 1235412-1235416
All existing tests are passing; now I need to add a few new tests to verify the extra expression combination features.
BTW, the mechanisms used to introduce these matchers into JUnit and TestNG are:
JUnit: Modified FilterFactory to parse the group expressions, combine them into a single matcher object (with the excludes matcher section inverted). Then, wrapped the matcher in a JUnit Filter implementation, and passed back in place of the old comma-separated listing of Filter instances.
TestNG: Modified AbstractDirectConfigurator to parse group expressions in much the same way as above (with JUnit), then set the matcher instance STATICALLY on a new IMethodSelector implementation. This static setter is my workaround for the fact that TestNG classloads / instantiates method selectors, rather than allowing instances to be injected directly. I didn't want to use a file-based approach, in case multiple TestNG processes ran simultaneously in a build (the selector would have to know the file name). The old groups / excludeGroups settings are commented out in favor of the new method selector.
These implementations may be controversial / flawed. If so, please provide some guidance as to a preferred alternative mechanism, since I'm relatively unfamiliar with the deep workings of these test frameworks.
was (Author: jdcasey):
Applied in revIds: 1235412-1235416
All existing tests are passing; now I need to add a few new tests to verify the extra expression combination features.
> Implement boolean expression to define test group to be run.
> ------------------------------------------------------------
>
> Key: SUREFIRE-809
> URL: https://jira.codehaus.org/browse/SUREFIRE-809
> Project: Maven Surefire
> Issue Type: New Feature
> Components: Junit 4.x support
> Affects Versions: 2.11
> Reporter: Ondrej Zizka
> Attachments: BooleanExpression.g, category-expression.jj
>
>
> This is an alternative to SUREFIRE-808.
> Instead of having hard-coded filtering structure combining two lists.
> an expression could be parsed and evaluated for each test.
> Each test would be "tagged" using
> {code}
> @Categories({ MyCateg1.class, MyCateg2.class, ... })
> {code}
> Surefire's `group` config param would be an expression like:
> {code}
> <groups>( Ejb AND (CommonCriteria OR Security) ) AND NOT( Clustering )<groups>
> {code}
> Presence of a category of given name would be evaluated as true, absence of it as false.
> Interface inheritance would be taken into account.
> This mechanism would provide unlimited possibilities of grouping tests, and would be very beneficial for huge testuites counting thousands of tests.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://jira.codehaus.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] (SUREFIRE-809) Implement boolean expression to define test
group to be run.
Posted by "Kristian Rosenvold (JIRA)" <ji...@codehaus.org>.
[ https://jira.codehaus.org/browse/SUREFIRE-809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=289576#comment-289576 ]
Kristian Rosenvold commented on SUREFIRE-809:
---------------------------------------------
Oops. Sorry about running the release here. Let me know if it needs to be cancelled ;) Or maybe this should be in the release notes ;)
> Implement boolean expression to define test group to be run.
> ------------------------------------------------------------
>
> Key: SUREFIRE-809
> URL: https://jira.codehaus.org/browse/SUREFIRE-809
> Project: Maven Surefire
> Issue Type: New Feature
> Components: Junit 4.x support
> Affects Versions: 2.11
> Reporter: Ondrej Zizka
> Attachments: BooleanExpression.g, category-expression.jj
>
>
> This is an alternative to SUREFIRE-808.
> Instead of having hard-coded filtering structure combining two lists.
> an expression could be parsed and evaluated for each test.
> Each test would be "tagged" using
> {code}
> @Categories({ MyCateg1.class, MyCateg2.class, ... })
> {code}
> Surefire's `group` config param would be an expression like:
> {code}
> <groups>( Ejb AND (CommonCriteria OR Security) ) AND NOT( Clustering )<groups>
> {code}
> Presence of a category of given name would be evaluated as true, absence of it as false.
> Interface inheritance would be taken into account.
> This mechanism would provide unlimited possibilities of grouping tests, and would be very beneficial for huge testuites counting thousands of tests.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://jira.codehaus.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] (SUREFIRE-809) Implement boolean expression to define test
group to be run.
Posted by "John Casey (JIRA)" <ji...@codehaus.org>.
[ https://jira.codehaus.org/browse/SUREFIRE-809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=289558#comment-289558 ]
John Casey edited comment on SUREFIRE-809 at 1/24/12 1:36 PM:
--------------------------------------------------------------
Applied in revIds: 1235412-1235416
All existing tests are passing; now I need to add a few new tests to verify the extra expression combination features.
BTW, the mechanisms used to introduce these matchers into JUnit and TestNG are:
*JUnit:*
Modified FilterFactory to parse the group expressions, combine them into a single matcher object (with the excludes matcher section inverted). Then, wrapped the matcher in a JUnit Filter implementation, and passed back in place of the old comma-separated listing of Filter instances.
*TestNG:*
Modified AbstractDirectConfigurator to parse group expressions in much the same way as above (with JUnit), then set the matcher instance STATICALLY on a new IMethodSelector implementation. This static setter is my workaround for the fact that TestNG classloads / instantiates method selectors, rather than allowing instances to be injected directly. I didn't want to use a file-based approach, in case multiple TestNG processes ran simultaneously in a build (the selector would have to know the file name). The old groups / excludeGroups settings are commented out in favor of the new method selector.
These implementations may be controversial / flawed. If so, please provide some guidance as to a preferred alternative mechanism, since I'm relatively unfamiliar with the deep workings of these test frameworks.
was (Author: jdcasey):
Applied in revIds: 1235412-1235416
All existing tests are passing; now I need to add a few new tests to verify the extra expression combination features.
BTW, the mechanisms used to introduce these matchers into JUnit and TestNG are:
JUnit: Modified FilterFactory to parse the group expressions, combine them into a single matcher object (with the excludes matcher section inverted). Then, wrapped the matcher in a JUnit Filter implementation, and passed back in place of the old comma-separated listing of Filter instances.
TestNG: Modified AbstractDirectConfigurator to parse group expressions in much the same way as above (with JUnit), then set the matcher instance STATICALLY on a new IMethodSelector implementation. This static setter is my workaround for the fact that TestNG classloads / instantiates method selectors, rather than allowing instances to be injected directly. I didn't want to use a file-based approach, in case multiple TestNG processes ran simultaneously in a build (the selector would have to know the file name). The old groups / excludeGroups settings are commented out in favor of the new method selector.
These implementations may be controversial / flawed. If so, please provide some guidance as to a preferred alternative mechanism, since I'm relatively unfamiliar with the deep workings of these test frameworks.
> Implement boolean expression to define test group to be run.
> ------------------------------------------------------------
>
> Key: SUREFIRE-809
> URL: https://jira.codehaus.org/browse/SUREFIRE-809
> Project: Maven Surefire
> Issue Type: New Feature
> Components: Junit 4.x support
> Affects Versions: 2.11
> Reporter: Ondrej Zizka
> Attachments: BooleanExpression.g, category-expression.jj
>
>
> This is an alternative to SUREFIRE-808.
> Instead of having hard-coded filtering structure combining two lists.
> an expression could be parsed and evaluated for each test.
> Each test would be "tagged" using
> {code}
> @Categories({ MyCateg1.class, MyCateg2.class, ... })
> {code}
> Surefire's `group` config param would be an expression like:
> {code}
> <groups>( Ejb AND (CommonCriteria OR Security) ) AND NOT( Clustering )<groups>
> {code}
> Presence of a category of given name would be evaluated as true, absence of it as false.
> Interface inheritance would be taken into account.
> This mechanism would provide unlimited possibilities of grouping tests, and would be very beneficial for huge testuites counting thousands of tests.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://jira.codehaus.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] (SUREFIRE-809) Implement boolean expression to define test
group to be run.
Posted by "Kristian Rosenvold (JIRA)" <ji...@codehaus.org>.
[ https://jira.codehaus.org/browse/SUREFIRE-809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=285797#comment-285797 ]
Kristian Rosenvold commented on SUREFIRE-809:
---------------------------------------------
Anyone implementing this should note that it will be running on the fork-side, which means that we have to shade any external dependency. It's also a clear advantage to keep the dependency profile lean, since it affects fork startup-time - which is extremely lean in its current state.
> Implement boolean expression to define test group to be run.
> ------------------------------------------------------------
>
> Key: SUREFIRE-809
> URL: https://jira.codehaus.org/browse/SUREFIRE-809
> Project: Maven Surefire
> Issue Type: Improvement
> Components: Junit 4.x support
> Affects Versions: 2.11
> Reporter: Ondrej Zizka
> Attachments: BooleanExpression.g
>
>
> This is an alternative to SUREFIRE-808.
> Instead of having hard-coded filtering structure combining two lists.
> an expression could be parsed and evaluated for each test.
> Each test would be "tagged" using
> {code}
> @Categories({ MyCateg1.class, MyCateg2.class, ... })
> {code}
> Surefire's `group` config param would be an expression like:
> {code}
> <groups>( Ejb AND (CommonCriteria OR Security) ) AND NOT( Clustering )<groups>
> {code}
> Presence of a category of given name would be evaluated as true, absence of it as false.
> Interface inheritance would be taken into account.
> This mechanism would provide unlimited possibilities of grouping tests, and would be very beneficial for huge testuites counting thousands of tests.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://jira.codehaus.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] (SUREFIRE-809) Implement boolean expression to define test
group to be run.
Posted by "John Casey (JIRA)" <ji...@codehaus.org>.
[ https://jira.codehaus.org/browse/SUREFIRE-809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
John Casey updated SUREFIRE-809:
--------------------------------
Component/s: TestNG support
Fix Version/s: 2.12
Assignee: John Casey
> Implement boolean expression to define test group to be run.
> ------------------------------------------------------------
>
> Key: SUREFIRE-809
> URL: https://jira.codehaus.org/browse/SUREFIRE-809
> Project: Maven Surefire
> Issue Type: New Feature
> Components: Junit 4.x support, TestNG support
> Affects Versions: 2.11
> Reporter: Ondrej Zizka
> Assignee: John Casey
> Fix For: 2.12
>
> Attachments: BooleanExpression.g, category-expression.jj
>
>
> This is an alternative to SUREFIRE-808.
> Instead of having hard-coded filtering structure combining two lists.
> an expression could be parsed and evaluated for each test.
> Each test would be "tagged" using
> {code}
> @Categories({ MyCateg1.class, MyCateg2.class, ... })
> {code}
> Surefire's `group` config param would be an expression like:
> {code}
> <groups>( Ejb AND (CommonCriteria OR Security) ) AND NOT( Clustering )<groups>
> {code}
> Presence of a category of given name would be evaluated as true, absence of it as false.
> Interface inheritance would be taken into account.
> This mechanism would provide unlimited possibilities of grouping tests, and would be very beneficial for huge testuites counting thousands of tests.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://jira.codehaus.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] (SUREFIRE-809) Implement boolean expression to define test
group to be run.
Posted by "Ondrej Zizka (JIRA)" <ji...@codehaus.org>.
[ https://jira.codehaus.org/browse/SUREFIRE-809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=285701#comment-285701 ]
Ondrej Zizka commented on SUREFIRE-809:
---------------------------------------
For parsing and evaluation, JEXL library could be used - http://commons.apache.org/jexl/ .
Other approach is to use ANTLR grammar, parse the expression and code the evaluation. Example of grammar is attached.
Also, there's older 2.x release of JEP library which was under GPL: http://sourceforge.net/projects/jep/
> Implement boolean expression to define test group to be run.
> ------------------------------------------------------------
>
> Key: SUREFIRE-809
> URL: https://jira.codehaus.org/browse/SUREFIRE-809
> Project: Maven Surefire
> Issue Type: Improvement
> Components: Junit 4.x support
> Affects Versions: 2.11
> Reporter: Ondrej Zizka
> Attachments: BooleanExpression.g
>
>
> This is an alternative to SUREFIRE-808.
> Instead of having hard-coded filtering structure combining two lists.
> an expression could be parsed and evaluated for each test.
> Each test would be "tagged" using
> {code}
> @Categories({ MyCateg1.class, MyCateg2.class, ... })
> {code}
> Surefire's `group` config param would be an expression like:
> {code}
> <groups>( Ejb AND (CommonCriteria OR Security) ) AND NOT( Clustering )<groups>
> {code}
> Presence of a category of given name would be evaluated as true, absence of it as false.
> Interface inheritance would be taken into account.
> This mechanism would provide unlimited possibilities of grouping tests, and would be very beneficial for huge testuites counting thousands of tests.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://jira.codehaus.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira