You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@maven.apache.org by "Alexander Klimetschek (JIRA)" <ji...@codehaus.org> on 2011/02/17 14:40:22 UTC
[jira] Created: (SUREFIRE-701) showSuccess option for test reports:
show reports only for failed tests
showSuccess option for test reports: show reports only for failed tests
-----------------------------------------------------------------------
Key: SUREFIRE-701
URL: http://jira.codehaus.org/browse/SUREFIRE-701
Project: Maven Surefire
Issue Type: Improvement
Components: Maven Surefire Plugin
Affects Versions: 2.7.3
Reporter: Alexander Klimetschek
Attachments: showsuccess.patch
Use case:
When you have a few tests, and some of them fail during development/refactoring, you find yourself repeatedly looking into target/surefire-reports to open the test report (txt) files with the errors and stack traces to analyze them. However, by default reports will be written also for successful tests, and both in txt and xml format (the xml files can already be omitted using disableXmlReport=false). This makes it hard to find the files that contain the failed tests.
Solution proposal:
Just as the maven-surefire-report-plugin already has a showSuccess option, that, if set to "false", will not report successful tests, the same should be available for the maven-surefire-plugin and the normal test goal.
Either set "-DshowSuccess=false" on the command line or define "<showSuccess>false</showSuccess>" in the maven-surefire-plugin configuration of the pom, and reports for successful tests should not be written to the reports directory.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (SUREFIRE-701) showSuccess option for test reports:
show reports only for failed tests
Posted by "Kristian Rosenvold (JIRA)" <ji...@codehaus.org>.
[ http://jira.codehaus.org/browse/SUREFIRE-701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kristian Rosenvold updated SUREFIRE-701:
----------------------------------------
Issue Type: New Feature (was: Improvement)
> showSuccess option for test reports: show reports only for failed tests
> -----------------------------------------------------------------------
>
> Key: SUREFIRE-701
> URL: http://jira.codehaus.org/browse/SUREFIRE-701
> Project: Maven Surefire
> Issue Type: New Feature
> Components: Maven Surefire Plugin
> Affects Versions: 2.7.3
> Reporter: Alexander Klimetschek
> Attachments: showsuccess.patch
>
>
> Use case:
> When you have a few tests, and some of them fail during development/refactoring, you find yourself repeatedly looking into target/surefire-reports to open the test report (txt) files with the errors and stack traces to analyze them. However, by default reports will be written also for successful tests, and both in txt and xml format (the xml files can already be omitted using disableXmlReport=false). This makes it hard to find the files that contain the failed tests.
> Solution proposal:
> Just as the maven-surefire-report-plugin already has a showSuccess option, that, if set to "false", will not report successful tests, the same should be available for the maven-surefire-plugin and the normal test goal.
> Either set "-DshowSuccess=false" on the command line or define "<showSuccess>false</showSuccess>" in the maven-surefire-plugin configuration of the pom, and reports for successful tests should not be written to the reports directory.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (SUREFIRE-701) showSuccess option for test
reports: show reports only for failed tests
Posted by "Alexander Klimetschek (JIRA)" <ji...@codehaus.org>.
[ http://jira.codehaus.org/browse/SUREFIRE-701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=256569#action_256569 ]
Alexander Klimetschek commented on SUREFIRE-701:
------------------------------------------------
> This should be the default behavior
I agree, but I didn't want to go that far in the patch ;-)
> showSuccess option for test reports: show reports only for failed tests
> -----------------------------------------------------------------------
>
> Key: SUREFIRE-701
> URL: http://jira.codehaus.org/browse/SUREFIRE-701
> Project: Maven Surefire
> Issue Type: Improvement
> Components: Maven Surefire Plugin
> Affects Versions: 2.7.3
> Reporter: Alexander Klimetschek
> Attachments: showsuccess.patch
>
>
> Use case:
> When you have a few tests, and some of them fail during development/refactoring, you find yourself repeatedly looking into target/surefire-reports to open the test report (txt) files with the errors and stack traces to analyze them. However, by default reports will be written also for successful tests, and both in txt and xml format (the xml files can already be omitted using disableXmlReport=false). This makes it hard to find the files that contain the failed tests.
> Solution proposal:
> Just as the maven-surefire-report-plugin already has a showSuccess option, that, if set to "false", will not report successful tests, the same should be available for the maven-surefire-plugin and the normal test goal.
> Either set "-DshowSuccess=false" on the command line or define "<showSuccess>false</showSuccess>" in the maven-surefire-plugin configuration of the pom, and reports for successful tests should not be written to the reports directory.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (SUREFIRE-701) showSuccess option for test
reports: show reports only for failed tests
Posted by "Alexander Klimetschek (JIRA)" <ji...@codehaus.org>.
[ http://jira.codehaus.org/browse/SUREFIRE-701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=256543#action_256543 ]
Alexander Klimetschek commented on SUREFIRE-701:
------------------------------------------------
Patch description:
Attached is a patch that simply removes report files at the end of test execution if it was successful (in AbstractFileReporter). Note: It is difficult to avoid writing the file in the first place, since test logs are streamed directly to the file as the test runs (no buffering) and you only know that a test was successful after it has completed. However, the remove-on-completion works fine. Most of the many code changes apply to all the places where the new configuration has to be stored and passed around. Since I had to extend the ReporterConfiguration constructor with a new boolean parameter, there were places where the invocations via reflection had to be changed. I hope I got them all right, normal mvn test calls work fine for me.
I haven't written a unit test for that, as I don't know which place would be the best (api tests or integration-tests). To be able to run "mvn test" in the surefire projects itself (all successful for me), I had to change the maven-surefire-plugin version in the root pom to "2.7.3-SNAPSHOT" - it seems that useSystemClassLoader=false doesn't help, as I get reflection exceptions with 2.7.2 (see below). However, with 2.7.3-SNAPSHOT you can't build from the reactor, as maven stops detecting a cyclic reference now ;-)
[INFO] --- maven-surefire-plugin:2.7.2:test (default-test) @ surefire-integration-tests ---
[INFO] Surefire report directory: /Users/alex/Day/code/maven-surefire/surefire-integration-tests/target/surefire-reports
org.apache.maven.surefire.util.SurefireReflectionException: java.lang.NoSuchMethodException: org.apache.maven.surefire.report.ReporterConfiguration.<init>(java.util.List, java.io.File, java.lang.Boolean, java.lang.Integer); nested exception is java.lang.NoSuchMethodException: org.apache.maven.surefire.report.ReporterConfiguration.<init>(java.util.List, java.io.File, java.lang.Boolean, java.lang.Integer)
java.lang.NoSuchMethodException: org.apache.maven.surefire.report.ReporterConfiguration.<init>(java.util.List, java.io.File, java.lang.Boolean, java.lang.Integer)
at java.lang.Class.getConstructor0(Class.java:2706)
at java.lang.Class.getConstructor(Class.java:1657)
at org.apache.maven.surefire.util.ReflectionUtils.getConstructor(ReflectionUtils.java:76)
at org.apache.maven.surefire.booter.SurefireReflector.createReporterConfiguration(SurefireReflector.java:183)
at org.apache.maven.surefire.booter.SurefireReflector.setReporterConfiguration(SurefireReflector.java:254)
at org.apache.maven.surefire.booter.SurefireReflector.setReporterConfigurationAware(SurefireReflector.java:247)
at org.apache.maven.surefire.booter.ProviderFactory.createProvider(ProviderFactory.java:67)
at org.apache.maven.surefire.booter.SurefireStarter.invokeProvider(SurefireStarter.java:146)
at org.apache.maven.surefire.booter.SurefireStarter.runSuitesInProcess(SurefireStarter.java:91)
at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:69)
As a side node, I think the code base could be simplified enormously if a generic configuration system would be used (like property lists); the use of reflection contradicts with the hight amount of type-enforcement done with the various config objects and type hierarchies. But I guess this is a sign of evolution :-)
> showSuccess option for test reports: show reports only for failed tests
> -----------------------------------------------------------------------
>
> Key: SUREFIRE-701
> URL: http://jira.codehaus.org/browse/SUREFIRE-701
> Project: Maven Surefire
> Issue Type: Improvement
> Components: Maven Surefire Plugin
> Affects Versions: 2.7.3
> Reporter: Alexander Klimetschek
> Attachments: showsuccess.patch
>
>
> Use case:
> When you have a few tests, and some of them fail during development/refactoring, you find yourself repeatedly looking into target/surefire-reports to open the test report (txt) files with the errors and stack traces to analyze them. However, by default reports will be written also for successful tests, and both in txt and xml format (the xml files can already be omitted using disableXmlReport=false). This makes it hard to find the files that contain the failed tests.
> Solution proposal:
> Just as the maven-surefire-report-plugin already has a showSuccess option, that, if set to "false", will not report successful tests, the same should be available for the maven-surefire-plugin and the normal test goal.
> Either set "-DshowSuccess=false" on the command line or define "<showSuccess>false</showSuccess>" in the maven-surefire-plugin configuration of the pom, and reports for successful tests should not be written to the reports directory.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (SUREFIRE-701) showSuccess option for test
reports: show reports only for failed tests
Posted by "Emmanuel Lécharny (JIRA)" <ji...@codehaus.org>.
[ http://jira.codehaus.org/browse/SUREFIRE-701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=256565#action_256565 ]
Emmanuel Lécharny commented on SUREFIRE-701:
--------------------------------------------
This should be the default behavior : not generating the (empty) text files for success. Who cares about them anyway ?
> showSuccess option for test reports: show reports only for failed tests
> -----------------------------------------------------------------------
>
> Key: SUREFIRE-701
> URL: http://jira.codehaus.org/browse/SUREFIRE-701
> Project: Maven Surefire
> Issue Type: Improvement
> Components: Maven Surefire Plugin
> Affects Versions: 2.7.3
> Reporter: Alexander Klimetschek
> Attachments: showsuccess.patch
>
>
> Use case:
> When you have a few tests, and some of them fail during development/refactoring, you find yourself repeatedly looking into target/surefire-reports to open the test report (txt) files with the errors and stack traces to analyze them. However, by default reports will be written also for successful tests, and both in txt and xml format (the xml files can already be omitted using disableXmlReport=false). This makes it hard to find the files that contain the failed tests.
> Solution proposal:
> Just as the maven-surefire-report-plugin already has a showSuccess option, that, if set to "false", will not report successful tests, the same should be available for the maven-surefire-plugin and the normal test goal.
> Either set "-DshowSuccess=false" on the command line or define "<showSuccess>false</showSuccess>" in the maven-surefire-plugin configuration of the pom, and reports for successful tests should not be written to the reports directory.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Issue Comment Edited: (SUREFIRE-701) showSuccess option for
test reports: show reports only for failed tests
Posted by "Alexander Klimetschek (JIRA)" <ji...@codehaus.org>.
[ http://jira.codehaus.org/browse/SUREFIRE-701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=256543#action_256543 ]
Alexander Klimetschek edited comment on SUREFIRE-701 at 2/17/11 7:45 AM:
-------------------------------------------------------------------------
Patch description:
Attached is a patch that simply removes report files at the end of test execution if it was successful (in AbstractFileReporter). Note: It is difficult to avoid writing the file in the first place, since test logs are streamed directly to the file as the test runs (no buffering) and you only know that a test was successful after it has completed. However, the remove-on-completion works fine. It is backwards-compatible, because the default value for the new option is "true", which will write successful tests as before. Most of the many code changes apply to all the places where the new configuration has to be stored and passed around. Since I had to extend the ReporterConfiguration constructor with a new boolean parameter, there were places where the invocations via reflection had to be changed. I hope I got them all right, normal mvn test calls work fine for me.
I haven't written a unit test for that, as I don't know which place would be the best (api tests or integration-tests). To be able to run "mvn test" in the surefire projects itself (all successful for me), I had to change the maven-surefire-plugin version in the root pom to "2.7.3-SNAPSHOT" - it seems that useSystemClassLoader=false doesn't help, as I get reflection exceptions with 2.7.2 (see below). However, with 2.7.3-SNAPSHOT you can't build from the reactor, as maven stops detecting a cyclic reference now ;-)
[INFO] --- maven-surefire-plugin:2.7.2:test (default-test) @ surefire-integration-tests ---
[INFO] Surefire report directory: /Users/alex/Day/code/maven-surefire/surefire-integration-tests/target/surefire-reports
org.apache.maven.surefire.util.SurefireReflectionException: java.lang.NoSuchMethodException: org.apache.maven.surefire.report.ReporterConfiguration.<init>(java.util.List, java.io.File, java.lang.Boolean, java.lang.Integer); nested exception is java.lang.NoSuchMethodException: org.apache.maven.surefire.report.ReporterConfiguration.<init>(java.util.List, java.io.File, java.lang.Boolean, java.lang.Integer)
java.lang.NoSuchMethodException: org.apache.maven.surefire.report.ReporterConfiguration.<init>(java.util.List, java.io.File, java.lang.Boolean, java.lang.Integer)
at java.lang.Class.getConstructor0(Class.java:2706)
at java.lang.Class.getConstructor(Class.java:1657)
at org.apache.maven.surefire.util.ReflectionUtils.getConstructor(ReflectionUtils.java:76)
at org.apache.maven.surefire.booter.SurefireReflector.createReporterConfiguration(SurefireReflector.java:183)
at org.apache.maven.surefire.booter.SurefireReflector.setReporterConfiguration(SurefireReflector.java:254)
at org.apache.maven.surefire.booter.SurefireReflector.setReporterConfigurationAware(SurefireReflector.java:247)
at org.apache.maven.surefire.booter.ProviderFactory.createProvider(ProviderFactory.java:67)
at org.apache.maven.surefire.booter.SurefireStarter.invokeProvider(SurefireStarter.java:146)
at org.apache.maven.surefire.booter.SurefireStarter.runSuitesInProcess(SurefireStarter.java:91)
at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:69)
As a side node, I think the code base could be simplified enormously if a generic configuration system would be used (like property lists); the use of reflection contradicts with the hight amount of type-enforcement done with the various config objects and type hierarchies. But I guess this is a sign of evolution :-)
was (Author: aklimets):
Patch description:
Attached is a patch that simply removes report files at the end of test execution if it was successful (in AbstractFileReporter). Note: It is difficult to avoid writing the file in the first place, since test logs are streamed directly to the file as the test runs (no buffering) and you only know that a test was successful after it has completed. However, the remove-on-completion works fine. Most of the many code changes apply to all the places where the new configuration has to be stored and passed around. Since I had to extend the ReporterConfiguration constructor with a new boolean parameter, there were places where the invocations via reflection had to be changed. I hope I got them all right, normal mvn test calls work fine for me.
I haven't written a unit test for that, as I don't know which place would be the best (api tests or integration-tests). To be able to run "mvn test" in the surefire projects itself (all successful for me), I had to change the maven-surefire-plugin version in the root pom to "2.7.3-SNAPSHOT" - it seems that useSystemClassLoader=false doesn't help, as I get reflection exceptions with 2.7.2 (see below). However, with 2.7.3-SNAPSHOT you can't build from the reactor, as maven stops detecting a cyclic reference now ;-)
[INFO] --- maven-surefire-plugin:2.7.2:test (default-test) @ surefire-integration-tests ---
[INFO] Surefire report directory: /Users/alex/Day/code/maven-surefire/surefire-integration-tests/target/surefire-reports
org.apache.maven.surefire.util.SurefireReflectionException: java.lang.NoSuchMethodException: org.apache.maven.surefire.report.ReporterConfiguration.<init>(java.util.List, java.io.File, java.lang.Boolean, java.lang.Integer); nested exception is java.lang.NoSuchMethodException: org.apache.maven.surefire.report.ReporterConfiguration.<init>(java.util.List, java.io.File, java.lang.Boolean, java.lang.Integer)
java.lang.NoSuchMethodException: org.apache.maven.surefire.report.ReporterConfiguration.<init>(java.util.List, java.io.File, java.lang.Boolean, java.lang.Integer)
at java.lang.Class.getConstructor0(Class.java:2706)
at java.lang.Class.getConstructor(Class.java:1657)
at org.apache.maven.surefire.util.ReflectionUtils.getConstructor(ReflectionUtils.java:76)
at org.apache.maven.surefire.booter.SurefireReflector.createReporterConfiguration(SurefireReflector.java:183)
at org.apache.maven.surefire.booter.SurefireReflector.setReporterConfiguration(SurefireReflector.java:254)
at org.apache.maven.surefire.booter.SurefireReflector.setReporterConfigurationAware(SurefireReflector.java:247)
at org.apache.maven.surefire.booter.ProviderFactory.createProvider(ProviderFactory.java:67)
at org.apache.maven.surefire.booter.SurefireStarter.invokeProvider(SurefireStarter.java:146)
at org.apache.maven.surefire.booter.SurefireStarter.runSuitesInProcess(SurefireStarter.java:91)
at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:69)
As a side node, I think the code base could be simplified enormously if a generic configuration system would be used (like property lists); the use of reflection contradicts with the hight amount of type-enforcement done with the various config objects and type hierarchies. But I guess this is a sign of evolution :-)
> showSuccess option for test reports: show reports only for failed tests
> -----------------------------------------------------------------------
>
> Key: SUREFIRE-701
> URL: http://jira.codehaus.org/browse/SUREFIRE-701
> Project: Maven Surefire
> Issue Type: Improvement
> Components: Maven Surefire Plugin
> Affects Versions: 2.7.3
> Reporter: Alexander Klimetschek
> Attachments: showsuccess.patch
>
>
> Use case:
> When you have a few tests, and some of them fail during development/refactoring, you find yourself repeatedly looking into target/surefire-reports to open the test report (txt) files with the errors and stack traces to analyze them. However, by default reports will be written also for successful tests, and both in txt and xml format (the xml files can already be omitted using disableXmlReport=false). This makes it hard to find the files that contain the failed tests.
> Solution proposal:
> Just as the maven-surefire-report-plugin already has a showSuccess option, that, if set to "false", will not report successful tests, the same should be available for the maven-surefire-plugin and the normal test goal.
> Either set "-DshowSuccess=false" on the command line or define "<showSuccess>false</showSuccess>" in the maven-surefire-plugin configuration of the pom, and reports for successful tests should not be written to the reports directory.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira