You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@maven.apache.org by "Kristian Rosenvold (JIRA)" <ji...@codehaus.org> on 2010/12/24 21:30:58 UTC
[jira] Updated: (SUREFIRE-611) surefire exit code should explicitly
distinguish between crashed and failed unit test(s)
[ http://jira.codehaus.org/browse/SUREFIRE-611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kristian Rosenvold updated SUREFIRE-611:
----------------------------------------
Component/s: Maven Surefire Plugin
> surefire exit code should explicitly distinguish between crashed and failed unit test(s)
> ----------------------------------------------------------------------------------------
>
> Key: SUREFIRE-611
> URL: http://jira.codehaus.org/browse/SUREFIRE-611
> Project: Maven Surefire
> Issue Type: Improvement
> Components: Maven Surefire Plugin
> Reporter: Tycho Lamerigts
> Attachments: tc-junit-crash.tar.gz
>
>
> - If a unit test crashes, surefire skips all remaining tests, prints a message "[ERROR] There are test failures. ..." to the console, and does NOT print a test summary report.
> - If one or more unit tests fail, maven completes remaining tests, prints a message "[ERROR] There are test failures. ..." to the console, and DOES print a test summary report.
> The only distinction between crash and fail is the presence (or absence) of a summary report.
> This is a problem for some tools (like TeamCity), because they typically don't pick up on the fact that many testcases were not run at all in case of a crashed unit test. Instead in case of a crashed test they simply report, for example, 100 tests succeeded, 0 failed. They do not report "50 tests skipped due to a crash after test #100", and it is quite hard for those tools to recognize this situation. If surefire would report a crash more explicitly, other tools can act accordingly.
> I have attached a small maven project to reproduce the problem. By commenting and uncommenting the marked line in file ATest.java, you can reproduce both fail and crash scenarios. In the crash scenario, Maven will correctly report a failed build (unless you use the -Dmaven.test.failure.ignore flag, in which case it even won't do that), but it fails to indicate there was a crash which caused tests (in this case 3) to be skipped.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira