You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@maven.apache.org by "A (JIRA)" <ji...@codehaus.org> on 2008/03/05 15:23:29 UTC
[jira] Created: (SUREFIRE-468) test timeout handling
test timeout handling
---------------------
Key: SUREFIRE-468
URL: http://jira.codehaus.org/browse/SUREFIRE-468
Project: Maven Surefire
Issue Type: Bug
Components: plugin, process forking
Affects Versions: 2.4.2
Reporter: A
When forkmode is always/prtest (probably that could be true for the last test and forkmode once) when one test hangs and timeout occurs, est suite execution stops and report file for the offending test not generated. That could mislead somebody to think all tests passed if all tests before the offending one passed.
AFAICT that should be synchronized between one of these:
1. CommandLineUtils.executeCommandLine()
2. SurefireBooter.fork()
3. SurefireBooter.run()
4. SurefirePlugin.execute()
Probably fork must detect a timeout. Then the timeout be gracefully handled by generating a report file for the test. Then continue execution of remaining tests.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Closed: (SUREFIRE-468) When tests timeout, report files on
disk are incorrect
Posted by "Kristian Rosenvold (JIRA)" <ji...@codehaus.org>.
[ http://jira.codehaus.org/browse/SUREFIRE-468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kristian Rosenvold closed SUREFIRE-468.
---------------------------------------
Resolution: Fixed
Fix Version/s: (was: Backlog)
2.7.2
Assignee: Kristian Rosenvold
Fixed in r1062022, updated IT
> When tests timeout, report files on disk are incorrect
> ------------------------------------------------------
>
> Key: SUREFIRE-468
> URL: http://jira.codehaus.org/browse/SUREFIRE-468
> Project: Maven Surefire
> Issue Type: Bug
> Components: Maven Surefire Plugin, process forking
> Affects Versions: 2.4.2
> Reporter: A
> Assignee: Kristian Rosenvold
> Fix For: 2.7.2
>
>
> When forkmode is always/prtest (probably that could be true for the last test and forkmode once) when one test hangs and timeout occurs, est suite execution stops and report file for the offending test not generated. That could mislead somebody to think all tests passed if all tests before the offending one passed.
> AFAICT that should be synchronized between one of these:
> 1. CommandLineUtils.executeCommandLine()
> 2. SurefireBooter.fork()
> 3. SurefireBooter.run()
> 4. SurefirePlugin.execute()
> Probably fork must detect a timeout. Then the timeout be gracefully handled by generating a report file for the test. Then continue execution of remaining tests.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (SUREFIRE-468) When tests timeout, report files
on disk are incorrect
Posted by "A (JIRA)" <ji...@codehaus.org>.
[ http://jira.codehaus.org/browse/SUREFIRE-468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=132943#action_132943 ]
A commented on SUREFIRE-468:
----------------------------
It should not stop execution of subsequent tests if "testFailureIgnore" system property is set. Sorry , I've forgot to include that in the issue description.
So the issues are really two:
1. report generation
2. not all tests executed even when "testFailureIgnore" system property is set.
> When tests timeout, report files on disk are incorrect
> ------------------------------------------------------
>
> Key: SUREFIRE-468
> URL: http://jira.codehaus.org/browse/SUREFIRE-468
> Project: Maven Surefire
> Issue Type: Bug
> Components: plugin, process forking
> Affects Versions: 2.4.2
> Reporter: A
> Fix For: 2.5
>
>
> When forkmode is always/prtest (probably that could be true for the last test and forkmode once) when one test hangs and timeout occurs, est suite execution stops and report file for the offending test not generated. That could mislead somebody to think all tests passed if all tests before the offending one passed.
> AFAICT that should be synchronized between one of these:
> 1. CommandLineUtils.executeCommandLine()
> 2. SurefireBooter.fork()
> 3. SurefireBooter.run()
> 4. SurefirePlugin.execute()
> Probably fork must detect a timeout. Then the timeout be gracefully handled by generating a report file for the test. Then continue execution of remaining tests.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (SUREFIRE-468) When tests timeout, report files on
disk are incorrect
Posted by "Stephen Connolly (JIRA)" <ji...@codehaus.org>.
[ http://jira.codehaus.org/browse/SUREFIRE-468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Stephen Connolly updated SUREFIRE-468:
--------------------------------------
Fix Version/s: (was: 2.5)
2.6
> When tests timeout, report files on disk are incorrect
> ------------------------------------------------------
>
> Key: SUREFIRE-468
> URL: http://jira.codehaus.org/browse/SUREFIRE-468
> Project: Maven Surefire
> Issue Type: Bug
> Components: plugin, process forking
> Affects Versions: 2.4.2
> Reporter: A
> Fix For: 2.6
>
>
> When forkmode is always/prtest (probably that could be true for the last test and forkmode once) when one test hangs and timeout occurs, est suite execution stops and report file for the offending test not generated. That could mislead somebody to think all tests passed if all tests before the offending one passed.
> AFAICT that should be synchronized between one of these:
> 1. CommandLineUtils.executeCommandLine()
> 2. SurefireBooter.fork()
> 3. SurefireBooter.run()
> 4. SurefirePlugin.execute()
> Probably fork must detect a timeout. Then the timeout be gracefully handled by generating a report file for the test. Then continue execution of remaining tests.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (SUREFIRE-468) When tests timeout, report files on
disk are incorrect
Posted by "Stephen Connolly (JIRA)" <ji...@codehaus.org>.
[ http://jira.codehaus.org/browse/SUREFIRE-468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Stephen Connolly updated SUREFIRE-468:
--------------------------------------
Fix Version/s: (was: 2.6)
Backlog
Moving to the backlog as it's not a regression and it is more pressing to get 2.6 out first
> When tests timeout, report files on disk are incorrect
> ------------------------------------------------------
>
> Key: SUREFIRE-468
> URL: http://jira.codehaus.org/browse/SUREFIRE-468
> Project: Maven Surefire
> Issue Type: Bug
> Components: Maven Surefire Plugin, process forking
> Affects Versions: 2.4.2
> Reporter: A
> Fix For: Backlog
>
>
> When forkmode is always/prtest (probably that could be true for the last test and forkmode once) when one test hangs and timeout occurs, est suite execution stops and report file for the offending test not generated. That could mislead somebody to think all tests passed if all tests before the offending one passed.
> AFAICT that should be synchronized between one of these:
> 1. CommandLineUtils.executeCommandLine()
> 2. SurefireBooter.fork()
> 3. SurefireBooter.run()
> 4. SurefirePlugin.execute()
> Probably fork must detect a timeout. Then the timeout be gracefully handled by generating a report file for the test. Then continue execution of remaining tests.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Reopened: (SUREFIRE-468) When tests timeout, report files on
disk are incorrect
Posted by "Kristian Rosenvold (JIRA)" <ji...@codehaus.org>.
[ http://jira.codehaus.org/browse/SUREFIRE-468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kristian Rosenvold reopened SUREFIRE-468:
-----------------------------------------
Testing SUREFIRE-705 reveals that this patch does not work well enough to stay in. Handling this issue must be done "differently".
> When tests timeout, report files on disk are incorrect
> ------------------------------------------------------
>
> Key: SUREFIRE-468
> URL: http://jira.codehaus.org/browse/SUREFIRE-468
> Project: Maven Surefire
> Issue Type: Bug
> Components: Maven Surefire Plugin, process forking
> Affects Versions: 2.4.2
> Reporter: A
> Assignee: Kristian Rosenvold
> Fix For: 2.7.2
>
>
> When forkmode is always/prtest (probably that could be true for the last test and forkmode once) when one test hangs and timeout occurs, est suite execution stops and report file for the offending test not generated. That could mislead somebody to think all tests passed if all tests before the offending one passed.
> AFAICT that should be synchronized between one of these:
> 1. CommandLineUtils.executeCommandLine()
> 2. SurefireBooter.fork()
> 3. SurefireBooter.run()
> 4. SurefirePlugin.execute()
> Probably fork must detect a timeout. Then the timeout be gracefully handled by generating a report file for the test. Then continue execution of remaining tests.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (SUREFIRE-468) When tests timeout, report files on
disk are incorrect
Posted by "Dan Fabulich (JIRA)" <ji...@codehaus.org>.
[ http://jira.codehaus.org/browse/SUREFIRE-468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dan Fabulich updated SUREFIRE-468:
----------------------------------
Fix Version/s: 2.5
Summary: When tests timeout, report files on disk are incorrect (was: test timeout handling)
When I run the "timeout-forked" integration test, it fails the Maven build, which is correct behavior.
However, it seems to always generate the TXT file, and sometimes generates the XML file and sometimes not. (Seems like it usually doesn't generate XML when using "mvn clean test", but usually does generate XML when using "mvn clean && mvn test".)
In all cases, all of the files remaining on disk seem to suggest that the test passed. For example, the TXT file says "Tests run: 1, Failures: 0," the XML file says <testsuite failures="0" tests="1">, etc.
> When tests timeout, report files on disk are incorrect
> ------------------------------------------------------
>
> Key: SUREFIRE-468
> URL: http://jira.codehaus.org/browse/SUREFIRE-468
> Project: Maven Surefire
> Issue Type: Bug
> Components: plugin, process forking
> Affects Versions: 2.4.2
> Reporter: A
> Fix For: 2.5
>
>
> When forkmode is always/prtest (probably that could be true for the last test and forkmode once) when one test hangs and timeout occurs, est suite execution stops and report file for the offending test not generated. That could mislead somebody to think all tests passed if all tests before the offending one passed.
> AFAICT that should be synchronized between one of these:
> 1. CommandLineUtils.executeCommandLine()
> 2. SurefireBooter.fork()
> 3. SurefireBooter.run()
> 4. SurefirePlugin.execute()
> Probably fork must detect a timeout. Then the timeout be gracefully handled by generating a report file for the test. Then continue execution of remaining tests.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (SUREFIRE-468) When tests timeout, report files on
disk are incorrect
Posted by "Kristian Rosenvold (JIRA)" <ji...@codehaus.org>.
[ http://jira.codehaus.org/browse/SUREFIRE-468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kristian Rosenvold updated SUREFIRE-468:
----------------------------------------
Fix Version/s: (was: 2.7.2)
> When tests timeout, report files on disk are incorrect
> ------------------------------------------------------
>
> Key: SUREFIRE-468
> URL: http://jira.codehaus.org/browse/SUREFIRE-468
> Project: Maven Surefire
> Issue Type: Bug
> Components: Maven Surefire Plugin, process forking
> Affects Versions: 2.4.2
> Reporter: A
> Assignee: Kristian Rosenvold
>
> When forkmode is always/prtest (probably that could be true for the last test and forkmode once) when one test hangs and timeout occurs, est suite execution stops and report file for the offending test not generated. That could mislead somebody to think all tests passed if all tests before the offending one passed.
> AFAICT that should be synchronized between one of these:
> 1. CommandLineUtils.executeCommandLine()
> 2. SurefireBooter.fork()
> 3. SurefireBooter.run()
> 4. SurefirePlugin.execute()
> Probably fork must detect a timeout. Then the timeout be gracefully handled by generating a report file for the test. Then continue execution of remaining tests.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira