You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@maven.apache.org by "Jesse Glick (JIRA)" <ji...@codehaus.org> on 2013/01/03 22:14:13 UTC
[jira] (SUREFIRE-946) Maven hangs on SIGTERM when using Surefire
forking (CommandLineUtils.ProcessHook)
Jesse Glick created SUREFIRE-946:
------------------------------------
Summary: Maven hangs on SIGTERM when using Surefire forking (CommandLineUtils.ProcessHook)
Key: SUREFIRE-946
URL: https://jira.codehaus.org/browse/SUREFIRE-946
Project: Maven Surefire
Issue Type: Bug
Affects Versions: 2.13
Reporter: Jesse Glick
Attachments: stack.txt
Java 7u7, Surefire with JUnit {{forkMode="perthread"}} + {{threadCount="1"}} + {{reuseForks="true"}}. After pressing Ctrl-C to stop the Maven test run, the process hangs and must be killed with SIGKILL. From the thread dump, {{CommandLineUtils.ProcessHook}} and {{StreamFeeder}} look responsible.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://jira.codehaus.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] (SUREFIRE-946) Maven hangs on SIGTERM when using Surefire
forking (CommandLineUtils.ProcessHook)
Posted by "Kristian Rosenvold (JIRA)" <ji...@codehaus.org>.
[ https://jira.codehaus.org/browse/SUREFIRE-946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=316632#comment-316632 ]
Kristian Rosenvold commented on SUREFIRE-946:
---------------------------------------------
@andreas I think this means LazyTestsToRun line 110 goes puking forever because we try to read a line that never arrives from the killed maven process. Adding a shutdown hook to the plugin that sends termination signal to the fork is probably the way to go. I would actually consider using this same hook to always terminate the forked process, since that's one of our oldest open issues.
There is always some dark magic voodo involved when reading/writing process input, I hope this will work.
> Maven hangs on SIGTERM when using Surefire forking (CommandLineUtils.ProcessHook)
> ---------------------------------------------------------------------------------
>
> Key: SUREFIRE-946
> URL: https://jira.codehaus.org/browse/SUREFIRE-946
> Project: Maven Surefire
> Issue Type: Bug
> Affects Versions: 2.13
> Reporter: Jesse Glick
> Attachments: stack.txt
>
>
> Java 7u7, Surefire with JUnit {{forkMode="perthread"}} + {{threadCount="1"}} + {{reuseForks="true"}}. After pressing Ctrl-C to stop the Maven test run, the process hangs and must be killed with SIGKILL. From the thread dump, {{CommandLineUtils.ProcessHook}} and {{StreamFeeder}} look responsible.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://jira.codehaus.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] (SUREFIRE-946) Maven hangs on SIGTERM when using Surefire
forking (CommandLineUtils.ProcessHook)
Posted by "Andreas Gudian (JIRA)" <ji...@codehaus.org>.
[ https://jira.codehaus.org/browse/SUREFIRE-946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=317050#comment-317050 ]
Andreas Gudian commented on SUREFIRE-946:
-----------------------------------------
It already has the ability to pass a timeout for the process. I would add another option to tell it to {{killProcessAfterTimeout}} - and then in that case let it call Process#destroy(). That should do it, right?
> Maven hangs on SIGTERM when using Surefire forking (CommandLineUtils.ProcessHook)
> ---------------------------------------------------------------------------------
>
> Key: SUREFIRE-946
> URL: https://jira.codehaus.org/browse/SUREFIRE-946
> Project: Maven Surefire
> Issue Type: Bug
> Affects Versions: 2.13
> Reporter: Jesse Glick
> Fix For: 2.14
>
> Attachments: stack.txt, SUREFIRE-946.patch
>
>
> Java 7u7, Surefire with JUnit {{forkMode="perthread"}} + {{threadCount="1"}} + {{reuseForks="true"}}. After pressing Ctrl-C to stop the Maven test run, the process hangs and must be killed with SIGKILL. From the thread dump, {{CommandLineUtils.ProcessHook}} and {{StreamFeeder}} look responsible.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://jira.codehaus.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] (SUREFIRE-946) Maven hangs on SIGTERM when using Surefire
forking (CommandLineUtils.ProcessHook)
Posted by "Kristian Rosenvold (JIRA)" <ji...@codehaus.org>.
[ https://jira.codehaus.org/browse/SUREFIRE-946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=317054#comment-317054 ]
Kristian Rosenvold commented on SUREFIRE-946:
---------------------------------------------
I'm thinking the shutdown hook could be "not used" when we terminate normally, so the successful flow of events does not have to depend on a shutdown hook - like it used to be. So we basically add & remove the shutdown hook in the normal flow of events, and send BYE without involving the hook. While the test runs, the hook is in place and will catch dirty stuff happening in the test ?
> Maven hangs on SIGTERM when using Surefire forking (CommandLineUtils.ProcessHook)
> ---------------------------------------------------------------------------------
>
> Key: SUREFIRE-946
> URL: https://jira.codehaus.org/browse/SUREFIRE-946
> Project: Maven Surefire
> Issue Type: Bug
> Affects Versions: 2.13
> Reporter: Jesse Glick
> Fix For: 2.14
>
> Attachments: stack.txt, SUREFIRE-946.patch
>
>
> Java 7u7, Surefire with JUnit {{forkMode="perthread"}} + {{threadCount="1"}} + {{reuseForks="true"}}. After pressing Ctrl-C to stop the Maven test run, the process hangs and must be killed with SIGKILL. From the thread dump, {{CommandLineUtils.ProcessHook}} and {{StreamFeeder}} look responsible.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://jira.codehaus.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] (SUREFIRE-946) Maven hangs on SIGTERM when using Surefire
forking (CommandLineUtils.ProcessHook)
Posted by "Kristian Rosenvold (JIRA)" <ji...@codehaus.org>.
[ https://jira.codehaus.org/browse/SUREFIRE-946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=317048#comment-317048 ]
Kristian Rosenvold commented on SUREFIRE-946:
---------------------------------------------
I think it depends if CommandlineUtils leaks the pid or allows killing by external handle. I know we have this in selenium but I don't remember here ;)
> Maven hangs on SIGTERM when using Surefire forking (CommandLineUtils.ProcessHook)
> ---------------------------------------------------------------------------------
>
> Key: SUREFIRE-946
> URL: https://jira.codehaus.org/browse/SUREFIRE-946
> Project: Maven Surefire
> Issue Type: Bug
> Affects Versions: 2.13
> Reporter: Jesse Glick
> Fix For: 2.14
>
> Attachments: stack.txt, SUREFIRE-946.patch
>
>
> Java 7u7, Surefire with JUnit {{forkMode="perthread"}} + {{threadCount="1"}} + {{reuseForks="true"}}. After pressing Ctrl-C to stop the Maven test run, the process hangs and must be killed with SIGKILL. From the thread dump, {{CommandLineUtils.ProcessHook}} and {{StreamFeeder}} look responsible.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://jira.codehaus.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] (SUREFIRE-946) Maven hangs on SIGTERM when using Surefire
forking (CommandLineUtils.ProcessHook)
Posted by "Jesse Glick (JIRA)" <ji...@codehaus.org>.
[ https://jira.codehaus.org/browse/SUREFIRE-946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=316896#comment-316896 ]
Jesse Glick commented on SUREFIRE-946:
--------------------------------------
Regardless of why the subprocess is failing to exit, the fact remains that the current {{ProcessHook}} tries to acquire a lock during a shutdown hook with no timeout and no guarantee that this lock will be released in a timely fashion. Making it more likely for the subprocess to exit cleanly may make this bug less likely to occur but will not completely prevent it. Supplementing Andreas fix with a patch to {{ProcessHook}} to impose a timeout on the call to {{Process.destroy}} still seems like a good idea.
> Maven hangs on SIGTERM when using Surefire forking (CommandLineUtils.ProcessHook)
> ---------------------------------------------------------------------------------
>
> Key: SUREFIRE-946
> URL: https://jira.codehaus.org/browse/SUREFIRE-946
> Project: Maven Surefire
> Issue Type: Bug
> Affects Versions: 2.13
> Reporter: Jesse Glick
> Attachments: stack.txt, SUREFIRE-946.patch
>
>
> Java 7u7, Surefire with JUnit {{forkMode="perthread"}} + {{threadCount="1"}} + {{reuseForks="true"}}. After pressing Ctrl-C to stop the Maven test run, the process hangs and must be killed with SIGKILL. From the thread dump, {{CommandLineUtils.ProcessHook}} and {{StreamFeeder}} look responsible.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://jira.codehaus.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] (SUREFIRE-946) Maven hangs on SIGTERM when using Surefire
forking (CommandLineUtils.ProcessHook)
Posted by "Andreas Gudian (JIRA)" <ji...@codehaus.org>.
[ https://jira.codehaus.org/browse/SUREFIRE-946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=316987#comment-316987 ]
Andreas Gudian commented on SUREFIRE-946:
-----------------------------------------
Great, thanks for your test.
Then for {{reuseForks=true}} the crash detection and behaviour when terminating the main process now at least works just as good or bad as for the other forked test executions in 2.13.4.
> Maven hangs on SIGTERM when using Surefire forking (CommandLineUtils.ProcessHook)
> ---------------------------------------------------------------------------------
>
> Key: SUREFIRE-946
> URL: https://jira.codehaus.org/browse/SUREFIRE-946
> Project: Maven Surefire
> Issue Type: Bug
> Affects Versions: 2.13
> Reporter: Jesse Glick
> Attachments: stack.txt, SUREFIRE-946.patch
>
>
> Java 7u7, Surefire with JUnit {{forkMode="perthread"}} + {{threadCount="1"}} + {{reuseForks="true"}}. After pressing Ctrl-C to stop the Maven test run, the process hangs and must be killed with SIGKILL. From the thread dump, {{CommandLineUtils.ProcessHook}} and {{StreamFeeder}} look responsible.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://jira.codehaus.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] (SUREFIRE-946) Maven hangs on SIGTERM when using Surefire
forking (CommandLineUtils.ProcessHook)
Posted by "Kristian Rosenvold (JIRA)" <ji...@codehaus.org>.
[ https://jira.codehaus.org/browse/SUREFIRE-946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=316623#comment-316623 ]
Kristian Rosenvold commented on SUREFIRE-946:
---------------------------------------------
Ca you find out if this is related to the "reuseForks" parameter, or if it also happens without it ?
> Maven hangs on SIGTERM when using Surefire forking (CommandLineUtils.ProcessHook)
> ---------------------------------------------------------------------------------
>
> Key: SUREFIRE-946
> URL: https://jira.codehaus.org/browse/SUREFIRE-946
> Project: Maven Surefire
> Issue Type: Bug
> Affects Versions: 2.13
> Reporter: Jesse Glick
> Attachments: stack.txt
>
>
> Java 7u7, Surefire with JUnit {{forkMode="perthread"}} + {{threadCount="1"}} + {{reuseForks="true"}}. After pressing Ctrl-C to stop the Maven test run, the process hangs and must be killed with SIGKILL. From the thread dump, {{CommandLineUtils.ProcessHook}} and {{StreamFeeder}} look responsible.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://jira.codehaus.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] (SUREFIRE-946) Maven hangs on SIGTERM when using Surefire
forking (CommandLineUtils.ProcessHook)
Posted by "Kristian Rosenvold (JIRA)" <ji...@codehaus.org>.
[ https://jira.codehaus.org/browse/SUREFIRE-946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kristian Rosenvold updated SUREFIRE-946:
----------------------------------------
Fix Version/s: 2.14
> Maven hangs on SIGTERM when using Surefire forking (CommandLineUtils.ProcessHook)
> ---------------------------------------------------------------------------------
>
> Key: SUREFIRE-946
> URL: https://jira.codehaus.org/browse/SUREFIRE-946
> Project: Maven Surefire
> Issue Type: Bug
> Affects Versions: 2.13
> Reporter: Jesse Glick
> Fix For: 2.14
>
> Attachments: stack.txt, SUREFIRE-946.patch
>
>
> Java 7u7, Surefire with JUnit {{forkMode="perthread"}} + {{threadCount="1"}} + {{reuseForks="true"}}. After pressing Ctrl-C to stop the Maven test run, the process hangs and must be killed with SIGKILL. From the thread dump, {{CommandLineUtils.ProcessHook}} and {{StreamFeeder}} look responsible.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://jira.codehaus.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] (SUREFIRE-946) Maven hangs on SIGTERM when using Surefire
forking (CommandLineUtils.ProcessHook)
Posted by "Jesse Glick (JIRA)" <ji...@codehaus.org>.
[ https://jira.codehaus.org/browse/SUREFIRE-946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=316922#comment-316922 ]
Jesse Glick commented on SUREFIRE-946:
--------------------------------------
Acquiring the lock:
{code:none}
"CommandlineUtils process shutdown hook" prio=10 tid=0x6757a800 nid=0x29b1 waiting for monitor entry [0x678ad000]
java.lang.Thread.State: BLOCKED (on object monitor)
at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
- waiting to lock <0xa7b292c8> (a java.lang.UNIXProcess$ProcessPipeOutputStream)
at java.io.FilterOutputStream.close(FilterOutputStream.java:157)
at java.lang.UNIXProcess.destroy(UNIXProcess.java:234)
at org.apache.maven.surefire.shade.org.apache.maven.shared.utils.cli.CommandLineUtils$ProcessHook.run(CommandLineUtils.java:74)
{code}
Testing fix: I will try to do so soon.
> Maven hangs on SIGTERM when using Surefire forking (CommandLineUtils.ProcessHook)
> ---------------------------------------------------------------------------------
>
> Key: SUREFIRE-946
> URL: https://jira.codehaus.org/browse/SUREFIRE-946
> Project: Maven Surefire
> Issue Type: Bug
> Affects Versions: 2.13
> Reporter: Jesse Glick
> Attachments: stack.txt, SUREFIRE-946.patch
>
>
> Java 7u7, Surefire with JUnit {{forkMode="perthread"}} + {{threadCount="1"}} + {{reuseForks="true"}}. After pressing Ctrl-C to stop the Maven test run, the process hangs and must be killed with SIGKILL. From the thread dump, {{CommandLineUtils.ProcessHook}} and {{StreamFeeder}} look responsible.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://jira.codehaus.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] (SUREFIRE-946) Maven hangs on SIGTERM when using Surefire
forking (CommandLineUtils.ProcessHook)
Posted by "Jesse Glick (JIRA)" <ji...@codehaus.org>.
[ https://jira.codehaus.org/browse/SUREFIRE-946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=316928#comment-316928 ]
Jesse Glick commented on SUREFIRE-946:
--------------------------------------
Linux 3.2.0 (Ubuntu 12.04), sorry for not mentioning this initially.
> Maven hangs on SIGTERM when using Surefire forking (CommandLineUtils.ProcessHook)
> ---------------------------------------------------------------------------------
>
> Key: SUREFIRE-946
> URL: https://jira.codehaus.org/browse/SUREFIRE-946
> Project: Maven Surefire
> Issue Type: Bug
> Affects Versions: 2.13
> Reporter: Jesse Glick
> Attachments: stack.txt, SUREFIRE-946.patch
>
>
> Java 7u7, Surefire with JUnit {{forkMode="perthread"}} + {{threadCount="1"}} + {{reuseForks="true"}}. After pressing Ctrl-C to stop the Maven test run, the process hangs and must be killed with SIGKILL. From the thread dump, {{CommandLineUtils.ProcessHook}} and {{StreamFeeder}} look responsible.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://jira.codehaus.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] (SUREFIRE-946) Maven hangs on SIGTERM when using Surefire
forking (CommandLineUtils.ProcessHook)
Posted by "Andreas Gudian (JIRA)" <ji...@codehaus.org>.
[ https://jira.codehaus.org/browse/SUREFIRE-946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=316921#comment-316921 ]
Andreas Gudian commented on SUREFIRE-946:
-----------------------------------------
@Kristian: what do you think, should we aim to add a heartbeat-feature for the the coming release?
> Maven hangs on SIGTERM when using Surefire forking (CommandLineUtils.ProcessHook)
> ---------------------------------------------------------------------------------
>
> Key: SUREFIRE-946
> URL: https://jira.codehaus.org/browse/SUREFIRE-946
> Project: Maven Surefire
> Issue Type: Bug
> Affects Versions: 2.13
> Reporter: Jesse Glick
> Attachments: stack.txt, SUREFIRE-946.patch
>
>
> Java 7u7, Surefire with JUnit {{forkMode="perthread"}} + {{threadCount="1"}} + {{reuseForks="true"}}. After pressing Ctrl-C to stop the Maven test run, the process hangs and must be killed with SIGKILL. From the thread dump, {{CommandLineUtils.ProcessHook}} and {{StreamFeeder}} look responsible.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://jira.codehaus.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] (SUREFIRE-946) Maven hangs on SIGTERM when using Surefire
forking (CommandLineUtils.ProcessHook)
Posted by "Andreas Gudian (JIRA)" <ji...@codehaus.org>.
[ https://jira.codehaus.org/browse/SUREFIRE-946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=316920#comment-316920 ]
Andreas Gudian commented on SUREFIRE-946:
-----------------------------------------
Where do you see {{ProcessHook}} acquiring a lock? I think terminating the forked VM is not the problem.
I think the (remaining) problem is if the forked VM terminates _without_ executing its own shutdown hook.
Do you have any means to test this fix for your scenario?
> Maven hangs on SIGTERM when using Surefire forking (CommandLineUtils.ProcessHook)
> ---------------------------------------------------------------------------------
>
> Key: SUREFIRE-946
> URL: https://jira.codehaus.org/browse/SUREFIRE-946
> Project: Maven Surefire
> Issue Type: Bug
> Affects Versions: 2.13
> Reporter: Jesse Glick
> Attachments: stack.txt, SUREFIRE-946.patch
>
>
> Java 7u7, Surefire with JUnit {{forkMode="perthread"}} + {{threadCount="1"}} + {{reuseForks="true"}}. After pressing Ctrl-C to stop the Maven test run, the process hangs and must be killed with SIGKILL. From the thread dump, {{CommandLineUtils.ProcessHook}} and {{StreamFeeder}} look responsible.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://jira.codehaus.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] (SUREFIRE-946) Maven hangs on SIGTERM when using Surefire
forking (CommandLineUtils.ProcessHook)
Posted by "Andreas Gudian (JIRA)" <ji...@codehaus.org>.
[ https://jira.codehaus.org/browse/SUREFIRE-946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=316923#comment-316923 ]
Andreas Gudian commented on SUREFIRE-946:
-----------------------------------------
Damn, I didn't see that. Then the fix might not work for you. What OS are you on?
> Maven hangs on SIGTERM when using Surefire forking (CommandLineUtils.ProcessHook)
> ---------------------------------------------------------------------------------
>
> Key: SUREFIRE-946
> URL: https://jira.codehaus.org/browse/SUREFIRE-946
> Project: Maven Surefire
> Issue Type: Bug
> Affects Versions: 2.13
> Reporter: Jesse Glick
> Attachments: stack.txt, SUREFIRE-946.patch
>
>
> Java 7u7, Surefire with JUnit {{forkMode="perthread"}} + {{threadCount="1"}} + {{reuseForks="true"}}. After pressing Ctrl-C to stop the Maven test run, the process hangs and must be killed with SIGKILL. From the thread dump, {{CommandLineUtils.ProcessHook}} and {{StreamFeeder}} look responsible.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://jira.codehaus.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] (SUREFIRE-946) Maven hangs on SIGTERM when using Surefire
forking (CommandLineUtils.ProcessHook)
Posted by "Andreas Gudian (JIRA)" <ji...@codehaus.org>.
[ https://jira.codehaus.org/browse/SUREFIRE-946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=317047#comment-317047 ]
Andreas Gudian commented on SUREFIRE-946:
-----------------------------------------
@Kristian:
* indeed, {{closed}} needs to be volatile. Then it should be thread-safe again.
* {{BOOTERCODE_CRASH}} is the way for the forked process to let the main process know that it terminated unexpectedly. The reason why I added this is to repair the crash-detection, which would not work without actively closing the TestProvidingInputStream at some point. See the extended CrashDetectionIT: it fails without that part of the fix (well, to be more precise: the test will hang). However, as I have mentioned in one of my comments above, this only fixes the crash detection for those cases where the VM is still able to execute the shutdown hook successfully. To really fix it for that case as well, I yet have not any other idea than to implement the heart-beat feature. Hmm, or perhaps if we extend the CommandLineUtils to allow passing a custom hook to execute before it starts waiting on the streams. That could work, I guess and I could remove the BOOTERCODE_CRASH stuff again... I'll try that... :)
* Integration-Tests: yeah. For the VM termination in the forked process, I extended the CrashDetectionIT. To test the brutal VM crash of the forked process, I will further extend it (and call Runtime#halt). The exact issue that Jesse had was that the main process tried to shut down, but the locks held in the TestProvidingInputStream prevented the shutdown hooks to finish. So what we would need is a test that sends a kill signal to the main process started by the IT. I don't know the internal guts of the Verifier - but is that possible? Ok, looked at Process#destroy() and it seems do-able. Requires some more work on CommandLineUtils, Verifier and everything in between. Sounds like fun :)
> Maven hangs on SIGTERM when using Surefire forking (CommandLineUtils.ProcessHook)
> ---------------------------------------------------------------------------------
>
> Key: SUREFIRE-946
> URL: https://jira.codehaus.org/browse/SUREFIRE-946
> Project: Maven Surefire
> Issue Type: Bug
> Affects Versions: 2.13
> Reporter: Jesse Glick
> Fix For: 2.14
>
> Attachments: stack.txt, SUREFIRE-946.patch
>
>
> Java 7u7, Surefire with JUnit {{forkMode="perthread"}} + {{threadCount="1"}} + {{reuseForks="true"}}. After pressing Ctrl-C to stop the Maven test run, the process hangs and must be killed with SIGKILL. From the thread dump, {{CommandLineUtils.ProcessHook}} and {{StreamFeeder}} look responsible.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://jira.codehaus.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] (SUREFIRE-946) Maven hangs on SIGTERM when using Surefire
forking (CommandLineUtils.ProcessHook)
Posted by "Kristian Rosenvold (JIRA)" <ji...@codehaus.org>.
[ https://jira.codehaus.org/browse/SUREFIRE-946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=317037#comment-317037 ]
Kristian Rosenvold edited comment on SUREFIRE-946 at 1/9/13 11:44 AM:
----------------------------------------------------------------------
I have looked at the patch, and applied it to trunk in e54dbd810f62fe723800f11279b881bff244b707
@andreas: I have a couple of minor considerations I'd like your input on;
TestProvidingInputStream has synchronized read method. As far as I can tell the use of this class is confined to a single thread? If this is not the case, then "close" must be synchronized (or "closed" volatile), since there is mismatched synchronization on this variable in the class. (Both reads and writes need to be synchronized for correctness). BUt it seems like this class is NotThreadSafe ?
In forkedBooter, I'm not sure I understand the added value provided by the ForkingRunListener.BOOTERCODE_CRASH, since by my meagre understanding, anything that is not BOOTERCODE_BYE implies a crash ?
(Which was the existing logic)
was (Author: krosenvold):
I have looked at the patch, and applied it to trunk in e54dbd810f62fe723800f11279b881bff244b707
@andread: I have a couple of minor considerations I'd like your input on;
TestProvidingInputStream has synchronized read method. As far as I can tell the use of this class is confined to a single thread? If this is not the case, then "close" must be synchronized (or "closed" volatile), since there is mismatched synchronization on this variable in the class. (Both reads and writes need to be synchronized for correctness). BUt it seems like this class is NotThreadSafe ?
In forkedBooter, I'm not sure I understand the added value provided by the ForkingRunListener.BOOTERCODE_CRASH, since by my meagre understanding, anything that is not BOOTERCODE_BYE implies a crash ?
(Which was the existing logic)
> Maven hangs on SIGTERM when using Surefire forking (CommandLineUtils.ProcessHook)
> ---------------------------------------------------------------------------------
>
> Key: SUREFIRE-946
> URL: https://jira.codehaus.org/browse/SUREFIRE-946
> Project: Maven Surefire
> Issue Type: Bug
> Affects Versions: 2.13
> Reporter: Jesse Glick
> Fix For: 2.14
>
> Attachments: stack.txt, SUREFIRE-946.patch
>
>
> Java 7u7, Surefire with JUnit {{forkMode="perthread"}} + {{threadCount="1"}} + {{reuseForks="true"}}. After pressing Ctrl-C to stop the Maven test run, the process hangs and must be killed with SIGKILL. From the thread dump, {{CommandLineUtils.ProcessHook}} and {{StreamFeeder}} look responsible.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://jira.codehaus.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] (SUREFIRE-946) Maven hangs on SIGTERM when using Surefire
forking (CommandLineUtils.ProcessHook)
Posted by "Andreas Gudian (JIRA)" <ji...@codehaus.org>.
[ https://jira.codehaus.org/browse/SUREFIRE-946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=317064#comment-317064 ]
Andreas Gudian commented on SUREFIRE-946:
-----------------------------------------
OK, check my latest push... It fixes all the broken ITs, it now even works in case the forked process calls Runtime#halt, and I removed BOOTERCODE_CRASH. Also added an IT for the initial problem described in this issue (Surefire946KillMainProcessInReusableForkIT), and extended the CrashDetectionIT to test reuseForks with Runtime#halt.
This still relys on Process#waitFor to work in case no timeout was given. So we're not getting better than in 2.12.4, but I hope it's not worse either.
Oh, and... I had to modify maven-verifier (for Surefire946KillMainProcessInReusableForkIT) and maven-shared-utils (for the IT and the actual fix). I pushed that to https://github.com/agudian/maven-shared/tree/SUREFIRE-946, but did not yet file a pull-request. I'd like your feedback first... :)
> Maven hangs on SIGTERM when using Surefire forking (CommandLineUtils.ProcessHook)
> ---------------------------------------------------------------------------------
>
> Key: SUREFIRE-946
> URL: https://jira.codehaus.org/browse/SUREFIRE-946
> Project: Maven Surefire
> Issue Type: Bug
> Affects Versions: 2.13
> Reporter: Jesse Glick
> Fix For: 2.14
>
> Attachments: stack.txt, SUREFIRE-946.patch
>
>
> Java 7u7, Surefire with JUnit {{forkMode="perthread"}} + {{threadCount="1"}} + {{reuseForks="true"}}. After pressing Ctrl-C to stop the Maven test run, the process hangs and must be killed with SIGKILL. From the thread dump, {{CommandLineUtils.ProcessHook}} and {{StreamFeeder}} look responsible.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://jira.codehaus.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] (SUREFIRE-946) Maven hangs on SIGTERM when using Surefire
forking (CommandLineUtils.ProcessHook)
Posted by "Kristian Rosenvold (JIRA)" <ji...@codehaus.org>.
[ https://jira.codehaus.org/browse/SUREFIRE-946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=316791#comment-316791 ]
Kristian Rosenvold commented on SUREFIRE-946:
---------------------------------------------
{begin rant}
Hundreds of tear-stained blog posts have been written about java process forking and the jdk issue tracker is littered with unsolved bugs in process forking. Unfortunately the people who wrote ProcessBuilder did not fully appreciate all the subtleties of these bugs. I spent quite a lot of time trying to make ProcessBuilder stable for the selenium project, and my conclusion was that ProcessBuilder does not handle the process streams correctly, and was broken and deadlock prone for all released jdk versions at the time I tested this (about a year ago).
This is the code that runs millions of forks every day and has done so for numerous years, in terms of stability and reliability it outperforms ProcessBuilder by a wide margin.
{end rant}
I have long since supressed the details of the numerous traumatic ways process forking can fail regarding stream handling. Looking at the code now, I think the issue might be that the streams are not closed in the shutdown hook (If I remember correctly the kill -3 will not run the finally block in executeCommandLineAsCallable). I think the process failing to exit is simply because
we're not closing the streams properly.
I'm thinking adding these to the current shutdown hook might do it:
if ( inputFeeder != null )
{
inputFeeder.close();
}
outputPumper.close();
errorPumper.close();
(It's probably wise trying to fix this one problem first before moving onto more esoteric stuff like heartbeats...)
I clearly seem to remember one of the tearstained blogposts about terminating forks that were waiting for output from stdin. The details escape me ;)
In this context we might consider switching rpc mechanism entirely to something like named pipes. The 2-way communications between two java processes using stdio is really one of the reasons I've held back on implementing this feature in the first place. Suggestions are welcome as I'm still thinking about this on the 5th year or so ;)
BTW: The shutdown hook is necessary for some reason (probably more bugs). The explanation can be found by looking at history for ant/commons-exec/plexus-utils. I believe the original commit in plexus has a fairly extensive issue history.
> Maven hangs on SIGTERM when using Surefire forking (CommandLineUtils.ProcessHook)
> ---------------------------------------------------------------------------------
>
> Key: SUREFIRE-946
> URL: https://jira.codehaus.org/browse/SUREFIRE-946
> Project: Maven Surefire
> Issue Type: Bug
> Affects Versions: 2.13
> Reporter: Jesse Glick
> Attachments: stack.txt
>
>
> Java 7u7, Surefire with JUnit {{forkMode="perthread"}} + {{threadCount="1"}} + {{reuseForks="true"}}. After pressing Ctrl-C to stop the Maven test run, the process hangs and must be killed with SIGKILL. From the thread dump, {{CommandLineUtils.ProcessHook}} and {{StreamFeeder}} look responsible.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://jira.codehaus.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] (SUREFIRE-946) Maven hangs on SIGTERM when using Surefire
forking (CommandLineUtils.ProcessHook)
Posted by "Kristian Rosenvold (JIRA)" <ji...@codehaus.org>.
[ https://jira.codehaus.org/browse/SUREFIRE-946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=317056#comment-317056 ]
Kristian Rosenvold commented on SUREFIRE-946:
---------------------------------------------
Process termination is a tremendous PITA. Do not expect this to work at all.
I think it might be safer to send a "TERMINATE" down the commlink and do System.exit() in the fork.
> Maven hangs on SIGTERM when using Surefire forking (CommandLineUtils.ProcessHook)
> ---------------------------------------------------------------------------------
>
> Key: SUREFIRE-946
> URL: https://jira.codehaus.org/browse/SUREFIRE-946
> Project: Maven Surefire
> Issue Type: Bug
> Affects Versions: 2.13
> Reporter: Jesse Glick
> Fix For: 2.14
>
> Attachments: stack.txt, SUREFIRE-946.patch
>
>
> Java 7u7, Surefire with JUnit {{forkMode="perthread"}} + {{threadCount="1"}} + {{reuseForks="true"}}. After pressing Ctrl-C to stop the Maven test run, the process hangs and must be killed with SIGKILL. From the thread dump, {{CommandLineUtils.ProcessHook}} and {{StreamFeeder}} look responsible.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://jira.codehaus.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] (SUREFIRE-946) Maven hangs on SIGTERM when using Surefire
forking (CommandLineUtils.ProcessHook)
Posted by "Andreas Gudian (JIRA)" <ji...@codehaus.org>.
[ https://jira.codehaus.org/browse/SUREFIRE-946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=316801#comment-316801 ]
Andreas Gudian commented on SUREFIRE-946:
-----------------------------------------
On my Windows PC, hitting Ctrl+C correctly kills all processes. From the stack dumps, it looked to me as if the _forked_ process was killed off without the main process knowing. So I executed some test classes from which one calls {{System.exit(0)}} in its test method. Bang, I got the exact same stack dumps.
I fixed that by adding a shutdown hook in the forked process that will say "bye" to the main process if everything went fine, and report an abnormal termination if something else went wrong (such as a test calling {{System.exit}}). The main process then closes the {{TestProvidingInputStream}}, allowing the {{StreamFeeder}} thread to terminate.
Still, in case the shutdown hooks are not executed (e.g. due to {{Runtime.halt(..)}} or some other brutal external process termination, we will end up having the same problem. We would be back with that heartbeat thing...
Pull-request is open.
> Maven hangs on SIGTERM when using Surefire forking (CommandLineUtils.ProcessHook)
> ---------------------------------------------------------------------------------
>
> Key: SUREFIRE-946
> URL: https://jira.codehaus.org/browse/SUREFIRE-946
> Project: Maven Surefire
> Issue Type: Bug
> Affects Versions: 2.13
> Reporter: Jesse Glick
> Attachments: stack.txt
>
>
> Java 7u7, Surefire with JUnit {{forkMode="perthread"}} + {{threadCount="1"}} + {{reuseForks="true"}}. After pressing Ctrl-C to stop the Maven test run, the process hangs and must be killed with SIGKILL. From the thread dump, {{CommandLineUtils.ProcessHook}} and {{StreamFeeder}} look responsible.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://jira.codehaus.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] (SUREFIRE-946) Maven hangs on SIGTERM when using Surefire
forking (CommandLineUtils.ProcessHook)
Posted by "Kristian Rosenvold (JIRA)" <ji...@codehaus.org>.
[ https://jira.codehaus.org/browse/SUREFIRE-946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kristian Rosenvold closed SUREFIRE-946.
---------------------------------------
Resolution: Fixed
Assignee: Kristian Rosenvold
Fixed in a series of patches ;)
> Maven hangs on SIGTERM when using Surefire forking (CommandLineUtils.ProcessHook)
> ---------------------------------------------------------------------------------
>
> Key: SUREFIRE-946
> URL: https://jira.codehaus.org/browse/SUREFIRE-946
> Project: Maven Surefire
> Issue Type: Bug
> Affects Versions: 2.13
> Reporter: Jesse Glick
> Assignee: Kristian Rosenvold
> Fix For: 2.14
>
> Attachments: stack.txt, SUREFIRE-946.patch
>
>
> Java 7u7, Surefire with JUnit {{forkMode="perthread"}} + {{threadCount="1"}} + {{reuseForks="true"}}. After pressing Ctrl-C to stop the Maven test run, the process hangs and must be killed with SIGKILL. From the thread dump, {{CommandLineUtils.ProcessHook}} and {{StreamFeeder}} look responsible.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://jira.codehaus.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] (SUREFIRE-946) Maven hangs on SIGTERM when using Surefire
forking (CommandLineUtils.ProcessHook)
Posted by "Jesse Glick (JIRA)" <ji...@codehaus.org>.
[ https://jira.codehaus.org/browse/SUREFIRE-946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=316986#comment-316986 ]
Jesse Glick commented on SUREFIRE-946:
--------------------------------------
Proposed fix works in my case at least.
FYI my test case is: https://github.com/jenkinsci/jenkins/pull/667 (just edit root {{pom.xml}} to use {{2.14-SNAPSHOT}} then run {{mvn -DskipTests install && mvn -f test/pom.xml surefire:test}} and press Ctrl-C after some tests have started)
> Maven hangs on SIGTERM when using Surefire forking (CommandLineUtils.ProcessHook)
> ---------------------------------------------------------------------------------
>
> Key: SUREFIRE-946
> URL: https://jira.codehaus.org/browse/SUREFIRE-946
> Project: Maven Surefire
> Issue Type: Bug
> Affects Versions: 2.13
> Reporter: Jesse Glick
> Attachments: stack.txt, SUREFIRE-946.patch
>
>
> Java 7u7, Surefire with JUnit {{forkMode="perthread"}} + {{threadCount="1"}} + {{reuseForks="true"}}. After pressing Ctrl-C to stop the Maven test run, the process hangs and must be killed with SIGKILL. From the thread dump, {{CommandLineUtils.ProcessHook}} and {{StreamFeeder}} look responsible.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://jira.codehaus.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] (SUREFIRE-946) Maven hangs on SIGTERM when using Surefire
forking (CommandLineUtils.ProcessHook)
Posted by "Kristian Rosenvold (JIRA)" <ji...@codehaus.org>.
[ https://jira.codehaus.org/browse/SUREFIRE-946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=317095#comment-317095 ]
Kristian Rosenvold commented on SUREFIRE-946:
---------------------------------------------
I'm not planning to re-release verifier right now; can we do without the changes for this patch ? Does that only affect the IT ?
> Maven hangs on SIGTERM when using Surefire forking (CommandLineUtils.ProcessHook)
> ---------------------------------------------------------------------------------
>
> Key: SUREFIRE-946
> URL: https://jira.codehaus.org/browse/SUREFIRE-946
> Project: Maven Surefire
> Issue Type: Bug
> Affects Versions: 2.13
> Reporter: Jesse Glick
> Fix For: 2.14
>
> Attachments: stack.txt, SUREFIRE-946.patch
>
>
> Java 7u7, Surefire with JUnit {{forkMode="perthread"}} + {{threadCount="1"}} + {{reuseForks="true"}}. After pressing Ctrl-C to stop the Maven test run, the process hangs and must be killed with SIGKILL. From the thread dump, {{CommandLineUtils.ProcessHook}} and {{StreamFeeder}} look responsible.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://jira.codehaus.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] (SUREFIRE-946) Maven hangs on SIGTERM when using Surefire
forking (CommandLineUtils.ProcessHook)
Posted by "Kristian Rosenvold (JIRA)" <ji...@codehaus.org>.
[ https://jira.codehaus.org/browse/SUREFIRE-946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=317053#comment-317053 ]
Kristian Rosenvold commented on SUREFIRE-946:
---------------------------------------------
Before that I think you need to take a look at ConsoleOutputIT, which was broken with this fix ;)
It fails on the assert for "Printline in shutdown hook", which I assume is because we send BYE in the shutdown hook, and the hook with soutv runs afterwards. ;)
This fails because the CI runs on jdk1.5 and that has a different iteration order of maps than 1.6, and the shutdown hooks are in a map; which means they dont have a predictable order.
> Maven hangs on SIGTERM when using Surefire forking (CommandLineUtils.ProcessHook)
> ---------------------------------------------------------------------------------
>
> Key: SUREFIRE-946
> URL: https://jira.codehaus.org/browse/SUREFIRE-946
> Project: Maven Surefire
> Issue Type: Bug
> Affects Versions: 2.13
> Reporter: Jesse Glick
> Fix For: 2.14
>
> Attachments: stack.txt, SUREFIRE-946.patch
>
>
> Java 7u7, Surefire with JUnit {{forkMode="perthread"}} + {{threadCount="1"}} + {{reuseForks="true"}}. After pressing Ctrl-C to stop the Maven test run, the process hangs and must be killed with SIGKILL. From the thread dump, {{CommandLineUtils.ProcessHook}} and {{StreamFeeder}} look responsible.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://jira.codehaus.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] (SUREFIRE-946) Maven hangs on SIGTERM when using Surefire
forking (CommandLineUtils.ProcessHook)
Posted by "Kristian Rosenvold (JIRA)" <ji...@codehaus.org>.
[ https://jira.codehaus.org/browse/SUREFIRE-946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=317037#comment-317037 ]
Kristian Rosenvold commented on SUREFIRE-946:
---------------------------------------------
I have looked at the patch, and applied it to trunk in e54dbd810f62fe723800f11279b881bff244b707
@andread: I have a couple of minor considerations I'd like your input on;
TestProvidingInputStream has synchronized read method. As far as I can tell the use of this class is confined to a single thread? If this is not the case, then "close" must be synchronized (or "closed" volatile), since there is mismatched synchronization on this variable in the class. (Both reads and writes need to be synchronized for correctness). BUt it seems like this class is NotThreadSafe ?
In forkedBooter, I'm not sure I understand the added value provided by the ForkingRunListener.BOOTERCODE_CRASH, since by my meagre understanding, anything that is not BOOTERCODE_BYE implies a crash ?
(Which was the existing logic)
> Maven hangs on SIGTERM when using Surefire forking (CommandLineUtils.ProcessHook)
> ---------------------------------------------------------------------------------
>
> Key: SUREFIRE-946
> URL: https://jira.codehaus.org/browse/SUREFIRE-946
> Project: Maven Surefire
> Issue Type: Bug
> Affects Versions: 2.13
> Reporter: Jesse Glick
> Fix For: 2.14
>
> Attachments: stack.txt, SUREFIRE-946.patch
>
>
> Java 7u7, Surefire with JUnit {{forkMode="perthread"}} + {{threadCount="1"}} + {{reuseForks="true"}}. After pressing Ctrl-C to stop the Maven test run, the process hangs and must be killed with SIGKILL. From the thread dump, {{CommandLineUtils.ProcessHook}} and {{StreamFeeder}} look responsible.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://jira.codehaus.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] (SUREFIRE-946) Maven hangs on SIGTERM when using Surefire
forking (CommandLineUtils.ProcessHook)
Posted by "Jesse Glick (JIRA)" <ji...@codehaus.org>.
[ https://jira.codehaus.org/browse/SUREFIRE-946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=316584#comment-316584 ]
Jesse Glick commented on SUREFIRE-946:
--------------------------------------
Java 7u9, sorry.
> Maven hangs on SIGTERM when using Surefire forking (CommandLineUtils.ProcessHook)
> ---------------------------------------------------------------------------------
>
> Key: SUREFIRE-946
> URL: https://jira.codehaus.org/browse/SUREFIRE-946
> Project: Maven Surefire
> Issue Type: Bug
> Affects Versions: 2.13
> Reporter: Jesse Glick
> Attachments: stack.txt
>
>
> Java 7u7, Surefire with JUnit {{forkMode="perthread"}} + {{threadCount="1"}} + {{reuseForks="true"}}. After pressing Ctrl-C to stop the Maven test run, the process hangs and must be killed with SIGKILL. From the thread dump, {{CommandLineUtils.ProcessHook}} and {{StreamFeeder}} look responsible.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://jira.codehaus.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] (SUREFIRE-946) Maven hangs on SIGTERM when using Surefire
forking (CommandLineUtils.ProcessHook)
Posted by "Kristian Rosenvold (JIRA)" <ji...@codehaus.org>.
[ https://jira.codehaus.org/browse/SUREFIRE-946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=316643#comment-316643 ]
Kristian Rosenvold commented on SUREFIRE-946:
---------------------------------------------
Doh; seems like we'd have to make you a committer for me to do that.
So in the meantime I'll just keep track of your work here ;)
> Maven hangs on SIGTERM when using Surefire forking (CommandLineUtils.ProcessHook)
> ---------------------------------------------------------------------------------
>
> Key: SUREFIRE-946
> URL: https://jira.codehaus.org/browse/SUREFIRE-946
> Project: Maven Surefire
> Issue Type: Bug
> Affects Versions: 2.13
> Reporter: Jesse Glick
> Attachments: stack.txt
>
>
> Java 7u7, Surefire with JUnit {{forkMode="perthread"}} + {{threadCount="1"}} + {{reuseForks="true"}}. After pressing Ctrl-C to stop the Maven test run, the process hangs and must be killed with SIGKILL. From the thread dump, {{CommandLineUtils.ProcessHook}} and {{StreamFeeder}} look responsible.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://jira.codehaus.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] (SUREFIRE-946) Maven hangs on SIGTERM when using Surefire
forking (CommandLineUtils.ProcessHook)
Posted by "Jesse Glick (JIRA)" <ji...@codehaus.org>.
[ https://jira.codehaus.org/browse/SUREFIRE-946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=316721#comment-316721 ]
Jesse Glick commented on SUREFIRE-946:
--------------------------------------
Seems to happen only when {{reuseForks="true"}}.
Regardless of possible logic errors reading lines that never arrive, the immediate problem is that shutdown hooks must be very conservative so they are sure to complete promptly, and {{CommandLineUtils.ProcessHook}} seems to be deadlocking; it should not call {{Process.destroy}} without a timeout. (It could fork a thread calling {{destroy}} and wait a second or two for it to complete, else give up.) I am not sure this shutdown hook is actually necessary at allAFAIK Java would destroy the spawned process at shutdown anyway.
Since {{CommandLineUtils}} seems to buggy in this respect, I would recommend not using it at all. Just use {{ProcessBuilder}} directly.
> Maven hangs on SIGTERM when using Surefire forking (CommandLineUtils.ProcessHook)
> ---------------------------------------------------------------------------------
>
> Key: SUREFIRE-946
> URL: https://jira.codehaus.org/browse/SUREFIRE-946
> Project: Maven Surefire
> Issue Type: Bug
> Affects Versions: 2.13
> Reporter: Jesse Glick
> Attachments: stack.txt
>
>
> Java 7u7, Surefire with JUnit {{forkMode="perthread"}} + {{threadCount="1"}} + {{reuseForks="true"}}. After pressing Ctrl-C to stop the Maven test run, the process hangs and must be killed with SIGKILL. From the thread dump, {{CommandLineUtils.ProcessHook}} and {{StreamFeeder}} look responsible.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://jira.codehaus.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] (SUREFIRE-946) Maven hangs on SIGTERM when using Surefire
forking (CommandLineUtils.ProcessHook)
Posted by "Andreas Gudian (JIRA)" <ji...@codehaus.org>.
[ https://jira.codehaus.org/browse/SUREFIRE-946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=316927#comment-316927 ]
Andreas Gudian commented on SUREFIRE-946:
-----------------------------------------
Ok, please try https://github.com/agudian/maven-surefire/tree/SUREFIRE-946 (https://github.com/agudian/maven-surefire/archive/SUREFIRE-946.zip)
> Maven hangs on SIGTERM when using Surefire forking (CommandLineUtils.ProcessHook)
> ---------------------------------------------------------------------------------
>
> Key: SUREFIRE-946
> URL: https://jira.codehaus.org/browse/SUREFIRE-946
> Project: Maven Surefire
> Issue Type: Bug
> Affects Versions: 2.13
> Reporter: Jesse Glick
> Attachments: stack.txt, SUREFIRE-946.patch
>
>
> Java 7u7, Surefire with JUnit {{forkMode="perthread"}} + {{threadCount="1"}} + {{reuseForks="true"}}. After pressing Ctrl-C to stop the Maven test run, the process hangs and must be killed with SIGKILL. From the thread dump, {{CommandLineUtils.ProcessHook}} and {{StreamFeeder}} look responsible.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://jira.codehaus.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] (SUREFIRE-946) Maven hangs on SIGTERM when using Surefire
forking (CommandLineUtils.ProcessHook)
Posted by "Kristian Rosenvold (JIRA)" <ji...@codehaus.org>.
[ https://jira.codehaus.org/browse/SUREFIRE-946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=316999#comment-316999 ]
Kristian Rosenvold commented on SUREFIRE-946:
---------------------------------------------
I want to study this issue/fix a little closer before committing anything, so I'll push this through once I get time to sit down properly with it, which will stil be a few days from now.
> Maven hangs on SIGTERM when using Surefire forking (CommandLineUtils.ProcessHook)
> ---------------------------------------------------------------------------------
>
> Key: SUREFIRE-946
> URL: https://jira.codehaus.org/browse/SUREFIRE-946
> Project: Maven Surefire
> Issue Type: Bug
> Affects Versions: 2.13
> Reporter: Jesse Glick
> Fix For: 2.14
>
> Attachments: stack.txt, SUREFIRE-946.patch
>
>
> Java 7u7, Surefire with JUnit {{forkMode="perthread"}} + {{threadCount="1"}} + {{reuseForks="true"}}. After pressing Ctrl-C to stop the Maven test run, the process hangs and must be killed with SIGKILL. From the thread dump, {{CommandLineUtils.ProcessHook}} and {{StreamFeeder}} look responsible.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://jira.codehaus.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] (SUREFIRE-946) Maven hangs on SIGTERM when using Surefire
forking (CommandLineUtils.ProcessHook)
Posted by "Andreas Gudian (JIRA)" <ji...@codehaus.org>.
[ https://jira.codehaus.org/browse/SUREFIRE-946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=316924#comment-316924 ]
Andreas Gudian commented on SUREFIRE-946:
-----------------------------------------
Give me some minutes before starting to compile, I already see what I have to do...
> Maven hangs on SIGTERM when using Surefire forking (CommandLineUtils.ProcessHook)
> ---------------------------------------------------------------------------------
>
> Key: SUREFIRE-946
> URL: https://jira.codehaus.org/browse/SUREFIRE-946
> Project: Maven Surefire
> Issue Type: Bug
> Affects Versions: 2.13
> Reporter: Jesse Glick
> Attachments: stack.txt, SUREFIRE-946.patch
>
>
> Java 7u7, Surefire with JUnit {{forkMode="perthread"}} + {{threadCount="1"}} + {{reuseForks="true"}}. After pressing Ctrl-C to stop the Maven test run, the process hangs and must be killed with SIGKILL. From the thread dump, {{CommandLineUtils.ProcessHook}} and {{StreamFeeder}} look responsible.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://jira.codehaus.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] (SUREFIRE-946) Maven hangs on SIGTERM when using Surefire
forking (CommandLineUtils.ProcessHook)
Posted by "Andreas Gudian (JIRA)" <ji...@codehaus.org>.
[ https://jira.codehaus.org/browse/SUREFIRE-946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=317055#comment-317055 ]
Andreas Gudian commented on SUREFIRE-946:
-----------------------------------------
I guess I'll first try to remove it all together by adding that callback to CommandLineUtils... Then there would only be a problem in case {{Process}} does not get when the forked process is termintated - does that happen?
> Maven hangs on SIGTERM when using Surefire forking (CommandLineUtils.ProcessHook)
> ---------------------------------------------------------------------------------
>
> Key: SUREFIRE-946
> URL: https://jira.codehaus.org/browse/SUREFIRE-946
> Project: Maven Surefire
> Issue Type: Bug
> Affects Versions: 2.13
> Reporter: Jesse Glick
> Fix For: 2.14
>
> Attachments: stack.txt, SUREFIRE-946.patch
>
>
> Java 7u7, Surefire with JUnit {{forkMode="perthread"}} + {{threadCount="1"}} + {{reuseForks="true"}}. After pressing Ctrl-C to stop the Maven test run, the process hangs and must be killed with SIGKILL. From the thread dump, {{CommandLineUtils.ProcessHook}} and {{StreamFeeder}} look responsible.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://jira.codehaus.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] (SUREFIRE-946) Maven hangs on SIGTERM when using Surefire
forking (CommandLineUtils.ProcessHook)
Posted by "Andreas Gudian (JIRA)" <ji...@codehaus.org>.
[ https://jira.codehaus.org/browse/SUREFIRE-946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=316635#comment-316635 ]
Andreas Gudian commented on SUREFIRE-946:
-----------------------------------------
@kristian: I'd like to give it a try. Can you assign the issue to me?
> Maven hangs on SIGTERM when using Surefire forking (CommandLineUtils.ProcessHook)
> ---------------------------------------------------------------------------------
>
> Key: SUREFIRE-946
> URL: https://jira.codehaus.org/browse/SUREFIRE-946
> Project: Maven Surefire
> Issue Type: Bug
> Affects Versions: 2.13
> Reporter: Jesse Glick
> Attachments: stack.txt
>
>
> Java 7u7, Surefire with JUnit {{forkMode="perthread"}} + {{threadCount="1"}} + {{reuseForks="true"}}. After pressing Ctrl-C to stop the Maven test run, the process hangs and must be killed with SIGKILL. From the thread dump, {{CommandLineUtils.ProcessHook}} and {{StreamFeeder}} look responsible.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://jira.codehaus.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] (SUREFIRE-946) Maven hangs on SIGTERM when using Surefire
forking (CommandLineUtils.ProcessHook)
Posted by "Andreas Gudian (JIRA)" <ji...@codehaus.org>.
[ https://jira.codehaus.org/browse/SUREFIRE-946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andreas Gudian updated SUREFIRE-946:
------------------------------------
Attachment: SUREFIRE-946.patch
Added patch file containing the commits of github pull request https://github.com/apache/maven-surefire/pull/20
> Maven hangs on SIGTERM when using Surefire forking (CommandLineUtils.ProcessHook)
> ---------------------------------------------------------------------------------
>
> Key: SUREFIRE-946
> URL: https://jira.codehaus.org/browse/SUREFIRE-946
> Project: Maven Surefire
> Issue Type: Bug
> Affects Versions: 2.13
> Reporter: Jesse Glick
> Attachments: stack.txt, SUREFIRE-946.patch
>
>
> Java 7u7, Surefire with JUnit {{forkMode="perthread"}} + {{threadCount="1"}} + {{reuseForks="true"}}. After pressing Ctrl-C to stop the Maven test run, the process hangs and must be killed with SIGKILL. From the thread dump, {{CommandLineUtils.ProcessHook}} and {{StreamFeeder}} look responsible.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://jira.codehaus.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] (SUREFIRE-946) Maven hangs on SIGTERM when using Surefire
forking (CommandLineUtils.ProcessHook)
Posted by "Andreas Gudian (JIRA)" <ji...@codehaus.org>.
[ https://jira.codehaus.org/browse/SUREFIRE-946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=317069#comment-317069 ]
Andreas Gudian commented on SUREFIRE-946:
-----------------------------------------
Having thought about this over night, I'm pretty sure my Surefire946KillMainProcessInReusableForkIT does not yet test anything, as a) verifier does not fork by default in the current configuration, and b) I'd have to add some code to check if the process already terminated (e.g. by checking if the thread named {{Thread.currentThread().getName()+ "-ProcessKiller"}} is still alive). I'll fix that tonight.
> Maven hangs on SIGTERM when using Surefire forking (CommandLineUtils.ProcessHook)
> ---------------------------------------------------------------------------------
>
> Key: SUREFIRE-946
> URL: https://jira.codehaus.org/browse/SUREFIRE-946
> Project: Maven Surefire
> Issue Type: Bug
> Affects Versions: 2.13
> Reporter: Jesse Glick
> Fix For: 2.14
>
> Attachments: stack.txt, SUREFIRE-946.patch
>
>
> Java 7u7, Surefire with JUnit {{forkMode="perthread"}} + {{threadCount="1"}} + {{reuseForks="true"}}. After pressing Ctrl-C to stop the Maven test run, the process hangs and must be killed with SIGKILL. From the thread dump, {{CommandLineUtils.ProcessHook}} and {{StreamFeeder}} look responsible.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://jira.codehaus.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] (SUREFIRE-946) Maven hangs on SIGTERM when using Surefire
forking (CommandLineUtils.ProcessHook)
Posted by "Andreas Gudian (JIRA)" <ji...@codehaus.org>.
[ https://jira.codehaus.org/browse/SUREFIRE-946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=316646#comment-316646 ]
Andreas Gudian commented on SUREFIRE-946:
-----------------------------------------
Fine with me :)
> Maven hangs on SIGTERM when using Surefire forking (CommandLineUtils.ProcessHook)
> ---------------------------------------------------------------------------------
>
> Key: SUREFIRE-946
> URL: https://jira.codehaus.org/browse/SUREFIRE-946
> Project: Maven Surefire
> Issue Type: Bug
> Affects Versions: 2.13
> Reporter: Jesse Glick
> Attachments: stack.txt
>
>
> Java 7u7, Surefire with JUnit {{forkMode="perthread"}} + {{threadCount="1"}} + {{reuseForks="true"}}. After pressing Ctrl-C to stop the Maven test run, the process hangs and must be killed with SIGKILL. From the thread dump, {{CommandLineUtils.ProcessHook}} and {{StreamFeeder}} look responsible.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://jira.codehaus.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] (SUREFIRE-946) Maven hangs on SIGTERM when using Surefire
forking (CommandLineUtils.ProcessHook)
Posted by "Kristian Rosenvold (JIRA)" <ji...@codehaus.org>.
[ https://jira.codehaus.org/browse/SUREFIRE-946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=317038#comment-317038 ]
Kristian Rosenvold commented on SUREFIRE-946:
---------------------------------------------
Btw the "timeout-forked" IT-project can be run with mvn -DsleepLength=1000000 -Dsurefire.version=2.14-SNAPSHOT -DforkMode=perthread -DreuseForks=true -DthreadCount=2 test
to test this issue. It might even be possble to make a testcase for this, /me wonders.
> Maven hangs on SIGTERM when using Surefire forking (CommandLineUtils.ProcessHook)
> ---------------------------------------------------------------------------------
>
> Key: SUREFIRE-946
> URL: https://jira.codehaus.org/browse/SUREFIRE-946
> Project: Maven Surefire
> Issue Type: Bug
> Affects Versions: 2.13
> Reporter: Jesse Glick
> Fix For: 2.14
>
> Attachments: stack.txt, SUREFIRE-946.patch
>
>
> Java 7u7, Surefire with JUnit {{forkMode="perthread"}} + {{threadCount="1"}} + {{reuseForks="true"}}. After pressing Ctrl-C to stop the Maven test run, the process hangs and must be killed with SIGKILL. From the thread dump, {{CommandLineUtils.ProcessHook}} and {{StreamFeeder}} look responsible.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://jira.codehaus.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira