[jira] [Updated] (KUDU-1669) Java ITClient test can orphan processes

["Adar Dembo (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/KUDU-1669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Adar Dembo updated KUDU-1669:
-----------------------------
    Attachment: org.apache.kudu.client.ITClient-output.txt.gz

Attaching the full test output from ITClient.

The test didn't actually fail, but later build-and-test.sh complained with:
{noformat}
21:22:16 All tests passed, yet some left behind their test output:
21:22:16 master-2-1475270228629
{noformat}


> Java ITClient test can orphan processes
> ---------------------------------------
>
>                 Key: KUDU-1669
>                 URL: https://issues.apache.org/jira/browse/KUDU-1669
>             Project: Kudu
>          Issue Type: Bug
>          Components: client
>    Affects Versions: 1.0.0
>            Reporter: Adar Dembo
>         Attachments: org.apache.kudu.client.ITClient-output.txt.gz
>
>
> Saw this with ITClient but it can happen to any test that uses a MiniCluster, restarts a process, and interrupts its own threads. Maybe ITClient is the only example of that.
> Here's the interesting stack trace:
> {noformat}
> 21:18:12.316 [ERROR - Thread-8] (ITClient.java:134) Couldn't restart a master
> java.lang.InterruptedException: sleep interrupted
>         at java.lang.Thread.sleep(Native Method)
>         at org.apache.kudu.client.MiniKuduCluster.configureAndStartProcess(MiniKuduCluster.java:239)
>         at org.apache.kudu.client.MiniKuduCluster.restartDeadProcessOnPort(MiniKuduCluster.java:282)
>         at org.apache.kudu.client.MiniKuduCluster.restartDeadMasterOnPort(MiniKuduCluster.java:256)
>         at org.apache.kudu.client.BaseKuduTest.restartLeaderMaster(BaseKuduTest.java:431)
>         at org.apache.kudu.client.ITClient$ChaosThread.restartMaster(ITClient.java:222)
>         at org.apache.kudu.client.ITClient$ChaosThread.run(ITClient.java:161)
>         at java.lang.Thread.run(Thread.java:745)
> {noformat}
> Because of this thread interruption, the newly restarted process never makes it into BaseKuduTest.masterProcesses, which means it isn't destroyed when the test class cleans up. All sorts of bad stuff can happen then. It's possible the process is completely orphaned on the test machine (though I imagine we'd kill it eventually). I noticed this because in one of my precommit test runs ITClient left behind a test directory; presumably because the orphaned master continued to write files even after the test directory was cleaned up.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)