[jira] [Created] (KUDU-1669) Java ITClient test can orphan processes

["Adar Dembo (JIRA)" <ji...@apache.org>]

Adar Dembo created KUDU-1669:
--------------------------------

             Summary: Java ITClient test can orphan processes
                 Key: KUDU-1669
                 URL: https://issues.apache.org/jira/browse/KUDU-1669
             Project: Kudu
          Issue Type: Bug
          Components: client
    Affects Versions: 1.0.0
            Reporter: Adar Dembo


Saw this with ITClient but it can happen to any test that uses a MiniCluster, restarts a process, and interrupts its own threads. Maybe ITClient is the only example of that.

Here's the interesting stack trace:
{noformat}
21:18:12.316 [ERROR - Thread-8] (ITClient.java:134) Couldn't restart a master
java.lang.InterruptedException: sleep interrupted
        at java.lang.Thread.sleep(Native Method)
        at org.apache.kudu.client.MiniKuduCluster.configureAndStartProcess(MiniKuduCluster.java:239)
        at org.apache.kudu.client.MiniKuduCluster.restartDeadProcessOnPort(MiniKuduCluster.java:282)
        at org.apache.kudu.client.MiniKuduCluster.restartDeadMasterOnPort(MiniKuduCluster.java:256)
        at org.apache.kudu.client.BaseKuduTest.restartLeaderMaster(BaseKuduTest.java:431)
        at org.apache.kudu.client.ITClient$ChaosThread.restartMaster(ITClient.java:222)
        at org.apache.kudu.client.ITClient$ChaosThread.run(ITClient.java:161)
        at java.lang.Thread.run(Thread.java:745)
{noformat}

Because of this thread interruption, the newly restarted process never makes it into BaseKuduTest.masterProcesses, which means it isn't destroyed when the test class cleans up. All sorts of bad stuff can happen then. It's possible the process is completely orphaned on the test machine (though I imagine we'd kill it eventually). I noticed this because in one of my precommit test runs ITClient left behind a test directory; presumably because the orphaned master continued to write files even after the test directory was cleaned up.




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)