[jira] [Created] (SPARK-16139) Audit tests for leaked threads

["Imran Rashid (JIRA)" <ji...@apache.org>]

Imran Rashid created SPARK-16139:
------------------------------------

             Summary: Audit tests for leaked threads
                 Key: SPARK-16139
                 URL: https://issues.apache.org/jira/browse/SPARK-16139
             Project: Spark
          Issue Type: Test
          Components: Tests
    Affects Versions: 2.0.0
            Reporter: Imran Rashid


Lots of our tests don't properly shutdown everything they create, and end up leaking lots of threads.  For example, {{TaskSetManagerSuite}} doesn't stop the extra {{TaskScheduler}} and {{DAGScheduler}} it creates.  There are a couple more instances I've run into recently, eg. in [{{DAGSchedulerSuite}}|https://github.com/apache/spark/commit/cf1995a97645f0b44c997f4fdbba631fd6b91a16#diff-f3b410b16818d8f34bb1eb4120a60d51R235 ]

I'm fixing these piecemeal when I see them (eg., TaskSetManagerSuite should be fixed by my pr for SPARK-16136), but it would be great to have a comprehensive audit and fix this across all tests.

This should be semi-automatable.  In {{SparkFunSuite}}, you could grab all threads before the tests starts, and after it completes.  Then you could clearly log all threads started after the test started but still going.  Unfortunately this isn't perfect, it seems that netty threads aren't killed immediately on shutdown, . Its OK if some of them linger beyond the test, so you may need to do some whitelisting based on thread-name & a little more manual inspection.  But you could at least clearly log the relevant info, so that after a jenkins run you could easily pull the info from the logs.

Bonus points if you can figure out some way to make this output visible outside of the logs, ideally even in the test report that makes it to github, but that isn't necessary, and unless its very easy probably best for a separate task.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org