[jira] [Resolved] (KUDU-1453) Spark executors leak kudu clients and netty threads

["Todd Lipcon (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/KUDU-1453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Todd Lipcon resolved KUDU-1453.
-------------------------------
       Resolution: Fixed
    Fix Version/s: 0.9.0

> Spark executors leak kudu clients and netty threads
> ---------------------------------------------------
>
>                 Key: KUDU-1453
>                 URL: https://issues.apache.org/jira/browse/KUDU-1453
>             Project: Kudu
>          Issue Type: Bug
>          Components: spark
>    Affects Versions: 0.8.0
>            Reporter: Todd Lipcon
>            Assignee: Brent Gardner
>            Priority: Blocker
>             Fix For: 0.9.0
>
>
> On a test cluster, every time I run a spark SQL query against a table, each of my spark worker tasks end up with another ~500 netty worker threads created. It seems like each Spark partition/task is creating its own KuduClient which then creates a bunch of worker threads and never cleans them up.
> I'm calling this a blocker since after ~20 queries or so, the machines woudl run out of threads and crash.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)