[jira] [Commented] (SPARK-672) Executor gets stuck in a "zombie" state after running out of memory

["Andrew Ash (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SPARK-672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14212020#comment-14212020 ] 

Andrew Ash commented on SPARK-672:
----------------------------------

[~mbautin] when an executor JVM is under extremely heavy GC load, it will often lock up, not even responding to normal kill commands and requiring a kill -9 to shut down.  I agree that Spark could behave better in these situations.

What is your preference for handling these issues -- maybe give the executor a timeout and kill -9 it after a certain period of time of non-responsiveness?

> Executor gets stuck in a "zombie" state after running out of memory
> -------------------------------------------------------------------
>
>                 Key: SPARK-672
>                 URL: https://issues.apache.org/jira/browse/SPARK-672
>             Project: Spark
>          Issue Type: Bug
>            Reporter: Mikhail Bautin
>         Attachments: executor_jstack.txt, executor_stderr.txt, standalone_worker_jstack.txt
>
>
> As a result of running a workload, an executor ran out of memory, but the executor process stayed up. Also (not sure this is related) the standalone worker process stayed up but disappeared from the master web UI.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org