You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Mark Grover (JIRA)" <ji...@apache.org> on 2015/08/10 21:01:46 UTC

[jira] [Commented] (SPARK-9790) [YARN] Expose in WebUI if NodeManager is the reason why executors were killed.

    [ https://issues.apache.org/jira/browse/SPARK-9790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14680570#comment-14680570 ] 

Mark Grover commented on SPARK-9790:
------------------------------------

I am working on this, will file a Work-In-Progress pull request soon.

> [YARN] Expose in WebUI if NodeManager is the reason why executors were killed.
> ------------------------------------------------------------------------------
>
>                 Key: SPARK-9790
>                 URL: https://issues.apache.org/jira/browse/SPARK-9790
>             Project: Spark
>          Issue Type: Bug
>          Components: YARN
>    Affects Versions: 1.4.1
>            Reporter: Mark Grover
>
> When an executor is killed by yarn because it exceeds the memory overhead, the only thing spark knows is that the executor is lost. The user has to go track search through the NM logs to figure out that its been killed by yarn.
> It would be much nicer if the spark-driver could be notified why the executor was killed. Ideally it could both log an explanatory message, and update the UI (and the eventLog) so that it was clear why the executor was lost.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org