You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Marcelo Vanzin (JIRA)" <ji...@apache.org> on 2015/11/03 17:52:27 UTC
[jira] [Resolved] (SPARK-9790) [YARN] Expose in WebUI if
NodeManager is the reason why executors were killed.
[ https://issues.apache.org/jira/browse/SPARK-9790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Marcelo Vanzin resolved SPARK-9790.
-----------------------------------
Resolution: Fixed
Assignee: Mark Grover
Fix Version/s: 1.6.0
> [YARN] Expose in WebUI if NodeManager is the reason why executors were killed.
> ------------------------------------------------------------------------------
>
> Key: SPARK-9790
> URL: https://issues.apache.org/jira/browse/SPARK-9790
> Project: Spark
> Issue Type: Improvement
> Components: YARN
> Affects Versions: 1.4.1
> Reporter: Mark Grover
> Assignee: Mark Grover
> Priority: Minor
> Fix For: 1.6.0
>
> Attachments: error_showing_in_UI.png
>
>
> When an executor is killed by yarn because it exceeds the memory overhead, the only thing spark knows is that the executor is lost. The user has to go track search through the NM logs to figure out that its been killed by yarn.
> It would be much nicer if the spark-driver could be notified why the executor was killed. Ideally it could both log an explanatory message, and update the UI (and the eventLog) so that it was clear why the executor was lost.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org