You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Alex Angelini (JIRA)" <ji...@apache.org> on 2015/08/12 05:58:45 UTC
[jira] [Commented] (SPARK-9745) Applications hangs when the last
executor fails with dynamic allocation
[ https://issues.apache.org/jira/browse/SPARK-9745?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14692875#comment-14692875 ]
Alex Angelini commented on SPARK-9745:
--------------------------------------
I was not able to reproduce this issue after merging: https://github.com/apache/spark/pull/7716
> Applications hangs when the last executor fails with dynamic allocation
> -----------------------------------------------------------------------
>
> Key: SPARK-9745
> URL: https://issues.apache.org/jira/browse/SPARK-9745
> Project: Spark
> Issue Type: Bug
> Components: PySpark, Scheduler, YARN
> Affects Versions: 1.5.0
> Environment: YARN + Pyspark + Dynamic Allocation
> Reporter: Alex Angelini
> Assignee: Andrew Or
> Priority: Blocker
> Attachments: am_hung_job.png, executors_hung_job.png, logs_hung_job.png, tasks_hung_job.png
>
>
> When a job has only a single executor remaining and that executor dies (due to something like an OOM), the application fails to notice that there are no executors left and it hangs indefinitely.
> This only happens when dynamic allocation is enabled.
> The following images were taken from a hung application with no executors:
> !logs_hung_job.png!
> ^^ *Notice how 1 executor was lost, but the application never requested it to be removed*
> !am_hung_job.png!
> !executors_hung_job.png!
> !tasks_hung_job.png!
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org