[jira] [Assigned] (SPARK-22123) Add latest failure reason for task set blacklist

["Saisai Shao (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SPARK-22123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Saisai Shao reassigned SPARK-22123:
-----------------------------------

    Assignee: zhoukang

> Add latest failure reason for task set blacklist
> ------------------------------------------------
>
>                 Key: SPARK-22123
>                 URL: https://issues.apache.org/jira/browse/SPARK-22123
>             Project: Spark
>          Issue Type: Improvement
>          Components: Scheduler, Spark Core
>    Affects Versions: 2.1.0, 2.2.0
>            Reporter: zhoukang
>            Assignee: zhoukang
>             Fix For: 2.3.0
>
>
> Till now , every job which aborted by completed blacklist just show log like below which has no more information:
> {code:java}
> Aborting $taskSet because task $indexInTaskSet (partition $partition) cannot run anywhere due to node and executor blacklist. Blacklisting behavior cannot run anywhere due to node and executor blacklist.Blacklisting behavior can be configured via spark.blacklist.*."
> {code}
> We could add most recent failure reason for taskset blacklist which can be showed on spark ui to let user know failure reason directly.
> An example after modifying:
> {code:java}
> Aborting TaskSet 0.0 because task 0 (partition 0) cannot run anywhere due to node and executor blacklist.
>  Most recent failure:
>  Some(Lost task 0.1 in stage 0.0 (TID 3,xxx, executor 1): java.lang.Exception: Fake error!
>  at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:73)
>  at org.apache.spark.scheduler.Task.run(Task.scala:99)
>  at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:305)
>  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>  at java.lang.Thread.run(Thread.java:745)
>  ). 
> Blacklisting behavior can be configured via spark.blacklist.*.
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org