You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Apache Spark (Jira)" <ji...@apache.org> on 2022/10/31 02:58:00 UTC

[jira] [Commented] (SPARK-40481) Ignore stage fetch failure caused by decommissioned executor

    [ https://issues.apache.org/jira/browse/SPARK-40481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17626353#comment-17626353 ] 

Apache Spark commented on SPARK-40481:
--------------------------------------

User 'warrenzhu25' has created a pull request for this issue:
https://github.com/apache/spark/pull/38441

> Ignore stage fetch failure caused by decommissioned executor
> ------------------------------------------------------------
>
>                 Key: SPARK-40481
>                 URL: https://issues.apache.org/jira/browse/SPARK-40481
>             Project: Spark
>          Issue Type: Improvement
>          Components: Spark Core
>    Affects Versions: 3.4.0
>            Reporter: Zhongwei Zhu
>            Assignee: Zhongwei Zhu
>            Priority: Minor
>             Fix For: 3.4.0
>
>
> When executor decommission is enabled, there would be many stage failure caused by FetchFailed from decommissioned executor, further causing whole job's failure. It would be better not to count such failure in `spark.stage.maxConsecutiveAttempts`



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org