You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "liziyan (Jira)" <ji...@apache.org> on 2024/11/27 11:52:00 UTC

[jira] [Comment Edited] (SPARK-50288) Executors of stage failed still alive even through stage has been retried

    [ https://issues.apache.org/jira/browse/SPARK-50288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17901455#comment-17901455 ] 

liziyan edited comment on SPARK-50288 at 11/27/24 11:51 AM:
------------------------------------------------------------

'spark.excludeOnFailure.enabled' might be helpful.


was (Author: JIRAUSER307819):
'spark.excludeOnFailure.enabled' might be help.

> Executors of stage failed still alive even through stage has been retried
> -------------------------------------------------------------------------
>
>                 Key: SPARK-50288
>                 URL: https://issues.apache.org/jira/browse/SPARK-50288
>             Project: Spark
>          Issue Type: Question
>          Components: Shuffle, Spark Core
>    Affects Versions: 3.3.0, 3.5.2
>            Reporter: Yu-Ting LIN
>            Priority: Major
>         Attachments: stage19_stage19retry1.png, trigger_duplicate_process.png
>
>
> We are executing spark dataframe API with foreachPartition and we observed something that we are not able to explain.
> In the figure that we provided, you can see that stage 19 has been retried due to some ShuffleOutputNotFound error and stage 19 has been retried 1.
> !stage19_stage19retry1.png|width=1150,height=603!
> However, we found that there are still some executors allocated for stage 19 in state RUNNING e.g. partition 371 and 200. In addition, as not yet finished, partition 371 and 200 have been resubmitted again in stage 19 (retry 1). 
> !trigger_duplicate_process.png|width=1144,height=402!
> Is there any configuration can help us make sure that all executor from the stage failed will be terminated before retry staged been triggered.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org