[jira] [Resolved] (SPARK-45498) Followup: Ignore task completion from old stage after retrying indeterminate stages

["Wenchen Fan (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SPARK-45498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Wenchen Fan resolved SPARK-45498.
---------------------------------
    Fix Version/s: 3.5.1
                   4.0.0
       Resolution: Fixed

Issue resolved by pull request 43326
[https://github.com/apache/spark/pull/43326]

> Followup: Ignore task completion from old stage after retrying indeterminate stages
> -----------------------------------------------------------------------------------
>
>                 Key: SPARK-45498
>                 URL: https://issues.apache.org/jira/browse/SPARK-45498
>             Project: Spark
>          Issue Type: Bug
>          Components: Spark Core
>    Affects Versions: 4.0.0, 3.5.1
>            Reporter: Mayur Bhosale
>            Assignee: Mayur Bhosale
>            Priority: Minor
>              Labels: pull-request-available
>             Fix For: 3.5.1, 4.0.0
>
>
> With SPARK-45182, we added a fix for not letting laggard tasks of the older attempts of the indeterminate stage from marking the partition has completed in map output tracker.
> When a task completes, DAG scheduler also notifies all the tasksets of the stage about that partition being completed. Tasksets would not schedule such task if they are not already scheduled. This is not correct for indeterminate stage, since we want to re-run all the tasks on re-attempt



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org