You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Venkata krishnan Sowrirajan (Jira)" <ji...@apache.org> on 2022/02/02 19:22:00 UTC

[jira] [Created] (SPARK-38092) Check if shuffleMergeId is the same as the current stage's shuffleMergeId before registering MergeStatus

Venkata krishnan Sowrirajan created SPARK-38092:
---------------------------------------------------

             Summary: Check if shuffleMergeId is the same as the current stage's shuffleMergeId before registering MergeStatus
                 Key: SPARK-38092
                 URL: https://issues.apache.org/jira/browse/SPARK-38092
             Project: Spark
          Issue Type: Sub-task
          Components: Shuffle
    Affects Versions: 3.2.1
            Reporter: Venkata krishnan Sowrirajan


Currently we have handled this in the handleShuffleMergeFinalized during finalization ensuring the finalize request is indeed for the current stage's shuffle dependency shuffleMergeId. The same check has to be done before registering merge statuses as well.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org