You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "EdisonWang (Jira)" <ji...@apache.org> on 2022/02/09 08:49:00 UTC

[jira] [Created] (SPARK-38160) Shuffle by rand could lead to incorrect answers when ShuffleFetchFailed happend

EdisonWang created SPARK-38160:
----------------------------------

             Summary: Shuffle by rand could lead to incorrect answers when ShuffleFetchFailed happend
                 Key: SPARK-38160
                 URL: https://issues.apache.org/jira/browse/SPARK-38160
             Project: Spark
          Issue Type: Bug
          Components: SQL
    Affects Versions: 3.3.0
            Reporter: EdisonWang


When we do shuffle on indeterminate expressions such as rand, and ShuffleFetchFailed happend, we may get incorrent result since it only retries failed map tasks.

We try to fix this by retry all upstream map tasks in this situation.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org