You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Tengfei Huang (Jira)" <ji...@apache.org> on 2025/06/04 02:37:00 UTC

[jira] [Commented] (SPARK-52395) Fail fast on the first shuffle block fetch failure

    [ https://issues.apache.org/jira/browse/SPARK-52395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17956012#comment-17956012 ] 

Tengfei Huang commented on SPARK-52395:
---------------------------------------

Will open a PR soon.

> Fail fast on the first shuffle block fetch failure
> --------------------------------------------------
>
>                 Key: SPARK-52395
>                 URL: https://issues.apache.org/jira/browse/SPARK-52395
>             Project: Spark
>          Issue Type: Improvement
>          Components: Spark Core
>    Affects Versions: 4.0.0
>            Reporter: Tengfei Huang
>            Priority: Major
>
> Shuffle blocks are handled in sequence to guarantee thread safety on handling shuffle data. This becomes an issue when the first block fetch failure is admitted but plenty of succeeded blocks are already in the queue. This leads to task runs for meaningless/wasted until the failed block is handled. 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org