[GitHub] [spark] liupc edited a comment on issue #27604: [SPARK-30849][CORE][SHUFFLE]Fix application failed due to failed to get MapStatuses broadcast block

[GitBox <gi...@apache.org>]

liupc edited a comment on issue #27604: [SPARK-30849][CORE][SHUFFLE]Fix application failed due to failed to get MapStatuses broadcast block
URL: https://github.com/apache/spark/pull/27604#issuecomment-595222617
 
 
   > But the max task failure is for the same task rather than several tasks in a same stage?
   
   Yes，I mean the fetch failure from the executor may cause several mapStatus to be removed and recomputed, thus several tasks will be re-executed in parent stage, so each of those task finish the cached mapStatuses will be invalidated, so the task of child stage 2 described in the jira may repeatedly encounter the IOException. @Ngone51 , I will try to write some tests or do some hack to reproduce this issue.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org