[GitHub] [spark] liupc edited a comment on issue #27604: [SPARK-30849][CORE][SHUFFLE]Fix application failed due to failed to get MapStatuses broadcast block

[GitBox <gi...@apache.org>]

liupc edited a comment on issue #27604: [SPARK-30849][CORE][SHUFFLE]Fix application failed due to failed to get MapStatuses broadcast block
URL: https://github.com/apache/spark/pull/27604#issuecomment-590319139
 
 
   > But I'm also surprised and doubt after reading JIRA ticket and error log that how does a same task hit this issue continuously 4 times? Is it only a coincidence or I miss something? 
   
   currently, when `registerMapOutput` we will invalidate the cached mapStatus broadcast, that means this mapStatus broadcast will be destroyed when the retrying tasks of the parent stage finished. So if there are serval tasks retried for parent stage, then there are large possibility that we will encounter this issue.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org