You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2023/09/23 18:25:00 UTC
[jira] [Updated] (SPARK-44345) Only log unknown shuffle map output as error when shuffle migration disabled
[ https://issues.apache.org/jira/browse/SPARK-44345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
ASF GitHub Bot updated SPARK-44345:
-----------------------------------
Labels: pull-request-available (was: )
> Only log unknown shuffle map output as error when shuffle migration disabled
> ----------------------------------------------------------------------------
>
> Key: SPARK-44345
> URL: https://issues.apache.org/jira/browse/SPARK-44345
> Project: Spark
> Issue Type: Improvement
> Components: Spark Core
> Affects Versions: 3.4.1
> Reporter: Zhongwei Zhu
> Priority: Minor
> Labels: pull-request-available
>
> When decommission and shuffle migration is enabled, there're lots of error message like `Asked to update map output for unknown shuffle` .
> As shuffle clean and unregister is done by ContextCleaner in an async way, when target block manager received the shuffle block from decommissioned block manager, then update shuffle location to map output tracker, but at that time, shuffle might have been unregistered. This should not consider as error.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org