You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Dongjoon Hyun (Jira)" <ji...@apache.org> on 2023/09/24 02:04:00 UTC
[jira] [Assigned] (SPARK-44345) Only log unknown shuffle map output as error when shuffle migration disabled
[ https://issues.apache.org/jira/browse/SPARK-44345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dongjoon Hyun reassigned SPARK-44345:
-------------------------------------
Assignee: Zhongwei Zhu
> Only log unknown shuffle map output as error when shuffle migration disabled
> ----------------------------------------------------------------------------
>
> Key: SPARK-44345
> URL: https://issues.apache.org/jira/browse/SPARK-44345
> Project: Spark
> Issue Type: Improvement
> Components: Spark Core
> Affects Versions: 3.4.1
> Reporter: Zhongwei Zhu
> Assignee: Zhongwei Zhu
> Priority: Minor
> Labels: pull-request-available
>
> When decommission and shuffle migration is enabled, there're lots of error message like `Asked to update map output for unknown shuffle` .
> As shuffle clean and unregister is done by ContextCleaner in an async way, when target block manager received the shuffle block from decommissioned block manager, then update shuffle location to map output tracker, but at that time, shuffle might have been unregistered. This should not consider as error.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org