You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Dongjoon Hyun (Jira)" <ji...@apache.org> on 2022/12/14 05:55:00 UTC
[jira] [Updated] (SPARK-39324) Log ExecutorDecommission as INFO level in TaskSchedulerImpl
[ https://issues.apache.org/jira/browse/SPARK-39324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dongjoon Hyun updated SPARK-39324:
----------------------------------
Parent: SPARK-41515
Issue Type: Sub-task (was: Improvement)
> Log ExecutorDecommission as INFO level in TaskSchedulerImpl
> -----------------------------------------------------------
>
> Key: SPARK-39324
> URL: https://issues.apache.org/jira/browse/SPARK-39324
> Project: Spark
> Issue Type: Sub-task
> Components: Spark Core
> Affects Versions: 3.4.0
> Reporter: Dongjoon Hyun
> Assignee: Dongjoon Hyun
> Priority: Major
> Fix For: 3.4.0
>
>
> Like the other module, `TaskSchedulerImpl` should log the decommission as `INFO` level.
> {code}
> 22/05/28 01:25:28 INFO KubernetesClusterSchedulerBackend: Decommission executors: 8
> 22/05/28 01:25:28 INFO KubernetesClusterSchedulerBackend: Notify executor 8 to decommissioning.
> 22/05/28 01:25:28 INFO BlockManagerMasterEndpoint: Mark BlockManagers (BlockManagerId(8, 100.103.40.13, 43353, None)) as being decommissioning.
> 22/05/28 01:25:29 ERROR TaskSchedulerImpl: Lost executor 8 on 100.103.40.13: Executor decommission.
> 22/05/28 01:25:29 INFO ExecutorMonitor: Executor 8 is removed. Remove reason statistics: ...
> 22/05/28 01:25:29 INFO DAGScheduler: Executor lost: 8 (epoch 7)
> 22/05/28 01:25:29 INFO BlockManagerMasterEndpoint: Trying to remove executor 8 from BlockManagerMaster.
> 22/05/28 01:25:29 INFO BlockManagerMasterEndpoint: Removing block manager BlockManagerId(8, 100.103.40.13, 43353, None)
> 22/05/28 01:25:29 INFO BlockManagerMaster: Removed 8 successfully in removeExecutor
> 22/05/28 01:25:29 INFO DAGScheduler: Shuffle files lost for executor: 8 (epoch 7)
> 22/05/28 01:25:34 INFO BlockManagerMaster: Removal of executor 8 requested
> 22/05/28 01:25:34 INFO BlockManagerMasterEndpoint: Trying to remove executor 8 from BlockManagerMaster.
> 22/05/28 01:25:34 INFO KubernetesClusterSchedulerBackend$KubernetesDriverEndpoint: Asked to remove non-existent executor 8
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org