You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "wuyi (Jira)" <ji...@apache.org> on 2021/08/18 12:56:00 UTC
[jira] [Created] (SPARK-36543) Decommission logs too frequent when
waiting migration to finish
wuyi created SPARK-36543:
----------------------------
Summary: Decommission logs too frequent when waiting migration to finish
Key: SPARK-36543
URL: https://issues.apache.org/jira/browse/SPARK-36543
Project: Spark
Issue Type: Sub-task
Components: Spark Core
Affects Versions: 3.1.0, 3.2.0, 3.3.0
Reporter: wuyi
{code:java}
21/08/18 08:14:31 INFO CoarseGrainedExecutorBackend: Checking to see if we can shutdown.
21/08/18 08:14:31 INFO CoarseGrainedExecutorBackend: No running tasks, checking migrations
21/08/18 08:14:31 INFO CoarseGrainedExecutorBackend: All blocks not yet migrated.
21/08/18 08:14:32 INFO CoarseGrainedExecutorBackend: Checking to see if we can shutdown.
21/08/18 08:14:32 INFO CoarseGrainedExecutorBackend: No running tasks, checking migrations
21/08/18 08:14:32 INFO CoarseGrainedExecutorBackend: All blocks not yet migrated.
21/08/18 08:14:33 INFO CoarseGrainedExecutorBackend: Checking to see if we can shutdown.
21/08/18 08:14:33 INFO CoarseGrainedExecutorBackend: No running tasks, checking migrations
21/08/18 08:14:33 INFO CoarseGrainedExecutorBackend: All blocks not yet migrated.
21/08/18 08:14:34 INFO CoarseGrainedExecutorBackend: Checking to see if we can shutdown.
21/08/18 08:14:34 INFO CoarseGrainedExecutorBackend: No running tasks, checking migrations
21/08/18 08:14:34 INFO CoarseGrainedExecutorBackend: All blocks not yet migrated.
21/08/18 08:14:35 INFO CoarseGrainedExecutorBackend: Checking to see if we can shutdown.
21/08/18 08:14:35 INFO CoarseGrainedExecutorBackend: No running tasks, checking migrations
21/08/18 08:14:35 INFO CoarseGrainedExecutorBackend: All blocks not yet migrated.
21/08/18 08:14:36 INFO CoarseGrainedExecutorBackend: Checking to see if we can shutdown.
21/08/18 08:14:36 INFO CoarseGrainedExecutorBackend: No running tasks, checking migrations
21/08/18 08:14:36 INFO CoarseGrainedExecutorBackend: All blocks not yet migrated.
21/08/18 08:14:37 INFO CoarseGrainedExecutorBackend: Checking to see if we can shutdown.
21/08/18 08:14:37 INFO CoarseGrainedExecutorBackend: No running tasks, checking migrations
21/08/18 08:14:37 INFO CoarseGrainedExecutorBackend: All blocks not yet migrated.
21/08/18 08:14:38 INFO CoarseGrainedExecutorBackend: Checking to see if we can shutdown.
21/08/18 08:14:38 INFO CoarseGrainedExecutorBackend: No running tasks, checking migrations
21/08/18 08:14:38 INFO CoarseGrainedExecutorBackend: All blocks not yet migrated.
21/08/18 08:14:39 INFO CoarseGrainedExecutorBackend: Checking to see if we can shutdown.
21/08/18 08:14:39 INFO CoarseGrainedExecutorBackend: No running tasks, checking migrations
21/08/18 08:14:39 INFO CoarseGrainedExecutorBackend: All blocks not yet migrated.
21/08/18 08:14:40 INFO CoarseGrainedExecutorBackend: Checking to see if we can shutdown.
21/08/18 08:14:40 INFO CoarseGrainedExecutorBackend: No running tasks, checking migrations
21/08/18 08:14:40 INFO CoarseGrainedExecutorBackend: All blocks not yet migrated.
21/08/18 08:14:41 INFO CoarseGrainedExecutorBackend: Checking to see if we can shutdown.
21/08/18 08:14:41 INFO CoarseGrainedExecutorBackend: No running tasks, checking migrations
21/08/18 08:14:41 INFO CoarseGrainedExecutorBackend: All blocks not yet migrated.
21/08/18 08:14:42 INFO CoarseGrainedExecutorBackend: Checking to see if we can shutdown.
21/08/18 08:14:42 INFO CoarseGrainedExecutorBackend: No running tasks, checking migrations
21/08/18 08:14:42 INFO CoarseGrainedExecutorBackend: All blocks not yet migrated.
21/08/18 08:14:43 INFO CoarseGrainedExecutorBackend: Checking to see if we can shutdown.
21/08/18 08:14:43 INFO CoarseGrainedExecutorBackend: No running tasks, checking migrations
21/08/18 08:14:43 INFO CoarseGrainedExecutorBackend: All blocks not yet migrated.
21/08/18 08:14:44 INFO CoarseGrainedExecutorBackend: Checking to see if we can shutdown.
21/08/18 08:14:44 INFO CoarseGrainedExecutorBackend: No running tasks, checking migrations
21/08/18 08:14:44 INFO CoarseGrainedExecutorBackend: All blocks not yet migrated.
...{code}
It takes some time to migrate data (shuffle or rdd). Logging per second is too frequent.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org