You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "chengning (JIRA)" <ji...@apache.org> on 2017/11/02 08:26:00 UTC
[jira] [Comment Edited] (SPARK-22424) Task not finished for a long
time in monitor UI, but I found it finished in logs
[ https://issues.apache.org/jira/browse/SPARK-22424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16235379#comment-16235379 ]
chengning edited comment on SPARK-22424 at 11/2/17 8:25 AM:
------------------------------------------------------------
I have another picture shows clearly
!1.png|thumbnail!
executor
17/11/01 16:40:55 INFO executor.CoarseGrainedExecutorBackend: Got assigned task 640603
17/11/01 16:40:55 INFO executor.Executor: Running task 3.0 in stage 8218.0 (TID 640603)
17/11/01 16:40:55 INFO spark.MapOutputTrackerWorker: Updating epoch to 2319 and clearing cache
17/11/01 16:40:55 INFO broadcast.TorrentBroadcast: Started reading broadcast variable 8218
17/11/01 16:40:55 INFO memory.MemoryStore: Block broadcast_8218_piece0 stored as bytes in memory (estimated size 15.2 KB, free 2.2 GB)
17/11/01 16:40:55 INFO broadcast.TorrentBroadcast: Reading broadcast variable 8218 took 6 ms
17/11/01 16:40:55 INFO memory.MemoryStore: Block broadcast_8218 stored as values in memory (estimated size 31.5 KB, free 2.2 GB)
17/11/01 16:40:55 INFO spark.MapOutputTrackerWorker: Don't have map outputs for shuffle 2318, fetching them
17/11/01 16:40:55 INFO spark.MapOutputTrackerWorker: Doing the fetch; tracker endpoint = NettyRpcEndpointRef(spark://MapOutputTracker@10.110.155.57:33084)
17/11/01 16:40:55 INFO spark.MapOutputTrackerWorker: Got the output locations
17/11/01 16:40:55 INFO storage.ShuffleBlockFetcherIterator: Getting 28 non-empty blocks out of 30 blocks
17/11/01 16:40:55 INFO storage.ShuffleBlockFetcherIterator: Started 27 remote fetches in 3 ms
17/11/01 16:40:55 INFO codegen.CodeGenerator: Code generated in 21.652093 ms
17/11/01 16:40:55 INFO executor.Executor: Finished task 3.0 in stage 8218.0 (TID 640603). 3554 bytes result sent to
driver
17/11/01 16:40:55 INFO scheduler.TaskSetManager: Starting task 3.0 in stage 8218.0 (TID 640603, Letv6CU621YYPS, executor 12, partition 3, PROCESS_LOCAL, 6324 bytes)
17/11/01 16:40:55 INFO scheduler.TaskSetManager: Finished task 3.0 in stage 8218.0 (TID 640603) in 167 ms on Letv6CU621YYPS (executor 12) (16/200)
17/11/01 16:40:55 ERROR scheduler.LiveListenerBus: Dropping SparkListenerEvent because no remaining room in event queue. This likely means one of the SparkListeners is too slow and cannot keep up with the rate at which tasks are being started by the scheduler.
was (Author: chengning):
I have another picture shows clearly
!1.png|thumbnail!
executor
17/11/01 16:40:55 INFO executor.CoarseGrainedExecutorBackend: Got assigned task 640603
17/11/01 16:40:55 INFO executor.Executor: Running task 3.0 in stage 8218.0 (TID 640603)
17/11/01 16:40:55 INFO spark.MapOutputTrackerWorker: Updating epoch to 2319 and clearing cache
17/11/01 16:40:55 INFO broadcast.TorrentBroadcast: Started reading broadcast variable 8218
17/11/01 16:40:55 INFO memory.MemoryStore: Block broadcast_8218_piece0 stored as bytes in memory (estimated size 15.2 KB, free 2.2 GB)
17/11/01 16:40:55 INFO broadcast.TorrentBroadcast: Reading broadcast variable 8218 took 6 ms
17/11/01 16:40:55 INFO memory.MemoryStore: Block broadcast_8218 stored as values in memory (estimated size 31.5 KB, free 2.2 GB)
17/11/01 16:40:55 INFO spark.MapOutputTrackerWorker: Don't have map outputs for shuffle 2318, fetching them
17/11/01 16:40:55 INFO spark.MapOutputTrackerWorker: Doing the fetch; tracker endpoint = NettyRpcEndpointRef(spark://MapOutputTracker@10.110.155.57:33084)
17/11/01 16:40:55 INFO spark.MapOutputTrackerWorker: Got the output locations
17/11/01 16:40:55 INFO storage.ShuffleBlockFetcherIterator: Getting 28 non-empty blocks out of 30 blocks
17/11/01 16:40:55 INFO storage.ShuffleBlockFetcherIterator: Started 27 remote fetches in 3 ms
17/11/01 16:40:55 INFO codegen.CodeGenerator: Code generated in 21.652093 ms
17/11/01 16:40:55 INFO executor.Executor: Finished task 3.0 in stage 8218.0 (TID 640603). 3554 bytes result sent to
driver
17/11/01 16:40:55 INFO scheduler.TaskSetManager: Starting task 3.0 in stage 8218.0 (TID 640603, Letv6CU621YYPS, executor 12, partition 3, PROCESS_LOCAL, 6324 bytes)
17/11/01 16:40:55 INFO scheduler.TaskSetManager: Finished task 3.0 in stage 8218.0 (TID 640603) in 167 ms on Letv6CU621YYPS (executor 12) (16/200)
17/11/01 16:40:55 ERROR scheduler.LiveListenerBus: Dropping SparkListenerEvent because no remaining room in event queue. This likely means one of the SparkListeners is too slow and cannot keep up with the rate at which tasks are being started by the scheduler.
> Task not finished for a long time in monitor UI, but I found it finished in logs
> --------------------------------------------------------------------------------
>
> Key: SPARK-22424
> URL: https://issues.apache.org/jira/browse/SPARK-22424
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
> Affects Versions: 2.1.0
> Reporter: chengning
> Priority: Blocking
> Attachments: 1.png, C33oL.jpg
>
>
> Task not finished for a long time in monitor UI, but I found it finished in logs
> Thanks a lot.
> !https://i.stack.imgur.com/C33oL.jpg!
> !C33oL.jpg|thumbnail!
> executor log:
> 17/09/29 17:32:28 INFO executor.CoarseGrainedExecutorBackend: Got assigned task 213492
> 17/09/29 17:32:28 INFO executor.Executor: Running task 52.0 in stage 2468.0 (TID 213492)
> 17/09/29 17:32:28 INFO storage.ShuffleBlockFetcherIterator: Getting 30 non-empty blocks out of 30 blocks
> 17/09/29 17:32:28 INFO storage.ShuffleBlockFetcherIterator: Started 29 remote fetches in 1 ms
> 17:32:28.447: tcPartition=7 ms
> 17/09/29 17:32:28 INFO executor.Executor: Finished task 52.0 in stage 2468.0 (TID 213492). 2755 bytes result sent to driver
> driver logļ¼:
> 17/09/29 17:32:28 INFO scheduler.TaskSetManager: Starting task 52.0 in stage 2468.0 (TID 213492, HMGQXD2, executor 1, partition 52, PROCESS_LOCAL, 6386 bytes)
> 17/09/29 17:32:28 INFO scheduler.TaskSetManager: Finished task 52.0 in stage 2468.0 (TID 213492) in 24 ms on HMGQXD2 (executor 1) (53/200)
> 17/09/29 17:32:28 INFO cluster.YarnScheduler: Removed TaskSet 2468.0, whose tasks have all completed, from pool
> 17/09/29 17:32:28 INFO scheduler.DAGScheduler: ResultStage 2468 (foreachPartition at Counter2.java:152) finished in 0.255 s
> 17/09/29 17:32:28 INFO scheduler.DAGScheduler: Job 1647 finished: foreachPartition at Counter2.java:152, took 0.415256 s
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org