You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Sean Owen (JIRA)" <ji...@apache.org> on 2016/07/24 08:15:20 UTC
[jira] [Resolved] (SPARK-16541) SparkTC application could not
shutdown successfully
[ https://issues.apache.org/jira/browse/SPARK-16541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sean Owen resolved SPARK-16541.
-------------------------------
Resolution: Cannot Reproduce
> SparkTC application could not shutdown successfully
> ----------------------------------------------------
>
> Key: SPARK-16541
> URL: https://issues.apache.org/jira/browse/SPARK-16541
> Project: Spark
> Issue Type: Bug
> Affects Versions: 2.0.0
> Reporter: Yesha Vora
>
> SparkTC application in yarn-client mode was stuck at 10% progress.
> {code} spark-submit --class org.apache.spark.examples.SparkTC --master yarn-client spark-examples-assembly_*.jar {code}
> It seems like SparkTC application tasks finished and printed "TC has 6254 edges.". after that while shutting down, spark application kept getting "ERROR LiveListenerBus: SparkListenerBus has already stopped! Dropping event SparkListenerExecutorMetricsUpdate"
> {code}
> 16/07/13 08:43:37 INFO DAGScheduler: ResultStage 283 (count at SparkTC.scala:71) finished in 42.357 s
> 16/07/13 08:43:37 INFO DAGScheduler: Job 13 finished: count at SparkTC.scala:71, took 43.137408 s
> TC has 6254 edges.
> 16/07/13 08:43:37 INFO ServerConnector: Stopped ServerConnector@5e0054a2{HTTP/1.1}{0.0.0.0:4040}
> 16/07/13 08:43:37 INFO ContextHandler: Stopped o.s.j.s.ServletContextHandler@7350a22{/stages/stage/kill,null,UNAVAILABLE}
> 16/07/13 08:43:37 INFO ContextHandler: Stopped o.s.j.s.ServletContextHandler@54d56a49{/api,null,UNAVAILABLE}
> 16/07/13 08:43:37 INFO ContextHandler: Stopped o.s.j.s.ServletContextHandler@2fd52e57{/,null,UNAVAILABLE}
> 16/07/13 08:43:37 INFO ContextHandler: Stopped o.s.j.s.ServletContextHandler@7737ff3{/static,null,UNAVAILABLE}
> 16/07/13 08:43:37 INFO ContextHandler: Stopped o.s.j.s.ServletContextHandler@499d9067{/executors/threadDump/json,null,UNAVAILABLE}
> 16/07/13 08:43:37 INFO ContextHandler: Stopped o.s.j.s.ServletContextHandler@40d0c2af{/executors/threadDump,null,UNAVAILABLE}
> 16/07/13 08:43:37 INFO ContextHandler: Stopped o.s.j.s.ServletContextHandler@44ce4013{/executors/json,null,UNAVAILABLE}
> 16/07/13 08:43:37 INFO ContextHandler: Stopped o.s.j.s.ServletContextHandler@59c9a28a{/executors,null,UNAVAILABLE}
> 16/07/13 08:43:37 INFO ContextHandler: Stopped o.s.j.s.ServletContextHandler@2e784443{/environment/json,null,UNAVAILABLE}
> 16/07/13 08:43:37 INFO ContextHandler: Stopped o.s.j.s.ServletContextHandler@10240ba4{/environment,null,UNAVAILABLE}
> 16/07/13 08:43:37 INFO ContextHandler: Stopped o.s.j.s.ServletContextHandler@4ee2dd22{/storage/rdd/json,null,UNAVAILABLE}
> 16/07/13 08:43:37 INFO ContextHandler: Stopped o.s.j.s.ServletContextHandler@79ab14cd{/storage/rdd,null,UNAVAILABLE}
> 16/07/13 08:43:37 INFO ContextHandler: Stopped o.s.j.s.ServletContextHandler@731d1285{/storage/json,null,UNAVAILABLE}
> 16/07/13 08:43:37 INFO ContextHandler: Stopped o.s.j.s.ServletContextHandler@72e46ea8{/storage,null,UNAVAILABLE}
> 16/07/13 08:43:37 INFO ContextHandler: Stopped o.s.j.s.ServletContextHandler@266dcdd5{/stages/pool/json,null,UNAVAILABLE}
> 16/07/13 08:43:37 INFO ContextHandler: Stopped o.s.j.s.ServletContextHandler@17ee6dd9{/stages/pool,null,UNAVAILABLE}
> 16/07/13 08:43:37 INFO ContextHandler: Stopped o.s.j.s.ServletContextHandler@717867ea{/stages/stage/json,null,UNAVAILABLE}
> 16/07/13 08:43:37 INFO ContextHandler: Stopped o.s.j.s.ServletContextHandler@74aaadcc{/stages/stage,null,UNAVAILABLE}
> 16/07/13 08:43:37 INFO ContextHandler: Stopped o.s.j.s.ServletContextHandler@14f35a42{/stages/json,null,UNAVAILABLE}
> 16/07/13 08:43:37 INFO ContextHandler: Stopped o.s.j.s.ServletContextHandler@27ec74f8{/stages,null,UNAVAILABLE}
> 16/07/13 08:43:37 INFO ContextHandler: Stopped o.s.j.s.ServletContextHandler@148ad9f9{/jobs/job/json,null,UNAVAILABLE}
> 16/07/13 08:43:37 INFO ContextHandler: Stopped o.s.j.s.ServletContextHandler@14445e4c{/jobs/job,null,UNAVAILABLE}
> 16/07/13 08:43:37 INFO ContextHandler: Stopped o.s.j.s.ServletContextHandler@6d1557ff{/jobs/json,null,UNAVAILABLE}
> 16/07/13 08:43:37 INFO ContextHandler: Stopped o.s.j.s.ServletContextHandler@aca62b1{/jobs,null,UNAVAILABLE}
> 16/07/13 08:43:37 INFO SparkUI: Stopped Spark web UI at http://xx.xx.xx:4040
> 16/07/13 08:43:37 ERROR LiveListenerBus: SparkListenerBus has already stopped! Dropping event SparkListenerExecutorMetricsUpdate(2,WrappedArray())
> 16/07/13 08:43:37 ERROR LiveListenerBus: SparkListenerBus has already stopped! Dropping event SparkListenerExecutorMetricsUpdate(1,WrappedArray())
> 16/07/13 08:43:47 ERROR LiveListenerBus: SparkListenerBus has already stopped! Dropping event SparkListenerExecutorMetricsUpdate(2,WrappedArray())
> 16/07/13 08:43:47 ERROR LiveListenerBus: SparkListenerBus has already stopped! Dropping event SparkListenerExecutorMetricsUpdate(1,WrappedArray())
> 16/07/13 08:43:57 ERROR LiveListenerBus: SparkListenerBus has already stopped! Dropping event SparkListenerExecutorMetricsUpdate(2,WrappedArray())
> 16/07/13 08:43:57 ERROR LiveListenerBus: SparkListenerBus has already stopped! Dropping event SparkListenerExecutorMetricsUpdate(1,WrappedArray())
> 16/07/13 08:44:07 ERROR LiveListenerBus: SparkListenerBus has already stopped! Dropping event SparkListenerExecutorMetricsUpdate(2,WrappedArray())
> 16/07/13 08:44:07 ERROR LiveListenerBus: SparkListenerBus has already stopped! Dropping event SparkListenerExecutorMetricsUpdate(1,WrappedArray())
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org