You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Hyukjin Kwon (Jira)" <ji...@apache.org> on 2020/01/06 06:15:00 UTC
[jira] [Resolved] (SPARK-30357) SparkContext: Invoking stop() from
shutdown hook
[ https://issues.apache.org/jira/browse/SPARK-30357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hyukjin Kwon resolved SPARK-30357.
----------------------------------
Resolution: Incomplete
> SparkContext: Invoking stop() from shutdown hook
> ------------------------------------------------
>
> Key: SPARK-30357
> URL: https://issues.apache.org/jira/browse/SPARK-30357
> Project: Spark
> Issue Type: Bug
> Components: Deploy, Kubernetes, MLlib, PySpark
> Affects Versions: 2.4.4
> Environment: pivotal kubernetes container service PKS having 3master nodes and 3 worker nodes
>
> OS : ubuntu for k8-s cluster
>
>
> Reporter: Veera
> Priority: Major
>
> I'm getting below error while running spark-submit job in kubernetes , i didn't get any specific error
> {code}
> 19/12/26 07:38:18 INFO SparkContext: Invoking stop() from shutdown hook
> 19/12/26 07:38:18 INFO SparkUI: Stopped Spark web UI at http://spark-ml-test-1577345808987-driver-svc.spark-jobs.svc:4040
> 19/12/26 07:38:18 INFO KubernetesClusterSchedulerBackend: Shutting down all executors
> 19/12/26 07:38:18 INFO KubernetesClusterSchedulerBackend$KubernetesDriverEndpoint: Asking each executor to shut down
> 19/12/26 07:38:18 WARN ExecutorPodsWatchSnapshotSource: Kubernetes client has been closed (this is expected if the application is shutting down.)
> 19/12/26 07:38:18 INFO MapOutputTrackerMasterEndpoint: MapOutputTrackerMasterEndpoint stopped!
> 19/12/26 07:38:18 INFO MemoryStore: MemoryStore cleared
> 19/12/26 07:38:18 INFO BlockManager: BlockManager stopped
> 19/12/26 07:38:18 INFO BlockManagerMaster: BlockManagerMaster stopped
> 19/12/26 07:38:18 INFO OutputCommitCoordinator$OutputCommitCoordinatorEndpoint: OutputCommitCoordinator stopped!
> 19/12/26 07:38:18 INFO SparkContext: Successfully stopped SparkContext
> 19/12/26 07:38:18 INFO ShutdownHookManager: Shutdown hook called
> 19/12/26 07:38:18 INFO ShutdownHookManager: Deleting directory /var/data/spark-ec98f5d3-022d-4eae-87aa-c23a96c77555/spark-b757b85f-3709-4215-8208-95db840a294a/pyspark-665955c9-0dc6-49ee-9b57-0392be430f10
> 19/12/26 07:38:18 INFO ShutdownHookManager: Deleting directory /tmp/spark-fa047fa8-b573-44aa-ab65-e235fb0d5b09
> 19/12/26 07:38:18 INFO ShutdownHookManager: Deleting directory /var/data/spark-ec98f5d3-022d-4eae-87aa-c23a96c77555/spark-b757b85f-3709-4215-8208-95db840a294a
> {code}
>
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org