You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Veera (Jira)" <ji...@apache.org> on 2019/12/26 09:21:00 UTC
[jira] [Created] (SPARK-30357) SparkContext: Invoking stop() from
shutdown hook
Veera created SPARK-30357:
-----------------------------
Summary: SparkContext: Invoking stop() from shutdown hook
Key: SPARK-30357
URL: https://issues.apache.org/jira/browse/SPARK-30357
Project: Spark
Issue Type: Bug
Components: Deploy, Kubernetes, MLlib, PySpark
Affects Versions: 2.4.4
Environment: pivotal kubernetes container service PKS having 3master nodes and 3 worker nodes
OS : ubuntu for k8-s cluster
Reporter: Veera
I'm getting below error while running spark-submit job in kubernetes , i didn't get any specific error
19/12/26 07:38:18 INFO SparkContext: Invoking stop() from shutdown hook
19/12/26 07:38:18 INFO SparkUI: Stopped Spark web UI at http://spark-ml-test-1577345808987-driver-svc.spark-jobs.svc:4040
19/12/26 07:38:18 INFO KubernetesClusterSchedulerBackend: Shutting down all executors
19/12/26 07:38:18 INFO KubernetesClusterSchedulerBackend$KubernetesDriverEndpoint: Asking each executor to shut down
19/12/26 07:38:18 WARN ExecutorPodsWatchSnapshotSource: Kubernetes client has been closed (this is expected if the application is shutting down.)
19/12/26 07:38:18 INFO MapOutputTrackerMasterEndpoint: MapOutputTrackerMasterEndpoint stopped!
19/12/26 07:38:18 INFO MemoryStore: MemoryStore cleared
19/12/26 07:38:18 INFO BlockManager: BlockManager stopped
19/12/26 07:38:18 INFO BlockManagerMaster: BlockManagerMaster stopped
19/12/26 07:38:18 INFO OutputCommitCoordinator$OutputCommitCoordinatorEndpoint: OutputCommitCoordinator stopped!
19/12/26 07:38:18 INFO SparkContext: Successfully stopped SparkContext
19/12/26 07:38:18 INFO ShutdownHookManager: Shutdown hook called
19/12/26 07:38:18 INFO ShutdownHookManager: Deleting directory /var/data/spark-ec98f5d3-022d-4eae-87aa-c23a96c77555/spark-b757b85f-3709-4215-8208-95db840a294a/pyspark-665955c9-0dc6-49ee-9b57-0392be430f10
19/12/26 07:38:18 INFO ShutdownHookManager: Deleting directory /tmp/spark-fa047fa8-b573-44aa-ab65-e235fb0d5b09
19/12/26 07:38:18 INFO ShutdownHookManager: Deleting directory /var/data/spark-ec98f5d3-022d-4eae-87aa-c23a96c77555/spark-b757b85f-3709-4215-8208-95db840a294a
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org