You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Yin Huai (JIRA)" <ji...@apache.org> on 2015/04/15 06:04:59 UTC
[jira] [Updated] (SPARK-4497) HiveThriftServer2 does not exit
properly on failure
[ https://issues.apache.org/jira/browse/SPARK-4497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Yin Huai updated SPARK-4497:
----------------------------
Description:
start thriftserver with
{{sbin/start-thriftserver.sh --master ...}}
If there is an error (in my case namenode is in standby mode) the driver shuts down properly:
{code}
14/11/19 16:32:58 ERROR HiveThriftServer2: Error starting HiveThriftServer2
....
14/11/19 16:32:59 INFO SparkUI: Stopped Spark web UI at http://myip:4040
14/11/19 16:32:59 INFO DAGScheduler: Stopping DAGScheduler
14/11/19 16:32:59 INFO SparkDeploySchedulerBackend: Shutting down all executors
14/11/19 16:32:59 INFO SparkDeploySchedulerBackend: Asking each executor to shut down
14/11/19 16:33:00 INFO MapOutputTrackerMasterActor: MapOutputTrackerActor stopped!
14/11/19 16:33:00 INFO MemoryStore: MemoryStore cleared
14/11/19 16:33:00 INFO BlockManager: BlockManager stopped
14/11/19 16:33:00 INFO BlockManagerMaster: BlockManagerMaster stopped
14/11/19 16:33:00 INFO SparkContext: Successfully stopped SparkContext
{code}
but trying to run {{sbin/start-thriftserver.sh --master ... }} again results in an error that Thrifserver is already running.
{{ps -aef|grep <offendingPID>}} shows
{code}
root 32334 1 0 16:32 ? 00:00:00 /usr/local/bin/java org.apache.spark.deploy.SparkSubmitDriverBootstrapper --class org.apache.spark.sql.hive.thriftserver.HiveThriftServer2 --master spark://myip:7077 --conf -spark.executor.extraJavaOptions=-verbose:gc -XX:-PrintGCDetails -XX:+PrintGCTimeStamps spark-internal --hiveconf hive.root.logger=INFO,console
{code}
This is problematic since we have a process that tries to restart the driver if it dies
was:
start thriftserver with
sbin/start-thriftserver.sh --master ...
If there is an error (in my case namenode is in standby mode) the driver shuts down properly:
14/11/19 16:32:58 ERROR HiveThriftServer2: Error starting HiveThriftServer2
....
14/11/19 16:32:59 INFO SparkUI: Stopped Spark web UI at http://myip:4040
14/11/19 16:32:59 INFO DAGScheduler: Stopping DAGScheduler
14/11/19 16:32:59 INFO SparkDeploySchedulerBackend: Shutting down all executors
14/11/19 16:32:59 INFO SparkDeploySchedulerBackend: Asking each executor to shut down
14/11/19 16:33:00 INFO MapOutputTrackerMasterActor: MapOutputTrackerActor stopped!
14/11/19 16:33:00 INFO MemoryStore: MemoryStore cleared
14/11/19 16:33:00 INFO BlockManager: BlockManager stopped
14/11/19 16:33:00 INFO BlockManagerMaster: BlockManagerMaster stopped
14/11/19 16:33:00 INFO SparkContext: Successfully stopped SparkContext
but trying to run sbin/start-thriftserver.sh --master ... again results in an error that Thrifserver is already running.
ps -aef|grep <offendingPID> shows
root 32334 1 0 16:32 ? 00:00:00 /usr/local/bin/java org.apache.spark.deploy.SparkSubmitDriverBootstrapper --class org.apache.spark.sql.hive.thriftserver.HiveThriftServer2 --master spark://myip:7077 --conf -spark.executor.extraJavaOptions=-verbose:gc -XX:-PrintGCDetails -XX:+PrintGCTimeStamps spark-internal --hiveconf hive.root.logger=INFO,console
This is problematic since we have a process that tries to restart the driver if it dies
> HiveThriftServer2 does not exit properly on failure
> ---------------------------------------------------
>
> Key: SPARK-4497
> URL: https://issues.apache.org/jira/browse/SPARK-4497
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 1.2.0
> Reporter: Yana Kadiyska
> Priority: Critical
>
> start thriftserver with
> {{sbin/start-thriftserver.sh --master ...}}
> If there is an error (in my case namenode is in standby mode) the driver shuts down properly:
> {code}
> 14/11/19 16:32:58 ERROR HiveThriftServer2: Error starting HiveThriftServer2
> ....
> 14/11/19 16:32:59 INFO SparkUI: Stopped Spark web UI at http://myip:4040
> 14/11/19 16:32:59 INFO DAGScheduler: Stopping DAGScheduler
> 14/11/19 16:32:59 INFO SparkDeploySchedulerBackend: Shutting down all executors
> 14/11/19 16:32:59 INFO SparkDeploySchedulerBackend: Asking each executor to shut down
> 14/11/19 16:33:00 INFO MapOutputTrackerMasterActor: MapOutputTrackerActor stopped!
> 14/11/19 16:33:00 INFO MemoryStore: MemoryStore cleared
> 14/11/19 16:33:00 INFO BlockManager: BlockManager stopped
> 14/11/19 16:33:00 INFO BlockManagerMaster: BlockManagerMaster stopped
> 14/11/19 16:33:00 INFO SparkContext: Successfully stopped SparkContext
> {code}
> but trying to run {{sbin/start-thriftserver.sh --master ... }} again results in an error that Thrifserver is already running.
> {{ps -aef|grep <offendingPID>}} shows
> {code}
> root 32334 1 0 16:32 ? 00:00:00 /usr/local/bin/java org.apache.spark.deploy.SparkSubmitDriverBootstrapper --class org.apache.spark.sql.hive.thriftserver.HiveThriftServer2 --master spark://myip:7077 --conf -spark.executor.extraJavaOptions=-verbose:gc -XX:-PrintGCDetails -XX:+PrintGCTimeStamps spark-internal --hiveconf hive.root.logger=INFO,console
> {code}
> This is problematic since we have a process that tries to restart the driver if it dies
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org