You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Patrick Wendell (JIRA)" <ji...@apache.org> on 2015/01/16 07:29:34 UTC
[jira] [Comment Edited] (SPARK-5176) Thrift server fails with
confusing error message when deploy-mode is cluster
[ https://issues.apache.org/jira/browse/SPARK-5176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14279869#comment-14279869 ]
Patrick Wendell edited comment on SPARK-5176 at 1/16/15 6:28 AM:
-----------------------------------------------------------------
Yes, we should add a check here similar to the existing ones for the thriftserver class:
https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala#L143
[~tpanning] are you interested in contributing this? If not, someone else will pick it up.
was (Author: pwendell):
Yes, we should add a check here similar to the existing ones for the thriftserver class:
https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala#L143
> Thrift server fails with confusing error message when deploy-mode is cluster
> ----------------------------------------------------------------------------
>
> Key: SPARK-5176
> URL: https://issues.apache.org/jira/browse/SPARK-5176
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 1.1.0, 1.2.0
> Reporter: Tom Panning
> Labels: starter
>
> With Spark 1.2.0, when I try to run
> {noformat}
> $SPARK_HOME/sbin/start-thriftserver.sh --deploy-mode cluster --master spark://xd-spark.xdata.data-tactics-corp.com:7077
> {noformat}
> The log output is
> {noformat}
> Spark assembly has been built with Hive, including Datanucleus jars on classpath
> Spark Command: /usr/java/latest/bin/java -cp ::/home/tpanning/Projects/spark/spark-1.2.0-bin-hadoop2.4/sbin/../conf:/home/tpanning/Projects/spark/spark-1.2.0-bin-hadoop2.4/lib/spark-assembly-1.2.0-hadoop2.4.0.jar:/home/tpanning/Projects/spark/spark-1.2.0-bin-hadoop2.4/lib/datanucleus-core-3.2.10.jar:/home/tpanning/Projects/spark/spark-1.2.0-bin-hadoop2.4/lib/datanucleus-rdbms-3.2.9.jar:/home/tpanning/Projects/spark/spark-1.2.0-bin-hadoop2.4/lib/datanucleus-api-jdo-3.2.6.jar -XX:MaxPermSize=128m -Xms512m -Xmx512m org.apache.spark.deploy.SparkSubmit --class org.apache.spark.sql.hive.thriftserver.HiveThriftServer2 --deploy-mode cluster --master spark://xd-spark.xdata.data-tactics-corp.com:7077 spark-internal
> ========================================
> Jar url 'spark-internal' is not in valid format.
> Must be a jar file path in URL format (e.g. hdfs://host:port/XX.jar, file:///XX.jar)
> Usage: DriverClient [options] launch <active-master> <jar-url> <main-class> [driver options]
> Usage: DriverClient kill <active-master> <driver-id>
> Options:
> -c CORES, --cores CORES Number of cores to request (default: 1)
> -m MEMORY, --memory MEMORY Megabytes of memory to request (default: 512)
> -s, --supervise Whether to restart the driver on failure
> -v, --verbose Print more debugging output
>
> Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties
> {noformat}
> I do not get this error if deploy-mode is set to client. The --deploy-mode option is described by the --help output, so I expected it to work. I checked, and this behavior seems to be present in Spark 1.1.0 as well.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org