You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by sr...@apache.org on 2018/10/17 14:56:19 UTC
spark git commit: [SPARK-25735][CORE][MINOR] Improve
start-thriftserver.sh: print clean usage and exit with code 1
Repository: spark
Updated Branches:
refs/heads/master 1901f0621 -> 24f5bbd77
[SPARK-25735][CORE][MINOR] Improve start-thriftserver.sh: print clean usage and exit with code 1
## What changes were proposed in this pull request?
Currently if we run
```
sh start-thriftserver.sh -h
```
we get
```
...
Thrift server options:
2018-10-15 21:45:39 INFO HiveThriftServer2:54 - Starting SparkContext
2018-10-15 21:45:40 INFO SparkContext:54 - Running Spark version 2.3.2
2018-10-15 21:45:40 WARN NativeCodeLoader:62 - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2018-10-15 21:45:40 ERROR SparkContext:91 - Error initializing SparkContext.
org.apache.spark.SparkException: A master URL must be set in your configuration
at org.apache.spark.SparkContext.<init>(SparkContext.scala:367)
at org.apache.spark.SparkContext$.getOrCreate(SparkContext.scala:2493)
at org.apache.spark.sql.SparkSession$Builder$$anonfun$7.apply(SparkSession.scala:934)
at org.apache.spark.sql.SparkSession$Builder$$anonfun$7.apply(SparkSession.scala:925)
at scala.Option.getOrElse(Option.scala:121)
at org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:925)
at org.apache.spark.sql.hive.thriftserver.SparkSQLEnv$.init(SparkSQLEnv.scala:48)
at org.apache.spark.sql.hive.thriftserver.HiveThriftServer2$.main(HiveThriftServer2.scala:79)
at org.apache.spark.sql.hive.thriftserver.HiveThriftServer2.main(HiveThriftServer2.scala)
2018-10-15 21:45:40 ERROR Utils:91 - Uncaught exception in thread main
```
After fix, the usage output is clean:
```
...
Thrift server options:
--hiveconf <property=value> Use value for given property
```
Also exit with code 1, to follow other scripts(this is the behavior of parsing option `-h` for other linux commands as well).
## How was this patch tested?
Manual test.
Closes #22727 from gengliangwang/stsUsage.
Authored-by: Gengliang Wang <ge...@databricks.com>
Signed-off-by: Sean Owen <se...@databricks.com>
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/24f5bbd7
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/24f5bbd7
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/24f5bbd7
Branch: refs/heads/master
Commit: 24f5bbd770033dacdea62555488bfffb61665279
Parents: 1901f06
Author: Gengliang Wang <ge...@databricks.com>
Authored: Wed Oct 17 09:56:17 2018 -0500
Committer: Sean Owen <se...@databricks.com>
Committed: Wed Oct 17 09:56:17 2018 -0500
----------------------------------------------------------------------
sbin/start-thriftserver.sh | 6 +++++-
.../spark/sql/hive/thriftserver/HiveThriftServer2.scala | 7 +++++++
2 files changed, 12 insertions(+), 1 deletion(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/spark/blob/24f5bbd7/sbin/start-thriftserver.sh
----------------------------------------------------------------------
diff --git a/sbin/start-thriftserver.sh b/sbin/start-thriftserver.sh
index f02f317..b1d3871 100755
--- a/sbin/start-thriftserver.sh
+++ b/sbin/start-thriftserver.sh
@@ -39,6 +39,10 @@ function usage {
pattern+="\|Spark Command: "
pattern+="\|======="
pattern+="\|--help"
+ pattern+="\|Using Spark's default log4j profile:"
+ pattern+="\|^log4j:"
+ pattern+="\|Started daemon with process name"
+ pattern+="\|Registered signal handler for"
"${SPARK_HOME}"/bin/spark-submit --help 2>&1 | grep -v Usage 1>&2
echo
@@ -48,7 +52,7 @@ function usage {
if [[ "$@" = *--help ]] || [[ "$@" = *-h ]]; then
usage
- exit 0
+ exit 1
fi
export SUBMIT_USAGE_FUNCTION=usage
http://git-wip-us.apache.org/repos/asf/spark/blob/24f5bbd7/sql/hive-thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/HiveThriftServer2.scala
----------------------------------------------------------------------
diff --git a/sql/hive-thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/HiveThriftServer2.scala b/sql/hive-thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/HiveThriftServer2.scala
index 7442c98..64d924f 100644
--- a/sql/hive-thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/HiveThriftServer2.scala
+++ b/sql/hive-thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/HiveThriftServer2.scala
@@ -71,6 +71,13 @@ object HiveThriftServer2 extends Logging {
}
def main(args: Array[String]) {
+ // If the arguments contains "-h" or "--help", print out the usage and exit.
+ if (args.contains("-h") || args.contains("--help")) {
+ HiveServer2.main(args)
+ // The following code should not be reachable. It is added to ensure the main function exits.
+ return
+ }
+
Utils.initDaemon(log)
val optionsProcessor = new HiveServer2.ServerOptionsProcessor("HiveThriftServer2")
optionsProcessor.parse(args)
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org