[jira] [Created] (SPARK-18340) Inconsistent error messages in launching scripts and hanging in sparkr script for wrong options

["Hyukjin Kwon (JIRA)" <ji...@apache.org>]

Hyukjin Kwon created SPARK-18340:
------------------------------------

             Summary: Inconsistent error messages in launching scripts and hanging in sparkr script for wrong options
                 Key: SPARK-18340
                 URL: https://issues.apache.org/jira/browse/SPARK-18340
             Project: Spark
          Issue Type: Bug
          Components: Spark Shell, Spark Submit
            Reporter: Hyukjin Kwon
            Priority: Minor


It seems there are some problems with handling wrong options as below:

*{{spark-submit}} script - this one looks fine

{code}
spark-submit --aabbcc
Error: Unrecognized option: --aabbcc

Usage: spark-submit [options] <app jar | python file> [app arguments]
Usage: spark-submit --kill [submission ID] --master [spark://...]
Usage: spark-submit --status [submission ID] --master [spark://...]
Usage: spark-submit run-example [options] example-class [example args]

Options:
  --master MASTER_URL         spark://host:port, mesos://host:port, yarn, or local.
  --deploy-mode DEPLOY_MODE   Whether to launch the driver program locally ("client") or
                              on one of the worker machines inside the cluster ("cluster")
                              (Default: client).
  --class CLASS_NAME          Your application's main class (for Java / Scala apps).
  --name NAME                 A name of your application.
...
{code}


*{{spark-sql}} script - this one looks fine

{code}
spark-sql --aabbcc
Unrecognized option: --aabbcc
usage: hive
 -d,--define <key=value>          Variable subsitution to apply to hive
                                  commands. e.g. -d A=B or --define A=B
    --database <databasename>     Specify the database to use
...
{code}

*{{sparkr}} script - this one might be a bit major because users possibly mistakenly put some wrong options with typos and the error message does not indicate the options are wrong.

{code}
sparkr --aabbcc

...

Error in sparkR.sparkContext(master, appName, sparkHome, sparkConfigMap,  :
  JVM is not ready after 10 seconds
>
{code}

*{{pyspark}} script - we could make the error message consistently with the others

{code}
pyspark --aabbcc
Exception in thread "main" java.lang.IllegalArgumentException: pyspark does not support any application options.
	at org.apache.spark.launcher.CommandBuilderUtils.checkArgument(CommandBuilderUtils.java:241)
	at org.apache.spark.launcher.SparkSubmitCommandBuilder.buildPySparkShellCommand(SparkSubmitCommandBuilder.java:290)
	at org.apache.spark.launcher.SparkSubmitCommandBuilder.buildCommand(SparkSubmitCommandBuilder.java:147)
	at org.apache.spark.launcher.Main.main(Main.java:86)
{code}




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org