[jira] [Created] (SPARK-25995) sparkR should ensure user args are after the argument used for the port

["Thomas Graves (JIRA)" <ji...@apache.org>]

Thomas Graves created SPARK-25995:
-------------------------------------

             Summary: sparkR should ensure user args are after the argument used for the port
                 Key: SPARK-25995
                 URL: https://issues.apache.org/jira/browse/SPARK-25995
             Project: Spark
          Issue Type: Bug
          Components: SparkR
    Affects Versions: 2.3.2
            Reporter: Thomas Graves


Currently if you run sparkR and accidentally specify an argument, it fails with a useless error message.  For example:

$SPARK_HOME/bin/sparkR  --master yarn --deploy-mode client fooarg

This gets turned into:

Launching java with spark-submit command spark-submit   "--master" "yarn" "--deploy-mode" "client" "sparkr-shell" "fooarg" /tmp/Rtmp6XBGz2/backend_port162806ea36bca

Notice that "fooarg" got put before /tmp file which is how R and jvm know which port to connect to.  SparkR eventually fails with timeout exception after 10 seconds.  

 

SparkR should either not allow args or make sure the order is correct so the backend_port is always first. see https://github.com/apache/spark/blob/master/R/pkg/R/sparkR.R#L129



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org