You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Hyukjin Kwon (JIRA)" <ji...@apache.org> on 2018/11/10 10:20:00 UTC
[jira] [Commented] (SPARK-25995) sparkR should ensure user args are
after the argument used for the port
[ https://issues.apache.org/jira/browse/SPARK-25995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16682322#comment-16682322 ]
Hyukjin Kwon commented on SPARK-25995:
--------------------------------------
Sounds related with SPARK-18340 as well.
> sparkR should ensure user args are after the argument used for the port
> -----------------------------------------------------------------------
>
> Key: SPARK-25995
> URL: https://issues.apache.org/jira/browse/SPARK-25995
> Project: Spark
> Issue Type: Bug
> Components: SparkR
> Affects Versions: 2.3.2
> Reporter: Thomas Graves
> Priority: Minor
>
> Currently if you run sparkR and accidentally specify an argument, it fails with a useless error message. For example:
> $SPARK_HOME/bin/sparkR --master yarn --deploy-mode client fooarg
> This gets turned into:
> Launching java with spark-submit command spark-submit "--master" "yarn" "--deploy-mode" "client" "sparkr-shell" "fooarg" /tmp/Rtmp6XBGz2/backend_port162806ea36bca
> Notice that "fooarg" got put before /tmp file which is how R and jvm know which port to connect to. SparkR eventually fails with timeout exception after 10 seconds.
>
> SparkR should either not allow args or make sure the order is correct so the backend_port is always first. see https://github.com/apache/spark/blob/master/R/pkg/R/sparkR.R#L129
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org