[jira] [Created] (SPARK-20016) SparkLauncher submit job failed when setConf with special charaters under windows

["Vincent Sun (JIRA)" <ji...@apache.org>]

Vincent Sun created SPARK-20016:
-----------------------------------

             Summary: SparkLauncher submit job failed when setConf with special charaters under windows
                 Key: SPARK-20016
                 URL: https://issues.apache.org/jira/browse/SPARK-20016
             Project: Spark
          Issue Type: Bug
          Components: Java API
    Affects Versions: 2.0.0
         Environment: windows 7, 8, 10, 2008, 2008R2, etc.
            Reporter: Vincent Sun


I am using sparkLauncher JAVA API to submit job to a remote spark cluster master. I have some configurations like --conf spark.executor.cores=2 need to pass to master. So I called SparkLauncher.setConf() method. It works fine under Linux/CentOS, but failed on my own desktop which is a windows 8 OS. After debug into the source code I found the reason is at:
quoteForBatchScript method of CommandBuilderUtils class

It will add quote '"' characters while there is '=' or some other kinds of special characters under windows system. And that cause the command cannot be executed success in windows cmd.







--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org