[jira] [Commented] (SPARK-17256) spark-submit.cmd cannot work if path has space and cut off double-quoted arguments

["Quanmao Liu (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SPARK-17256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15438534#comment-15438534 ] 

Quanmao Liu commented on SPARK-17256:
-------------------------------------

I've pushed a pull request : https://github.com/apache/spark/pull/14807

> spark-submit.cmd cannot work if path has space and cut off double-quoted arguments
> ----------------------------------------------------------------------------------
>
>                 Key: SPARK-17256
>                 URL: https://issues.apache.org/jira/browse/SPARK-17256
>             Project: Spark
>          Issue Type: Bug
>          Components: Deploy
>    Affects Versions: 2.0.0
>         Environment: Windows
>            Reporter: Quanmao Liu
>              Labels: cut-off, double-quotes
>   Original Estimate: 5m
>  Remaining Estimate: 5m
>
> The key problem is  { cmd /V /E /C "~%dp0spark-xxx.cmd" } cannot accept arguments has double quotes, even if itself of "spark-xxx.cmd".
> Double quoted command in spark/bin/spark-submit.cmd (and spark-class.cmd etc) as following:
>  cmd /V /E /C "%~dp0spark-submit2.cmd" %*
> Will cut off the argument that double quoted, and not effect if full path of spark-submit.cmd has space.
> Example case :
> spark-submit.cmd --jars xxxx  "jdbc:mysql://localhost:3306/lzdb?user=guest&password=abc123" my_table
> (1) Cannot just start spark-submit.cmd if it's full path has space, for example  "D:\opengit\spark\bin - Copy\spark-submit.cmd" :
> D:\opengit\spark\bin - Copy>
> spark-submit.cmd --jars just-to-start "jdbc:mysql://localhost:3306/lzdb?user=guest&password=abc123" my_table
> ---- will get erorr : 
> 'D:\opengit\spark\bin' is not recognized as an internal or external command,
> operable program or batch file.
> 'password' is not recognized as an internal or external command,
> operable program or batch file.
> (2) spark-submit.cmd can be started but will cut off arguments if path's no space . for example of above but no space:
> D:\opengit\spark\bin>
> spark-submit.cmd --jars just-to-start "jdbc:mysql://localhost:3306/lzdb?user=guest&password=abc123" my_table
> ----- will get error :
> 'password' is not recognized as an internal or external command,
> operable program or batch file.
> (1) A solution for this is just removing the "cmd /V /E /C" in the spark/bin/xxx.cmd files , but this will has possible "pollution" to the environment variables in the same cmd window (and later derived cmd window like started in it by "start cmd").
> But this will solve the space in path problem.
> (2) A mitigation is checking before adding double quotes, and added if really need. This just works well if path no space.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org