[jira] [Resolved] (SPARK-21399) Difficult to debug code in spark-class

["Marcelo Vanzin (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SPARK-21399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Marcelo Vanzin resolved SPARK-21399.
------------------------------------
    Resolution: Invalid

Sorry, but you not understanding shell code is not a reason to rewrite it... there are reasons why the code does what it does and why simpler approaches don't work.

> Difficult to debug code in spark-class
> --------------------------------------
>
>                 Key: SPARK-21399
>                 URL: https://issues.apache.org/jira/browse/SPARK-21399
>             Project: Spark
>          Issue Type: Improvement
>          Components: Spark Shell, Spark Submit
>    Affects Versions: 2.1.1
>         Environment: Running spark on a version of hottub.
>            Reporter: Mahesh
>            Priority: Minor
>
> The code below is difficult to debug, and its not clear what is happening, it appears to use some kind of shell magic, if possible it should be replaced with an easier to understand code. Specifically there needs to be a log of the exact java command that is finally invoked. 
> Using input redirection means that adding comments to the build_command will break the code ,I am not sure but the below code for build_command is not equivalent to one in the code, although semantically it should be.
> build_command() {
>   "$RUNNER" -Xmx128m -cp "$LAUNCH_CLASSPATH" org.apache.spark.launcher.Main "$@"
>   printf "%d\0" $?
>   #printf "Something\0"
> }
> *Original code below*
> *build_command() {
>   "$RUNNER" -Xmx128m -cp "$LAUNCH_CLASSPATH" org.apache.spark.launcher.Main "$@"
>   printf "%d\0" $?
> }
> CMD=()
> while IFS= read -d '' -r ARG; do
>   CMD+=("$ARG")
> done < <(build_command "$@")*



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org