[jira] [Created] (SPARK-21399) Difficult to debug code in spark-class

["Mahesh (JIRA)" <ji...@apache.org>]

Mahesh created SPARK-21399:
------------------------------

             Summary: Difficult to debug code in spark-class
                 Key: SPARK-21399
                 URL: https://issues.apache.org/jira/browse/SPARK-21399
             Project: Spark
          Issue Type: Bug
          Components: Spark Shell, Spark Submit
    Affects Versions: 2.1.1
         Environment: Running spark on a version of hottub.
            Reporter: Mahesh
            Priority: Minor


The code below is difficult to debug, and its not clear what is happening, it appears to use some kind of shell magic, if possible it should be replaced with an easier to understand code. Specifically there needs to be a log of the exact java command that is finally invoked. 

Using input redirection means that adding comments to the build_command will break the code ,I am not sure but the below code for build_command is not equivalent to one in the code, although semantically it should be.

build_command() {
  "$RUNNER" -Xmx128m -cp "$LAUNCH_CLASSPATH" org.apache.spark.launcher.Main "$@"
  printf "%d\0" $?
  #printf "Something\0"
}



# The launcher library will print arguments separated by a NULL character, to allow arguments with
# characters that would be otherwise interpreted by the shell. Read that in a while loop, populating
# an array that will be used to exec the final command.
#
# The exit code of the launcher is appended to the output, so the parent shell removes it from the
# command array and checks the value to see if the launcher succeeded.
build_command() {
  "$RUNNER" -Xmx128m -cp "$LAUNCH_CLASSPATH" org.apache.spark.launcher.Main "$@"
  printf "%d\0" $?
}

CMD=()
while IFS= read -d '' -r ARG; do
  CMD+=("$ARG")
done < <(build_command "$@")




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org