[jira] [Created] (SPARK-29070) Allow SparkLauncher to return full spark-submit command line

["Jeff Evans (Jira)" <ji...@apache.org>]

Jeff Evans created SPARK-29070:
----------------------------------

             Summary: Allow SparkLauncher to return full spark-submit command line
                 Key: SPARK-29070
                 URL: https://issues.apache.org/jira/browse/SPARK-29070
             Project: Spark
          Issue Type: Improvement
          Components: Spark Submit
    Affects Versions: 2.4.5
            Reporter: Jeff Evans


{{org.apache.spark.launcher.SparkLauncher}} wraps a {{ProcessBuilder}}, and builds up a full command line to {{spark-submit}} using a builder pattern.  When {{startApplication}} is finally called, a 

In scenarios where another application is submitting to Spark, it would be extremely useful from a support and debugging standpoint to be able to see the full {{spark-submit}} command that is actually used (so that the same submission can be tested standalone, arguments tweaked, etc.).  Currently, the only way this gets captured is to {{stderr}} if the {{SPARK_PRINT_LAUNCH_COMMAND}} environment variable is set.  This is cumbersome in the context of an application that is wrapping Spark and already using the APIs.

I propose simply adding a getter method to {{SparkSubmit}} that allows clients to retrieve what the full command line will be, so they can log this however they wish (or do anything else with it).



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org