[jira] [Commented] (SPARK-38242) Sort the SparkSubmit debug output

["Apache Spark (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SPARK-38242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17493970#comment-17493970 ] 

Apache Spark commented on SPARK-38242:
--------------------------------------

User 'martin-g' has created a pull request for this issue:
https://github.com/apache/spark/pull/35556

> Sort the SparkSubmit debug output 
> ----------------------------------
>
>                 Key: SPARK-38242
>                 URL: https://issues.apache.org/jira/browse/SPARK-38242
>             Project: Spark
>          Issue Type: Wish
>          Components: Spark Submit
>    Affects Versions: 3.2.1
>            Reporter: Martin Tzvetanov Grigorov
>            Priority: Minor
>
> When '-verbose' is passed to SparkSubmit it prints some useful debug information: Main class, Argument and Spark config.
> I find it a bit hard to find information there because the arguments/configs are shuffled. I suggest to sort those before printing them.
>  
> {code:java}
>  Main class:
> org.apache.spark.deploy.k8s.submit.KubernetesClientApplication
> Arguments:
> --main-class
> --primary-java-resource
> local:///opt/spark/examples/jars/spark-examples_2.13-3.3.0-SNAPSHOT.jar
> org.apache.spark.examples.SparkPi
> Spark config:
> (spark.app.name,spark-on-k8s-app)
> (spark.app.submitTime,1645106476125)
> (spark.driver.cores,1)
> (spark.driver.extraJavaOptions,-Dio.netty.tryReflectionSetAccessible=true)
> (spark.driver.memory,2048m)
> (spark.dynamicAllocation.enabled,true)
> (spark.dynamicAllocation.shuffleTracking.enabled,true)
> (spark.executor.cores,2)
> (spark.executor.extraJavaOptions,-Dio.netty.tryReflectionSetAccessible=true)
> (spark.executor.instances,3)
> (spark.executor.memory,2048m)
> (spark.jars,local:///opt/spark/examples/jars/spark-examples_2.13-3.3.0-SNAPSHOT.jar)
> (spark.kubernetes.allocation.batch.delay,1)
> (spark.kubernetes.allocation.batch.size,3)
> (spark.kubernetes.authenticate.driver.serviceAccountName,spark-account-name)
> (spark.kubernetes.driver.container.image,spark/spark:3.3.0-SNAPSHOT-scala_2.13-11-jre-slim)
> (spark.kubernetes.executor.container.image,spark/spark:3.3.0-SNAPSHOT-scala_2.13-11-jre-slim)
> (spark.kubernetes.namespace,spark-on-k8s)
> (spark.master,k8s://https://192.168.49.2:8443)
> (spark.network.timeout,300)
> (spark.submit.deployMode,cluster)
> (spark.submit.pyFiles,)
> Classpath elements:{code}
>  
> The "Parsed arguments:" order is hardcoded at org.apache.spark.deploy.SparkSubmitArguments#toString, so they are still shuffled.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org