[GitHub] [spark] srowen commented on a change in pull request #35556: [SPARK-38242][CORE] Sort the SparkSubmit debug output

[GitBox <gi...@apache.org>]

srowen commented on a change in pull request #35556:
URL: https://github.com/apache/spark/pull/35556#discussion_r810503007



##########
File path: core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala
##########
@@ -906,10 +906,10 @@ private[spark] class SparkSubmit extends Logging {
 
     if (args.verbose) {
       logInfo(s"Main class:\n$childMainClass")
-      logInfo(s"Arguments:\n${childArgs.mkString("\n")}")
+      logInfo(s"Arguments:\n${childArgs.sorted.mkString("\n")}")

Review comment:
       Ah wait a sec, we can't sort these. Order of args matters; you might have `--flag value`. You can see the issue in your example, where the arg order doesn't reflect what was passed and loses this information

##########
File path: core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala
##########
@@ -906,10 +906,10 @@ private[spark] class SparkSubmit extends Logging {
 
     if (args.verbose) {
       logInfo(s"Main class:\n$childMainClass")
-      logInfo(s"Arguments:\n${childArgs.mkString("\n")}")
+      logInfo(s"Arguments:\n${childArgs.sorted.mkString("\n")}")
       // sysProps may contain sensitive information, so redact before printing
-      logInfo(s"Spark config:\n${Utils.redact(sparkConf.getAll.toMap).mkString("\n")}")
-      logInfo(s"Classpath elements:\n${childClasspath.mkString("\n")}")
+      logInfo(s"Spark config:\n${Utils.redact(sparkConf.getAll.toMap).sorted.mkString("\n")}")
+      logInfo(s"Classpath elements:\n${childClasspath.sorted.mkString("\n")}")

Review comment:
       Likewise classpath order matters, so not sure we should do this.
   Spark configs ... yeah order should not matter, so that should be sorted.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org