[GitHub] [spark] hehuiyuan commented on a change in pull request #24009: [k8s]Unify the three variables' name : pod name prefix in kubernetes / spark.app.name in spark ui / spark-app-name in pod's annotations

[GitBox <gi...@apache.org>]

hehuiyuan commented on a change in pull request #24009: [k8s]Unify the three variables' name :       pod name prefix in kubernetes / spark.app.name in spark ui  /  spark-app-name in pod's annotations
URL: https://github.com/apache/spark/pull/24009#discussion_r263628017
 
 

 ##########
 File path: core/src/main/scala/org/apache/spark/SparkConf.scala
 ##########
 @@ -115,9 +115,13 @@ class SparkConf(loadDefaults: Boolean) extends Cloneable with Logging with Seria
     set("spark.master", master)
   }
 
-  /** Set a name for your application. Shown in the Spark web UI. */
+  /** Set a name for your application. Shown in the Spark web UI.
+    * For spark on kubernetes,Unify the three variables' name :
 
 Review comment:
     /** Set a name for your application. Shown in the Spark web UI.
       * For spark on kubernetes,Unify the three variables' name :
       *     pod name prefix in kubernetes / spark.app.name in spark ui  /  spark-app-name in pod's annotations
       * */
   
   It uses the value of "-- name"  to name the pod name for spark on k8s when submitting tasks by spark-submit . Meanwhile KubernetesClientApplication calls start function ：
   `start(args: Array[String], conf: SparkConf)`
   in conf parameter, the value of  spark.app.name equals the value of "--name".
   But the value of spark.app.name is not coincident.
   `val conf = new SparkConf().**setAppName("JavaNetworkWordCount")`
   `    _ui =
         if (conf.getBoolean("spark.ui.enabled", true)) {
           Some(SparkUI.create(Some(this), _statusStore, _conf, _env.securityManager, appName, "",
             startTime))
         } else {
           // For tests, do not enable the UI
           None
         }`
   
   Create an UI by calling SparkUI's create function which contains _conf  parameter.
   The value of spark.app.name is setting by calling setAppName method.
     
   KubernetesClientApplication.start  and  SparkUI.create  ,both functions have SparkConf parameter. They have the possibility of inconsistency.
   
   Refer to the pictures and content in the previous comments
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org