[GitHub] [spark] ulysses-you commented on a diff in pull request #42049: [SPARK-44466][SQL] Exclude configs starting with `SPARK_DRIVER_PREFIX` and `SPARK_EXECUTOR_PREFIX` from modifiedConfigs

["ulysses-you (via GitHub)" <gi...@apache.org>]

ulysses-you commented on code in PR #42049:
URL: https://github.com/apache/spark/pull/42049#discussion_r1270153264


##########
sql/core/src/main/scala/org/apache/spark/sql/execution/SQLExecution.scala:
##########
@@ -97,7 +98,11 @@ object SQLExecution {
 
       val globalConfigs = sparkSession.sharedState.conf.getAll.toMap
       val modifiedConfigs = sparkSession.sessionState.conf.getAllConfs
-        .filterNot(kv => globalConfigs.get(kv._1).contains(kv._2))
+        .filterNot { case (key, value) =>
+          key.startsWith(SPARK_DRIVER_PREFIX) ||
+            key.startsWith(SPARK_EXECUTOR_PREFIX) ||
+            globalConfigs.get(key).contains(value)

Review Comment:
   make sense to me as these configs are kind of static configs, but cc @HyukjinKwon I'm not sure if python side depends on any configs with these prefix at runtime



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org