[GitHub] [spark] cloud-fan commented on a change in pull request #27041: [SPARK-25061][SQL] Precedence for ThriftServer hiveconf commandline parameter

[GitBox <gi...@apache.org>]

cloud-fan commented on a change in pull request #27041: [SPARK-25061][SQL] Precedence for ThriftServer hiveconf commandline parameter
URL: https://github.com/apache/spark/pull/27041#discussion_r380460761
 
 

 ##########
 File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveClientImpl.scala
 ##########
 @@ -176,9 +176,18 @@ private[hive] class HiveClientImpl(
     // is not set to builtin. When spark.sql.hive.metastore.jars is builtin, the classpath
     // has hive-site.xml. So, HiveConf will use that to override its default values.
     // 2: we set all spark confs to this hiveConf.
-    // 3: we set all entries in config to this hiveConf.
+    // 3: we take the conf passed as --hiveconf which would be set as system properties
+    // by org.apache.hive.service.server.HiveServer2.ServerOptionsProcessor.parse in
+    // org.apache.spark.sql.hive.thriftserver.HiveThriftServer2.main.
+    // 4: we set all entries in extraConfig to this hiveConf which have the highest precedence.
+    // To summarize, the order of precedence will be
+    // hadoopConf < sparkConf < overrideProps < extraConfig
+
+    // not to lose command line overwritten properties
+    // make a copy overridden props so that it can be reinserted finally
+    val overriddenHiveProps = HiveConf.getConfSystemProperties.asScala
 
 Review comment:
   so we totally ignore the --hive-conf previously?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org