You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "danny0405 (via GitHub)" <gi...@apache.org> on 2023/08/06 04:57:20 UTC

[GitHub] [hudi] danny0405 commented on a diff in pull request #9221: [HUDI-6550] Add Hadoop conf to HiveConf for HiveSyncConfig

danny0405 commented on code in PR #9221:
URL: https://github.com/apache/hudi/pull/9221#discussion_r1285152739


##########
hudi-sync/hudi-hive-sync/src/main/java/org/apache/hudi/hive/HiveSyncConfig.java:
##########
@@ -98,8 +98,9 @@ public HiveSyncConfig(Properties props) {
 
   public HiveSyncConfig(Properties props, Configuration hadoopConf) {
     super(props, hadoopConf);
-    HiveConf hiveConf = hadoopConf instanceof HiveConf
-        ? (HiveConf) hadoopConf : new HiveConf(hadoopConf, HiveConf.class);
+    HiveConf hiveConf = new HiveConf();
+    // HiveConf needs to load Hadoop conf to allow instantiation via AWSGlueClientFactory
+    hiveConf.addResource(hadoopConf);

Review Comment:
   > but it's possible that there are other configs/custom configs passed in via Spark session,
   
   Is this a classical way people pass around hive options with spark?
   
   > An alternative solution would be always pass hadoopConf to HiveConf constructor
   
   Does it introduce too much overhead then?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org