[GitHub] [iceberg] openinx commented on a change in pull request #2799: Flink: merge the configurations of hive-conf-dir to CatalogLoader

[GitBox <gi...@apache.org>]

openinx commented on a change in pull request #2799:
URL: https://github.com/apache/iceberg/pull/2799#discussion_r667571844



##########
File path: flink/src/main/java/org/apache/iceberg/flink/CatalogLoader.java
##########
@@ -51,7 +51,9 @@ static CatalogLoader hadoop(String name, Configuration hadoopConf, Map<String, S
   }
 
   static CatalogLoader hive(String name, Configuration hadoopConf, Map<String, String> properties) {
-    return new HiveCatalogLoader(name, hadoopConf, properties);
+    String hiveConfDir = properties.get(FlinkCatalogFactory.HIVE_CONF_DIR);
+    Configuration newHadoopConf = FlinkCatalogFactory.mergeHiveConf(hadoopConf, hiveConfDir);

Review comment:
       You are using flink datastream API to submit the flink jobs right ?  In theory,  the `hadoopConf` cloud be passed as any hadoop `Configuration` if your want (For example, you can add hive-site.xml to `hadoopConf` outside this method).  Another side,  we could load the `HiveCatalog` without specifying any hive-site.xml configurations because we will set all the necessary key-values configurations from `propeties` to `hiveConf` ( see [code](https://github.com/apache/iceberg/blob/90225d6c9413016d611e2ce5eff37db1bc1b4fc5/hive-metastore/src/main/java/org/apache/iceberg/hive/HiveCatalog.java#L96)). So all you need to do is filing the correct hive properties in the `Map<String, String> properties`: 
   
   ```java
   Map<String, String> properties = Maps.newHashMap();
   properties.set("uri", "thrift://localhost:9083");
   properties.set("warehouse", "hdfs://nn:9090/path/to/warehouse");
   properties.set("clients", "5");
   ```
   
   Then you cloud load the correct `HiveCatalog`.    So I think we don't need to have this changes in our master codebase. 




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org