[GitHub] [iceberg] vinitamaloo-asu commented on issue #8277: Unable to create Iceberg table with HiveCatalog (RuntimeMetaException: Failed to connect to Hive Metastore)

["vinitamaloo-asu (via GitHub)" <gi...@apache.org>]

vinitamaloo-asu commented on issue #8277:
URL: https://github.com/apache/iceberg/issues/8277#issuecomment-1672685374

   I am trying to connect to a local embedded metastore that's why kept this uri empty.
   
   
   From my testing, I observed that when I am initializing a new HiveCatalog, it does not create HiveConf with SparkConf and only has hadoop configuration.
   When SparkSession is initially configured, it creates a new HiveConf which has both sparkconf and hadoopConfiguration.
   
   Snippet from HiveClientImpl(newHiveConf):
   `    val confMap = (hadoopConf.iterator().asScala.map(kv => kv.getKey -> kv.getValue) ++
         sparkConf.getAll.toMap ++ extraConfig).toMap
       confMap.foreach { case (k, v) => hiveConf.set(k, v) }
       SQLConf.get.redactOptions(confMap).foreach { case (k, v) =>
         logDebug(s"Applying Hadoop/Hive/Spark and extra properties to Hive Conf:$k=$v")
       }`
   
   
   
   And since my new hivecatalog does not have sparkConfig attached it also does not have all the hive configurations which I set during the sparkSession builder phase.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org