[GitHub] [iceberg] jackye1995 commented on issue #2176: Running iceberg with spark 3 in local mode

[GitBox <gi...@apache.org>]

jackye1995 commented on issue #2176:
URL: https://github.com/apache/iceberg/issues/2176#issuecomment-769967215


   You are using hive catalog for your `spark_catalog` catalog, and hadoop catalog for your `local` catalog.
   
   For hive catalog, you need to also specify the `uri` config, and you also need `warehouse` config:
   
   ```
   .config("spark.sql.catalog.spark_catalog", "org.apache.iceberg.spark.SparkSessionCatalog")
   .config("spark.sql.catalog.spark_catalog.type", "hive")
   .config("spark.sql.catalog.spark_catalog.uri", "https://my.hive.cluster.com:9083")
   .config("spark.sql.catalog.spark_catalog.warehouse", "/warehouse")
   ```
   
   Combine with Russell's answer, the complete config should be:
   
   ```java
   .config("spark.sql.catalog.spark_catalog", "org.apache.iceberg.spark.SparkSessionCatalog")
   .config("spark.sql.catalog.spark_catalog.type", "hive")
   .config("spark.sql.catalog.spark_catalog.uri", "https://my.hive.ip.com:9083")
   .config("spark.sql.catalog.spark_catalog.warehouse", "/warehouse-hive") // I use a different path here to distinguish the two catalogs
   .config("spark.sql.catalog.local", "org.apache.iceberg.spark.SparkSessionCatalog")
   .config("spark.sql.catalog.local.type", "hadoop")
   .config("spark.sql.catalog.local.warehouse", "/warehouse")
   ```


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org