[GitHub] [iceberg] RussellSpitzer commented on a change in pull request #1784: Fix Resolving of SparkSession Table's Metadata Tables

[GitBox <gi...@apache.org>]

RussellSpitzer commented on a change in pull request #1784:
URL: https://github.com/apache/iceberg/pull/1784#discussion_r526401342



##########
File path: spark/src/main/java/org/apache/iceberg/actions/BaseSparkAction.java
##########
@@ -137,7 +138,19 @@
     }
     // Try catalog based name based resolution
     try {
-      return spark.table(tableName + "." + type);
+      if (tableName.startsWith("spark_catalog")) {
+        // Do to the design of Spark, we cannot pass multi element namespaces to the session catalog
+        // We also don't know whether the Catalog is Hive or Hadoop Based, so we will try to load it
+        // in the hive manner first, then fall back and try the location if we have completely run out of options
+        // TODO remove this when we have Spark workaround for multipart identifiers in SparkSessionCatalog
+        try {
+          return noCatalogReader.load(tableName.replaceFirst("spark_catalog\\.", "") + "." + type);

Review comment:
       One huge drawback to this is we can't actually use any Catalog Options that have been specified, so if we have options in the future than alter the way a Metadata table is read we won't have them here.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org