[GitHub] [iceberg] rdblue commented on a change in pull request #1161: Fix errors when use removeOrphanFiles By HadoopCatalog

[GitBox <gi...@apache.org>]

rdblue commented on a change in pull request #1161:
URL: https://github.com/apache/iceberg/pull/1161#discussion_r449655254



##########
File path: spark/src/main/java/org/apache/iceberg/actions/BaseAction.java
##########
@@ -30,8 +31,11 @@ protected String metadataTableName(MetadataTableType type) {
     String tableName = table().toString();
     if (tableName.contains("/")) {
       return tableName + "#" + type;
-    } else if (tableName.startsWith("hadoop.") || tableName.startsWith("hive.")) {
-      // HiveCatalog and HadoopCatalog prepend a logical name which we need to drop for Spark 2.4
+    } else if (tableName.startsWith("hadoop.")) {
+      // HadoopCatalog tableName style is 'hadoop.ns.tb'

Review comment:
       I don't think this comment is correct because the solution is to convert to a path table reference.
   
   If i understand correctly, the problem in Spark 2.4 is that IcebergSource uses the Hive catalog or it uses HadoopTables to load a path. The table you're passing in was loaded by HadoopCatalog, so it doesn't work because that catalog is not available. I agree that the solution is to convert a Hadoop table to a path reference, but then this comment should explain what's happening: a HadoopCatalog was used to load the table, so convert to the path.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org