[GitHub] [iceberg] RussellSpitzer commented on a diff in pull request #4325: Spark:Skip corrupt files in Spark Procedures and Actions

[GitBox <gi...@apache.org>]

RussellSpitzer commented on code in PR #4325:
URL: https://github.com/apache/iceberg/pull/4325#discussion_r867346285


##########
data/src/main/java/org/apache/iceberg/data/TableMigrationUtil.java:
##########
@@ -121,9 +156,13 @@ private TableMigrationUtil() {
           datafiles[index] = buildDataFile(fileStatus.get(index), partitionKey, spec, metrics, "orc");
         });
       } else {
-        throw new UnsupportedOperationException("Unknown partition format: " + format);
+        if (skipCorruptFiles) {
+          LOG.warn("Unknown partition format: {}", format);

Review Comment:
   In the warn can you just add 
   `"Skipping unknown partition format: {} - {}", format, partitionUri)`
   Or whatever makes sense so a user has the full context here?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org