You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@iceberg.apache.org by GitBox <gi...@apache.org> on 2022/02/07 19:12:51 UTC

[GitHub] [iceberg] dubeme commented on issue #4051: How do I make Iceberg ignore corrupt (parquet, ORC, avro) file?

dubeme commented on issue #4051:
URL: https://github.com/apache/iceberg/issues/4051#issuecomment-1031821580


   > Ah I misread you are looking for such an option. That we could do, I think it wouldn't be that hard to put a try catch in add-files that is only used if a flag "skip-corrupt-files" or something is set.
   
   I see you read things like I do 😂 Yes, I want to Ignore corrupt files during Table import via _add_files_. Ideally controlled with a flag. 
   
   I've been looking at the code path, and it seems `TableMigrationUtil.listPartition` (https://github.com/apache/iceberg/blob/2208b24617f2c64bb02f1a7aec79f3bbaabdc1cc/data/src/main/java/org/apache/iceberg/data/TableMigrationUtil.java#L106) is the place to start?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org