[GitHub] [iceberg] rdblue commented on issue #2553: Spark filters do not work on int96 timestamp columns

[GitBox <gi...@apache.org>]

rdblue commented on issue #2553:
URL: https://github.com/apache/iceberg/issues/2553#issuecomment-833724970


   The metrics setting is for Iceberg metrics and this is happening when filtering row groups in Parquet. That's why the metrics setting doesn't help. We just need to update Iceberg's [ParquetMetricsRowGroupFilter](https://github.com/apache/iceberg/blob/master/parquet/src/main/java/org/apache/iceberg/parquet/ParquetMetricsRowGroupFilter.java#L97) to ignore INT96 stats. I thought that Parquet would not return them in the first place, which is why I think we overlooked this. Would you like to open a PR with the fix, @thesquelched?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org