You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "mbutrovich (via GitHub)" <gi...@apache.org> on 2026/04/16 03:08:05 UTC
[PR] deps: bump iceberg-rust to latest after picking up fixes for #3856 and #3860 [datafusion-comet]
mbutrovich opened a new pull request, #3958:
URL: https://github.com/apache/datafusion-comet/pull/3958
## Which issue does this PR close?
Closes #3856.
Closes #3860.
## Rationale for this change
Two bugs in iceberg-rust were causing incorrect results and errors in Comet's native Iceberg scan:
1. INT96 timestamps were read with the wrong TimeUnit (nanoseconds instead of microseconds), producing ~1170 year offsets on migrated tables with far-future dates.
2. `PredicateConverter` mapped incorrect Parquet leaf column indices for migrated tables containing nested types (struct/array/map), causing "column isn't a root column" errors when filter predicates were pushed down.
Both were fixed upstream in iceberg-rust ([#2301](https://github.com/apache/iceberg-rust/pull/2301), [#2307](https://github.com/apache/iceberg-rust/pull/2307)).
## What changes are included in this PR?
- Bump `iceberg` and `iceberg-storage-opendal` dependencies to iceberg-rust commit `a2f067d` which includes both fixes.
- Add two regression tests to `CometIcebergNativeSuite`.
## How are these changes tested?
- `migration - INT96 timestamp`: writes Parquet with INT96 timestamps (flat, struct, array, map) using dates outside the nanosecond range, imports into Iceberg, and verifies Comet reads match Spark/raw Parquet.
- `filter with nested types in migrated table`: creates a migrated table with struct/array/map columns and a trailing flat column, then verifies filter predicates on that flat column work correctly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] deps: bump iceberg-rust to latest after picking up fixes for #3856 and #3860 [datafusion-comet]
Posted by "mbutrovich (via GitHub)" <gi...@apache.org>.
mbutrovich merged PR #3958:
URL: https://github.com/apache/datafusion-comet/pull/3958
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org