[I] Adopt a pin-bump policy for the iceberg-rust dependency [datafusion-comet]

["andygrove (via GitHub)" <gi...@apache.org>]

andygrove opened a new issue, #5645:
URL: https://github.com/apache/datafusion-comet/issues/5645

   ### What is the problem the feature request solves?
   
   `native/Cargo.toml` pins `iceberg` and `iceberg-storage-opendal` to a git revision (`3d84c81353b1b23b6e4ae8eea8f8a021cc6927a7`) rather than a release. The pin is what the native Iceberg scan and writer are tested against, but nothing decides when it moves, and #5636 is the first bug caused by it drifting behind an upstream fix (partition-path escaping, apache/iceberg-rust#2875). With the writer now producing files that iceberg-java reads back, a stale pin can turn an upstream bug fix into a Comet correctness bug.
   
   ### Describe the potential solution
   
   Write down and follow a policy, for example:
   
   - Bump the pin at least once per Comet release cycle, and immediately when an upstream fix affects file bytes, manifest metadata, or partition layout.
   - Prefer tagged iceberg-rust releases over arbitrary revisions once a release contains everything Comet needs, so the version is visible in `Cargo.lock` and in the docs.
   - Each bump runs the Iceberg suites on all four Spark/Iceberg profiles plus the manifest-metrics parity tests, since those are what catch behavior changes in the writer.
   - Track "waiting on upstream" items (see the eligibility-restrictions issue) so a bump can also lift restrictions.
   - Note the bump in the release notes, since users reading Comet-written tables with iceberg-java care which writer semantics they got.
   
   ### Additional context
   
   Part of the native Iceberg writes epic. Related: #5636.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org