Re: [PR] Fix commit idempotence of DynamicIcebergSink [iceberg]

["aiborodin (via GitHub)" <gi...@apache.org>]

aiborodin commented on PR #14092:
URL: https://github.com/apache/iceberg/pull/14092#issuecomment-3305370201

   > 1. Fixing a potential issue in DynamicCommitter with processing multiple WriteResult for a given table.
   > 2. Refactoring the commit logic to produce one snapshot per spec change.
   > 
   > It would make things easier to review if we fixed (1) and then (2), perhaps in separate commits. I need to dig my head a bit deeper into the changes to understand them properly.
   
   @mxm How do you envision splitting these two points? The root problem is `DynamicWriteResultAggregator` producing multiple commit requests per table, branch, and checkpointId triplet. The `DynamicCommitter` is agnostic to this and commits upstream requests; it doesn't have any special handling for partition spec changes. I cleaned up `DynamicCommitter` to reflect a single commit expectation, which feels like it should be part of this change.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org