You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "aiborodin (via GitHub)" <gi...@apache.org> on 2025/10/30 07:54:04 UTC
[PR] Fix commit duplication in DynamicIcebergSink [iceberg]
aiborodin opened a new pull request, #14445:
URL: https://github.com/apache/iceberg/pull/14445
Fix the race condition in https://github.com/apache/iceberg/issues/14425 by validating that all new parent snapshots, fetched by the table refresh in `SnapshotProducer.apply()`, have a committed checkpoint id strictly lower than the checkpoint id of the currently staged committable. Abort the commit to prevent duplication otherwise.
We should also add tests and validate other updates (eg `ReplacePartitions`).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Fix commit duplication in DynamicIcebergSink [iceberg]
Posted by "aiborodin (via GitHub)" <gi...@apache.org>.
aiborodin commented on PR #14445:
URL: https://github.com/apache/iceberg/pull/14445#issuecomment-3479133462
@pvary @mrcnc I pushed an alternative solution in https://github.com/apache/iceberg/pull/14484, which extends `BaseRowDelta` and `BaseReplacePartitions` operations in Flink packages instead of adding API methods to `RowDelta` and `ReplacePartitions`.
Please let me know which solution you think works best.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Fix commit duplication in DynamicIcebergSink [iceberg]
Posted by "aiborodin (via GitHub)" <gi...@apache.org>.
aiborodin commented on PR #14445:
URL: https://github.com/apache/iceberg/pull/14445#issuecomment-3477501350
@pvary @mrcnc I added unit and integration tests reproducing the issue in https://github.com/apache/iceberg/issues/14425. I would appreciate it if someone could take a look.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Fix commit duplication in DynamicIcebergSink [iceberg]
Posted by "aiborodin (via GitHub)" <gi...@apache.org>.
aiborodin commented on PR #14445:
URL: https://github.com/apache/iceberg/pull/14445#issuecomment-3495801153
The new PR is raised in https://github.com/apache/iceberg/pull/14517.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Fix commit duplication in DynamicIcebergSink [iceberg]
Posted by "aiborodin (via GitHub)" <gi...@apache.org>.
aiborodin closed pull request #14445: Fix commit duplication in DynamicIcebergSink
URL: https://github.com/apache/iceberg/pull/14445
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Fix commit duplication in DynamicIcebergSink [iceberg]
Posted by "aiborodin (via GitHub)" <gi...@apache.org>.
aiborodin commented on PR #14445:
URL: https://github.com/apache/iceberg/pull/14445#issuecomment-3471481567
@pvary Could you please take a look?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org