You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "mxm (via GitHub)" <gi...@apache.org> on 2026/03/19 12:48:47 UTC
[PR] Flink: Fix non-deterministic operator UIDs in DynamicIcebergSink [iceberg]
mxm opened a new pull request, #15687:
URL: https://github.com/apache/iceberg/pull/15687
The pre-commit topology operator UID included a random UUID (sinkId), which changed on every sink instantiation. This prevented Flink from restoring state from checkpoints/savepoints because operator UIDs didn't match across full redeploys. This did not affect simple failures / restarts because the JobGraph is not re-generated then.
Use a fixed suffix for the UID, consistent with the other operators (generator, updater, sink) which already use deterministic UIDs. The sinkId is still used in the committer for file separation.
I deliberately did not change the UID syntax. Despite the bug, users will still be able to restore state, e.g. via Flink's `allowNonRestoredState`, but we must keep the stateful operators UUIds like the committer consistent.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Flink: Fix non-deterministic operator UIDs in DynamicIcebergSink [iceberg]
Posted by "pvary (via GitHub)" <gi...@apache.org>.
pvary commented on PR #15687:
URL: https://github.com/apache/iceberg/pull/15687#issuecomment-4099004659
Merged to main.
Thanks @mxm for the PR and the offline explanation!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Flink: Fix non-deterministic operator UIDs in DynamicIcebergSink [iceberg]
Posted by "mxm (via GitHub)" <gi...@apache.org>.
mxm commented on PR #15687:
URL: https://github.com/apache/iceberg/pull/15687#issuecomment-4099186894
Thanks @pvary!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Flink: Fix non-deterministic operator UIDs in DynamicIcebergSink [iceberg]
Posted by "pvary (via GitHub)" <gi...@apache.org>.
pvary merged PR #15687:
URL: https://github.com/apache/iceberg/pull/15687
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org