You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "herbherbherb (via GitHub)" <gi...@apache.org> on 2026/03/25 17:34:21 UTC
[PR] Flink Sink V2: Add PostCommitHook plugin interface [iceberg]
herbherbherb opened a new pull request, #15769:
URL: https://github.com/apache/iceberg/pull/15769
### Summary
Adds a `PostCommitHook` plugin interface to `IcebergSink` that is invoked after each successful Iceberg commit. This enables use cases like updating table properties with watermark metadata, triggering notifications, recording metrics, or synchronizing external metadata stores.
Resolves #15768.
### Changes
- New: `PostCommitHook.java` -- `@FunctionalInterface` with a single method: `void afterCommit(long snapshotId, Map<String, String> summary)`
- Modified: `IcebergCommitter` -- accepts optional hook, invokes it after successful commit
- Modified: `IcebergSink.Builder` -- new `postCommitHook()` method, passed through to committer
### Compatibility
- No behavioral change when the hook is not set (null default)
- No changes to public API signatures of existing methods
- Fully backward compatible
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Flink Sink V2: Add PostCommitHook plugin interface [iceberg]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #15769:
URL: https://github.com/apache/iceberg/pull/15769#issuecomment-5040480375
This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If you think that is incorrect, or the pull request requires review, you can revive the PR at any time.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Flink Sink V2: Add PostCommitHook plugin interface [iceberg]
Posted by "herbherbherb (via GitHub)" <gi...@apache.org>.
herbherbherb commented on PR #15769:
URL: https://github.com/apache/iceberg/pull/15769#issuecomment-4363907931
comment to keep the PR open, waiting for committee's review
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Flink Sink V2: Add PostCommitHook plugin interface [iceberg]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #15769:
URL: https://github.com/apache/iceberg/pull/15769#issuecomment-4975454290
This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the dev@iceberg.apache.org list. Thank you for your contributions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Flink Sink V2: Add PostCommitHook plugin interface [iceberg]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #15769:
URL: https://github.com/apache/iceberg/pull/15769#issuecomment-4696824559
This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the dev@iceberg.apache.org list. Thank you for your contributions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Flink Sink V2: Add PostCommitHook plugin interface [iceberg]
Posted by "herbherbherb (via GitHub)" <gi...@apache.org>.
herbherbherb commented on PR #15769:
URL: https://github.com/apache/iceberg/pull/15769#issuecomment-4305009028
@pvary Thank you for the feedback [here](https://github.com/apache/iceberg/pull/15316#issuecomment-4069625607). I completely understand your concern regarding exposing the internal components of `IcebergSink` and forcing the community to maintain those contracts long-term.
Based on your suggestion, I took a step back and revised the architecture to avoid exposing those internals. Instead of changing access modifiers, I am proposing we add a few independent hooks:
1. OutputFileFactoryProvider — custom `OutputFileFactory` creation (https://github.com/apache/iceberg/issues/15763 /
https://github.com/apache/iceberg/pull/15764)
2. PostCommitHook — callback after successful Iceberg commit (https://github.com/apache/iceberg/issues/15768 /
https://github.com/apache/iceberg/pull/15769)
3. writeObserver — observes each record written and produces per-checkpoint metadata that flows through the sink pipeline (https://github.com/apache/iceberg/issues/15783 / https://github.com/apache/iceberg/pull/15784)
Does this plugin-based approach alleviate your concerns about locking in the internal architecture? If this looks like a better direction, I would really appreciate it if you could take a look and review these individual PRs when you have a chance, much appreciated.
Also tagging @bryanck
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Flink Sink V2: Add PostCommitHook plugin interface [iceberg]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed pull request #15769: Flink Sink V2: Add PostCommitHook plugin interface
URL: https://github.com/apache/iceberg/pull/15769
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Flink Sink V2: Add PostCommitHook plugin interface [iceberg]
Posted by "herbherbherb (via GitHub)" <gi...@apache.org>.
herbherbherb commented on PR #15769:
URL: https://github.com/apache/iceberg/pull/15769#issuecomment-4698568806
commenting to keep the PR open, will follow-up with next step
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Flink Sink V2: Add PostCommitHook plugin interface [iceberg]
Posted by "pvary (via GitHub)" <gi...@apache.org>.
pvary commented on PR #15769:
URL: https://github.com/apache/iceberg/pull/15769#issuecomment-4437777362
These are sweeping changes. Please create a proposal, start a discussion on the dev list, or during the Community Sync.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org