You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@polaris.apache.org by "adnanhemani (via GitHub)" <gi...@apache.org> on 2025/11/18 09:43:54 UTC

[PR] Fixes phantom event emission; Introduces StageCommitTableEvent [polaris]

adnanhemani opened a new pull request, #3073:
URL: https://github.com/apache/polaris/pull/3073

This PR introduces a new event type `StageCommitTableEvent`, which triggers only in the case of a transactional commit at the time the staged commit is generated (before the commit is actually applied). This allows us to reserve the `AfterCommitTableEvent` solely for when the commit has been successfully applied.

In order to do this, I've made a slight addition to the `TransactionWorkspaceMetaStoreManager` class to add the staging of events, mirroring exactly how we stage updates. Noting that there is no unit testing for this class, I've added this as well.

Additionally, E2E testing for the events emissions in the CommitTransaction API has been added, looking both at the happy and sad paths.

## Checklist
- [X] 🛡️ Don't disclose security issues! (contact security@apache.org)
- [X] 🔗 Clearly explained why the changes are needed, or linked related issues: Fixes #2990.
- [X] 🧪 Added/updated tests with good coverage, or manually tested (and explained how)
- [X] 💡 Added comments for complex logic
- [N/A] 🧾 Updated `CHANGELOG.md` (if needed)
- [N/A] 📚 Updated documentation in `site/content/in-dev/unreleased` (if needed)

Closes #2990.

--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] Fixes phantom event emission; Introduces StageCommitTableEvent [polaris]

Posted by "adnanhemani (via GitHub)" <gi...@apache.org>.

adnanhemani commented on PR #3073:
URL: https://github.com/apache/polaris/pull/3073#issuecomment-3608548974

   Converting to draft as #3195 will also fix the base GH issue (#2990). Once #3195 merges, we can close this PR altogether.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] Fixes phantom event emission; Introduces StageCommitTableEvent [polaris]

Posted by "adnanhemani (via GitHub)" <gi...@apache.org>.

adnanhemani closed pull request #3073: Fixes phantom event emission; Introduces StageCommitTableEvent
URL: https://github.com/apache/polaris/pull/3073


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] Fixes phantom event emission; Introduces StageCommitTableEvent [polaris]

Posted by "adnanhemani (via GitHub)" <gi...@apache.org>.

adnanhemani commented on PR #3073:
URL: https://github.com/apache/polaris/pull/3073#issuecomment-3634413177

   Closed as #3195 was merged.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] Fixes phantom event emission; Introduces StageCommitTableEvent [polaris]

Posted by "dimas-b (via GitHub)" <gi...@apache.org>.

dimas-b commented on code in PR #3073:
URL: https://github.com/apache/polaris/pull/3073#discussion_r2543827519


##########
runtime/service/src/main/java/org/apache/polaris/service/catalog/iceberg/IcebergCatalog.java:
##########
@@ -1553,9 +1554,20 @@ public void doCommit(TableMetadata base, TableMetadata metadata) {
         updateTableLike(tableIdentifier, entity);
       }
 
-      polarisEventListener.onAfterCommitTable(
-          new IcebergRestCatalogEvents.AfterCommitTableEvent(
-              catalogName, tableIdentifier, base, metadata));
+      if (getMetaStoreManager() instanceof TransactionWorkspaceMetaStoreManager) {

Review Comment:
   This `if` + `instanceof` looks concerning to me. I do believe the whole events design can and should be done in a way that processing logic does not require knowing specific types of pluggable components.
   
   `TransactionalWorkspaceMetaStoreManager` in itself has API design [concerns from my POV](https://lists.apache.org/thread/rf5orxs815zs4h64p4rwp03q3pbgxb5r), so I believe this area of the code needs refactoring in general, so adding logic that depends on specific types is both risky and complicates future refactoring.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] Fixes phantom event emission; Introduces StageCommitTableEvent [polaris]

Posted by "adnanhemani (via GitHub)" <gi...@apache.org>.

adnanhemani commented on PR #3073:
URL: https://github.com/apache/polaris/pull/3073#issuecomment-3546516698

   cc @dimas-b
   
   Please let me know what you think with regards to your suggestion of needing a README/extended documentation. Also, should we add this to the CHANGELOG? That a new event type has been added?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org