[PR] [core] Wait for global index in visibility callback [paimon]

["JingsongLi (via GitHub)" <gi...@apache.org>]

JingsongLi opened a new pull request, #8265:
URL: https://github.com/apache/paimon/pull/8265

   ## Summary
   
   Support waiting for newly committed row-tracking data to be covered by existing global indexes when `visibility-callback.enabled` is enabled. This extends the visibility callback beyond compaction so batch writes do not return before global index visibility catches up.
   
   ## Changes
   
   - Enable `VisibilityWaitCallback` for row-tracking tables when visibility callback is configured.
   - Track row id ranges from newly added append data files and wait until each existing global index covers those ranges.
   - Preserve the existing deletion-vector and postpone-bucket compaction wait behavior.
   - Update the option documentation and generated configuration docs.
   - Add a Java test that verifies a write waits until incremental BTree global index build commits.
   
   ## Testing
   
   - `mvn -pl paimon-api,paimon-core -DskipTests compile`
   - `mvn -pl paimon-core -Dtest=VisibilityWaitCallbackTest test`
   - `git diff --check`
   
   ## Notes
   
   This applies only to batch or bounded stream commits, matching the existing visibility callback behavior.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] [core] Wait for global index in visibility callback [paimon]

["JingsongLi (via GitHub)" <gi...@apache.org>]

JingsongLi merged PR #8265:
URL: https://github.com/apache/paimon/pull/8265


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org