You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@paimon.apache.org by "XiaoHongbo-Hope (via GitHub)" <gi...@apache.org> on 2026/07/30 14:15:15 UTC

[PR] [core] Drop concurrently-built global indexes at materialize-deletion compaction [paimon]

XiaoHongbo-Hope opened a new pull request, #8947:
URL: https://github.com/apache/paimon/pull/8947

   ### Purpose
   
   `DataEvolutionCompactGlobalIndexDropper` drops global indexes that a
   `materialize-deletion` compaction invalidates (the compaction renumbers row-ids,
   so pre-materialization indexes become stale). It scanned the compaction's **base**
   snapshot for indexes to drop. A global index built concurrently and committed
   **after** the compaction planned still encodes pre-materialization row-ids, but is
   not visible at the base snapshot, so it survives — and a later vector search
   resolves its stale row-ids against renumbered data.
   
   Latent on master today (index build on DV tables is currently blocked); reachable
   once Lumina indexes on DV tables are allowed. Self-contained; submitted
   independently.
   
   ### Change
   
   Scan the **latest** snapshot for stale indexes instead of the compaction base, so
   concurrently-committed indexes are also dropped (falling back to the base snapshot
   when no newer one exists).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [core] Drop concurrently-built global indexes at materialize-deletion compaction [paimon]

Posted by "JingsongLi (via GitHub)" <gi...@apache.org>.
JingsongLi merged PR #8947:
URL: https://github.com/apache/paimon/pull/8947


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [core] Drop concurrently-built global indexes at materialize-deletion compaction [paimon]

Posted by "JingsongLi (via GitHub)" <gi...@apache.org>.
JingsongLi commented on PR #8947:
URL: https://github.com/apache/paimon/pull/8947#issuecomment-5138472354

   +1


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org