You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@paimon.apache.org by "steFaiz (via GitHub)" <gi...@apache.org> on 2026/07/21 06:35:56 UTC

[PR] [core] detect concurrent row-id reassignment and data-evolution compaction [paimon]

steFaiz opened a new pull request, #8773:
URL: https://github.com/apache/paimon/pull/8773

   ### Purpose
   The main purpose is to ensure data evolution DV works with row-id reassignment.
   
   I added some tests and find an another issue: Concurrent row-id reassignment and DataEvolution Compaction:
   
     A compaction may be planned using an old RowID range. If RowID reassignment commits before that compaction,
     committing the stale compaction could restore the old RowIDs. For example, reassignment moves [0, 9] to [10,
     19], but a stale compaction may move them back to [0, 9].
   
   So I added row-id existence check for compact commit:
   1. Collecting existing data ranges with `MERGE Adjacent`(while non-compact commit will not merge)
   2. Each generated compaction files should be contained by old merge row ranges (by part and bucket)
   
   ### Tests
   1. ConflictDetectionTest
   2. DataEvolutionDeletionTest


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] [core] detect concurrent row-id reassignment and data-evolution compaction [paimon]

Posted by "JingsongLi (via GitHub)" <gi...@apache.org>.
JingsongLi commented on PR #8773:
URL: https://github.com/apache/paimon/pull/8773#issuecomment-5031404270

   +1


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] [core] detect concurrent row-id reassignment and data-evolution compaction [paimon]

Posted by "JingsongLi (via GitHub)" <gi...@apache.org>.
JingsongLi merged PR #8773:
URL: https://github.com/apache/paimon/pull/8773


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org