You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "hudi-bot (via GitHub)" <gi...@apache.org> on 2025/11/30 03:21:49 UTC
[I] Support out of order rollback blocks in AbstractHoodieLogRecordReader [hudi]
hudi-bot opened a new issue, #14437:
URL: https://github.com/apache/hudi/issues/14437
Currently the rollback block refers to previous data block for invalidating. In multiwriter scenario the block need not be previous block. So, need to support out of order rollback blocks.
## JIRA info
- Link: https://issues.apache.org/jira/browse/HUDI-3919
- Type: Improvement
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Support out of order rollback blocks in AbstractHoodieLogRecordReader [hudi]
Posted by "nsivabalan (via GitHub)" <gi...@apache.org>.
nsivabalan commented on issue #14437:
URL: https://github.com/apache/hudi/issues/14437#issuecomment-4795158848
This issue appears to be resolved on current `master`. The class was renamed to `BaseHoodieLogRecordReader`, and `scanInternal` now uses a two-pass scan that explicitly supports out-of-order rollback blocks in multi-writer scenarios.
The Javadoc at `hudi-common/src/main/java/org/apache/hudi/common/table/log/BaseHoodieLogRecordReader.java` lines 224-248 calls out the exact scenario from this issue:
> Scanning blocks is easy to do in single writer mode, where the rollback block is right after the effected data blocks. With multi-writer mode the blocks can be out of sync. An example scenario. `B1, B2, B3, B4, R1(B3), B5`. In this case, rollback block R1 is invalidating the B3 which is not the previous block.
In pass 1 (lines 272-354), each rollback command block is resolved via the `TARGET_INSTANT_TIME` header and the targeted instant's blocks are removed from `instantToBlocksMap` regardless of position (lines 334-345). Pass 2 (lines 370-411) then iterates instants in reverse order to correctly position compacted blocks.
The two-pass logic was introduced in commit `86a1efbff1` (HUDI-3900, PR #5958). Closing as resolved seems appropriate — please reopen if you can reproduce on a recent release.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Support out of order rollback blocks in AbstractHoodieLogRecordReader [hudi]
Posted by "nsivabalan (via GitHub)" <gi...@apache.org>.
nsivabalan closed issue #14437: Support out of order rollback blocks in AbstractHoodieLogRecordReader
URL: https://github.com/apache/hudi/issues/14437
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Support out of order rollback blocks in AbstractHoodieLogRecordReader [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on issue #14437:
URL: https://github.com/apache/hudi/issues/14437#issuecomment-3630145135
### Linked PR(s)
- https://github.com/apache/hudi/pull/5341
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org