You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@hudi.apache.org by "hudi-bot (via GitHub)" <gi...@apache.org> on 2025/11/30 06:15:22 UTC

[I] Apply rollback commits from data table as rollbacks in MDT instead of Delta commit [hudi]

hudi-bot opened a new issue, #15653:
URL: https://github.com/apache/hudi/issues/15653

   As of now, any rollback in DT is another DC in MDT. this may not scale for record level index in MDT since we have to add 1000s of delete records and finally have to resolve all valid and invalid records. So, its better to rollback the commit in MDT as well instead of doing a DC. 
   
    
   
   Impact: 
   
   record level index is unusable w/o this change. While fixing other rollback related tickets, do consider this as a possible option if this simplifies other fixes. 
   
   ## JIRA info
   
   - Link: https://issues.apache.org/jira/browse/HUDI-5463
   - Type: Bug
   - Epic: https://issues.apache.org/jira/browse/HUDI-1292
   - Fix version(s):
     - 1.1.0
   
   
   ---
   
   
   ## Comments
   
   14/Mar/23 02:11;danny0405;The failed instant in DT may never written into the MDT yet, so what's the expected behavior of the rollback then?;;;


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] Apply rollback commits from data table as rollbacks in MDT instead of Delta commit [hudi]

Posted by "deepakpanda93 (via GitHub)" <gi...@apache.org>.

deepakpanda93 closed issue #15653: Apply rollback commits from data table as rollbacks in MDT instead of Delta commit
URL: https://github.com/apache/hudi/issues/15653


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] Apply rollback commits from data table as rollbacks in MDT instead of Delta commit [hudi]

Posted by "deepakpanda93 (via GitHub)" <gi...@apache.org>.

deepakpanda93 commented on issue #15653:
URL: https://github.com/apache/hudi/issues/15653#issuecomment-4891403256

This is implemented on current 1.x. Data-table rollbacks are no longer applied to the metadata table as delta commits that append large numbers of delete records. Instead (HoodieTableMetadataUtil.convertMetadataToRecords for rollback metadata):

- We only need to handle the FILES partition here (a MOR rollback may add a new log file that must be recorded). All other partitions — including RECORD_INDEX (RLI), column_stats, etc. — are handled by an actual rollback of the deltacommit that added those records.

- So RLI rollbacks are done by rolling back the MDT deltacommit rather than resolving 1000s of delete records, which is exactly what this ticket requested and why RLI scales in current releases. The mechanism continues to be hardened (e.g. #18160 post-commit rollback of MDT deltacommits in 1.2.0, #18033, #18279).

Regarding the earlier question about a failed DT instant that was never written to the MDT: there's simply no MDT deltacommit to roll back in that case, and the FILES-partition record path covers any log file added by the rollback.

Closing as resolved on current 1.x.

--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] Apply rollback commits from data table as rollbacks in MDT instead of Delta commit [hudi]

Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.

hudi-bot commented on issue #15653:
URL: https://github.com/apache/hudi/issues/15653#issuecomment-3593840663

   This issue belongs to a closed JIRA Epic: https://issues.apache.org/jira/browse/HUDI-1292


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org