You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "hudi-bot (via GitHub)" <gi...@apache.org> on 2025/11/30 08:04:12 UTC
[I] Relax the restriction for Spark MDT rollback [hudi]
hudi-bot opened a new issue, #16013:
URL: https://github.com/apache/hudi/issues/16013
## JIRA info
- Link: https://issues.apache.org/jira/browse/HUDI-6645
- Type: Bug
- Fix version(s):
- 1.1.0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Relax the restriction for Spark MDT rollback [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on issue #16013:
URL: https://github.com/apache/hudi/issues/16013#issuecomment-3630173386
### Linked PR(s)
- https://github.com/apache/hudi/pull/9364
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Relax the restriction for Spark MDT rollback [hudi]
Posted by "nsivabalan (via GitHub)" <gi...@apache.org>.
nsivabalan closed issue #16013: Relax the restriction for Spark MDT rollback
URL: https://github.com/apache/hudi/issues/16013
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Relax the restriction for Spark MDT rollback [hudi]
Posted by "nsivabalan (via GitHub)" <gi...@apache.org>.
nsivabalan commented on issue #16013:
URL: https://github.com/apache/hudi/issues/16013#issuecomment-4795516766
This appears to be addressed on current `master`. The original PR #9364 was closed unmerged, but the same goal — relaxing the Spark MDT rollback restriction — was achieved through two later commits:
1. **[HUDI-7510] (#10874, commit `9b094e628d6e`)** — Loosened the base `validateRollback` check in `HoodieBackedTableMetadataWriter` and removed the Flink-only override of `validateRollback`, so Spark and Flink now share the same relaxed logic.
2. **PR #14008 (commit `d290b8124a48`)** — Further relaxed by skipping validation entirely for pending (non-completed) MDT deltacommits and switching the comparison to use the rolled-back instant's *completion time* against the compaction's requested time (with a length guard).
Current code: `hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadataWriter.java:1851`. Neither `SparkHoodieBackedTableMetadataWriter` nor `FlinkHoodieBackedTableMetadataWriter` override `validateRollback` anymore.
If the specific Spark scenario you had in mind is still hitting this exception, please share a stack trace / repro and we can reopen — otherwise this can likely be closed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org