You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "vamshikrishnakyatham (via GitHub)" <gi...@apache.org> on 2025/09/29 03:22:57 UTC
[I] HoodieRollbackException During Clean Operation [hudi]
vamshikrishnakyatham opened a new issue, #14007:
URL: https://github.com/apache/hudi/issues/14007
### Bug Description
**What happened:**
Clean operation fails when attempting to rollback commits that appear chronologically before a compaction in the timeline but were not actually included in the compaction scope. The validation logic incorrectly blocks rollback of these instants (commits requested before compaction but completed after compaction).
**What you expected:**
Clean operation should successfully rollback commits that were not actually compacted, regardless of their position in the timeline relative to compaction timestamps.
**Steps to reproduce:**
1. Perform multiple delta commits on a Hudi table
2. Trigger a compaction while some commits are still pending
3. Run a clean operation that attempts to rollback one of these commits because of timeout
We observe HoodieRollbackException due to incorrect timeline based validation
### Environment
**Hudi version:** 1.1.0
**Query engine:** (Spark/Flink/Trino etc)
**Relevant configs:**
### Logs and Stack Trace
org.apache.hudi.exception.HoodieRollbackException: Failed to rollback s3a://xyz 20250925012123905
at org.apache.hudi.client.BaseHoodieTableServiceClient.rollback(BaseHoodieTableServiceClient.java:1231)
at org.apache.hudi.client.BaseHoodieTableServiceClient.rollback(BaseHoodieTableServiceClient.java:1165)
at org.apache.hudi.client.BaseHoodieTableServiceClient.rollbackFailedWrites(BaseHoodieTableServiceClient.java:1088)
at org.apache.hudi.client.BaseHoodieTableServiceClient.rollbackFailedWrites(BaseHoodieTableServiceClient.java:1070)
at org.apache.hudi.client.BaseHoodieTableServiceClient.rollbackFailedWrites(BaseHoodieTableServiceClient.java:1065)
at org.apache.hudi.client.BaseHoodieTableServiceClient.lambda$clean$cbd664b5$1(BaseHoodieTableServiceClient.java:846)
at org.apache.hudi.common.util.CleanerUtils.rollbackFailedWrites(CleanerUtils.java:217)
at org.apache.hudi.client.BaseHoodieTableServiceClient.clean(BaseHoodieTableServiceClient.java:845)
at org.apache.hudi.client.BaseHoodieWriteClient.clean(BaseHoodieWriteClient.java:930)
at com.onehouse.hudi.table.standaloneservices.CleanTask.runTask(CleanTask.java:33)
at com.onehouse.hudi.table.standaloneservices.TableServiceTaskRunner.executeSingleTask(TableServiceTaskRunner.java:130)
at com.onehouse.hudi.table.standaloneservices.TableServiceTaskRunner.executeTasks(TableServiceTaskRunner.java:123)
at com.onehouse.hudi.table.standaloneservices.TableServiceTaskRunner.run(TableServiceTaskRunner.java:92)
at com.onehouse.util.TaskFuture.lambda$new$0(TaskFuture.java:33)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: org.apache.hudi.exception.HoodieMetadataException: Commit being rolled back 20250925012123905 is earlier than the latest compaction 20250925012523368. There are 8 deltacommits after this compaction: [[20250925012447357__20250925013432341__deltacommit__COMPLETED], [20250925012518125__20250925012831379__deltacommit__COMPLETED], [20250925012851950__20250925013157886__deltacommit__COMPLETED], [20250925013447468__20250925014634434__deltacommit__COMPLETED], [20250925013748434__20250925014142539__deltacommit__COMPLETED], [20250925014147869__20250925014458106__deltacommit__COMPLETED], [20250925014647316__20250925015522155__deltacommit__COMPLETED], [20250925015718106__20250925020108423__deltacommit__COMPLETED]]
at org.apache.hudi.metadata.HoodieBackedTableMetadataWriter.validateRollback(HoodieBackedTableMetadataWriter.java:1665)
at org.apache.hudi.metadata.HoodieBackedTableMetadataWriter.update(HoodieBackedTableMetadataWriter.java:1635)
at org.apache.hudi.table.action.BaseActionExecutor.writeTableMetadata(BaseActionExecutor.java:114)
at org.apache.hudi.table.action.rollback.BaseRollbackActionExecutor.finishRollback(BaseRollbackActionExecutor.java:279)
at org.apache.hudi.table.action.rollback.BaseRollbackActionExecutor.runRollback(BaseRollbackActionExecutor.java:125)
at org.apache.hudi.table.action.rollback.BaseRollbackActionExecutor.execute(BaseRollbackActionExecutor.java:153)
at org.apache.hudi.table.HoodieSparkMergeOnReadTable.rollback(HoodieSparkMergeOnReadTable.java:218)
at org.apache.hudi.client.BaseHoodieTableServiceClient.rollback(BaseHoodieTableServiceClient.java:1218)
... 17 more
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] HoodieRollbackException During Clean Operation - MDT [hudi]
Posted by "danny0405 (via GitHub)" <gi...@apache.org>.
danny0405 closed issue #14007: HoodieRollbackException During Clean Operation - MDT
URL: https://github.com/apache/hudi/issues/14007
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org