You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "hudi-bot (via GitHub)" <gi...@apache.org> on 2025/11/30 04:30:55 UTC
[I] Ability to clean up dangling data files using hudi-cli [hudi]
hudi-bot opened a new issue, #14889:
URL: https://github.com/apache/hudi/issues/14889
See https://github.com/apache/hudi/issues/3739
Scenario: commits archived but data files not cleaned up because cleaning frequency is lesser than tha of archival.
## JIRA info
- Link: https://issues.apache.org/jira/browse/HUDI-2580
- Type: Task
- Epic: https://issues.apache.org/jira/browse/HUDI-1388
---
## Comments
13/Dec/21 20:29;shivnarayan;[~guoyihua] : Can you add any follow up items here. From what I know, we had some tests to be added for the tool.
;;;
---
14/Jan/22 00:32;guoyihua;HoodieRepairTool is a stand-alone Spark app now and it's not hooked up with hudi-cli. I'll follow up on this.;;;
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Ability to clean up dangling data files using hudi-cli [hudi]
Posted by "rangareddy (via GitHub)" <gi...@apache.org>.
rangareddy closed issue #14889: Ability to clean up dangling data files using hudi-cli
URL: https://github.com/apache/hudi/issues/14889
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Ability to clean up dangling data files using hudi-cli [hudi]
Posted by "rangareddy (via GitHub)" <gi...@apache.org>.
rangareddy commented on issue #14889:
URL: https://github.com/apache/hudi/issues/14889#issuecomment-3664318714
We have implemented the [HoodieRepairTool](https://github.com/apache/hudi/blob/master/hudi-utilities/src/main/java/org/apache/hudi/utilities/HoodieRepairTool.java), a Spark-based utility designed to repair Hudi tables by identifying and removing dangling base and log files.
**Example Command:**
```sh
spark-submit \
--class org.apache.hudi.utilities.HoodieRepairTool \
--driver-memory 4g \
--executor-memory 1g \
--conf spark.serializer=org.apache.spark.serializer.KryoSerializer \
--conf spark.sql.catalogImplementation=hive \
--conf spark.sql.extensions=org.apache.spark.sql.hudi.HoodieSparkSessionExtension \
$HUDI_DIR/packaging/hudi-utilities-bundle/target/hudi-utilities-bundle_2.12-0.11.0-SNAPSHOT.jar \
--mode repair \
--base-path base_path \
--backup-path backup_path \
--start-instant-time ts1 \
--end-instant-time ts2 \
--assume-date-partitioning
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Ability to clean up dangling data files using hudi-cli [hudi]
Posted by "rangareddy (via GitHub)" <gi...@apache.org>.
rangareddy commented on issue #14889:
URL: https://github.com/apache/hudi/issues/14889#issuecomment-3664325911
We have implemented an alternative process for managing orphaned data files; therefore, this issue is being closed as resolved.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org