You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "hudi-bot (via GitHub)" <gi...@apache.org> on 2025/11/30 05:46:11 UTC
[I] PartitionsForFullCleaning in CleanPlanner is using FileSystemBasedListing [hudi]
hudi-bot opened a new issue, #15382:
URL: https://github.com/apache/hudi/issues/15382
[https://github.com/apache/hudi/pull/5169/files#diff-f3f9696ca040bd9f581a1122b1c5cf0409b3ba4525672fbf3328e9f2c40eec66]
Looks like we are doing file system based full listing when partitionsForFullCleaning is being invoked. When the patch was landed, DELETE_PARTITION was synchronous and hence the fix. but later we fixed DELETE_PARTITION to be lazy. and so, we can use metadata table if enabled.
## JIRA info
- Link: https://issues.apache.org/jira/browse/HUDI-4749
- Type: Bug
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] PartitionsForFullCleaning in CleanPlanner is using FileSystemBasedListing [hudi]
Posted by "nsivabalan (via GitHub)" <gi...@apache.org>.
nsivabalan closed issue #15382: PartitionsForFullCleaning in CleanPlanner is using FileSystemBasedListing
URL: https://github.com/apache/hudi/issues/15382
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] PartitionsForFullCleaning in CleanPlanner is using FileSystemBasedListing [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on issue #15382:
URL: https://github.com/apache/hudi/issues/15382#issuecomment-3630144791
### Linked PR(s)
- https://github.com/apache/hudi/pull/6548
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] PartitionsForFullCleaning in CleanPlanner is using FileSystemBasedListing [hudi]
Posted by "nsivabalan (via GitHub)" <gi...@apache.org>.
nsivabalan commented on issue #15382:
URL: https://github.com/apache/hudi/issues/15382#issuecomment-4795358681
This appears to be already fixed on master.
[HUDI-6039 / PR #8384](https://github.com/apache/hudi/pull/8384) (commit `16cb9c45d6010846749ee7963d48811aad0cf327`, 2023-04-21) removed the direct `FileSystemBackedTableMetadata` instantiation in `CleanPlanner.getPartitionPathsForFullCleaning` that was added by PR #5169.
Current code at `hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/action/clean/CleanPlanner.java:302`:
```java
allPartitionPaths = hoodieTable.getTableMetadata().getAllPartitionPaths();
```
`HoodieTable.getTableMetadata()` returns the metadata-table-backed implementation when MDT is enabled (see `HoodieTable.java:1234-1246`), so full-clean partition listing now goes through MDT rather than always falling back to FS-based listing.
Suggest closing this issue and the linked HUDI-4749 ticket as resolved unless there's a separate scenario still using FS listing that I'm missing.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org