You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "nsivabalan (via GitHub)" <gi...@apache.org> on 2026/04/18 02:05:28 UTC
[PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
nsivabalan opened a new pull request, #18526:
URL: https://github.com/apache/hudi/pull/18526
### Describe the issue this Pull Request addresses
Closes #17866
Users need the ability to back up certain partitions to a separate location and remove them from the original Hudi table, with the option to restore them later. This is driven by use cases like partition TTL where stashed data needs to be retained for a grace period before permanent deletion.
### Summary and Changelog
Adds a new spark-submit tool HoodieStashPartitionsTool that moves partition data to a stash location and issues a deletePartitions operation. The tool uses a custom SparkPreCommitValidator to atomically move files before the replace commit completes.
Stash operation:
1. Pre-check: if files exist in the stash location from a prior failed attempt, move them back to source (ensures deletePartitions sees complete partition state)
2. Call deletePartitions with StashPartitionsPreCommitValidator configured — the validator moves files to the stash location before commit
3. On commit success, the replace commit marks file groups as replaced; partition data is already at the stash location
Restore: Users use the standard insert_overwrite write operation reading from the stash location (no new tooling).
####Changelog:
- StashPartitionRenameHelper — pluggable interface for moving partition files between source and stash locations
- DefaultStashPartitionRenameHelper — default implementation using copy + delete (custom implementations can leverage efficient FS-level
rename APIs)
- StashPartitionsPreCommitValidator — special-purpose pre-commit validator that moves files to stash before deletePartitions commit.
Overrides validate() to extract affected partitions from partitionToReplaceFileIds. Guards against RLI-enabled tables.
- HoodieStashPartitionsTool — spark-submit tool with three modes: stash, rollback_stash, dry_run
- New configs: hoodie.stash.partitions.target.path, hoodie.stash.rename.helper.class
### Impact
- No changes to public APIs or storage format. The tool is a standalone utility in hudi-utilities.
- Pre-commit validators are used in an unconventional way (performing side-effects rather than read-only checks). This is explicitly
documented as a special-purpose pattern not to be followed.
- Tables with Record Level Index (RLI) enabled are not supported — the validator fails fast with a clear error message.
- The StashPartitionRenameHelper interface allows custom implementations for environments with efficient rename APIs (e.g., HDFS atomic
rename).
### Risk Level
Low.
- All new code — no modifications to existing Hudi write paths, timeline operations, or metadata table logic.
- The tool delegates to the existing deletePartitions API for all timeline/MDT operations.
- The pre-commit validator is only activated when explicitly configured by the tool; it does not affect any existing write operations.
- Failure scenarios are handled: the tool's pre-check recovers partial stash state on retry, and the validator is idempotent (skips already-stashed partitions).
### Documentation Update
- New tool usage documentation needed for HoodieStashPartitionsTool (spark-submit command, CLI arguments, modes).
- New config keys to document:
- hoodie.stash.partitions.target.path — target stash directory (required when using the stash validator)
- hoodie.stash.rename.helper.class — custom rename helper class (optional, defaults to DefaultStashPartitionRenameHelper)
### Contributor's checklist
- [ ] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute)
- [ ] Enough context is provided in the sections above
- [ ] Adequate tests were added if applicable
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-4272475965
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `0%` with `223 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 39.71%. Comparing base ([`7bcb8be`](https://app.codecov.io/gh/apache/hudi/commit/7bcb8be7c153010998574f591c7ff3799192eda6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`e8b5a0a`](https://app.codecov.io/gh/apache/hudi/commit/e8b5a0a62cea99be0a184e0b6f11149b49850806?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 3 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | 0.00% | [156 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | 0.00% | [40 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | 0.00% | [27 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
> :exclamation: There is a different number of reports uploaded between BASE (7bcb8be) and HEAD (e8b5a0a). Click for more details.
>
> <details><summary>HEAD has 27 uploads less than BASE</summary>
>
>| Flag | BASE (7bcb8be) | HEAD (e8b5a0a) |
>|------|------|------|
>|spark-client-hadoop-common|1|0|
>|spark-scala-tests|10|1|
>|spark-java-tests|15|0|
>|common-and-other-modules|1|0|
>|utilities|1|0|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18526 +/- ##
=============================================
- Coverage 68.88% 39.71% -29.18%
+ Complexity 28279 12411 -15868
=============================================
Files 2464 1947 -517
Lines 135568 105315 -30253
Branches 16446 12685 -3761
=============================================
- Hits 93385 41825 -51560
- Misses 34801 58524 +23723
+ Partials 7382 4966 -2416
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.70% <0.00%> (-0.06%)` | :arrow_down: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `33.01% <0.00%> (-12.43%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | `0.00% <0.00%> (ø)` | |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | `0.00% <0.00%> (ø)` | |
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | `0.00% <0.00%> (ø)` | |
... and [1847 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18526/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-4272567771
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `78.47534%` with `48 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 68.89%. Comparing base ([`7bcb8be`](https://app.codecov.io/gh/apache/hudi/commit/7bcb8be7c153010998574f591c7ff3799192eda6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`e8b5a0a`](https://app.codecov.io/gh/apache/hudi/commit/e8b5a0a62cea99be0a184e0b6f11149b49850806?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 3 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | 76.92% | [28 Missing and 8 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | 75.00% | [6 Missing and 4 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | 92.59% | [1 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18526 +/- ##
============================================
+ Coverage 68.88% 68.89% +0.01%
- Complexity 28279 28317 +38
============================================
Files 2464 2467 +3
Lines 135568 135817 +249
Branches 16446 16468 +22
============================================
+ Hits 93385 93574 +189
- Misses 34801 34846 +45
- Partials 7382 7397 +15
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.59% <11.21%> (-0.05%)` | :arrow_down: |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.70% <0.00%> (-0.06%)` | :arrow_down: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.46% <77.61%> (+0.06%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.83% <0.00%> (-0.10%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.35% <0.00%> (-0.09%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `38.31% <74.88%> (+0.14%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | `92.59% <92.59%> (ø)` | |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | `75.00% <75.00%> (ø)` | |
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | `76.92% <76.92%> (ø)` | |
... and [10 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18526/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-4272501340
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `24.66368%` with `168 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 58.88%. Comparing base ([`7bcb8be`](https://app.codecov.io/gh/apache/hudi/commit/7bcb8be7c153010998574f591c7ff3799192eda6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`e8b5a0a`](https://app.codecov.io/gh/apache/hudi/commit/e8b5a0a62cea99be0a184e0b6f11149b49850806?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 3 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | 0.00% | [156 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | 75.00% | [6 Missing and 4 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | 92.59% | [1 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
> :exclamation: There is a different number of reports uploaded between BASE (7bcb8be) and HEAD (e8b5a0a). Click for more details.
>
> <details><summary>HEAD has 16 uploads less than BASE</summary>
>
>| Flag | BASE (7bcb8be) | HEAD (e8b5a0a) |
>|------|------|------|
>|spark-scala-tests|10|5|
>|spark-java-tests|15|5|
>|utilities|1|0|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18526 +/- ##
=============================================
- Coverage 68.88% 58.88% -10.00%
+ Complexity 28279 24216 -4063
=============================================
Files 2464 2467 +3
Lines 135568 135817 +249
Branches 16446 16468 +22
=============================================
- Hits 93385 79976 -13409
- Misses 34801 48998 +14197
+ Partials 7382 6843 -539
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.59% <11.21%> (-0.05%)` | :arrow_down: |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.70% <0.00%> (-0.06%)` | :arrow_down: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.46% <77.61%> (+0.06%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `32.29% <0.00%> (-16.64%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `32.48% <0.00%> (-12.96%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | `92.59% <92.59%> (ø)` | |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | `75.00% <75.00%> (ø)` | |
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | `0.00% <0.00%> (ø)` | |
... and [660 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18526/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-4285810604
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `78.47534%` with `48 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 68.79%. Comparing base ([`7bcb8be`](https://app.codecov.io/gh/apache/hudi/commit/7bcb8be7c153010998574f591c7ff3799192eda6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`891eaf2`](https://app.codecov.io/gh/apache/hudi/commit/891eaf2e1427bf93b9c7ff9626628c30e1c9c01d?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 15 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | 76.92% | [28 Missing and 8 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | 75.00% | [6 Missing and 4 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | 92.59% | [1 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18526 +/- ##
============================================
- Coverage 68.88% 68.79% -0.10%
- Complexity 28279 28361 +82
============================================
Files 2464 2471 +7
Lines 135568 136218 +650
Branches 16446 16521 +75
============================================
+ Hits 93385 93707 +322
- Misses 34801 35095 +294
- Partials 7382 7416 +34
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.56% <11.21%> (-0.08%)` | :arrow_down: |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.75% <0.00%> (+<0.01%)` | :arrow_up: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.47% <77.61%> (+0.07%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.83% <0.00%> (-0.09%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.34% <0.00%> (-0.10%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `38.30% <74.88%> (+0.13%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | `92.59% <92.59%> (ø)` | |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | `75.00% <75.00%> (ø)` | |
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | `76.92% <76.92%> (ø)` | |
... and [25 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18526/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-4284481883
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `78.47534%` with `48 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 65.92%. Comparing base ([`7bcb8be`](https://app.codecov.io/gh/apache/hudi/commit/7bcb8be7c153010998574f591c7ff3799192eda6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`891eaf2`](https://app.codecov.io/gh/apache/hudi/commit/891eaf2e1427bf93b9c7ff9626628c30e1c9c01d?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 11 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | 76.92% | [28 Missing and 8 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | 75.00% | [6 Missing and 4 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | 92.59% | [1 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18526 +/- ##
============================================
- Coverage 68.88% 65.92% -2.96%
+ Complexity 28279 27123 -1156
============================================
Files 2464 2471 +7
Lines 135568 136218 +650
Branches 16446 16521 +75
============================================
- Hits 93385 89808 -3577
- Misses 34801 38926 +4125
- Partials 7382 7484 +102
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.56% <11.21%> (-0.08%)` | :arrow_down: |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.75% <0.00%> (+<0.01%)` | :arrow_up: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.47% <77.61%> (+0.07%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `32.28% <0.00%> (-16.64%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.81% <0.00%> (-0.64%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `38.30% <74.88%> (+0.13%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | `92.59% <92.59%> (ø)` | |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | `75.00% <75.00%> (ø)` | |
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | `76.92% <76.92%> (ø)` | |
... and [388 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18526/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-4284509411
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `78.47534%` with `48 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 67.69%. Comparing base ([`7bcb8be`](https://app.codecov.io/gh/apache/hudi/commit/7bcb8be7c153010998574f591c7ff3799192eda6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`891eaf2`](https://app.codecov.io/gh/apache/hudi/commit/891eaf2e1427bf93b9c7ff9626628c30e1c9c01d?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 11 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | 76.92% | [28 Missing and 8 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | 75.00% | [6 Missing and 4 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | 92.59% | [1 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18526 +/- ##
============================================
- Coverage 68.88% 67.69% -1.20%
+ Complexity 28279 27880 -399
============================================
Files 2464 2471 +7
Lines 135568 136218 +650
Branches 16446 16521 +75
============================================
- Hits 93385 92206 -1179
- Misses 34801 36547 +1746
- Partials 7382 7465 +83
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.56% <11.21%> (-0.08%)` | :arrow_down: |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.75% <0.00%> (+<0.01%)` | :arrow_up: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.47% <77.61%> (+0.07%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `42.32% <0.00%> (-6.60%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.19% <0.00%> (-0.25%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `38.30% <74.88%> (+0.13%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | `92.59% <92.59%> (ø)` | |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | `75.00% <75.00%> (ø)` | |
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | `76.92% <76.92%> (ø)` | |
... and [208 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18526/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-4300468942
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "67f1e57f0dffe02bbe2d7bacfaa8f1aaa4be8c88",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=13351",
"triggerID" : "67f1e57f0dffe02bbe2d7bacfaa8f1aaa4be8c88",
"triggerType" : "PUSH"
}, {
"hash" : "e8b5a0a62cea99be0a184e0b6f11149b49850806",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=13354",
"triggerID" : "e8b5a0a62cea99be0a184e0b6f11149b49850806",
"triggerType" : "PUSH"
}, {
"hash" : "891eaf2e1427bf93b9c7ff9626628c30e1c9c01d",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=13391",
"triggerID" : "891eaf2e1427bf93b9c7ff9626628c30e1c9c01d",
"triggerType" : "PUSH"
}, {
"hash" : "6d8dedd061ed183c52471df09355b85e5a60099a",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=13453",
"triggerID" : "6d8dedd061ed183c52471df09355b85e5a60099a",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 891eaf2e1427bf93b9c7ff9626628c30e1c9c01d Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=13391)
* 6d8dedd061ed183c52471df09355b85e5a60099a Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=13453)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-5235342042
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `80.83832%` with `64 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 68.68%. Comparing base ([`7bcb8be`](https://app.codecov.io/gh/apache/hudi/commit/7bcb8be7c153010998574f591c7ff3799192eda6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`6d8dedd`](https://app.codecov.io/gh/apache/hudi/commit/6d8dedd061ed183c52471df09355b85e5a60099a?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 476 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | 79.84% | [43 Missing and 10 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | 77.50% | [5 Missing and 4 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | 93.54% | [1 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18526 +/- ##
============================================
- Coverage 68.88% 68.68% -0.21%
- Complexity 28279 28473 +194
============================================
Files 2464 2478 +14
Lines 135568 136972 +1404
Branches 16446 16653 +207
============================================
+ Hits 93385 94077 +692
- Misses 34801 35324 +523
- Partials 7382 7571 +189
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.39% <13.77%> (-0.25%)` | :arrow_down: |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.74% <0.00%> (-0.02%)` | :arrow_down: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.64% <77.46%> (+0.23%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `49.25% <0.00%> (+0.32%)` | :arrow_up: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.18% <0.00%> (-0.26%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `37.87% <77.54%> (-0.30%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | `93.54% <93.54%> (ø)` | |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | `77.50% <77.50%> (ø)` | |
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | `79.84% <79.84%> (ø)` | |
... and [79 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18526/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
- :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "codecov-commenter (via GitHub) via commits" <co...@hudi.apache.org>.
codecov-commenter commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-6560824345
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `80.83832%` with `64 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 68.68%. Comparing base ([`7bcb8be`](https://app.codecov.io/gh/apache/hudi/commit/7bcb8be7c153010998574f591c7ff3799192eda6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`6d8dedd`](https://app.codecov.io/gh/apache/hudi/commit/6d8dedd061ed183c52471df09355b85e5a60099a?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 820 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | 79.84% | [43 Missing and 10 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | 77.50% | [5 Missing and 4 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | 93.54% | [1 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18526 +/- ##
============================================
- Coverage 68.88% 68.68% -0.21%
- Complexity 28279 28473 +194
============================================
Files 2464 2478 +14
Lines 135568 136972 +1404
Branches 16446 16653 +207
============================================
+ Hits 93385 94077 +692
- Misses 34801 35324 +523
- Partials 7382 7571 +189
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.39% <13.77%> (-0.25%)` | :arrow_down: |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.74% <0.00%> (-0.02%)` | :arrow_down: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.64% <77.46%> (+0.23%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `49.25% <0.00%> (+0.32%)` | :arrow_up: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.18% <0.00%> (-0.26%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `37.87% <77.54%> (-0.30%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | `93.54% <93.54%> (ø)` | |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | `77.50% <77.50%> (ø)` | |
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | `79.84% <79.84%> (ø)` | |
... and [79 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18526/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
- :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-5195150402
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `80.83832%` with `64 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 68.68%. Comparing base ([`7bcb8be`](https://app.codecov.io/gh/apache/hudi/commit/7bcb8be7c153010998574f591c7ff3799192eda6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`6d8dedd`](https://app.codecov.io/gh/apache/hudi/commit/6d8dedd061ed183c52471df09355b85e5a60099a?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 461 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | 79.84% | [43 Missing and 10 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | 77.50% | [5 Missing and 4 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | 93.54% | [1 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18526 +/- ##
============================================
- Coverage 68.88% 68.68% -0.21%
- Complexity 28279 28473 +194
============================================
Files 2464 2478 +14
Lines 135568 136972 +1404
Branches 16446 16653 +207
============================================
+ Hits 93385 94077 +692
- Misses 34801 35324 +523
- Partials 7382 7571 +189
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.39% <13.77%> (-0.25%)` | :arrow_down: |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.74% <0.00%> (-0.02%)` | :arrow_down: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.64% <77.46%> (+0.23%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `49.25% <0.00%> (+0.32%)` | :arrow_up: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.18% <0.00%> (-0.26%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `37.87% <77.54%> (-0.30%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | `93.54% <93.54%> (ø)` | |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | `77.50% <77.50%> (ø)` | |
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | `79.84% <79.84%> (ø)` | |
... and [79 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18526/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
- :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-6394509971
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `80.83832%` with `64 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 68.68%. Comparing base ([`7bcb8be`](https://app.codecov.io/gh/apache/hudi/commit/7bcb8be7c153010998574f591c7ff3799192eda6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`6d8dedd`](https://app.codecov.io/gh/apache/hudi/commit/6d8dedd061ed183c52471df09355b85e5a60099a?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 802 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | 79.84% | [43 Missing and 10 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | 77.50% | [5 Missing and 4 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | 93.54% | [1 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18526 +/- ##
============================================
- Coverage 68.88% 68.68% -0.21%
- Complexity 28279 28473 +194
============================================
Files 2464 2478 +14
Lines 135568 136972 +1404
Branches 16446 16653 +207
============================================
+ Hits 93385 94077 +692
- Misses 34801 35324 +523
- Partials 7382 7571 +189
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.39% <13.77%> (-0.25%)` | :arrow_down: |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.74% <0.00%> (-0.02%)` | :arrow_down: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.64% <77.46%> (+0.23%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `49.25% <0.00%> (+0.32%)` | :arrow_up: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.18% <0.00%> (-0.26%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `37.87% <77.54%> (-0.30%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | `93.54% <93.54%> (ø)` | |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | `77.50% <77.50%> (ø)` | |
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | `79.84% <79.84%> (ø)` | |
... and [79 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18526/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
- :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-5279943432
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `80.83832%` with `64 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 68.68%. Comparing base ([`7bcb8be`](https://app.codecov.io/gh/apache/hudi/commit/7bcb8be7c153010998574f591c7ff3799192eda6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`6d8dedd`](https://app.codecov.io/gh/apache/hudi/commit/6d8dedd061ed183c52471df09355b85e5a60099a?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 488 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | 79.84% | [43 Missing and 10 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | 77.50% | [5 Missing and 4 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | 93.54% | [1 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18526 +/- ##
============================================
- Coverage 68.88% 68.68% -0.21%
- Complexity 28279 28473 +194
============================================
Files 2464 2478 +14
Lines 135568 136972 +1404
Branches 16446 16653 +207
============================================
+ Hits 93385 94077 +692
- Misses 34801 35324 +523
- Partials 7382 7571 +189
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.39% <13.77%> (-0.25%)` | :arrow_down: |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.74% <0.00%> (-0.02%)` | :arrow_down: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.64% <77.46%> (+0.23%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `49.25% <0.00%> (+0.32%)` | :arrow_up: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.18% <0.00%> (-0.26%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `37.87% <77.54%> (-0.30%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | `93.54% <93.54%> (ø)` | |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | `77.50% <77.50%> (ø)` | |
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | `79.84% <79.84%> (ø)` | |
... and [79 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18526/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
- :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-4300598353
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `16.46707%` with `279 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 45.73%. Comparing base ([`7bcb8be`](https://app.codecov.io/gh/apache/hudi/commit/7bcb8be7c153010998574f591c7ff3799192eda6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`6d8dedd`](https://app.codecov.io/gh/apache/hudi/commit/6d8dedd061ed183c52471df09355b85e5a60099a?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 20 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | 0.00% | [263 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | 77.50% | [5 Missing and 4 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | 77.41% | [5 Missing and 2 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
> :exclamation: There is a different number of reports uploaded between BASE (7bcb8be) and HEAD (6d8dedd). Click for more details.
>
> <details><summary>HEAD has 26 uploads less than BASE</summary>
>
>| Flag | BASE (7bcb8be) | HEAD (6d8dedd) |
>|------|------|------|
>|spark-scala-tests|10|0|
>|spark-java-tests|15|1|
>|common-and-other-modules|1|0|
>|utilities|1|0|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18526 +/- ##
=============================================
- Coverage 68.88% 45.73% -23.15%
+ Complexity 28279 14860 -13419
=============================================
Files 2464 1954 -510
Lines 135568 106099 -29469
Branches 16446 12768 -3678
=============================================
- Hits 93385 48528 -44857
- Misses 34801 52593 +17792
+ Partials 7382 4978 -2404
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.74% <0.00%> (-0.02%)` | :arrow_down: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.64% <77.46%> (+0.23%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `33.21% <0.00%> (-15.72%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | `77.41% <77.41%> (ø)` | |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | `77.50% <77.50%> (ø)` | |
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | `0.00% <0.00%> (ø)` | |
... and [1664 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18526/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-4300676151
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `23.05389%` with `257 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 65.60%. Comparing base ([`7bcb8be`](https://app.codecov.io/gh/apache/hudi/commit/7bcb8be7c153010998574f591c7ff3799192eda6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`6d8dedd`](https://app.codecov.io/gh/apache/hudi/commit/6d8dedd061ed183c52471df09355b85e5a60099a?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 21 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | 6.46% | [245 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | 77.50% | [5 Missing and 4 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | 93.54% | [1 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18526 +/- ##
============================================
- Coverage 68.88% 65.60% -3.28%
+ Complexity 28279 27189 -1090
============================================
Files 2464 2478 +14
Lines 135568 136972 +1404
Branches 16446 16653 +207
============================================
- Hits 93385 89867 -3518
- Misses 34801 39716 +4915
- Partials 7382 7389 +7
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.39% <13.77%> (-0.25%)` | :arrow_down: |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.74% <0.00%> (-0.02%)` | :arrow_down: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.64% <77.46%> (+0.23%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `42.88% <0.00%> (-6.05%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.62% <0.00%> (-0.83%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | `93.54% <93.54%> (ø)` | |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | `77.50% <77.50%> (ø)` | |
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | `6.46% <6.46%> (ø)` | |
... and [344 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18526/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-4272483382
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `23.31839%` with `171 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 46.65%. Comparing base ([`7bcb8be`](https://app.codecov.io/gh/apache/hudi/commit/7bcb8be7c153010998574f591c7ff3799192eda6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`e8b5a0a`](https://app.codecov.io/gh/apache/hudi/commit/e8b5a0a62cea99be0a184e0b6f11149b49850806?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 3 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | 0.00% | [156 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | 75.00% | [6 Missing and 4 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | 81.48% | [3 Missing and 2 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
> :exclamation: There is a different number of reports uploaded between BASE (7bcb8be) and HEAD (e8b5a0a). Click for more details.
>
> <details><summary>HEAD has 26 uploads less than BASE</summary>
>
>| Flag | BASE (7bcb8be) | HEAD (e8b5a0a) |
>|------|------|------|
>|spark-scala-tests|10|1|
>|spark-java-tests|15|0|
>|common-and-other-modules|1|0|
>|utilities|1|0|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18526 +/- ##
=============================================
- Coverage 68.88% 46.65% -22.24%
+ Complexity 28279 15051 -13228
=============================================
Files 2464 1947 -517
Lines 135568 105335 -30233
Branches 16446 12703 -3743
=============================================
- Hits 93385 49141 -44244
- Misses 34801 51151 +16350
+ Partials 7382 5043 -2339
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.70% <0.00%> (-0.06%)` | :arrow_down: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.46% <77.61%> (+0.06%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `33.01% <0.00%> (-12.43%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | `81.48% <81.48%> (ø)` | |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | `75.00% <75.00%> (ø)` | |
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | `0.00% <0.00%> (ø)` | |
... and [1639 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18526/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-4272512788
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `78.47534%` with `48 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 66.02%. Comparing base ([`7bcb8be`](https://app.codecov.io/gh/apache/hudi/commit/7bcb8be7c153010998574f591c7ff3799192eda6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`e8b5a0a`](https://app.codecov.io/gh/apache/hudi/commit/e8b5a0a62cea99be0a184e0b6f11149b49850806?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 3 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | 76.92% | [28 Missing and 8 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | 75.00% | [6 Missing and 4 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | 92.59% | [1 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18526 +/- ##
============================================
- Coverage 68.88% 66.02% -2.87%
+ Complexity 28279 27078 -1201
============================================
Files 2464 2467 +3
Lines 135568 135817 +249
Branches 16446 16468 +22
============================================
- Hits 93385 89672 -3713
- Misses 34801 38679 +3878
- Partials 7382 7466 +84
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.59% <11.21%> (-0.05%)` | :arrow_down: |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.70% <0.00%> (-0.06%)` | :arrow_down: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.46% <77.61%> (+0.06%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `32.29% <0.00%> (-16.64%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.81% <0.00%> (-0.63%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `38.31% <74.88%> (+0.14%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | `92.59% <92.59%> (ø)` | |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | `75.00% <75.00%> (ø)` | |
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | `76.92% <76.92%> (ø)` | |
... and [374 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18526/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-4284433275
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `23.31839%` with `171 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 46.16%. Comparing base ([`7bcb8be`](https://app.codecov.io/gh/apache/hudi/commit/7bcb8be7c153010998574f591c7ff3799192eda6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`891eaf2`](https://app.codecov.io/gh/apache/hudi/commit/891eaf2e1427bf93b9c7ff9626628c30e1c9c01d?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 11 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | 0.00% | [156 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | 75.00% | [6 Missing and 4 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | 81.48% | [3 Missing and 2 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
> :exclamation: There is a different number of reports uploaded between BASE (7bcb8be) and HEAD (891eaf2). Click for more details.
>
> <details><summary>HEAD has 25 uploads less than BASE</summary>
>
>| Flag | BASE (7bcb8be) | HEAD (891eaf2) |
>|------|------|------|
>|spark-scala-tests|10|2|
>|spark-java-tests|15|0|
>|common-and-other-modules|1|0|
>|utilities|1|0|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18526 +/- ##
=============================================
- Coverage 68.88% 46.16% -22.73%
+ Complexity 28279 15213 -13066
=============================================
Files 2464 1965 -499
Lines 135568 107868 -27700
Branches 16446 13266 -3180
=============================================
- Hits 93385 49794 -43591
- Misses 34801 52909 +18108
+ Partials 7382 5165 -2217
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.75% <0.00%> (+<0.01%)` | :arrow_up: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.47% <77.61%> (+0.07%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `32.86% <0.00%> (-12.59%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | `81.48% <81.48%> (ø)` | |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | `75.00% <75.00%> (ø)` | |
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | `0.00% <0.00%> (ø)` | |
... and [1616 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18526/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-4300498651
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `0%` with `31 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 44.74%. Comparing base ([`7bcb8be`](https://app.codecov.io/gh/apache/hudi/commit/7bcb8be7c153010998574f591c7ff3799192eda6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`6d8dedd`](https://app.codecov.io/gh/apache/hudi/commit/6d8dedd061ed183c52471df09355b85e5a60099a?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 20 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | 0.00% | [31 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
> :exclamation: There is a different number of reports uploaded between BASE (7bcb8be) and HEAD (6d8dedd). Click for more details.
>
> <details><summary>HEAD has 28 uploads less than BASE</summary>
>
>| Flag | BASE (7bcb8be) | HEAD (6d8dedd) |
>|------|------|------|
>|spark-client-hadoop-common|1|0|
>|spark-scala-tests|10|0|
>|spark-java-tests|15|0|
>|common-and-other-modules|1|0|
>|utilities|1|0|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18526 +/- ##
=============================================
- Coverage 68.88% 44.74% -24.14%
+ Complexity 28279 8507 -19772
=============================================
Files 2464 1199 -1265
Lines 135568 62458 -73110
Branches 16446 6747 -9699
=============================================
- Hits 93385 27947 -65438
+ Misses 34801 31453 -3348
+ Partials 7382 3058 -4324
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.74% <0.00%> (-0.02%)` | :arrow_down: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | `0.00% <0.00%> (ø)` | |
... and [2066 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18526/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-6528392781
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `80.83832%` with `64 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 68.68%. Comparing base ([`7bcb8be`](https://app.codecov.io/gh/apache/hudi/commit/7bcb8be7c153010998574f591c7ff3799192eda6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`6d8dedd`](https://app.codecov.io/gh/apache/hudi/commit/6d8dedd061ed183c52471df09355b85e5a60099a?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 819 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | 79.84% | [43 Missing and 10 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | 77.50% | [5 Missing and 4 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | 93.54% | [1 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18526 +/- ##
============================================
- Coverage 68.88% 68.68% -0.21%
- Complexity 28279 28473 +194
============================================
Files 2464 2478 +14
Lines 135568 136972 +1404
Branches 16446 16653 +207
============================================
+ Hits 93385 94077 +692
- Misses 34801 35324 +523
- Partials 7382 7571 +189
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.39% <13.77%> (-0.25%)` | :arrow_down: |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.74% <0.00%> (-0.02%)` | :arrow_down: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.64% <77.46%> (+0.23%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `49.25% <0.00%> (+0.32%)` | :arrow_up: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.18% <0.00%> (-0.26%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `37.87% <77.54%> (-0.30%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | `93.54% <93.54%> (ø)` | |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | `77.50% <77.50%> (ø)` | |
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | `79.84% <79.84%> (ø)` | |
... and [79 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18526/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
- :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-6528375526
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `80.83832%` with `64 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 68.68%. Comparing base ([`7bcb8be`](https://app.codecov.io/gh/apache/hudi/commit/7bcb8be7c153010998574f591c7ff3799192eda6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`6d8dedd`](https://app.codecov.io/gh/apache/hudi/commit/6d8dedd061ed183c52471df09355b85e5a60099a?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 819 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | 79.84% | [43 Missing and 10 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | 77.50% | [5 Missing and 4 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | 93.54% | [1 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18526 +/- ##
============================================
- Coverage 68.88% 68.68% -0.21%
- Complexity 28279 28473 +194
============================================
Files 2464 2478 +14
Lines 135568 136972 +1404
Branches 16446 16653 +207
============================================
+ Hits 93385 94077 +692
- Misses 34801 35324 +523
- Partials 7382 7571 +189
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.39% <13.77%> (-0.25%)` | :arrow_down: |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.74% <0.00%> (-0.02%)` | :arrow_down: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.64% <77.46%> (+0.23%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `49.25% <0.00%> (+0.32%)` | :arrow_up: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.18% <0.00%> (-0.26%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `37.87% <77.54%> (-0.30%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | `93.54% <93.54%> (ø)` | |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | `77.50% <77.50%> (ø)` | |
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | `79.84% <79.84%> (ø)` | |
... and [79 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18526/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
- :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-4272558781
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `78.47534%` with `48 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 68.89%. Comparing base ([`7bcb8be`](https://app.codecov.io/gh/apache/hudi/commit/7bcb8be7c153010998574f591c7ff3799192eda6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`e8b5a0a`](https://app.codecov.io/gh/apache/hudi/commit/e8b5a0a62cea99be0a184e0b6f11149b49850806?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 3 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | 76.92% | [28 Missing and 8 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | 75.00% | [6 Missing and 4 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | 92.59% | [1 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18526 +/- ##
============================================
+ Coverage 68.88% 68.89% +0.01%
- Complexity 28279 28317 +38
============================================
Files 2464 2467 +3
Lines 135568 135817 +249
Branches 16446 16468 +22
============================================
+ Hits 93385 93571 +186
- Misses 34801 34848 +47
- Partials 7382 7398 +16
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.59% <11.21%> (-0.05%)` | :arrow_down: |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.70% <0.00%> (-0.06%)` | :arrow_down: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.46% <77.61%> (+0.06%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.79% <0.00%> (-0.14%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.35% <0.00%> (-0.09%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `38.31% <74.88%> (+0.14%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | `92.59% <92.59%> (ø)` | |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | `75.00% <75.00%> (ø)` | |
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | `76.92% <76.92%> (ø)` | |
... and [12 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18526/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-4300678550
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `23.05389%` with `257 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 65.73%. Comparing base ([`7bcb8be`](https://app.codecov.io/gh/apache/hudi/commit/7bcb8be7c153010998574f591c7ff3799192eda6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`6d8dedd`](https://app.codecov.io/gh/apache/hudi/commit/6d8dedd061ed183c52471df09355b85e5a60099a?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 21 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | 6.46% | [245 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | 77.50% | [5 Missing and 4 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | 93.54% | [1 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18526 +/- ##
============================================
- Coverage 68.88% 65.73% -3.16%
+ Complexity 28279 27262 -1017
============================================
Files 2464 2478 +14
Lines 135568 136972 +1404
Branches 16446 16653 +207
============================================
- Hits 93385 90037 -3348
- Misses 34801 39490 +4689
- Partials 7382 7445 +63
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.39% <13.77%> (-0.25%)` | :arrow_down: |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.74% <0.00%> (-0.02%)` | :arrow_down: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.64% <77.46%> (+0.23%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `42.88% <0.00%> (-6.05%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.80% <0.00%> (-0.64%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | `93.54% <93.54%> (ø)` | |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | `77.50% <77.50%> (ø)` | |
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | `6.46% <6.46%> (ø)` | |
... and [335 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18526/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-4300698475
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `80.83832%` with `64 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 68.67%. Comparing base ([`7bcb8be`](https://app.codecov.io/gh/apache/hudi/commit/7bcb8be7c153010998574f591c7ff3799192eda6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`6d8dedd`](https://app.codecov.io/gh/apache/hudi/commit/6d8dedd061ed183c52471df09355b85e5a60099a?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 21 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | 79.84% | [43 Missing and 10 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | 77.50% | [5 Missing and 4 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | 93.54% | [1 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18526 +/- ##
============================================
- Coverage 68.88% 68.67% -0.21%
- Complexity 28279 28471 +192
============================================
Files 2464 2478 +14
Lines 135568 136972 +1404
Branches 16446 16653 +207
============================================
+ Hits 93385 94070 +685
- Misses 34801 35328 +527
- Partials 7382 7574 +192
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.39% <13.77%> (-0.25%)` | :arrow_down: |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.74% <0.00%> (-0.02%)` | :arrow_down: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.64% <77.46%> (+0.23%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `49.15% <0.00%> (+0.22%)` | :arrow_up: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.18% <0.00%> (-0.26%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `37.87% <77.54%> (-0.30%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | `93.54% <93.54%> (ø)` | |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | `77.50% <77.50%> (ø)` | |
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | `79.84% <79.84%> (ø)` | |
... and [82 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18526/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-4300693729
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `80.83832%` with `64 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 68.60%. Comparing base ([`7bcb8be`](https://app.codecov.io/gh/apache/hudi/commit/7bcb8be7c153010998574f591c7ff3799192eda6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`6d8dedd`](https://app.codecov.io/gh/apache/hudi/commit/6d8dedd061ed183c52471df09355b85e5a60099a?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 21 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | 79.84% | [43 Missing and 10 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | 77.50% | [5 Missing and 4 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | 93.54% | [1 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18526 +/- ##
============================================
- Coverage 68.88% 68.60% -0.29%
- Complexity 28279 28421 +142
============================================
Files 2464 2478 +14
Lines 135568 136972 +1404
Branches 16446 16653 +207
============================================
+ Hits 93385 93968 +583
- Misses 34801 35431 +630
- Partials 7382 7573 +191
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.39% <13.77%> (-0.25%)` | :arrow_down: |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.74% <0.00%> (-0.02%)` | :arrow_down: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.64% <77.46%> (+0.23%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.92% <0.00%> (-0.01%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.18% <0.00%> (-0.26%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `37.87% <77.54%> (-0.30%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | `93.54% <93.54%> (ø)` | |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | `77.50% <77.50%> (ø)` | |
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | `79.84% <79.84%> (ø)` | |
... and [92 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18526/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-4300686949
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `23.05389%` with `257 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 66.01%. Comparing base ([`7bcb8be`](https://app.codecov.io/gh/apache/hudi/commit/7bcb8be7c153010998574f591c7ff3799192eda6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`6d8dedd`](https://app.codecov.io/gh/apache/hudi/commit/6d8dedd061ed183c52471df09355b85e5a60099a?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 21 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | 6.46% | [245 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | 77.50% | [5 Missing and 4 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | 93.54% | [1 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18526 +/- ##
============================================
- Coverage 68.88% 66.01% -2.88%
+ Complexity 28279 27431 -848
============================================
Files 2464 2478 +14
Lines 135568 136972 +1404
Branches 16446 16653 +207
============================================
- Hits 93385 90421 -2964
- Misses 34801 39026 +4225
- Partials 7382 7525 +143
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.39% <13.77%> (-0.25%)` | :arrow_down: |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.74% <0.00%> (-0.02%)` | :arrow_down: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.64% <77.46%> (+0.23%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.48% <0.00%> (-3.45%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.18% <0.00%> (-0.26%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | `93.54% <93.54%> (ø)` | |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | `77.50% <77.50%> (ø)` | |
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | `6.46% <6.46%> (ø)` | |
... and [297 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18526/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "nsivabalan (via GitHub)" <gi...@apache.org>.
nsivabalan commented on code in PR #18526:
URL: https://github.com/apache/hudi/pull/18526#discussion_r3127248812
##########
hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/client/validator/StashPartitionsPreCommitValidator.java:
##########
@@ -0,0 +1,181 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.hudi.client.validator;
+
+import org.apache.hudi.client.WriteStatus;
+import org.apache.hudi.common.config.HoodieMetadataConfig;
+import org.apache.hudi.common.data.HoodieData;
+import org.apache.hudi.common.engine.HoodieEngineContext;
+import org.apache.hudi.common.util.ReflectionUtils;
+import org.apache.hudi.common.util.StringUtils;
+import org.apache.hudi.common.util.ValidationUtils;
+import org.apache.hudi.config.HoodieWriteConfig;
+import org.apache.hudi.exception.HoodieValidationException;
+import org.apache.hudi.storage.HoodieStorage;
+import org.apache.hudi.storage.StoragePath;
+import org.apache.hudi.storage.StoragePathInfo;
+import org.apache.hudi.table.HoodieSparkTable;
+import org.apache.hudi.table.action.HoodieWriteMetadata;
+
+import org.apache.spark.sql.Dataset;
+import org.apache.spark.sql.Row;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.IOException;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+
+/**
+ * A special-purpose pre-commit validator that moves partition files to a stash
+ * location before a deletePartitions commit completes.
+ *
+ * <p><strong>WARNING: This is NOT a standard validator pattern.</strong>
+ * Unlike typical pre-commit validators that are read-only checks, this validator
+ * performs destructive side-effects (moving/deleting files). It is designed exclusively
+ * for use with the {@code HoodieStashPartitionsTool} and the DELETE_PARTITION operation.
+ * Do NOT use this as a pattern for other validators.</p>
+ *
+ * <h3>Why move files inside a pre-commit validator?</h3>
+ *
+ * <p>The stash operation needs to both move data files to a backup location and mark
+ * the partitions as deleted via a replace commit. These two actions must be coupled:
+ * if the replace commit lands on the timeline but the file move has not happened yet,
+ * the Hudi cleaner will eventually delete the physical data files for the replaced
+ * file groups (based on its retention policy), permanently losing the data that was
+ * intended to be stashed.
+ *
+ * <p>By performing the file move inside this pre-commit validator, the move happens
+ * <b>before</b> the replace commit is finalized. If the move fails, the validator
+ * throws an exception, the commit does not land, and the table remains unchanged.
+ * This guarantees that data is never in a state where it has been marked as deleted
+ * but has not been safely backed up to the stash location.</p>
+ *
+ * <p>This validator requires the following configs to be set:
+ * <ul>
+ * <li>{@link #STASH_PATH_CONFIG} — the target stash directory</li>
+ * <li>{@link #RENAME_HELPER_CLASS_CONFIG} (optional) — custom rename helper class</li>
+ * </ul>
+ *
+ * <p>Guards:
+ * <ul>
+ * <li>Fails if RLI (Record Level Index) is enabled on the table</li>
+ * <li>Must only be used with DELETE_PARTITION operation</li>
+ * </ul>
+ */
+public class StashPartitionsPreCommitValidator<T, I, K, O extends HoodieData<WriteStatus>>
+ extends SparkPreCommitValidator<T, I, K, O> {
+
+ private static final Logger LOG = LoggerFactory.getLogger(StashPartitionsPreCommitValidator.class);
+
+ /**
+ * Config key for the stash target directory path.
+ */
+ public static final String STASH_PATH_CONFIG = "hoodie.stash.partitions.target.path";
+
+ /**
+ * Config key for the custom rename helper class. Defaults to {@link DefaultStashPartitionRenameHelper}.
+ */
+ public static final String RENAME_HELPER_CLASS_CONFIG = "hoodie.stash.rename.helper.class";
+
+ public StashPartitionsPreCommitValidator(HoodieSparkTable<T> table,
+ HoodieEngineContext engineContext,
+ HoodieWriteConfig writeConfig) {
+ super(table, engineContext, writeConfig);
+ }
+
+ /**
+ * Override validate to extract affected partitions from partitionToReplaceFileIds rather than
+ * write stats, since deletePartitions produces empty write stats but populates the replacement map.
+ */
+ @Override
+ public void validate(String instantTime, HoodieWriteMetadata<O> writeResult,
Review Comment:
let me add this to the StashPartitionTool and bail out early if condition is not met
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-4300632028
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `23.05389%` with `257 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 58.61%. Comparing base ([`7bcb8be`](https://app.codecov.io/gh/apache/hudi/commit/7bcb8be7c153010998574f591c7ff3799192eda6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`6d8dedd`](https://app.codecov.io/gh/apache/hudi/commit/6d8dedd061ed183c52471df09355b85e5a60099a?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 21 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | 6.46% | [245 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | 77.50% | [5 Missing and 4 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | 93.54% | [1 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
> :exclamation: There is a different number of reports uploaded between BASE (7bcb8be) and HEAD (6d8dedd). Click for more details.
>
> <details><summary>HEAD has 17 uploads less than BASE</summary>
>
>| Flag | BASE (7bcb8be) | HEAD (6d8dedd) |
>|------|------|------|
>|spark-scala-tests|10|5|
>|spark-java-tests|15|4|
>|utilities|1|0|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18526 +/- ##
=============================================
- Coverage 68.88% 58.61% -10.27%
+ Complexity 28279 24322 -3957
=============================================
Files 2464 2478 +14
Lines 135568 136958 +1390
Branches 16446 16651 +205
=============================================
- Hits 93385 80279 -13106
- Misses 34801 49794 +14993
+ Partials 7382 6885 -497
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.39% <13.77%> (-0.25%)` | :arrow_down: |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.74% <0.00%> (-0.02%)` | :arrow_down: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.64% <77.46%> (+0.23%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `32.18% <0.00%> (-16.75%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `32.36% <0.00%> (-13.09%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | `93.54% <93.54%> (ø)` | |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | `77.50% <77.50%> (ø)` | |
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | `6.46% <6.46%> (ø)` | |
... and [682 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18526/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-4300613668
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `16.46707%` with `279 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 48.26%. Comparing base ([`7bcb8be`](https://app.codecov.io/gh/apache/hudi/commit/7bcb8be7c153010998574f591c7ff3799192eda6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`6d8dedd`](https://app.codecov.io/gh/apache/hudi/commit/6d8dedd061ed183c52471df09355b85e5a60099a?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 21 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | 0.00% | [263 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | 77.50% | [5 Missing and 4 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | 77.41% | [5 Missing and 2 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
> :exclamation: There is a different number of reports uploaded between BASE (7bcb8be) and HEAD (6d8dedd). Click for more details.
>
> <details><summary>HEAD has 22 uploads less than BASE</summary>
>
>| Flag | BASE (7bcb8be) | HEAD (6d8dedd) |
>|------|------|------|
>|spark-scala-tests|10|4|
>|spark-java-tests|15|1|
>|common-and-other-modules|1|0|
>|utilities|1|0|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18526 +/- ##
=============================================
- Coverage 68.88% 48.26% -20.62%
+ Complexity 28279 16350 -11929
=============================================
Files 2464 1989 -475
Lines 135568 110942 -24626
Branches 16446 13898 -2548
=============================================
- Hits 93385 53546 -39839
- Misses 34801 51850 +17049
+ Partials 7382 5546 -1836
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.74% <0.00%> (-0.02%)` | :arrow_down: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.64% <77.46%> (+0.23%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `33.21% <0.00%> (-15.72%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `32.44% <0.00%> (-13.00%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | `77.41% <77.41%> (ø)` | |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | `77.50% <77.50%> (ø)` | |
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | `0.00% <0.00%> (ø)` | |
... and [1562 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18526/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-4284451480
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `23.31839%` with `171 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 47.95%. Comparing base ([`7bcb8be`](https://app.codecov.io/gh/apache/hudi/commit/7bcb8be7c153010998574f591c7ff3799192eda6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`891eaf2`](https://app.codecov.io/gh/apache/hudi/commit/891eaf2e1427bf93b9c7ff9626628c30e1c9c01d?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 11 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | 0.00% | [156 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | 75.00% | [6 Missing and 4 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | 81.48% | [3 Missing and 2 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
> :exclamation: There is a different number of reports uploaded between BASE (7bcb8be) and HEAD (891eaf2). Click for more details.
>
> <details><summary>HEAD has 20 uploads less than BASE</summary>
>
>| Flag | BASE (7bcb8be) | HEAD (891eaf2) |
>|------|------|------|
>|spark-scala-tests|10|5|
>|spark-java-tests|15|2|
>|common-and-other-modules|1|0|
>|utilities|1|0|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18526 +/- ##
=============================================
- Coverage 68.88% 47.95% -20.93%
+ Complexity 28279 16514 -11765
=============================================
Files 2464 2011 -453
Lines 135568 112821 -22747
Branches 16446 14328 -2118
=============================================
- Hits 93385 54106 -39279
- Misses 34801 53038 +18237
+ Partials 7382 5677 -1705
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.75% <0.00%> (+<0.01%)` | :arrow_up: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.47% <77.61%> (+0.07%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `33.07% <0.00%> (-15.86%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `32.48% <0.00%> (-12.96%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | `81.48% <81.48%> (ø)` | |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | `75.00% <75.00%> (ø)` | |
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | `0.00% <0.00%> (ø)` | |
... and [1534 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18526/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-4284539140
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `78.47534%` with `48 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 68.78%. Comparing base ([`7bcb8be`](https://app.codecov.io/gh/apache/hudi/commit/7bcb8be7c153010998574f591c7ff3799192eda6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`891eaf2`](https://app.codecov.io/gh/apache/hudi/commit/891eaf2e1427bf93b9c7ff9626628c30e1c9c01d?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 11 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | 76.92% | [28 Missing and 8 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | 75.00% | [6 Missing and 4 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | 92.59% | [1 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18526 +/- ##
============================================
- Coverage 68.88% 68.78% -0.10%
- Complexity 28279 28360 +81
============================================
Files 2464 2471 +7
Lines 135568 136218 +650
Branches 16446 16521 +75
============================================
+ Hits 93385 93702 +317
- Misses 34801 35098 +297
- Partials 7382 7418 +36
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.56% <11.21%> (-0.08%)` | :arrow_down: |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.75% <0.00%> (+<0.01%)` | :arrow_up: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.47% <77.61%> (+0.07%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.71% <0.00%> (-0.21%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.34% <0.00%> (-0.10%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `38.30% <74.88%> (+0.13%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | `92.59% <92.59%> (ø)` | |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | `75.00% <75.00%> (ø)` | |
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | `76.92% <76.92%> (ø)` | |
... and [29 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18526/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-5126305787
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `80.83832%` with `64 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 68.68%. Comparing base ([`7bcb8be`](https://app.codecov.io/gh/apache/hudi/commit/7bcb8be7c153010998574f591c7ff3799192eda6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`6d8dedd`](https://app.codecov.io/gh/apache/hudi/commit/6d8dedd061ed183c52471df09355b85e5a60099a?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 417 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | 79.84% | [43 Missing and 10 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | 77.50% | [5 Missing and 4 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | 93.54% | [1 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18526 +/- ##
============================================
- Coverage 68.88% 68.68% -0.21%
- Complexity 28279 28473 +194
============================================
Files 2464 2478 +14
Lines 135568 136972 +1404
Branches 16446 16653 +207
============================================
+ Hits 93385 94077 +692
- Misses 34801 35324 +523
- Partials 7382 7571 +189
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.39% <13.77%> (-0.25%)` | :arrow_down: |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.74% <0.00%> (-0.02%)` | :arrow_down: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.64% <77.46%> (+0.23%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `49.25% <0.00%> (+0.32%)` | :arrow_up: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.18% <0.00%> (-0.26%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `37.87% <77.54%> (-0.30%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | `93.54% <93.54%> (ø)` | |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | `77.50% <77.50%> (ø)` | |
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | `79.84% <79.84%> (ø)` | |
... and [79 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18526/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
- :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-4272407314
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "67f1e57f0dffe02bbe2d7bacfaa8f1aaa4be8c88",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=13351",
"triggerID" : "67f1e57f0dffe02bbe2d7bacfaa8f1aaa4be8c88",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 67f1e57f0dffe02bbe2d7bacfaa8f1aaa4be8c88 Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=13351)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-4272494473
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `23.31839%` with `171 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 47.96%. Comparing base ([`7bcb8be`](https://app.codecov.io/gh/apache/hudi/commit/7bcb8be7c153010998574f591c7ff3799192eda6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`e8b5a0a`](https://app.codecov.io/gh/apache/hudi/commit/e8b5a0a62cea99be0a184e0b6f11149b49850806?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 3 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | 0.00% | [156 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | 75.00% | [6 Missing and 4 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | 81.48% | [3 Missing and 2 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
> :exclamation: There is a different number of reports uploaded between BASE (7bcb8be) and HEAD (e8b5a0a). Click for more details.
>
> <details><summary>HEAD has 20 uploads less than BASE</summary>
>
>| Flag | BASE (7bcb8be) | HEAD (e8b5a0a) |
>|------|------|------|
>|spark-scala-tests|10|5|
>|spark-java-tests|15|2|
>|common-and-other-modules|1|0|
>|utilities|1|0|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18526 +/- ##
=============================================
- Coverage 68.88% 47.96% -20.92%
+ Complexity 28279 16514 -11765
=============================================
Files 2464 2011 -453
Lines 135568 112793 -22775
Branches 16446 14326 -2120
=============================================
- Hits 93385 54101 -39284
- Misses 34801 53014 +18213
+ Partials 7382 5678 -1704
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.70% <0.00%> (-0.06%)` | :arrow_down: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.46% <77.61%> (+0.06%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `33.08% <0.00%> (-15.85%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `32.48% <0.00%> (-12.96%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | `81.48% <81.48%> (ø)` | |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | `75.00% <75.00%> (ø)` | |
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | `0.00% <0.00%> (ø)` | |
... and [1532 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18526/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-4272493616
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `23.31839%` with `171 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 48.42%. Comparing base ([`7bcb8be`](https://app.codecov.io/gh/apache/hudi/commit/7bcb8be7c153010998574f591c7ff3799192eda6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`e8b5a0a`](https://app.codecov.io/gh/apache/hudi/commit/e8b5a0a62cea99be0a184e0b6f11149b49850806?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 3 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | 0.00% | [156 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | 75.00% | [6 Missing and 4 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | 81.48% | [3 Missing and 2 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
> :exclamation: There is a different number of reports uploaded between BASE (7bcb8be) and HEAD (e8b5a0a). Click for more details.
>
> <details><summary>HEAD has 22 uploads less than BASE</summary>
>
>| Flag | BASE (7bcb8be) | HEAD (e8b5a0a) |
>|------|------|------|
>|spark-scala-tests|10|4|
>|spark-java-tests|15|1|
>|common-and-other-modules|1|0|
>|utilities|1|0|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18526 +/- ##
=============================================
- Coverage 68.88% 48.42% -20.46%
+ Complexity 28279 16279 -12000
=============================================
Files 2464 1982 -482
Lines 135568 110190 -25378
Branches 16446 13772 -2674
=============================================
- Hits 93385 53364 -40021
- Misses 34801 51306 +16505
+ Partials 7382 5520 -1862
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.70% <0.00%> (-0.06%)` | :arrow_down: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.46% <77.61%> (+0.06%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `33.45% <0.00%> (-15.47%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `32.61% <0.00%> (-12.83%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | `81.48% <81.48%> (ø)` | |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | `75.00% <75.00%> (ø)` | |
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | `0.00% <0.00%> (ø)` | |
... and [1549 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18526/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-4272524442
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `78.47534%` with `48 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 68.03%. Comparing base ([`7bcb8be`](https://app.codecov.io/gh/apache/hudi/commit/7bcb8be7c153010998574f591c7ff3799192eda6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`e8b5a0a`](https://app.codecov.io/gh/apache/hudi/commit/e8b5a0a62cea99be0a184e0b6f11149b49850806?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 3 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | 76.92% | [28 Missing and 8 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | 75.00% | [6 Missing and 4 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | 92.59% | [1 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18526 +/- ##
============================================
- Coverage 68.88% 68.03% -0.86%
+ Complexity 28279 27958 -321
============================================
Files 2464 2467 +3
Lines 135568 135817 +249
Branches 16446 16468 +22
============================================
- Hits 93385 92399 -986
- Misses 34801 35942 +1141
- Partials 7382 7476 +94
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.59% <11.21%> (-0.05%)` | :arrow_down: |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.70% <0.00%> (-0.06%)` | :arrow_down: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.46% <77.61%> (+0.06%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.07% <0.00%> (-3.86%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.35% <0.00%> (-0.09%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `38.31% <74.88%> (+0.14%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | `92.59% <92.59%> (ø)` | |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | `75.00% <75.00%> (ø)` | |
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | `76.92% <76.92%> (ø)` | |
... and [154 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18526/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-4272528156
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `78.47534%` with `48 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 68.82%. Comparing base ([`7bcb8be`](https://app.codecov.io/gh/apache/hudi/commit/7bcb8be7c153010998574f591c7ff3799192eda6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`e8b5a0a`](https://app.codecov.io/gh/apache/hudi/commit/e8b5a0a62cea99be0a184e0b6f11149b49850806?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 3 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | 76.92% | [28 Missing and 8 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | 75.00% | [6 Missing and 4 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | 92.59% | [1 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18526 +/- ##
============================================
- Coverage 68.88% 68.82% -0.07%
+ Complexity 28279 28271 -8
============================================
Files 2464 2467 +3
Lines 135568 135817 +249
Branches 16446 16468 +22
============================================
+ Hits 93385 93473 +88
- Misses 34801 34928 +127
- Partials 7382 7416 +34
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.59% <11.21%> (-0.05%)` | :arrow_down: |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.70% <0.00%> (-0.06%)` | :arrow_down: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.46% <77.61%> (+0.06%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.58% <0.00%> (-0.34%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.35% <0.00%> (-0.09%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `38.31% <74.88%> (+0.14%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | `92.59% <92.59%> (ø)` | |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | `75.00% <75.00%> (ø)` | |
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | `76.92% <76.92%> (ø)` | |
... and [29 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18526/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-4300453762
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "67f1e57f0dffe02bbe2d7bacfaa8f1aaa4be8c88",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=13351",
"triggerID" : "67f1e57f0dffe02bbe2d7bacfaa8f1aaa4be8c88",
"triggerType" : "PUSH"
}, {
"hash" : "e8b5a0a62cea99be0a184e0b6f11149b49850806",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=13354",
"triggerID" : "e8b5a0a62cea99be0a184e0b6f11149b49850806",
"triggerType" : "PUSH"
}, {
"hash" : "891eaf2e1427bf93b9c7ff9626628c30e1c9c01d",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=13391",
"triggerID" : "891eaf2e1427bf93b9c7ff9626628c30e1c9c01d",
"triggerType" : "PUSH"
}, {
"hash" : "6d8dedd061ed183c52471df09355b85e5a60099a",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "6d8dedd061ed183c52471df09355b85e5a60099a",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 891eaf2e1427bf93b9c7ff9626628c30e1c9c01d Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=13391)
* 6d8dedd061ed183c52471df09355b85e5a60099a UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-4300609784
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `16.46707%` with `279 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 48.26%. Comparing base ([`7bcb8be`](https://app.codecov.io/gh/apache/hudi/commit/7bcb8be7c153010998574f591c7ff3799192eda6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`6d8dedd`](https://app.codecov.io/gh/apache/hudi/commit/6d8dedd061ed183c52471df09355b85e5a60099a?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 20 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | 0.00% | [263 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | 77.50% | [5 Missing and 4 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | 77.41% | [5 Missing and 2 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
> :exclamation: There is a different number of reports uploaded between BASE (7bcb8be) and HEAD (6d8dedd). Click for more details.
>
> <details><summary>HEAD has 22 uploads less than BASE</summary>
>
>| Flag | BASE (7bcb8be) | HEAD (6d8dedd) |
>|------|------|------|
>|spark-scala-tests|10|4|
>|spark-java-tests|15|1|
>|common-and-other-modules|1|0|
>|utilities|1|0|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18526 +/- ##
=============================================
- Coverage 68.88% 48.26% -20.62%
+ Complexity 28279 16350 -11929
=============================================
Files 2464 1989 -475
Lines 135568 110942 -24626
Branches 16446 13898 -2548
=============================================
- Hits 93385 53546 -39839
- Misses 34801 51850 +17049
+ Partials 7382 5546 -1836
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.74% <0.00%> (-0.02%)` | :arrow_down: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.64% <77.46%> (+0.23%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `33.21% <0.00%> (-15.72%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `32.44% <0.00%> (-13.00%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | `77.41% <77.41%> (ø)` | |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | `77.50% <77.50%> (ø)` | |
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | `0.00% <0.00%> (ø)` | |
... and [1562 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18526/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-4300592466
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `77.46479%` with `16 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 54.08%. Comparing base ([`7bcb8be`](https://app.codecov.io/gh/apache/hudi/commit/7bcb8be7c153010998574f591c7ff3799192eda6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`6d8dedd`](https://app.codecov.io/gh/apache/hudi/commit/6d8dedd061ed183c52471df09355b85e5a60099a?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 20 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | 77.50% | [5 Missing and 4 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | 77.41% | [5 Missing and 2 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
> :exclamation: There is a different number of reports uploaded between BASE (7bcb8be) and HEAD (6d8dedd). Click for more details.
>
> <details><summary>HEAD has 27 uploads less than BASE</summary>
>
>| Flag | BASE (7bcb8be) | HEAD (6d8dedd) |
>|------|------|------|
>|spark-scala-tests|10|0|
>|spark-java-tests|15|0|
>|common-and-other-modules|1|0|
>|utilities|1|0|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18526 +/- ##
=============================================
- Coverage 68.88% 54.08% -14.80%
+ Complexity 28279 12383 -15896
=============================================
Files 2464 1430 -1034
Lines 135568 71541 -64027
Branches 16446 8099 -8347
=============================================
- Hits 93385 38695 -54690
+ Misses 34801 29387 -5414
+ Partials 7382 3459 -3923
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.74% <0.00%> (-0.02%)` | :arrow_down: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.64% <77.46%> (+0.23%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | `77.41% <77.41%> (ø)` | |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | `77.50% <77.50%> (ø)` | |
... and [1816 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18526/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-4284465541
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `24.66368%` with `168 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 58.81%. Comparing base ([`7bcb8be`](https://app.codecov.io/gh/apache/hudi/commit/7bcb8be7c153010998574f591c7ff3799192eda6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`891eaf2`](https://app.codecov.io/gh/apache/hudi/commit/891eaf2e1427bf93b9c7ff9626628c30e1c9c01d?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 11 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | 0.00% | [156 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | 75.00% | [6 Missing and 4 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | 92.59% | [1 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
> :exclamation: There is a different number of reports uploaded between BASE (7bcb8be) and HEAD (891eaf2). Click for more details.
>
> <details><summary>HEAD has 16 uploads less than BASE</summary>
>
>| Flag | BASE (7bcb8be) | HEAD (891eaf2) |
>|------|------|------|
>|spark-scala-tests|10|5|
>|spark-java-tests|15|5|
>|utilities|1|0|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18526 +/- ##
=============================================
- Coverage 68.88% 58.81% -10.07%
+ Complexity 28279 24263 -4016
=============================================
Files 2464 2471 +7
Lines 135568 136218 +650
Branches 16446 16521 +75
=============================================
- Hits 93385 80116 -13269
- Misses 34801 49243 +14442
+ Partials 7382 6859 -523
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.56% <11.21%> (-0.08%)` | :arrow_down: |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.75% <0.00%> (+<0.01%)` | :arrow_up: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.47% <77.61%> (+0.07%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `32.28% <0.00%> (-16.64%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `32.48% <0.00%> (-12.96%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | `92.59% <92.59%> (ø)` | |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | `75.00% <75.00%> (ø)` | |
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | `0.00% <0.00%> (ø)` | |
... and [673 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18526/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "nsivabalan (via GitHub)" <gi...@apache.org>.
nsivabalan commented on code in PR #18526:
URL: https://github.com/apache/hudi/pull/18526#discussion_r3121011722
##########
hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/client/validator/StashPartitionsPreCommitValidator.java:
##########
@@ -0,0 +1,181 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.hudi.client.validator;
+
+import org.apache.hudi.client.WriteStatus;
+import org.apache.hudi.common.config.HoodieMetadataConfig;
+import org.apache.hudi.common.data.HoodieData;
+import org.apache.hudi.common.engine.HoodieEngineContext;
+import org.apache.hudi.common.util.ReflectionUtils;
+import org.apache.hudi.common.util.StringUtils;
+import org.apache.hudi.common.util.ValidationUtils;
+import org.apache.hudi.config.HoodieWriteConfig;
+import org.apache.hudi.exception.HoodieValidationException;
+import org.apache.hudi.storage.HoodieStorage;
+import org.apache.hudi.storage.StoragePath;
+import org.apache.hudi.storage.StoragePathInfo;
+import org.apache.hudi.table.HoodieSparkTable;
+import org.apache.hudi.table.action.HoodieWriteMetadata;
+
+import org.apache.spark.sql.Dataset;
+import org.apache.spark.sql.Row;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.IOException;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+
+/**
+ * A special-purpose pre-commit validator that moves partition files to a stash
+ * location before a deletePartitions commit completes.
+ *
+ * <p><strong>WARNING: This is NOT a standard validator pattern.</strong>
+ * Unlike typical pre-commit validators that are read-only checks, this validator
+ * performs destructive side-effects (moving/deleting files). It is designed exclusively
+ * for use with the {@code HoodieStashPartitionsTool} and the DELETE_PARTITION operation.
+ * Do NOT use this as a pattern for other validators.</p>
+ *
+ * <h3>Why move files inside a pre-commit validator?</h3>
+ *
+ * <p>The stash operation needs to both move data files to a backup location and mark
+ * the partitions as deleted via a replace commit. These two actions must be coupled:
+ * if the replace commit lands on the timeline but the file move has not happened yet,
+ * the Hudi cleaner will eventually delete the physical data files for the replaced
+ * file groups (based on its retention policy), permanently losing the data that was
+ * intended to be stashed.
+ *
+ * <p>By performing the file move inside this pre-commit validator, the move happens
+ * <b>before</b> the replace commit is finalized. If the move fails, the validator
+ * throws an exception, the commit does not land, and the table remains unchanged.
+ * This guarantees that data is never in a state where it has been marked as deleted
+ * but has not been safely backed up to the stash location.</p>
+ *
+ * <p>This validator requires the following configs to be set:
+ * <ul>
+ * <li>{@link #STASH_PATH_CONFIG} — the target stash directory</li>
+ * <li>{@link #RENAME_HELPER_CLASS_CONFIG} (optional) — custom rename helper class</li>
+ * </ul>
+ *
+ * <p>Guards:
+ * <ul>
+ * <li>Fails if RLI (Record Level Index) is enabled on the table</li>
+ * <li>Must only be used with DELETE_PARTITION operation</li>
+ * </ul>
+ */
+public class StashPartitionsPreCommitValidator<T, I, K, O extends HoodieData<WriteStatus>>
+ extends SparkPreCommitValidator<T, I, K, O> {
+
+ private static final Logger LOG = LoggerFactory.getLogger(StashPartitionsPreCommitValidator.class);
+
+ /**
+ * Config key for the stash target directory path.
+ */
+ public static final String STASH_PATH_CONFIG = "hoodie.stash.partitions.target.path";
+
+ /**
+ * Config key for the custom rename helper class. Defaults to {@link DefaultStashPartitionRenameHelper}.
+ */
+ public static final String RENAME_HELPER_CLASS_CONFIG = "hoodie.stash.rename.helper.class";
+
+ public StashPartitionsPreCommitValidator(HoodieSparkTable<T> table,
+ HoodieEngineContext engineContext,
+ HoodieWriteConfig writeConfig) {
+ super(table, engineContext, writeConfig);
+ }
+
+ /**
+ * Override validate to extract affected partitions from partitionToReplaceFileIds rather than
+ * write stats, since deletePartitions produces empty write stats but populates the replacement map.
+ */
+ @Override
+ public void validate(String instantTime, HoodieWriteMetadata<O> writeResult,
Review Comment:
synced up f2f.
lets add validations on only latest file slices. and only parquet files.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "nsivabalan (via GitHub)" <gi...@apache.org>.
nsivabalan commented on code in PR #18526:
URL: https://github.com/apache/hudi/pull/18526#discussion_r3120991992
##########
hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/client/validator/StashPartitionsPreCommitValidator.java:
##########
@@ -0,0 +1,181 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.hudi.client.validator;
+
+import org.apache.hudi.client.WriteStatus;
+import org.apache.hudi.common.config.HoodieMetadataConfig;
+import org.apache.hudi.common.data.HoodieData;
+import org.apache.hudi.common.engine.HoodieEngineContext;
+import org.apache.hudi.common.util.ReflectionUtils;
+import org.apache.hudi.common.util.StringUtils;
+import org.apache.hudi.common.util.ValidationUtils;
+import org.apache.hudi.config.HoodieWriteConfig;
+import org.apache.hudi.exception.HoodieValidationException;
+import org.apache.hudi.storage.HoodieStorage;
+import org.apache.hudi.storage.StoragePath;
+import org.apache.hudi.storage.StoragePathInfo;
+import org.apache.hudi.table.HoodieSparkTable;
+import org.apache.hudi.table.action.HoodieWriteMetadata;
+
+import org.apache.spark.sql.Dataset;
+import org.apache.spark.sql.Row;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.IOException;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+
+/**
+ * A special-purpose pre-commit validator that moves partition files to a stash
+ * location before a deletePartitions commit completes.
+ *
+ * <p><strong>WARNING: This is NOT a standard validator pattern.</strong>
+ * Unlike typical pre-commit validators that are read-only checks, this validator
+ * performs destructive side-effects (moving/deleting files). It is designed exclusively
+ * for use with the {@code HoodieStashPartitionsTool} and the DELETE_PARTITION operation.
+ * Do NOT use this as a pattern for other validators.</p>
+ *
+ * <h3>Why move files inside a pre-commit validator?</h3>
+ *
+ * <p>The stash operation needs to both move data files to a backup location and mark
+ * the partitions as deleted via a replace commit. These two actions must be coupled:
+ * if the replace commit lands on the timeline but the file move has not happened yet,
+ * the Hudi cleaner will eventually delete the physical data files for the replaced
+ * file groups (based on its retention policy), permanently losing the data that was
+ * intended to be stashed.
+ *
+ * <p>By performing the file move inside this pre-commit validator, the move happens
+ * <b>before</b> the replace commit is finalized. If the move fails, the validator
+ * throws an exception, the commit does not land, and the table remains unchanged.
+ * This guarantees that data is never in a state where it has been marked as deleted
+ * but has not been safely backed up to the stash location.</p>
+ *
+ * <p>This validator requires the following configs to be set:
+ * <ul>
+ * <li>{@link #STASH_PATH_CONFIG} — the target stash directory</li>
+ * <li>{@link #RENAME_HELPER_CLASS_CONFIG} (optional) — custom rename helper class</li>
+ * </ul>
+ *
+ * <p>Guards:
+ * <ul>
+ * <li>Fails if RLI (Record Level Index) is enabled on the table</li>
+ * <li>Must only be used with DELETE_PARTITION operation</li>
+ * </ul>
+ */
+public class StashPartitionsPreCommitValidator<T, I, K, O extends HoodieData<WriteStatus>>
+ extends SparkPreCommitValidator<T, I, K, O> {
+
+ private static final Logger LOG = LoggerFactory.getLogger(StashPartitionsPreCommitValidator.class);
+
+ /**
+ * Config key for the stash target directory path.
+ */
+ public static final String STASH_PATH_CONFIG = "hoodie.stash.partitions.target.path";
+
+ /**
+ * Config key for the custom rename helper class. Defaults to {@link DefaultStashPartitionRenameHelper}.
+ */
+ public static final String RENAME_HELPER_CLASS_CONFIG = "hoodie.stash.rename.helper.class";
+
+ public StashPartitionsPreCommitValidator(HoodieSparkTable<T> table,
+ HoodieEngineContext engineContext,
+ HoodieWriteConfig writeConfig) {
+ super(table, engineContext, writeConfig);
+ }
+
+ /**
+ * Override validate to extract affected partitions from partitionToReplaceFileIds rather than
+ * write stats, since deletePartitions produces empty write stats but populates the replacement map.
+ */
+ @Override
+ public void validate(String instantTime, HoodieWriteMetadata<O> writeResult,
+ Dataset<Row> before, Dataset<Row> after) throws HoodieValidationException {
+ Set<String> partitionsAffected = new HashSet<>(writeResult.getPartitionToReplaceFileIds().keySet());
+ if (partitionsAffected.isEmpty()) {
+ // Fall back to write stats if partitionToReplaceFileIds is not populated
+ partitionsAffected = getPartitionsModified(writeResult);
+ }
+ validateRecordsBeforeAndAfter(before, after, partitionsAffected);
+ }
+
+ @Override
+ protected void validateRecordsBeforeAndAfter(Dataset<Row> before,
+ Dataset<Row> after,
+ Set<String> partitionsAffected) {
+ // Guard: ensure RLI is not enabled
+ HoodieMetadataConfig metadataConfig = getWriteConfig().getMetadataConfig();
Review Comment:
we should dis allow SI though.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-5126300900
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `80.83832%` with `64 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 68.68%. Comparing base ([`7bcb8be`](https://app.codecov.io/gh/apache/hudi/commit/7bcb8be7c153010998574f591c7ff3799192eda6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`6d8dedd`](https://app.codecov.io/gh/apache/hudi/commit/6d8dedd061ed183c52471df09355b85e5a60099a?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 417 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | 79.84% | [43 Missing and 10 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | 77.50% | [5 Missing and 4 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | 93.54% | [1 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18526 +/- ##
============================================
- Coverage 68.88% 68.68% -0.21%
- Complexity 28279 28473 +194
============================================
Files 2464 2478 +14
Lines 135568 136972 +1404
Branches 16446 16653 +207
============================================
+ Hits 93385 94077 +692
- Misses 34801 35324 +523
- Partials 7382 7571 +189
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.39% <13.77%> (-0.25%)` | :arrow_down: |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.74% <0.00%> (-0.02%)` | :arrow_down: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.64% <77.46%> (+0.23%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `49.25% <0.00%> (+0.32%)` | :arrow_up: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.18% <0.00%> (-0.26%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `37.87% <77.54%> (-0.30%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | `93.54% <93.54%> (ø)` | |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | `77.50% <77.50%> (ø)` | |
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | `79.84% <79.84%> (ø)` | |
... and [79 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18526/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
- :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-4284464057
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `24.66368%` with `168 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 58.76%. Comparing base ([`7bcb8be`](https://app.codecov.io/gh/apache/hudi/commit/7bcb8be7c153010998574f591c7ff3799192eda6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`891eaf2`](https://app.codecov.io/gh/apache/hudi/commit/891eaf2e1427bf93b9c7ff9626628c30e1c9c01d?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 11 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | 0.00% | [156 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | 75.00% | [6 Missing and 4 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | 92.59% | [1 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
> :exclamation: There is a different number of reports uploaded between BASE (7bcb8be) and HEAD (891eaf2). Click for more details.
>
> <details><summary>HEAD has 18 uploads less than BASE</summary>
>
>| Flag | BASE (7bcb8be) | HEAD (891eaf2) |
>|------|------|------|
>|spark-scala-tests|10|5|
>|spark-java-tests|15|3|
>|utilities|1|0|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18526 +/- ##
=============================================
- Coverage 68.88% 58.76% -10.12%
+ Complexity 28279 24243 -4036
=============================================
Files 2464 2471 +7
Lines 135568 136218 +650
Branches 16446 16521 +75
=============================================
- Hits 93385 80050 -13335
- Misses 34801 49301 +14500
+ Partials 7382 6867 -515
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.56% <11.21%> (-0.08%)` | :arrow_down: |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.75% <0.00%> (+<0.01%)` | :arrow_up: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.47% <77.61%> (+0.07%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `33.11% <0.00%> (-15.82%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `32.48% <0.00%> (-12.96%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | `92.59% <92.59%> (ø)` | |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | `75.00% <75.00%> (ø)` | |
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | `0.00% <0.00%> (ø)` | |
... and [673 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18526/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-4284506958
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `78.47534%` with `48 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 67.56%. Comparing base ([`7bcb8be`](https://app.codecov.io/gh/apache/hudi/commit/7bcb8be7c153010998574f591c7ff3799192eda6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`891eaf2`](https://app.codecov.io/gh/apache/hudi/commit/891eaf2e1427bf93b9c7ff9626628c30e1c9c01d?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 11 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | 76.92% | [28 Missing and 8 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | 75.00% | [6 Missing and 4 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | 92.59% | [1 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18526 +/- ##
============================================
- Coverage 68.88% 67.56% -1.33%
+ Complexity 28279 27817 -462
============================================
Files 2464 2471 +7
Lines 135568 136218 +650
Branches 16446 16521 +75
============================================
- Hits 93385 92031 -1354
- Misses 34801 36744 +1943
- Partials 7382 7443 +61
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.56% <11.21%> (-0.08%)` | :arrow_down: |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.75% <0.00%> (+<0.01%)` | :arrow_up: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.47% <77.61%> (+0.07%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `42.32% <0.00%> (-6.60%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.98% <0.00%> (-0.46%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `38.30% <74.88%> (+0.13%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | `92.59% <92.59%> (ø)` | |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | `75.00% <75.00%> (ø)` | |
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | `76.92% <76.92%> (ø)` | |
... and [229 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18526/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-4284546078
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `78.47534%` with `48 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 68.79%. Comparing base ([`7bcb8be`](https://app.codecov.io/gh/apache/hudi/commit/7bcb8be7c153010998574f591c7ff3799192eda6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`891eaf2`](https://app.codecov.io/gh/apache/hudi/commit/891eaf2e1427bf93b9c7ff9626628c30e1c9c01d?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 11 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | 76.92% | [28 Missing and 8 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | 75.00% | [6 Missing and 4 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | 92.59% | [1 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18526 +/- ##
============================================
- Coverage 68.88% 68.79% -0.10%
- Complexity 28279 28361 +82
============================================
Files 2464 2471 +7
Lines 135568 136218 +650
Branches 16446 16521 +75
============================================
+ Hits 93385 93705 +320
- Misses 34801 35097 +296
- Partials 7382 7416 +34
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.56% <11.21%> (-0.08%)` | :arrow_down: |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.75% <0.00%> (+<0.01%)` | :arrow_up: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.47% <77.61%> (+0.07%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.75% <0.00%> (-0.18%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.34% <0.00%> (-0.10%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `38.30% <74.88%> (+0.13%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | `92.59% <92.59%> (ø)` | |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | `75.00% <75.00%> (ø)` | |
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | `76.92% <76.92%> (ø)` | |
... and [27 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18526/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-4272719857
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "67f1e57f0dffe02bbe2d7bacfaa8f1aaa4be8c88",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=13351",
"triggerID" : "67f1e57f0dffe02bbe2d7bacfaa8f1aaa4be8c88",
"triggerType" : "PUSH"
}, {
"hash" : "e8b5a0a62cea99be0a184e0b6f11149b49850806",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=13354",
"triggerID" : "e8b5a0a62cea99be0a184e0b6f11149b49850806",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* e8b5a0a62cea99be0a184e0b6f11149b49850806 Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=13354)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-4953821809
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `80.83832%` with `64 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 68.68%. Comparing base ([`7bcb8be`](https://app.codecov.io/gh/apache/hudi/commit/7bcb8be7c153010998574f591c7ff3799192eda6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`6d8dedd`](https://app.codecov.io/gh/apache/hudi/commit/6d8dedd061ed183c52471df09355b85e5a60099a?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 330 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | 79.84% | [43 Missing and 10 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | 77.50% | [5 Missing and 4 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | 93.54% | [1 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18526 +/- ##
============================================
- Coverage 68.88% 68.68% -0.21%
- Complexity 28279 28473 +194
============================================
Files 2464 2478 +14
Lines 135568 136972 +1404
Branches 16446 16653 +207
============================================
+ Hits 93385 94077 +692
- Misses 34801 35324 +523
- Partials 7382 7571 +189
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.39% <13.77%> (-0.25%)` | :arrow_down: |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.74% <0.00%> (-0.02%)` | :arrow_down: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.64% <77.46%> (+0.23%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `49.25% <0.00%> (+0.32%)` | :arrow_up: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.18% <0.00%> (-0.26%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `37.87% <77.54%> (-0.30%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | `93.54% <93.54%> (ø)` | |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | `77.50% <77.50%> (ø)` | |
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | `79.84% <79.84%> (ø)` | |
... and [79 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18526/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-4272498099
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `23.31839%` with `171 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 48.03%. Comparing base ([`7bcb8be`](https://app.codecov.io/gh/apache/hudi/commit/7bcb8be7c153010998574f591c7ff3799192eda6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`e8b5a0a`](https://app.codecov.io/gh/apache/hudi/commit/e8b5a0a62cea99be0a184e0b6f11149b49850806?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 3 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | 0.00% | [156 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | 75.00% | [6 Missing and 4 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | 81.48% | [3 Missing and 2 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
> :exclamation: There is a different number of reports uploaded between BASE (7bcb8be) and HEAD (e8b5a0a). Click for more details.
>
> <details><summary>HEAD has 17 uploads less than BASE</summary>
>
>| Flag | BASE (7bcb8be) | HEAD (e8b5a0a) |
>|------|------|------|
>|spark-scala-tests|10|5|
>|spark-java-tests|15|5|
>|common-and-other-modules|1|0|
>|utilities|1|0|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18526 +/- ##
=============================================
- Coverage 68.88% 48.03% -20.86%
+ Complexity 28279 16535 -11744
=============================================
Files 2464 2011 -453
Lines 135568 112793 -22775
Branches 16446 14326 -2120
=============================================
- Hits 93385 54178 -39207
- Misses 34801 52952 +18151
+ Partials 7382 5663 -1719
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.70% <0.00%> (-0.06%)` | :arrow_down: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.46% <77.61%> (+0.06%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `32.29% <0.00%> (-16.64%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `32.48% <0.00%> (-12.96%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | `81.48% <81.48%> (ø)` | |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | `75.00% <75.00%> (ø)` | |
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | `0.00% <0.00%> (ø)` | |
... and [1532 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18526/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-4272532163
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `78.47534%` with `48 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 68.89%. Comparing base ([`7bcb8be`](https://app.codecov.io/gh/apache/hudi/commit/7bcb8be7c153010998574f591c7ff3799192eda6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`e8b5a0a`](https://app.codecov.io/gh/apache/hudi/commit/e8b5a0a62cea99be0a184e0b6f11149b49850806?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 3 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | 76.92% | [28 Missing and 8 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | 75.00% | [6 Missing and 4 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | 92.59% | [1 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18526 +/- ##
==========================================
Coverage 68.88% 68.89%
- Complexity 28279 28316 +37
==========================================
Files 2464 2467 +3
Lines 135568 135817 +249
Branches 16446 16468 +22
==========================================
+ Hits 93385 93567 +182
- Misses 34801 34851 +50
- Partials 7382 7399 +17
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.59% <11.21%> (-0.05%)` | :arrow_down: |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.70% <0.00%> (-0.06%)` | :arrow_down: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.46% <77.61%> (+0.06%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.72% <0.00%> (-0.20%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.35% <0.00%> (-0.09%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `38.31% <74.88%> (+0.14%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | `92.59% <92.59%> (ø)` | |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | `75.00% <75.00%> (ø)` | |
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | `76.92% <76.92%> (ø)` | |
... and [14 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18526/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-4272537949
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `78.47534%` with `48 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 68.89%. Comparing base ([`7bcb8be`](https://app.codecov.io/gh/apache/hudi/commit/7bcb8be7c153010998574f591c7ff3799192eda6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`e8b5a0a`](https://app.codecov.io/gh/apache/hudi/commit/e8b5a0a62cea99be0a184e0b6f11149b49850806?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 3 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | 76.92% | [28 Missing and 8 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | 75.00% | [6 Missing and 4 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | 92.59% | [1 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18526 +/- ##
==========================================
Coverage 68.88% 68.89%
- Complexity 28279 28317 +38
==========================================
Files 2464 2467 +3
Lines 135568 135817 +249
Branches 16446 16468 +22
==========================================
+ Hits 93385 93569 +184
- Misses 34801 34850 +49
- Partials 7382 7398 +16
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.59% <11.21%> (-0.05%)` | :arrow_down: |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.70% <0.00%> (-0.06%)` | :arrow_down: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.46% <77.61%> (+0.06%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.76% <0.00%> (-0.17%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.35% <0.00%> (-0.09%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `38.31% <74.88%> (+0.14%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | `92.59% <92.59%> (ø)` | |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | `75.00% <75.00%> (ø)` | |
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | `76.92% <76.92%> (ø)` | |
... and [14 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18526/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "nsivabalan (via GitHub)" <gi...@apache.org>.
nsivabalan commented on code in PR #18526:
URL: https://github.com/apache/hudi/pull/18526#discussion_r3127222000
##########
hudi-utilities/src/main/java/org/apache/hudi/utilities/HoodieStashPartitionsTool.java:
##########
@@ -0,0 +1,403 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.hudi.utilities;
+
+import org.apache.hudi.client.HoodieWriteResult;
+import org.apache.hudi.client.SparkRDDWriteClient;
+import org.apache.hudi.client.validator.DefaultStashPartitionRenameHelper;
+import org.apache.hudi.client.validator.StashPartitionRenameHelper;
+import org.apache.hudi.client.validator.StashPartitionsPreCommitValidator;
+import org.apache.hudi.common.config.TypedProperties;
+import org.apache.hudi.common.model.HoodieRecordPayload;
+import org.apache.hudi.common.table.HoodieTableMetaClient;
+import org.apache.hudi.common.table.timeline.HoodieTimeline;
+import org.apache.hudi.common.util.Option;
+import org.apache.hudi.common.util.ReflectionUtils;
+import org.apache.hudi.common.util.StringUtils;
+import org.apache.hudi.config.HoodiePreCommitValidatorConfig;
+import org.apache.hudi.exception.HoodieException;
+import org.apache.hudi.hadoop.fs.HadoopFSUtils;
+import org.apache.hudi.storage.HoodieStorage;
+import org.apache.hudi.storage.HoodieStorageUtils;
+import org.apache.hudi.storage.StoragePath;
+import org.apache.hudi.storage.StoragePathInfo;
+
+import com.beust.jcommander.JCommander;
+import com.beust.jcommander.Parameter;
+import org.apache.hadoop.fs.Path;
+import org.apache.spark.api.java.JavaSparkContext;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.IOException;
+import java.io.Serializable;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+
+/**
+ * A spark-submit tool for stashing (backing up and removing) Hudi table partitions.
+ *
+ * <p>This tool supports three modes:
+ * <ul>
+ * <li><b>stash</b> — Moves data files from specified partitions to a stash location
+ * and issues a deletePartitions operation on the Hudi table. Uses a custom
+ * {@link StashPartitionsPreCommitValidator} to move files atomically before commit.</li>
+ * <li><b>rollback_stash</b> — Copies any data found in the stash location back to the
+ * original partition paths. Used when the user no longer wants to proceed with stashing.</li>
+ * <li><b>dry_run</b> — Shows which partitions and file groups would be affected.</li>
+ * </ul>
+ *
+ * <h3>Why a pre-commit validator instead of moving files in the tool?</h3>
+ *
+ * <p>A naive approach would be to first run {@code deletePartitions} in the tool, and then
+ * move the data files to the stash location as a separate step. However, this is unsafe:
+ * if the tool crashes or fails after {@code deletePartitions} succeeds but before the data
+ * is moved to the stash location, the data is at risk. Once the replace commit from
+ * {@code deletePartitions} lands on the timeline, the Hudi cleaner will eventually delete
+ * the physical data files for the replaced file groups based on its configured retention
+ * policy (e.g., retain last N commits). If the cleaner runs before the tool is retried,
+ * the data intended for stashing is permanently lost.
+ *
+ * <p>By using a {@link StashPartitionsPreCommitValidator}, the file move happens
+ * <b>before</b> the {@code deletePartitions} commit is finalized. If the move fails, the
+ * commit does not land, and the table remains in its original state — the cleaner has
+ * nothing to clean. This ensures that data is never in a state where it has been marked
+ * as deleted (via a replace commit) but has not yet been safely copied to the stash location.
+ *
+ * <p>For restoring stashed partitions back into the Hudi table, use the standard
+ * {@code insert_overwrite} write operation reading from the stash location.
+ *
+ * <p>Example usage:
+ * <pre>
+ * spark-submit \
+ * --class org.apache.hudi.utilities.HoodieStashPartitionsTool \
+ * --master local[*] \
+ * --driver-memory 1g \
+ * --executor-memory 1g \
+ * hudi-utilities-bundle.jar \
+ * --base-path /path/to/hudi/table \
+ * --table-name myTable \
+ * --stash-path /path/to/stash/folder \
+ * --partitions datestr=2023-01-01,datestr=2023-01-02 \
+ * --mode stash
+ * </pre>
+ */
+public class HoodieStashPartitionsTool implements Serializable {
+
+ private static final long serialVersionUID = 1L;
+ private static final Logger LOG = LoggerFactory.getLogger(HoodieStashPartitionsTool.class);
+
+ private final transient JavaSparkContext jsc;
+ private final Config cfg;
+ private final TypedProperties props;
+ private final HoodieTableMetaClient metaClient;
+
+ public HoodieStashPartitionsTool(JavaSparkContext jsc, Config cfg) {
+ this.jsc = jsc;
+ this.cfg = cfg;
+ this.props = cfg.propsFilePath == null
+ ? UtilHelpers.buildProperties(cfg.configs)
+ : readConfigFromFileSystem(jsc, cfg);
+ this.metaClient = HoodieTableMetaClient.builder()
+ .setConf(HadoopFSUtils.getStorageConfWithCopy(jsc.hadoopConfiguration()))
+ .setBasePath(cfg.basePath)
+ .setLoadActiveTimelineOnLoad(true)
+ .build();
+ }
+
+ public enum Mode {
+ STASH,
+ ROLLBACK_STASH,
+ DRY_RUN
+ }
+
+ public static class Config implements Serializable {
+ @Parameter(names = {"--base-path", "-sp"}, description = "Base path for the Hudi table.", required = true)
+ public String basePath = null;
+
+ @Parameter(names = {"--table-name", "-tn"}, description = "Table name.", required = true)
+ public String tableName = null;
+
+ @Parameter(names = {"--stash-path"}, description = "Target path for stashing partition data.", required = true)
+ public String stashPath = null;
+
+ @Parameter(names = {"--partitions", "-p"}, description = "Comma separated list of partitions to stash.", required = true)
+ public String partitions = null;
+
+ @Parameter(names = {"--mode", "-m"}, description = "Set job mode: "
+ + "\"stash\" to move partition data to stash location and delete partitions; "
+ + "\"rollback_stash\" to copy stashed data back to table and abort stashing; "
+ + "\"dry_run\" to preview which partitions and files would be affected.", required = true)
+ public String runningMode = null;
+
+ @Parameter(names = {"--parallelism", "-pl"}, description = "Parallelism for hoodie operations.", required = false)
+ public int parallelism = 1500;
+
+ @Parameter(names = {"--rename-helper-class"}, description = "Fully qualified class name of a custom "
+ + "StashPartitionRenameHelper implementation. Defaults to DefaultStashPartitionRenameHelper.", required = false)
+ public String renameHelperClass = null;
+
+ @Parameter(names = {"--spark-master", "-ms"}, description = "Spark master.", required = false)
+ public String sparkMaster = null;
+
+ @Parameter(names = {"--spark-memory", "-sm"}, description = "Spark memory to use.", required = false)
+ public String sparkMemory = "1g";
+
+ @Parameter(names = {"--props"}, description = "Path to properties file on localfs or dfs, with configurations for "
+ + "hoodie client for stashing partitions.")
+ public String propsFilePath = null;
+
+ @Parameter(names = {"--hoodie-conf"}, description = "Any configuration that can be set in the properties file "
+ + "(using the CLI parameter \"--props\") can also be passed via command line using this parameter. This can be repeated.",
+ splitter = IdentitySplitter.class)
+ public List<String> configs = new ArrayList<>();
+
+ @Parameter(names = {"--help", "-h"}, help = true)
+ public Boolean help = false;
+
+ @Override
+ public String toString() {
+ return "HoodieStashPartitionsToolConfig {\n"
+ + " --base-path " + basePath + ", \n"
+ + " --table-name " + tableName + ", \n"
+ + " --stash-path " + stashPath + ", \n"
+ + " --partitions " + partitions + ", \n"
+ + " --mode " + runningMode + ", \n"
+ + " --parallelism " + parallelism + ", \n"
+ + " --rename-helper-class " + renameHelperClass + ", \n"
+ + " --spark-master " + sparkMaster + ", \n"
+ + " --spark-memory " + sparkMemory + ", \n"
+ + " --props " + propsFilePath + ", \n"
+ + " --hoodie-conf " + configs
+ + "\n}";
+ }
+ }
+
+ public static void main(String[] args) {
+ final Config cfg = new Config();
+ JCommander cmd = new JCommander(cfg, null, args);
+ if (cfg.help || args.length == 0) {
+ cmd.usage();
+ System.exit(1);
+ }
+ Map<String, String> sparkConfigMap = new HashMap<>();
+ sparkConfigMap.put("spark.executor.memory", cfg.sparkMemory);
+ JavaSparkContext jsc = UtilHelpers.buildSparkContext("Hoodie-Stash-Partitions",
+ cfg.sparkMaster, false, sparkConfigMap);
+
+ HoodieStashPartitionsTool tool = new HoodieStashPartitionsTool(jsc, cfg);
+ try {
+ tool.run();
+ } catch (Throwable throwable) {
+ LOG.error("Fail to run stash partitions for " + cfg, throwable);
+ } finally {
+ jsc.stop();
+ }
+ }
+
+ public void run() {
+ try {
+ LOG.info(cfg.toString());
+ Mode mode = Mode.valueOf(cfg.runningMode.toUpperCase());
+ switch (mode) {
+ case STASH:
+ LOG.info(" ****** The Hoodie Stash Partitions Tool is in stash mode ****** ");
+ doStash();
+ break;
+ case ROLLBACK_STASH:
+ LOG.info(" ****** The Hoodie Stash Partitions Tool is in rollback_stash mode ****** ");
+ doRollbackStash();
+ break;
+ case DRY_RUN:
+ LOG.info(" ****** The Hoodie Stash Partitions Tool is in dry-run mode ****** ");
+ doDryRun();
+ break;
+ default:
+ LOG.info("Unsupported running mode [" + cfg.runningMode + "], quit the job directly");
+ }
+ } catch (Exception e) {
+ throw new HoodieException("Unable to stash partitions in " + cfg.basePath, e);
+ }
+ }
+
+ /**
+ * Stash partitions: pre-check to recover any partial stash from a prior failed attempt,
+ * then issue deletePartitions with custom pre-commit validator. Hudi internally handles
+ * rollback of any prior failed instants. The pre-commit validator handles already-stashed
+ * partitions gracefully (source empty -> skip).
+ *
+ * <p>The pre-check runs regardless of whether MDT is enabled. Even with MDT, a prior failed
+ * attempt may have had its pre-commit validator move files to the stash location before
+ * crashing. Those files need to be moved back so that the subsequent deletePartitions
+ * sees the complete partition state.
+ */
+ private void doStash() {
+ List<String> partitionsToStash = Arrays.asList(cfg.partitions.split(","));
+
+ preCheckAndRecoverPartialStash(partitionsToStash);
+
+ // Set up props with stash-specific configs for the pre-commit validator
+ TypedProperties stashProps = new TypedProperties();
+ stashProps.putAll(props);
+ stashProps.setProperty(HoodiePreCommitValidatorConfig.VALIDATOR_CLASS_NAMES.key(),
+ StashPartitionsPreCommitValidator.class.getName());
+ stashProps.setProperty(StashPartitionsPreCommitValidator.STASH_PATH_CONFIG, cfg.stashPath);
+ if (!StringUtils.isNullOrEmpty(cfg.renameHelperClass)) {
+ stashProps.setProperty(StashPartitionsPreCommitValidator.RENAME_HELPER_CLASS_CONFIG, cfg.renameHelperClass);
+ }
+
+ try (SparkRDDWriteClient<HoodieRecordPayload> client =
+ UtilHelpers.createHoodieClient(jsc, cfg.basePath, "", cfg.parallelism, Option.empty(), stashProps)) {
+ String instantTime = client.startCommit(HoodieTimeline.REPLACE_COMMIT_ACTION);
+ LOG.info("Starting deletePartitions with instant {} for partitions: {}", instantTime, partitionsToStash);
+ HoodieWriteResult result = client.deletePartitions(partitionsToStash, instantTime);
+ client.commit(instantTime, result.getWriteStatuses(), Option.empty(), HoodieTimeline.REPLACE_COMMIT_ACTION,
+ result.getPartitionToReplaceFileIds(), Option.empty());
+ LOG.info("Successfully stashed partitions: {}", partitionsToStash);
+ }
+ }
+
+ /**
+ * Pre-check: if files exist in the stash location from a prior failed attempt,
+ * copy them back to source before retrying deletePartitions. This ensures the
+ * subsequent deletePartitions sees the complete partition state and creates
+ * a correct replace commit. Runs for both MDT and non-MDT tables, since the
+ * pre-commit validator may have moved files before crashing.
+ */
+ private void preCheckAndRecoverPartialStash(List<String> partitions) {
+ LOG.info("Running pre-check for partial stash recovery (non-MDT table).");
+ HoodieStorage storage = HoodieStorageUtils.getStorage(
+ cfg.basePath, metaClient.getStorageConf());
+ StashPartitionRenameHelper renameHelper = createRenameHelper();
+ StoragePath basePath = metaClient.getBasePath();
+
+ for (String partition : partitions) {
+ StoragePath stashPartitionPath = new StoragePath(cfg.stashPath, partition);
+ StoragePath sourcePartitionPath = new StoragePath(basePath, partition);
+
+ try {
+ if (!storage.exists(stashPartitionPath) || isDirectoryEmpty(storage, stashPartitionPath)) {
+ LOG.info("Partition {} has no stashed data from prior attempt, skipping pre-check.", partition);
+ continue;
+ }
+ // Files exist in stash — copy them back to source to ensure deletePartitions
+ // sees all files and creates a complete replace commit.
+ LOG.info("Found stashed files for partition {} from prior attempt. Moving back to source.", partition);
+ renameHelper.movePartitionFiles(storage, stashPartitionPath, sourcePartitionPath);
+ LOG.info("Recovered partition {} back to source.", partition);
+ } catch (IOException e) {
+ throw new HoodieException("Failed to recover partition " + partition + " from stash during pre-check.", e);
+ }
+ }
+ }
+
+ /**
+ * Rollback stash: copy any data from stash location back to original partition paths.
+ * This is for when the user no longer wants to proceed with stashing.
+ */
+ private void doRollbackStash() {
+ List<String> partitions = Arrays.asList(cfg.partitions.split(","));
+ HoodieStorage storage = HoodieStorageUtils.getStorage(
+ cfg.basePath, metaClient.getStorageConf());
+ StashPartitionRenameHelper renameHelper = createRenameHelper();
+ StoragePath basePath = metaClient.getBasePath();
+
+ for (String partition : partitions) {
+ StoragePath stashPartitionPath = new StoragePath(cfg.stashPath, partition);
+ StoragePath sourcePartitionPath = new StoragePath(basePath, partition);
+
+ try {
+ if (!storage.exists(stashPartitionPath) || isDirectoryEmpty(storage, stashPartitionPath)) {
+ LOG.info("No stashed data found for partition {}, skipping.", partition);
+ continue;
+ }
+ LOG.info("Rolling back stash for partition {}: moving from {} to {}", partition, stashPartitionPath, sourcePartitionPath);
+ renameHelper.movePartitionFiles(storage, stashPartitionPath, sourcePartitionPath);
+ LOG.info("Successfully rolled back stash for partition {}.", partition);
+ } catch (IOException e) {
+ throw new HoodieException("Failed to rollback stash for partition: " + partition, e);
+ }
+ }
+ LOG.info("Stash rollback complete.");
+ }
+
+ /**
+ * Dry run: show partitions and file groups that would be affected.
+ */
+ private void doDryRun() {
+ try (SparkRDDWriteClient<HoodieRecordPayload> client =
+ UtilHelpers.createHoodieClient(jsc, cfg.basePath, "", cfg.parallelism, Option.empty(), props)) {
+ org.apache.hudi.table.HoodieSparkTable<HoodieRecordPayload> table =
+ org.apache.hudi.table.HoodieSparkTable.create(client.getConfig(), client.getEngineContext());
+ List<String> parts = Arrays.asList(cfg.partitions.split(","));
+ HoodieStorage storage = HoodieStorageUtils.getStorage(
+ cfg.basePath, metaClient.getStorageConf());
+ StoragePath basePath = metaClient.getBasePath();
+
+ LOG.info("=== Dry Run: Partitions and files that would be stashed ===");
+ for (String partition : parts) {
+ StoragePath sourcePartitionPath = new StoragePath(basePath, partition);
+ StoragePath stashPartitionPath = new StoragePath(cfg.stashPath, partition);
+
+ List<String> fileIds = table.getSliceView()
+ .getLatestFileSlices(partition)
+ .map(fs -> fs.getFileId())
+ .distinct()
+ .collect(Collectors.toList());
+
+ boolean sourceExists;
+ boolean stashExists;
+ try {
+ sourceExists = storage.exists(sourcePartitionPath) && !isDirectoryEmpty(storage, sourcePartitionPath);
+ stashExists = storage.exists(stashPartitionPath) && !isDirectoryEmpty(storage, stashPartitionPath);
+ } catch (IOException e) {
+ throw new HoodieException("Failed to check paths for partition: " + partition, e);
+ }
+
+ LOG.info("Partition: {}", partition);
+ LOG.info(" Source path: {} (exists={})", sourcePartitionPath, sourceExists);
+ LOG.info(" Stash path: {} (exists={})", stashPartitionPath, stashExists);
+ LOG.info(" File group IDs: {}", fileIds);
+ }
+ }
+ }
+
+ private boolean isDirectoryEmpty(HoodieStorage storage, StoragePath path) throws IOException {
+ List<StoragePathInfo> entries = storage.listDirectEntries(path);
+ return entries.isEmpty();
+ }
+
+ private StashPartitionRenameHelper createRenameHelper() {
Review Comment:
sorry, not sure what do you mean by this. the pre commit validator gets instantiated using reflection. So, we cannot instantiate a renameHelper in our tool and make the preCommitValidator use the same instance.
or did you mean something else
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-4300644388
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `23.05389%` with `257 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 58.61%. Comparing base ([`7bcb8be`](https://app.codecov.io/gh/apache/hudi/commit/7bcb8be7c153010998574f591c7ff3799192eda6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`6d8dedd`](https://app.codecov.io/gh/apache/hudi/commit/6d8dedd061ed183c52471df09355b85e5a60099a?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 21 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | 6.46% | [245 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | 77.50% | [5 Missing and 4 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | 93.54% | [1 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
> :exclamation: There is a different number of reports uploaded between BASE (7bcb8be) and HEAD (6d8dedd). Click for more details.
>
> <details><summary>HEAD has 16 uploads less than BASE</summary>
>
>| Flag | BASE (7bcb8be) | HEAD (6d8dedd) |
>|------|------|------|
>|spark-scala-tests|10|5|
>|spark-java-tests|15|5|
>|utilities|1|0|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18526 +/- ##
=============================================
- Coverage 68.88% 58.61% -10.27%
+ Complexity 28279 24322 -3957
=============================================
Files 2464 2478 +14
Lines 135568 136958 +1390
Branches 16446 16651 +205
=============================================
- Hits 93385 80280 -13105
- Misses 34801 49794 +14993
+ Partials 7382 6884 -498
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.39% <13.77%> (-0.25%)` | :arrow_down: |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.74% <0.00%> (-0.02%)` | :arrow_down: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.64% <77.46%> (+0.23%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `32.18% <0.00%> (-16.75%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `32.36% <0.00%> (-13.09%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | `93.54% <93.54%> (ø)` | |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | `77.50% <77.50%> (ø)` | |
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | `6.46% <6.46%> (ø)` | |
... and [682 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18526/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-4300627591
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `16.46707%` with `279 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 47.87%. Comparing base ([`7bcb8be`](https://app.codecov.io/gh/apache/hudi/commit/7bcb8be7c153010998574f591c7ff3799192eda6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`6d8dedd`](https://app.codecov.io/gh/apache/hudi/commit/6d8dedd061ed183c52471df09355b85e5a60099a?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 21 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | 0.00% | [263 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | 77.50% | [5 Missing and 4 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | 77.41% | [5 Missing and 2 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
> :exclamation: There is a different number of reports uploaded between BASE (7bcb8be) and HEAD (6d8dedd). Click for more details.
>
> <details><summary>HEAD has 19 uploads less than BASE</summary>
>
>| Flag | BASE (7bcb8be) | HEAD (6d8dedd) |
>|------|------|------|
>|spark-scala-tests|10|5|
>|spark-java-tests|15|3|
>|common-and-other-modules|1|0|
>|utilities|1|0|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18526 +/- ##
=============================================
- Coverage 68.88% 47.87% -21.01%
+ Complexity 28279 16593 -11686
=============================================
Files 2464 2018 -446
Lines 135568 113561 -22007
Branches 16446 14458 -1988
=============================================
- Hits 93385 54371 -39014
- Misses 34801 53487 +18686
+ Partials 7382 5703 -1679
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.74% <0.00%> (-0.02%)` | :arrow_down: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.64% <77.46%> (+0.23%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `32.51% <0.00%> (-16.42%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `32.36% <0.00%> (-13.09%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | `77.41% <77.41%> (ø)` | |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | `77.50% <77.50%> (ø)` | |
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | `0.00% <0.00%> (ø)` | |
... and [1545 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18526/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-4300657695
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `23.05389%` with `257 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 65.57%. Comparing base ([`7bcb8be`](https://app.codecov.io/gh/apache/hudi/commit/7bcb8be7c153010998574f591c7ff3799192eda6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`6d8dedd`](https://app.codecov.io/gh/apache/hudi/commit/6d8dedd061ed183c52471df09355b85e5a60099a?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 21 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | 6.46% | [245 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | 77.50% | [5 Missing and 4 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | 93.54% | [1 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18526 +/- ##
============================================
- Coverage 68.88% 65.57% -3.32%
+ Complexity 28279 27174 -1105
============================================
Files 2464 2478 +14
Lines 135568 136958 +1390
Branches 16446 16651 +205
============================================
- Hits 93385 89804 -3581
- Misses 34801 39786 +4985
+ Partials 7382 7368 -14
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.39% <13.77%> (-0.25%)` | :arrow_down: |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.74% <0.00%> (-0.02%)` | :arrow_down: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.64% <77.46%> (+0.23%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `42.74% <0.00%> (-6.18%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.62% <0.00%> (-0.83%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | `93.54% <93.54%> (ø)` | |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | `77.50% <77.50%> (ø)` | |
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | `6.46% <6.46%> (ø)` | |
... and [349 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18526/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-4988453501
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `80.83832%` with `64 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 68.68%. Comparing base ([`7bcb8be`](https://app.codecov.io/gh/apache/hudi/commit/7bcb8be7c153010998574f591c7ff3799192eda6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`6d8dedd`](https://app.codecov.io/gh/apache/hudi/commit/6d8dedd061ed183c52471df09355b85e5a60099a?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 347 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | 79.84% | [43 Missing and 10 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | 77.50% | [5 Missing and 4 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | 93.54% | [1 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18526 +/- ##
============================================
- Coverage 68.88% 68.68% -0.21%
- Complexity 28279 28473 +194
============================================
Files 2464 2478 +14
Lines 135568 136972 +1404
Branches 16446 16653 +207
============================================
+ Hits 93385 94077 +692
- Misses 34801 35324 +523
- Partials 7382 7571 +189
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.39% <13.77%> (-0.25%)` | :arrow_down: |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.74% <0.00%> (-0.02%)` | :arrow_down: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.64% <77.46%> (+0.23%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `49.25% <0.00%> (+0.32%)` | :arrow_up: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.18% <0.00%> (-0.26%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `37.87% <77.54%> (-0.30%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | `93.54% <93.54%> (ø)` | |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | `77.50% <77.50%> (ø)` | |
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | `79.84% <79.84%> (ø)` | |
... and [79 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18526/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-6407354430
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `80.83832%` with `64 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 68.68%. Comparing base ([`7bcb8be`](https://app.codecov.io/gh/apache/hudi/commit/7bcb8be7c153010998574f591c7ff3799192eda6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`6d8dedd`](https://app.codecov.io/gh/apache/hudi/commit/6d8dedd061ed183c52471df09355b85e5a60099a?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 802 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | 79.84% | [43 Missing and 10 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | 77.50% | [5 Missing and 4 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | 93.54% | [1 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18526 +/- ##
============================================
- Coverage 68.88% 68.68% -0.21%
- Complexity 28279 28473 +194
============================================
Files 2464 2478 +14
Lines 135568 136972 +1404
Branches 16446 16653 +207
============================================
+ Hits 93385 94077 +692
- Misses 34801 35324 +523
- Partials 7382 7571 +189
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.39% <13.77%> (-0.25%)` | :arrow_down: |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.74% <0.00%> (-0.02%)` | :arrow_down: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.64% <77.46%> (+0.23%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `49.25% <0.00%> (+0.32%)` | :arrow_up: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.18% <0.00%> (-0.26%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `37.87% <77.54%> (-0.30%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | `93.54% <93.54%> (ø)` | |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | `77.50% <77.50%> (ø)` | |
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | `79.84% <79.84%> (ø)` | |
... and [79 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18526/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
- :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "hudi-agent (via GitHub)" <gi...@apache.org>.
hudi-agent commented on code in PR #18526:
URL: https://github.com/apache/hudi/pull/18526#discussion_r3113645340
##########
hudi-utilities/src/main/java/org/apache/hudi/utilities/HoodieStashPartitionsTool.java:
##########
@@ -0,0 +1,403 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.hudi.utilities;
+
+import org.apache.hudi.client.HoodieWriteResult;
+import org.apache.hudi.client.SparkRDDWriteClient;
+import org.apache.hudi.client.validator.DefaultStashPartitionRenameHelper;
+import org.apache.hudi.client.validator.StashPartitionRenameHelper;
+import org.apache.hudi.client.validator.StashPartitionsPreCommitValidator;
+import org.apache.hudi.common.config.TypedProperties;
+import org.apache.hudi.common.model.HoodieRecordPayload;
+import org.apache.hudi.common.table.HoodieTableMetaClient;
+import org.apache.hudi.common.table.timeline.HoodieTimeline;
+import org.apache.hudi.common.util.Option;
+import org.apache.hudi.common.util.ReflectionUtils;
+import org.apache.hudi.common.util.StringUtils;
+import org.apache.hudi.config.HoodiePreCommitValidatorConfig;
+import org.apache.hudi.exception.HoodieException;
+import org.apache.hudi.hadoop.fs.HadoopFSUtils;
+import org.apache.hudi.storage.HoodieStorage;
+import org.apache.hudi.storage.HoodieStorageUtils;
+import org.apache.hudi.storage.StoragePath;
+import org.apache.hudi.storage.StoragePathInfo;
+
+import com.beust.jcommander.JCommander;
+import com.beust.jcommander.Parameter;
+import org.apache.hadoop.fs.Path;
+import org.apache.spark.api.java.JavaSparkContext;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.IOException;
+import java.io.Serializable;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+
+/**
+ * A spark-submit tool for stashing (backing up and removing) Hudi table partitions.
+ *
+ * <p>This tool supports three modes:
+ * <ul>
+ * <li><b>stash</b> — Moves data files from specified partitions to a stash location
+ * and issues a deletePartitions operation on the Hudi table. Uses a custom
+ * {@link StashPartitionsPreCommitValidator} to move files atomically before commit.</li>
+ * <li><b>rollback_stash</b> — Copies any data found in the stash location back to the
+ * original partition paths. Used when the user no longer wants to proceed with stashing.</li>
+ * <li><b>dry_run</b> — Shows which partitions and file groups would be affected.</li>
+ * </ul>
+ *
+ * <h3>Why a pre-commit validator instead of moving files in the tool?</h3>
+ *
+ * <p>A naive approach would be to first run {@code deletePartitions} in the tool, and then
+ * move the data files to the stash location as a separate step. However, this is unsafe:
+ * if the tool crashes or fails after {@code deletePartitions} succeeds but before the data
+ * is moved to the stash location, the data is at risk. Once the replace commit from
+ * {@code deletePartitions} lands on the timeline, the Hudi cleaner will eventually delete
+ * the physical data files for the replaced file groups based on its configured retention
+ * policy (e.g., retain last N commits). If the cleaner runs before the tool is retried,
+ * the data intended for stashing is permanently lost.
+ *
+ * <p>By using a {@link StashPartitionsPreCommitValidator}, the file move happens
+ * <b>before</b> the {@code deletePartitions} commit is finalized. If the move fails, the
+ * commit does not land, and the table remains in its original state — the cleaner has
+ * nothing to clean. This ensures that data is never in a state where it has been marked
+ * as deleted (via a replace commit) but has not yet been safely copied to the stash location.
+ *
+ * <p>For restoring stashed partitions back into the Hudi table, use the standard
+ * {@code insert_overwrite} write operation reading from the stash location.
+ *
+ * <p>Example usage:
+ * <pre>
+ * spark-submit \
+ * --class org.apache.hudi.utilities.HoodieStashPartitionsTool \
+ * --master local[*] \
+ * --driver-memory 1g \
+ * --executor-memory 1g \
+ * hudi-utilities-bundle.jar \
+ * --base-path /path/to/hudi/table \
+ * --table-name myTable \
+ * --stash-path /path/to/stash/folder \
+ * --partitions datestr=2023-01-01,datestr=2023-01-02 \
+ * --mode stash
+ * </pre>
+ */
+public class HoodieStashPartitionsTool implements Serializable {
+
+ private static final long serialVersionUID = 1L;
+ private static final Logger LOG = LoggerFactory.getLogger(HoodieStashPartitionsTool.class);
+
+ private final transient JavaSparkContext jsc;
+ private final Config cfg;
+ private final TypedProperties props;
+ private final HoodieTableMetaClient metaClient;
+
+ public HoodieStashPartitionsTool(JavaSparkContext jsc, Config cfg) {
+ this.jsc = jsc;
+ this.cfg = cfg;
+ this.props = cfg.propsFilePath == null
+ ? UtilHelpers.buildProperties(cfg.configs)
+ : readConfigFromFileSystem(jsc, cfg);
+ this.metaClient = HoodieTableMetaClient.builder()
+ .setConf(HadoopFSUtils.getStorageConfWithCopy(jsc.hadoopConfiguration()))
+ .setBasePath(cfg.basePath)
+ .setLoadActiveTimelineOnLoad(true)
+ .build();
+ }
+
+ public enum Mode {
+ STASH,
+ ROLLBACK_STASH,
+ DRY_RUN
+ }
+
+ public static class Config implements Serializable {
+ @Parameter(names = {"--base-path", "-sp"}, description = "Base path for the Hudi table.", required = true)
+ public String basePath = null;
+
+ @Parameter(names = {"--table-name", "-tn"}, description = "Table name.", required = true)
+ public String tableName = null;
+
+ @Parameter(names = {"--stash-path"}, description = "Target path for stashing partition data.", required = true)
+ public String stashPath = null;
+
+ @Parameter(names = {"--partitions", "-p"}, description = "Comma separated list of partitions to stash.", required = true)
+ public String partitions = null;
+
+ @Parameter(names = {"--mode", "-m"}, description = "Set job mode: "
+ + "\"stash\" to move partition data to stash location and delete partitions; "
+ + "\"rollback_stash\" to copy stashed data back to table and abort stashing; "
+ + "\"dry_run\" to preview which partitions and files would be affected.", required = true)
+ public String runningMode = null;
+
+ @Parameter(names = {"--parallelism", "-pl"}, description = "Parallelism for hoodie operations.", required = false)
+ public int parallelism = 1500;
+
+ @Parameter(names = {"--rename-helper-class"}, description = "Fully qualified class name of a custom "
+ + "StashPartitionRenameHelper implementation. Defaults to DefaultStashPartitionRenameHelper.", required = false)
+ public String renameHelperClass = null;
+
+ @Parameter(names = {"--spark-master", "-ms"}, description = "Spark master.", required = false)
+ public String sparkMaster = null;
+
+ @Parameter(names = {"--spark-memory", "-sm"}, description = "Spark memory to use.", required = false)
+ public String sparkMemory = "1g";
+
+ @Parameter(names = {"--props"}, description = "Path to properties file on localfs or dfs, with configurations for "
+ + "hoodie client for stashing partitions.")
+ public String propsFilePath = null;
+
+ @Parameter(names = {"--hoodie-conf"}, description = "Any configuration that can be set in the properties file "
+ + "(using the CLI parameter \"--props\") can also be passed via command line using this parameter. This can be repeated.",
+ splitter = IdentitySplitter.class)
+ public List<String> configs = new ArrayList<>();
+
+ @Parameter(names = {"--help", "-h"}, help = true)
+ public Boolean help = false;
+
+ @Override
+ public String toString() {
+ return "HoodieStashPartitionsToolConfig {\n"
+ + " --base-path " + basePath + ", \n"
+ + " --table-name " + tableName + ", \n"
+ + " --stash-path " + stashPath + ", \n"
+ + " --partitions " + partitions + ", \n"
+ + " --mode " + runningMode + ", \n"
+ + " --parallelism " + parallelism + ", \n"
+ + " --rename-helper-class " + renameHelperClass + ", \n"
+ + " --spark-master " + sparkMaster + ", \n"
+ + " --spark-memory " + sparkMemory + ", \n"
+ + " --props " + propsFilePath + ", \n"
+ + " --hoodie-conf " + configs
+ + "\n}";
+ }
+ }
+
+ public static void main(String[] args) {
+ final Config cfg = new Config();
+ JCommander cmd = new JCommander(cfg, null, args);
+ if (cfg.help || args.length == 0) {
+ cmd.usage();
+ System.exit(1);
+ }
+ Map<String, String> sparkConfigMap = new HashMap<>();
+ sparkConfigMap.put("spark.executor.memory", cfg.sparkMemory);
+ JavaSparkContext jsc = UtilHelpers.buildSparkContext("Hoodie-Stash-Partitions",
+ cfg.sparkMaster, false, sparkConfigMap);
+
+ HoodieStashPartitionsTool tool = new HoodieStashPartitionsTool(jsc, cfg);
+ try {
+ tool.run();
+ } catch (Throwable throwable) {
+ LOG.error("Fail to run stash partitions for " + cfg, throwable);
+ } finally {
+ jsc.stop();
+ }
+ }
+
+ public void run() {
+ try {
+ LOG.info(cfg.toString());
+ Mode mode = Mode.valueOf(cfg.runningMode.toUpperCase());
+ switch (mode) {
+ case STASH:
+ LOG.info(" ****** The Hoodie Stash Partitions Tool is in stash mode ****** ");
+ doStash();
+ break;
+ case ROLLBACK_STASH:
+ LOG.info(" ****** The Hoodie Stash Partitions Tool is in rollback_stash mode ****** ");
+ doRollbackStash();
+ break;
+ case DRY_RUN:
+ LOG.info(" ****** The Hoodie Stash Partitions Tool is in dry-run mode ****** ");
+ doDryRun();
+ break;
+ default:
+ LOG.info("Unsupported running mode [" + cfg.runningMode + "], quit the job directly");
+ }
+ } catch (Exception e) {
+ throw new HoodieException("Unable to stash partitions in " + cfg.basePath, e);
+ }
+ }
+
+ /**
+ * Stash partitions: pre-check to recover any partial stash from a prior failed attempt,
+ * then issue deletePartitions with custom pre-commit validator. Hudi internally handles
+ * rollback of any prior failed instants. The pre-commit validator handles already-stashed
+ * partitions gracefully (source empty -> skip).
+ *
+ * <p>The pre-check runs regardless of whether MDT is enabled. Even with MDT, a prior failed
+ * attempt may have had its pre-commit validator move files to the stash location before
+ * crashing. Those files need to be moved back so that the subsequent deletePartitions
+ * sees the complete partition state.
+ */
+ private void doStash() {
+ List<String> partitionsToStash = Arrays.asList(cfg.partitions.split(","));
+
+ preCheckAndRecoverPartialStash(partitionsToStash);
+
+ // Set up props with stash-specific configs for the pre-commit validator
+ TypedProperties stashProps = new TypedProperties();
+ stashProps.putAll(props);
+ stashProps.setProperty(HoodiePreCommitValidatorConfig.VALIDATOR_CLASS_NAMES.key(),
+ StashPartitionsPreCommitValidator.class.getName());
+ stashProps.setProperty(StashPartitionsPreCommitValidator.STASH_PATH_CONFIG, cfg.stashPath);
+ if (!StringUtils.isNullOrEmpty(cfg.renameHelperClass)) {
+ stashProps.setProperty(StashPartitionsPreCommitValidator.RENAME_HELPER_CLASS_CONFIG, cfg.renameHelperClass);
+ }
+
+ try (SparkRDDWriteClient<HoodieRecordPayload> client =
+ UtilHelpers.createHoodieClient(jsc, cfg.basePath, "", cfg.parallelism, Option.empty(), stashProps)) {
+ String instantTime = client.startCommit(HoodieTimeline.REPLACE_COMMIT_ACTION);
+ LOG.info("Starting deletePartitions with instant {} for partitions: {}", instantTime, partitionsToStash);
+ HoodieWriteResult result = client.deletePartitions(partitionsToStash, instantTime);
+ client.commit(instantTime, result.getWriteStatuses(), Option.empty(), HoodieTimeline.REPLACE_COMMIT_ACTION,
+ result.getPartitionToReplaceFileIds(), Option.empty());
+ LOG.info("Successfully stashed partitions: {}", partitionsToStash);
+ }
+ }
+
+ /**
+ * Pre-check: if files exist in the stash location from a prior failed attempt,
+ * copy them back to source before retrying deletePartitions. This ensures the
+ * subsequent deletePartitions sees the complete partition state and creates
+ * a correct replace commit. Runs for both MDT and non-MDT tables, since the
+ * pre-commit validator may have moved files before crashing.
+ */
+ private void preCheckAndRecoverPartialStash(List<String> partitions) {
+ LOG.info("Running pre-check for partial stash recovery (non-MDT table).");
+ HoodieStorage storage = HoodieStorageUtils.getStorage(
+ cfg.basePath, metaClient.getStorageConf());
+ StashPartitionRenameHelper renameHelper = createRenameHelper();
+ StoragePath basePath = metaClient.getBasePath();
+
+ for (String partition : partitions) {
+ StoragePath stashPartitionPath = new StoragePath(cfg.stashPath, partition);
+ StoragePath sourcePartitionPath = new StoragePath(basePath, partition);
+
+ try {
+ if (!storage.exists(stashPartitionPath) || isDirectoryEmpty(storage, stashPartitionPath)) {
+ LOG.info("Partition {} has no stashed data from prior attempt, skipping pre-check.", partition);
+ continue;
+ }
+ // Files exist in stash — copy them back to source to ensure deletePartitions
+ // sees all files and creates a complete replace commit.
+ LOG.info("Found stashed files for partition {} from prior attempt. Moving back to source.", partition);
+ renameHelper.movePartitionFiles(storage, stashPartitionPath, sourcePartitionPath);
+ LOG.info("Recovered partition {} back to source.", partition);
+ } catch (IOException e) {
+ throw new HoodieException("Failed to recover partition " + partition + " from stash during pre-check.", e);
+ }
+ }
+ }
Review Comment:
_⚠️ Potential issue_ | _🟡 Minor_
**Fix misleading log message.**
Line 289 says "(non-MDT table)" but the method's Javadoc (lines 285-286) explicitly states this runs "for both MDT and non-MDT tables." The log message is incorrect and could confuse operators reviewing logs.
<details>
<summary>🔧 Proposed fix</summary>
```diff
private void preCheckAndRecoverPartialStash(List<String> partitions) {
- LOG.info("Running pre-check for partial stash recovery (non-MDT table).");
+ LOG.info("Running pre-check for partial stash recovery.");
HoodieStorage storage = HoodieStorageUtils.getStorage(
```
</details>
<!-- suggestion_start -->
<details>
<summary>📝 Committable suggestion</summary>
> ‼️ **IMPORTANT**
> Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
```suggestion
private void preCheckAndRecoverPartialStash(List<String> partitions) {
LOG.info("Running pre-check for partial stash recovery.");
HoodieStorage storage = HoodieStorageUtils.getStorage(
cfg.basePath, metaClient.getStorageConf());
StashPartitionRenameHelper renameHelper = createRenameHelper();
StoragePath basePath = metaClient.getBasePath();
for (String partition : partitions) {
StoragePath stashPartitionPath = new StoragePath(cfg.stashPath, partition);
StoragePath sourcePartitionPath = new StoragePath(basePath, partition);
try {
if (!storage.exists(stashPartitionPath) || isDirectoryEmpty(storage, stashPartitionPath)) {
LOG.info("Partition {} has no stashed data from prior attempt, skipping pre-check.", partition);
continue;
}
// Files exist in stash — copy them back to source to ensure deletePartitions
// sees all files and creates a complete replace commit.
LOG.info("Found stashed files for partition {} from prior attempt. Moving back to source.", partition);
renameHelper.movePartitionFiles(storage, stashPartitionPath, sourcePartitionPath);
LOG.info("Recovered partition {} back to source.", partition);
} catch (IOException e) {
throw new HoodieException("Failed to recover partition " + partition + " from stash during pre-check.", e);
}
}
}
```
</details>
<!-- suggestion_end -->
<details>
<summary>🤖 Prompt for AI Agents</summary>
```
Verify each finding against the current code and only fix it if needed.
In
`@hudi-utilities/src/main/java/org/apache/hudi/utilities/HoodieStashPartitionsTool.java`
around lines 288 - 313, Update the misleading log message in
preCheckAndRecoverPartialStash: replace the current LOG.info("Running pre-check
for partial stash recovery (non-MDT table).") with a message that reflects this
runs for both MDT and non-MDT tables (e.g., "Running pre-check for partial stash
recovery (MDT and non-MDT tables).") so it matches the method Javadoc; ensure
you change the LOG.info call in the
HoodieStashPartitionsTool.preCheckAndRecoverPartialStash method and nothing
else.
```
</details>
<!-- fingerprinting:phantom:medusa:ocelot:c14c17bc-2986-49b3-8206-d8c65ea2a4a3 -->
<!-- This is an auto-generated comment by CodeRabbit -->
— *CodeRabbit* ([original](https://github.com/hudi-agent/hudi/pull/8#discussion_r3113644201)) (source:comment#3113644201)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-4284256403
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "67f1e57f0dffe02bbe2d7bacfaa8f1aaa4be8c88",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=13351",
"triggerID" : "67f1e57f0dffe02bbe2d7bacfaa8f1aaa4be8c88",
"triggerType" : "PUSH"
}, {
"hash" : "e8b5a0a62cea99be0a184e0b6f11149b49850806",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=13354",
"triggerID" : "e8b5a0a62cea99be0a184e0b6f11149b49850806",
"triggerType" : "PUSH"
}, {
"hash" : "891eaf2e1427bf93b9c7ff9626628c30e1c9c01d",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=13391",
"triggerID" : "891eaf2e1427bf93b9c7ff9626628c30e1c9c01d",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* e8b5a0a62cea99be0a184e0b6f11149b49850806 Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=13354)
* 891eaf2e1427bf93b9c7ff9626628c30e1c9c01d Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=13391)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-4284528917
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `78.47534%` with `48 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 67.99%. Comparing base ([`7bcb8be`](https://app.codecov.io/gh/apache/hudi/commit/7bcb8be7c153010998574f591c7ff3799192eda6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`891eaf2`](https://app.codecov.io/gh/apache/hudi/commit/891eaf2e1427bf93b9c7ff9626628c30e1c9c01d?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 11 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | 76.92% | [28 Missing and 8 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | 75.00% | [6 Missing and 4 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | 92.59% | [1 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18526 +/- ##
============================================
- Coverage 68.88% 67.99% -0.89%
+ Complexity 28279 28048 -231
============================================
Files 2464 2471 +7
Lines 135568 136218 +650
Branches 16446 16521 +75
============================================
- Hits 93385 92628 -757
- Misses 34801 36113 +1312
- Partials 7382 7477 +95
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.56% <11.21%> (-0.08%)` | :arrow_down: |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.75% <0.00%> (+<0.01%)` | :arrow_up: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.47% <77.61%> (+0.07%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.19% <0.00%> (-3.73%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.34% <0.00%> (-0.10%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `38.30% <74.88%> (+0.13%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | `92.59% <92.59%> (ø)` | |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | `75.00% <75.00%> (ø)` | |
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | `76.92% <76.92%> (ø)` | |
... and [156 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18526/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-4284525503
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `78.47534%` with `48 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 67.97%. Comparing base ([`7bcb8be`](https://app.codecov.io/gh/apache/hudi/commit/7bcb8be7c153010998574f591c7ff3799192eda6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`891eaf2`](https://app.codecov.io/gh/apache/hudi/commit/891eaf2e1427bf93b9c7ff9626628c30e1c9c01d?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 11 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | 76.92% | [28 Missing and 8 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | 75.00% | [6 Missing and 4 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | 92.59% | [1 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18526 +/- ##
============================================
- Coverage 68.88% 67.97% -0.91%
+ Complexity 28279 28040 -239
============================================
Files 2464 2471 +7
Lines 135568 136218 +650
Branches 16446 16521 +75
============================================
- Hits 93385 92599 -786
- Misses 34801 36140 +1339
- Partials 7382 7479 +97
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.56% <11.21%> (-0.08%)` | :arrow_down: |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.75% <0.00%> (+<0.01%)` | :arrow_up: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.47% <77.61%> (+0.07%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.15% <0.00%> (-3.77%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.34% <0.00%> (-0.10%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `38.30% <74.88%> (+0.13%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | `92.59% <92.59%> (ø)` | |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | `75.00% <75.00%> (ø)` | |
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | `76.92% <76.92%> (ø)` | |
... and [163 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18526/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-4284443953
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `23.31839%` with `171 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 47.91%. Comparing base ([`7bcb8be`](https://app.codecov.io/gh/apache/hudi/commit/7bcb8be7c153010998574f591c7ff3799192eda6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`891eaf2`](https://app.codecov.io/gh/apache/hudi/commit/891eaf2e1427bf93b9c7ff9626628c30e1c9c01d?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 11 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | 0.00% | [156 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | 75.00% | [6 Missing and 4 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | 81.48% | [3 Missing and 2 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
> :exclamation: There is a different number of reports uploaded between BASE (7bcb8be) and HEAD (891eaf2). Click for more details.
>
> <details><summary>HEAD has 21 uploads less than BASE</summary>
>
>| Flag | BASE (7bcb8be) | HEAD (891eaf2) |
>|------|------|------|
>|spark-scala-tests|10|5|
>|spark-java-tests|15|1|
>|common-and-other-modules|1|0|
>|utilities|1|0|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18526 +/- ##
=============================================
- Coverage 68.88% 47.91% -20.97%
+ Complexity 28279 16500 -11779
=============================================
Files 2464 2011 -453
Lines 135568 112821 -22747
Branches 16446 14328 -2118
=============================================
- Hits 93385 54060 -39325
- Misses 34801 53079 +18278
+ Partials 7382 5682 -1700
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.75% <0.00%> (+<0.01%)` | :arrow_up: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.47% <77.61%> (+0.07%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `33.47% <0.00%> (-15.46%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `32.48% <0.00%> (-12.96%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | `81.48% <81.48%> (ø)` | |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | `75.00% <75.00%> (ø)` | |
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | `0.00% <0.00%> (ø)` | |
... and [1535 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18526/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-4284612246
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `78.47534%` with `48 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 68.79%. Comparing base ([`7bcb8be`](https://app.codecov.io/gh/apache/hudi/commit/7bcb8be7c153010998574f591c7ff3799192eda6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`891eaf2`](https://app.codecov.io/gh/apache/hudi/commit/891eaf2e1427bf93b9c7ff9626628c30e1c9c01d?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 11 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | 76.92% | [28 Missing and 8 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | 75.00% | [6 Missing and 4 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | 92.59% | [1 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18526 +/- ##
============================================
- Coverage 68.88% 68.79% -0.10%
- Complexity 28279 28361 +82
============================================
Files 2464 2471 +7
Lines 135568 136218 +650
Branches 16446 16521 +75
============================================
+ Hits 93385 93707 +322
- Misses 34801 35095 +294
- Partials 7382 7416 +34
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.56% <11.21%> (-0.08%)` | :arrow_down: |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.75% <0.00%> (+<0.01%)` | :arrow_up: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.47% <77.61%> (+0.07%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.83% <0.00%> (-0.09%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.34% <0.00%> (-0.10%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `38.30% <74.88%> (+0.13%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | `92.59% <92.59%> (ø)` | |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | `75.00% <75.00%> (ø)` | |
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | `76.92% <76.92%> (ø)` | |
... and [25 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18526/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "nsivabalan (via GitHub)" <gi...@apache.org>.
nsivabalan commented on code in PR #18526:
URL: https://github.com/apache/hudi/pull/18526#discussion_r3126664932
##########
hudi-utilities/src/main/java/org/apache/hudi/utilities/HoodieStashPartitionsTool.java:
##########
@@ -0,0 +1,386 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.hudi.utilities;
+
+import org.apache.hudi.client.HoodieWriteResult;
+import org.apache.hudi.client.SparkRDDWriteClient;
+import org.apache.hudi.client.validator.DefaultStashPartitionRenameHelper;
+import org.apache.hudi.client.validator.StashPartitionRenameHelper;
+import org.apache.hudi.client.validator.StashPartitionsPreCommitValidator;
+import org.apache.hudi.common.config.TypedProperties;
+import org.apache.hudi.common.model.HoodieRecordPayload;
+import org.apache.hudi.common.table.HoodieTableMetaClient;
+import org.apache.hudi.common.table.timeline.HoodieTimeline;
+import org.apache.hudi.common.util.Option;
+import org.apache.hudi.common.util.ReflectionUtils;
+import org.apache.hudi.common.util.StringUtils;
+import org.apache.hudi.config.HoodiePreCommitValidatorConfig;
+import org.apache.hudi.exception.HoodieException;
+import org.apache.hudi.hadoop.fs.HadoopFSUtils;
+import org.apache.hudi.storage.HoodieStorage;
+import org.apache.hudi.storage.HoodieStorageUtils;
+import org.apache.hudi.storage.StoragePath;
+import org.apache.hudi.storage.StoragePathInfo;
+
+import com.beust.jcommander.JCommander;
+import com.beust.jcommander.Parameter;
+import org.apache.hadoop.fs.Path;
+import org.apache.spark.api.java.JavaSparkContext;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.IOException;
+import java.io.Serializable;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+
+/**
+ * A spark-submit tool for stashing (backing up and removing) Hudi table partitions.
+ *
+ * <p>This tool supports three modes:
+ * <ul>
+ * <li><b>stash</b> — Moves data files from specified partitions to a stash location
+ * and issues a deletePartitions operation on the Hudi table. Uses a custom
+ * {@link StashPartitionsPreCommitValidator} to move files atomically before commit.</li>
+ * <li><b>rollback_stash</b> — Copies any data found in the stash location back to the
+ * original partition paths. Used when the user no longer wants to proceed with stashing.</li>
+ * <li><b>dry_run</b> — Shows which partitions and file groups would be affected.</li>
+ * </ul>
+ *
+ * <p>For restoring stashed partitions back into the Hudi table, use the standard
+ * {@code insert_overwrite} write operation reading from the stash location.
+ *
+ * <p>Example usage:
+ * <pre>
+ * spark-submit \
+ * --class org.apache.hudi.utilities.HoodieStashPartitionsTool \
+ * --master local[*] \
+ * --driver-memory 1g \
+ * --executor-memory 1g \
+ * hudi-utilities-bundle.jar \
+ * --base-path /path/to/hudi/table \
+ * --table-name myTable \
+ * --stash-path /path/to/stash/folder \
+ * --partitions datestr=2023-01-01,datestr=2023-01-02 \
+ * --mode stash
+ * </pre>
+ */
+public class HoodieStashPartitionsTool implements Serializable {
+
+ private static final long serialVersionUID = 1L;
+ private static final Logger LOG = LoggerFactory.getLogger(HoodieStashPartitionsTool.class);
+
+ private final transient JavaSparkContext jsc;
+ private final Config cfg;
+ private final TypedProperties props;
+ private final HoodieTableMetaClient metaClient;
+
+ public HoodieStashPartitionsTool(JavaSparkContext jsc, Config cfg) {
+ this.jsc = jsc;
+ this.cfg = cfg;
+ this.props = cfg.propsFilePath == null
+ ? UtilHelpers.buildProperties(cfg.configs)
+ : readConfigFromFileSystem(jsc, cfg);
+ this.metaClient = HoodieTableMetaClient.builder()
+ .setConf(HadoopFSUtils.getStorageConfWithCopy(jsc.hadoopConfiguration()))
+ .setBasePath(cfg.basePath)
+ .setLoadActiveTimelineOnLoad(true)
+ .build();
+ }
+
+ public enum Mode {
+ STASH,
+ ROLLBACK_STASH,
+ DRY_RUN
+ }
+
+ public static class Config implements Serializable {
+ @Parameter(names = {"--base-path", "-sp"}, description = "Base path for the Hudi table.", required = true)
+ public String basePath = null;
+
+ @Parameter(names = {"--table-name", "-tn"}, description = "Table name.", required = true)
+ public String tableName = null;
+
+ @Parameter(names = {"--stash-path"}, description = "Target path for stashing partition data.", required = true)
+ public String stashPath = null;
+
+ @Parameter(names = {"--partitions", "-p"}, description = "Comma separated list of partitions to stash.", required = true)
+ public String partitions = null;
+
+ @Parameter(names = {"--mode", "-m"}, description = "Set job mode: "
+ + "\"stash\" to move partition data to stash location and delete partitions; "
+ + "\"rollback_stash\" to copy stashed data back to table and abort stashing; "
+ + "\"dry_run\" to preview which partitions and files would be affected.", required = true)
+ public String runningMode = null;
+
+ @Parameter(names = {"--parallelism", "-pl"}, description = "Parallelism for hoodie operations.", required = false)
+ public int parallelism = 1500;
+
+ @Parameter(names = {"--rename-helper-class"}, description = "Fully qualified class name of a custom "
+ + "StashPartitionRenameHelper implementation. Defaults to DefaultStashPartitionRenameHelper.", required = false)
+ public String renameHelperClass = null;
+
+ @Parameter(names = {"--spark-master", "-ms"}, description = "Spark master.", required = false)
+ public String sparkMaster = null;
+
+ @Parameter(names = {"--spark-memory", "-sm"}, description = "Spark memory to use.", required = false)
+ public String sparkMemory = "1g";
+
+ @Parameter(names = {"--props"}, description = "Path to properties file on localfs or dfs, with configurations for "
+ + "hoodie client for stashing partitions.")
+ public String propsFilePath = null;
+
+ @Parameter(names = {"--hoodie-conf"}, description = "Any configuration that can be set in the properties file "
+ + "(using the CLI parameter \"--props\") can also be passed via command line using this parameter. This can be repeated.",
+ splitter = IdentitySplitter.class)
+ public List<String> configs = new ArrayList<>();
+
+ @Parameter(names = {"--help", "-h"}, help = true)
+ public Boolean help = false;
+
+ @Override
+ public String toString() {
+ return "HoodieStashPartitionsToolConfig {\n"
+ + " --base-path " + basePath + ", \n"
+ + " --table-name " + tableName + ", \n"
+ + " --stash-path " + stashPath + ", \n"
+ + " --partitions " + partitions + ", \n"
+ + " --mode " + runningMode + ", \n"
+ + " --parallelism " + parallelism + ", \n"
+ + " --rename-helper-class " + renameHelperClass + ", \n"
+ + " --spark-master " + sparkMaster + ", \n"
+ + " --spark-memory " + sparkMemory + ", \n"
+ + " --props " + propsFilePath + ", \n"
+ + " --hoodie-conf " + configs
+ + "\n}";
+ }
+ }
+
+ public static void main(String[] args) {
+ final Config cfg = new Config();
+ JCommander cmd = new JCommander(cfg, null, args);
+ if (cfg.help || args.length == 0) {
+ cmd.usage();
+ System.exit(1);
+ }
+ Map<String, String> sparkConfigMap = new HashMap<>();
+ sparkConfigMap.put("spark.executor.memory", cfg.sparkMemory);
+ JavaSparkContext jsc = UtilHelpers.buildSparkContext("Hoodie-Stash-Partitions",
+ cfg.sparkMaster, false, sparkConfigMap);
+
+ HoodieStashPartitionsTool tool = new HoodieStashPartitionsTool(jsc, cfg);
+ try {
+ tool.run();
+ } catch (Throwable throwable) {
+ LOG.error("Fail to run stash partitions for " + cfg, throwable);
+ } finally {
+ jsc.stop();
+ }
+ }
+
+ public void run() {
+ try {
+ LOG.info(cfg.toString());
+ Mode mode = Mode.valueOf(cfg.runningMode.toUpperCase());
+ switch (mode) {
+ case STASH:
+ LOG.info(" ****** The Hoodie Stash Partitions Tool is in stash mode ****** ");
+ doStash();
+ break;
+ case ROLLBACK_STASH:
+ LOG.info(" ****** The Hoodie Stash Partitions Tool is in rollback_stash mode ****** ");
+ doRollbackStash();
+ break;
+ case DRY_RUN:
+ LOG.info(" ****** The Hoodie Stash Partitions Tool is in dry-run mode ****** ");
+ doDryRun();
+ break;
+ default:
+ LOG.info("Unsupported running mode [" + cfg.runningMode + "], quit the job directly");
+ }
+ } catch (Exception e) {
+ throw new HoodieException("Unable to stash partitions in " + cfg.basePath, e);
+ }
+ }
+
+ /**
+ * Stash partitions: pre-check to recover any partial stash from a prior failed attempt,
+ * then issue deletePartitions with custom pre-commit validator. Hudi internally handles
+ * rollback of any prior failed instants. The pre-commit validator handles already-stashed
+ * partitions gracefully (source empty -> skip).
+ *
+ * <p>The pre-check runs regardless of whether MDT is enabled. Even with MDT, a prior failed
+ * attempt may have had its pre-commit validator move files to the stash location before
+ * crashing. Those files need to be moved back so that the subsequent deletePartitions
+ * sees the complete partition state.
+ */
+ private void doStash() {
+ List<String> partitionsToStash = Arrays.asList(cfg.partitions.split(","));
+
+ preCheckAndRecoverPartialStash(partitionsToStash);
+
+ // Set up props with stash-specific configs for the pre-commit validator
Review Comment:
sure. fixing it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-4300611597
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `16.46707%` with `279 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 48.26%. Comparing base ([`7bcb8be`](https://app.codecov.io/gh/apache/hudi/commit/7bcb8be7c153010998574f591c7ff3799192eda6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`6d8dedd`](https://app.codecov.io/gh/apache/hudi/commit/6d8dedd061ed183c52471df09355b85e5a60099a?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 20 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | 0.00% | [263 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | 77.50% | [5 Missing and 4 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | 77.41% | [5 Missing and 2 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
> :exclamation: There is a different number of reports uploaded between BASE (7bcb8be) and HEAD (6d8dedd). Click for more details.
>
> <details><summary>HEAD has 22 uploads less than BASE</summary>
>
>| Flag | BASE (7bcb8be) | HEAD (6d8dedd) |
>|------|------|------|
>|spark-scala-tests|10|4|
>|spark-java-tests|15|1|
>|common-and-other-modules|1|0|
>|utilities|1|0|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18526 +/- ##
=============================================
- Coverage 68.88% 48.26% -20.62%
+ Complexity 28279 16350 -11929
=============================================
Files 2464 1989 -475
Lines 135568 110942 -24626
Branches 16446 13898 -2548
=============================================
- Hits 93385 53546 -39839
- Misses 34801 51850 +17049
+ Partials 7382 5546 -1836
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.74% <0.00%> (-0.02%)` | :arrow_down: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.64% <77.46%> (+0.23%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `33.21% <0.00%> (-15.72%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `32.44% <0.00%> (-13.00%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | `77.41% <77.41%> (ø)` | |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | `77.50% <77.50%> (ø)` | |
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | `0.00% <0.00%> (ø)` | |
... and [1562 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18526/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-4300712426
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `80.83832%` with `64 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 68.68%. Comparing base ([`7bcb8be`](https://app.codecov.io/gh/apache/hudi/commit/7bcb8be7c153010998574f591c7ff3799192eda6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`6d8dedd`](https://app.codecov.io/gh/apache/hudi/commit/6d8dedd061ed183c52471df09355b85e5a60099a?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 22 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | 79.84% | [43 Missing and 10 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | 77.50% | [5 Missing and 4 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | 93.54% | [1 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18526 +/- ##
============================================
- Coverage 68.88% 68.68% -0.21%
- Complexity 28279 28472 +193
============================================
Files 2464 2478 +14
Lines 135568 136972 +1404
Branches 16446 16653 +207
============================================
+ Hits 93385 94074 +689
- Misses 34801 35326 +525
- Partials 7382 7572 +190
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.39% <13.77%> (-0.25%)` | :arrow_down: |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.74% <0.00%> (-0.02%)` | :arrow_down: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.64% <77.46%> (+0.23%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `49.18% <0.00%> (+0.26%)` | :arrow_up: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.18% <0.00%> (-0.26%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `37.87% <77.54%> (-0.30%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | `93.54% <93.54%> (ø)` | |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | `77.50% <77.50%> (ø)` | |
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | `79.84% <79.84%> (ø)` | |
... and [81 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18526/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-4272496389
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `23.31839%` with `171 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 48.03%. Comparing base ([`7bcb8be`](https://app.codecov.io/gh/apache/hudi/commit/7bcb8be7c153010998574f591c7ff3799192eda6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`e8b5a0a`](https://app.codecov.io/gh/apache/hudi/commit/e8b5a0a62cea99be0a184e0b6f11149b49850806?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 3 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | 0.00% | [156 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | 75.00% | [6 Missing and 4 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | 81.48% | [3 Missing and 2 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
> :exclamation: There is a different number of reports uploaded between BASE (7bcb8be) and HEAD (e8b5a0a). Click for more details.
>
> <details><summary>HEAD has 18 uploads less than BASE</summary>
>
>| Flag | BASE (7bcb8be) | HEAD (e8b5a0a) |
>|------|------|------|
>|spark-scala-tests|10|5|
>|spark-java-tests|15|4|
>|common-and-other-modules|1|0|
>|utilities|1|0|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18526 +/- ##
=============================================
- Coverage 68.88% 48.03% -20.86%
+ Complexity 28279 16534 -11745
=============================================
Files 2464 2011 -453
Lines 135568 112793 -22775
Branches 16446 14326 -2120
=============================================
- Hits 93385 54176 -39209
- Misses 34801 52952 +18151
+ Partials 7382 5665 -1717
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.70% <0.00%> (-0.06%)` | :arrow_down: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.46% <77.61%> (+0.06%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `32.29% <0.00%> (-16.64%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `32.48% <0.00%> (-12.96%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | `81.48% <81.48%> (ø)` | |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | `75.00% <75.00%> (ø)` | |
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | `0.00% <0.00%> (ø)` | |
... and [1532 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18526/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-4272449068
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `0%` with `27 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 44.70%. Comparing base ([`7bcb8be`](https://app.codecov.io/gh/apache/hudi/commit/7bcb8be7c153010998574f591c7ff3799192eda6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`e8b5a0a`](https://app.codecov.io/gh/apache/hudi/commit/e8b5a0a62cea99be0a184e0b6f11149b49850806?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 3 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | 0.00% | [27 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
> :exclamation: There is a different number of reports uploaded between BASE (7bcb8be) and HEAD (e8b5a0a). Click for more details.
>
> <details><summary>HEAD has 28 uploads less than BASE</summary>
>
>| Flag | BASE (7bcb8be) | HEAD (e8b5a0a) |
>|------|------|------|
>|spark-client-hadoop-common|1|0|
>|spark-scala-tests|10|0|
>|spark-java-tests|15|0|
>|common-and-other-modules|1|0|
>|utilities|1|0|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18526 +/- ##
=============================================
- Coverage 68.88% 44.70% -24.19%
+ Complexity 28279 8480 -19799
=============================================
Files 2464 1200 -1264
Lines 135568 62378 -73190
Branches 16446 6737 -9709
=============================================
- Hits 93385 27884 -65501
+ Misses 34801 31454 -3347
+ Partials 7382 3040 -4342
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.70% <0.00%> (-0.06%)` | :arrow_down: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | `0.00% <0.00%> (ø)` | |
... and [2066 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18526/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-4272400387
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "67f1e57f0dffe02bbe2d7bacfaa8f1aaa4be8c88",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "67f1e57f0dffe02bbe2d7bacfaa8f1aaa4be8c88",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 67f1e57f0dffe02bbe2d7bacfaa8f1aaa4be8c88 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "yihua (via GitHub)" <gi...@apache.org>.
yihua commented on code in PR #18526:
URL: https://github.com/apache/hudi/pull/18526#discussion_r3105528956
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/validator/DefaultStashPartitionRenameHelper.java:
##########
@@ -0,0 +1,92 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.hudi.client.validator;
+
+import org.apache.hudi.io.util.FileIOUtils;
+import org.apache.hudi.storage.HoodieStorage;
+import org.apache.hudi.storage.StoragePath;
+import org.apache.hudi.storage.StoragePathInfo;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.IOException;
+import java.util.List;
+import java.util.Set;
+import java.util.stream.Collectors;
+
+/**
+ * Default implementation of {@link StashPartitionRenameHelper} that copies files
+ * individually from source to target and then deletes them from source.
+ * <p>
+ * This implementation is correct but may be slow for large partitions on cloud
+ * storage. For production use on HDFS or cloud storage with efficient rename APIs,
+ * a custom implementation can be plugged in via the
+ * {@code hoodie.stash.rename.helper.class} configuration.
+ */
+public class DefaultStashPartitionRenameHelper implements StashPartitionRenameHelper {
+
+ private static final long serialVersionUID = 1L;
+ private static final Logger LOG = LoggerFactory.getLogger(DefaultStashPartitionRenameHelper.class);
+
+ @Override
+ public void movePartitionFiles(HoodieStorage storage, StoragePath sourcePath, StoragePath targetPath) throws IOException {
+ // Ensure target directory exists
+ if (!storage.exists(targetPath)) {
+ storage.createDirectory(targetPath);
+ }
+
+ List<StoragePathInfo> sourceFiles = storage.listDirectEntries(sourcePath);
+ if (sourceFiles.isEmpty()) {
+ LOG.info("Source path {} has no files, nothing to move.", sourcePath);
+ return;
+ }
+
+ // Determine which files already exist in target (from a prior partial attempt)
+ Set<String> existingTargetFileNames = getExistingFileNames(storage, targetPath);
+
+ // Copy files not already in target
+ for (StoragePathInfo sourceFileInfo : sourceFiles) {
+ String fileName = sourceFileInfo.getPath().getName();
+ if (existingTargetFileNames.contains(fileName)) {
+ LOG.info("File {} already exists in target {}, skipping copy.", fileName, targetPath);
+ } else {
+ StoragePath destFilePath = new StoragePath(targetPath, fileName);
+ LOG.info("Copying file {} to {}", sourceFileInfo.getPath(), destFilePath);
+ FileIOUtils.copy(storage, sourceFileInfo.getPath(), destFilePath);
+ }
+ }
+
+ // Delete all source files after successful copy
+ for (StoragePathInfo sourceFileInfo : sourceFiles) {
+ LOG.info("Deleting source file {}", sourceFileInfo.getPath());
+ storage.deleteFile(sourceFileInfo.getPath());
+ }
+ }
+
+ private Set<String> getExistingFileNames(HoodieStorage storage, StoragePath targetPath) throws IOException {
+ if (!storage.exists(targetPath)) {
+ return java.util.Collections.emptySet();
+ }
Review Comment:
🤖 nit: `java.util.Collections` is used as a fully-qualified name here rather than an import — could you add `import java.util.Collections;` at the top and use `Collections.emptySet()` instead?
<sub><i>- Generated by an AI agent and may contain mistakes. Please verify any suggestions before applying.</i></sub>
##########
hudi-utilities/src/main/java/org/apache/hudi/utilities/HoodieStashPartitionsTool.java:
##########
@@ -0,0 +1,386 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.hudi.utilities;
+
+import org.apache.hudi.client.HoodieWriteResult;
+import org.apache.hudi.client.SparkRDDWriteClient;
+import org.apache.hudi.client.validator.DefaultStashPartitionRenameHelper;
+import org.apache.hudi.client.validator.StashPartitionRenameHelper;
+import org.apache.hudi.client.validator.StashPartitionsPreCommitValidator;
+import org.apache.hudi.common.config.TypedProperties;
+import org.apache.hudi.common.model.HoodieRecordPayload;
+import org.apache.hudi.common.table.HoodieTableMetaClient;
+import org.apache.hudi.common.table.timeline.HoodieTimeline;
+import org.apache.hudi.common.util.Option;
+import org.apache.hudi.common.util.ReflectionUtils;
+import org.apache.hudi.common.util.StringUtils;
+import org.apache.hudi.config.HoodiePreCommitValidatorConfig;
+import org.apache.hudi.exception.HoodieException;
+import org.apache.hudi.hadoop.fs.HadoopFSUtils;
+import org.apache.hudi.storage.HoodieStorage;
+import org.apache.hudi.storage.HoodieStorageUtils;
+import org.apache.hudi.storage.StoragePath;
+import org.apache.hudi.storage.StoragePathInfo;
+
+import com.beust.jcommander.JCommander;
+import com.beust.jcommander.Parameter;
+import org.apache.hadoop.fs.Path;
+import org.apache.spark.api.java.JavaSparkContext;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.IOException;
+import java.io.Serializable;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+
+/**
+ * A spark-submit tool for stashing (backing up and removing) Hudi table partitions.
+ *
+ * <p>This tool supports three modes:
+ * <ul>
+ * <li><b>stash</b> — Moves data files from specified partitions to a stash location
+ * and issues a deletePartitions operation on the Hudi table. Uses a custom
+ * {@link StashPartitionsPreCommitValidator} to move files atomically before commit.</li>
+ * <li><b>rollback_stash</b> — Copies any data found in the stash location back to the
+ * original partition paths. Used when the user no longer wants to proceed with stashing.</li>
+ * <li><b>dry_run</b> — Shows which partitions and file groups would be affected.</li>
+ * </ul>
+ *
+ * <p>For restoring stashed partitions back into the Hudi table, use the standard
+ * {@code insert_overwrite} write operation reading from the stash location.
+ *
+ * <p>Example usage:
+ * <pre>
+ * spark-submit \
+ * --class org.apache.hudi.utilities.HoodieStashPartitionsTool \
+ * --master local[*] \
+ * --driver-memory 1g \
+ * --executor-memory 1g \
+ * hudi-utilities-bundle.jar \
+ * --base-path /path/to/hudi/table \
+ * --table-name myTable \
+ * --stash-path /path/to/stash/folder \
+ * --partitions datestr=2023-01-01,datestr=2023-01-02 \
+ * --mode stash
+ * </pre>
+ */
+public class HoodieStashPartitionsTool implements Serializable {
+
+ private static final long serialVersionUID = 1L;
+ private static final Logger LOG = LoggerFactory.getLogger(HoodieStashPartitionsTool.class);
+
+ private final transient JavaSparkContext jsc;
+ private final Config cfg;
+ private final TypedProperties props;
+ private final HoodieTableMetaClient metaClient;
+
+ public HoodieStashPartitionsTool(JavaSparkContext jsc, Config cfg) {
+ this.jsc = jsc;
+ this.cfg = cfg;
+ this.props = cfg.propsFilePath == null
+ ? UtilHelpers.buildProperties(cfg.configs)
+ : readConfigFromFileSystem(jsc, cfg);
+ this.metaClient = HoodieTableMetaClient.builder()
+ .setConf(HadoopFSUtils.getStorageConfWithCopy(jsc.hadoopConfiguration()))
+ .setBasePath(cfg.basePath)
+ .setLoadActiveTimelineOnLoad(true)
+ .build();
+ }
+
+ public enum Mode {
+ STASH,
+ ROLLBACK_STASH,
+ DRY_RUN
+ }
+
+ public static class Config implements Serializable {
+ @Parameter(names = {"--base-path", "-sp"}, description = "Base path for the Hudi table.", required = true)
+ public String basePath = null;
+
+ @Parameter(names = {"--table-name", "-tn"}, description = "Table name.", required = true)
+ public String tableName = null;
+
+ @Parameter(names = {"--stash-path"}, description = "Target path for stashing partition data.", required = true)
+ public String stashPath = null;
+
+ @Parameter(names = {"--partitions", "-p"}, description = "Comma separated list of partitions to stash.", required = true)
+ public String partitions = null;
+
+ @Parameter(names = {"--mode", "-m"}, description = "Set job mode: "
+ + "\"stash\" to move partition data to stash location and delete partitions; "
+ + "\"rollback_stash\" to copy stashed data back to table and abort stashing; "
+ + "\"dry_run\" to preview which partitions and files would be affected.", required = true)
+ public String runningMode = null;
+
+ @Parameter(names = {"--parallelism", "-pl"}, description = "Parallelism for hoodie operations.", required = false)
+ public int parallelism = 1500;
+
+ @Parameter(names = {"--rename-helper-class"}, description = "Fully qualified class name of a custom "
+ + "StashPartitionRenameHelper implementation. Defaults to DefaultStashPartitionRenameHelper.", required = false)
+ public String renameHelperClass = null;
+
+ @Parameter(names = {"--spark-master", "-ms"}, description = "Spark master.", required = false)
+ public String sparkMaster = null;
+
+ @Parameter(names = {"--spark-memory", "-sm"}, description = "Spark memory to use.", required = false)
+ public String sparkMemory = "1g";
+
+ @Parameter(names = {"--props"}, description = "Path to properties file on localfs or dfs, with configurations for "
+ + "hoodie client for stashing partitions.")
+ public String propsFilePath = null;
+
+ @Parameter(names = {"--hoodie-conf"}, description = "Any configuration that can be set in the properties file "
+ + "(using the CLI parameter \"--props\") can also be passed via command line using this parameter. This can be repeated.",
+ splitter = IdentitySplitter.class)
+ public List<String> configs = new ArrayList<>();
+
+ @Parameter(names = {"--help", "-h"}, help = true)
+ public Boolean help = false;
+
+ @Override
+ public String toString() {
+ return "HoodieStashPartitionsToolConfig {\n"
+ + " --base-path " + basePath + ", \n"
+ + " --table-name " + tableName + ", \n"
+ + " --stash-path " + stashPath + ", \n"
+ + " --partitions " + partitions + ", \n"
+ + " --mode " + runningMode + ", \n"
+ + " --parallelism " + parallelism + ", \n"
+ + " --rename-helper-class " + renameHelperClass + ", \n"
+ + " --spark-master " + sparkMaster + ", \n"
+ + " --spark-memory " + sparkMemory + ", \n"
+ + " --props " + propsFilePath + ", \n"
+ + " --hoodie-conf " + configs
+ + "\n}";
+ }
+ }
+
+ public static void main(String[] args) {
+ final Config cfg = new Config();
+ JCommander cmd = new JCommander(cfg, null, args);
+ if (cfg.help || args.length == 0) {
+ cmd.usage();
+ System.exit(1);
+ }
+ Map<String, String> sparkConfigMap = new HashMap<>();
+ sparkConfigMap.put("spark.executor.memory", cfg.sparkMemory);
+ JavaSparkContext jsc = UtilHelpers.buildSparkContext("Hoodie-Stash-Partitions",
+ cfg.sparkMaster, false, sparkConfigMap);
+
+ HoodieStashPartitionsTool tool = new HoodieStashPartitionsTool(jsc, cfg);
+ try {
+ tool.run();
+ } catch (Throwable throwable) {
+ LOG.error("Fail to run stash partitions for " + cfg, throwable);
+ } finally {
+ jsc.stop();
+ }
+ }
+
+ public void run() {
+ try {
+ LOG.info(cfg.toString());
+ Mode mode = Mode.valueOf(cfg.runningMode.toUpperCase());
+ switch (mode) {
+ case STASH:
+ LOG.info(" ****** The Hoodie Stash Partitions Tool is in stash mode ****** ");
+ doStash();
+ break;
+ case ROLLBACK_STASH:
+ LOG.info(" ****** The Hoodie Stash Partitions Tool is in rollback_stash mode ****** ");
+ doRollbackStash();
+ break;
+ case DRY_RUN:
+ LOG.info(" ****** The Hoodie Stash Partitions Tool is in dry-run mode ****** ");
+ doDryRun();
+ break;
+ default:
+ LOG.info("Unsupported running mode [" + cfg.runningMode + "], quit the job directly");
+ }
+ } catch (Exception e) {
+ throw new HoodieException("Unable to stash partitions in " + cfg.basePath, e);
+ }
+ }
+
+ /**
+ * Stash partitions: pre-check to recover any partial stash from a prior failed attempt,
+ * then issue deletePartitions with custom pre-commit validator. Hudi internally handles
+ * rollback of any prior failed instants. The pre-commit validator handles already-stashed
+ * partitions gracefully (source empty -> skip).
+ *
+ * <p>The pre-check runs regardless of whether MDT is enabled. Even with MDT, a prior failed
+ * attempt may have had its pre-commit validator move files to the stash location before
+ * crashing. Those files need to be moved back so that the subsequent deletePartitions
+ * sees the complete partition state.
+ */
+ private void doStash() {
+ List<String> partitionsToStash = Arrays.asList(cfg.partitions.split(","));
+
+ preCheckAndRecoverPartialStash(partitionsToStash);
+
+ // Set up props with stash-specific configs for the pre-commit validator
+ TypedProperties stashProps = new TypedProperties();
+ stashProps.putAll(props);
+ stashProps.setProperty(HoodiePreCommitValidatorConfig.VALIDATOR_CLASS_NAMES.key(),
+ StashPartitionsPreCommitValidator.class.getName());
+ stashProps.setProperty(StashPartitionsPreCommitValidator.STASH_PATH_CONFIG, cfg.stashPath);
+ if (!StringUtils.isNullOrEmpty(cfg.renameHelperClass)) {
+ stashProps.setProperty(StashPartitionsPreCommitValidator.RENAME_HELPER_CLASS_CONFIG, cfg.renameHelperClass);
+ }
+
+ try (SparkRDDWriteClient<HoodieRecordPayload> client =
+ UtilHelpers.createHoodieClient(jsc, cfg.basePath, "", cfg.parallelism, Option.empty(), stashProps)) {
+ String instantTime = client.startCommit(HoodieTimeline.REPLACE_COMMIT_ACTION);
+ LOG.info("Starting deletePartitions with instant {} for partitions: {}", instantTime, partitionsToStash);
+ HoodieWriteResult result = client.deletePartitions(partitionsToStash, instantTime);
+ client.commit(instantTime, result.getWriteStatuses(), Option.empty(), HoodieTimeline.REPLACE_COMMIT_ACTION,
+ result.getPartitionToReplaceFileIds(), Option.empty());
+ LOG.info("Successfully stashed partitions: {}", partitionsToStash);
+ }
+ }
+
+ /**
+ * Pre-check: if files exist in the stash location from a prior failed attempt,
+ * copy them back to source before retrying deletePartitions. This ensures the
+ * subsequent deletePartitions sees the complete partition state and creates
+ * a correct replace commit. Runs for both MDT and non-MDT tables, since the
+ * pre-commit validator may have moved files before crashing.
+ */
+ private void preCheckAndRecoverPartialStash(List<String> partitions) {
+ LOG.info("Running pre-check for partial stash recovery (non-MDT table).");
+ HoodieStorage storage = HoodieStorageUtils.getStorage(
+ cfg.basePath, metaClient.getStorageConf());
+ StashPartitionRenameHelper renameHelper = createRenameHelper();
+ StoragePath basePath = metaClient.getBasePath();
+
+ for (String partition : partitions) {
+ StoragePath stashPartitionPath = new StoragePath(cfg.stashPath, partition);
+ StoragePath sourcePartitionPath = new StoragePath(basePath, partition);
+
+ try {
+ if (!storage.exists(stashPartitionPath) || isDirectoryEmpty(storage, stashPartitionPath)) {
+ LOG.info("Partition {} has no stashed data from prior attempt, skipping pre-check.", partition);
+ continue;
+ }
+ // Files exist in stash — copy them back to source to ensure deletePartitions
+ // sees all files and creates a complete replace commit.
+ LOG.info("Found stashed files for partition {} from prior attempt. Moving back to source.", partition);
+ renameHelper.movePartitionFiles(storage, stashPartitionPath, sourcePartitionPath);
+ LOG.info("Recovered partition {} back to source.", partition);
+ } catch (IOException e) {
+ throw new HoodieException("Failed to recover partition " + partition + " from stash during pre-check.", e);
+ }
+ }
+ }
+
+ /**
+ * Rollback stash: copy any data from stash location back to original partition paths.
+ * This is for when the user no longer wants to proceed with stashing.
+ */
+ private void doRollbackStash() {
+ List<String> partitions = Arrays.asList(cfg.partitions.split(","));
+ HoodieStorage storage = HoodieStorageUtils.getStorage(
+ cfg.basePath, metaClient.getStorageConf());
+ StashPartitionRenameHelper renameHelper = createRenameHelper();
+ StoragePath basePath = metaClient.getBasePath();
+
+ for (String partition : partitions) {
+ StoragePath stashPartitionPath = new StoragePath(cfg.stashPath, partition);
+ StoragePath sourcePartitionPath = new StoragePath(basePath, partition);
+
+ try {
+ if (!storage.exists(stashPartitionPath) || isDirectoryEmpty(storage, stashPartitionPath)) {
+ LOG.info("No stashed data found for partition {}, skipping.", partition);
+ continue;
+ }
+ LOG.info("Rolling back stash for partition {}: moving from {} to {}", partition, stashPartitionPath, sourcePartitionPath);
+ renameHelper.movePartitionFiles(storage, stashPartitionPath, sourcePartitionPath);
+ LOG.info("Successfully rolled back stash for partition {}.", partition);
+ } catch (IOException e) {
+ throw new HoodieException("Failed to rollback stash for partition: " + partition, e);
+ }
+ }
+ LOG.info("Stash rollback complete.");
+ }
+
+ /**
+ * Dry run: show partitions and file groups that would be affected.
+ */
+ private void doDryRun() {
+ try (SparkRDDWriteClient<HoodieRecordPayload> client =
+ UtilHelpers.createHoodieClient(jsc, cfg.basePath, "", cfg.parallelism, Option.empty(), props)) {
+ org.apache.hudi.table.HoodieSparkTable<HoodieRecordPayload> table =
+ org.apache.hudi.table.HoodieSparkTable.create(client.getConfig(), client.getEngineContext());
Review Comment:
🤖 nit: `org.apache.hudi.table.HoodieSparkTable` is spelled out as a fully-qualified name in the method body — could you add a proper import and use the short form `HoodieSparkTable`? (The same class is cleanly imported in `StashPartitionsPreCommitValidator`.)
<sub><i>- Generated by an AI agent and may contain mistakes. Please verify any suggestions before applying.</i></sub>
##########
hudi-utilities/src/main/java/org/apache/hudi/utilities/HoodieStashPartitionsTool.java:
##########
@@ -0,0 +1,386 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.hudi.utilities;
+
+import org.apache.hudi.client.HoodieWriteResult;
+import org.apache.hudi.client.SparkRDDWriteClient;
+import org.apache.hudi.client.validator.DefaultStashPartitionRenameHelper;
+import org.apache.hudi.client.validator.StashPartitionRenameHelper;
+import org.apache.hudi.client.validator.StashPartitionsPreCommitValidator;
+import org.apache.hudi.common.config.TypedProperties;
+import org.apache.hudi.common.model.HoodieRecordPayload;
+import org.apache.hudi.common.table.HoodieTableMetaClient;
+import org.apache.hudi.common.table.timeline.HoodieTimeline;
+import org.apache.hudi.common.util.Option;
+import org.apache.hudi.common.util.ReflectionUtils;
+import org.apache.hudi.common.util.StringUtils;
+import org.apache.hudi.config.HoodiePreCommitValidatorConfig;
+import org.apache.hudi.exception.HoodieException;
+import org.apache.hudi.hadoop.fs.HadoopFSUtils;
+import org.apache.hudi.storage.HoodieStorage;
+import org.apache.hudi.storage.HoodieStorageUtils;
+import org.apache.hudi.storage.StoragePath;
+import org.apache.hudi.storage.StoragePathInfo;
+
+import com.beust.jcommander.JCommander;
+import com.beust.jcommander.Parameter;
+import org.apache.hadoop.fs.Path;
+import org.apache.spark.api.java.JavaSparkContext;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.IOException;
+import java.io.Serializable;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+
+/**
+ * A spark-submit tool for stashing (backing up and removing) Hudi table partitions.
+ *
+ * <p>This tool supports three modes:
+ * <ul>
+ * <li><b>stash</b> — Moves data files from specified partitions to a stash location
+ * and issues a deletePartitions operation on the Hudi table. Uses a custom
+ * {@link StashPartitionsPreCommitValidator} to move files atomically before commit.</li>
+ * <li><b>rollback_stash</b> — Copies any data found in the stash location back to the
+ * original partition paths. Used when the user no longer wants to proceed with stashing.</li>
+ * <li><b>dry_run</b> — Shows which partitions and file groups would be affected.</li>
+ * </ul>
+ *
+ * <p>For restoring stashed partitions back into the Hudi table, use the standard
+ * {@code insert_overwrite} write operation reading from the stash location.
+ *
+ * <p>Example usage:
+ * <pre>
+ * spark-submit \
+ * --class org.apache.hudi.utilities.HoodieStashPartitionsTool \
+ * --master local[*] \
+ * --driver-memory 1g \
+ * --executor-memory 1g \
+ * hudi-utilities-bundle.jar \
+ * --base-path /path/to/hudi/table \
+ * --table-name myTable \
+ * --stash-path /path/to/stash/folder \
+ * --partitions datestr=2023-01-01,datestr=2023-01-02 \
+ * --mode stash
+ * </pre>
+ */
+public class HoodieStashPartitionsTool implements Serializable {
+
+ private static final long serialVersionUID = 1L;
+ private static final Logger LOG = LoggerFactory.getLogger(HoodieStashPartitionsTool.class);
+
+ private final transient JavaSparkContext jsc;
+ private final Config cfg;
+ private final TypedProperties props;
+ private final HoodieTableMetaClient metaClient;
+
+ public HoodieStashPartitionsTool(JavaSparkContext jsc, Config cfg) {
+ this.jsc = jsc;
+ this.cfg = cfg;
+ this.props = cfg.propsFilePath == null
+ ? UtilHelpers.buildProperties(cfg.configs)
+ : readConfigFromFileSystem(jsc, cfg);
+ this.metaClient = HoodieTableMetaClient.builder()
+ .setConf(HadoopFSUtils.getStorageConfWithCopy(jsc.hadoopConfiguration()))
+ .setBasePath(cfg.basePath)
+ .setLoadActiveTimelineOnLoad(true)
+ .build();
+ }
+
+ public enum Mode {
+ STASH,
+ ROLLBACK_STASH,
+ DRY_RUN
+ }
+
+ public static class Config implements Serializable {
+ @Parameter(names = {"--base-path", "-sp"}, description = "Base path for the Hudi table.", required = true)
+ public String basePath = null;
+
+ @Parameter(names = {"--table-name", "-tn"}, description = "Table name.", required = true)
+ public String tableName = null;
+
+ @Parameter(names = {"--stash-path"}, description = "Target path for stashing partition data.", required = true)
+ public String stashPath = null;
+
+ @Parameter(names = {"--partitions", "-p"}, description = "Comma separated list of partitions to stash.", required = true)
+ public String partitions = null;
+
+ @Parameter(names = {"--mode", "-m"}, description = "Set job mode: "
+ + "\"stash\" to move partition data to stash location and delete partitions; "
+ + "\"rollback_stash\" to copy stashed data back to table and abort stashing; "
+ + "\"dry_run\" to preview which partitions and files would be affected.", required = true)
+ public String runningMode = null;
+
+ @Parameter(names = {"--parallelism", "-pl"}, description = "Parallelism for hoodie operations.", required = false)
+ public int parallelism = 1500;
+
+ @Parameter(names = {"--rename-helper-class"}, description = "Fully qualified class name of a custom "
+ + "StashPartitionRenameHelper implementation. Defaults to DefaultStashPartitionRenameHelper.", required = false)
+ public String renameHelperClass = null;
+
+ @Parameter(names = {"--spark-master", "-ms"}, description = "Spark master.", required = false)
+ public String sparkMaster = null;
+
+ @Parameter(names = {"--spark-memory", "-sm"}, description = "Spark memory to use.", required = false)
+ public String sparkMemory = "1g";
+
+ @Parameter(names = {"--props"}, description = "Path to properties file on localfs or dfs, with configurations for "
+ + "hoodie client for stashing partitions.")
+ public String propsFilePath = null;
+
+ @Parameter(names = {"--hoodie-conf"}, description = "Any configuration that can be set in the properties file "
+ + "(using the CLI parameter \"--props\") can also be passed via command line using this parameter. This can be repeated.",
+ splitter = IdentitySplitter.class)
+ public List<String> configs = new ArrayList<>();
+
+ @Parameter(names = {"--help", "-h"}, help = true)
+ public Boolean help = false;
+
+ @Override
+ public String toString() {
+ return "HoodieStashPartitionsToolConfig {\n"
+ + " --base-path " + basePath + ", \n"
+ + " --table-name " + tableName + ", \n"
+ + " --stash-path " + stashPath + ", \n"
+ + " --partitions " + partitions + ", \n"
+ + " --mode " + runningMode + ", \n"
+ + " --parallelism " + parallelism + ", \n"
+ + " --rename-helper-class " + renameHelperClass + ", \n"
+ + " --spark-master " + sparkMaster + ", \n"
+ + " --spark-memory " + sparkMemory + ", \n"
+ + " --props " + propsFilePath + ", \n"
+ + " --hoodie-conf " + configs
+ + "\n}";
+ }
+ }
+
+ public static void main(String[] args) {
+ final Config cfg = new Config();
+ JCommander cmd = new JCommander(cfg, null, args);
+ if (cfg.help || args.length == 0) {
+ cmd.usage();
+ System.exit(1);
+ }
+ Map<String, String> sparkConfigMap = new HashMap<>();
+ sparkConfigMap.put("spark.executor.memory", cfg.sparkMemory);
+ JavaSparkContext jsc = UtilHelpers.buildSparkContext("Hoodie-Stash-Partitions",
+ cfg.sparkMaster, false, sparkConfigMap);
+
+ HoodieStashPartitionsTool tool = new HoodieStashPartitionsTool(jsc, cfg);
+ try {
+ tool.run();
+ } catch (Throwable throwable) {
+ LOG.error("Fail to run stash partitions for " + cfg, throwable);
+ } finally {
+ jsc.stop();
+ }
+ }
+
+ public void run() {
+ try {
+ LOG.info(cfg.toString());
+ Mode mode = Mode.valueOf(cfg.runningMode.toUpperCase());
+ switch (mode) {
+ case STASH:
+ LOG.info(" ****** The Hoodie Stash Partitions Tool is in stash mode ****** ");
+ doStash();
+ break;
+ case ROLLBACK_STASH:
+ LOG.info(" ****** The Hoodie Stash Partitions Tool is in rollback_stash mode ****** ");
+ doRollbackStash();
+ break;
+ case DRY_RUN:
+ LOG.info(" ****** The Hoodie Stash Partitions Tool is in dry-run mode ****** ");
+ doDryRun();
+ break;
+ default:
+ LOG.info("Unsupported running mode [" + cfg.runningMode + "], quit the job directly");
+ }
+ } catch (Exception e) {
+ throw new HoodieException("Unable to stash partitions in " + cfg.basePath, e);
+ }
+ }
+
+ /**
+ * Stash partitions: pre-check to recover any partial stash from a prior failed attempt,
+ * then issue deletePartitions with custom pre-commit validator. Hudi internally handles
+ * rollback of any prior failed instants. The pre-commit validator handles already-stashed
+ * partitions gracefully (source empty -> skip).
+ *
+ * <p>The pre-check runs regardless of whether MDT is enabled. Even with MDT, a prior failed
+ * attempt may have had its pre-commit validator move files to the stash location before
+ * crashing. Those files need to be moved back so that the subsequent deletePartitions
+ * sees the complete partition state.
+ */
+ private void doStash() {
+ List<String> partitionsToStash = Arrays.asList(cfg.partitions.split(","));
+
+ preCheckAndRecoverPartialStash(partitionsToStash);
+
+ // Set up props with stash-specific configs for the pre-commit validator
+ TypedProperties stashProps = new TypedProperties();
+ stashProps.putAll(props);
+ stashProps.setProperty(HoodiePreCommitValidatorConfig.VALIDATOR_CLASS_NAMES.key(),
+ StashPartitionsPreCommitValidator.class.getName());
+ stashProps.setProperty(StashPartitionsPreCommitValidator.STASH_PATH_CONFIG, cfg.stashPath);
+ if (!StringUtils.isNullOrEmpty(cfg.renameHelperClass)) {
+ stashProps.setProperty(StashPartitionsPreCommitValidator.RENAME_HELPER_CLASS_CONFIG, cfg.renameHelperClass);
+ }
+
+ try (SparkRDDWriteClient<HoodieRecordPayload> client =
+ UtilHelpers.createHoodieClient(jsc, cfg.basePath, "", cfg.parallelism, Option.empty(), stashProps)) {
+ String instantTime = client.startCommit(HoodieTimeline.REPLACE_COMMIT_ACTION);
+ LOG.info("Starting deletePartitions with instant {} for partitions: {}", instantTime, partitionsToStash);
+ HoodieWriteResult result = client.deletePartitions(partitionsToStash, instantTime);
+ client.commit(instantTime, result.getWriteStatuses(), Option.empty(), HoodieTimeline.REPLACE_COMMIT_ACTION,
+ result.getPartitionToReplaceFileIds(), Option.empty());
+ LOG.info("Successfully stashed partitions: {}", partitionsToStash);
+ }
+ }
+
+ /**
+ * Pre-check: if files exist in the stash location from a prior failed attempt,
+ * copy them back to source before retrying deletePartitions. This ensures the
+ * subsequent deletePartitions sees the complete partition state and creates
+ * a correct replace commit. Runs for both MDT and non-MDT tables, since the
+ * pre-commit validator may have moved files before crashing.
+ */
+ private void preCheckAndRecoverPartialStash(List<String> partitions) {
+ LOG.info("Running pre-check for partial stash recovery (non-MDT table).");
+ HoodieStorage storage = HoodieStorageUtils.getStorage(
Review Comment:
🤖 nit: the log message says "(non-MDT table)" but the Javadoc directly above this method says it "Runs for both MDT and non-MDT tables" — the parenthetical will mislead someone grep-ing logs on an MDT table. Could you update it to something like `"Running pre-check for partial stash recovery."`?
<sub><i>- Generated by an AI agent and may contain mistakes. Please verify any suggestions before applying.</i></sub>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-6395797077
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `80.83832%` with `64 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 68.68%. Comparing base ([`7bcb8be`](https://app.codecov.io/gh/apache/hudi/commit/7bcb8be7c153010998574f591c7ff3799192eda6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`6d8dedd`](https://app.codecov.io/gh/apache/hudi/commit/6d8dedd061ed183c52471df09355b85e5a60099a?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 802 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | 79.84% | [43 Missing and 10 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | 77.50% | [5 Missing and 4 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | 93.54% | [1 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18526 +/- ##
============================================
- Coverage 68.88% 68.68% -0.21%
- Complexity 28279 28473 +194
============================================
Files 2464 2478 +14
Lines 135568 136972 +1404
Branches 16446 16653 +207
============================================
+ Hits 93385 94077 +692
- Misses 34801 35324 +523
- Partials 7382 7571 +189
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.39% <13.77%> (-0.25%)` | :arrow_down: |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.74% <0.00%> (-0.02%)` | :arrow_down: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.64% <77.46%> (+0.23%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `49.25% <0.00%> (+0.32%)` | :arrow_up: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.18% <0.00%> (-0.26%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `37.87% <77.54%> (-0.30%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | `93.54% <93.54%> (ø)` | |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | `77.50% <77.50%> (ø)` | |
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | `79.84% <79.84%> (ø)` | |
... and [79 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18526/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
- :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-6398911210
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `80.83832%` with `64 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 68.68%. Comparing base ([`7bcb8be`](https://app.codecov.io/gh/apache/hudi/commit/7bcb8be7c153010998574f591c7ff3799192eda6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`6d8dedd`](https://app.codecov.io/gh/apache/hudi/commit/6d8dedd061ed183c52471df09355b85e5a60099a?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 802 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | 79.84% | [43 Missing and 10 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | 77.50% | [5 Missing and 4 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | 93.54% | [1 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18526 +/- ##
============================================
- Coverage 68.88% 68.68% -0.21%
- Complexity 28279 28473 +194
============================================
Files 2464 2478 +14
Lines 135568 136972 +1404
Branches 16446 16653 +207
============================================
+ Hits 93385 94077 +692
- Misses 34801 35324 +523
- Partials 7382 7571 +189
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.39% <13.77%> (-0.25%)` | :arrow_down: |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.74% <0.00%> (-0.02%)` | :arrow_down: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.64% <77.46%> (+0.23%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `49.25% <0.00%> (+0.32%)` | :arrow_up: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.18% <0.00%> (-0.26%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `37.87% <77.54%> (-0.30%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | `93.54% <93.54%> (ø)` | |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | `77.50% <77.50%> (ø)` | |
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | `79.84% <79.84%> (ø)` | |
... and [79 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18526/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
- :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-4284441976
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `23.31839%` with `171 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 47.91%. Comparing base ([`7bcb8be`](https://app.codecov.io/gh/apache/hudi/commit/7bcb8be7c153010998574f591c7ff3799192eda6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`891eaf2`](https://app.codecov.io/gh/apache/hudi/commit/891eaf2e1427bf93b9c7ff9626628c30e1c9c01d?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 11 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | 0.00% | [156 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | 75.00% | [6 Missing and 4 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | 81.48% | [3 Missing and 2 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
> :exclamation: There is a different number of reports uploaded between BASE (7bcb8be) and HEAD (891eaf2). Click for more details.
>
> <details><summary>HEAD has 21 uploads less than BASE</summary>
>
>| Flag | BASE (7bcb8be) | HEAD (891eaf2) |
>|------|------|------|
>|spark-scala-tests|10|5|
>|spark-java-tests|15|1|
>|common-and-other-modules|1|0|
>|utilities|1|0|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18526 +/- ##
=============================================
- Coverage 68.88% 47.91% -20.97%
+ Complexity 28279 16500 -11779
=============================================
Files 2464 2011 -453
Lines 135568 112821 -22747
Branches 16446 14328 -2118
=============================================
- Hits 93385 54060 -39325
- Misses 34801 53079 +18278
+ Partials 7382 5682 -1700
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.75% <0.00%> (+<0.01%)` | :arrow_up: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.47% <77.61%> (+0.07%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `33.47% <0.00%> (-15.46%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `32.48% <0.00%> (-12.96%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | `81.48% <81.48%> (ø)` | |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | `75.00% <75.00%> (ø)` | |
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | `0.00% <0.00%> (ø)` | |
... and [1535 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18526/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-4284363471
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `0%` with `27 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 44.75%. Comparing base ([`7bcb8be`](https://app.codecov.io/gh/apache/hudi/commit/7bcb8be7c153010998574f591c7ff3799192eda6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`891eaf2`](https://app.codecov.io/gh/apache/hudi/commit/891eaf2e1427bf93b9c7ff9626628c30e1c9c01d?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 11 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | 0.00% | [27 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
> :exclamation: There is a different number of reports uploaded between BASE (7bcb8be) and HEAD (891eaf2). Click for more details.
>
> <details><summary>HEAD has 28 uploads less than BASE</summary>
>
>| Flag | BASE (7bcb8be) | HEAD (891eaf2) |
>|------|------|------|
>|spark-client-hadoop-common|1|0|
>|spark-scala-tests|10|0|
>|spark-java-tests|15|0|
>|common-and-other-modules|1|0|
>|utilities|1|0|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18526 +/- ##
=============================================
- Coverage 68.88% 44.75% -24.13%
+ Complexity 28279 8499 -19780
=============================================
Files 2464 1199 -1265
Lines 135568 62369 -73199
Branches 16446 6737 -9709
=============================================
- Hits 93385 27916 -65469
+ Misses 34801 31402 -3399
+ Partials 7382 3051 -4331
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.75% <0.00%> (+<0.01%)` | :arrow_up: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | `0.00% <0.00%> (ø)` | |
... and [2066 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18526/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-4957496670
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `80.83832%` with `64 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 68.68%. Comparing base ([`7bcb8be`](https://app.codecov.io/gh/apache/hudi/commit/7bcb8be7c153010998574f591c7ff3799192eda6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`6d8dedd`](https://app.codecov.io/gh/apache/hudi/commit/6d8dedd061ed183c52471df09355b85e5a60099a?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 332 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | 79.84% | [43 Missing and 10 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | 77.50% | [5 Missing and 4 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | 93.54% | [1 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18526 +/- ##
============================================
- Coverage 68.88% 68.68% -0.21%
- Complexity 28279 28473 +194
============================================
Files 2464 2478 +14
Lines 135568 136972 +1404
Branches 16446 16653 +207
============================================
+ Hits 93385 94077 +692
- Misses 34801 35324 +523
- Partials 7382 7571 +189
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.39% <13.77%> (-0.25%)` | :arrow_down: |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.74% <0.00%> (-0.02%)` | :arrow_down: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.64% <77.46%> (+0.23%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `49.25% <0.00%> (+0.32%)` | :arrow_up: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.18% <0.00%> (-0.26%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `37.87% <77.54%> (-0.30%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | `93.54% <93.54%> (ø)` | |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | `77.50% <77.50%> (ø)` | |
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | `79.84% <79.84%> (ø)` | |
... and [79 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18526/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-4957498833
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `80.83832%` with `64 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 68.68%. Comparing base ([`7bcb8be`](https://app.codecov.io/gh/apache/hudi/commit/7bcb8be7c153010998574f591c7ff3799192eda6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`6d8dedd`](https://app.codecov.io/gh/apache/hudi/commit/6d8dedd061ed183c52471df09355b85e5a60099a?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 332 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | 79.84% | [43 Missing and 10 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | 77.50% | [5 Missing and 4 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | 93.54% | [1 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18526 +/- ##
============================================
- Coverage 68.88% 68.68% -0.21%
- Complexity 28279 28473 +194
============================================
Files 2464 2478 +14
Lines 135568 136972 +1404
Branches 16446 16653 +207
============================================
+ Hits 93385 94077 +692
- Misses 34801 35324 +523
- Partials 7382 7571 +189
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.39% <13.77%> (-0.25%)` | :arrow_down: |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.74% <0.00%> (-0.02%)` | :arrow_down: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.64% <77.46%> (+0.23%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `49.25% <0.00%> (+0.32%)` | :arrow_up: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.18% <0.00%> (-0.26%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `37.87% <77.54%> (-0.30%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | `93.54% <93.54%> (ø)` | |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | `77.50% <77.50%> (ø)` | |
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | `79.84% <79.84%> (ø)` | |
... and [79 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18526/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-5186210795
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `80.83832%` with `64 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 68.68%. Comparing base ([`7bcb8be`](https://app.codecov.io/gh/apache/hudi/commit/7bcb8be7c153010998574f591c7ff3799192eda6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`6d8dedd`](https://app.codecov.io/gh/apache/hudi/commit/6d8dedd061ed183c52471df09355b85e5a60099a?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 461 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | 79.84% | [43 Missing and 10 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | 77.50% | [5 Missing and 4 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | 93.54% | [1 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18526 +/- ##
============================================
- Coverage 68.88% 68.68% -0.21%
- Complexity 28279 28473 +194
============================================
Files 2464 2478 +14
Lines 135568 136972 +1404
Branches 16446 16653 +207
============================================
+ Hits 93385 94077 +692
- Misses 34801 35324 +523
- Partials 7382 7571 +189
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.39% <13.77%> (-0.25%)` | :arrow_down: |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.74% <0.00%> (-0.02%)` | :arrow_down: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.64% <77.46%> (+0.23%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `49.25% <0.00%> (+0.32%)` | :arrow_up: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.18% <0.00%> (-0.26%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `37.87% <77.54%> (-0.30%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | `93.54% <93.54%> (ø)` | |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | `77.50% <77.50%> (ø)` | |
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | `79.84% <79.84%> (ø)` | |
... and [79 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18526/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
- :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "hudi-agent (via GitHub)" <gi...@apache.org>.
hudi-agent commented on code in PR #18526:
URL: https://github.com/apache/hudi/pull/18526#discussion_r3127445821
##########
hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/client/validator/StashPartitionsPreCommitValidator.java:
##########
@@ -0,0 +1,195 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.hudi.client.validator;
+
+import org.apache.hudi.client.WriteStatus;
+import org.apache.hudi.common.config.HoodieMetadataConfig;
+import org.apache.hudi.common.data.HoodieData;
+import org.apache.hudi.common.engine.HoodieEngineContext;
+import org.apache.hudi.common.util.ReflectionUtils;
+import org.apache.hudi.common.util.StringUtils;
+import org.apache.hudi.common.util.ValidationUtils;
+import org.apache.hudi.config.HoodieWriteConfig;
+import org.apache.hudi.exception.HoodieValidationException;
+import org.apache.hudi.storage.HoodieStorage;
+import org.apache.hudi.storage.StoragePath;
+import org.apache.hudi.storage.StoragePathInfo;
+import org.apache.hudi.table.HoodieSparkTable;
+import org.apache.hudi.table.action.HoodieWriteMetadata;
+
+import org.apache.spark.sql.Dataset;
+import org.apache.spark.sql.Row;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.IOException;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+
+/**
+ * A special-purpose pre-commit validator that moves partition files to a stash
+ * location before a deletePartitions commit completes.
+ *
+ * <p><strong>WARNING: This is NOT a standard validator pattern.</strong>
+ * Unlike typical pre-commit validators that are read-only checks, this validator
+ * performs destructive side-effects (moving/deleting files). It is designed exclusively
+ * for use with the {@code HoodieStashPartitionsTool} and the DELETE_PARTITION operation.
+ * Do NOT use this as a pattern for other validators.</p>
+ *
+ * <h3>Why move files inside a pre-commit validator?</h3>
+ *
+ * <p>The stash operation needs to both move data files to a backup location and mark
+ * the partitions as deleted via a replace commit. These two actions must be coupled:
+ * if the replace commit lands on the timeline but the file move has not happened yet,
+ * the Hudi cleaner will eventually delete the physical data files for the replaced
+ * file groups (based on its retention policy), permanently losing the data that was
+ * intended to be stashed.
+ *
+ * <p>By performing the file move inside this pre-commit validator, the move happens
+ * <b>before</b> the replace commit is finalized. If the move fails, the validator
+ * throws an exception, the commit does not land, and the table remains unchanged.
+ * This guarantees that data is never in a state where it has been marked as deleted
+ * but has not been safely backed up to the stash location.</p>
+ *
+ * <p>This validator requires the following configs to be set:
+ * <ul>
+ * <li>{@link #STASH_PATH_CONFIG} — the target stash directory</li>
+ * <li>{@link #RENAME_HELPER_CLASS_CONFIG} (optional) — custom rename helper class</li>
+ * </ul>
+ *
+ * <p>Guards:
+ * <ul>
+ * <li>Fails if RLI (Record Level Index) is enabled on the table</li>
+ * <li>Must only be used with DELETE_PARTITION operation</li>
+ * </ul>
+ */
+public class StashPartitionsPreCommitValidator<T, I, K, O extends HoodieData<WriteStatus>>
+ extends SparkPreCommitValidator<T, I, K, O> {
+
+ private static final Logger LOG = LoggerFactory.getLogger(StashPartitionsPreCommitValidator.class);
+
+ /**
+ * Config key for the stash target directory path.
+ */
+ public static final String STASH_PATH_CONFIG = "hoodie.stash.partitions.target.path";
+
+ /**
+ * Config key for the custom rename helper class. Defaults to {@link DefaultStashPartitionRenameHelper}.
+ */
+ public static final String RENAME_HELPER_CLASS_CONFIG = "hoodie.stash.rename.helper.class";
+
+ public StashPartitionsPreCommitValidator(HoodieSparkTable<T> table,
+ HoodieEngineContext engineContext,
+ HoodieWriteConfig writeConfig) {
+ super(table, engineContext, writeConfig);
+ }
+
+ /**
+ * Override validate to extract affected partitions from partitionToReplaceFileIds rather than
+ * write stats, since deletePartitions produces empty write stats but populates the replacement map.
+ */
+ @Override
+ public void validate(String instantTime, HoodieWriteMetadata<O> writeResult,
+ Dataset<Row> before, Dataset<Row> after) throws HoodieValidationException {
+ Set<String> partitionsAffected = new HashSet<>(writeResult.getPartitionToReplaceFileIds().keySet());
+ validateRecordsBeforeAndAfter(before, after, partitionsAffected);
+ }
+
+ /**
+ * Moves partition files to the stash location before the replace commit lands.
+ *
+ * <p><b>Partial failure semantics:</b> if the move fails partway through the partition list
+ * (e.g., partition k succeeded but partition k+1 throws), the exception aborts the commit —
+ * but partitions 0..k have already had their files physically moved to the stash location.
+ * The on-disk layout is now partially mutated even though no commit landed. This validator
+ * does NOT attempt to roll back already-moved partitions on failure.
+ *
+ * <p>The {@code HoodieStashPartitionsTool} compensates for this via its pre-check, which
+ * detects partially stashed partitions and restores them before retrying. Callers using this
+ * validator outside the tool must perform their own recovery (restore files from stash back
+ * to source) before any subsequent write to the affected partitions.
+ */
+ @Override
+ protected void validateRecordsBeforeAndAfter(Dataset<Row> before,
+ Dataset<Row> after,
+ Set<String> partitionsAffected) {
+ // Guard: ensure RLI and secondary index are not enabled
+ HoodieMetadataConfig metadataConfig = getWriteConfig().getMetadataConfig();
+ ValidationUtils.checkState(
+ !metadataConfig.isRecordLevelIndexEnabled() && !metadataConfig.isGlobalRecordLevelIndexEnabled(),
+ "StashPartitionsPreCommitValidator does not support tables with Record Level Index (RLI) enabled. "
+ + "Disable RLI before using the stash partitions tool.");
+ ValidationUtils.checkState(
Review Comment:
🤖 RLI/Global-RLI/SI are now blocked, but other MDT indexes that carry partition-level state (column_stats, partition_stats, bloom_filters, expression_index) aren't checked. Is that intentional? If any of those can end up with stale entries after a raw file move that bypasses normal write plumbing, it might be worth either disallowing them here too, or clarifying in javadoc/docs which index combinations are supported.
<sub><i>- Generated by an AI agent and may contain mistakes. Please verify any suggestions before applying.</i></sub>
##########
hudi-utilities/src/test/java/org/apache/hudi/utilities/TestHoodieStashPartitionsTool.java:
##########
@@ -0,0 +1,488 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.hudi.utilities;
+
+import org.apache.hudi.client.SparkRDDWriteClient;
+import org.apache.hudi.client.WriteClientTestUtils;
+import org.apache.hudi.client.WriteStatus;
+import org.apache.hudi.client.validator.DefaultStashPartitionRenameHelper;
+import org.apache.hudi.common.config.HoodieMetadataConfig;
+import org.apache.hudi.common.model.HoodieRecord;
+import org.apache.hudi.common.model.HoodieTableType;
+import org.apache.hudi.common.table.HoodieTableMetaClient;
+import org.apache.hudi.common.testutils.HoodieTestDataGenerator;
+import org.apache.hudi.config.HoodieWriteConfig;
+import org.apache.hudi.storage.HoodieStorage;
+import org.apache.hudi.storage.StoragePath;
+import org.apache.hudi.storage.StoragePathInfo;
+import org.apache.hudi.testutils.SparkClientFunctionalTestHarness;
+
+import org.apache.spark.api.java.JavaRDD;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+import org.junit.jupiter.params.ParameterizedTest;
+import org.junit.jupiter.params.provider.ValueSource;
+
+import java.io.IOException;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.util.List;
+import java.util.stream.Collectors;
+
+import static org.apache.hudi.common.testutils.HoodieTestDataGenerator.DEFAULT_FIRST_PARTITION_PATH;
+import static org.apache.hudi.common.testutils.HoodieTestDataGenerator.DEFAULT_SECOND_PARTITION_PATH;
+import static org.apache.hudi.common.testutils.HoodieTestDataGenerator.DEFAULT_THIRD_PARTITION_PATH;
+import static org.apache.hudi.testutils.Assertions.assertNoWriteErrors;
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertFalse;
+import static org.junit.jupiter.api.Assertions.assertTrue;
+
+/**
+ * Tests for {@link HoodieStashPartitionsTool}.
+ */
+public class TestHoodieStashPartitionsTool extends SparkClientFunctionalTestHarness {
+
+ private HoodieTableMetaClient metaClient;
+ private HoodieTestDataGenerator dataGen;
+ private String stashPath;
+
+ @BeforeEach
+ public void init() throws IOException {
+ metaClient = getHoodieMetaClient(HoodieTableType.COPY_ON_WRITE);
+ dataGen = new HoodieTestDataGenerator();
+ Path stashDir = tempDir.resolve("stash");
+ Files.createDirectories(stashDir);
+ stashPath = stashDir.toAbsolutePath().toUri().toString();
+ }
+
+ /**
+ * Given: A COW table with data in multiple partitions.
+ * When: Stash tool runs in stash mode for one partition.
+ * Then: Files are moved to stash location, partition is deleted from table,
+ * and a replace commit exists on the timeline.
+ */
+ @ParameterizedTest
+ @ValueSource(booleans = {true, false})
+ public void testStashSinglePartition(boolean enableMetadata) throws IOException {
+ insertRecords("001", 20, enableMetadata);
+
+ HoodieStorage storage = hoodieStorage();
+ StoragePath sourcePartition = new StoragePath(basePath(), DEFAULT_FIRST_PARTITION_PATH);
+ StoragePath stashPartition = new StoragePath(stashPath, DEFAULT_FIRST_PARTITION_PATH);
+
+ assertTrue(storage.exists(sourcePartition));
+ assertFalse(storage.listDirectEntries(sourcePartition).isEmpty());
+
+ HoodieStashPartitionsTool.Config cfg = buildConfig(DEFAULT_FIRST_PARTITION_PATH, "stash", enableMetadata);
+ new HoodieStashPartitionsTool(jsc(), cfg).run();
+
+ assertTrue(storage.exists(stashPartition));
+ assertFalse(storage.listDirectEntries(stashPartition).isEmpty());
+ assertSourcePartitionEmpty(storage, sourcePartition);
+
+ HoodieTableMetaClient reloadedMetaClient = HoodieTableMetaClient.reload(metaClient);
+ assertFalse(reloadedMetaClient.getActiveTimeline().getCompletedReplaceTimeline().empty());
+
+ // Other partitions should be untouched
+ StoragePath secondPartition = new StoragePath(basePath(), DEFAULT_SECOND_PARTITION_PATH);
+ assertTrue(storage.exists(secondPartition));
+ assertFalse(storage.listDirectEntries(secondPartition).isEmpty());
+ }
+
+ /**
+ * Given: A COW table with data, and one partition already stashed.
+ * When: Stash tool runs in stash mode for the same partition again (retry/MDT case).
+ * Then: The validator skips the already-stashed partition gracefully.
+ */
+ @ParameterizedTest
+ @ValueSource(booleans = {true, false})
+ public void testStashAlreadyStashedPartitionIsIdempotent(boolean enableMetadata) throws IOException {
+ insertRecords("001", 20, enableMetadata);
+
+ HoodieStashPartitionsTool.Config cfg = buildConfig(DEFAULT_FIRST_PARTITION_PATH, "stash", enableMetadata);
+ new HoodieStashPartitionsTool(jsc(), cfg).run();
+
+ HoodieStorage storage = hoodieStorage();
+ StoragePath stashPartition = new StoragePath(stashPath, DEFAULT_FIRST_PARTITION_PATH);
+ List<StoragePathInfo> stashFilesAfterFirst = storage.listDirectEntries(stashPartition);
+
+ // Run again
+ new HoodieStashPartitionsTool(jsc(), cfg).run();
+
+ List<StoragePathInfo> stashFilesAfterSecond = storage.listDirectEntries(stashPartition);
+ assertEquals(stashFilesAfterFirst.size(), stashFilesAfterSecond.size());
+ }
+
+ /**
+ * Given: A COW table with data in multiple partitions.
+ * When: Stash tool runs in stash mode for multiple partitions.
+ * Then: All specified partitions are moved to stash location.
+ */
+ @ParameterizedTest
+ @ValueSource(booleans = {true, false})
+ public void testStashMultiplePartitions(boolean enableMetadata) throws IOException {
+ insertRecords("001", 30, enableMetadata);
+
+ String partitions = DEFAULT_FIRST_PARTITION_PATH + "," + DEFAULT_SECOND_PARTITION_PATH;
+ HoodieStashPartitionsTool.Config cfg = buildConfig(partitions, "stash", enableMetadata);
+ new HoodieStashPartitionsTool(jsc(), cfg).run();
+
+ HoodieStorage storage = hoodieStorage();
+ for (String partition : new String[]{DEFAULT_FIRST_PARTITION_PATH, DEFAULT_SECOND_PARTITION_PATH}) {
+ StoragePath stashPartition = new StoragePath(stashPath, partition);
+ assertTrue(storage.exists(stashPartition));
+ assertFalse(storage.listDirectEntries(stashPartition).isEmpty());
+ assertSourcePartitionEmpty(storage, new StoragePath(basePath(), partition));
+ }
+
+ // Third partition untouched
+ StoragePath thirdPartition = new StoragePath(basePath(), DEFAULT_THIRD_PARTITION_PATH);
+ assertTrue(storage.exists(thirdPartition));
+ assertFalse(storage.listDirectEntries(thirdPartition).isEmpty());
+ }
+
+ /**
+ * Given: A stash fully completed (replace commit landed, data in stash, no active files).
+ * When: Rollback stash is run.
+ * Then: Rollback is skipped — files stay in stash, source remains empty. The user must
+ * use insert_overwrite to restore the partition.
+ */
+ @ParameterizedTest
+ @ValueSource(booleans = {true, false})
+ public void testRollbackStashSkipsCompletedStash(boolean enableMetadata) throws IOException {
+ insertRecords("001", 20, enableMetadata);
+
+ HoodieStorage storage = hoodieStorage();
+ StoragePath sourcePartition = new StoragePath(basePath(), DEFAULT_FIRST_PARTITION_PATH);
+ StoragePath stashPartition = new StoragePath(stashPath, DEFAULT_FIRST_PARTITION_PATH);
+
+ // Stash the partition (replace commit lands)
+ new HoodieStashPartitionsTool(jsc(), buildConfig(DEFAULT_FIRST_PARTITION_PATH, "stash", enableMetadata)).run();
+ assertSourcePartitionEmpty(storage, sourcePartition);
+ assertFalse(storage.listDirectEntries(stashPartition).isEmpty());
+
+ // Run rollback — should be a no-op since stash fully completed
+ new HoodieStashPartitionsTool(jsc(), buildConfig(DEFAULT_FIRST_PARTITION_PATH, "rollback_stash", enableMetadata)).run();
+
+ // Stash files should still be in stash (not moved back)
+ assertFalse(storage.listDirectEntries(stashPartition).isEmpty(),
+ "Stash files should remain in stash after rollback of a completed stash");
+ // Source should still be empty
+ assertSourcePartitionEmpty(storage, sourcePartition);
+ }
+
+ /**
+ * Given: A partial stash attempt — pre-commit validator moved files to stash but the
+ * commit never landed, so partition still has active files and stash has data.
+ * When: Rollback stash is run.
+ * Then: Files are restored from stash back to source, making the partition whole again.
+ */
+ @ParameterizedTest
+ @ValueSource(booleans = {true, false})
+ public void testRollbackStashRestoresPartialAttempt(boolean enableMetadata) throws IOException {
+ insertRecords("001", 20, enableMetadata);
+
+ HoodieStorage storage = hoodieStorage();
+ StoragePath sourcePartition = new StoragePath(basePath(), DEFAULT_FIRST_PARTITION_PATH);
+ List<StoragePathInfo> originalFiles = storage.listDirectEntries(sourcePartition);
+ int originalFileCount = originalFiles.size();
+ assertTrue(originalFileCount > 0);
+
+ // Simulate partial stash: the rename helper copies all files to stash first, then deletes
+ // from source. A crash after copy but before delete leaves files in both locations.
+ // We simulate this by copying the source files to stash without deleting them from source.
+ StoragePath stashPartition = new StoragePath(stashPath, DEFAULT_FIRST_PARTITION_PATH);
+ storage.createDirectory(stashPartition);
+ for (StoragePathInfo fileInfo : originalFiles) {
+ StoragePath destFile = new StoragePath(stashPartition, fileInfo.getPath().getName());
+ org.apache.hudi.io.util.FileIOUtils.copy(storage, fileInfo.getPath(), destFile);
+ }
+
+ // Verify: partition still has files, stash also has data
+ assertFalse(storage.listDirectEntries(sourcePartition).isEmpty());
+ assertFalse(storage.listDirectEntries(stashPartition).isEmpty());
+
+ // Run rollback — should restore stash data back to source
+ new HoodieStashPartitionsTool(jsc(), buildConfig(DEFAULT_FIRST_PARTITION_PATH, "rollback_stash", enableMetadata)).run();
+
+ // Stash should be empty after rollback
+ assertTrue(storage.listDirectEntries(stashPartition).isEmpty(),
+ "Stash should be empty after rollback of partial attempt");
+ // Source should still have all its files
+ assertFalse(storage.listDirectEntries(sourcePartition).isEmpty());
+ }
+
+ /**
+ * Given: No partition has been stashed (no stash data), partition has active files.
+ * When: Rollback stash is run.
+ * Then: Tool completes without error (no-op), partition is untouched.
+ */
+ @ParameterizedTest
+ @ValueSource(booleans = {true, false})
+ public void testRollbackStashNoOpWhenNothingStashed(boolean enableMetadata) throws IOException {
+ insertRecords("001", 10, enableMetadata);
+
+ HoodieStorage storage = hoodieStorage();
+ StoragePath sourcePartition = new StoragePath(basePath(), DEFAULT_FIRST_PARTITION_PATH);
+ int fileCountBefore = storage.listDirectEntries(sourcePartition).size();
+
+ HoodieStashPartitionsTool.Config cfg = buildConfig(DEFAULT_FIRST_PARTITION_PATH, "rollback_stash", enableMetadata);
+ new HoodieStashPartitionsTool(jsc(), cfg).run();
+
+ // Source should be untouched
+ assertEquals(fileCountBefore, storage.listDirectEntries(sourcePartition).size());
+ }
+
+ /**
+ * Given: No stash data and no active files in the partition.
+ * When: Rollback stash is run.
+ * Then: Tool completes without error (no-op, warns).
+ */
+ @ParameterizedTest
+ @ValueSource(booleans = {true, false})
+ public void testRollbackStashNoOpWhenNoDataAnywhere(boolean enableMetadata) throws IOException {
+ insertRecords("001", 20, enableMetadata);
+
+ // Stash the partition first (so it gets deleted), then clean up stash manually
+ // to simulate a state where there's no data anywhere.
+ new HoodieStashPartitionsTool(jsc(), buildConfig(DEFAULT_FIRST_PARTITION_PATH, "stash", enableMetadata)).run();
+
+ HoodieStorage storage = hoodieStorage();
+ StoragePath stashPartition = new StoragePath(stashPath, DEFAULT_FIRST_PARTITION_PATH);
+ // Remove stash data to simulate the "no data anywhere" state
+ for (StoragePathInfo file : storage.listDirectEntries(stashPartition)) {
+ storage.deleteFile(file.getPath());
+ }
+ assertTrue(storage.listDirectEntries(stashPartition).isEmpty());
+
+ // Run rollback — should be a no-op with warning
+ new HoodieStashPartitionsTool(jsc(), buildConfig(DEFAULT_FIRST_PARTITION_PATH, "rollback_stash", enableMetadata)).run();
+
+ // Nothing should change
+ StoragePath sourcePartition = new StoragePath(basePath(), DEFAULT_FIRST_PARTITION_PATH);
+ assertSourcePartitionEmpty(storage, sourcePartition);
+ }
+
+ /**
+ * Given: A COW table with data.
+ * When: Dry run mode is executed.
+ * Then: No files are moved, no commits created.
+ */
+ @ParameterizedTest
+ @ValueSource(booleans = {true, false})
+ public void testDryRunDoesNotModifyTable(boolean enableMetadata) throws IOException {
+ insertRecords("001", 20, enableMetadata);
+
+ HoodieStorage storage = hoodieStorage();
+ StoragePath sourcePartition = new StoragePath(basePath(), DEFAULT_FIRST_PARTITION_PATH);
+ List<StoragePathInfo> filesBefore = storage.listDirectEntries(sourcePartition);
+ int commitCountBefore = metaClient.getActiveTimeline().countInstants();
+
+ new HoodieStashPartitionsTool(jsc(), buildConfig(DEFAULT_FIRST_PARTITION_PATH, "dry_run", enableMetadata)).run();
+
+ List<StoragePathInfo> filesAfter = storage.listDirectEntries(sourcePartition);
+ assertEquals(filesBefore.size(), filesAfter.size());
+
+ StoragePath stashPartition = new StoragePath(stashPath, DEFAULT_FIRST_PARTITION_PATH);
+ assertFalse(storage.exists(stashPartition) && !storage.listDirectEntries(stashPartition).isEmpty());
+
+ HoodieTableMetaClient reloaded = HoodieTableMetaClient.reload(metaClient);
+ assertEquals(commitCountBefore, reloaded.getActiveTimeline().countInstants());
+ }
+
+ /**
+ * Scenario (a): Stash for multiple partitions was started. Operation failed before reaching
+ * the pre-commit validator (no files moved, no commit). Retry should succeed without issues.
+ *
+ * We simulate this by manually creating a requested replace commit instant (as if startCommit
+ * ran but deletePartitions failed), then retrying with the tool.
+ */
+ @ParameterizedTest
+ @ValueSource(booleans = {true, false})
+ public void testRetryAfterFailureBeforePreCommitValidator(boolean enableMetadata) throws IOException {
+ insertRecords("001", 30, enableMetadata);
+
+ HoodieStorage storage = hoodieStorage();
+ String partitions = DEFAULT_FIRST_PARTITION_PATH + "," + DEFAULT_SECOND_PARTITION_PATH;
+
+ // Verify all partitions have data and nothing is in stash
+ for (String partition : new String[]{DEFAULT_FIRST_PARTITION_PATH, DEFAULT_SECOND_PARTITION_PATH}) {
+ StoragePath src = new StoragePath(basePath(), partition);
+ assertTrue(storage.exists(src));
+ assertFalse(storage.listDirectEntries(src).isEmpty());
+ }
+
+ // The first attempt "failed" before reaching the validator — no files moved, no commit.
+ // This is the initial state. Just retry the tool — it should work from scratch.
+ HoodieStashPartitionsTool.Config cfg = buildConfig(partitions, "stash", enableMetadata);
+ new HoodieStashPartitionsTool(jsc(), cfg).run();
+
+ // Verify success
+ for (String partition : new String[]{DEFAULT_FIRST_PARTITION_PATH, DEFAULT_SECOND_PARTITION_PATH}) {
+ StoragePath stashPartition = new StoragePath(stashPath, partition);
+ assertTrue(storage.exists(stashPartition));
+ assertFalse(storage.listDirectEntries(stashPartition).isEmpty());
+ assertSourcePartitionEmpty(storage, new StoragePath(basePath(), partition));
+ }
+ }
+
+ /**
+ * Scenario (b): Stash for multiple partitions was started. The pre-commit validator
+ * stashed some partitions (moved files) but crashed before completing. On retry, the
+ * tool's pre-check (non-MDT) should recover partial state, and the validator (MDT)
+ * should handle it. The stash should eventually succeed.
+ */
+ @ParameterizedTest
+ @ValueSource(booleans = {true, false})
+ public void testRetryAfterValidatorPartiallyStashedAndCrashed(boolean enableMetadata) throws IOException {
+ insertRecords("001", 30, enableMetadata);
+
+ HoodieStorage storage = hoodieStorage();
+ String[] partitions = {DEFAULT_FIRST_PARTITION_PATH, DEFAULT_SECOND_PARTITION_PATH, DEFAULT_THIRD_PARTITION_PATH};
+
+ // Simulate partial stash: move first partition's files to stash (as if validator did it),
+ // but no commit landed.
+ DefaultStashPartitionRenameHelper renameHelper = new DefaultStashPartitionRenameHelper();
+ StoragePath src = new StoragePath(basePath(), DEFAULT_FIRST_PARTITION_PATH);
+ StoragePath dest = new StoragePath(stashPath, DEFAULT_FIRST_PARTITION_PATH);
+ renameHelper.stashPartitionFiles(storage, src, dest);
+
+ // Verify partial state: first partition in stash, others untouched
+ assertTrue(storage.exists(dest));
+ assertFalse(storage.listDirectEntries(dest).isEmpty());
+ assertSourcePartitionEmpty(storage, src);
+
+ // Retry with the tool — should handle partial state and complete all partitions
+ String allPartitions = String.join(",", partitions);
+ HoodieStashPartitionsTool.Config cfg = buildConfig(allPartitions, "stash", enableMetadata);
+ new HoodieStashPartitionsTool(jsc(), cfg).run();
+
+ // Verify: all partitions stashed
+ for (String partition : partitions) {
+ StoragePath stashPartition = new StoragePath(stashPath, partition);
+ assertTrue(storage.exists(stashPartition), "Stash should exist for: " + partition);
+ assertFalse(storage.listDirectEntries(stashPartition).isEmpty(),
+ "Stash should have files for: " + partition);
+ assertSourcePartitionEmpty(storage, new StoragePath(basePath(), partition));
+ }
+
+ HoodieTableMetaClient reloaded = HoodieTableMetaClient.reload(metaClient);
+ assertFalse(reloaded.getActiveTimeline().getCompletedReplaceTimeline().empty());
+ }
+
+ /**
+ * Scenario (d): Pre-commit validator succeeded for all partitions (files moved), but
+ * crashed just after that (before the commit landed). On retry, the tool should handle
+ * the state where all files are in stash but no commit exists, and succeed.
+ *
+ * We simulate this by running a real stash (which lands the replace commit), then deleting
+ * the completed replace commit from the timeline. This leaves the table in a state where
+ * files are in stash, source is empty, and no replace commit exists — exactly the scenario
+ * where the validator succeeded but the commit failed to persist.
+ */
+ @Test
+ public void testRetryAfterValidatorSucceededButCrashedBeforeCommit() throws IOException {
+ insertRecords("001", 30, true);
+
+ HoodieStorage storage = hoodieStorage();
+ String[] partitions = {DEFAULT_FIRST_PARTITION_PATH, DEFAULT_SECOND_PARTITION_PATH, DEFAULT_THIRD_PARTITION_PATH};
+ String allPartitions = String.join(",", partitions);
+
+ // Step 1: Run a real stash — this moves files and lands a replace commit
+ HoodieStashPartitionsTool.Config stashCfg = buildConfig(allPartitions, "stash", true);
+ new HoodieStashPartitionsTool(jsc(), stashCfg).run();
+
+ // Verify stash completed: files in stash, source empty, replace commit exists
+ for (String partition : partitions) {
+ StoragePath stashPartition = new StoragePath(stashPath, partition);
+ assertTrue(storage.exists(stashPartition));
+ assertFalse(storage.listDirectEntries(stashPartition).isEmpty());
+ assertSourcePartitionEmpty(storage, new StoragePath(basePath(), partition));
+ }
+ metaClient = HoodieTableMetaClient.reload(metaClient);
+ assertFalse(metaClient.getActiveTimeline().getCompletedReplaceTimeline().empty());
Review Comment:
🤖 The new simulation runs a real stash (which lands the replace commit and updates MDT) and then deletes only the DT timeline's replace instant. In the real "validator succeeded but commit crashed" scenario, neither DT nor MDT would have a committed replace — so MDT still "knows" the partition was replaced. With `enableMetadata=true`, does `evaluatePartitionStates` then see `partitionHasFiles=false` and classify all partitions as `ALREADY_STASHED`, causing the retry to short-circuit without re-running deletePartitions? If so, the assertion that a replace commit exists after retry may be passing for a different reason than intended (e.g. the stale DT instant state leaves a lingering commit), or may only hold because MDT is also implicitly reset. Could you confirm what path is actually being exercised here?
<sub><i>- Generated by an AI agent and may contain mistakes. Please verify any suggestions before applying.</i></sub>
##########
hudi-utilities/src/main/java/org/apache/hudi/utilities/HoodieStashPartitionsTool.java:
##########
@@ -0,0 +1,629 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.hudi.utilities;
+
+import org.apache.hudi.client.HoodieWriteResult;
+import org.apache.hudi.client.SparkRDDWriteClient;
+import org.apache.hudi.client.validator.DefaultStashPartitionRenameHelper;
+import org.apache.hudi.client.validator.StashPartitionRenameHelper;
+import org.apache.hudi.client.validator.StashPartitionsPreCommitValidator;
+import org.apache.hudi.common.config.HoodieMetadataConfig;
+import org.apache.hudi.common.config.TypedProperties;
+import org.apache.hudi.common.engine.HoodieLocalEngineContext;
+import org.apache.hudi.common.fs.FSUtils;
+import org.apache.hudi.common.model.FileSlice;
+import org.apache.hudi.common.model.HoodieFileGroup;
+import org.apache.hudi.common.model.HoodieRecordPayload;
+import org.apache.hudi.common.table.HoodieTableMetaClient;
+import org.apache.hudi.common.table.timeline.HoodieTimeline;
+import org.apache.hudi.common.table.view.HoodieTableFileSystemView;
+import org.apache.hudi.common.util.Option;
+import org.apache.hudi.common.util.ReflectionUtils;
+import org.apache.hudi.common.util.StringUtils;
+import org.apache.hudi.config.HoodiePreCommitValidatorConfig;
+import org.apache.hudi.exception.HoodieException;
+import org.apache.hudi.exception.HoodieValidationException;
+import org.apache.hudi.hadoop.fs.HadoopFSUtils;
+import org.apache.hudi.metadata.HoodieTableMetadata;
+import org.apache.hudi.storage.HoodieStorage;
+import org.apache.hudi.storage.HoodieStorageUtils;
+import org.apache.hudi.storage.StoragePath;
+import org.apache.hudi.storage.StoragePathInfo;
+import org.apache.hudi.table.HoodieSparkTable;
+
+import com.beust.jcommander.JCommander;
+import com.beust.jcommander.Parameter;
+import org.apache.hadoop.fs.Path;
+import org.apache.spark.api.java.JavaSparkContext;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.IOException;
+import java.io.Serializable;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.HashMap;
+import java.util.LinkedHashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+
+/**
+ * A spark-submit tool for stashing (backing up and removing) Hudi table partitions.
+ *
+ * <p>This tool supports three modes:
+ * <ul>
+ * <li><b>stash</b> — Moves data files from specified partitions to a stash location
+ * and issues a deletePartitions operation on the Hudi table. Uses a custom
+ * {@link StashPartitionsPreCommitValidator} to move files atomically before commit.</li>
+ * <li><b>rollback_stash</b> — Copies any data found in the stash location back to the
+ * original partition paths. Used when the user no longer wants to proceed with stashing.</li>
+ * <li><b>dry_run</b> — Shows which partitions and file groups would be affected.</li>
+ * </ul>
+ *
+ * <h3>Why a pre-commit validator instead of moving files in the tool?</h3>
+ *
+ * <p>A naive approach would be to first run {@code deletePartitions} in the tool, and then
+ * move the data files to the stash location as a separate step. However, this is unsafe:
+ * if the tool crashes or fails after {@code deletePartitions} succeeds but before the data
+ * is moved to the stash location, the data is at risk. Once the replace commit from
+ * {@code deletePartitions} lands on the timeline, the Hudi cleaner will eventually delete
+ * the physical data files for the replaced file groups based on its configured retention
+ * policy (e.g., retain last N commits). If the cleaner runs before the tool is retried,
+ * the data intended for stashing is permanently lost.
+ *
+ * <p>By using a {@link StashPartitionsPreCommitValidator}, the file move happens
+ * <b>before</b> the {@code deletePartitions} commit is finalized. If the move fails, the
+ * commit does not land, and the table remains in its original state — the cleaner has
+ * nothing to clean. This ensures that data is never in a state where it has been marked
+ * as deleted (via a replace commit) but has not yet been safely copied to the stash location.
+ *
+ * <p>For restoring stashed partitions back into the Hudi table, use the standard
+ * {@code insert_overwrite} write operation reading from the stash location.
+ *
+ * <p>Example usage:
+ * <pre>
+ * spark-submit \
+ * --class org.apache.hudi.utilities.HoodieStashPartitionsTool \
+ * --master local[*] \
+ * --driver-memory 1g \
+ * --executor-memory 1g \
+ * hudi-utilities-bundle.jar \
+ * --base-path /path/to/hudi/table \
+ * --table-name myTable \
+ * --stash-path /path/to/stash/folder \
+ * --partitions datestr=2023-01-01,datestr=2023-01-02 \
+ * --mode stash
+ * </pre>
+ */
+public class HoodieStashPartitionsTool implements Serializable {
+
+ private static final long serialVersionUID = 1L;
+ private static final Logger LOG = LoggerFactory.getLogger(HoodieStashPartitionsTool.class);
+
+ private final transient JavaSparkContext jsc;
+ private final Config cfg;
+ private final TypedProperties props;
+ private final HoodieTableMetaClient metaClient;
+
+ public HoodieStashPartitionsTool(JavaSparkContext jsc, Config cfg) {
+ this.jsc = jsc;
+ this.cfg = cfg;
+ this.props = cfg.propsFilePath == null
+ ? UtilHelpers.buildProperties(cfg.configs)
+ : readConfigFromFileSystem(jsc, cfg);
+ this.metaClient = HoodieTableMetaClient.builder()
+ .setConf(HadoopFSUtils.getStorageConfWithCopy(jsc.hadoopConfiguration()))
+ .setBasePath(cfg.basePath)
+ .setLoadActiveTimelineOnLoad(true)
+ .build();
+ }
+
+ public enum Mode {
+ STASH,
+ ROLLBACK_STASH,
+ DRY_RUN
+ }
+
+ public static class Config implements Serializable {
+ @Parameter(names = {"--base-path", "-sp"}, description = "Base path for the Hudi table.", required = true)
+ public String basePath = null;
+
+ @Parameter(names = {"--table-name", "-tn"}, description = "Table name.", required = true)
+ public String tableName = null;
+
+ @Parameter(names = {"--stash-path"}, description = "Target path for stashing partition data.", required = true)
+ public String stashPath = null;
+
+ @Parameter(names = {"--partitions", "-p"}, description = "Comma separated list of partitions to stash.", required = true)
+ public String partitions = null;
+
+ @Parameter(names = {"--mode", "-m"}, description = "Set job mode: "
+ + "\"stash\" to move partition data to stash location and delete partitions; "
+ + "\"rollback_stash\" to copy stashed data back to table and abort stashing; "
+ + "\"dry_run\" to preview which partitions and files would be affected.", required = true)
+ public String runningMode = null;
+
+ @Parameter(names = {"--parallelism", "-pl"}, description = "Parallelism for hoodie operations.", required = false)
+ public int parallelism = 1500;
+
+ @Parameter(names = {"--rename-helper-class"}, description = "Fully qualified class name of a custom "
+ + "StashPartitionRenameHelper implementation. Defaults to DefaultStashPartitionRenameHelper.", required = false)
+ public String renameHelperClass = null;
+
+ @Parameter(names = {"--spark-master", "-ms"}, description = "Spark master.", required = false)
+ public String sparkMaster = null;
+
+ @Parameter(names = {"--spark-memory", "-sm"}, description = "Spark memory to use.", required = false)
+ public String sparkMemory = "1g";
+
+ @Parameter(names = {"--props"}, description = "Path to properties file on localfs or dfs, with configurations for "
+ + "hoodie client for stashing partitions.")
+ public String propsFilePath = null;
+
+ @Parameter(names = {"--hoodie-conf"}, description = "Any configuration that can be set in the properties file "
+ + "(using the CLI parameter \"--props\") can also be passed via command line using this parameter. This can be repeated.",
+ splitter = IdentitySplitter.class)
+ public List<String> configs = new ArrayList<>();
+
+ @Parameter(names = {"--help", "-h"}, help = true)
+ public Boolean help = false;
+
+ @Override
+ public String toString() {
+ return "HoodieStashPartitionsToolConfig {\n"
+ + " --base-path " + basePath + ", \n"
+ + " --table-name " + tableName + ", \n"
+ + " --stash-path " + stashPath + ", \n"
+ + " --partitions " + partitions + ", \n"
+ + " --mode " + runningMode + ", \n"
+ + " --parallelism " + parallelism + ", \n"
+ + " --rename-helper-class " + renameHelperClass + ", \n"
+ + " --spark-master " + sparkMaster + ", \n"
+ + " --spark-memory " + sparkMemory + ", \n"
+ + " --props " + propsFilePath + ", \n"
+ + " --hoodie-conf " + configs
+ + "\n}";
+ }
+ }
+
+ public static void main(String[] args) {
+ final Config cfg = new Config();
+ JCommander cmd = new JCommander(cfg, null, args);
+ if (cfg.help || args.length == 0) {
+ cmd.usage();
+ System.exit(1);
+ }
+ Map<String, String> sparkConfigMap = new HashMap<>();
+ sparkConfigMap.put("spark.executor.memory", cfg.sparkMemory);
+ JavaSparkContext jsc = UtilHelpers.buildSparkContext("Hoodie-Stash-Partitions",
+ cfg.sparkMaster, false, sparkConfigMap);
+
+ HoodieStashPartitionsTool tool = new HoodieStashPartitionsTool(jsc, cfg);
+ try {
+ tool.run();
+ } catch (Throwable throwable) {
+ LOG.error("Fail to run stash partitions for " + cfg, throwable);
+ throw new HoodieException("Fail to run stash partitions for " + cfg, throwable);
+ } finally {
+ jsc.stop();
+ }
+ }
+
+ public void run() {
+ try {
+ LOG.info(cfg.toString());
+ Mode mode = Mode.valueOf(cfg.runningMode.toUpperCase());
+ switch (mode) {
+ case STASH:
+ LOG.info(" ****** The Hoodie Stash Partitions Tool is in stash mode ****** ");
+ doStash();
+ break;
+ case ROLLBACK_STASH:
+ LOG.info(" ****** The Hoodie Stash Partitions Tool is in rollback_stash mode ****** ");
+ doRollbackStash();
+ break;
+ case DRY_RUN:
+ LOG.info(" ****** The Hoodie Stash Partitions Tool is in dry-run mode ****** ");
+ doDryRun();
+ break;
+ default:
+ LOG.info("Unsupported running mode [" + cfg.runningMode + "], quit the job directly");
+ }
+ } catch (Exception e) {
+ throw new HoodieException("Unable to stash partitions in " + cfg.basePath, e);
+ }
+ }
+
+ /**
+ * Stash partitions: validates partition state using table metadata, recovers any partial
+ * stash from a prior failed attempt, then issues deletePartitions with a custom pre-commit
+ * validator.
+ *
+ * <p>The pre-check uses {@link HoodieTableMetadata} (MDT-backed when available) to determine
+ * whether each partition still has active files. This allows the tool to distinguish between:
+ * <ul>
+ * <li>A prior successful stash (partition deleted + data in stash) → no-op, return early</li>
+ * <li>A partial prior attempt (partition has files + data in stash) → recover from stash, retry</li>
+ * <li>A fresh run (partition has files + no stash data) → proceed normally</li>
+ * </ul>
+ */
+ private void doStash() {
+ List<String> partitionsToStash = Arrays.asList(cfg.partitions.split(","));
+
+ if (!preCheckAndRecoverPartialStash(partitionsToStash)) {
+ // All partitions already stashed successfully — nothing to do.
+ return;
+ }
+
+ // Validate that all partitions are in a clean, compacted state before stashing.
+ validatePartitionsForStash(partitionsToStash);
+
+ // Set up props with stash-specific configs for the pre-commit validator
+ TypedProperties stashProps = new TypedProperties();
+ stashProps.putAll(props);
+ stashProps.setProperty(HoodiePreCommitValidatorConfig.VALIDATOR_CLASS_NAMES.key(),
+ StashPartitionsPreCommitValidator.class.getName());
+ stashProps.setProperty(StashPartitionsPreCommitValidator.STASH_PATH_CONFIG, cfg.stashPath);
+ if (!StringUtils.isNullOrEmpty(cfg.renameHelperClass)) {
+ stashProps.setProperty(StashPartitionsPreCommitValidator.RENAME_HELPER_CLASS_CONFIG, cfg.renameHelperClass);
+ }
+
+ try (SparkRDDWriteClient<HoodieRecordPayload> client =
+ UtilHelpers.createHoodieClient(jsc, cfg.basePath, "", cfg.parallelism, Option.empty(), stashProps)) {
+ String instantTime = client.startCommit(HoodieTimeline.REPLACE_COMMIT_ACTION);
+ LOG.info("Starting deletePartitions with instant {} for partitions: {}", instantTime, partitionsToStash);
+ HoodieWriteResult result = client.deletePartitions(partitionsToStash, instantTime);
+ client.commit(instantTime, result.getWriteStatuses(), Option.empty(), HoodieTimeline.REPLACE_COMMIT_ACTION,
+ result.getPartitionToReplaceFileIds(), Option.empty());
+ LOG.info("Successfully stashed partitions: {}", partitionsToStash);
+ }
+ }
+
+ /**
+ * Pre-check: evaluates partition states and recovers any partial stash from a prior
+ * failed attempt. For partitions with a partial stash (active files + stash data),
+ * restores files from stash back to source so deletePartitions sees the complete state.
+ *
+ * @return true if deletePartitions should proceed, false if all partitions are already stashed
+ */
+ private boolean preCheckAndRecoverPartialStash(List<String> partitions) {
+ LOG.info("Running pre-check: validating partition state via table metadata.");
+ Map<String, PartitionStashState> stateMap = evaluatePartitionStates(partitions);
+ HoodieStorage storage = HoodieStorageUtils.getStorage(cfg.basePath, metaClient.getStorageConf());
+ StashPartitionRenameHelper renameHelper = createRenameHelper();
+ StoragePath basePath = metaClient.getBasePath();
+ int alreadyStashedCount = 0;
+
+ for (Map.Entry<String, PartitionStashState> entry : stateMap.entrySet()) {
+ String partition = entry.getKey();
+ PartitionStashState state = entry.getValue();
+
+ switch (state) {
+ case ALREADY_STASHED:
+ LOG.info("Partition {} already stashed (no active files in table, data exists in stash). Skipping.", partition);
+ alreadyStashedCount++;
+ break;
+ case PARTIAL_STASH:
+ LOG.info("Partition {} has active files and stash data from a prior partial attempt. "
+ + "Restoring stash data back to source before retry.", partition);
+ restorePartitionFromStash(renameHelper, storage, partition, basePath);
+ break;
+ case NOT_STASHED:
+ LOG.info("Partition {} has active files and no prior stash data. Will proceed with stash.", partition);
+ break;
+ case NO_DATA:
+ LOG.warn("Partition {} has no active files and no stash data. "
+ + "It may have been deleted by another operation.", partition);
+ break;
+ default:
+ break;
+ }
+ }
+
+ if (alreadyStashedCount == partitions.size()) {
+ LOG.info("All {} partition(s) are already stashed successfully. Nothing to do.", partitions.size());
+ return false;
+ }
+ return true;
+ }
Review Comment:
🤖 If a subset of the provided partitions come back as `ALREADY_STASHED` and the rest as `NOT_STASHED`, this proceeds with the full partition list into deletePartitions and `validatePartitionsForStash`. For the already-stashed ones the partition is already gone on disk (and from MDT), so validateFileGroups vacuously passes and the replace commit attempts to delete a partition that's not present. Is that the intended behavior, or should already-stashed partitions be filtered out of the list before the next stages so the commit operates only on partitions that actually have data? A mid-run partial success today could put the user into this state (some partitions stashed + committed, others failed before reaching the commit).
<sub><i>- Generated by an AI agent and may contain mistakes. Please verify any suggestions before applying.</i></sub>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-4300695672
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `80.83832%` with `64 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 68.67%. Comparing base ([`7bcb8be`](https://app.codecov.io/gh/apache/hudi/commit/7bcb8be7c153010998574f591c7ff3799192eda6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`6d8dedd`](https://app.codecov.io/gh/apache/hudi/commit/6d8dedd061ed183c52471df09355b85e5a60099a?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 21 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | 79.84% | [43 Missing and 10 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | 77.50% | [5 Missing and 4 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | 93.54% | [1 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18526 +/- ##
============================================
- Coverage 68.88% 68.67% -0.21%
- Complexity 28279 28471 +192
============================================
Files 2464 2478 +14
Lines 135568 136972 +1404
Branches 16446 16653 +207
============================================
+ Hits 93385 94070 +685
- Misses 34801 35328 +527
- Partials 7382 7574 +192
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.39% <13.77%> (-0.25%)` | :arrow_down: |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.74% <0.00%> (-0.02%)` | :arrow_down: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.64% <77.46%> (+0.23%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `49.15% <0.00%> (+0.22%)` | :arrow_up: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.18% <0.00%> (-0.26%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `37.87% <77.54%> (-0.30%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | `93.54% <93.54%> (ø)` | |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | `77.50% <77.50%> (ø)` | |
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | `79.84% <79.84%> (ø)` | |
... and [82 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18526/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-4300680724
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `23.05389%` with `257 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 65.75%. Comparing base ([`7bcb8be`](https://app.codecov.io/gh/apache/hudi/commit/7bcb8be7c153010998574f591c7ff3799192eda6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`6d8dedd`](https://app.codecov.io/gh/apache/hudi/commit/6d8dedd061ed183c52471df09355b85e5a60099a?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 21 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | 6.46% | [245 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | 77.50% | [5 Missing and 4 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | 93.54% | [1 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18526 +/- ##
============================================
- Coverage 68.88% 65.75% -3.14%
+ Complexity 28279 27274 -1005
============================================
Files 2464 2478 +14
Lines 135568 136972 +1404
Branches 16446 16653 +207
============================================
- Hits 93385 90061 -3324
- Misses 34801 39485 +4684
- Partials 7382 7426 +44
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.39% <13.77%> (-0.25%)` | :arrow_down: |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.74% <0.00%> (-0.02%)` | :arrow_down: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.64% <77.46%> (+0.23%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.34% <0.00%> (-3.59%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.80% <0.00%> (-0.64%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | `93.54% <93.54%> (ø)` | |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | `77.50% <77.50%> (ø)` | |
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | `6.46% <6.46%> (ø)` | |
... and [325 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18526/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-4300795791
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "67f1e57f0dffe02bbe2d7bacfaa8f1aaa4be8c88",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=13351",
"triggerID" : "67f1e57f0dffe02bbe2d7bacfaa8f1aaa4be8c88",
"triggerType" : "PUSH"
}, {
"hash" : "e8b5a0a62cea99be0a184e0b6f11149b49850806",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=13354",
"triggerID" : "e8b5a0a62cea99be0a184e0b6f11149b49850806",
"triggerType" : "PUSH"
}, {
"hash" : "891eaf2e1427bf93b9c7ff9626628c30e1c9c01d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=13391",
"triggerID" : "891eaf2e1427bf93b9c7ff9626628c30e1c9c01d",
"triggerType" : "PUSH"
}, {
"hash" : "6d8dedd061ed183c52471df09355b85e5a60099a",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=13453",
"triggerID" : "6d8dedd061ed183c52471df09355b85e5a60099a",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 6d8dedd061ed183c52471df09355b85e5a60099a Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=13453)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-4272503167
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `78.47534%` with `48 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 61.46%. Comparing base ([`7bcb8be`](https://app.codecov.io/gh/apache/hudi/commit/7bcb8be7c153010998574f591c7ff3799192eda6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`e8b5a0a`](https://app.codecov.io/gh/apache/hudi/commit/e8b5a0a62cea99be0a184e0b6f11149b49850806?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 3 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | 76.92% | [28 Missing and 8 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | 75.00% | [6 Missing and 4 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | 92.59% | [1 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
> :exclamation: There is a different number of reports uploaded between BASE (7bcb8be) and HEAD (e8b5a0a). Click for more details.
>
> <details><summary>HEAD has 15 uploads less than BASE</summary>
>
>| Flag | BASE (7bcb8be) | HEAD (e8b5a0a) |
>|------|------|------|
>|spark-scala-tests|10|5|
>|spark-java-tests|15|5|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18526 +/- ##
============================================
- Coverage 68.88% 61.46% -7.43%
+ Complexity 28279 25198 -3081
============================================
Files 2464 2467 +3
Lines 135568 135817 +249
Branches 16446 16468 +22
============================================
- Hits 93385 83476 -9909
- Misses 34801 45336 +10535
+ Partials 7382 7005 -377
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.59% <11.21%> (-0.05%)` | :arrow_down: |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.70% <0.00%> (-0.06%)` | :arrow_down: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.46% <77.61%> (+0.06%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `32.29% <0.00%> (-16.64%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `32.48% <0.00%> (-12.96%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `38.31% <74.88%> (+0.14%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | `92.59% <92.59%> (ø)` | |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | `75.00% <75.00%> (ø)` | |
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | `76.92% <76.92%> (ø)` | |
... and [543 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18526/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-4272490186
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `23.31839%` with `171 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 45.62%. Comparing base ([`7bcb8be`](https://app.codecov.io/gh/apache/hudi/commit/7bcb8be7c153010998574f591c7ff3799192eda6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`e8b5a0a`](https://app.codecov.io/gh/apache/hudi/commit/e8b5a0a62cea99be0a184e0b6f11149b49850806?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 3 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | 0.00% | [156 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | 75.00% | [6 Missing and 4 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | 81.48% | [3 Missing and 2 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
> :exclamation: There is a different number of reports uploaded between BASE (7bcb8be) and HEAD (e8b5a0a). Click for more details.
>
> <details><summary>HEAD has 23 uploads less than BASE</summary>
>
>| Flag | BASE (7bcb8be) | HEAD (e8b5a0a) |
>|------|------|------|
>|spark-scala-tests|10|4|
>|spark-java-tests|15|0|
>|common-and-other-modules|1|0|
>|utilities|1|0|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18526 +/- ##
=============================================
- Coverage 68.88% 45.62% -23.26%
+ Complexity 28279 15375 -12904
=============================================
Files 2464 1982 -482
Lines 135568 110190 -25378
Branches 16446 13772 -2674
=============================================
- Hits 93385 50278 -43107
- Misses 34801 54608 +19807
+ Partials 7382 5304 -2078
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.70% <0.00%> (-0.06%)` | :arrow_down: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.46% <77.61%> (+0.06%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `32.61% <0.00%> (-12.83%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | `81.48% <81.48%> (ø)` | |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | `75.00% <75.00%> (ø)` | |
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | `0.00% <0.00%> (ø)` | |
... and [1598 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18526/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-4272525181
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `78.47534%` with `48 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 68.05%. Comparing base ([`7bcb8be`](https://app.codecov.io/gh/apache/hudi/commit/7bcb8be7c153010998574f591c7ff3799192eda6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`e8b5a0a`](https://app.codecov.io/gh/apache/hudi/commit/e8b5a0a62cea99be0a184e0b6f11149b49850806?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 3 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | 76.92% | [28 Missing and 8 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | 75.00% | [6 Missing and 4 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | 92.59% | [1 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18526 +/- ##
============================================
- Coverage 68.88% 68.05% -0.84%
+ Complexity 28279 27965 -314
============================================
Files 2464 2467 +3
Lines 135568 135817 +249
Branches 16446 16468 +22
============================================
- Hits 93385 92427 -958
- Misses 34801 35915 +1114
- Partials 7382 7475 +93
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.59% <11.21%> (-0.05%)` | :arrow_down: |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.70% <0.00%> (-0.06%)` | :arrow_down: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.46% <77.61%> (+0.06%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.11% <0.00%> (-3.82%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.35% <0.00%> (-0.09%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `38.31% <74.88%> (+0.14%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | `92.59% <92.59%> (ø)` | |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | `75.00% <75.00%> (ø)` | |
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | `76.92% <76.92%> (ø)` | |
... and [148 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18526/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-6406479172
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `80.83832%` with `64 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 68.68%. Comparing base ([`7bcb8be`](https://app.codecov.io/gh/apache/hudi/commit/7bcb8be7c153010998574f591c7ff3799192eda6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`6d8dedd`](https://app.codecov.io/gh/apache/hudi/commit/6d8dedd061ed183c52471df09355b85e5a60099a?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 802 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | 79.84% | [43 Missing and 10 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | 77.50% | [5 Missing and 4 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | 93.54% | [1 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18526 +/- ##
============================================
- Coverage 68.88% 68.68% -0.21%
- Complexity 28279 28473 +194
============================================
Files 2464 2478 +14
Lines 135568 136972 +1404
Branches 16446 16653 +207
============================================
+ Hits 93385 94077 +692
- Misses 34801 35324 +523
- Partials 7382 7571 +189
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.39% <13.77%> (-0.25%)` | :arrow_down: |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.74% <0.00%> (-0.02%)` | :arrow_down: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.64% <77.46%> (+0.23%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `49.25% <0.00%> (+0.32%)` | :arrow_up: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.18% <0.00%> (-0.26%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `37.87% <77.54%> (-0.30%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | `93.54% <93.54%> (ø)` | |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | `77.50% <77.50%> (ø)` | |
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | `79.84% <79.84%> (ø)` | |
... and [79 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18526/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
- :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "codecov-commenter (via GitHub) via commits" <co...@hudi.apache.org>.
codecov-commenter commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-6543696075
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `80.83832%` with `64 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 68.68%. Comparing base ([`7bcb8be`](https://app.codecov.io/gh/apache/hudi/commit/7bcb8be7c153010998574f591c7ff3799192eda6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`6d8dedd`](https://app.codecov.io/gh/apache/hudi/commit/6d8dedd061ed183c52471df09355b85e5a60099a?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 820 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | 79.84% | [43 Missing and 10 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | 77.50% | [5 Missing and 4 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | 93.54% | [1 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18526 +/- ##
============================================
- Coverage 68.88% 68.68% -0.21%
- Complexity 28279 28473 +194
============================================
Files 2464 2478 +14
Lines 135568 136972 +1404
Branches 16446 16653 +207
============================================
+ Hits 93385 94077 +692
- Misses 34801 35324 +523
- Partials 7382 7571 +189
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.39% <13.77%> (-0.25%)` | :arrow_down: |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.74% <0.00%> (-0.02%)` | :arrow_down: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.64% <77.46%> (+0.23%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `49.25% <0.00%> (+0.32%)` | :arrow_up: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.18% <0.00%> (-0.26%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `37.87% <77.54%> (-0.30%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | `93.54% <93.54%> (ø)` | |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | `77.50% <77.50%> (ø)` | |
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | `79.84% <79.84%> (ø)` | |
... and [79 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18526/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
- :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-6528384015
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `80.83832%` with `64 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 68.68%. Comparing base ([`7bcb8be`](https://app.codecov.io/gh/apache/hudi/commit/7bcb8be7c153010998574f591c7ff3799192eda6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`6d8dedd`](https://app.codecov.io/gh/apache/hudi/commit/6d8dedd061ed183c52471df09355b85e5a60099a?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 819 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | 79.84% | [43 Missing and 10 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | 77.50% | [5 Missing and 4 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | 93.54% | [1 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18526 +/- ##
============================================
- Coverage 68.88% 68.68% -0.21%
- Complexity 28279 28473 +194
============================================
Files 2464 2478 +14
Lines 135568 136972 +1404
Branches 16446 16653 +207
============================================
+ Hits 93385 94077 +692
- Misses 34801 35324 +523
- Partials 7382 7571 +189
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.39% <13.77%> (-0.25%)` | :arrow_down: |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.74% <0.00%> (-0.02%)` | :arrow_down: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.64% <77.46%> (+0.23%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `49.25% <0.00%> (+0.32%)` | :arrow_up: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.18% <0.00%> (-0.26%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `37.87% <77.54%> (-0.30%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | `93.54% <93.54%> (ø)` | |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | `77.50% <77.50%> (ø)` | |
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | `79.84% <79.84%> (ø)` | |
... and [79 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18526/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
- :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-4284602855
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `78.47534%` with `48 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 68.79%. Comparing base ([`7bcb8be`](https://app.codecov.io/gh/apache/hudi/commit/7bcb8be7c153010998574f591c7ff3799192eda6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`891eaf2`](https://app.codecov.io/gh/apache/hudi/commit/891eaf2e1427bf93b9c7ff9626628c30e1c9c01d?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 11 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | 76.92% | [28 Missing and 8 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | 75.00% | [6 Missing and 4 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | 92.59% | [1 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18526 +/- ##
============================================
- Coverage 68.88% 68.79% -0.10%
- Complexity 28279 28361 +82
============================================
Files 2464 2471 +7
Lines 135568 136218 +650
Branches 16446 16521 +75
============================================
+ Hits 93385 93707 +322
- Misses 34801 35095 +294
- Partials 7382 7416 +34
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.56% <11.21%> (-0.08%)` | :arrow_down: |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.75% <0.00%> (+<0.01%)` | :arrow_up: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.47% <77.61%> (+0.07%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.78% <0.00%> (-0.14%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.34% <0.00%> (-0.10%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `38.30% <74.88%> (+0.13%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | `92.59% <92.59%> (ø)` | |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | `75.00% <75.00%> (ø)` | |
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | `76.92% <76.92%> (ø)` | |
... and [25 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18526/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "yihua (via GitHub)" <gi...@apache.org>.
yihua commented on code in PR #18526:
URL: https://github.com/apache/hudi/pull/18526#discussion_r3105532167
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/validator/DefaultStashPartitionRenameHelper.java:
##########
@@ -0,0 +1,92 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.hudi.client.validator;
+
+import org.apache.hudi.io.util.FileIOUtils;
+import org.apache.hudi.storage.HoodieStorage;
+import org.apache.hudi.storage.StoragePath;
+import org.apache.hudi.storage.StoragePathInfo;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.IOException;
+import java.util.List;
+import java.util.Set;
+import java.util.stream.Collectors;
+
+/**
+ * Default implementation of {@link StashPartitionRenameHelper} that copies files
+ * individually from source to target and then deletes them from source.
+ * <p>
+ * This implementation is correct but may be slow for large partitions on cloud
+ * storage. For production use on HDFS or cloud storage with efficient rename APIs,
+ * a custom implementation can be plugged in via the
+ * {@code hoodie.stash.rename.helper.class} configuration.
+ */
+public class DefaultStashPartitionRenameHelper implements StashPartitionRenameHelper {
+
+ private static final long serialVersionUID = 1L;
+ private static final Logger LOG = LoggerFactory.getLogger(DefaultStashPartitionRenameHelper.class);
+
+ @Override
+ public void movePartitionFiles(HoodieStorage storage, StoragePath sourcePath, StoragePath targetPath) throws IOException {
+ // Ensure target directory exists
+ if (!storage.exists(targetPath)) {
+ storage.createDirectory(targetPath);
+ }
+
+ List<StoragePathInfo> sourceFiles = storage.listDirectEntries(sourcePath);
+ if (sourceFiles.isEmpty()) {
+ LOG.info("Source path {} has no files, nothing to move.", sourcePath);
+ return;
+ }
+
+ // Determine which files already exist in target (from a prior partial attempt)
+ Set<String> existingTargetFileNames = getExistingFileNames(storage, targetPath);
+
+ // Copy files not already in target
+ for (StoragePathInfo sourceFileInfo : sourceFiles) {
+ String fileName = sourceFileInfo.getPath().getName();
+ if (existingTargetFileNames.contains(fileName)) {
+ LOG.info("File {} already exists in target {}, skipping copy.", fileName, targetPath);
+ } else {
+ StoragePath destFilePath = new StoragePath(targetPath, fileName);
+ LOG.info("Copying file {} to {}", sourceFileInfo.getPath(), destFilePath);
+ FileIOUtils.copy(storage, sourceFileInfo.getPath(), destFilePath);
+ }
+ }
Review Comment:
🤖 The 'already in target' check only matches by file name. If a previous attempt was killed mid-copy and left a truncated file in the target, this will treat it as fully copied and skip, then delete the source copy — silently losing the complete data. Given that this helper is explicitly documented as needing idempotency across partial failures, comparing file size (and/or using a tmp-name + atomic rename in the copy) would make recovery safer.
<sub><i>- Generated by an AI agent and may contain mistakes. Please verify any suggestions before applying.</i></sub>
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/validator/DefaultStashPartitionRenameHelper.java:
##########
@@ -0,0 +1,92 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.hudi.client.validator;
+
+import org.apache.hudi.io.util.FileIOUtils;
+import org.apache.hudi.storage.HoodieStorage;
+import org.apache.hudi.storage.StoragePath;
+import org.apache.hudi.storage.StoragePathInfo;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.IOException;
+import java.util.List;
+import java.util.Set;
+import java.util.stream.Collectors;
+
+/**
+ * Default implementation of {@link StashPartitionRenameHelper} that copies files
+ * individually from source to target and then deletes them from source.
+ * <p>
+ * This implementation is correct but may be slow for large partitions on cloud
+ * storage. For production use on HDFS or cloud storage with efficient rename APIs,
+ * a custom implementation can be plugged in via the
+ * {@code hoodie.stash.rename.helper.class} configuration.
+ */
+public class DefaultStashPartitionRenameHelper implements StashPartitionRenameHelper {
+
+ private static final long serialVersionUID = 1L;
+ private static final Logger LOG = LoggerFactory.getLogger(DefaultStashPartitionRenameHelper.class);
+
+ @Override
+ public void movePartitionFiles(HoodieStorage storage, StoragePath sourcePath, StoragePath targetPath) throws IOException {
+ // Ensure target directory exists
+ if (!storage.exists(targetPath)) {
+ storage.createDirectory(targetPath);
+ }
+
+ List<StoragePathInfo> sourceFiles = storage.listDirectEntries(sourcePath);
+ if (sourceFiles.isEmpty()) {
+ LOG.info("Source path {} has no files, nothing to move.", sourcePath);
+ return;
+ }
+
+ // Determine which files already exist in target (from a prior partial attempt)
Review Comment:
🤖 `listDirectEntries` returns both files and subdirectories. If a partition directory happens to contain a subdir (or e.g. a stray `.hoodie_partition_metadata`-like nested path in some storage layouts), the copy / deleteFile loop below would fail or skip silently. Is it worth asserting `sourceFileInfo.isFile()` (or filtering out directories) before the copy, so we fail loudly on unexpected layouts rather than producing a partial move?
<sub><i>- Generated by an AI agent and may contain mistakes. Please verify any suggestions before applying.</i></sub>
##########
hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/client/validator/StashPartitionsPreCommitValidator.java:
##########
@@ -0,0 +1,181 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.hudi.client.validator;
+
+import org.apache.hudi.client.WriteStatus;
+import org.apache.hudi.common.config.HoodieMetadataConfig;
+import org.apache.hudi.common.data.HoodieData;
+import org.apache.hudi.common.engine.HoodieEngineContext;
+import org.apache.hudi.common.util.ReflectionUtils;
+import org.apache.hudi.common.util.StringUtils;
+import org.apache.hudi.common.util.ValidationUtils;
+import org.apache.hudi.config.HoodieWriteConfig;
+import org.apache.hudi.exception.HoodieValidationException;
+import org.apache.hudi.storage.HoodieStorage;
+import org.apache.hudi.storage.StoragePath;
+import org.apache.hudi.storage.StoragePathInfo;
+import org.apache.hudi.table.HoodieSparkTable;
+import org.apache.hudi.table.action.HoodieWriteMetadata;
+
+import org.apache.spark.sql.Dataset;
+import org.apache.spark.sql.Row;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.IOException;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+
+/**
+ * A special-purpose pre-commit validator that moves partition files to a stash
+ * location before a deletePartitions commit completes.
+ *
+ * <p><strong>WARNING: This is NOT a standard validator pattern.</strong>
+ * Unlike typical pre-commit validators that are read-only checks, this validator
+ * performs destructive side-effects (moving/deleting files). It is designed exclusively
+ * for use with the {@code HoodieStashPartitionsTool} and the DELETE_PARTITION operation.
+ * Do NOT use this as a pattern for other validators.</p>
+ *
+ * <h3>Why move files inside a pre-commit validator?</h3>
+ *
+ * <p>The stash operation needs to both move data files to a backup location and mark
+ * the partitions as deleted via a replace commit. These two actions must be coupled:
+ * if the replace commit lands on the timeline but the file move has not happened yet,
+ * the Hudi cleaner will eventually delete the physical data files for the replaced
+ * file groups (based on its retention policy), permanently losing the data that was
+ * intended to be stashed.
+ *
+ * <p>By performing the file move inside this pre-commit validator, the move happens
+ * <b>before</b> the replace commit is finalized. If the move fails, the validator
+ * throws an exception, the commit does not land, and the table remains unchanged.
+ * This guarantees that data is never in a state where it has been marked as deleted
+ * but has not been safely backed up to the stash location.</p>
+ *
+ * <p>This validator requires the following configs to be set:
+ * <ul>
+ * <li>{@link #STASH_PATH_CONFIG} — the target stash directory</li>
+ * <li>{@link #RENAME_HELPER_CLASS_CONFIG} (optional) — custom rename helper class</li>
+ * </ul>
+ *
+ * <p>Guards:
+ * <ul>
+ * <li>Fails if RLI (Record Level Index) is enabled on the table</li>
+ * <li>Must only be used with DELETE_PARTITION operation</li>
+ * </ul>
+ */
+public class StashPartitionsPreCommitValidator<T, I, K, O extends HoodieData<WriteStatus>>
+ extends SparkPreCommitValidator<T, I, K, O> {
+
+ private static final Logger LOG = LoggerFactory.getLogger(StashPartitionsPreCommitValidator.class);
+
+ /**
+ * Config key for the stash target directory path.
+ */
+ public static final String STASH_PATH_CONFIG = "hoodie.stash.partitions.target.path";
+
+ /**
+ * Config key for the custom rename helper class. Defaults to {@link DefaultStashPartitionRenameHelper}.
+ */
+ public static final String RENAME_HELPER_CLASS_CONFIG = "hoodie.stash.rename.helper.class";
+
+ public StashPartitionsPreCommitValidator(HoodieSparkTable<T> table,
+ HoodieEngineContext engineContext,
+ HoodieWriteConfig writeConfig) {
+ super(table, engineContext, writeConfig);
+ }
+
+ /**
+ * Override validate to extract affected partitions from partitionToReplaceFileIds rather than
+ * write stats, since deletePartitions produces empty write stats but populates the replacement map.
+ */
+ @Override
+ public void validate(String instantTime, HoodieWriteMetadata<O> writeResult,
+ Dataset<Row> before, Dataset<Row> after) throws HoodieValidationException {
+ Set<String> partitionsAffected = new HashSet<>(writeResult.getPartitionToReplaceFileIds().keySet());
+ if (partitionsAffected.isEmpty()) {
+ // Fall back to write stats if partitionToReplaceFileIds is not populated
+ partitionsAffected = getPartitionsModified(writeResult);
+ }
+ validateRecordsBeforeAndAfter(before, after, partitionsAffected);
+ }
+
+ @Override
+ protected void validateRecordsBeforeAndAfter(Dataset<Row> before,
+ Dataset<Row> after,
+ Set<String> partitionsAffected) {
+ // Guard: ensure RLI is not enabled
+ HoodieMetadataConfig metadataConfig = getWriteConfig().getMetadataConfig();
+ ValidationUtils.checkState(
+ !metadataConfig.isRecordLevelIndexEnabled() && !metadataConfig.isGlobalRecordLevelIndexEnabled(),
+ "StashPartitionsPreCommitValidator does not support tables with Record Level Index (RLI) enabled. "
+ + "Disable RLI before using the stash partitions tool.");
+
+ // Read stash path from config
+ String stashPath = getWriteConfig().getProps().getProperty(STASH_PATH_CONFIG);
+ ValidationUtils.checkState(!StringUtils.isNullOrEmpty(stashPath),
+ "Stash path must be configured via " + STASH_PATH_CONFIG);
+
+ // Instantiate rename helper
+ StashPartitionRenameHelper renameHelper = createRenameHelper();
+
+ HoodieStorage storage = getHoodieTable().getStorage();
+ StoragePath basePath = getHoodieTable().getMetaClient().getBasePath();
+
+ LOG.info("Stashing {} partition(s) to {}", partitionsAffected.size(), stashPath);
+
+ for (String partition : partitionsAffected) {
+ StoragePath sourcePartitionPath = new StoragePath(basePath, partition);
+ StoragePath targetPartitionPath = new StoragePath(stashPath, partition);
+
+ try {
Review Comment:
🤖 If the move fails partway through the partition list (partition k succeeded, partition k+1 throws), the exception aborts the commit — but partitions 0..k have already had their files moved to stash and are NOT moved back. The table's file-system layout on disk is now partially mutated even though no commit landed. The tool's pre-check compensates on the next run, but callers using this validator outside the tool (or without the pre-check) would see inconsistent on-disk state. Worth adding a javadoc warning that on failure the caller must run recovery before any subsequent write, or catch-and-roll-back within this loop.
<sub><i>- Generated by an AI agent and may contain mistakes. Please verify any suggestions before applying.</i></sub>
##########
hudi-utilities/src/main/java/org/apache/hudi/utilities/HoodieStashPartitionsTool.java:
##########
@@ -0,0 +1,386 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.hudi.utilities;
+
+import org.apache.hudi.client.HoodieWriteResult;
+import org.apache.hudi.client.SparkRDDWriteClient;
+import org.apache.hudi.client.validator.DefaultStashPartitionRenameHelper;
+import org.apache.hudi.client.validator.StashPartitionRenameHelper;
+import org.apache.hudi.client.validator.StashPartitionsPreCommitValidator;
+import org.apache.hudi.common.config.TypedProperties;
+import org.apache.hudi.common.model.HoodieRecordPayload;
+import org.apache.hudi.common.table.HoodieTableMetaClient;
+import org.apache.hudi.common.table.timeline.HoodieTimeline;
+import org.apache.hudi.common.util.Option;
+import org.apache.hudi.common.util.ReflectionUtils;
+import org.apache.hudi.common.util.StringUtils;
+import org.apache.hudi.config.HoodiePreCommitValidatorConfig;
+import org.apache.hudi.exception.HoodieException;
+import org.apache.hudi.hadoop.fs.HadoopFSUtils;
+import org.apache.hudi.storage.HoodieStorage;
+import org.apache.hudi.storage.HoodieStorageUtils;
+import org.apache.hudi.storage.StoragePath;
+import org.apache.hudi.storage.StoragePathInfo;
+
+import com.beust.jcommander.JCommander;
+import com.beust.jcommander.Parameter;
+import org.apache.hadoop.fs.Path;
+import org.apache.spark.api.java.JavaSparkContext;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.IOException;
+import java.io.Serializable;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+
+/**
+ * A spark-submit tool for stashing (backing up and removing) Hudi table partitions.
+ *
+ * <p>This tool supports three modes:
+ * <ul>
+ * <li><b>stash</b> — Moves data files from specified partitions to a stash location
+ * and issues a deletePartitions operation on the Hudi table. Uses a custom
+ * {@link StashPartitionsPreCommitValidator} to move files atomically before commit.</li>
+ * <li><b>rollback_stash</b> — Copies any data found in the stash location back to the
+ * original partition paths. Used when the user no longer wants to proceed with stashing.</li>
+ * <li><b>dry_run</b> — Shows which partitions and file groups would be affected.</li>
+ * </ul>
+ *
+ * <p>For restoring stashed partitions back into the Hudi table, use the standard
+ * {@code insert_overwrite} write operation reading from the stash location.
+ *
+ * <p>Example usage:
+ * <pre>
+ * spark-submit \
+ * --class org.apache.hudi.utilities.HoodieStashPartitionsTool \
+ * --master local[*] \
+ * --driver-memory 1g \
+ * --executor-memory 1g \
+ * hudi-utilities-bundle.jar \
+ * --base-path /path/to/hudi/table \
+ * --table-name myTable \
+ * --stash-path /path/to/stash/folder \
+ * --partitions datestr=2023-01-01,datestr=2023-01-02 \
+ * --mode stash
+ * </pre>
+ */
+public class HoodieStashPartitionsTool implements Serializable {
+
+ private static final long serialVersionUID = 1L;
+ private static final Logger LOG = LoggerFactory.getLogger(HoodieStashPartitionsTool.class);
+
+ private final transient JavaSparkContext jsc;
+ private final Config cfg;
+ private final TypedProperties props;
+ private final HoodieTableMetaClient metaClient;
+
+ public HoodieStashPartitionsTool(JavaSparkContext jsc, Config cfg) {
+ this.jsc = jsc;
+ this.cfg = cfg;
+ this.props = cfg.propsFilePath == null
+ ? UtilHelpers.buildProperties(cfg.configs)
+ : readConfigFromFileSystem(jsc, cfg);
+ this.metaClient = HoodieTableMetaClient.builder()
+ .setConf(HadoopFSUtils.getStorageConfWithCopy(jsc.hadoopConfiguration()))
+ .setBasePath(cfg.basePath)
+ .setLoadActiveTimelineOnLoad(true)
+ .build();
+ }
+
+ public enum Mode {
+ STASH,
+ ROLLBACK_STASH,
+ DRY_RUN
+ }
+
+ public static class Config implements Serializable {
+ @Parameter(names = {"--base-path", "-sp"}, description = "Base path for the Hudi table.", required = true)
+ public String basePath = null;
+
+ @Parameter(names = {"--table-name", "-tn"}, description = "Table name.", required = true)
+ public String tableName = null;
+
+ @Parameter(names = {"--stash-path"}, description = "Target path for stashing partition data.", required = true)
+ public String stashPath = null;
+
+ @Parameter(names = {"--partitions", "-p"}, description = "Comma separated list of partitions to stash.", required = true)
+ public String partitions = null;
+
+ @Parameter(names = {"--mode", "-m"}, description = "Set job mode: "
+ + "\"stash\" to move partition data to stash location and delete partitions; "
+ + "\"rollback_stash\" to copy stashed data back to table and abort stashing; "
+ + "\"dry_run\" to preview which partitions and files would be affected.", required = true)
+ public String runningMode = null;
+
+ @Parameter(names = {"--parallelism", "-pl"}, description = "Parallelism for hoodie operations.", required = false)
+ public int parallelism = 1500;
+
+ @Parameter(names = {"--rename-helper-class"}, description = "Fully qualified class name of a custom "
+ + "StashPartitionRenameHelper implementation. Defaults to DefaultStashPartitionRenameHelper.", required = false)
+ public String renameHelperClass = null;
+
+ @Parameter(names = {"--spark-master", "-ms"}, description = "Spark master.", required = false)
+ public String sparkMaster = null;
+
+ @Parameter(names = {"--spark-memory", "-sm"}, description = "Spark memory to use.", required = false)
+ public String sparkMemory = "1g";
+
+ @Parameter(names = {"--props"}, description = "Path to properties file on localfs or dfs, with configurations for "
+ + "hoodie client for stashing partitions.")
+ public String propsFilePath = null;
+
+ @Parameter(names = {"--hoodie-conf"}, description = "Any configuration that can be set in the properties file "
+ + "(using the CLI parameter \"--props\") can also be passed via command line using this parameter. This can be repeated.",
+ splitter = IdentitySplitter.class)
+ public List<String> configs = new ArrayList<>();
+
+ @Parameter(names = {"--help", "-h"}, help = true)
+ public Boolean help = false;
+
+ @Override
+ public String toString() {
+ return "HoodieStashPartitionsToolConfig {\n"
+ + " --base-path " + basePath + ", \n"
+ + " --table-name " + tableName + ", \n"
+ + " --stash-path " + stashPath + ", \n"
+ + " --partitions " + partitions + ", \n"
+ + " --mode " + runningMode + ", \n"
+ + " --parallelism " + parallelism + ", \n"
+ + " --rename-helper-class " + renameHelperClass + ", \n"
+ + " --spark-master " + sparkMaster + ", \n"
+ + " --spark-memory " + sparkMemory + ", \n"
+ + " --props " + propsFilePath + ", \n"
+ + " --hoodie-conf " + configs
+ + "\n}";
+ }
+ }
+
+ public static void main(String[] args) {
+ final Config cfg = new Config();
+ JCommander cmd = new JCommander(cfg, null, args);
+ if (cfg.help || args.length == 0) {
+ cmd.usage();
+ System.exit(1);
+ }
+ Map<String, String> sparkConfigMap = new HashMap<>();
+ sparkConfigMap.put("spark.executor.memory", cfg.sparkMemory);
+ JavaSparkContext jsc = UtilHelpers.buildSparkContext("Hoodie-Stash-Partitions",
+ cfg.sparkMaster, false, sparkConfigMap);
+
+ HoodieStashPartitionsTool tool = new HoodieStashPartitionsTool(jsc, cfg);
+ try {
+ tool.run();
+ } catch (Throwable throwable) {
+ LOG.error("Fail to run stash partitions for " + cfg, throwable);
+ } finally {
+ jsc.stop();
+ }
+ }
+
+ public void run() {
+ try {
+ LOG.info(cfg.toString());
+ Mode mode = Mode.valueOf(cfg.runningMode.toUpperCase());
+ switch (mode) {
+ case STASH:
+ LOG.info(" ****** The Hoodie Stash Partitions Tool is in stash mode ****** ");
+ doStash();
+ break;
+ case ROLLBACK_STASH:
+ LOG.info(" ****** The Hoodie Stash Partitions Tool is in rollback_stash mode ****** ");
+ doRollbackStash();
+ break;
+ case DRY_RUN:
+ LOG.info(" ****** The Hoodie Stash Partitions Tool is in dry-run mode ****** ");
+ doDryRun();
+ break;
+ default:
+ LOG.info("Unsupported running mode [" + cfg.runningMode + "], quit the job directly");
+ }
+ } catch (Exception e) {
+ throw new HoodieException("Unable to stash partitions in " + cfg.basePath, e);
+ }
+ }
+
+ /**
+ * Stash partitions: pre-check to recover any partial stash from a prior failed attempt,
+ * then issue deletePartitions with custom pre-commit validator. Hudi internally handles
+ * rollback of any prior failed instants. The pre-commit validator handles already-stashed
+ * partitions gracefully (source empty -> skip).
+ *
+ * <p>The pre-check runs regardless of whether MDT is enabled. Even with MDT, a prior failed
+ * attempt may have had its pre-commit validator move files to the stash location before
+ * crashing. Those files need to be moved back so that the subsequent deletePartitions
+ * sees the complete partition state.
+ */
+ private void doStash() {
+ List<String> partitionsToStash = Arrays.asList(cfg.partitions.split(","));
+
+ preCheckAndRecoverPartialStash(partitionsToStash);
+
+ // Set up props with stash-specific configs for the pre-commit validator
+ TypedProperties stashProps = new TypedProperties();
+ stashProps.putAll(props);
+ stashProps.setProperty(HoodiePreCommitValidatorConfig.VALIDATOR_CLASS_NAMES.key(),
+ StashPartitionsPreCommitValidator.class.getName());
+ stashProps.setProperty(StashPartitionsPreCommitValidator.STASH_PATH_CONFIG, cfg.stashPath);
+ if (!StringUtils.isNullOrEmpty(cfg.renameHelperClass)) {
+ stashProps.setProperty(StashPartitionsPreCommitValidator.RENAME_HELPER_CLASS_CONFIG, cfg.renameHelperClass);
+ }
+
+ try (SparkRDDWriteClient<HoodieRecordPayload> client =
+ UtilHelpers.createHoodieClient(jsc, cfg.basePath, "", cfg.parallelism, Option.empty(), stashProps)) {
+ String instantTime = client.startCommit(HoodieTimeline.REPLACE_COMMIT_ACTION);
+ LOG.info("Starting deletePartitions with instant {} for partitions: {}", instantTime, partitionsToStash);
+ HoodieWriteResult result = client.deletePartitions(partitionsToStash, instantTime);
+ client.commit(instantTime, result.getWriteStatuses(), Option.empty(), HoodieTimeline.REPLACE_COMMIT_ACTION,
+ result.getPartitionToReplaceFileIds(), Option.empty());
+ LOG.info("Successfully stashed partitions: {}", partitionsToStash);
+ }
+ }
+
+ /**
+ * Pre-check: if files exist in the stash location from a prior failed attempt,
+ * copy them back to source before retrying deletePartitions. This ensures the
+ * subsequent deletePartitions sees the complete partition state and creates
+ * a correct replace commit. Runs for both MDT and non-MDT tables, since the
+ * pre-commit validator may have moved files before crashing.
+ */
+ private void preCheckAndRecoverPartialStash(List<String> partitions) {
+ LOG.info("Running pre-check for partial stash recovery (non-MDT table).");
+ HoodieStorage storage = HoodieStorageUtils.getStorage(
+ cfg.basePath, metaClient.getStorageConf());
+ StashPartitionRenameHelper renameHelper = createRenameHelper();
+ StoragePath basePath = metaClient.getBasePath();
+
+ for (String partition : partitions) {
+ StoragePath stashPartitionPath = new StoragePath(cfg.stashPath, partition);
+ StoragePath sourcePartitionPath = new StoragePath(basePath, partition);
+
+ try {
+ if (!storage.exists(stashPartitionPath) || isDirectoryEmpty(storage, stashPartitionPath)) {
+ LOG.info("Partition {} has no stashed data from prior attempt, skipping pre-check.", partition);
+ continue;
+ }
+ // Files exist in stash — copy them back to source to ensure deletePartitions
+ // sees all files and creates a complete replace commit.
Review Comment:
🤖 `rollback_stash` copies data from stash back to source with no check against the Hudi timeline. If the stash commit already landed (successful stash), running rollback_stash repopulates the source directory but the replace commit still marks those file groups as deleted — readers won't see the restored data, and the cleaner will eventually delete these files. Could rollback_stash fail fast (or at least warn loudly) when a completed replace commit for these partitions exists? The javadoc says it's meant for 'aborting in-progress stashing', but nothing enforces that.
<sub><i>- Generated by an AI agent and may contain mistakes. Please verify any suggestions before applying.</i></sub>
##########
hudi-utilities/src/main/java/org/apache/hudi/utilities/HoodieStashPartitionsTool.java:
##########
@@ -0,0 +1,386 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.hudi.utilities;
+
+import org.apache.hudi.client.HoodieWriteResult;
+import org.apache.hudi.client.SparkRDDWriteClient;
+import org.apache.hudi.client.validator.DefaultStashPartitionRenameHelper;
+import org.apache.hudi.client.validator.StashPartitionRenameHelper;
+import org.apache.hudi.client.validator.StashPartitionsPreCommitValidator;
+import org.apache.hudi.common.config.TypedProperties;
+import org.apache.hudi.common.model.HoodieRecordPayload;
+import org.apache.hudi.common.table.HoodieTableMetaClient;
+import org.apache.hudi.common.table.timeline.HoodieTimeline;
+import org.apache.hudi.common.util.Option;
+import org.apache.hudi.common.util.ReflectionUtils;
+import org.apache.hudi.common.util.StringUtils;
+import org.apache.hudi.config.HoodiePreCommitValidatorConfig;
+import org.apache.hudi.exception.HoodieException;
+import org.apache.hudi.hadoop.fs.HadoopFSUtils;
+import org.apache.hudi.storage.HoodieStorage;
+import org.apache.hudi.storage.HoodieStorageUtils;
+import org.apache.hudi.storage.StoragePath;
+import org.apache.hudi.storage.StoragePathInfo;
+
+import com.beust.jcommander.JCommander;
+import com.beust.jcommander.Parameter;
+import org.apache.hadoop.fs.Path;
+import org.apache.spark.api.java.JavaSparkContext;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.IOException;
+import java.io.Serializable;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+
+/**
+ * A spark-submit tool for stashing (backing up and removing) Hudi table partitions.
+ *
+ * <p>This tool supports three modes:
+ * <ul>
+ * <li><b>stash</b> — Moves data files from specified partitions to a stash location
+ * and issues a deletePartitions operation on the Hudi table. Uses a custom
+ * {@link StashPartitionsPreCommitValidator} to move files atomically before commit.</li>
+ * <li><b>rollback_stash</b> — Copies any data found in the stash location back to the
+ * original partition paths. Used when the user no longer wants to proceed with stashing.</li>
+ * <li><b>dry_run</b> — Shows which partitions and file groups would be affected.</li>
+ * </ul>
+ *
+ * <p>For restoring stashed partitions back into the Hudi table, use the standard
+ * {@code insert_overwrite} write operation reading from the stash location.
+ *
+ * <p>Example usage:
+ * <pre>
+ * spark-submit \
+ * --class org.apache.hudi.utilities.HoodieStashPartitionsTool \
+ * --master local[*] \
+ * --driver-memory 1g \
+ * --executor-memory 1g \
+ * hudi-utilities-bundle.jar \
+ * --base-path /path/to/hudi/table \
+ * --table-name myTable \
+ * --stash-path /path/to/stash/folder \
+ * --partitions datestr=2023-01-01,datestr=2023-01-02 \
+ * --mode stash
+ * </pre>
+ */
+public class HoodieStashPartitionsTool implements Serializable {
+
+ private static final long serialVersionUID = 1L;
+ private static final Logger LOG = LoggerFactory.getLogger(HoodieStashPartitionsTool.class);
+
+ private final transient JavaSparkContext jsc;
+ private final Config cfg;
+ private final TypedProperties props;
+ private final HoodieTableMetaClient metaClient;
+
+ public HoodieStashPartitionsTool(JavaSparkContext jsc, Config cfg) {
+ this.jsc = jsc;
+ this.cfg = cfg;
+ this.props = cfg.propsFilePath == null
+ ? UtilHelpers.buildProperties(cfg.configs)
+ : readConfigFromFileSystem(jsc, cfg);
+ this.metaClient = HoodieTableMetaClient.builder()
+ .setConf(HadoopFSUtils.getStorageConfWithCopy(jsc.hadoopConfiguration()))
+ .setBasePath(cfg.basePath)
+ .setLoadActiveTimelineOnLoad(true)
+ .build();
+ }
+
+ public enum Mode {
+ STASH,
+ ROLLBACK_STASH,
+ DRY_RUN
+ }
+
+ public static class Config implements Serializable {
+ @Parameter(names = {"--base-path", "-sp"}, description = "Base path for the Hudi table.", required = true)
+ public String basePath = null;
+
+ @Parameter(names = {"--table-name", "-tn"}, description = "Table name.", required = true)
+ public String tableName = null;
+
+ @Parameter(names = {"--stash-path"}, description = "Target path for stashing partition data.", required = true)
+ public String stashPath = null;
+
+ @Parameter(names = {"--partitions", "-p"}, description = "Comma separated list of partitions to stash.", required = true)
+ public String partitions = null;
+
+ @Parameter(names = {"--mode", "-m"}, description = "Set job mode: "
+ + "\"stash\" to move partition data to stash location and delete partitions; "
+ + "\"rollback_stash\" to copy stashed data back to table and abort stashing; "
+ + "\"dry_run\" to preview which partitions and files would be affected.", required = true)
+ public String runningMode = null;
+
+ @Parameter(names = {"--parallelism", "-pl"}, description = "Parallelism for hoodie operations.", required = false)
+ public int parallelism = 1500;
+
+ @Parameter(names = {"--rename-helper-class"}, description = "Fully qualified class name of a custom "
+ + "StashPartitionRenameHelper implementation. Defaults to DefaultStashPartitionRenameHelper.", required = false)
+ public String renameHelperClass = null;
+
+ @Parameter(names = {"--spark-master", "-ms"}, description = "Spark master.", required = false)
+ public String sparkMaster = null;
+
+ @Parameter(names = {"--spark-memory", "-sm"}, description = "Spark memory to use.", required = false)
+ public String sparkMemory = "1g";
+
+ @Parameter(names = {"--props"}, description = "Path to properties file on localfs or dfs, with configurations for "
+ + "hoodie client for stashing partitions.")
+ public String propsFilePath = null;
+
+ @Parameter(names = {"--hoodie-conf"}, description = "Any configuration that can be set in the properties file "
+ + "(using the CLI parameter \"--props\") can also be passed via command line using this parameter. This can be repeated.",
+ splitter = IdentitySplitter.class)
+ public List<String> configs = new ArrayList<>();
+
+ @Parameter(names = {"--help", "-h"}, help = true)
+ public Boolean help = false;
+
+ @Override
+ public String toString() {
+ return "HoodieStashPartitionsToolConfig {\n"
+ + " --base-path " + basePath + ", \n"
+ + " --table-name " + tableName + ", \n"
+ + " --stash-path " + stashPath + ", \n"
+ + " --partitions " + partitions + ", \n"
+ + " --mode " + runningMode + ", \n"
+ + " --parallelism " + parallelism + ", \n"
+ + " --rename-helper-class " + renameHelperClass + ", \n"
+ + " --spark-master " + sparkMaster + ", \n"
+ + " --spark-memory " + sparkMemory + ", \n"
+ + " --props " + propsFilePath + ", \n"
+ + " --hoodie-conf " + configs
+ + "\n}";
+ }
+ }
+
+ public static void main(String[] args) {
+ final Config cfg = new Config();
+ JCommander cmd = new JCommander(cfg, null, args);
+ if (cfg.help || args.length == 0) {
+ cmd.usage();
+ System.exit(1);
+ }
+ Map<String, String> sparkConfigMap = new HashMap<>();
+ sparkConfigMap.put("spark.executor.memory", cfg.sparkMemory);
+ JavaSparkContext jsc = UtilHelpers.buildSparkContext("Hoodie-Stash-Partitions",
+ cfg.sparkMaster, false, sparkConfigMap);
+
+ HoodieStashPartitionsTool tool = new HoodieStashPartitionsTool(jsc, cfg);
+ try {
+ tool.run();
+ } catch (Throwable throwable) {
+ LOG.error("Fail to run stash partitions for " + cfg, throwable);
+ } finally {
+ jsc.stop();
+ }
+ }
+
+ public void run() {
+ try {
+ LOG.info(cfg.toString());
+ Mode mode = Mode.valueOf(cfg.runningMode.toUpperCase());
+ switch (mode) {
+ case STASH:
+ LOG.info(" ****** The Hoodie Stash Partitions Tool is in stash mode ****** ");
+ doStash();
+ break;
+ case ROLLBACK_STASH:
+ LOG.info(" ****** The Hoodie Stash Partitions Tool is in rollback_stash mode ****** ");
+ doRollbackStash();
+ break;
+ case DRY_RUN:
+ LOG.info(" ****** The Hoodie Stash Partitions Tool is in dry-run mode ****** ");
+ doDryRun();
+ break;
+ default:
+ LOG.info("Unsupported running mode [" + cfg.runningMode + "], quit the job directly");
+ }
+ } catch (Exception e) {
+ throw new HoodieException("Unable to stash partitions in " + cfg.basePath, e);
+ }
+ }
+
+ /**
+ * Stash partitions: pre-check to recover any partial stash from a prior failed attempt,
+ * then issue deletePartitions with custom pre-commit validator. Hudi internally handles
+ * rollback of any prior failed instants. The pre-commit validator handles already-stashed
+ * partitions gracefully (source empty -> skip).
+ *
+ * <p>The pre-check runs regardless of whether MDT is enabled. Even with MDT, a prior failed
+ * attempt may have had its pre-commit validator move files to the stash location before
+ * crashing. Those files need to be moved back so that the subsequent deletePartitions
+ * sees the complete partition state.
+ */
+ private void doStash() {
+ List<String> partitionsToStash = Arrays.asList(cfg.partitions.split(","));
+
+ preCheckAndRecoverPartialStash(partitionsToStash);
+
+ // Set up props with stash-specific configs for the pre-commit validator
Review Comment:
🤖 The pre-check unconditionally moves files back from stash to source whenever it sees data in the stash dir. But if a prior `stash` run actually succeeded (replace commit landed) and the user re-invokes `stash` with the same partitions, this will move data back into the source dirs even though the timeline still marks those file groups as replaced — leaving the table in an inconsistent state where files physically exist under partitions that Hudi considers deleted (and the cleaner will eventually remove them). Could the pre-check first verify that no completed replace commit exists for these partitions at the last instant (or at least error out with a clear message) before moving data back?
<sub><i>- Generated by an AI agent and may contain mistakes. Please verify any suggestions before applying.</i></sub>
##########
hudi-utilities/src/main/java/org/apache/hudi/utilities/HoodieStashPartitionsTool.java:
##########
@@ -0,0 +1,386 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.hudi.utilities;
+
+import org.apache.hudi.client.HoodieWriteResult;
+import org.apache.hudi.client.SparkRDDWriteClient;
+import org.apache.hudi.client.validator.DefaultStashPartitionRenameHelper;
+import org.apache.hudi.client.validator.StashPartitionRenameHelper;
+import org.apache.hudi.client.validator.StashPartitionsPreCommitValidator;
+import org.apache.hudi.common.config.TypedProperties;
+import org.apache.hudi.common.model.HoodieRecordPayload;
+import org.apache.hudi.common.table.HoodieTableMetaClient;
+import org.apache.hudi.common.table.timeline.HoodieTimeline;
+import org.apache.hudi.common.util.Option;
+import org.apache.hudi.common.util.ReflectionUtils;
+import org.apache.hudi.common.util.StringUtils;
+import org.apache.hudi.config.HoodiePreCommitValidatorConfig;
+import org.apache.hudi.exception.HoodieException;
+import org.apache.hudi.hadoop.fs.HadoopFSUtils;
+import org.apache.hudi.storage.HoodieStorage;
+import org.apache.hudi.storage.HoodieStorageUtils;
+import org.apache.hudi.storage.StoragePath;
+import org.apache.hudi.storage.StoragePathInfo;
+
+import com.beust.jcommander.JCommander;
+import com.beust.jcommander.Parameter;
+import org.apache.hadoop.fs.Path;
+import org.apache.spark.api.java.JavaSparkContext;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.IOException;
+import java.io.Serializable;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+
+/**
+ * A spark-submit tool for stashing (backing up and removing) Hudi table partitions.
+ *
+ * <p>This tool supports three modes:
+ * <ul>
+ * <li><b>stash</b> — Moves data files from specified partitions to a stash location
+ * and issues a deletePartitions operation on the Hudi table. Uses a custom
+ * {@link StashPartitionsPreCommitValidator} to move files atomically before commit.</li>
+ * <li><b>rollback_stash</b> — Copies any data found in the stash location back to the
+ * original partition paths. Used when the user no longer wants to proceed with stashing.</li>
+ * <li><b>dry_run</b> — Shows which partitions and file groups would be affected.</li>
+ * </ul>
+ *
+ * <p>For restoring stashed partitions back into the Hudi table, use the standard
+ * {@code insert_overwrite} write operation reading from the stash location.
+ *
+ * <p>Example usage:
+ * <pre>
+ * spark-submit \
+ * --class org.apache.hudi.utilities.HoodieStashPartitionsTool \
+ * --master local[*] \
+ * --driver-memory 1g \
+ * --executor-memory 1g \
+ * hudi-utilities-bundle.jar \
+ * --base-path /path/to/hudi/table \
+ * --table-name myTable \
+ * --stash-path /path/to/stash/folder \
+ * --partitions datestr=2023-01-01,datestr=2023-01-02 \
+ * --mode stash
+ * </pre>
+ */
+public class HoodieStashPartitionsTool implements Serializable {
+
+ private static final long serialVersionUID = 1L;
+ private static final Logger LOG = LoggerFactory.getLogger(HoodieStashPartitionsTool.class);
+
+ private final transient JavaSparkContext jsc;
+ private final Config cfg;
+ private final TypedProperties props;
+ private final HoodieTableMetaClient metaClient;
+
+ public HoodieStashPartitionsTool(JavaSparkContext jsc, Config cfg) {
+ this.jsc = jsc;
+ this.cfg = cfg;
+ this.props = cfg.propsFilePath == null
+ ? UtilHelpers.buildProperties(cfg.configs)
+ : readConfigFromFileSystem(jsc, cfg);
+ this.metaClient = HoodieTableMetaClient.builder()
+ .setConf(HadoopFSUtils.getStorageConfWithCopy(jsc.hadoopConfiguration()))
+ .setBasePath(cfg.basePath)
+ .setLoadActiveTimelineOnLoad(true)
+ .build();
+ }
+
+ public enum Mode {
+ STASH,
+ ROLLBACK_STASH,
+ DRY_RUN
+ }
+
+ public static class Config implements Serializable {
+ @Parameter(names = {"--base-path", "-sp"}, description = "Base path for the Hudi table.", required = true)
+ public String basePath = null;
+
+ @Parameter(names = {"--table-name", "-tn"}, description = "Table name.", required = true)
+ public String tableName = null;
+
+ @Parameter(names = {"--stash-path"}, description = "Target path for stashing partition data.", required = true)
+ public String stashPath = null;
+
+ @Parameter(names = {"--partitions", "-p"}, description = "Comma separated list of partitions to stash.", required = true)
+ public String partitions = null;
+
+ @Parameter(names = {"--mode", "-m"}, description = "Set job mode: "
+ + "\"stash\" to move partition data to stash location and delete partitions; "
+ + "\"rollback_stash\" to copy stashed data back to table and abort stashing; "
+ + "\"dry_run\" to preview which partitions and files would be affected.", required = true)
+ public String runningMode = null;
+
+ @Parameter(names = {"--parallelism", "-pl"}, description = "Parallelism for hoodie operations.", required = false)
+ public int parallelism = 1500;
+
+ @Parameter(names = {"--rename-helper-class"}, description = "Fully qualified class name of a custom "
+ + "StashPartitionRenameHelper implementation. Defaults to DefaultStashPartitionRenameHelper.", required = false)
+ public String renameHelperClass = null;
+
+ @Parameter(names = {"--spark-master", "-ms"}, description = "Spark master.", required = false)
+ public String sparkMaster = null;
+
+ @Parameter(names = {"--spark-memory", "-sm"}, description = "Spark memory to use.", required = false)
+ public String sparkMemory = "1g";
+
+ @Parameter(names = {"--props"}, description = "Path to properties file on localfs or dfs, with configurations for "
+ + "hoodie client for stashing partitions.")
+ public String propsFilePath = null;
+
+ @Parameter(names = {"--hoodie-conf"}, description = "Any configuration that can be set in the properties file "
+ + "(using the CLI parameter \"--props\") can also be passed via command line using this parameter. This can be repeated.",
+ splitter = IdentitySplitter.class)
+ public List<String> configs = new ArrayList<>();
+
+ @Parameter(names = {"--help", "-h"}, help = true)
+ public Boolean help = false;
+
+ @Override
+ public String toString() {
+ return "HoodieStashPartitionsToolConfig {\n"
+ + " --base-path " + basePath + ", \n"
+ + " --table-name " + tableName + ", \n"
+ + " --stash-path " + stashPath + ", \n"
+ + " --partitions " + partitions + ", \n"
+ + " --mode " + runningMode + ", \n"
+ + " --parallelism " + parallelism + ", \n"
+ + " --rename-helper-class " + renameHelperClass + ", \n"
+ + " --spark-master " + sparkMaster + ", \n"
+ + " --spark-memory " + sparkMemory + ", \n"
+ + " --props " + propsFilePath + ", \n"
Review Comment:
🤖 `main` catches `Throwable`, logs it, then falls through to `jsc.stop()` and the JVM exits with code 0. For a spark-submit operational tool like this one, silent success on failure is dangerous — orchestrators (Airflow, cron, etc.) would mark the run as successful even when the stash failed partway. Could you add a `System.exit(1)` (or rethrow) after the error log so the non-zero exit propagates?
<sub><i>- Generated by an AI agent and may contain mistakes. Please verify any suggestions before applying.</i></sub>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "yihua (via GitHub)" <gi...@apache.org>.
yihua commented on code in PR #18526:
URL: https://github.com/apache/hudi/pull/18526#discussion_r3105599121
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/validator/DefaultStashPartitionRenameHelper.java:
##########
@@ -0,0 +1,92 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.hudi.client.validator;
+
+import org.apache.hudi.io.util.FileIOUtils;
+import org.apache.hudi.storage.HoodieStorage;
+import org.apache.hudi.storage.StoragePath;
+import org.apache.hudi.storage.StoragePathInfo;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.IOException;
+import java.util.List;
+import java.util.Set;
+import java.util.stream.Collectors;
+
+/**
+ * Default implementation of {@link StashPartitionRenameHelper} that copies files
+ * individually from source to target and then deletes them from source.
+ * <p>
+ * This implementation is correct but may be slow for large partitions on cloud
+ * storage. For production use on HDFS or cloud storage with efficient rename APIs,
+ * a custom implementation can be plugged in via the
+ * {@code hoodie.stash.rename.helper.class} configuration.
+ */
+public class DefaultStashPartitionRenameHelper implements StashPartitionRenameHelper {
+
+ private static final long serialVersionUID = 1L;
+ private static final Logger LOG = LoggerFactory.getLogger(DefaultStashPartitionRenameHelper.class);
+
+ @Override
+ public void movePartitionFiles(HoodieStorage storage, StoragePath sourcePath, StoragePath targetPath) throws IOException {
+ // Ensure target directory exists
+ if (!storage.exists(targetPath)) {
+ storage.createDirectory(targetPath);
+ }
+
+ List<StoragePathInfo> sourceFiles = storage.listDirectEntries(sourcePath);
+ if (sourceFiles.isEmpty()) {
+ LOG.info("Source path {} has no files, nothing to move.", sourcePath);
+ return;
+ }
+
+ // Determine which files already exist in target (from a prior partial attempt)
+ Set<String> existingTargetFileNames = getExistingFileNames(storage, targetPath);
+
+ // Copy files not already in target
+ for (StoragePathInfo sourceFileInfo : sourceFiles) {
+ String fileName = sourceFileInfo.getPath().getName();
+ if (existingTargetFileNames.contains(fileName)) {
+ LOG.info("File {} already exists in target {}, skipping copy.", fileName, targetPath);
+ } else {
+ StoragePath destFilePath = new StoragePath(targetPath, fileName);
+ LOG.info("Copying file {} to {}", sourceFileInfo.getPath(), destFilePath);
+ FileIOUtils.copy(storage, sourceFileInfo.getPath(), destFilePath);
+ }
+ }
+
+ // Delete all source files after successful copy
+ for (StoragePathInfo sourceFileInfo : sourceFiles) {
+ LOG.info("Deleting source file {}", sourceFileInfo.getPath());
+ storage.deleteFile(sourceFileInfo.getPath());
Review Comment:
_⚠️ Potential issue_ | _🔴 Critical_
<details>
<summary>🧩 Analysis chain</summary>
🏁 Script executed:
```shell
#!/bin/bash
set -euo pipefail
# Verify helper logic: skip-existing + copy + delete sequence
rg -n -C3 'existingTargetFileNames|FileIOUtils.copy|deleteFile|listDirectEntries' \
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/validator/DefaultStashPartitionRenameHelper.java
# Verify copy semantics in FileIOUtils
rg -n -C25 'public static void copy\(' \
hudi-io/src/main/java/org/apache/hudi/io/util/FileIOUtils.java
```
Repository: yihua/hudi
Length of output: 4358
---
🏁 Script executed:
```shell
#!/bin/bash
set -euo pipefail
# Find listDirectEntries definition to understand what it returns
rg -n 'listDirectEntries' \
hudi-io/src/main/java/org/apache/hudi/storage/HoodieStorage.java \
-A 5 | head -50
# Also check if there's any filtering for files only
rg -n 'isDirectory|isFile' \
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/validator/DefaultStashPartitionRenameHelper.java
```
Repository: yihua/hudi
Length of output: 1278
---
🏁 Script executed:
```shell
#!/bin/bash
set -euo pipefail
# Find concrete implementations of HoodieStorage to see what listDirectEntries returns
fd -e java -path '*/storage/*' | head -20 | xargs -I {} grep -l 'class.*HoodieStorage' {} | head -5
# Search for implementations more broadly
rg 'class.*HoodieStorage' -t java | grep -v abstract | head -10
```
Repository: yihua/hudi
Length of output: 222
---
🏁 Script executed:
```shell
#!/bin/bash
set -euo pipefail
# Find concrete implementations of HoodieStorage
rg 'class.*extends.*HoodieStorage|class.*implements.*HoodieStorage' --type java | head -10
# Also check what StoragePathInfo contains
rg -A 20 'class StoragePathInfo' --type java | head -40
```
Repository: yihua/hudi
Length of output: 3068
---
🏁 Script executed:
```shell
#!/bin/bash
set -euo pipefail
# Check HoodieHadoopStorage implementation of deleteFile
rg -n 'public.*deleteFile' \
hudi-hadoop-common/src/main/java/org/apache/hudi/storage/hadoop/HoodieHadoopStorage.java \
-A 10 | head -40
# Also check if copy operation would fail on directories
rg -n 'public.*void copy' \
hudi-io/src/main/java/org/apache/hudi/io/util/FileIOUtils.java \
-B 2 -A 15 | head -50
```
Repository: yihua/hudi
Length of output: 1813
---
</details>
**Prevent data loss when retrying after interrupted copies.**
At line 68, existence-by-name is treated as success. With line 73 using non-atomic streaming copy (1024-byte chunks), a crash during copy leaves a partial destination file. On retry, the existence check treats this incomplete file as fully copied and line 80 deletes the source without validation, causing data loss.
Require completion validation before skipping recopy and deleting source—for example, temp-path copy followed by atomic rename, or verified size/checksum comparison.
Additionally, `listDirectEntries()` on line 56 returns both files and directories (indicated by the `isDirectory` flag in `StoragePathInfo`), but the code does not filter directories. Attempting to copy or delete directories will throw exceptions at lines 73 and 80.
<details>
<summary>🤖 Prompt for AI Agents</summary>
```
Verify each finding against the current code and only fix it if needed.
In
`@hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/validator/DefaultStashPartitionRenameHelper.java`
around lines 63 - 80, The current DefaultStashPartitionRenameHelper logic treats
existence-by-name as a successful copy and iterates over all entries from
listDirectEntries() without filtering directories, which can leave partial
destination files on interrupted copies and attempt to copy/delete directories.
Fix by filtering sourceFiles to only regular files (StoragePathInfo.isDirectory
== false) before processing, perform copies to a temporary path and then
atomically rename to the final StoragePath (replace direct FileIOUtils.copy
calls), and validate completion (e.g., compare sizes or checksums via
getExistingFileNames or an added helper) before skipping recopy or calling
storage.deleteFile; update getExistingFileNames usage to check validated
completion rather than name-only presence.
```
</details>
<!-- fingerprinting:phantom:poseidon:hawk -->
<!-- This is an auto-generated comment by CodeRabbit -->
— *CodeRabbit* ([original](https://github.com/yihua/hudi/pull/51#discussion_r3105547968)) (source:comment#3105547968)
##########
hudi-utilities/src/main/java/org/apache/hudi/utilities/HoodieStashPartitionsTool.java:
##########
@@ -0,0 +1,386 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.hudi.utilities;
+
+import org.apache.hudi.client.HoodieWriteResult;
+import org.apache.hudi.client.SparkRDDWriteClient;
+import org.apache.hudi.client.validator.DefaultStashPartitionRenameHelper;
+import org.apache.hudi.client.validator.StashPartitionRenameHelper;
+import org.apache.hudi.client.validator.StashPartitionsPreCommitValidator;
+import org.apache.hudi.common.config.TypedProperties;
+import org.apache.hudi.common.model.HoodieRecordPayload;
+import org.apache.hudi.common.table.HoodieTableMetaClient;
+import org.apache.hudi.common.table.timeline.HoodieTimeline;
+import org.apache.hudi.common.util.Option;
+import org.apache.hudi.common.util.ReflectionUtils;
+import org.apache.hudi.common.util.StringUtils;
+import org.apache.hudi.config.HoodiePreCommitValidatorConfig;
+import org.apache.hudi.exception.HoodieException;
+import org.apache.hudi.hadoop.fs.HadoopFSUtils;
+import org.apache.hudi.storage.HoodieStorage;
+import org.apache.hudi.storage.HoodieStorageUtils;
+import org.apache.hudi.storage.StoragePath;
+import org.apache.hudi.storage.StoragePathInfo;
+
+import com.beust.jcommander.JCommander;
+import com.beust.jcommander.Parameter;
+import org.apache.hadoop.fs.Path;
+import org.apache.spark.api.java.JavaSparkContext;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.IOException;
+import java.io.Serializable;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+
+/**
+ * A spark-submit tool for stashing (backing up and removing) Hudi table partitions.
+ *
+ * <p>This tool supports three modes:
+ * <ul>
+ * <li><b>stash</b> — Moves data files from specified partitions to a stash location
+ * and issues a deletePartitions operation on the Hudi table. Uses a custom
+ * {@link StashPartitionsPreCommitValidator} to move files atomically before commit.</li>
+ * <li><b>rollback_stash</b> — Copies any data found in the stash location back to the
+ * original partition paths. Used when the user no longer wants to proceed with stashing.</li>
+ * <li><b>dry_run</b> — Shows which partitions and file groups would be affected.</li>
+ * </ul>
+ *
+ * <p>For restoring stashed partitions back into the Hudi table, use the standard
+ * {@code insert_overwrite} write operation reading from the stash location.
+ *
+ * <p>Example usage:
+ * <pre>
+ * spark-submit \
+ * --class org.apache.hudi.utilities.HoodieStashPartitionsTool \
+ * --master local[*] \
+ * --driver-memory 1g \
+ * --executor-memory 1g \
+ * hudi-utilities-bundle.jar \
+ * --base-path /path/to/hudi/table \
+ * --table-name myTable \
+ * --stash-path /path/to/stash/folder \
+ * --partitions datestr=2023-01-01,datestr=2023-01-02 \
+ * --mode stash
+ * </pre>
+ */
+public class HoodieStashPartitionsTool implements Serializable {
+
+ private static final long serialVersionUID = 1L;
+ private static final Logger LOG = LoggerFactory.getLogger(HoodieStashPartitionsTool.class);
+
+ private final transient JavaSparkContext jsc;
+ private final Config cfg;
+ private final TypedProperties props;
+ private final HoodieTableMetaClient metaClient;
+
+ public HoodieStashPartitionsTool(JavaSparkContext jsc, Config cfg) {
+ this.jsc = jsc;
+ this.cfg = cfg;
+ this.props = cfg.propsFilePath == null
+ ? UtilHelpers.buildProperties(cfg.configs)
+ : readConfigFromFileSystem(jsc, cfg);
+ this.metaClient = HoodieTableMetaClient.builder()
+ .setConf(HadoopFSUtils.getStorageConfWithCopy(jsc.hadoopConfiguration()))
+ .setBasePath(cfg.basePath)
+ .setLoadActiveTimelineOnLoad(true)
+ .build();
+ }
+
+ public enum Mode {
+ STASH,
+ ROLLBACK_STASH,
+ DRY_RUN
+ }
+
+ public static class Config implements Serializable {
+ @Parameter(names = {"--base-path", "-sp"}, description = "Base path for the Hudi table.", required = true)
+ public String basePath = null;
+
+ @Parameter(names = {"--table-name", "-tn"}, description = "Table name.", required = true)
+ public String tableName = null;
+
+ @Parameter(names = {"--stash-path"}, description = "Target path for stashing partition data.", required = true)
+ public String stashPath = null;
+
+ @Parameter(names = {"--partitions", "-p"}, description = "Comma separated list of partitions to stash.", required = true)
+ public String partitions = null;
+
+ @Parameter(names = {"--mode", "-m"}, description = "Set job mode: "
+ + "\"stash\" to move partition data to stash location and delete partitions; "
+ + "\"rollback_stash\" to copy stashed data back to table and abort stashing; "
+ + "\"dry_run\" to preview which partitions and files would be affected.", required = true)
+ public String runningMode = null;
+
+ @Parameter(names = {"--parallelism", "-pl"}, description = "Parallelism for hoodie operations.", required = false)
+ public int parallelism = 1500;
+
+ @Parameter(names = {"--rename-helper-class"}, description = "Fully qualified class name of a custom "
+ + "StashPartitionRenameHelper implementation. Defaults to DefaultStashPartitionRenameHelper.", required = false)
+ public String renameHelperClass = null;
+
+ @Parameter(names = {"--spark-master", "-ms"}, description = "Spark master.", required = false)
+ public String sparkMaster = null;
+
+ @Parameter(names = {"--spark-memory", "-sm"}, description = "Spark memory to use.", required = false)
+ public String sparkMemory = "1g";
+
+ @Parameter(names = {"--props"}, description = "Path to properties file on localfs or dfs, with configurations for "
+ + "hoodie client for stashing partitions.")
+ public String propsFilePath = null;
+
+ @Parameter(names = {"--hoodie-conf"}, description = "Any configuration that can be set in the properties file "
+ + "(using the CLI parameter \"--props\") can also be passed via command line using this parameter. This can be repeated.",
+ splitter = IdentitySplitter.class)
+ public List<String> configs = new ArrayList<>();
+
+ @Parameter(names = {"--help", "-h"}, help = true)
+ public Boolean help = false;
+
+ @Override
+ public String toString() {
+ return "HoodieStashPartitionsToolConfig {\n"
+ + " --base-path " + basePath + ", \n"
+ + " --table-name " + tableName + ", \n"
+ + " --stash-path " + stashPath + ", \n"
+ + " --partitions " + partitions + ", \n"
+ + " --mode " + runningMode + ", \n"
+ + " --parallelism " + parallelism + ", \n"
+ + " --rename-helper-class " + renameHelperClass + ", \n"
+ + " --spark-master " + sparkMaster + ", \n"
+ + " --spark-memory " + sparkMemory + ", \n"
+ + " --props " + propsFilePath + ", \n"
+ + " --hoodie-conf " + configs
+ + "\n}";
+ }
+ }
+
+ public static void main(String[] args) {
+ final Config cfg = new Config();
+ JCommander cmd = new JCommander(cfg, null, args);
+ if (cfg.help || args.length == 0) {
+ cmd.usage();
+ System.exit(1);
+ }
+ Map<String, String> sparkConfigMap = new HashMap<>();
+ sparkConfigMap.put("spark.executor.memory", cfg.sparkMemory);
+ JavaSparkContext jsc = UtilHelpers.buildSparkContext("Hoodie-Stash-Partitions",
+ cfg.sparkMaster, false, sparkConfigMap);
+
+ HoodieStashPartitionsTool tool = new HoodieStashPartitionsTool(jsc, cfg);
+ try {
+ tool.run();
+ } catch (Throwable throwable) {
+ LOG.error("Fail to run stash partitions for " + cfg, throwable);
+ } finally {
+ jsc.stop();
+ }
Review Comment:
_⚠️ Potential issue_ | _🟠 Major_
**Exit with non-zero status when execution fails.**
At Line 195, failures are logged but the process still exits successfully. This breaks automation and monitoring around this CLI.
<details>
<summary>Proposed fix</summary>
```diff
- try {
+ int exitCode = 0;
+ try {
tool.run();
} catch (Throwable throwable) {
LOG.error("Fail to run stash partitions for " + cfg, throwable);
+ exitCode = 1;
} finally {
jsc.stop();
}
+ if (exitCode != 0) {
+ System.exit(exitCode);
+ }
```
</details>
<!-- suggestion_start -->
<details>
<summary>📝 Committable suggestion</summary>
> ‼️ **IMPORTANT**
> Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
```suggestion
int exitCode = 0;
try {
tool.run();
} catch (Throwable throwable) {
LOG.error("Fail to run stash partitions for " + cfg, throwable);
exitCode = 1;
} finally {
jsc.stop();
}
if (exitCode != 0) {
System.exit(exitCode);
}
```
</details>
<!-- suggestion_end -->
<details>
<summary>🤖 Prompt for AI Agents</summary>
```
Verify each finding against the current code and only fix it if needed.
In
`@hudi-utilities/src/main/java/org/apache/hudi/utilities/HoodieStashPartitionsTool.java`
around lines 193 - 199, The catch in HoodieStashPartitionsTool around tool.run()
logs failures but never signals an error exit; update the catch block in the
main execution path (where tool.run() is invoked in HoodieStashPartitionsTool)
to either rethrow the exception or call System.exit with a non-zero status
(e.g., System.exit(1)) after logging the error so the process exits with
failure; ensure jsc.stop() still runs in the finally block to clean up
SparkContext.
```
</details>
<!-- fingerprinting:phantom:poseidon:hawk -->
<!-- This is an auto-generated comment by CodeRabbit -->
— *CodeRabbit* ([original](https://github.com/yihua/hudi/pull/51#discussion_r3105547976)) (source:comment#3105547976)
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/validator/DefaultStashPartitionRenameHelper.java:
##########
@@ -0,0 +1,92 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.hudi.client.validator;
+
+import org.apache.hudi.io.util.FileIOUtils;
+import org.apache.hudi.storage.HoodieStorage;
+import org.apache.hudi.storage.StoragePath;
+import org.apache.hudi.storage.StoragePathInfo;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.IOException;
+import java.util.List;
+import java.util.Set;
+import java.util.stream.Collectors;
+
+/**
+ * Default implementation of {@link StashPartitionRenameHelper} that copies files
+ * individually from source to target and then deletes them from source.
+ * <p>
+ * This implementation is correct but may be slow for large partitions on cloud
+ * storage. For production use on HDFS or cloud storage with efficient rename APIs,
+ * a custom implementation can be plugged in via the
+ * {@code hoodie.stash.rename.helper.class} configuration.
+ */
+public class DefaultStashPartitionRenameHelper implements StashPartitionRenameHelper {
+
+ private static final long serialVersionUID = 1L;
+ private static final Logger LOG = LoggerFactory.getLogger(DefaultStashPartitionRenameHelper.class);
+
+ @Override
+ public void movePartitionFiles(HoodieStorage storage, StoragePath sourcePath, StoragePath targetPath) throws IOException {
+ // Ensure target directory exists
+ if (!storage.exists(targetPath)) {
+ storage.createDirectory(targetPath);
+ }
+
+ List<StoragePathInfo> sourceFiles = storage.listDirectEntries(sourcePath);
+ if (sourceFiles.isEmpty()) {
+ LOG.info("Source path {} has no files, nothing to move.", sourcePath);
+ return;
+ }
+
+ // Determine which files already exist in target (from a prior partial attempt)
+ Set<String> existingTargetFileNames = getExistingFileNames(storage, targetPath);
+
+ // Copy files not already in target
+ for (StoragePathInfo sourceFileInfo : sourceFiles) {
+ String fileName = sourceFileInfo.getPath().getName();
+ if (existingTargetFileNames.contains(fileName)) {
+ LOG.info("File {} already exists in target {}, skipping copy.", fileName, targetPath);
+ } else {
+ StoragePath destFilePath = new StoragePath(targetPath, fileName);
+ LOG.info("Copying file {} to {}", sourceFileInfo.getPath(), destFilePath);
+ FileIOUtils.copy(storage, sourceFileInfo.getPath(), destFilePath);
+ }
+ }
+
+ // Delete all source files after successful copy
+ for (StoragePathInfo sourceFileInfo : sourceFiles) {
+ LOG.info("Deleting source file {}", sourceFileInfo.getPath());
+ storage.deleteFile(sourceFileInfo.getPath());
Review Comment:
_⚠️ Potential issue_ | _🟠 Major_
**Handle directory entries explicitly before moving.**
`listDirectEntries` can return directories, but Lines 73 and 80 apply file operations unconditionally. That can fail on nested paths and leave partitions partially moved. Please recurse for directories (or reject them explicitly) instead of treating every entry as a file.
<details>
<summary>🤖 Prompt for AI Agents</summary>
```
Verify each finding against the current code and only fix it if needed.
In
`@hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/validator/DefaultStashPartitionRenameHelper.java`
around lines 56 - 80, DefaultStashPartitionRenameHelper currently treats every
entry returned by listDirectEntries as a file (iterating sourceFiles and calling
FileIOUtils.copy and storage.deleteFile) which will fail for directory entries;
update the loop that processes sourceFiles to explicitly check StoragePathInfo
(or its Path) for being a directory and either recurse into sub-entries to
copy/delete recursively or skip/reject directories with a clear error; ensure
the same check is applied when building existingTargetFileNames
(getExistingFileNames) so directories are handled consistently, and preserve
existing logging (e.g., the "Copying file ..." and "Deleting source file ..."
messages) only for actual files.
```
</details>
<!-- fingerprinting:phantom:poseidon:hawk -->
<!-- This is an auto-generated comment by CodeRabbit -->
— *CodeRabbit* ([original](https://github.com/yihua/hudi/pull/51#discussion_r3105547965)) (source:comment#3105547965)
##########
hudi-utilities/src/main/java/org/apache/hudi/utilities/HoodieStashPartitionsTool.java:
##########
@@ -0,0 +1,386 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.hudi.utilities;
+
+import org.apache.hudi.client.HoodieWriteResult;
+import org.apache.hudi.client.SparkRDDWriteClient;
+import org.apache.hudi.client.validator.DefaultStashPartitionRenameHelper;
+import org.apache.hudi.client.validator.StashPartitionRenameHelper;
+import org.apache.hudi.client.validator.StashPartitionsPreCommitValidator;
+import org.apache.hudi.common.config.TypedProperties;
+import org.apache.hudi.common.model.HoodieRecordPayload;
+import org.apache.hudi.common.table.HoodieTableMetaClient;
+import org.apache.hudi.common.table.timeline.HoodieTimeline;
+import org.apache.hudi.common.util.Option;
+import org.apache.hudi.common.util.ReflectionUtils;
+import org.apache.hudi.common.util.StringUtils;
+import org.apache.hudi.config.HoodiePreCommitValidatorConfig;
+import org.apache.hudi.exception.HoodieException;
+import org.apache.hudi.hadoop.fs.HadoopFSUtils;
+import org.apache.hudi.storage.HoodieStorage;
+import org.apache.hudi.storage.HoodieStorageUtils;
+import org.apache.hudi.storage.StoragePath;
+import org.apache.hudi.storage.StoragePathInfo;
+
+import com.beust.jcommander.JCommander;
+import com.beust.jcommander.Parameter;
+import org.apache.hadoop.fs.Path;
+import org.apache.spark.api.java.JavaSparkContext;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.IOException;
+import java.io.Serializable;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+
+/**
+ * A spark-submit tool for stashing (backing up and removing) Hudi table partitions.
+ *
+ * <p>This tool supports three modes:
+ * <ul>
+ * <li><b>stash</b> — Moves data files from specified partitions to a stash location
+ * and issues a deletePartitions operation on the Hudi table. Uses a custom
+ * {@link StashPartitionsPreCommitValidator} to move files atomically before commit.</li>
+ * <li><b>rollback_stash</b> — Copies any data found in the stash location back to the
+ * original partition paths. Used when the user no longer wants to proceed with stashing.</li>
+ * <li><b>dry_run</b> — Shows which partitions and file groups would be affected.</li>
+ * </ul>
+ *
+ * <p>For restoring stashed partitions back into the Hudi table, use the standard
+ * {@code insert_overwrite} write operation reading from the stash location.
+ *
+ * <p>Example usage:
+ * <pre>
+ * spark-submit \
+ * --class org.apache.hudi.utilities.HoodieStashPartitionsTool \
+ * --master local[*] \
+ * --driver-memory 1g \
+ * --executor-memory 1g \
+ * hudi-utilities-bundle.jar \
+ * --base-path /path/to/hudi/table \
+ * --table-name myTable \
+ * --stash-path /path/to/stash/folder \
+ * --partitions datestr=2023-01-01,datestr=2023-01-02 \
+ * --mode stash
+ * </pre>
+ */
+public class HoodieStashPartitionsTool implements Serializable {
+
+ private static final long serialVersionUID = 1L;
+ private static final Logger LOG = LoggerFactory.getLogger(HoodieStashPartitionsTool.class);
+
+ private final transient JavaSparkContext jsc;
+ private final Config cfg;
+ private final TypedProperties props;
+ private final HoodieTableMetaClient metaClient;
+
+ public HoodieStashPartitionsTool(JavaSparkContext jsc, Config cfg) {
+ this.jsc = jsc;
+ this.cfg = cfg;
+ this.props = cfg.propsFilePath == null
+ ? UtilHelpers.buildProperties(cfg.configs)
+ : readConfigFromFileSystem(jsc, cfg);
+ this.metaClient = HoodieTableMetaClient.builder()
+ .setConf(HadoopFSUtils.getStorageConfWithCopy(jsc.hadoopConfiguration()))
+ .setBasePath(cfg.basePath)
+ .setLoadActiveTimelineOnLoad(true)
+ .build();
+ }
+
+ public enum Mode {
+ STASH,
+ ROLLBACK_STASH,
+ DRY_RUN
+ }
+
+ public static class Config implements Serializable {
+ @Parameter(names = {"--base-path", "-sp"}, description = "Base path for the Hudi table.", required = true)
+ public String basePath = null;
+
+ @Parameter(names = {"--table-name", "-tn"}, description = "Table name.", required = true)
+ public String tableName = null;
+
+ @Parameter(names = {"--stash-path"}, description = "Target path for stashing partition data.", required = true)
+ public String stashPath = null;
+
+ @Parameter(names = {"--partitions", "-p"}, description = "Comma separated list of partitions to stash.", required = true)
+ public String partitions = null;
+
+ @Parameter(names = {"--mode", "-m"}, description = "Set job mode: "
+ + "\"stash\" to move partition data to stash location and delete partitions; "
+ + "\"rollback_stash\" to copy stashed data back to table and abort stashing; "
+ + "\"dry_run\" to preview which partitions and files would be affected.", required = true)
+ public String runningMode = null;
+
+ @Parameter(names = {"--parallelism", "-pl"}, description = "Parallelism for hoodie operations.", required = false)
+ public int parallelism = 1500;
+
+ @Parameter(names = {"--rename-helper-class"}, description = "Fully qualified class name of a custom "
+ + "StashPartitionRenameHelper implementation. Defaults to DefaultStashPartitionRenameHelper.", required = false)
+ public String renameHelperClass = null;
+
+ @Parameter(names = {"--spark-master", "-ms"}, description = "Spark master.", required = false)
+ public String sparkMaster = null;
+
+ @Parameter(names = {"--spark-memory", "-sm"}, description = "Spark memory to use.", required = false)
+ public String sparkMemory = "1g";
+
+ @Parameter(names = {"--props"}, description = "Path to properties file on localfs or dfs, with configurations for "
+ + "hoodie client for stashing partitions.")
+ public String propsFilePath = null;
+
+ @Parameter(names = {"--hoodie-conf"}, description = "Any configuration that can be set in the properties file "
+ + "(using the CLI parameter \"--props\") can also be passed via command line using this parameter. This can be repeated.",
+ splitter = IdentitySplitter.class)
+ public List<String> configs = new ArrayList<>();
+
+ @Parameter(names = {"--help", "-h"}, help = true)
+ public Boolean help = false;
+
+ @Override
+ public String toString() {
+ return "HoodieStashPartitionsToolConfig {\n"
+ + " --base-path " + basePath + ", \n"
+ + " --table-name " + tableName + ", \n"
+ + " --stash-path " + stashPath + ", \n"
+ + " --partitions " + partitions + ", \n"
+ + " --mode " + runningMode + ", \n"
+ + " --parallelism " + parallelism + ", \n"
+ + " --rename-helper-class " + renameHelperClass + ", \n"
+ + " --spark-master " + sparkMaster + ", \n"
+ + " --spark-memory " + sparkMemory + ", \n"
+ + " --props " + propsFilePath + ", \n"
+ + " --hoodie-conf " + configs
+ + "\n}";
+ }
+ }
+
+ public static void main(String[] args) {
+ final Config cfg = new Config();
+ JCommander cmd = new JCommander(cfg, null, args);
+ if (cfg.help || args.length == 0) {
+ cmd.usage();
+ System.exit(1);
+ }
+ Map<String, String> sparkConfigMap = new HashMap<>();
+ sparkConfigMap.put("spark.executor.memory", cfg.sparkMemory);
+ JavaSparkContext jsc = UtilHelpers.buildSparkContext("Hoodie-Stash-Partitions",
+ cfg.sparkMaster, false, sparkConfigMap);
+
+ HoodieStashPartitionsTool tool = new HoodieStashPartitionsTool(jsc, cfg);
+ try {
+ tool.run();
+ } catch (Throwable throwable) {
+ LOG.error("Fail to run stash partitions for " + cfg, throwable);
+ } finally {
+ jsc.stop();
+ }
+ }
+
+ public void run() {
+ try {
+ LOG.info(cfg.toString());
+ Mode mode = Mode.valueOf(cfg.runningMode.toUpperCase());
+ switch (mode) {
+ case STASH:
+ LOG.info(" ****** The Hoodie Stash Partitions Tool is in stash mode ****** ");
+ doStash();
+ break;
+ case ROLLBACK_STASH:
+ LOG.info(" ****** The Hoodie Stash Partitions Tool is in rollback_stash mode ****** ");
+ doRollbackStash();
+ break;
+ case DRY_RUN:
+ LOG.info(" ****** The Hoodie Stash Partitions Tool is in dry-run mode ****** ");
+ doDryRun();
+ break;
+ default:
+ LOG.info("Unsupported running mode [" + cfg.runningMode + "], quit the job directly");
+ }
+ } catch (Exception e) {
+ throw new HoodieException("Unable to stash partitions in " + cfg.basePath, e);
+ }
+ }
+
+ /**
+ * Stash partitions: pre-check to recover any partial stash from a prior failed attempt,
+ * then issue deletePartitions with custom pre-commit validator. Hudi internally handles
+ * rollback of any prior failed instants. The pre-commit validator handles already-stashed
+ * partitions gracefully (source empty -> skip).
+ *
+ * <p>The pre-check runs regardless of whether MDT is enabled. Even with MDT, a prior failed
+ * attempt may have had its pre-commit validator move files to the stash location before
+ * crashing. Those files need to be moved back so that the subsequent deletePartitions
+ * sees the complete partition state.
+ */
+ private void doStash() {
+ List<String> partitionsToStash = Arrays.asList(cfg.partitions.split(","));
Review Comment:
_⚠️ Potential issue_ | _🟠 Major_
**Normalize and validate partition input before use.**
Raw `split(",")` allows empty/whitespace tokens. An empty token can resolve to an unintended path scope. Parse once (trim + filter empties) and fail fast if the result is empty.
<details>
<summary>Proposed fix</summary>
```diff
- List<String> partitionsToStash = Arrays.asList(cfg.partitions.split(","));
+ List<String> partitionsToStash = parsePartitions(cfg.partitions);
@@
- List<String> partitions = Arrays.asList(cfg.partitions.split(","));
+ List<String> partitions = parsePartitions(cfg.partitions);
@@
- List<String> parts = Arrays.asList(cfg.partitions.split(","));
+ List<String> parts = parsePartitions(cfg.partitions);
```
```diff
+ private List<String> parsePartitions(String rawPartitions) {
+ List<String> parsed = Arrays.stream(rawPartitions.split(","))
+ .map(String::trim)
+ .filter(s -> !s.isEmpty())
+ .collect(Collectors.toList());
+ if (parsed.isEmpty()) {
+ throw new HoodieException("No valid partitions were provided.");
+ }
+ return parsed;
+ }
```
</details>
Also applies to: 303-303, 336-336
<details>
<summary>🤖 Prompt for AI Agents</summary>
```
Verify each finding against the current code and only fix it if needed.
In
`@hudi-utilities/src/main/java/org/apache/hudi/utilities/HoodieStashPartitionsTool.java`
at line 239, Normalize and validate the cfg.partitions string before using it:
replace direct Arrays.asList(cfg.partitions.split(",")) with a single parsed
list that trims each token, filters out empty/whitespace tokens, and collects
the results into partitionsToStash (and the other occurrences referenced by
variable partitionsToStash); if the resulting list is empty, throw/exit fast
with a clear error explaining that no valid partitions were provided. Ensure
this logic is used wherever partitions are parsed (e.g., in
HoodieStashPartitionsTool methods that assign partitionsToStash).
```
</details>
<!-- fingerprinting:phantom:poseidon:hawk -->
<!-- This is an auto-generated comment by CodeRabbit -->
— *CodeRabbit* ([original](https://github.com/yihua/hudi/pull/51#discussion_r3105547979)) (source:comment#3105547979)
##########
hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/client/validator/StashPartitionsPreCommitValidator.java:
##########
@@ -0,0 +1,181 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.hudi.client.validator;
+
+import org.apache.hudi.client.WriteStatus;
+import org.apache.hudi.common.config.HoodieMetadataConfig;
+import org.apache.hudi.common.data.HoodieData;
+import org.apache.hudi.common.engine.HoodieEngineContext;
+import org.apache.hudi.common.util.ReflectionUtils;
+import org.apache.hudi.common.util.StringUtils;
+import org.apache.hudi.common.util.ValidationUtils;
+import org.apache.hudi.config.HoodieWriteConfig;
+import org.apache.hudi.exception.HoodieValidationException;
+import org.apache.hudi.storage.HoodieStorage;
+import org.apache.hudi.storage.StoragePath;
+import org.apache.hudi.storage.StoragePathInfo;
+import org.apache.hudi.table.HoodieSparkTable;
+import org.apache.hudi.table.action.HoodieWriteMetadata;
+
+import org.apache.spark.sql.Dataset;
+import org.apache.spark.sql.Row;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.IOException;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+
+/**
+ * A special-purpose pre-commit validator that moves partition files to a stash
+ * location before a deletePartitions commit completes.
+ *
+ * <p><strong>WARNING: This is NOT a standard validator pattern.</strong>
+ * Unlike typical pre-commit validators that are read-only checks, this validator
+ * performs destructive side-effects (moving/deleting files). It is designed exclusively
+ * for use with the {@code HoodieStashPartitionsTool} and the DELETE_PARTITION operation.
+ * Do NOT use this as a pattern for other validators.</p>
+ *
+ * <h3>Why move files inside a pre-commit validator?</h3>
+ *
+ * <p>The stash operation needs to both move data files to a backup location and mark
+ * the partitions as deleted via a replace commit. These two actions must be coupled:
+ * if the replace commit lands on the timeline but the file move has not happened yet,
+ * the Hudi cleaner will eventually delete the physical data files for the replaced
+ * file groups (based on its retention policy), permanently losing the data that was
+ * intended to be stashed.
+ *
+ * <p>By performing the file move inside this pre-commit validator, the move happens
+ * <b>before</b> the replace commit is finalized. If the move fails, the validator
+ * throws an exception, the commit does not land, and the table remains unchanged.
+ * This guarantees that data is never in a state where it has been marked as deleted
+ * but has not been safely backed up to the stash location.</p>
+ *
+ * <p>This validator requires the following configs to be set:
+ * <ul>
+ * <li>{@link #STASH_PATH_CONFIG} — the target stash directory</li>
+ * <li>{@link #RENAME_HELPER_CLASS_CONFIG} (optional) — custom rename helper class</li>
+ * </ul>
+ *
+ * <p>Guards:
+ * <ul>
+ * <li>Fails if RLI (Record Level Index) is enabled on the table</li>
+ * <li>Must only be used with DELETE_PARTITION operation</li>
+ * </ul>
+ */
+public class StashPartitionsPreCommitValidator<T, I, K, O extends HoodieData<WriteStatus>>
+ extends SparkPreCommitValidator<T, I, K, O> {
+
+ private static final Logger LOG = LoggerFactory.getLogger(StashPartitionsPreCommitValidator.class);
+
+ /**
+ * Config key for the stash target directory path.
+ */
+ public static final String STASH_PATH_CONFIG = "hoodie.stash.partitions.target.path";
+
+ /**
+ * Config key for the custom rename helper class. Defaults to {@link DefaultStashPartitionRenameHelper}.
+ */
+ public static final String RENAME_HELPER_CLASS_CONFIG = "hoodie.stash.rename.helper.class";
+
+ public StashPartitionsPreCommitValidator(HoodieSparkTable<T> table,
+ HoodieEngineContext engineContext,
+ HoodieWriteConfig writeConfig) {
+ super(table, engineContext, writeConfig);
+ }
+
+ /**
+ * Override validate to extract affected partitions from partitionToReplaceFileIds rather than
+ * write stats, since deletePartitions produces empty write stats but populates the replacement map.
+ */
+ @Override
+ public void validate(String instantTime, HoodieWriteMetadata<O> writeResult,
+ Dataset<Row> before, Dataset<Row> after) throws HoodieValidationException {
+ Set<String> partitionsAffected = new HashSet<>(writeResult.getPartitionToReplaceFileIds().keySet());
+ if (partitionsAffected.isEmpty()) {
+ // Fall back to write stats if partitionToReplaceFileIds is not populated
+ partitionsAffected = getPartitionsModified(writeResult);
+ }
+ validateRecordsBeforeAndAfter(before, after, partitionsAffected);
Review Comment:
_⚠️ Potential issue_ | _🔴 Critical_
**Do not fallback to write-stats partitions for this destructive validator.**
At Line 113, falling back to `getPartitionsModified(...)` allows this validator to move files for non-`DELETE_PARTITION` writes if configured incorrectly. This class should fail fast unless `partitionToReplaceFileIds` is populated.
<details>
<summary>Proposed fix</summary>
```diff
- Set<String> partitionsAffected = new HashSet<>(writeResult.getPartitionToReplaceFileIds().keySet());
- if (partitionsAffected.isEmpty()) {
- // Fall back to write stats if partitionToReplaceFileIds is not populated
- partitionsAffected = getPartitionsModified(writeResult);
- }
+ Set<String> partitionsAffected = new HashSet<>(writeResult.getPartitionToReplaceFileIds().keySet());
+ ValidationUtils.checkState(!partitionsAffected.isEmpty(),
+ "StashPartitionsPreCommitValidator must be used only with DELETE_PARTITION replace commits.");
validateRecordsBeforeAndAfter(before, after, partitionsAffected);
```
</details>
<!-- suggestion_start -->
<details>
<summary>📝 Committable suggestion</summary>
> ‼️ **IMPORTANT**
> Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
```suggestion
Set<String> partitionsAffected = new HashSet<>(writeResult.getPartitionToReplaceFileIds().keySet());
ValidationUtils.checkState(!partitionsAffected.isEmpty(),
"StashPartitionsPreCommitValidator must be used only with DELETE_PARTITION replace commits.");
validateRecordsBeforeAndAfter(before, after, partitionsAffected);
```
</details>
<!-- suggestion_end -->
<details>
<summary>🤖 Prompt for AI Agents</summary>
```
Verify each finding against the current code and only fix it if needed.
In
`@hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/client/validator/StashPartitionsPreCommitValidator.java`
around lines 112 - 117, The validator must not fall back to write-stats; in
StashPartitionsPreCommitValidator replace the current fallback logic that sets
partitionsAffected from getPartitionsModified(writeResult) with a fail-fast
behavior: when writeResult.getPartitionToReplaceFileIds() is empty, throw a
clear IllegalStateException (or HoodieCommitException) indicating
partitionToReplaceFileIds is required for this destructive validator, otherwise
proceed to call validateRecordsBeforeAndAfter(before, after, partitionsAffected)
using the set from writeResult.getPartitionToReplaceFileIds().keySet().
```
</details>
<!-- fingerprinting:phantom:poseidon:hawk -->
<!-- This is an auto-generated comment by CodeRabbit -->
— *CodeRabbit* ([original](https://github.com/yihua/hudi/pull/51#discussion_r3105547971)) (source:comment#3105547971)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-4273311189
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `78.47534%` with `48 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 68.89%. Comparing base ([`7bcb8be`](https://app.codecov.io/gh/apache/hudi/commit/7bcb8be7c153010998574f591c7ff3799192eda6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`e8b5a0a`](https://app.codecov.io/gh/apache/hudi/commit/e8b5a0a62cea99be0a184e0b6f11149b49850806?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 3 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | 76.92% | [28 Missing and 8 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | 75.00% | [6 Missing and 4 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | 92.59% | [1 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18526 +/- ##
============================================
+ Coverage 68.88% 68.89% +0.01%
- Complexity 28279 28317 +38
============================================
Files 2464 2467 +3
Lines 135568 135817 +249
Branches 16446 16468 +22
============================================
+ Hits 93385 93574 +189
- Misses 34801 34846 +45
- Partials 7382 7397 +15
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.59% <11.21%> (-0.05%)` | :arrow_down: |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.70% <0.00%> (-0.06%)` | :arrow_down: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.46% <77.61%> (+0.06%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.83% <0.00%> (-0.10%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.35% <0.00%> (-0.09%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `38.31% <74.88%> (+0.14%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | `92.59% <92.59%> (ø)` | |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | `75.00% <75.00%> (ø)` | |
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | `76.92% <76.92%> (ø)` | |
... and [10 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18526/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-4284460077
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `23.31839%` with `171 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 47.96%. Comparing base ([`7bcb8be`](https://app.codecov.io/gh/apache/hudi/commit/7bcb8be7c153010998574f591c7ff3799192eda6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`891eaf2`](https://app.codecov.io/gh/apache/hudi/commit/891eaf2e1427bf93b9c7ff9626628c30e1c9c01d?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 11 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | 0.00% | [156 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | 75.00% | [6 Missing and 4 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | 81.48% | [3 Missing and 2 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
> :exclamation: There is a different number of reports uploaded between BASE (7bcb8be) and HEAD (891eaf2). Click for more details.
>
> <details><summary>HEAD has 19 uploads less than BASE</summary>
>
>| Flag | BASE (7bcb8be) | HEAD (891eaf2) |
>|------|------|------|
>|spark-scala-tests|10|5|
>|spark-java-tests|15|3|
>|common-and-other-modules|1|0|
>|utilities|1|0|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18526 +/- ##
=============================================
- Coverage 68.88% 47.96% -20.92%
+ Complexity 28279 16515 -11764
=============================================
Files 2464 2011 -453
Lines 135568 112821 -22747
Branches 16446 14328 -2118
=============================================
- Hits 93385 54115 -39270
- Misses 34801 53036 +18235
+ Partials 7382 5670 -1712
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.75% <0.00%> (+<0.01%)` | :arrow_up: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.47% <77.61%> (+0.07%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `33.11% <0.00%> (-15.82%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `32.48% <0.00%> (-12.96%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | `81.48% <81.48%> (ø)` | |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | `75.00% <75.00%> (ø)` | |
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | `0.00% <0.00%> (ø)` | |
... and [1534 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18526/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "nsivabalan (via GitHub)" <gi...@apache.org>.
nsivabalan commented on code in PR #18526:
URL: https://github.com/apache/hudi/pull/18526#discussion_r3126896750
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/validator/DefaultStashPartitionRenameHelper.java:
##########
@@ -0,0 +1,92 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.hudi.client.validator;
+
+import org.apache.hudi.io.util.FileIOUtils;
+import org.apache.hudi.storage.HoodieStorage;
+import org.apache.hudi.storage.StoragePath;
+import org.apache.hudi.storage.StoragePathInfo;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.IOException;
+import java.util.List;
+import java.util.Set;
+import java.util.stream.Collectors;
+
+/**
+ * Default implementation of {@link StashPartitionRenameHelper} that copies files
+ * individually from source to target and then deletes them from source.
+ * <p>
+ * This implementation is correct but may be slow for large partitions on cloud
+ * storage. For production use on HDFS or cloud storage with efficient rename APIs,
+ * a custom implementation can be plugged in via the
+ * {@code hoodie.stash.rename.helper.class} configuration.
+ */
+public class DefaultStashPartitionRenameHelper implements StashPartitionRenameHelper {
+
+ private static final long serialVersionUID = 1L;
+ private static final Logger LOG = LoggerFactory.getLogger(DefaultStashPartitionRenameHelper.class);
+
+ @Override
+ public void movePartitionFiles(HoodieStorage storage, StoragePath sourcePath, StoragePath targetPath) throws IOException {
+ // Ensure target directory exists
+ if (!storage.exists(targetPath)) {
+ storage.createDirectory(targetPath);
+ }
+
+ List<StoragePathInfo> sourceFiles = storage.listDirectEntries(sourcePath);
+ if (sourceFiles.isEmpty()) {
+ LOG.info("Source path {} has no files, nothing to move.", sourcePath);
+ return;
+ }
+
+ // Determine which files already exist in target (from a prior partial attempt)
+ Set<String> existingTargetFileNames = getExistingFileNames(storage, targetPath);
+
+ // Copy files not already in target
+ for (StoragePathInfo sourceFileInfo : sourceFiles) {
+ String fileName = sourceFileInfo.getPath().getName();
+ if (existingTargetFileNames.contains(fileName)) {
+ LOG.info("File {} already exists in target {}, skipping copy.", fileName, targetPath);
+ } else {
+ StoragePath destFilePath = new StoragePath(targetPath, fileName);
+ LOG.info("Copying file {} to {}", sourceFileInfo.getPath(), destFilePath);
+ FileIOUtils.copy(storage, sourceFileInfo.getPath(), destFilePath);
+ }
+ }
Review Comment:
preCheckAndRecoverPartialStash takes care of copying back any partially copied/moved to stash location back to the source location
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-4300625193
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `16.46707%` with `279 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 47.87%. Comparing base ([`7bcb8be`](https://app.codecov.io/gh/apache/hudi/commit/7bcb8be7c153010998574f591c7ff3799192eda6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`6d8dedd`](https://app.codecov.io/gh/apache/hudi/commit/6d8dedd061ed183c52471df09355b85e5a60099a?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 21 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | 0.00% | [263 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | 77.50% | [5 Missing and 4 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | 77.41% | [5 Missing and 2 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
> :exclamation: There is a different number of reports uploaded between BASE (7bcb8be) and HEAD (6d8dedd). Click for more details.
>
> <details><summary>HEAD has 19 uploads less than BASE</summary>
>
>| Flag | BASE (7bcb8be) | HEAD (6d8dedd) |
>|------|------|------|
>|spark-scala-tests|10|5|
>|spark-java-tests|15|3|
>|common-and-other-modules|1|0|
>|utilities|1|0|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18526 +/- ##
=============================================
- Coverage 68.88% 47.87% -21.01%
+ Complexity 28279 16593 -11686
=============================================
Files 2464 2018 -446
Lines 135568 113561 -22007
Branches 16446 14458 -1988
=============================================
- Hits 93385 54371 -39014
- Misses 34801 53487 +18686
+ Partials 7382 5703 -1679
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.74% <0.00%> (-0.02%)` | :arrow_down: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.64% <77.46%> (+0.23%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `32.51% <0.00%> (-16.42%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `32.36% <0.00%> (-13.09%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | `77.41% <77.41%> (ø)` | |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | `77.50% <77.50%> (ø)` | |
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | `0.00% <0.00%> (ø)` | |
... and [1545 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18526/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-4272495165
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `23.31839%` with `171 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 48.00%. Comparing base ([`7bcb8be`](https://app.codecov.io/gh/apache/hudi/commit/7bcb8be7c153010998574f591c7ff3799192eda6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`e8b5a0a`](https://app.codecov.io/gh/apache/hudi/commit/e8b5a0a62cea99be0a184e0b6f11149b49850806?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 3 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | 0.00% | [156 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | 75.00% | [6 Missing and 4 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | 81.48% | [3 Missing and 2 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
> :exclamation: There is a different number of reports uploaded between BASE (7bcb8be) and HEAD (e8b5a0a). Click for more details.
>
> <details><summary>HEAD has 19 uploads less than BASE</summary>
>
>| Flag | BASE (7bcb8be) | HEAD (e8b5a0a) |
>|------|------|------|
>|spark-scala-tests|10|5|
>|spark-java-tests|15|3|
>|common-and-other-modules|1|0|
>|utilities|1|0|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18526 +/- ##
=============================================
- Coverage 68.88% 48.00% -20.89%
+ Complexity 28279 16524 -11755
=============================================
Files 2464 2011 -453
Lines 135568 112793 -22775
Branches 16446 14326 -2120
=============================================
- Hits 93385 54143 -39242
- Misses 34801 52981 +18180
+ Partials 7382 5669 -1713
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.70% <0.00%> (-0.06%)` | :arrow_down: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.46% <77.61%> (+0.06%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `32.69% <0.00%> (-16.23%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `32.48% <0.00%> (-12.96%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | `81.48% <81.48%> (ø)` | |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | `75.00% <75.00%> (ø)` | |
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | `0.00% <0.00%> (ø)` | |
... and [1532 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18526/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-4272489687
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `23.31839%` with `171 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 45.62%. Comparing base ([`7bcb8be`](https://app.codecov.io/gh/apache/hudi/commit/7bcb8be7c153010998574f591c7ff3799192eda6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`e8b5a0a`](https://app.codecov.io/gh/apache/hudi/commit/e8b5a0a62cea99be0a184e0b6f11149b49850806?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 3 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | 0.00% | [156 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | 75.00% | [6 Missing and 4 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | 81.48% | [3 Missing and 2 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
> :exclamation: There is a different number of reports uploaded between BASE (7bcb8be) and HEAD (e8b5a0a). Click for more details.
>
> <details><summary>HEAD has 24 uploads less than BASE</summary>
>
>| Flag | BASE (7bcb8be) | HEAD (e8b5a0a) |
>|------|------|------|
>|spark-scala-tests|10|3|
>|spark-java-tests|15|0|
>|common-and-other-modules|1|0|
>|utilities|1|0|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18526 +/- ##
=============================================
- Coverage 68.88% 45.62% -23.26%
+ Complexity 28279 15375 -12904
=============================================
Files 2464 1982 -482
Lines 135568 110185 -25383
Branches 16446 13766 -2680
=============================================
- Hits 93385 50274 -43111
- Misses 34801 54606 +19805
+ Partials 7382 5305 -2077
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.70% <0.00%> (-0.06%)` | :arrow_down: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.46% <77.61%> (+0.06%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `32.61% <0.00%> (-12.84%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | `81.48% <81.48%> (ø)` | |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | `75.00% <75.00%> (ø)` | |
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | `0.00% <0.00%> (ø)` | |
... and [1600 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18526/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-4272518556
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `78.47534%` with `48 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 66.28%. Comparing base ([`7bcb8be`](https://app.codecov.io/gh/apache/hudi/commit/7bcb8be7c153010998574f591c7ff3799192eda6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`e8b5a0a`](https://app.codecov.io/gh/apache/hudi/commit/e8b5a0a62cea99be0a184e0b6f11149b49850806?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 3 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | 76.92% | [28 Missing and 8 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | 75.00% | [6 Missing and 4 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | 92.59% | [1 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18526 +/- ##
============================================
- Coverage 68.88% 66.28% -2.61%
+ Complexity 28279 27209 -1070
============================================
Files 2464 2467 +3
Lines 135568 135817 +249
Branches 16446 16468 +22
============================================
- Hits 93385 90022 -3363
- Misses 34801 38260 +3459
- Partials 7382 7535 +153
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.59% <11.21%> (-0.05%)` | :arrow_down: |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.70% <0.00%> (-0.06%)` | :arrow_down: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.46% <77.61%> (+0.06%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `32.29% <0.00%> (-16.64%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.20% <0.00%> (-0.25%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `38.31% <74.88%> (+0.14%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | `92.59% <92.59%> (ø)` | |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | `75.00% <75.00%> (ø)` | |
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | `76.92% <76.92%> (ø)` | |
... and [341 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18526/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-4272559491
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "67f1e57f0dffe02bbe2d7bacfaa8f1aaa4be8c88",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=13351",
"triggerID" : "67f1e57f0dffe02bbe2d7bacfaa8f1aaa4be8c88",
"triggerType" : "PUSH"
}, {
"hash" : "e8b5a0a62cea99be0a184e0b6f11149b49850806",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=13354",
"triggerID" : "e8b5a0a62cea99be0a184e0b6f11149b49850806",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 67f1e57f0dffe02bbe2d7bacfaa8f1aaa4be8c88 Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=13351)
* e8b5a0a62cea99be0a184e0b6f11149b49850806 Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=13354)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-5211669469
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `80.83832%` with `64 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 68.68%. Comparing base ([`7bcb8be`](https://app.codecov.io/gh/apache/hudi/commit/7bcb8be7c153010998574f591c7ff3799192eda6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`6d8dedd`](https://app.codecov.io/gh/apache/hudi/commit/6d8dedd061ed183c52471df09355b85e5a60099a?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 471 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | 79.84% | [43 Missing and 10 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | 77.50% | [5 Missing and 4 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | 93.54% | [1 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18526 +/- ##
============================================
- Coverage 68.88% 68.68% -0.21%
- Complexity 28279 28473 +194
============================================
Files 2464 2478 +14
Lines 135568 136972 +1404
Branches 16446 16653 +207
============================================
+ Hits 93385 94077 +692
- Misses 34801 35324 +523
- Partials 7382 7571 +189
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.39% <13.77%> (-0.25%)` | :arrow_down: |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.74% <0.00%> (-0.02%)` | :arrow_down: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.64% <77.46%> (+0.23%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `49.25% <0.00%> (+0.32%)` | :arrow_up: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.18% <0.00%> (-0.26%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `37.87% <77.54%> (-0.30%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | `93.54% <93.54%> (ø)` | |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | `77.50% <77.50%> (ø)` | |
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | `79.84% <79.84%> (ø)` | |
... and [79 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18526/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
- :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "codecov-commenter (via GitHub) via commits" <co...@hudi.apache.org>.
codecov-commenter commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-6543684653
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `80.83832%` with `64 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 68.68%. Comparing base ([`7bcb8be`](https://app.codecov.io/gh/apache/hudi/commit/7bcb8be7c153010998574f591c7ff3799192eda6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`6d8dedd`](https://app.codecov.io/gh/apache/hudi/commit/6d8dedd061ed183c52471df09355b85e5a60099a?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 820 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | 79.84% | [43 Missing and 10 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | 77.50% | [5 Missing and 4 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | 93.54% | [1 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18526 +/- ##
============================================
- Coverage 68.88% 68.68% -0.21%
- Complexity 28279 28473 +194
============================================
Files 2464 2478 +14
Lines 135568 136972 +1404
Branches 16446 16653 +207
============================================
+ Hits 93385 94077 +692
- Misses 34801 35324 +523
- Partials 7382 7571 +189
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.39% <13.77%> (-0.25%)` | :arrow_down: |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.74% <0.00%> (-0.02%)` | :arrow_down: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.64% <77.46%> (+0.23%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `49.25% <0.00%> (+0.32%)` | :arrow_up: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.18% <0.00%> (-0.26%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `37.87% <77.54%> (-0.30%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | `93.54% <93.54%> (ø)` | |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | `77.50% <77.50%> (ø)` | |
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | `79.84% <79.84%> (ø)` | |
... and [79 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18526/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
- :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "codecov-commenter (via GitHub) via commits" <co...@hudi.apache.org>.
codecov-commenter commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-6536895936
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `80.83832%` with `64 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 68.68%. Comparing base ([`7bcb8be`](https://app.codecov.io/gh/apache/hudi/commit/7bcb8be7c153010998574f591c7ff3799192eda6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`6d8dedd`](https://app.codecov.io/gh/apache/hudi/commit/6d8dedd061ed183c52471df09355b85e5a60099a?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 819 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | 79.84% | [43 Missing and 10 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | 77.50% | [5 Missing and 4 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | 93.54% | [1 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18526 +/- ##
============================================
- Coverage 68.88% 68.68% -0.21%
- Complexity 28279 28473 +194
============================================
Files 2464 2478 +14
Lines 135568 136972 +1404
Branches 16446 16653 +207
============================================
+ Hits 93385 94077 +692
- Misses 34801 35324 +523
- Partials 7382 7571 +189
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.39% <13.77%> (-0.25%)` | :arrow_down: |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.74% <0.00%> (-0.02%)` | :arrow_down: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.64% <77.46%> (+0.23%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `49.25% <0.00%> (+0.32%)` | :arrow_up: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.18% <0.00%> (-0.26%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `37.87% <77.54%> (-0.30%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | `93.54% <93.54%> (ø)` | |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | `77.50% <77.50%> (ø)` | |
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | `79.84% <79.84%> (ø)` | |
... and [79 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18526/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
- :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-6530290545
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `80.83832%` with `64 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 68.68%. Comparing base ([`7bcb8be`](https://app.codecov.io/gh/apache/hudi/commit/7bcb8be7c153010998574f591c7ff3799192eda6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`6d8dedd`](https://app.codecov.io/gh/apache/hudi/commit/6d8dedd061ed183c52471df09355b85e5a60099a?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 819 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | 79.84% | [43 Missing and 10 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | 77.50% | [5 Missing and 4 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | 93.54% | [1 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18526 +/- ##
============================================
- Coverage 68.88% 68.68% -0.21%
- Complexity 28279 28473 +194
============================================
Files 2464 2478 +14
Lines 135568 136972 +1404
Branches 16446 16653 +207
============================================
+ Hits 93385 94077 +692
- Misses 34801 35324 +523
- Partials 7382 7571 +189
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.39% <13.77%> (-0.25%)` | :arrow_down: |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.74% <0.00%> (-0.02%)` | :arrow_down: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.64% <77.46%> (+0.23%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `49.25% <0.00%> (+0.32%)` | :arrow_up: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.18% <0.00%> (-0.26%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `37.87% <77.54%> (-0.30%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | `93.54% <93.54%> (ø)` | |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | `77.50% <77.50%> (ø)` | |
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | `79.84% <79.84%> (ø)` | |
... and [79 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18526/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
- :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-6531620999
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `80.83832%` with `64 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 68.68%. Comparing base ([`7bcb8be`](https://app.codecov.io/gh/apache/hudi/commit/7bcb8be7c153010998574f591c7ff3799192eda6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`6d8dedd`](https://app.codecov.io/gh/apache/hudi/commit/6d8dedd061ed183c52471df09355b85e5a60099a?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 819 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | 79.84% | [43 Missing and 10 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | 77.50% | [5 Missing and 4 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | 93.54% | [1 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18526 +/- ##
============================================
- Coverage 68.88% 68.68% -0.21%
- Complexity 28279 28473 +194
============================================
Files 2464 2478 +14
Lines 135568 136972 +1404
Branches 16446 16653 +207
============================================
+ Hits 93385 94077 +692
- Misses 34801 35324 +523
- Partials 7382 7571 +189
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.39% <13.77%> (-0.25%)` | :arrow_down: |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.74% <0.00%> (-0.02%)` | :arrow_down: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.64% <77.46%> (+0.23%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `49.25% <0.00%> (+0.32%)` | :arrow_up: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.18% <0.00%> (-0.26%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `37.87% <77.54%> (-0.30%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | `93.54% <93.54%> (ø)` | |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | `77.50% <77.50%> (ø)` | |
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | `79.84% <79.84%> (ø)` | |
... and [79 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18526/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
- :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-4272489050
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `23.31839%` with `171 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 46.17%. Comparing base ([`7bcb8be`](https://app.codecov.io/gh/apache/hudi/commit/7bcb8be7c153010998574f591c7ff3799192eda6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`e8b5a0a`](https://app.codecov.io/gh/apache/hudi/commit/e8b5a0a62cea99be0a184e0b6f11149b49850806?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 3 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | 0.00% | [156 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | 75.00% | [6 Missing and 4 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | 81.48% | [3 Missing and 2 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
> :exclamation: There is a different number of reports uploaded between BASE (7bcb8be) and HEAD (e8b5a0a). Click for more details.
>
> <details><summary>HEAD has 25 uploads less than BASE</summary>
>
>| Flag | BASE (7bcb8be) | HEAD (e8b5a0a) |
>|------|------|------|
>|spark-scala-tests|10|2|
>|spark-java-tests|15|0|
>|common-and-other-modules|1|0|
>|utilities|1|0|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18526 +/- ##
=============================================
- Coverage 68.88% 46.17% -22.72%
+ Complexity 28279 15214 -13065
=============================================
Files 2464 1965 -499
Lines 135568 107840 -27728
Branches 16446 13264 -3182
=============================================
- Hits 93385 49794 -43591
- Misses 34801 52881 +18080
+ Partials 7382 5165 -2217
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.70% <0.00%> (-0.06%)` | :arrow_down: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.46% <77.61%> (+0.06%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `32.87% <0.00%> (-12.57%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | `81.48% <81.48%> (ø)` | |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | `75.00% <75.00%> (ø)` | |
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | `0.00% <0.00%> (ø)` | |
... and [1614 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18526/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-4272523331
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `78.47534%` with `48 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 66.38%. Comparing base ([`7bcb8be`](https://app.codecov.io/gh/apache/hudi/commit/7bcb8be7c153010998574f591c7ff3799192eda6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`e8b5a0a`](https://app.codecov.io/gh/apache/hudi/commit/e8b5a0a62cea99be0a184e0b6f11149b49850806?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 3 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | 76.92% | [28 Missing and 8 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | 75.00% | [6 Missing and 4 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | 92.59% | [1 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18526 +/- ##
============================================
- Coverage 68.88% 66.38% -2.51%
+ Complexity 28279 27274 -1005
============================================
Files 2464 2467 +3
Lines 135568 135817 +249
Branches 16446 16468 +22
============================================
- Hits 93385 90159 -3226
- Misses 34801 38079 +3278
- Partials 7382 7579 +197
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.59% <11.21%> (-0.05%)` | :arrow_down: |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.70% <0.00%> (-0.06%)` | :arrow_down: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.46% <77.61%> (+0.06%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `32.29% <0.00%> (-16.64%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.35% <0.00%> (-0.09%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `38.31% <74.88%> (+0.14%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | `92.59% <92.59%> (ø)` | |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | `75.00% <75.00%> (ø)` | |
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | `76.92% <76.92%> (ø)` | |
... and [334 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18526/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-4272516106
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `78.47534%` with `48 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 66.14%. Comparing base ([`7bcb8be`](https://app.codecov.io/gh/apache/hudi/commit/7bcb8be7c153010998574f591c7ff3799192eda6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`e8b5a0a`](https://app.codecov.io/gh/apache/hudi/commit/e8b5a0a62cea99be0a184e0b6f11149b49850806?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 3 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | 76.92% | [28 Missing and 8 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | 75.00% | [6 Missing and 4 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | 92.59% | [1 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18526 +/- ##
============================================
- Coverage 68.88% 66.14% -2.75%
+ Complexity 28279 27145 -1134
============================================
Files 2464 2467 +3
Lines 135568 135817 +249
Branches 16446 16468 +22
============================================
- Hits 93385 89832 -3553
- Misses 34801 38472 +3671
- Partials 7382 7513 +131
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.59% <11.21%> (-0.05%)` | :arrow_down: |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.70% <0.00%> (-0.06%)` | :arrow_down: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.46% <77.61%> (+0.06%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `32.29% <0.00%> (-16.64%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.99% <0.00%> (-0.45%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `38.31% <74.88%> (+0.14%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | `92.59% <92.59%> (ø)` | |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | `75.00% <75.00%> (ø)` | |
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | `76.92% <76.92%> (ø)` | |
... and [363 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18526/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-4272530662
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `78.47534%` with `48 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 68.89%. Comparing base ([`7bcb8be`](https://app.codecov.io/gh/apache/hudi/commit/7bcb8be7c153010998574f591c7ff3799192eda6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`e8b5a0a`](https://app.codecov.io/gh/apache/hudi/commit/e8b5a0a62cea99be0a184e0b6f11149b49850806?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 3 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | 76.92% | [28 Missing and 8 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | 75.00% | [6 Missing and 4 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | 92.59% | [1 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18526 +/- ##
==========================================
Coverage 68.88% 68.89%
- Complexity 28279 28316 +37
==========================================
Files 2464 2467 +3
Lines 135568 135817 +249
Branches 16446 16468 +22
==========================================
+ Hits 93385 93566 +181
- Misses 34801 34852 +51
- Partials 7382 7399 +17
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.59% <11.21%> (-0.05%)` | :arrow_down: |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.70% <0.00%> (-0.06%)` | :arrow_down: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.46% <77.61%> (+0.06%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.72% <0.00%> (-0.21%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.35% <0.00%> (-0.09%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `38.31% <74.88%> (+0.14%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | `92.59% <92.59%> (ø)` | |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | `75.00% <75.00%> (ø)` | |
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | `76.92% <76.92%> (ø)` | |
... and [15 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18526/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-4272528033
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "67f1e57f0dffe02bbe2d7bacfaa8f1aaa4be8c88",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=13351",
"triggerID" : "67f1e57f0dffe02bbe2d7bacfaa8f1aaa4be8c88",
"triggerType" : "PUSH"
}, {
"hash" : "e8b5a0a62cea99be0a184e0b6f11149b49850806",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=13354",
"triggerID" : "e8b5a0a62cea99be0a184e0b6f11149b49850806",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 67f1e57f0dffe02bbe2d7bacfaa8f1aaa4be8c88 Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=13351)
* e8b5a0a62cea99be0a184e0b6f11149b49850806 Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=13354)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-4272559421
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `78.47534%` with `48 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 68.89%. Comparing base ([`7bcb8be`](https://app.codecov.io/gh/apache/hudi/commit/7bcb8be7c153010998574f591c7ff3799192eda6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`e8b5a0a`](https://app.codecov.io/gh/apache/hudi/commit/e8b5a0a62cea99be0a184e0b6f11149b49850806?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 3 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | 76.92% | [28 Missing and 8 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | 75.00% | [6 Missing and 4 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | 92.59% | [1 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18526 +/- ##
============================================
+ Coverage 68.88% 68.89% +0.01%
- Complexity 28279 28317 +38
============================================
Files 2464 2467 +3
Lines 135568 135817 +249
Branches 16446 16468 +22
============================================
+ Hits 93385 93572 +187
- Misses 34801 34848 +47
- Partials 7382 7397 +15
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.59% <11.21%> (-0.05%)` | :arrow_down: |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.70% <0.00%> (-0.06%)` | :arrow_down: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.46% <77.61%> (+0.06%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.82% <0.00%> (-0.10%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.35% <0.00%> (-0.09%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `38.31% <74.88%> (+0.14%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | `92.59% <92.59%> (ø)` | |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | `75.00% <75.00%> (ø)` | |
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | `76.92% <76.92%> (ø)` | |
... and [11 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18526/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-5190889860
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `80.83832%` with `64 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 68.68%. Comparing base ([`7bcb8be`](https://app.codecov.io/gh/apache/hudi/commit/7bcb8be7c153010998574f591c7ff3799192eda6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`6d8dedd`](https://app.codecov.io/gh/apache/hudi/commit/6d8dedd061ed183c52471df09355b85e5a60099a?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 461 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | 79.84% | [43 Missing and 10 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | 77.50% | [5 Missing and 4 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | 93.54% | [1 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18526 +/- ##
============================================
- Coverage 68.88% 68.68% -0.21%
- Complexity 28279 28473 +194
============================================
Files 2464 2478 +14
Lines 135568 136972 +1404
Branches 16446 16653 +207
============================================
+ Hits 93385 94077 +692
- Misses 34801 35324 +523
- Partials 7382 7571 +189
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.39% <13.77%> (-0.25%)` | :arrow_down: |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.74% <0.00%> (-0.02%)` | :arrow_down: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.64% <77.46%> (+0.23%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `49.25% <0.00%> (+0.32%)` | :arrow_up: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.18% <0.00%> (-0.26%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `37.87% <77.54%> (-0.30%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | `93.54% <93.54%> (ø)` | |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | `77.50% <77.50%> (ø)` | |
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | `79.84% <79.84%> (ø)` | |
... and [79 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18526/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
- :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-4284240819
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "67f1e57f0dffe02bbe2d7bacfaa8f1aaa4be8c88",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=13351",
"triggerID" : "67f1e57f0dffe02bbe2d7bacfaa8f1aaa4be8c88",
"triggerType" : "PUSH"
}, {
"hash" : "e8b5a0a62cea99be0a184e0b6f11149b49850806",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=13354",
"triggerID" : "e8b5a0a62cea99be0a184e0b6f11149b49850806",
"triggerType" : "PUSH"
}, {
"hash" : "891eaf2e1427bf93b9c7ff9626628c30e1c9c01d",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "891eaf2e1427bf93b9c7ff9626628c30e1c9c01d",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* e8b5a0a62cea99be0a184e0b6f11149b49850806 Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=13354)
* 891eaf2e1427bf93b9c7ff9626628c30e1c9c01d UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-4284511010
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `78.47534%` with `48 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 67.78%. Comparing base ([`7bcb8be`](https://app.codecov.io/gh/apache/hudi/commit/7bcb8be7c153010998574f591c7ff3799192eda6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`891eaf2`](https://app.codecov.io/gh/apache/hudi/commit/891eaf2e1427bf93b9c7ff9626628c30e1c9c01d?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 11 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | 76.92% | [28 Missing and 8 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | 75.00% | [6 Missing and 4 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | 92.59% | [1 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18526 +/- ##
============================================
- Coverage 68.88% 67.78% -1.10%
+ Complexity 28279 27942 -337
============================================
Files 2464 2471 +7
Lines 135568 136218 +650
Branches 16446 16521 +75
============================================
- Hits 93385 92341 -1044
- Misses 34801 36369 +1568
- Partials 7382 7508 +126
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.56% <11.21%> (-0.08%)` | :arrow_down: |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.75% <0.00%> (+<0.01%)` | :arrow_up: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.47% <77.61%> (+0.07%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `42.32% <0.00%> (-6.60%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.34% <0.00%> (-0.10%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `38.30% <74.88%> (+0.13%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | `92.59% <92.59%> (ø)` | |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | `75.00% <75.00%> (ø)` | |
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | `76.92% <76.92%> (ø)` | |
... and [201 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18526/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-4284606290
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `78.47534%` with `48 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 68.79%. Comparing base ([`7bcb8be`](https://app.codecov.io/gh/apache/hudi/commit/7bcb8be7c153010998574f591c7ff3799192eda6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`891eaf2`](https://app.codecov.io/gh/apache/hudi/commit/891eaf2e1427bf93b9c7ff9626628c30e1c9c01d?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 11 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | 76.92% | [28 Missing and 8 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | 75.00% | [6 Missing and 4 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | 92.59% | [1 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18526 +/- ##
============================================
- Coverage 68.88% 68.79% -0.10%
- Complexity 28279 28361 +82
============================================
Files 2464 2471 +7
Lines 135568 136218 +650
Branches 16446 16521 +75
============================================
+ Hits 93385 93707 +322
- Misses 34801 35095 +294
- Partials 7382 7416 +34
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.56% <11.21%> (-0.08%)` | :arrow_down: |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.75% <0.00%> (+<0.01%)` | :arrow_up: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.47% <77.61%> (+0.07%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.83% <0.00%> (-0.10%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.34% <0.00%> (-0.10%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `38.30% <74.88%> (+0.13%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | `92.59% <92.59%> (ø)` | |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | `75.00% <75.00%> (ø)` | |
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | `76.92% <76.92%> (ø)` | |
... and [25 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18526/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "kbuci (via GitHub)" <gi...@apache.org>.
kbuci commented on code in PR #18526:
URL: https://github.com/apache/hudi/pull/18526#discussion_r3120937105
##########
hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/client/validator/StashPartitionsPreCommitValidator.java:
##########
@@ -0,0 +1,181 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.hudi.client.validator;
+
+import org.apache.hudi.client.WriteStatus;
+import org.apache.hudi.common.config.HoodieMetadataConfig;
+import org.apache.hudi.common.data.HoodieData;
+import org.apache.hudi.common.engine.HoodieEngineContext;
+import org.apache.hudi.common.util.ReflectionUtils;
+import org.apache.hudi.common.util.StringUtils;
+import org.apache.hudi.common.util.ValidationUtils;
+import org.apache.hudi.config.HoodieWriteConfig;
+import org.apache.hudi.exception.HoodieValidationException;
+import org.apache.hudi.storage.HoodieStorage;
+import org.apache.hudi.storage.StoragePath;
+import org.apache.hudi.storage.StoragePathInfo;
+import org.apache.hudi.table.HoodieSparkTable;
+import org.apache.hudi.table.action.HoodieWriteMetadata;
+
+import org.apache.spark.sql.Dataset;
+import org.apache.spark.sql.Row;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.IOException;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+
+/**
+ * A special-purpose pre-commit validator that moves partition files to a stash
+ * location before a deletePartitions commit completes.
+ *
+ * <p><strong>WARNING: This is NOT a standard validator pattern.</strong>
+ * Unlike typical pre-commit validators that are read-only checks, this validator
+ * performs destructive side-effects (moving/deleting files). It is designed exclusively
+ * for use with the {@code HoodieStashPartitionsTool} and the DELETE_PARTITION operation.
+ * Do NOT use this as a pattern for other validators.</p>
+ *
+ * <h3>Why move files inside a pre-commit validator?</h3>
+ *
+ * <p>The stash operation needs to both move data files to a backup location and mark
+ * the partitions as deleted via a replace commit. These two actions must be coupled:
+ * if the replace commit lands on the timeline but the file move has not happened yet,
+ * the Hudi cleaner will eventually delete the physical data files for the replaced
+ * file groups (based on its retention policy), permanently losing the data that was
+ * intended to be stashed.
+ *
+ * <p>By performing the file move inside this pre-commit validator, the move happens
+ * <b>before</b> the replace commit is finalized. If the move fails, the validator
+ * throws an exception, the commit does not land, and the table remains unchanged.
+ * This guarantees that data is never in a state where it has been marked as deleted
+ * but has not been safely backed up to the stash location.</p>
+ *
+ * <p>This validator requires the following configs to be set:
+ * <ul>
+ * <li>{@link #STASH_PATH_CONFIG} — the target stash directory</li>
+ * <li>{@link #RENAME_HELPER_CLASS_CONFIG} (optional) — custom rename helper class</li>
+ * </ul>
+ *
+ * <p>Guards:
+ * <ul>
+ * <li>Fails if RLI (Record Level Index) is enabled on the table</li>
+ * <li>Must only be used with DELETE_PARTITION operation</li>
+ * </ul>
+ */
+public class StashPartitionsPreCommitValidator<T, I, K, O extends HoodieData<WriteStatus>>
+ extends SparkPreCommitValidator<T, I, K, O> {
+
+ private static final Logger LOG = LoggerFactory.getLogger(StashPartitionsPreCommitValidator.class);
+
+ /**
+ * Config key for the stash target directory path.
+ */
+ public static final String STASH_PATH_CONFIG = "hoodie.stash.partitions.target.path";
+
+ /**
+ * Config key for the custom rename helper class. Defaults to {@link DefaultStashPartitionRenameHelper}.
+ */
+ public static final String RENAME_HELPER_CLASS_CONFIG = "hoodie.stash.rename.helper.class";
+
+ public StashPartitionsPreCommitValidator(HoodieSparkTable<T> table,
+ HoodieEngineContext engineContext,
+ HoodieWriteConfig writeConfig) {
+ super(table, engineContext, writeConfig);
+ }
+
+ /**
+ * Override validate to extract affected partitions from partitionToReplaceFileIds rather than
+ * write stats, since deletePartitions produces empty write stats but populates the replacement map.
+ */
+ @Override
+ public void validate(String instantTime, HoodieWriteMetadata<O> writeResult,
Review Comment:
We should also validate that the partitions don't have any uncleaned files right?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "nsivabalan (via GitHub)" <gi...@apache.org>.
nsivabalan commented on code in PR #18526:
URL: https://github.com/apache/hudi/pull/18526#discussion_r3127235132
##########
hudi-utilities/src/main/java/org/apache/hudi/utilities/HoodieStashPartitionsTool.java:
##########
@@ -0,0 +1,403 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.hudi.utilities;
+
+import org.apache.hudi.client.HoodieWriteResult;
+import org.apache.hudi.client.SparkRDDWriteClient;
+import org.apache.hudi.client.validator.DefaultStashPartitionRenameHelper;
+import org.apache.hudi.client.validator.StashPartitionRenameHelper;
+import org.apache.hudi.client.validator.StashPartitionsPreCommitValidator;
+import org.apache.hudi.common.config.TypedProperties;
+import org.apache.hudi.common.model.HoodieRecordPayload;
+import org.apache.hudi.common.table.HoodieTableMetaClient;
+import org.apache.hudi.common.table.timeline.HoodieTimeline;
+import org.apache.hudi.common.util.Option;
+import org.apache.hudi.common.util.ReflectionUtils;
+import org.apache.hudi.common.util.StringUtils;
+import org.apache.hudi.config.HoodiePreCommitValidatorConfig;
+import org.apache.hudi.exception.HoodieException;
+import org.apache.hudi.hadoop.fs.HadoopFSUtils;
+import org.apache.hudi.storage.HoodieStorage;
+import org.apache.hudi.storage.HoodieStorageUtils;
+import org.apache.hudi.storage.StoragePath;
+import org.apache.hudi.storage.StoragePathInfo;
+
+import com.beust.jcommander.JCommander;
+import com.beust.jcommander.Parameter;
+import org.apache.hadoop.fs.Path;
+import org.apache.spark.api.java.JavaSparkContext;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.IOException;
+import java.io.Serializable;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+
+/**
+ * A spark-submit tool for stashing (backing up and removing) Hudi table partitions.
+ *
+ * <p>This tool supports three modes:
+ * <ul>
+ * <li><b>stash</b> — Moves data files from specified partitions to a stash location
+ * and issues a deletePartitions operation on the Hudi table. Uses a custom
+ * {@link StashPartitionsPreCommitValidator} to move files atomically before commit.</li>
+ * <li><b>rollback_stash</b> — Copies any data found in the stash location back to the
+ * original partition paths. Used when the user no longer wants to proceed with stashing.</li>
+ * <li><b>dry_run</b> — Shows which partitions and file groups would be affected.</li>
+ * </ul>
+ *
+ * <h3>Why a pre-commit validator instead of moving files in the tool?</h3>
+ *
+ * <p>A naive approach would be to first run {@code deletePartitions} in the tool, and then
+ * move the data files to the stash location as a separate step. However, this is unsafe:
+ * if the tool crashes or fails after {@code deletePartitions} succeeds but before the data
+ * is moved to the stash location, the data is at risk. Once the replace commit from
+ * {@code deletePartitions} lands on the timeline, the Hudi cleaner will eventually delete
+ * the physical data files for the replaced file groups based on its configured retention
+ * policy (e.g., retain last N commits). If the cleaner runs before the tool is retried,
+ * the data intended for stashing is permanently lost.
+ *
+ * <p>By using a {@link StashPartitionsPreCommitValidator}, the file move happens
+ * <b>before</b> the {@code deletePartitions} commit is finalized. If the move fails, the
+ * commit does not land, and the table remains in its original state — the cleaner has
+ * nothing to clean. This ensures that data is never in a state where it has been marked
+ * as deleted (via a replace commit) but has not yet been safely copied to the stash location.
+ *
+ * <p>For restoring stashed partitions back into the Hudi table, use the standard
+ * {@code insert_overwrite} write operation reading from the stash location.
+ *
+ * <p>Example usage:
+ * <pre>
+ * spark-submit \
+ * --class org.apache.hudi.utilities.HoodieStashPartitionsTool \
+ * --master local[*] \
+ * --driver-memory 1g \
+ * --executor-memory 1g \
+ * hudi-utilities-bundle.jar \
+ * --base-path /path/to/hudi/table \
+ * --table-name myTable \
+ * --stash-path /path/to/stash/folder \
+ * --partitions datestr=2023-01-01,datestr=2023-01-02 \
+ * --mode stash
+ * </pre>
+ */
+public class HoodieStashPartitionsTool implements Serializable {
+
+ private static final long serialVersionUID = 1L;
+ private static final Logger LOG = LoggerFactory.getLogger(HoodieStashPartitionsTool.class);
+
+ private final transient JavaSparkContext jsc;
+ private final Config cfg;
+ private final TypedProperties props;
+ private final HoodieTableMetaClient metaClient;
+
+ public HoodieStashPartitionsTool(JavaSparkContext jsc, Config cfg) {
+ this.jsc = jsc;
+ this.cfg = cfg;
+ this.props = cfg.propsFilePath == null
+ ? UtilHelpers.buildProperties(cfg.configs)
+ : readConfigFromFileSystem(jsc, cfg);
+ this.metaClient = HoodieTableMetaClient.builder()
+ .setConf(HadoopFSUtils.getStorageConfWithCopy(jsc.hadoopConfiguration()))
+ .setBasePath(cfg.basePath)
+ .setLoadActiveTimelineOnLoad(true)
+ .build();
+ }
+
+ public enum Mode {
+ STASH,
+ ROLLBACK_STASH,
+ DRY_RUN
+ }
+
+ public static class Config implements Serializable {
+ @Parameter(names = {"--base-path", "-sp"}, description = "Base path for the Hudi table.", required = true)
+ public String basePath = null;
+
+ @Parameter(names = {"--table-name", "-tn"}, description = "Table name.", required = true)
+ public String tableName = null;
+
+ @Parameter(names = {"--stash-path"}, description = "Target path for stashing partition data.", required = true)
+ public String stashPath = null;
+
+ @Parameter(names = {"--partitions", "-p"}, description = "Comma separated list of partitions to stash.", required = true)
+ public String partitions = null;
+
+ @Parameter(names = {"--mode", "-m"}, description = "Set job mode: "
+ + "\"stash\" to move partition data to stash location and delete partitions; "
+ + "\"rollback_stash\" to copy stashed data back to table and abort stashing; "
+ + "\"dry_run\" to preview which partitions and files would be affected.", required = true)
+ public String runningMode = null;
+
+ @Parameter(names = {"--parallelism", "-pl"}, description = "Parallelism for hoodie operations.", required = false)
+ public int parallelism = 1500;
+
+ @Parameter(names = {"--rename-helper-class"}, description = "Fully qualified class name of a custom "
+ + "StashPartitionRenameHelper implementation. Defaults to DefaultStashPartitionRenameHelper.", required = false)
+ public String renameHelperClass = null;
+
+ @Parameter(names = {"--spark-master", "-ms"}, description = "Spark master.", required = false)
+ public String sparkMaster = null;
+
+ @Parameter(names = {"--spark-memory", "-sm"}, description = "Spark memory to use.", required = false)
+ public String sparkMemory = "1g";
+
+ @Parameter(names = {"--props"}, description = "Path to properties file on localfs or dfs, with configurations for "
+ + "hoodie client for stashing partitions.")
+ public String propsFilePath = null;
+
+ @Parameter(names = {"--hoodie-conf"}, description = "Any configuration that can be set in the properties file "
+ + "(using the CLI parameter \"--props\") can also be passed via command line using this parameter. This can be repeated.",
+ splitter = IdentitySplitter.class)
+ public List<String> configs = new ArrayList<>();
+
+ @Parameter(names = {"--help", "-h"}, help = true)
+ public Boolean help = false;
+
+ @Override
+ public String toString() {
+ return "HoodieStashPartitionsToolConfig {\n"
+ + " --base-path " + basePath + ", \n"
+ + " --table-name " + tableName + ", \n"
+ + " --stash-path " + stashPath + ", \n"
+ + " --partitions " + partitions + ", \n"
+ + " --mode " + runningMode + ", \n"
+ + " --parallelism " + parallelism + ", \n"
+ + " --rename-helper-class " + renameHelperClass + ", \n"
+ + " --spark-master " + sparkMaster + ", \n"
+ + " --spark-memory " + sparkMemory + ", \n"
+ + " --props " + propsFilePath + ", \n"
+ + " --hoodie-conf " + configs
+ + "\n}";
+ }
+ }
+
+ public static void main(String[] args) {
+ final Config cfg = new Config();
+ JCommander cmd = new JCommander(cfg, null, args);
+ if (cfg.help || args.length == 0) {
+ cmd.usage();
+ System.exit(1);
+ }
+ Map<String, String> sparkConfigMap = new HashMap<>();
+ sparkConfigMap.put("spark.executor.memory", cfg.sparkMemory);
+ JavaSparkContext jsc = UtilHelpers.buildSparkContext("Hoodie-Stash-Partitions",
+ cfg.sparkMaster, false, sparkConfigMap);
+
+ HoodieStashPartitionsTool tool = new HoodieStashPartitionsTool(jsc, cfg);
+ try {
+ tool.run();
+ } catch (Throwable throwable) {
+ LOG.error("Fail to run stash partitions for " + cfg, throwable);
+ } finally {
+ jsc.stop();
+ }
+ }
+
+ public void run() {
+ try {
+ LOG.info(cfg.toString());
+ Mode mode = Mode.valueOf(cfg.runningMode.toUpperCase());
+ switch (mode) {
+ case STASH:
+ LOG.info(" ****** The Hoodie Stash Partitions Tool is in stash mode ****** ");
+ doStash();
+ break;
+ case ROLLBACK_STASH:
+ LOG.info(" ****** The Hoodie Stash Partitions Tool is in rollback_stash mode ****** ");
+ doRollbackStash();
+ break;
+ case DRY_RUN:
+ LOG.info(" ****** The Hoodie Stash Partitions Tool is in dry-run mode ****** ");
+ doDryRun();
+ break;
+ default:
+ LOG.info("Unsupported running mode [" + cfg.runningMode + "], quit the job directly");
+ }
+ } catch (Exception e) {
+ throw new HoodieException("Unable to stash partitions in " + cfg.basePath, e);
+ }
+ }
+
+ /**
+ * Stash partitions: pre-check to recover any partial stash from a prior failed attempt,
+ * then issue deletePartitions with custom pre-commit validator. Hudi internally handles
+ * rollback of any prior failed instants. The pre-commit validator handles already-stashed
+ * partitions gracefully (source empty -> skip).
+ *
+ * <p>The pre-check runs regardless of whether MDT is enabled. Even with MDT, a prior failed
+ * attempt may have had its pre-commit validator move files to the stash location before
+ * crashing. Those files need to be moved back so that the subsequent deletePartitions
+ * sees the complete partition state.
+ */
+ private void doStash() {
+ List<String> partitionsToStash = Arrays.asList(cfg.partitions.split(","));
+
+ preCheckAndRecoverPartialStash(partitionsToStash);
+
+ // Set up props with stash-specific configs for the pre-commit validator
+ TypedProperties stashProps = new TypedProperties();
+ stashProps.putAll(props);
+ stashProps.setProperty(HoodiePreCommitValidatorConfig.VALIDATOR_CLASS_NAMES.key(),
+ StashPartitionsPreCommitValidator.class.getName());
+ stashProps.setProperty(StashPartitionsPreCommitValidator.STASH_PATH_CONFIG, cfg.stashPath);
+ if (!StringUtils.isNullOrEmpty(cfg.renameHelperClass)) {
+ stashProps.setProperty(StashPartitionsPreCommitValidator.RENAME_HELPER_CLASS_CONFIG, cfg.renameHelperClass);
+ }
+
+ try (SparkRDDWriteClient<HoodieRecordPayload> client =
+ UtilHelpers.createHoodieClient(jsc, cfg.basePath, "", cfg.parallelism, Option.empty(), stashProps)) {
+ String instantTime = client.startCommit(HoodieTimeline.REPLACE_COMMIT_ACTION);
+ LOG.info("Starting deletePartitions with instant {} for partitions: {}", instantTime, partitionsToStash);
+ HoodieWriteResult result = client.deletePartitions(partitionsToStash, instantTime);
+ client.commit(instantTime, result.getWriteStatuses(), Option.empty(), HoodieTimeline.REPLACE_COMMIT_ACTION,
+ result.getPartitionToReplaceFileIds(), Option.empty());
+ LOG.info("Successfully stashed partitions: {}", partitionsToStash);
+ }
+ }
+
+ /**
+ * Pre-check: if files exist in the stash location from a prior failed attempt,
+ * copy them back to source before retrying deletePartitions. This ensures the
+ * subsequent deletePartitions sees the complete partition state and creates
+ * a correct replace commit. Runs for both MDT and non-MDT tables, since the
+ * pre-commit validator may have moved files before crashing.
+ */
+ private void preCheckAndRecoverPartialStash(List<String> partitions) {
Review Comment:
sure. there are very few minor difference. will work on code dedup
##########
hudi-utilities/src/main/java/org/apache/hudi/utilities/HoodieStashPartitionsTool.java:
##########
@@ -0,0 +1,403 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.hudi.utilities;
+
+import org.apache.hudi.client.HoodieWriteResult;
+import org.apache.hudi.client.SparkRDDWriteClient;
+import org.apache.hudi.client.validator.DefaultStashPartitionRenameHelper;
+import org.apache.hudi.client.validator.StashPartitionRenameHelper;
+import org.apache.hudi.client.validator.StashPartitionsPreCommitValidator;
+import org.apache.hudi.common.config.TypedProperties;
+import org.apache.hudi.common.model.HoodieRecordPayload;
+import org.apache.hudi.common.table.HoodieTableMetaClient;
+import org.apache.hudi.common.table.timeline.HoodieTimeline;
+import org.apache.hudi.common.util.Option;
+import org.apache.hudi.common.util.ReflectionUtils;
+import org.apache.hudi.common.util.StringUtils;
+import org.apache.hudi.config.HoodiePreCommitValidatorConfig;
+import org.apache.hudi.exception.HoodieException;
+import org.apache.hudi.hadoop.fs.HadoopFSUtils;
+import org.apache.hudi.storage.HoodieStorage;
+import org.apache.hudi.storage.HoodieStorageUtils;
+import org.apache.hudi.storage.StoragePath;
+import org.apache.hudi.storage.StoragePathInfo;
+
+import com.beust.jcommander.JCommander;
+import com.beust.jcommander.Parameter;
+import org.apache.hadoop.fs.Path;
+import org.apache.spark.api.java.JavaSparkContext;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.IOException;
+import java.io.Serializable;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+
+/**
+ * A spark-submit tool for stashing (backing up and removing) Hudi table partitions.
+ *
+ * <p>This tool supports three modes:
+ * <ul>
+ * <li><b>stash</b> — Moves data files from specified partitions to a stash location
+ * and issues a deletePartitions operation on the Hudi table. Uses a custom
+ * {@link StashPartitionsPreCommitValidator} to move files atomically before commit.</li>
+ * <li><b>rollback_stash</b> — Copies any data found in the stash location back to the
+ * original partition paths. Used when the user no longer wants to proceed with stashing.</li>
+ * <li><b>dry_run</b> — Shows which partitions and file groups would be affected.</li>
+ * </ul>
+ *
+ * <h3>Why a pre-commit validator instead of moving files in the tool?</h3>
+ *
+ * <p>A naive approach would be to first run {@code deletePartitions} in the tool, and then
+ * move the data files to the stash location as a separate step. However, this is unsafe:
+ * if the tool crashes or fails after {@code deletePartitions} succeeds but before the data
+ * is moved to the stash location, the data is at risk. Once the replace commit from
+ * {@code deletePartitions} lands on the timeline, the Hudi cleaner will eventually delete
+ * the physical data files for the replaced file groups based on its configured retention
+ * policy (e.g., retain last N commits). If the cleaner runs before the tool is retried,
+ * the data intended for stashing is permanently lost.
+ *
+ * <p>By using a {@link StashPartitionsPreCommitValidator}, the file move happens
+ * <b>before</b> the {@code deletePartitions} commit is finalized. If the move fails, the
+ * commit does not land, and the table remains in its original state — the cleaner has
+ * nothing to clean. This ensures that data is never in a state where it has been marked
+ * as deleted (via a replace commit) but has not yet been safely copied to the stash location.
+ *
+ * <p>For restoring stashed partitions back into the Hudi table, use the standard
+ * {@code insert_overwrite} write operation reading from the stash location.
+ *
+ * <p>Example usage:
+ * <pre>
+ * spark-submit \
+ * --class org.apache.hudi.utilities.HoodieStashPartitionsTool \
+ * --master local[*] \
+ * --driver-memory 1g \
+ * --executor-memory 1g \
+ * hudi-utilities-bundle.jar \
+ * --base-path /path/to/hudi/table \
+ * --table-name myTable \
+ * --stash-path /path/to/stash/folder \
+ * --partitions datestr=2023-01-01,datestr=2023-01-02 \
+ * --mode stash
+ * </pre>
+ */
+public class HoodieStashPartitionsTool implements Serializable {
+
+ private static final long serialVersionUID = 1L;
+ private static final Logger LOG = LoggerFactory.getLogger(HoodieStashPartitionsTool.class);
+
+ private final transient JavaSparkContext jsc;
+ private final Config cfg;
+ private final TypedProperties props;
+ private final HoodieTableMetaClient metaClient;
+
+ public HoodieStashPartitionsTool(JavaSparkContext jsc, Config cfg) {
+ this.jsc = jsc;
+ this.cfg = cfg;
+ this.props = cfg.propsFilePath == null
+ ? UtilHelpers.buildProperties(cfg.configs)
+ : readConfigFromFileSystem(jsc, cfg);
+ this.metaClient = HoodieTableMetaClient.builder()
+ .setConf(HadoopFSUtils.getStorageConfWithCopy(jsc.hadoopConfiguration()))
+ .setBasePath(cfg.basePath)
+ .setLoadActiveTimelineOnLoad(true)
+ .build();
+ }
+
+ public enum Mode {
+ STASH,
+ ROLLBACK_STASH,
+ DRY_RUN
+ }
+
+ public static class Config implements Serializable {
+ @Parameter(names = {"--base-path", "-sp"}, description = "Base path for the Hudi table.", required = true)
+ public String basePath = null;
+
+ @Parameter(names = {"--table-name", "-tn"}, description = "Table name.", required = true)
+ public String tableName = null;
+
+ @Parameter(names = {"--stash-path"}, description = "Target path for stashing partition data.", required = true)
+ public String stashPath = null;
+
+ @Parameter(names = {"--partitions", "-p"}, description = "Comma separated list of partitions to stash.", required = true)
+ public String partitions = null;
+
+ @Parameter(names = {"--mode", "-m"}, description = "Set job mode: "
+ + "\"stash\" to move partition data to stash location and delete partitions; "
+ + "\"rollback_stash\" to copy stashed data back to table and abort stashing; "
+ + "\"dry_run\" to preview which partitions and files would be affected.", required = true)
+ public String runningMode = null;
+
+ @Parameter(names = {"--parallelism", "-pl"}, description = "Parallelism for hoodie operations.", required = false)
+ public int parallelism = 1500;
+
+ @Parameter(names = {"--rename-helper-class"}, description = "Fully qualified class name of a custom "
+ + "StashPartitionRenameHelper implementation. Defaults to DefaultStashPartitionRenameHelper.", required = false)
+ public String renameHelperClass = null;
+
+ @Parameter(names = {"--spark-master", "-ms"}, description = "Spark master.", required = false)
+ public String sparkMaster = null;
+
+ @Parameter(names = {"--spark-memory", "-sm"}, description = "Spark memory to use.", required = false)
+ public String sparkMemory = "1g";
+
+ @Parameter(names = {"--props"}, description = "Path to properties file on localfs or dfs, with configurations for "
+ + "hoodie client for stashing partitions.")
+ public String propsFilePath = null;
+
+ @Parameter(names = {"--hoodie-conf"}, description = "Any configuration that can be set in the properties file "
+ + "(using the CLI parameter \"--props\") can also be passed via command line using this parameter. This can be repeated.",
+ splitter = IdentitySplitter.class)
+ public List<String> configs = new ArrayList<>();
+
+ @Parameter(names = {"--help", "-h"}, help = true)
+ public Boolean help = false;
+
+ @Override
+ public String toString() {
+ return "HoodieStashPartitionsToolConfig {\n"
+ + " --base-path " + basePath + ", \n"
+ + " --table-name " + tableName + ", \n"
+ + " --stash-path " + stashPath + ", \n"
+ + " --partitions " + partitions + ", \n"
+ + " --mode " + runningMode + ", \n"
+ + " --parallelism " + parallelism + ", \n"
+ + " --rename-helper-class " + renameHelperClass + ", \n"
+ + " --spark-master " + sparkMaster + ", \n"
+ + " --spark-memory " + sparkMemory + ", \n"
+ + " --props " + propsFilePath + ", \n"
+ + " --hoodie-conf " + configs
+ + "\n}";
+ }
+ }
+
+ public static void main(String[] args) {
+ final Config cfg = new Config();
+ JCommander cmd = new JCommander(cfg, null, args);
+ if (cfg.help || args.length == 0) {
+ cmd.usage();
+ System.exit(1);
+ }
+ Map<String, String> sparkConfigMap = new HashMap<>();
+ sparkConfigMap.put("spark.executor.memory", cfg.sparkMemory);
+ JavaSparkContext jsc = UtilHelpers.buildSparkContext("Hoodie-Stash-Partitions",
+ cfg.sparkMaster, false, sparkConfigMap);
+
+ HoodieStashPartitionsTool tool = new HoodieStashPartitionsTool(jsc, cfg);
+ try {
+ tool.run();
+ } catch (Throwable throwable) {
+ LOG.error("Fail to run stash partitions for " + cfg, throwable);
+ } finally {
+ jsc.stop();
+ }
+ }
+
+ public void run() {
+ try {
+ LOG.info(cfg.toString());
+ Mode mode = Mode.valueOf(cfg.runningMode.toUpperCase());
+ switch (mode) {
+ case STASH:
+ LOG.info(" ****** The Hoodie Stash Partitions Tool is in stash mode ****** ");
+ doStash();
+ break;
+ case ROLLBACK_STASH:
+ LOG.info(" ****** The Hoodie Stash Partitions Tool is in rollback_stash mode ****** ");
+ doRollbackStash();
+ break;
+ case DRY_RUN:
+ LOG.info(" ****** The Hoodie Stash Partitions Tool is in dry-run mode ****** ");
+ doDryRun();
+ break;
+ default:
+ LOG.info("Unsupported running mode [" + cfg.runningMode + "], quit the job directly");
+ }
+ } catch (Exception e) {
+ throw new HoodieException("Unable to stash partitions in " + cfg.basePath, e);
+ }
+ }
+
+ /**
+ * Stash partitions: pre-check to recover any partial stash from a prior failed attempt,
+ * then issue deletePartitions with custom pre-commit validator. Hudi internally handles
+ * rollback of any prior failed instants. The pre-commit validator handles already-stashed
+ * partitions gracefully (source empty -> skip).
+ *
+ * <p>The pre-check runs regardless of whether MDT is enabled. Even with MDT, a prior failed
+ * attempt may have had its pre-commit validator move files to the stash location before
+ * crashing. Those files need to be moved back so that the subsequent deletePartitions
+ * sees the complete partition state.
+ */
+ private void doStash() {
+ List<String> partitionsToStash = Arrays.asList(cfg.partitions.split(","));
+
+ preCheckAndRecoverPartialStash(partitionsToStash);
+
+ // Set up props with stash-specific configs for the pre-commit validator
+ TypedProperties stashProps = new TypedProperties();
+ stashProps.putAll(props);
+ stashProps.setProperty(HoodiePreCommitValidatorConfig.VALIDATOR_CLASS_NAMES.key(),
+ StashPartitionsPreCommitValidator.class.getName());
+ stashProps.setProperty(StashPartitionsPreCommitValidator.STASH_PATH_CONFIG, cfg.stashPath);
+ if (!StringUtils.isNullOrEmpty(cfg.renameHelperClass)) {
+ stashProps.setProperty(StashPartitionsPreCommitValidator.RENAME_HELPER_CLASS_CONFIG, cfg.renameHelperClass);
+ }
+
+ try (SparkRDDWriteClient<HoodieRecordPayload> client =
+ UtilHelpers.createHoodieClient(jsc, cfg.basePath, "", cfg.parallelism, Option.empty(), stashProps)) {
+ String instantTime = client.startCommit(HoodieTimeline.REPLACE_COMMIT_ACTION);
+ LOG.info("Starting deletePartitions with instant {} for partitions: {}", instantTime, partitionsToStash);
+ HoodieWriteResult result = client.deletePartitions(partitionsToStash, instantTime);
+ client.commit(instantTime, result.getWriteStatuses(), Option.empty(), HoodieTimeline.REPLACE_COMMIT_ACTION,
+ result.getPartitionToReplaceFileIds(), Option.empty());
+ LOG.info("Successfully stashed partitions: {}", partitionsToStash);
+ }
+ }
+
+ /**
+ * Pre-check: if files exist in the stash location from a prior failed attempt,
+ * copy them back to source before retrying deletePartitions. This ensures the
+ * subsequent deletePartitions sees the complete partition state and creates
+ * a correct replace commit. Runs for both MDT and non-MDT tables, since the
+ * pre-commit validator may have moved files before crashing.
+ */
+ private void preCheckAndRecoverPartialStash(List<String> partitions) {
+ LOG.info("Running pre-check for partial stash recovery (non-MDT table).");
+ HoodieStorage storage = HoodieStorageUtils.getStorage(
+ cfg.basePath, metaClient.getStorageConf());
+ StashPartitionRenameHelper renameHelper = createRenameHelper();
+ StoragePath basePath = metaClient.getBasePath();
+
+ for (String partition : partitions) {
+ StoragePath stashPartitionPath = new StoragePath(cfg.stashPath, partition);
+ StoragePath sourcePartitionPath = new StoragePath(basePath, partition);
+
+ try {
+ if (!storage.exists(stashPartitionPath) || isDirectoryEmpty(storage, stashPartitionPath)) {
+ LOG.info("Partition {} has no stashed data from prior attempt, skipping pre-check.", partition);
+ continue;
+ }
+ // Files exist in stash — copy them back to source to ensure deletePartitions
+ // sees all files and creates a complete replace commit.
+ LOG.info("Found stashed files for partition {} from prior attempt. Moving back to source.", partition);
+ renameHelper.movePartitionFiles(storage, stashPartitionPath, sourcePartitionPath);
Review Comment:
sure
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "nsivabalan (via GitHub)" <gi...@apache.org>.
nsivabalan commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-4300437595
hey @kbuci : all feedback has been addressed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-4300606001
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `16.46707%` with `279 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 48.60%. Comparing base ([`7bcb8be`](https://app.codecov.io/gh/apache/hudi/commit/7bcb8be7c153010998574f591c7ff3799192eda6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`6d8dedd`](https://app.codecov.io/gh/apache/hudi/commit/6d8dedd061ed183c52471df09355b85e5a60099a?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 20 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | 0.00% | [263 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | 77.50% | [5 Missing and 4 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | 77.41% | [5 Missing and 2 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
> :exclamation: There is a different number of reports uploaded between BASE (7bcb8be) and HEAD (6d8dedd). Click for more details.
>
> <details><summary>HEAD has 25 uploads less than BASE</summary>
>
>| Flag | BASE (7bcb8be) | HEAD (6d8dedd) |
>|------|------|------|
>|spark-scala-tests|10|1|
>|spark-java-tests|15|1|
>|common-and-other-modules|1|0|
>|utilities|1|0|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18526 +/- ##
=============================================
- Coverage 68.88% 48.60% -20.29%
+ Complexity 28279 16082 -12197
=============================================
Files 2464 1971 -493
Lines 135568 108546 -27022
Branches 16446 13380 -3066
=============================================
- Hits 93385 52757 -40628
- Misses 34801 50433 +15632
+ Partials 7382 5356 -2026
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.74% <0.00%> (-0.02%)` | :arrow_down: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.64% <77.46%> (+0.23%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `33.21% <0.00%> (-15.72%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `32.84% <0.00%> (-12.60%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | `77.41% <77.41%> (ø)` | |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | `77.50% <77.50%> (ø)` | |
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | `0.00% <0.00%> (ø)` | |
... and [1579 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18526/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-4300633608
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `23.05389%` with `257 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 58.61%. Comparing base ([`7bcb8be`](https://app.codecov.io/gh/apache/hudi/commit/7bcb8be7c153010998574f591c7ff3799192eda6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`6d8dedd`](https://app.codecov.io/gh/apache/hudi/commit/6d8dedd061ed183c52471df09355b85e5a60099a?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 21 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | 6.46% | [245 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | 77.50% | [5 Missing and 4 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | 93.54% | [1 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
> :exclamation: There is a different number of reports uploaded between BASE (7bcb8be) and HEAD (6d8dedd). Click for more details.
>
> <details><summary>HEAD has 17 uploads less than BASE</summary>
>
>| Flag | BASE (7bcb8be) | HEAD (6d8dedd) |
>|------|------|------|
>|spark-scala-tests|10|5|
>|spark-java-tests|15|4|
>|utilities|1|0|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18526 +/- ##
=============================================
- Coverage 68.88% 58.61% -10.27%
+ Complexity 28279 24322 -3957
=============================================
Files 2464 2478 +14
Lines 135568 136958 +1390
Branches 16446 16651 +205
=============================================
- Hits 93385 80279 -13106
- Misses 34801 49794 +14993
+ Partials 7382 6885 -497
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.39% <13.77%> (-0.25%)` | :arrow_down: |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.74% <0.00%> (-0.02%)` | :arrow_down: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.64% <77.46%> (+0.23%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `32.18% <0.00%> (-16.75%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `32.36% <0.00%> (-13.09%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | `93.54% <93.54%> (ø)` | |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | `77.50% <77.50%> (ø)` | |
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | `6.46% <6.46%> (ø)` | |
... and [682 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18526/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-4300782625
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `80.83832%` with `64 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 68.68%. Comparing base ([`7bcb8be`](https://app.codecov.io/gh/apache/hudi/commit/7bcb8be7c153010998574f591c7ff3799192eda6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`6d8dedd`](https://app.codecov.io/gh/apache/hudi/commit/6d8dedd061ed183c52471df09355b85e5a60099a?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 22 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | 79.84% | [43 Missing and 10 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | 77.50% | [5 Missing and 4 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | 93.54% | [1 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18526 +/- ##
============================================
- Coverage 68.88% 68.68% -0.21%
- Complexity 28279 28473 +194
============================================
Files 2464 2478 +14
Lines 135568 136972 +1404
Branches 16446 16653 +207
============================================
+ Hits 93385 94077 +692
- Misses 34801 35324 +523
- Partials 7382 7571 +189
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.39% <13.77%> (-0.25%)` | :arrow_down: |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.74% <0.00%> (-0.02%)` | :arrow_down: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.64% <77.46%> (+0.23%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `49.25% <0.00%> (+0.32%)` | :arrow_up: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.18% <0.00%> (-0.26%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `37.87% <77.54%> (-0.30%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | `93.54% <93.54%> (ø)` | |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | `77.50% <77.50%> (ø)` | |
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | `79.84% <79.84%> (ø)` | |
... and [79 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18526/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-4272530003
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `78.47534%` with `48 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 68.82%. Comparing base ([`7bcb8be`](https://app.codecov.io/gh/apache/hudi/commit/7bcb8be7c153010998574f591c7ff3799192eda6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`e8b5a0a`](https://app.codecov.io/gh/apache/hudi/commit/e8b5a0a62cea99be0a184e0b6f11149b49850806?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 3 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | 76.92% | [28 Missing and 8 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | 75.00% | [6 Missing and 4 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | 92.59% | [1 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18526 +/- ##
============================================
- Coverage 68.88% 68.82% -0.07%
+ Complexity 28279 28271 -8
============================================
Files 2464 2467 +3
Lines 135568 135817 +249
Branches 16446 16468 +22
============================================
+ Hits 93385 93475 +90
- Misses 34801 34928 +127
- Partials 7382 7414 +32
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.59% <11.21%> (-0.05%)` | :arrow_down: |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.70% <0.00%> (-0.06%)` | :arrow_down: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.46% <77.61%> (+0.06%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.58% <0.00%> (-0.34%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.35% <0.00%> (-0.09%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `38.31% <74.88%> (+0.14%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | `92.59% <92.59%> (ø)` | |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | `75.00% <75.00%> (ø)` | |
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | `76.92% <76.92%> (ø)` | |
... and [28 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18526/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-4272414850
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "67f1e57f0dffe02bbe2d7bacfaa8f1aaa4be8c88",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=13351",
"triggerID" : "67f1e57f0dffe02bbe2d7bacfaa8f1aaa4be8c88",
"triggerType" : "PUSH"
}, {
"hash" : "e8b5a0a62cea99be0a184e0b6f11149b49850806",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e8b5a0a62cea99be0a184e0b6f11149b49850806",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 67f1e57f0dffe02bbe2d7bacfaa8f1aaa4be8c88 Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=13351)
* e8b5a0a62cea99be0a184e0b6f11149b49850806 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-4284407658
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `77.61194%` with `15 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 53.95%. Comparing base ([`7bcb8be`](https://app.codecov.io/gh/apache/hudi/commit/7bcb8be7c153010998574f591c7ff3799192eda6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`891eaf2`](https://app.codecov.io/gh/apache/hudi/commit/891eaf2e1427bf93b9c7ff9626628c30e1c9c01d?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 11 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | 75.00% | [6 Missing and 4 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | 81.48% | [3 Missing and 2 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
> :exclamation: There is a different number of reports uploaded between BASE (7bcb8be) and HEAD (891eaf2). Click for more details.
>
> <details><summary>HEAD has 27 uploads less than BASE</summary>
>
>| Flag | BASE (7bcb8be) | HEAD (891eaf2) |
>|------|------|------|
>|spark-scala-tests|10|0|
>|spark-java-tests|15|0|
>|common-and-other-modules|1|0|
>|utilities|1|0|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18526 +/- ##
=============================================
- Coverage 68.88% 53.95% -14.94%
+ Complexity 28279 12336 -15943
=============================================
Files 2464 1430 -1034
Lines 135568 71421 -64147
Branches 16446 8078 -8368
=============================================
- Hits 93385 38533 -54852
+ Misses 34801 29443 -5358
+ Partials 7382 3445 -3937
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.75% <0.00%> (+<0.01%)` | :arrow_up: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.47% <77.61%> (+0.07%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | `81.48% <81.48%> (ø)` | |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | `75.00% <75.00%> (ø)` | |
... and [1815 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18526/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-4284532949
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `78.47534%` with `48 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 68.78%. Comparing base ([`7bcb8be`](https://app.codecov.io/gh/apache/hudi/commit/7bcb8be7c153010998574f591c7ff3799192eda6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`891eaf2`](https://app.codecov.io/gh/apache/hudi/commit/891eaf2e1427bf93b9c7ff9626628c30e1c9c01d?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 11 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | 76.92% | [28 Missing and 8 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | 75.00% | [6 Missing and 4 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | 92.59% | [1 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18526 +/- ##
============================================
- Coverage 68.88% 68.78% -0.10%
- Complexity 28279 28359 +80
============================================
Files 2464 2471 +7
Lines 135568 136218 +650
Branches 16446 16521 +75
============================================
+ Hits 93385 93698 +313
- Misses 34801 35101 +300
- Partials 7382 7419 +37
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.56% <11.21%> (-0.08%)` | :arrow_down: |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.75% <0.00%> (+<0.01%)` | :arrow_up: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.47% <77.61%> (+0.07%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.71% <0.00%> (-0.22%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.34% <0.00%> (-0.10%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `38.30% <74.88%> (+0.13%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | `92.59% <92.59%> (ø)` | |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | `75.00% <75.00%> (ø)` | |
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | `76.92% <76.92%> (ø)` | |
... and [32 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18526/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-4284469201
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `78.47534%` with `48 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 61.38%. Comparing base ([`7bcb8be`](https://app.codecov.io/gh/apache/hudi/commit/7bcb8be7c153010998574f591c7ff3799192eda6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`891eaf2`](https://app.codecov.io/gh/apache/hudi/commit/891eaf2e1427bf93b9c7ff9626628c30e1c9c01d?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 11 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | 76.92% | [28 Missing and 8 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | 75.00% | [6 Missing and 4 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | 92.59% | [1 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
> :exclamation: There is a different number of reports uploaded between BASE (7bcb8be) and HEAD (891eaf2). Click for more details.
>
> <details><summary>HEAD has 15 uploads less than BASE</summary>
>
>| Flag | BASE (7bcb8be) | HEAD (891eaf2) |
>|------|------|------|
>|spark-scala-tests|10|5|
>|spark-java-tests|15|5|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18526 +/- ##
============================================
- Coverage 68.88% 61.38% -7.51%
+ Complexity 28279 25243 -3036
============================================
Files 2464 2471 +7
Lines 135568 136218 +650
Branches 16446 16521 +75
============================================
- Hits 93385 83612 -9773
- Misses 34801 45583 +10782
+ Partials 7382 7023 -359
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.56% <11.21%> (-0.08%)` | :arrow_down: |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.75% <0.00%> (+<0.01%)` | :arrow_up: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.47% <77.61%> (+0.07%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `32.28% <0.00%> (-16.64%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `32.48% <0.00%> (-12.96%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `38.30% <74.88%> (+0.13%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | `92.59% <92.59%> (ø)` | |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | `75.00% <75.00%> (ø)` | |
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | `76.92% <76.92%> (ø)` | |
... and [557 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18526/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-4284421556
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `23.31839%` with `171 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 46.63%. Comparing base ([`7bcb8be`](https://app.codecov.io/gh/apache/hudi/commit/7bcb8be7c153010998574f591c7ff3799192eda6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`891eaf2`](https://app.codecov.io/gh/apache/hudi/commit/891eaf2e1427bf93b9c7ff9626628c30e1c9c01d?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 11 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | 0.00% | [156 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | 75.00% | [6 Missing and 4 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | 81.48% | [3 Missing and 2 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
> :exclamation: There is a different number of reports uploaded between BASE (7bcb8be) and HEAD (891eaf2). Click for more details.
>
> <details><summary>HEAD has 26 uploads less than BASE</summary>
>
>| Flag | BASE (7bcb8be) | HEAD (891eaf2) |
>|------|------|------|
>|spark-scala-tests|10|1|
>|spark-java-tests|15|0|
>|common-and-other-modules|1|0|
>|utilities|1|0|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18526 +/- ##
=============================================
- Coverage 68.88% 46.63% -22.25%
+ Complexity 28279 15050 -13229
=============================================
Files 2464 1947 -517
Lines 135568 105363 -30205
Branches 16446 12705 -3741
=============================================
- Hits 93385 49141 -44244
- Misses 34801 51179 +16378
+ Partials 7382 5043 -2339
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.75% <0.00%> (+<0.01%)` | :arrow_up: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.47% <77.61%> (+0.07%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `33.00% <0.00%> (-12.44%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | `81.48% <81.48%> (ø)` | |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | `75.00% <75.00%> (ø)` | |
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | `0.00% <0.00%> (ø)` | |
... and [1641 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18526/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-4300686416
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `23.05389%` with `257 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 66.01%. Comparing base ([`7bcb8be`](https://app.codecov.io/gh/apache/hudi/commit/7bcb8be7c153010998574f591c7ff3799192eda6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`6d8dedd`](https://app.codecov.io/gh/apache/hudi/commit/6d8dedd061ed183c52471df09355b85e5a60099a?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 21 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | 6.46% | [245 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | 77.50% | [5 Missing and 4 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | 93.54% | [1 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18526 +/- ##
============================================
- Coverage 68.88% 66.01% -2.88%
+ Complexity 28279 27431 -848
============================================
Files 2464 2478 +14
Lines 135568 136972 +1404
Branches 16446 16653 +207
============================================
- Hits 93385 90421 -2964
- Misses 34801 39026 +4225
- Partials 7382 7525 +143
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.39% <13.77%> (-0.25%)` | :arrow_down: |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.74% <0.00%> (-0.02%)` | :arrow_down: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.64% <77.46%> (+0.23%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.48% <0.00%> (-3.45%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.18% <0.00%> (-0.26%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | `93.54% <93.54%> (ø)` | |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | `77.50% <77.50%> (ø)` | |
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | `6.46% <6.46%> (ø)` | |
... and [297 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18526/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-4300728819
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `80.83832%` with `64 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 68.68%. Comparing base ([`7bcb8be`](https://app.codecov.io/gh/apache/hudi/commit/7bcb8be7c153010998574f591c7ff3799192eda6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`6d8dedd`](https://app.codecov.io/gh/apache/hudi/commit/6d8dedd061ed183c52471df09355b85e5a60099a?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 22 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | 79.84% | [43 Missing and 10 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | 77.50% | [5 Missing and 4 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | 93.54% | [1 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18526 +/- ##
============================================
- Coverage 68.88% 68.68% -0.21%
- Complexity 28279 28473 +194
============================================
Files 2464 2478 +14
Lines 135568 136972 +1404
Branches 16446 16653 +207
============================================
+ Hits 93385 94077 +692
- Misses 34801 35324 +523
- Partials 7382 7571 +189
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.39% <13.77%> (-0.25%)` | :arrow_down: |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.74% <0.00%> (-0.02%)` | :arrow_down: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.64% <77.46%> (+0.23%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `49.22% <0.00%> (+0.29%)` | :arrow_up: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.18% <0.00%> (-0.26%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `37.87% <77.54%> (-0.30%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | `93.54% <93.54%> (ø)` | |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | `77.50% <77.50%> (ø)` | |
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | `79.84% <79.84%> (ø)` | |
... and [79 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18526/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-4300764620
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `80.83832%` with `64 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 68.68%. Comparing base ([`7bcb8be`](https://app.codecov.io/gh/apache/hudi/commit/7bcb8be7c153010998574f591c7ff3799192eda6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`6d8dedd`](https://app.codecov.io/gh/apache/hudi/commit/6d8dedd061ed183c52471df09355b85e5a60099a?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 22 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | 79.84% | [43 Missing and 10 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | 77.50% | [5 Missing and 4 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | 93.54% | [1 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18526 +/- ##
============================================
- Coverage 68.88% 68.68% -0.21%
- Complexity 28279 28473 +194
============================================
Files 2464 2478 +14
Lines 135568 136972 +1404
Branches 16446 16653 +207
============================================
+ Hits 93385 94077 +692
- Misses 34801 35324 +523
- Partials 7382 7571 +189
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.39% <13.77%> (-0.25%)` | :arrow_down: |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.74% <0.00%> (-0.02%)` | :arrow_down: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.64% <77.46%> (+0.23%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `49.25% <0.00%> (+0.32%)` | :arrow_up: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.18% <0.00%> (-0.26%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `37.87% <77.54%> (-0.30%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | `93.54% <93.54%> (ø)` | |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | `77.50% <77.50%> (ø)` | |
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | `79.84% <79.84%> (ø)` | |
... and [79 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18526/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-4300619375
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `16.46707%` with `279 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 47.77%. Comparing base ([`7bcb8be`](https://app.codecov.io/gh/apache/hudi/commit/7bcb8be7c153010998574f591c7ff3799192eda6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`6d8dedd`](https://app.codecov.io/gh/apache/hudi/commit/6d8dedd061ed183c52471df09355b85e5a60099a?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 21 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | 0.00% | [263 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | 77.50% | [5 Missing and 4 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | 77.41% | [5 Missing and 2 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
> :exclamation: There is a different number of reports uploaded between BASE (7bcb8be) and HEAD (6d8dedd). Click for more details.
>
> <details><summary>HEAD has 21 uploads less than BASE</summary>
>
>| Flag | BASE (7bcb8be) | HEAD (6d8dedd) |
>|------|------|------|
>|spark-scala-tests|10|5|
>|spark-java-tests|15|1|
>|common-and-other-modules|1|0|
>|utilities|1|0|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18526 +/- ##
=============================================
- Coverage 68.88% 47.77% -21.11%
+ Complexity 28279 16567 -11712
=============================================
Files 2464 2018 -446
Lines 135568 113561 -22007
Branches 16446 14458 -1988
=============================================
- Hits 93385 54259 -39126
- Misses 34801 53582 +18781
+ Partials 7382 5720 -1662
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.74% <0.00%> (-0.02%)` | :arrow_down: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.64% <77.46%> (+0.23%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `33.21% <0.00%> (-15.72%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `32.36% <0.00%> (-13.09%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | `77.41% <77.41%> (ø)` | |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | `77.50% <77.50%> (ø)` | |
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | `0.00% <0.00%> (ø)` | |
... and [1546 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18526/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-4300659066
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `23.05389%` with `257 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 65.57%. Comparing base ([`7bcb8be`](https://app.codecov.io/gh/apache/hudi/commit/7bcb8be7c153010998574f591c7ff3799192eda6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`6d8dedd`](https://app.codecov.io/gh/apache/hudi/commit/6d8dedd061ed183c52471df09355b85e5a60099a?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 21 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | 6.46% | [245 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | 77.50% | [5 Missing and 4 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | 93.54% | [1 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18526 +/- ##
============================================
- Coverage 68.88% 65.57% -3.32%
+ Complexity 28279 27174 -1105
============================================
Files 2464 2478 +14
Lines 135568 136958 +1390
Branches 16446 16651 +205
============================================
- Hits 93385 89804 -3581
- Misses 34801 39786 +4985
+ Partials 7382 7368 -14
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.39% <13.77%> (-0.25%)` | :arrow_down: |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.74% <0.00%> (-0.02%)` | :arrow_down: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.64% <77.46%> (+0.23%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `42.74% <0.00%> (-6.18%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.62% <0.00%> (-0.83%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | `93.54% <93.54%> (ø)` | |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | `77.50% <77.50%> (ø)` | |
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | `6.46% <6.46%> (ø)` | |
... and [349 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18526/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-4300683255
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `23.05389%` with `257 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 65.91%. Comparing base ([`7bcb8be`](https://app.codecov.io/gh/apache/hudi/commit/7bcb8be7c153010998574f591c7ff3799192eda6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`6d8dedd`](https://app.codecov.io/gh/apache/hudi/commit/6d8dedd061ed183c52471df09355b85e5a60099a?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 21 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | 6.46% | [245 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | 77.50% | [5 Missing and 4 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | 93.54% | [1 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18526 +/- ##
============================================
- Coverage 68.88% 65.91% -2.98%
+ Complexity 28279 27363 -916
============================================
Files 2464 2478 +14
Lines 135568 136972 +1404
Branches 16446 16653 +207
============================================
- Hits 93385 90282 -3103
- Misses 34801 39215 +4414
- Partials 7382 7475 +93
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.39% <13.77%> (-0.25%)` | :arrow_down: |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.74% <0.00%> (-0.02%)` | :arrow_down: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.64% <77.46%> (+0.23%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.48% <0.00%> (-3.45%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.01% <0.00%> (-0.43%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | `93.54% <93.54%> (ø)` | |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | `77.50% <77.50%> (ø)` | |
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | `6.46% <6.46%> (ø)` | |
... and [300 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18526/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-4284660707
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "67f1e57f0dffe02bbe2d7bacfaa8f1aaa4be8c88",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=13351",
"triggerID" : "67f1e57f0dffe02bbe2d7bacfaa8f1aaa4be8c88",
"triggerType" : "PUSH"
}, {
"hash" : "e8b5a0a62cea99be0a184e0b6f11149b49850806",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=13354",
"triggerID" : "e8b5a0a62cea99be0a184e0b6f11149b49850806",
"triggerType" : "PUSH"
}, {
"hash" : "891eaf2e1427bf93b9c7ff9626628c30e1c9c01d",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=13391",
"triggerID" : "891eaf2e1427bf93b9c7ff9626628c30e1c9c01d",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 891eaf2e1427bf93b9c7ff9626628c30e1c9c01d Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=13391)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "kbuci (via GitHub)" <gi...@apache.org>.
kbuci commented on code in PR #18526:
URL: https://github.com/apache/hudi/pull/18526#discussion_r3120869530
##########
hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/client/validator/StashPartitionsPreCommitValidator.java:
##########
@@ -0,0 +1,181 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.hudi.client.validator;
+
+import org.apache.hudi.client.WriteStatus;
+import org.apache.hudi.common.config.HoodieMetadataConfig;
+import org.apache.hudi.common.data.HoodieData;
+import org.apache.hudi.common.engine.HoodieEngineContext;
+import org.apache.hudi.common.util.ReflectionUtils;
+import org.apache.hudi.common.util.StringUtils;
+import org.apache.hudi.common.util.ValidationUtils;
+import org.apache.hudi.config.HoodieWriteConfig;
+import org.apache.hudi.exception.HoodieValidationException;
+import org.apache.hudi.storage.HoodieStorage;
+import org.apache.hudi.storage.StoragePath;
+import org.apache.hudi.storage.StoragePathInfo;
+import org.apache.hudi.table.HoodieSparkTable;
+import org.apache.hudi.table.action.HoodieWriteMetadata;
+
+import org.apache.spark.sql.Dataset;
+import org.apache.spark.sql.Row;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.IOException;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+
+/**
+ * A special-purpose pre-commit validator that moves partition files to a stash
+ * location before a deletePartitions commit completes.
+ *
+ * <p><strong>WARNING: This is NOT a standard validator pattern.</strong>
+ * Unlike typical pre-commit validators that are read-only checks, this validator
+ * performs destructive side-effects (moving/deleting files). It is designed exclusively
+ * for use with the {@code HoodieStashPartitionsTool} and the DELETE_PARTITION operation.
+ * Do NOT use this as a pattern for other validators.</p>
+ *
+ * <h3>Why move files inside a pre-commit validator?</h3>
+ *
+ * <p>The stash operation needs to both move data files to a backup location and mark
+ * the partitions as deleted via a replace commit. These two actions must be coupled:
+ * if the replace commit lands on the timeline but the file move has not happened yet,
+ * the Hudi cleaner will eventually delete the physical data files for the replaced
+ * file groups (based on its retention policy), permanently losing the data that was
+ * intended to be stashed.
+ *
+ * <p>By performing the file move inside this pre-commit validator, the move happens
+ * <b>before</b> the replace commit is finalized. If the move fails, the validator
+ * throws an exception, the commit does not land, and the table remains unchanged.
+ * This guarantees that data is never in a state where it has been marked as deleted
+ * but has not been safely backed up to the stash location.</p>
+ *
+ * <p>This validator requires the following configs to be set:
+ * <ul>
+ * <li>{@link #STASH_PATH_CONFIG} — the target stash directory</li>
+ * <li>{@link #RENAME_HELPER_CLASS_CONFIG} (optional) — custom rename helper class</li>
+ * </ul>
+ *
+ * <p>Guards:
+ * <ul>
+ * <li>Fails if RLI (Record Level Index) is enabled on the table</li>
+ * <li>Must only be used with DELETE_PARTITION operation</li>
+ * </ul>
+ */
+public class StashPartitionsPreCommitValidator<T, I, K, O extends HoodieData<WriteStatus>>
+ extends SparkPreCommitValidator<T, I, K, O> {
+
+ private static final Logger LOG = LoggerFactory.getLogger(StashPartitionsPreCommitValidator.class);
+
+ /**
+ * Config key for the stash target directory path.
+ */
+ public static final String STASH_PATH_CONFIG = "hoodie.stash.partitions.target.path";
+
+ /**
+ * Config key for the custom rename helper class. Defaults to {@link DefaultStashPartitionRenameHelper}.
+ */
+ public static final String RENAME_HELPER_CLASS_CONFIG = "hoodie.stash.rename.helper.class";
+
+ public StashPartitionsPreCommitValidator(HoodieSparkTable<T> table,
+ HoodieEngineContext engineContext,
+ HoodieWriteConfig writeConfig) {
+ super(table, engineContext, writeConfig);
+ }
+
+ /**
+ * Override validate to extract affected partitions from partitionToReplaceFileIds rather than
+ * write stats, since deletePartitions produces empty write stats but populates the replacement map.
+ */
+ @Override
+ public void validate(String instantTime, HoodieWriteMetadata<O> writeResult,
+ Dataset<Row> before, Dataset<Row> after) throws HoodieValidationException {
+ Set<String> partitionsAffected = new HashSet<>(writeResult.getPartitionToReplaceFileIds().keySet());
+ if (partitionsAffected.isEmpty()) {
+ // Fall back to write stats if partitionToReplaceFileIds is not populated
Review Comment:
Should we throw an exception in this case?
##########
hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/client/validator/StashPartitionsPreCommitValidator.java:
##########
@@ -0,0 +1,181 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.hudi.client.validator;
+
+import org.apache.hudi.client.WriteStatus;
+import org.apache.hudi.common.config.HoodieMetadataConfig;
+import org.apache.hudi.common.data.HoodieData;
+import org.apache.hudi.common.engine.HoodieEngineContext;
+import org.apache.hudi.common.util.ReflectionUtils;
+import org.apache.hudi.common.util.StringUtils;
+import org.apache.hudi.common.util.ValidationUtils;
+import org.apache.hudi.config.HoodieWriteConfig;
+import org.apache.hudi.exception.HoodieValidationException;
+import org.apache.hudi.storage.HoodieStorage;
+import org.apache.hudi.storage.StoragePath;
+import org.apache.hudi.storage.StoragePathInfo;
+import org.apache.hudi.table.HoodieSparkTable;
+import org.apache.hudi.table.action.HoodieWriteMetadata;
+
+import org.apache.spark.sql.Dataset;
+import org.apache.spark.sql.Row;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.IOException;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+
+/**
+ * A special-purpose pre-commit validator that moves partition files to a stash
+ * location before a deletePartitions commit completes.
+ *
+ * <p><strong>WARNING: This is NOT a standard validator pattern.</strong>
+ * Unlike typical pre-commit validators that are read-only checks, this validator
+ * performs destructive side-effects (moving/deleting files). It is designed exclusively
+ * for use with the {@code HoodieStashPartitionsTool} and the DELETE_PARTITION operation.
+ * Do NOT use this as a pattern for other validators.</p>
+ *
+ * <h3>Why move files inside a pre-commit validator?</h3>
+ *
+ * <p>The stash operation needs to both move data files to a backup location and mark
+ * the partitions as deleted via a replace commit. These two actions must be coupled:
+ * if the replace commit lands on the timeline but the file move has not happened yet,
+ * the Hudi cleaner will eventually delete the physical data files for the replaced
+ * file groups (based on its retention policy), permanently losing the data that was
+ * intended to be stashed.
+ *
+ * <p>By performing the file move inside this pre-commit validator, the move happens
+ * <b>before</b> the replace commit is finalized. If the move fails, the validator
+ * throws an exception, the commit does not land, and the table remains unchanged.
+ * This guarantees that data is never in a state where it has been marked as deleted
+ * but has not been safely backed up to the stash location.</p>
+ *
+ * <p>This validator requires the following configs to be set:
+ * <ul>
+ * <li>{@link #STASH_PATH_CONFIG} — the target stash directory</li>
+ * <li>{@link #RENAME_HELPER_CLASS_CONFIG} (optional) — custom rename helper class</li>
+ * </ul>
+ *
+ * <p>Guards:
+ * <ul>
+ * <li>Fails if RLI (Record Level Index) is enabled on the table</li>
+ * <li>Must only be used with DELETE_PARTITION operation</li>
+ * </ul>
+ */
+public class StashPartitionsPreCommitValidator<T, I, K, O extends HoodieData<WriteStatus>>
+ extends SparkPreCommitValidator<T, I, K, O> {
+
+ private static final Logger LOG = LoggerFactory.getLogger(StashPartitionsPreCommitValidator.class);
+
+ /**
+ * Config key for the stash target directory path.
+ */
+ public static final String STASH_PATH_CONFIG = "hoodie.stash.partitions.target.path";
+
+ /**
+ * Config key for the custom rename helper class. Defaults to {@link DefaultStashPartitionRenameHelper}.
+ */
+ public static final String RENAME_HELPER_CLASS_CONFIG = "hoodie.stash.rename.helper.class";
+
+ public StashPartitionsPreCommitValidator(HoodieSparkTable<T> table,
+ HoodieEngineContext engineContext,
+ HoodieWriteConfig writeConfig) {
+ super(table, engineContext, writeConfig);
+ }
+
+ /**
+ * Override validate to extract affected partitions from partitionToReplaceFileIds rather than
+ * write stats, since deletePartitions produces empty write stats but populates the replacement map.
+ */
+ @Override
+ public void validate(String instantTime, HoodieWriteMetadata<O> writeResult,
+ Dataset<Row> before, Dataset<Row> after) throws HoodieValidationException {
+ Set<String> partitionsAffected = new HashSet<>(writeResult.getPartitionToReplaceFileIds().keySet());
+ if (partitionsAffected.isEmpty()) {
+ // Fall back to write stats if partitionToReplaceFileIds is not populated
+ partitionsAffected = getPartitionsModified(writeResult);
+ }
+ validateRecordsBeforeAndAfter(before, after, partitionsAffected);
+ }
+
+ @Override
+ protected void validateRecordsBeforeAndAfter(Dataset<Row> before,
+ Dataset<Row> after,
+ Set<String> partitionsAffected) {
+ // Guard: ensure RLI is not enabled
+ HoodieMetadataConfig metadataConfig = getWriteConfig().getMetadataConfig();
Review Comment:
Do we also need to disallow other MDT partitions, like column_stats?
##########
hudi-utilities/src/main/java/org/apache/hudi/utilities/HoodieStashPartitionsTool.java:
##########
@@ -0,0 +1,403 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.hudi.utilities;
+
+import org.apache.hudi.client.HoodieWriteResult;
+import org.apache.hudi.client.SparkRDDWriteClient;
+import org.apache.hudi.client.validator.DefaultStashPartitionRenameHelper;
+import org.apache.hudi.client.validator.StashPartitionRenameHelper;
+import org.apache.hudi.client.validator.StashPartitionsPreCommitValidator;
+import org.apache.hudi.common.config.TypedProperties;
+import org.apache.hudi.common.model.HoodieRecordPayload;
+import org.apache.hudi.common.table.HoodieTableMetaClient;
+import org.apache.hudi.common.table.timeline.HoodieTimeline;
+import org.apache.hudi.common.util.Option;
+import org.apache.hudi.common.util.ReflectionUtils;
+import org.apache.hudi.common.util.StringUtils;
+import org.apache.hudi.config.HoodiePreCommitValidatorConfig;
+import org.apache.hudi.exception.HoodieException;
+import org.apache.hudi.hadoop.fs.HadoopFSUtils;
+import org.apache.hudi.storage.HoodieStorage;
+import org.apache.hudi.storage.HoodieStorageUtils;
+import org.apache.hudi.storage.StoragePath;
+import org.apache.hudi.storage.StoragePathInfo;
+
+import com.beust.jcommander.JCommander;
+import com.beust.jcommander.Parameter;
+import org.apache.hadoop.fs.Path;
+import org.apache.spark.api.java.JavaSparkContext;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.IOException;
+import java.io.Serializable;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+
+/**
+ * A spark-submit tool for stashing (backing up and removing) Hudi table partitions.
+ *
+ * <p>This tool supports three modes:
+ * <ul>
+ * <li><b>stash</b> — Moves data files from specified partitions to a stash location
+ * and issues a deletePartitions operation on the Hudi table. Uses a custom
+ * {@link StashPartitionsPreCommitValidator} to move files atomically before commit.</li>
+ * <li><b>rollback_stash</b> — Copies any data found in the stash location back to the
+ * original partition paths. Used when the user no longer wants to proceed with stashing.</li>
+ * <li><b>dry_run</b> — Shows which partitions and file groups would be affected.</li>
+ * </ul>
+ *
+ * <h3>Why a pre-commit validator instead of moving files in the tool?</h3>
+ *
+ * <p>A naive approach would be to first run {@code deletePartitions} in the tool, and then
+ * move the data files to the stash location as a separate step. However, this is unsafe:
+ * if the tool crashes or fails after {@code deletePartitions} succeeds but before the data
+ * is moved to the stash location, the data is at risk. Once the replace commit from
+ * {@code deletePartitions} lands on the timeline, the Hudi cleaner will eventually delete
+ * the physical data files for the replaced file groups based on its configured retention
+ * policy (e.g., retain last N commits). If the cleaner runs before the tool is retried,
+ * the data intended for stashing is permanently lost.
+ *
+ * <p>By using a {@link StashPartitionsPreCommitValidator}, the file move happens
+ * <b>before</b> the {@code deletePartitions} commit is finalized. If the move fails, the
+ * commit does not land, and the table remains in its original state — the cleaner has
+ * nothing to clean. This ensures that data is never in a state where it has been marked
+ * as deleted (via a replace commit) but has not yet been safely copied to the stash location.
+ *
+ * <p>For restoring stashed partitions back into the Hudi table, use the standard
+ * {@code insert_overwrite} write operation reading from the stash location.
+ *
+ * <p>Example usage:
+ * <pre>
+ * spark-submit \
+ * --class org.apache.hudi.utilities.HoodieStashPartitionsTool \
+ * --master local[*] \
+ * --driver-memory 1g \
+ * --executor-memory 1g \
+ * hudi-utilities-bundle.jar \
+ * --base-path /path/to/hudi/table \
+ * --table-name myTable \
+ * --stash-path /path/to/stash/folder \
+ * --partitions datestr=2023-01-01,datestr=2023-01-02 \
+ * --mode stash
+ * </pre>
+ */
+public class HoodieStashPartitionsTool implements Serializable {
+
+ private static final long serialVersionUID = 1L;
+ private static final Logger LOG = LoggerFactory.getLogger(HoodieStashPartitionsTool.class);
+
+ private final transient JavaSparkContext jsc;
+ private final Config cfg;
+ private final TypedProperties props;
+ private final HoodieTableMetaClient metaClient;
+
+ public HoodieStashPartitionsTool(JavaSparkContext jsc, Config cfg) {
+ this.jsc = jsc;
+ this.cfg = cfg;
+ this.props = cfg.propsFilePath == null
+ ? UtilHelpers.buildProperties(cfg.configs)
+ : readConfigFromFileSystem(jsc, cfg);
+ this.metaClient = HoodieTableMetaClient.builder()
+ .setConf(HadoopFSUtils.getStorageConfWithCopy(jsc.hadoopConfiguration()))
+ .setBasePath(cfg.basePath)
+ .setLoadActiveTimelineOnLoad(true)
+ .build();
+ }
+
+ public enum Mode {
+ STASH,
+ ROLLBACK_STASH,
+ DRY_RUN
+ }
+
+ public static class Config implements Serializable {
+ @Parameter(names = {"--base-path", "-sp"}, description = "Base path for the Hudi table.", required = true)
+ public String basePath = null;
+
+ @Parameter(names = {"--table-name", "-tn"}, description = "Table name.", required = true)
+ public String tableName = null;
+
+ @Parameter(names = {"--stash-path"}, description = "Target path for stashing partition data.", required = true)
+ public String stashPath = null;
+
+ @Parameter(names = {"--partitions", "-p"}, description = "Comma separated list of partitions to stash.", required = true)
+ public String partitions = null;
+
+ @Parameter(names = {"--mode", "-m"}, description = "Set job mode: "
+ + "\"stash\" to move partition data to stash location and delete partitions; "
+ + "\"rollback_stash\" to copy stashed data back to table and abort stashing; "
+ + "\"dry_run\" to preview which partitions and files would be affected.", required = true)
+ public String runningMode = null;
+
+ @Parameter(names = {"--parallelism", "-pl"}, description = "Parallelism for hoodie operations.", required = false)
+ public int parallelism = 1500;
+
+ @Parameter(names = {"--rename-helper-class"}, description = "Fully qualified class name of a custom "
+ + "StashPartitionRenameHelper implementation. Defaults to DefaultStashPartitionRenameHelper.", required = false)
+ public String renameHelperClass = null;
+
+ @Parameter(names = {"--spark-master", "-ms"}, description = "Spark master.", required = false)
+ public String sparkMaster = null;
+
+ @Parameter(names = {"--spark-memory", "-sm"}, description = "Spark memory to use.", required = false)
+ public String sparkMemory = "1g";
+
+ @Parameter(names = {"--props"}, description = "Path to properties file on localfs or dfs, with configurations for "
+ + "hoodie client for stashing partitions.")
+ public String propsFilePath = null;
+
+ @Parameter(names = {"--hoodie-conf"}, description = "Any configuration that can be set in the properties file "
+ + "(using the CLI parameter \"--props\") can also be passed via command line using this parameter. This can be repeated.",
+ splitter = IdentitySplitter.class)
+ public List<String> configs = new ArrayList<>();
+
+ @Parameter(names = {"--help", "-h"}, help = true)
+ public Boolean help = false;
+
+ @Override
+ public String toString() {
+ return "HoodieStashPartitionsToolConfig {\n"
+ + " --base-path " + basePath + ", \n"
+ + " --table-name " + tableName + ", \n"
+ + " --stash-path " + stashPath + ", \n"
+ + " --partitions " + partitions + ", \n"
+ + " --mode " + runningMode + ", \n"
+ + " --parallelism " + parallelism + ", \n"
+ + " --rename-helper-class " + renameHelperClass + ", \n"
+ + " --spark-master " + sparkMaster + ", \n"
+ + " --spark-memory " + sparkMemory + ", \n"
+ + " --props " + propsFilePath + ", \n"
+ + " --hoodie-conf " + configs
+ + "\n}";
+ }
+ }
+
+ public static void main(String[] args) {
+ final Config cfg = new Config();
+ JCommander cmd = new JCommander(cfg, null, args);
+ if (cfg.help || args.length == 0) {
+ cmd.usage();
+ System.exit(1);
+ }
+ Map<String, String> sparkConfigMap = new HashMap<>();
+ sparkConfigMap.put("spark.executor.memory", cfg.sparkMemory);
+ JavaSparkContext jsc = UtilHelpers.buildSparkContext("Hoodie-Stash-Partitions",
+ cfg.sparkMaster, false, sparkConfigMap);
+
+ HoodieStashPartitionsTool tool = new HoodieStashPartitionsTool(jsc, cfg);
+ try {
+ tool.run();
+ } catch (Throwable throwable) {
+ LOG.error("Fail to run stash partitions for " + cfg, throwable);
+ } finally {
+ jsc.stop();
+ }
+ }
+
+ public void run() {
+ try {
+ LOG.info(cfg.toString());
+ Mode mode = Mode.valueOf(cfg.runningMode.toUpperCase());
+ switch (mode) {
+ case STASH:
+ LOG.info(" ****** The Hoodie Stash Partitions Tool is in stash mode ****** ");
+ doStash();
+ break;
+ case ROLLBACK_STASH:
+ LOG.info(" ****** The Hoodie Stash Partitions Tool is in rollback_stash mode ****** ");
+ doRollbackStash();
+ break;
+ case DRY_RUN:
+ LOG.info(" ****** The Hoodie Stash Partitions Tool is in dry-run mode ****** ");
+ doDryRun();
+ break;
+ default:
+ LOG.info("Unsupported running mode [" + cfg.runningMode + "], quit the job directly");
+ }
+ } catch (Exception e) {
+ throw new HoodieException("Unable to stash partitions in " + cfg.basePath, e);
+ }
+ }
+
+ /**
+ * Stash partitions: pre-check to recover any partial stash from a prior failed attempt,
+ * then issue deletePartitions with custom pre-commit validator. Hudi internally handles
+ * rollback of any prior failed instants. The pre-commit validator handles already-stashed
+ * partitions gracefully (source empty -> skip).
+ *
+ * <p>The pre-check runs regardless of whether MDT is enabled. Even with MDT, a prior failed
+ * attempt may have had its pre-commit validator move files to the stash location before
+ * crashing. Those files need to be moved back so that the subsequent deletePartitions
+ * sees the complete partition state.
+ */
+ private void doStash() {
+ List<String> partitionsToStash = Arrays.asList(cfg.partitions.split(","));
+
+ preCheckAndRecoverPartialStash(partitionsToStash);
+
+ // Set up props with stash-specific configs for the pre-commit validator
+ TypedProperties stashProps = new TypedProperties();
+ stashProps.putAll(props);
+ stashProps.setProperty(HoodiePreCommitValidatorConfig.VALIDATOR_CLASS_NAMES.key(),
+ StashPartitionsPreCommitValidator.class.getName());
+ stashProps.setProperty(StashPartitionsPreCommitValidator.STASH_PATH_CONFIG, cfg.stashPath);
+ if (!StringUtils.isNullOrEmpty(cfg.renameHelperClass)) {
+ stashProps.setProperty(StashPartitionsPreCommitValidator.RENAME_HELPER_CLASS_CONFIG, cfg.renameHelperClass);
+ }
+
+ try (SparkRDDWriteClient<HoodieRecordPayload> client =
+ UtilHelpers.createHoodieClient(jsc, cfg.basePath, "", cfg.parallelism, Option.empty(), stashProps)) {
+ String instantTime = client.startCommit(HoodieTimeline.REPLACE_COMMIT_ACTION);
+ LOG.info("Starting deletePartitions with instant {} for partitions: {}", instantTime, partitionsToStash);
+ HoodieWriteResult result = client.deletePartitions(partitionsToStash, instantTime);
+ client.commit(instantTime, result.getWriteStatuses(), Option.empty(), HoodieTimeline.REPLACE_COMMIT_ACTION,
+ result.getPartitionToReplaceFileIds(), Option.empty());
+ LOG.info("Successfully stashed partitions: {}", partitionsToStash);
+ }
+ }
+
+ /**
+ * Pre-check: if files exist in the stash location from a prior failed attempt,
+ * copy them back to source before retrying deletePartitions. This ensures the
+ * subsequent deletePartitions sees the complete partition state and creates
+ * a correct replace commit. Runs for both MDT and non-MDT tables, since the
+ * pre-commit validator may have moved files before crashing.
+ */
+ private void preCheckAndRecoverPartialStash(List<String> partitions) {
+ LOG.info("Running pre-check for partial stash recovery (non-MDT table).");
+ HoodieStorage storage = HoodieStorageUtils.getStorage(
+ cfg.basePath, metaClient.getStorageConf());
+ StashPartitionRenameHelper renameHelper = createRenameHelper();
+ StoragePath basePath = metaClient.getBasePath();
+
+ for (String partition : partitions) {
+ StoragePath stashPartitionPath = new StoragePath(cfg.stashPath, partition);
+ StoragePath sourcePartitionPath = new StoragePath(basePath, partition);
+
+ try {
+ if (!storage.exists(stashPartitionPath) || isDirectoryEmpty(storage, stashPartitionPath)) {
+ LOG.info("Partition {} has no stashed data from prior attempt, skipping pre-check.", partition);
+ continue;
+ }
+ // Files exist in stash — copy them back to source to ensure deletePartitions
+ // sees all files and creates a complete replace commit.
+ LOG.info("Found stashed files for partition {} from prior attempt. Moving back to source.", partition);
+ renameHelper.movePartitionFiles(storage, stashPartitionPath, sourcePartitionPath);
Review Comment:
Based on this, I'm wondering now if we actually want create a different API in the `StashPartitionRenameHelper` from this. This would allow us to create a more optimal (for our production env) implementation for `StashPartitionRenameHelper` that, for HDFS:
- does a `rename` of entire partition for stashing. But for recovery does a `rename` of each file back from stash location to source dataset
##########
hudi-utilities/src/main/java/org/apache/hudi/utilities/HoodieStashPartitionsTool.java:
##########
@@ -0,0 +1,403 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.hudi.utilities;
+
+import org.apache.hudi.client.HoodieWriteResult;
+import org.apache.hudi.client.SparkRDDWriteClient;
+import org.apache.hudi.client.validator.DefaultStashPartitionRenameHelper;
+import org.apache.hudi.client.validator.StashPartitionRenameHelper;
+import org.apache.hudi.client.validator.StashPartitionsPreCommitValidator;
+import org.apache.hudi.common.config.TypedProperties;
+import org.apache.hudi.common.model.HoodieRecordPayload;
+import org.apache.hudi.common.table.HoodieTableMetaClient;
+import org.apache.hudi.common.table.timeline.HoodieTimeline;
+import org.apache.hudi.common.util.Option;
+import org.apache.hudi.common.util.ReflectionUtils;
+import org.apache.hudi.common.util.StringUtils;
+import org.apache.hudi.config.HoodiePreCommitValidatorConfig;
+import org.apache.hudi.exception.HoodieException;
+import org.apache.hudi.hadoop.fs.HadoopFSUtils;
+import org.apache.hudi.storage.HoodieStorage;
+import org.apache.hudi.storage.HoodieStorageUtils;
+import org.apache.hudi.storage.StoragePath;
+import org.apache.hudi.storage.StoragePathInfo;
+
+import com.beust.jcommander.JCommander;
+import com.beust.jcommander.Parameter;
+import org.apache.hadoop.fs.Path;
+import org.apache.spark.api.java.JavaSparkContext;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.IOException;
+import java.io.Serializable;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+
+/**
+ * A spark-submit tool for stashing (backing up and removing) Hudi table partitions.
+ *
+ * <p>This tool supports three modes:
+ * <ul>
+ * <li><b>stash</b> — Moves data files from specified partitions to a stash location
+ * and issues a deletePartitions operation on the Hudi table. Uses a custom
+ * {@link StashPartitionsPreCommitValidator} to move files atomically before commit.</li>
+ * <li><b>rollback_stash</b> — Copies any data found in the stash location back to the
+ * original partition paths. Used when the user no longer wants to proceed with stashing.</li>
+ * <li><b>dry_run</b> — Shows which partitions and file groups would be affected.</li>
+ * </ul>
+ *
+ * <h3>Why a pre-commit validator instead of moving files in the tool?</h3>
+ *
+ * <p>A naive approach would be to first run {@code deletePartitions} in the tool, and then
+ * move the data files to the stash location as a separate step. However, this is unsafe:
+ * if the tool crashes or fails after {@code deletePartitions} succeeds but before the data
+ * is moved to the stash location, the data is at risk. Once the replace commit from
+ * {@code deletePartitions} lands on the timeline, the Hudi cleaner will eventually delete
+ * the physical data files for the replaced file groups based on its configured retention
+ * policy (e.g., retain last N commits). If the cleaner runs before the tool is retried,
+ * the data intended for stashing is permanently lost.
+ *
+ * <p>By using a {@link StashPartitionsPreCommitValidator}, the file move happens
+ * <b>before</b> the {@code deletePartitions} commit is finalized. If the move fails, the
+ * commit does not land, and the table remains in its original state — the cleaner has
+ * nothing to clean. This ensures that data is never in a state where it has been marked
+ * as deleted (via a replace commit) but has not yet been safely copied to the stash location.
+ *
+ * <p>For restoring stashed partitions back into the Hudi table, use the standard
+ * {@code insert_overwrite} write operation reading from the stash location.
+ *
+ * <p>Example usage:
+ * <pre>
+ * spark-submit \
+ * --class org.apache.hudi.utilities.HoodieStashPartitionsTool \
+ * --master local[*] \
+ * --driver-memory 1g \
+ * --executor-memory 1g \
+ * hudi-utilities-bundle.jar \
+ * --base-path /path/to/hudi/table \
+ * --table-name myTable \
+ * --stash-path /path/to/stash/folder \
+ * --partitions datestr=2023-01-01,datestr=2023-01-02 \
+ * --mode stash
+ * </pre>
+ */
+public class HoodieStashPartitionsTool implements Serializable {
+
+ private static final long serialVersionUID = 1L;
+ private static final Logger LOG = LoggerFactory.getLogger(HoodieStashPartitionsTool.class);
+
+ private final transient JavaSparkContext jsc;
+ private final Config cfg;
+ private final TypedProperties props;
+ private final HoodieTableMetaClient metaClient;
+
+ public HoodieStashPartitionsTool(JavaSparkContext jsc, Config cfg) {
+ this.jsc = jsc;
+ this.cfg = cfg;
+ this.props = cfg.propsFilePath == null
+ ? UtilHelpers.buildProperties(cfg.configs)
+ : readConfigFromFileSystem(jsc, cfg);
+ this.metaClient = HoodieTableMetaClient.builder()
+ .setConf(HadoopFSUtils.getStorageConfWithCopy(jsc.hadoopConfiguration()))
+ .setBasePath(cfg.basePath)
+ .setLoadActiveTimelineOnLoad(true)
+ .build();
+ }
+
+ public enum Mode {
+ STASH,
+ ROLLBACK_STASH,
+ DRY_RUN
+ }
+
+ public static class Config implements Serializable {
+ @Parameter(names = {"--base-path", "-sp"}, description = "Base path for the Hudi table.", required = true)
+ public String basePath = null;
+
+ @Parameter(names = {"--table-name", "-tn"}, description = "Table name.", required = true)
+ public String tableName = null;
+
+ @Parameter(names = {"--stash-path"}, description = "Target path for stashing partition data.", required = true)
+ public String stashPath = null;
+
+ @Parameter(names = {"--partitions", "-p"}, description = "Comma separated list of partitions to stash.", required = true)
+ public String partitions = null;
+
+ @Parameter(names = {"--mode", "-m"}, description = "Set job mode: "
+ + "\"stash\" to move partition data to stash location and delete partitions; "
+ + "\"rollback_stash\" to copy stashed data back to table and abort stashing; "
+ + "\"dry_run\" to preview which partitions and files would be affected.", required = true)
+ public String runningMode = null;
+
+ @Parameter(names = {"--parallelism", "-pl"}, description = "Parallelism for hoodie operations.", required = false)
+ public int parallelism = 1500;
+
+ @Parameter(names = {"--rename-helper-class"}, description = "Fully qualified class name of a custom "
+ + "StashPartitionRenameHelper implementation. Defaults to DefaultStashPartitionRenameHelper.", required = false)
+ public String renameHelperClass = null;
+
+ @Parameter(names = {"--spark-master", "-ms"}, description = "Spark master.", required = false)
+ public String sparkMaster = null;
+
+ @Parameter(names = {"--spark-memory", "-sm"}, description = "Spark memory to use.", required = false)
+ public String sparkMemory = "1g";
+
+ @Parameter(names = {"--props"}, description = "Path to properties file on localfs or dfs, with configurations for "
+ + "hoodie client for stashing partitions.")
+ public String propsFilePath = null;
+
+ @Parameter(names = {"--hoodie-conf"}, description = "Any configuration that can be set in the properties file "
+ + "(using the CLI parameter \"--props\") can also be passed via command line using this parameter. This can be repeated.",
+ splitter = IdentitySplitter.class)
+ public List<String> configs = new ArrayList<>();
+
+ @Parameter(names = {"--help", "-h"}, help = true)
+ public Boolean help = false;
+
+ @Override
+ public String toString() {
+ return "HoodieStashPartitionsToolConfig {\n"
+ + " --base-path " + basePath + ", \n"
+ + " --table-name " + tableName + ", \n"
+ + " --stash-path " + stashPath + ", \n"
+ + " --partitions " + partitions + ", \n"
+ + " --mode " + runningMode + ", \n"
+ + " --parallelism " + parallelism + ", \n"
+ + " --rename-helper-class " + renameHelperClass + ", \n"
+ + " --spark-master " + sparkMaster + ", \n"
+ + " --spark-memory " + sparkMemory + ", \n"
+ + " --props " + propsFilePath + ", \n"
+ + " --hoodie-conf " + configs
+ + "\n}";
+ }
+ }
+
+ public static void main(String[] args) {
+ final Config cfg = new Config();
+ JCommander cmd = new JCommander(cfg, null, args);
+ if (cfg.help || args.length == 0) {
+ cmd.usage();
+ System.exit(1);
+ }
+ Map<String, String> sparkConfigMap = new HashMap<>();
+ sparkConfigMap.put("spark.executor.memory", cfg.sparkMemory);
+ JavaSparkContext jsc = UtilHelpers.buildSparkContext("Hoodie-Stash-Partitions",
+ cfg.sparkMaster, false, sparkConfigMap);
+
+ HoodieStashPartitionsTool tool = new HoodieStashPartitionsTool(jsc, cfg);
+ try {
+ tool.run();
+ } catch (Throwable throwable) {
+ LOG.error("Fail to run stash partitions for " + cfg, throwable);
+ } finally {
+ jsc.stop();
+ }
+ }
+
+ public void run() {
+ try {
+ LOG.info(cfg.toString());
+ Mode mode = Mode.valueOf(cfg.runningMode.toUpperCase());
+ switch (mode) {
+ case STASH:
+ LOG.info(" ****** The Hoodie Stash Partitions Tool is in stash mode ****** ");
+ doStash();
+ break;
+ case ROLLBACK_STASH:
+ LOG.info(" ****** The Hoodie Stash Partitions Tool is in rollback_stash mode ****** ");
+ doRollbackStash();
+ break;
+ case DRY_RUN:
+ LOG.info(" ****** The Hoodie Stash Partitions Tool is in dry-run mode ****** ");
+ doDryRun();
+ break;
+ default:
+ LOG.info("Unsupported running mode [" + cfg.runningMode + "], quit the job directly");
+ }
+ } catch (Exception e) {
+ throw new HoodieException("Unable to stash partitions in " + cfg.basePath, e);
+ }
+ }
+
+ /**
+ * Stash partitions: pre-check to recover any partial stash from a prior failed attempt,
+ * then issue deletePartitions with custom pre-commit validator. Hudi internally handles
+ * rollback of any prior failed instants. The pre-commit validator handles already-stashed
+ * partitions gracefully (source empty -> skip).
+ *
+ * <p>The pre-check runs regardless of whether MDT is enabled. Even with MDT, a prior failed
+ * attempt may have had its pre-commit validator move files to the stash location before
+ * crashing. Those files need to be moved back so that the subsequent deletePartitions
+ * sees the complete partition state.
+ */
+ private void doStash() {
+ List<String> partitionsToStash = Arrays.asList(cfg.partitions.split(","));
+
+ preCheckAndRecoverPartialStash(partitionsToStash);
+
+ // Set up props with stash-specific configs for the pre-commit validator
+ TypedProperties stashProps = new TypedProperties();
+ stashProps.putAll(props);
+ stashProps.setProperty(HoodiePreCommitValidatorConfig.VALIDATOR_CLASS_NAMES.key(),
+ StashPartitionsPreCommitValidator.class.getName());
+ stashProps.setProperty(StashPartitionsPreCommitValidator.STASH_PATH_CONFIG, cfg.stashPath);
+ if (!StringUtils.isNullOrEmpty(cfg.renameHelperClass)) {
+ stashProps.setProperty(StashPartitionsPreCommitValidator.RENAME_HELPER_CLASS_CONFIG, cfg.renameHelperClass);
+ }
+
+ try (SparkRDDWriteClient<HoodieRecordPayload> client =
+ UtilHelpers.createHoodieClient(jsc, cfg.basePath, "", cfg.parallelism, Option.empty(), stashProps)) {
+ String instantTime = client.startCommit(HoodieTimeline.REPLACE_COMMIT_ACTION);
+ LOG.info("Starting deletePartitions with instant {} for partitions: {}", instantTime, partitionsToStash);
+ HoodieWriteResult result = client.deletePartitions(partitionsToStash, instantTime);
+ client.commit(instantTime, result.getWriteStatuses(), Option.empty(), HoodieTimeline.REPLACE_COMMIT_ACTION,
+ result.getPartitionToReplaceFileIds(), Option.empty());
+ LOG.info("Successfully stashed partitions: {}", partitionsToStash);
+ }
+ }
+
+ /**
+ * Pre-check: if files exist in the stash location from a prior failed attempt,
+ * copy them back to source before retrying deletePartitions. This ensures the
+ * subsequent deletePartitions sees the complete partition state and creates
+ * a correct replace commit. Runs for both MDT and non-MDT tables, since the
+ * pre-commit validator may have moved files before crashing.
+ */
+ private void preCheckAndRecoverPartialStash(List<String> partitions) {
+ LOG.info("Running pre-check for partial stash recovery (non-MDT table).");
+ HoodieStorage storage = HoodieStorageUtils.getStorage(
+ cfg.basePath, metaClient.getStorageConf());
+ StashPartitionRenameHelper renameHelper = createRenameHelper();
+ StoragePath basePath = metaClient.getBasePath();
+
+ for (String partition : partitions) {
+ StoragePath stashPartitionPath = new StoragePath(cfg.stashPath, partition);
+ StoragePath sourcePartitionPath = new StoragePath(basePath, partition);
+
+ try {
+ if (!storage.exists(stashPartitionPath) || isDirectoryEmpty(storage, stashPartitionPath)) {
+ LOG.info("Partition {} has no stashed data from prior attempt, skipping pre-check.", partition);
+ continue;
+ }
+ // Files exist in stash — copy them back to source to ensure deletePartitions
+ // sees all files and creates a complete replace commit.
+ LOG.info("Found stashed files for partition {} from prior attempt. Moving back to source.", partition);
+ renameHelper.movePartitionFiles(storage, stashPartitionPath, sourcePartitionPath);
+ LOG.info("Recovered partition {} back to source.", partition);
+ } catch (IOException e) {
+ throw new HoodieException("Failed to recover partition " + partition + " from stash during pre-check.", e);
+ }
+ }
+ }
+
+ /**
+ * Rollback stash: copy any data from stash location back to original partition paths.
+ * This is for when the user no longer wants to proceed with stashing.
+ */
+ private void doRollbackStash() {
+ List<String> partitions = Arrays.asList(cfg.partitions.split(","));
+ HoodieStorage storage = HoodieStorageUtils.getStorage(
+ cfg.basePath, metaClient.getStorageConf());
+ StashPartitionRenameHelper renameHelper = createRenameHelper();
+ StoragePath basePath = metaClient.getBasePath();
+
+ for (String partition : partitions) {
+ StoragePath stashPartitionPath = new StoragePath(cfg.stashPath, partition);
+ StoragePath sourcePartitionPath = new StoragePath(basePath, partition);
+
+ try {
+ if (!storage.exists(stashPartitionPath) || isDirectoryEmpty(storage, stashPartitionPath)) {
+ LOG.info("No stashed data found for partition {}, skipping.", partition);
+ continue;
+ }
+ LOG.info("Rolling back stash for partition {}: moving from {} to {}", partition, stashPartitionPath, sourcePartitionPath);
+ renameHelper.movePartitionFiles(storage, stashPartitionPath, sourcePartitionPath);
+ LOG.info("Successfully rolled back stash for partition {}.", partition);
+ } catch (IOException e) {
+ throw new HoodieException("Failed to rollback stash for partition: " + partition, e);
+ }
+ }
+ LOG.info("Stash rollback complete.");
+ }
+
+ /**
+ * Dry run: show partitions and file groups that would be affected.
+ */
+ private void doDryRun() {
+ try (SparkRDDWriteClient<HoodieRecordPayload> client =
+ UtilHelpers.createHoodieClient(jsc, cfg.basePath, "", cfg.parallelism, Option.empty(), props)) {
+ org.apache.hudi.table.HoodieSparkTable<HoodieRecordPayload> table =
+ org.apache.hudi.table.HoodieSparkTable.create(client.getConfig(), client.getEngineContext());
+ List<String> parts = Arrays.asList(cfg.partitions.split(","));
+ HoodieStorage storage = HoodieStorageUtils.getStorage(
+ cfg.basePath, metaClient.getStorageConf());
+ StoragePath basePath = metaClient.getBasePath();
+
+ LOG.info("=== Dry Run: Partitions and files that would be stashed ===");
+ for (String partition : parts) {
+ StoragePath sourcePartitionPath = new StoragePath(basePath, partition);
+ StoragePath stashPartitionPath = new StoragePath(cfg.stashPath, partition);
+
+ List<String> fileIds = table.getSliceView()
+ .getLatestFileSlices(partition)
+ .map(fs -> fs.getFileId())
+ .distinct()
+ .collect(Collectors.toList());
+
+ boolean sourceExists;
+ boolean stashExists;
+ try {
+ sourceExists = storage.exists(sourcePartitionPath) && !isDirectoryEmpty(storage, sourcePartitionPath);
+ stashExists = storage.exists(stashPartitionPath) && !isDirectoryEmpty(storage, stashPartitionPath);
+ } catch (IOException e) {
+ throw new HoodieException("Failed to check paths for partition: " + partition, e);
+ }
+
+ LOG.info("Partition: {}", partition);
+ LOG.info(" Source path: {} (exists={})", sourcePartitionPath, sourceExists);
+ LOG.info(" Stash path: {} (exists={})", stashPartitionPath, stashExists);
+ LOG.info(" File group IDs: {}", fileIds);
+ }
+ }
+ }
+
+ private boolean isDirectoryEmpty(HoodieStorage storage, StoragePath path) throws IOException {
+ List<StoragePathInfo> entries = storage.listDirectEntries(path);
+ return entries.isEmpty();
+ }
+
+ private StashPartitionRenameHelper createRenameHelper() {
Review Comment:
can we re-use same helper in StashPartitionsPreCommitValidator ?
##########
hudi-utilities/src/main/java/org/apache/hudi/utilities/HoodieStashPartitionsTool.java:
##########
@@ -0,0 +1,403 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.hudi.utilities;
+
+import org.apache.hudi.client.HoodieWriteResult;
+import org.apache.hudi.client.SparkRDDWriteClient;
+import org.apache.hudi.client.validator.DefaultStashPartitionRenameHelper;
+import org.apache.hudi.client.validator.StashPartitionRenameHelper;
+import org.apache.hudi.client.validator.StashPartitionsPreCommitValidator;
+import org.apache.hudi.common.config.TypedProperties;
+import org.apache.hudi.common.model.HoodieRecordPayload;
+import org.apache.hudi.common.table.HoodieTableMetaClient;
+import org.apache.hudi.common.table.timeline.HoodieTimeline;
+import org.apache.hudi.common.util.Option;
+import org.apache.hudi.common.util.ReflectionUtils;
+import org.apache.hudi.common.util.StringUtils;
+import org.apache.hudi.config.HoodiePreCommitValidatorConfig;
+import org.apache.hudi.exception.HoodieException;
+import org.apache.hudi.hadoop.fs.HadoopFSUtils;
+import org.apache.hudi.storage.HoodieStorage;
+import org.apache.hudi.storage.HoodieStorageUtils;
+import org.apache.hudi.storage.StoragePath;
+import org.apache.hudi.storage.StoragePathInfo;
+
+import com.beust.jcommander.JCommander;
+import com.beust.jcommander.Parameter;
+import org.apache.hadoop.fs.Path;
+import org.apache.spark.api.java.JavaSparkContext;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.IOException;
+import java.io.Serializable;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+
+/**
+ * A spark-submit tool for stashing (backing up and removing) Hudi table partitions.
+ *
+ * <p>This tool supports three modes:
+ * <ul>
+ * <li><b>stash</b> — Moves data files from specified partitions to a stash location
+ * and issues a deletePartitions operation on the Hudi table. Uses a custom
+ * {@link StashPartitionsPreCommitValidator} to move files atomically before commit.</li>
+ * <li><b>rollback_stash</b> — Copies any data found in the stash location back to the
+ * original partition paths. Used when the user no longer wants to proceed with stashing.</li>
+ * <li><b>dry_run</b> — Shows which partitions and file groups would be affected.</li>
+ * </ul>
+ *
+ * <h3>Why a pre-commit validator instead of moving files in the tool?</h3>
+ *
+ * <p>A naive approach would be to first run {@code deletePartitions} in the tool, and then
+ * move the data files to the stash location as a separate step. However, this is unsafe:
+ * if the tool crashes or fails after {@code deletePartitions} succeeds but before the data
+ * is moved to the stash location, the data is at risk. Once the replace commit from
+ * {@code deletePartitions} lands on the timeline, the Hudi cleaner will eventually delete
+ * the physical data files for the replaced file groups based on its configured retention
+ * policy (e.g., retain last N commits). If the cleaner runs before the tool is retried,
+ * the data intended for stashing is permanently lost.
+ *
+ * <p>By using a {@link StashPartitionsPreCommitValidator}, the file move happens
+ * <b>before</b> the {@code deletePartitions} commit is finalized. If the move fails, the
+ * commit does not land, and the table remains in its original state — the cleaner has
+ * nothing to clean. This ensures that data is never in a state where it has been marked
+ * as deleted (via a replace commit) but has not yet been safely copied to the stash location.
+ *
+ * <p>For restoring stashed partitions back into the Hudi table, use the standard
+ * {@code insert_overwrite} write operation reading from the stash location.
+ *
+ * <p>Example usage:
+ * <pre>
+ * spark-submit \
+ * --class org.apache.hudi.utilities.HoodieStashPartitionsTool \
+ * --master local[*] \
+ * --driver-memory 1g \
+ * --executor-memory 1g \
+ * hudi-utilities-bundle.jar \
+ * --base-path /path/to/hudi/table \
+ * --table-name myTable \
+ * --stash-path /path/to/stash/folder \
+ * --partitions datestr=2023-01-01,datestr=2023-01-02 \
+ * --mode stash
+ * </pre>
+ */
+public class HoodieStashPartitionsTool implements Serializable {
+
+ private static final long serialVersionUID = 1L;
+ private static final Logger LOG = LoggerFactory.getLogger(HoodieStashPartitionsTool.class);
+
+ private final transient JavaSparkContext jsc;
+ private final Config cfg;
+ private final TypedProperties props;
+ private final HoodieTableMetaClient metaClient;
+
+ public HoodieStashPartitionsTool(JavaSparkContext jsc, Config cfg) {
+ this.jsc = jsc;
+ this.cfg = cfg;
+ this.props = cfg.propsFilePath == null
+ ? UtilHelpers.buildProperties(cfg.configs)
+ : readConfigFromFileSystem(jsc, cfg);
+ this.metaClient = HoodieTableMetaClient.builder()
+ .setConf(HadoopFSUtils.getStorageConfWithCopy(jsc.hadoopConfiguration()))
+ .setBasePath(cfg.basePath)
+ .setLoadActiveTimelineOnLoad(true)
+ .build();
+ }
+
+ public enum Mode {
+ STASH,
+ ROLLBACK_STASH,
+ DRY_RUN
+ }
+
+ public static class Config implements Serializable {
+ @Parameter(names = {"--base-path", "-sp"}, description = "Base path for the Hudi table.", required = true)
+ public String basePath = null;
+
+ @Parameter(names = {"--table-name", "-tn"}, description = "Table name.", required = true)
+ public String tableName = null;
+
+ @Parameter(names = {"--stash-path"}, description = "Target path for stashing partition data.", required = true)
+ public String stashPath = null;
+
+ @Parameter(names = {"--partitions", "-p"}, description = "Comma separated list of partitions to stash.", required = true)
+ public String partitions = null;
+
+ @Parameter(names = {"--mode", "-m"}, description = "Set job mode: "
+ + "\"stash\" to move partition data to stash location and delete partitions; "
+ + "\"rollback_stash\" to copy stashed data back to table and abort stashing; "
+ + "\"dry_run\" to preview which partitions and files would be affected.", required = true)
+ public String runningMode = null;
+
+ @Parameter(names = {"--parallelism", "-pl"}, description = "Parallelism for hoodie operations.", required = false)
+ public int parallelism = 1500;
+
+ @Parameter(names = {"--rename-helper-class"}, description = "Fully qualified class name of a custom "
+ + "StashPartitionRenameHelper implementation. Defaults to DefaultStashPartitionRenameHelper.", required = false)
+ public String renameHelperClass = null;
+
+ @Parameter(names = {"--spark-master", "-ms"}, description = "Spark master.", required = false)
+ public String sparkMaster = null;
+
+ @Parameter(names = {"--spark-memory", "-sm"}, description = "Spark memory to use.", required = false)
+ public String sparkMemory = "1g";
+
+ @Parameter(names = {"--props"}, description = "Path to properties file on localfs or dfs, with configurations for "
+ + "hoodie client for stashing partitions.")
+ public String propsFilePath = null;
+
+ @Parameter(names = {"--hoodie-conf"}, description = "Any configuration that can be set in the properties file "
+ + "(using the CLI parameter \"--props\") can also be passed via command line using this parameter. This can be repeated.",
+ splitter = IdentitySplitter.class)
+ public List<String> configs = new ArrayList<>();
+
+ @Parameter(names = {"--help", "-h"}, help = true)
+ public Boolean help = false;
+
+ @Override
+ public String toString() {
+ return "HoodieStashPartitionsToolConfig {\n"
+ + " --base-path " + basePath + ", \n"
+ + " --table-name " + tableName + ", \n"
+ + " --stash-path " + stashPath + ", \n"
+ + " --partitions " + partitions + ", \n"
+ + " --mode " + runningMode + ", \n"
+ + " --parallelism " + parallelism + ", \n"
+ + " --rename-helper-class " + renameHelperClass + ", \n"
+ + " --spark-master " + sparkMaster + ", \n"
+ + " --spark-memory " + sparkMemory + ", \n"
+ + " --props " + propsFilePath + ", \n"
+ + " --hoodie-conf " + configs
+ + "\n}";
+ }
+ }
+
+ public static void main(String[] args) {
+ final Config cfg = new Config();
+ JCommander cmd = new JCommander(cfg, null, args);
+ if (cfg.help || args.length == 0) {
+ cmd.usage();
+ System.exit(1);
+ }
+ Map<String, String> sparkConfigMap = new HashMap<>();
+ sparkConfigMap.put("spark.executor.memory", cfg.sparkMemory);
+ JavaSparkContext jsc = UtilHelpers.buildSparkContext("Hoodie-Stash-Partitions",
+ cfg.sparkMaster, false, sparkConfigMap);
+
+ HoodieStashPartitionsTool tool = new HoodieStashPartitionsTool(jsc, cfg);
+ try {
+ tool.run();
+ } catch (Throwable throwable) {
+ LOG.error("Fail to run stash partitions for " + cfg, throwable);
+ } finally {
+ jsc.stop();
+ }
+ }
+
+ public void run() {
+ try {
+ LOG.info(cfg.toString());
+ Mode mode = Mode.valueOf(cfg.runningMode.toUpperCase());
+ switch (mode) {
+ case STASH:
+ LOG.info(" ****** The Hoodie Stash Partitions Tool is in stash mode ****** ");
+ doStash();
+ break;
+ case ROLLBACK_STASH:
+ LOG.info(" ****** The Hoodie Stash Partitions Tool is in rollback_stash mode ****** ");
+ doRollbackStash();
+ break;
+ case DRY_RUN:
+ LOG.info(" ****** The Hoodie Stash Partitions Tool is in dry-run mode ****** ");
+ doDryRun();
+ break;
+ default:
+ LOG.info("Unsupported running mode [" + cfg.runningMode + "], quit the job directly");
+ }
+ } catch (Exception e) {
+ throw new HoodieException("Unable to stash partitions in " + cfg.basePath, e);
+ }
+ }
+
+ /**
+ * Stash partitions: pre-check to recover any partial stash from a prior failed attempt,
+ * then issue deletePartitions with custom pre-commit validator. Hudi internally handles
+ * rollback of any prior failed instants. The pre-commit validator handles already-stashed
+ * partitions gracefully (source empty -> skip).
+ *
+ * <p>The pre-check runs regardless of whether MDT is enabled. Even with MDT, a prior failed
+ * attempt may have had its pre-commit validator move files to the stash location before
+ * crashing. Those files need to be moved back so that the subsequent deletePartitions
+ * sees the complete partition state.
+ */
+ private void doStash() {
+ List<String> partitionsToStash = Arrays.asList(cfg.partitions.split(","));
+
+ preCheckAndRecoverPartialStash(partitionsToStash);
+
+ // Set up props with stash-specific configs for the pre-commit validator
+ TypedProperties stashProps = new TypedProperties();
+ stashProps.putAll(props);
+ stashProps.setProperty(HoodiePreCommitValidatorConfig.VALIDATOR_CLASS_NAMES.key(),
+ StashPartitionsPreCommitValidator.class.getName());
+ stashProps.setProperty(StashPartitionsPreCommitValidator.STASH_PATH_CONFIG, cfg.stashPath);
+ if (!StringUtils.isNullOrEmpty(cfg.renameHelperClass)) {
+ stashProps.setProperty(StashPartitionsPreCommitValidator.RENAME_HELPER_CLASS_CONFIG, cfg.renameHelperClass);
+ }
+
+ try (SparkRDDWriteClient<HoodieRecordPayload> client =
+ UtilHelpers.createHoodieClient(jsc, cfg.basePath, "", cfg.parallelism, Option.empty(), stashProps)) {
+ String instantTime = client.startCommit(HoodieTimeline.REPLACE_COMMIT_ACTION);
+ LOG.info("Starting deletePartitions with instant {} for partitions: {}", instantTime, partitionsToStash);
+ HoodieWriteResult result = client.deletePartitions(partitionsToStash, instantTime);
+ client.commit(instantTime, result.getWriteStatuses(), Option.empty(), HoodieTimeline.REPLACE_COMMIT_ACTION,
+ result.getPartitionToReplaceFileIds(), Option.empty());
+ LOG.info("Successfully stashed partitions: {}", partitionsToStash);
+ }
+ }
+
+ /**
+ * Pre-check: if files exist in the stash location from a prior failed attempt,
+ * copy them back to source before retrying deletePartitions. This ensures the
+ * subsequent deletePartitions sees the complete partition state and creates
+ * a correct replace commit. Runs for both MDT and non-MDT tables, since the
+ * pre-commit validator may have moved files before crashing.
+ */
+ private void preCheckAndRecoverPartialStash(List<String> partitions) {
Review Comment:
Whats the difference between this function and doRollbackStash ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-5159914729
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `80.83832%` with `64 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 68.68%. Comparing base ([`7bcb8be`](https://app.codecov.io/gh/apache/hudi/commit/7bcb8be7c153010998574f591c7ff3799192eda6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`6d8dedd`](https://app.codecov.io/gh/apache/hudi/commit/6d8dedd061ed183c52471df09355b85e5a60099a?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 444 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | 79.84% | [43 Missing and 10 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | 77.50% | [5 Missing and 4 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | 93.54% | [1 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18526 +/- ##
============================================
- Coverage 68.88% 68.68% -0.21%
- Complexity 28279 28473 +194
============================================
Files 2464 2478 +14
Lines 135568 136972 +1404
Branches 16446 16653 +207
============================================
+ Hits 93385 94077 +692
- Misses 34801 35324 +523
- Partials 7382 7571 +189
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.39% <13.77%> (-0.25%)` | :arrow_down: |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.74% <0.00%> (-0.02%)` | :arrow_down: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.64% <77.46%> (+0.23%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `49.25% <0.00%> (+0.32%)` | :arrow_up: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.18% <0.00%> (-0.26%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `37.87% <77.54%> (-0.30%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | `93.54% <93.54%> (ø)` | |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | `77.50% <77.50%> (ø)` | |
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | `79.84% <79.84%> (ø)` | |
... and [79 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18526/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
- :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(utilities): Add HoodieStashPartitionsTool for backing up and removing partitions [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18526:
URL: https://github.com/apache/hudi/pull/18526#issuecomment-4523087719
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `80.83832%` with `64 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 68.68%. Comparing base ([`7bcb8be`](https://app.codecov.io/gh/apache/hudi/commit/7bcb8be7c153010998574f591c7ff3799192eda6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`6d8dedd`](https://app.codecov.io/gh/apache/hudi/commit/6d8dedd061ed183c52471df09355b85e5a60099a?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 132 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | 79.84% | [43 Missing and 10 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | 77.50% | [5 Missing and 4 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | 93.54% | [1 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18526 +/- ##
============================================
- Coverage 68.88% 68.68% -0.21%
- Complexity 28279 28473 +194
============================================
Files 2464 2478 +14
Lines 135568 136972 +1404
Branches 16446 16653 +207
============================================
+ Hits 93385 94077 +692
- Misses 34801 35324 +523
- Partials 7382 7571 +189
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.39% <13.77%> (-0.25%)` | :arrow_down: |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.74% <0.00%> (-0.02%)` | :arrow_down: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.64% <77.46%> (+0.23%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `49.25% <0.00%> (+0.32%)` | :arrow_up: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.18% <0.00%> (-0.26%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18526/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `37.87% <77.54%> (-0.30%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18526?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...t/validator/DefaultStashPartitionRenameHelper.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FDefaultStashPartitionRenameHelper.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC92YWxpZGF0b3IvRGVmYXVsdFN0YXNoUGFydGl0aW9uUmVuYW1lSGVscGVyLmphdmE=) | `93.54% <93.54%> (ø)` | |
| [...t/validator/StashPartitionsPreCommitValidator.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2Fvalidator%2FStashPartitionsPreCommitValidator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L3ZhbGlkYXRvci9TdGFzaFBhcnRpdGlvbnNQcmVDb21taXRWYWxpZGF0b3IuamF2YQ==) | `77.50% <77.50%> (ø)` | |
| [...ache/hudi/utilities/HoodieStashPartitionsTool.java](https://app.codecov.io/gh/apache/hudi/pull/18526?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2FHoodieStashPartitionsTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL0hvb2RpZVN0YXNoUGFydGl0aW9uc1Rvb2wuamF2YQ==) | `79.84% <79.84%> (ø)` | |
... and [79 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18526/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org