You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "prashantwason (via GitHub)" <gi...@apache.org> on 2026/03/29 01:10:04 UTC
[PR] [HUDI-18410] Fix Flink upgrade blocking dispatcher thread causing heartbeat timeout [hudi]
prashantwason opened a new pull request, #18411:
URL: https://github.com/apache/hudi/pull/18411
## Summary
- Move table version upgrade (`tryUpgrade()`, `initMetadataTable()`, `restoreEvents()`) from the Pekko dispatcher thread to the coordinator's single-threaded FIFO executor in `StreamWriteOperatorCoordinator.start()`, preventing heartbeat timeout during long-running upgrades
- Optimize `SevenToEightUpgradeHandler.upgradeToLSMTimeline()` to use a larger batch size (500 vs default 10) and call `compactAndClean()` once at the end instead of after every batch
- Fixes https://github.com/apache/hudi/issues/18410
## Problem
When upgrading a Hudi table with many archived timeline actions (e.g., v7→v8 LSM timeline migration), the upgrade runs synchronously on the Pekko dispatcher thread in `StreamWriteOperatorCoordinator.start()`. Each batch of 10 actions triggers ~5 remote storage operations (parquet write, manifest update, compaction), and with hundreds of actions, the dispatcher thread is blocked for 90+ seconds. This prevents heartbeat responses, causing the ResourceManager to disconnect the JobManager.
## Solution
1. **Threading fix**: Create the `NonThrownExecutor` before the upgrade and submit the heavy initialization as the first FIFO task. Since all event handling also goes through this executor, the upgrade is guaranteed to complete before any events are processed. The Pekko dispatcher thread returns immediately, allowing heartbeats to flow.
2. **I/O optimization**: Use batch size of 500 (vs default 10) and single `compactAndClean()` at end, reducing remote storage operations from ~250 to ~6.
## Test plan
- [x] `TestStreamWriteOperatorCoordinator` — all 36 tests pass
- [x] Verified `setExecutor()` in test helper calls `executor.close()` which waits for task completion (`waitForTasksFinish=true`), so the upgrade task completes before the mock executor replaces it
- [x] Confirmed the fix is needed on apache/master — identical vulnerable code present
🤖 Generated with [Claude Code](https://claude.com/claude-code)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-18410] Fix Flink upgrade blocking dispatcher thread causing heartbeat timeout [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18411:
URL: https://github.com/apache/hudi/pull/18411#issuecomment-4149216671
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `66.66667%` with `1 line` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 48.05%. Comparing base ([`1eb97b3`](https://app.codecov.io/gh/apache/hudi/commit/1eb97b31826eabee4fb9c4bf78dc44d2bdb2dad0?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`10ccb3d`](https://app.codecov.io/gh/apache/hudi/commit/10ccb3d818b0e218b08db4e0777b531e494a9708?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...hudi/table/upgrade/SevenToEightUpgradeHandler.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Ftable%2Fupgrade%2FSevenToEightUpgradeHandler.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL3RhYmxlL3VwZ3JhZGUvU2V2ZW5Ub0VpZ2h0VXBncmFkZUhhbmRsZXIuamF2YQ==) | 66.66% | [1 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
> :exclamation: There is a different number of reports uploaded between BASE (1eb97b3) and HEAD (10ccb3d). Click for more details.
>
> <details><summary>HEAD has 20 uploads less than BASE</summary>
>
>| Flag | BASE (1eb97b3) | HEAD (10ccb3d) |
>|------|------|------|
>|spark-scala-tests|10|5|
>|spark-java-tests|15|2|
>|common-and-other-modules|1|0|
>|utilities|1|0|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18411 +/- ##
=============================================
- Coverage 68.21% 48.05% -20.16%
+ Complexity 27709 16316 -11393
=============================================
Files 2440 1996 -444
Lines 134249 111058 -23191
Branches 16179 14012 -2167
=============================================
- Hits 91578 53369 -38209
- Misses 35565 52126 +16561
+ Partials 7106 5563 -1543
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.98% <66.66%> (+0.05%)` | :arrow_up: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.32% <0.00%> (+<0.01%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `33.31% <66.66%> (-15.40%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `32.59% <0.00%> (-12.65%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...hudi/table/upgrade/SevenToEightUpgradeHandler.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Ftable%2Fupgrade%2FSevenToEightUpgradeHandler.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL3RhYmxlL3VwZ3JhZGUvU2V2ZW5Ub0VpZ2h0VXBncmFkZUhhbmRsZXIuamF2YQ==) | `62.14% <66.66%> (-25.19%)` | :arrow_down: |
... and [1511 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18411/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-18410] Optimize LSM timeline migration batch size and compaction during upgrade [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18411:
URL: https://github.com/apache/hudi/pull/18411#issuecomment-4157466133
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `66.66667%` with `1 line` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 48.21%. Comparing base ([`1eb97b3`](https://app.codecov.io/gh/apache/hudi/commit/1eb97b31826eabee4fb9c4bf78dc44d2bdb2dad0?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`cd5f54b`](https://app.codecov.io/gh/apache/hudi/commit/cd5f54b2de81575b98c515152c82fd71f950b2b4?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 6 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...hudi/table/upgrade/SevenToEightUpgradeHandler.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Ftable%2Fupgrade%2FSevenToEightUpgradeHandler.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL3RhYmxlL3VwZ3JhZGUvU2V2ZW5Ub0VpZ2h0VXBncmFkZUhhbmRsZXIuamF2YQ==) | 66.66% | [1 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
> :exclamation: There is a different number of reports uploaded between BASE (1eb97b3) and HEAD (cd5f54b). Click for more details.
>
> <details><summary>HEAD has 18 uploads less than BASE</summary>
>
>| Flag | BASE (1eb97b3) | HEAD (cd5f54b) |
>|------|------|------|
>|spark-scala-tests|10|5|
>|spark-java-tests|15|4|
>|common-and-other-modules|1|0|
>|utilities|1|0|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18411 +/- ##
=============================================
- Coverage 68.21% 48.21% -20.00%
+ Complexity 27709 16379 -11330
=============================================
Files 2440 1995 -445
Lines 134249 111150 -23099
Branches 16179 14042 -2137
=============================================
- Hits 91578 53593 -37985
- Misses 35565 51967 +16402
+ Partials 7106 5590 -1516
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.97% <66.66%> (+0.04%)` | :arrow_up: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.40% <0.00%> (+0.08%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `32.48% <66.66%> (-16.23%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `32.69% <0.00%> (-12.55%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...hudi/table/upgrade/SevenToEightUpgradeHandler.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Ftable%2Fupgrade%2FSevenToEightUpgradeHandler.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL3RhYmxlL3VwZ3JhZGUvU2V2ZW5Ub0VpZ2h0VXBncmFkZUhhbmRsZXIuamF2YQ==) | `62.14% <66.66%> (-25.19%)` | :arrow_down: |
... and [1511 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18411/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-18410] Optimize LSM timeline migration batch size and compaction during upgrade [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18411:
URL: https://github.com/apache/hudi/pull/18411#issuecomment-4157581469
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `87.50000%` with `3 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 68.25%. Comparing base ([`1eb97b3`](https://app.codecov.io/gh/apache/hudi/commit/1eb97b31826eabee4fb9c4bf78dc44d2bdb2dad0?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`cd5f54b`](https://app.codecov.io/gh/apache/hudi/commit/cd5f54b2de81575b98c515152c82fd71f950b2b4?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 6 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/sink/StreamWriteOperatorCoordinator.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-flink-datasource%2Fhudi-flink%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fsink%2FStreamWriteOperatorCoordinator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1mbGluay1kYXRhc291cmNlL2h1ZGktZmxpbmsvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvc2luay9TdHJlYW1Xcml0ZU9wZXJhdG9yQ29vcmRpbmF0b3IuamF2YQ==) | 90.47% | [1 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...hudi/table/upgrade/SevenToEightUpgradeHandler.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Ftable%2Fupgrade%2FSevenToEightUpgradeHandler.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL3RhYmxlL3VwZ3JhZGUvU2V2ZW5Ub0VpZ2h0VXBncmFkZUhhbmRsZXIuamF2YQ==) | 66.66% | [0 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18411 +/- ##
============================================
+ Coverage 68.21% 68.25% +0.04%
- Complexity 27709 27760 +51
============================================
Files 2440 2439 -1
Lines 134249 134358 +109
Branches 16179 16210 +31
============================================
+ Hits 91578 91708 +130
+ Misses 35565 35549 -16
+ Partials 7106 7101 -5
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.35% <87.50%> (+0.03%)` | :arrow_up: |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.97% <66.66%> (+0.04%)` | :arrow_up: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.40% <0.00%> (+0.08%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.71% <66.66%> (+<0.01%)` | :arrow_up: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.29% <0.00%> (+0.06%)` | :arrow_up: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `38.37% <66.66%> (+<0.01%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...hudi/table/upgrade/SevenToEightUpgradeHandler.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Ftable%2Fupgrade%2FSevenToEightUpgradeHandler.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL3RhYmxlL3VwZ3JhZGUvU2V2ZW5Ub0VpZ2h0VXBncmFkZUhhbmRsZXIuamF2YQ==) | `85.00% <66.66%> (-2.33%)` | :arrow_down: |
| [...ache/hudi/sink/StreamWriteOperatorCoordinator.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-flink-datasource%2Fhudi-flink%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fsink%2FStreamWriteOperatorCoordinator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1mbGluay1kYXRhc291cmNlL2h1ZGktZmxpbmsvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvc2luay9TdHJlYW1Xcml0ZU9wZXJhdG9yQ29vcmRpbmF0b3IuamF2YQ==) | `81.38% <90.47%> (+0.34%)` | :arrow_up: |
... and [43 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18411/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-18410] Optimize LSM timeline migration batch size and compaction during upgrade [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18411:
URL: https://github.com/apache/hudi/pull/18411#issuecomment-4157405227
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `66.66667%` with `1 line` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 53.95%. Comparing base ([`1eb97b3`](https://app.codecov.io/gh/apache/hudi/commit/1eb97b31826eabee4fb9c4bf78dc44d2bdb2dad0?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`cd5f54b`](https://app.codecov.io/gh/apache/hudi/commit/cd5f54b2de81575b98c515152c82fd71f950b2b4?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 6 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...hudi/table/upgrade/SevenToEightUpgradeHandler.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Ftable%2Fupgrade%2FSevenToEightUpgradeHandler.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL3RhYmxlL3VwZ3JhZGUvU2V2ZW5Ub0VpZ2h0VXBncmFkZUhhbmRsZXIuamF2YQ==) | 66.66% | [1 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
> :exclamation: There is a different number of reports uploaded between BASE (1eb97b3) and HEAD (cd5f54b). Click for more details.
>
> <details><summary>HEAD has 27 uploads less than BASE</summary>
>
>| Flag | BASE (1eb97b3) | HEAD (cd5f54b) |
>|------|------|------|
>|spark-scala-tests|10|0|
>|spark-java-tests|15|0|
>|common-and-other-modules|1|0|
>|utilities|1|0|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18411 +/- ##
=============================================
- Coverage 68.21% 53.95% -14.26%
+ Complexity 27709 12193 -15516
=============================================
Files 2440 1423 -1017
Lines 134249 70499 -63750
Branches 16179 7920 -8259
=============================================
- Hits 91578 38041 -53537
+ Misses 35565 29053 -6512
+ Partials 7106 3405 -3701
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.97% <66.66%> (+0.04%)` | :arrow_up: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.40% <0.00%> (+0.08%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...hudi/table/upgrade/SevenToEightUpgradeHandler.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Ftable%2Fupgrade%2FSevenToEightUpgradeHandler.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL3RhYmxlL3VwZ3JhZGUvU2V2ZW5Ub0VpZ2h0VXBncmFkZUhhbmRsZXIuamF2YQ==) | `57.85% <66.66%> (-29.47%)` | :arrow_down: |
... and [1791 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18411/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-18410] Fix Flink upgrade blocking dispatcher thread causing heartbeat timeout [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18411:
URL: https://github.com/apache/hudi/pull/18411#issuecomment-4149234125
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `66.66667%` with `1 line` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 64.79%. Comparing base ([`1eb97b3`](https://app.codecov.io/gh/apache/hudi/commit/1eb97b31826eabee4fb9c4bf78dc44d2bdb2dad0?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`10ccb3d`](https://app.codecov.io/gh/apache/hudi/commit/10ccb3d818b0e218b08db4e0777b531e494a9708?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...hudi/table/upgrade/SevenToEightUpgradeHandler.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Ftable%2Fupgrade%2FSevenToEightUpgradeHandler.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL3RhYmxlL3VwZ3JhZGUvU2V2ZW5Ub0VpZ2h0VXBncmFkZUhhbmRsZXIuamF2YQ==) | 66.66% | [0 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18411 +/- ##
============================================
- Coverage 68.21% 64.79% -3.43%
+ Complexity 27709 22046 -5663
============================================
Files 2440 1996 -444
Lines 134249 111058 -23191
Branches 16179 14012 -2167
============================================
- Hits 91578 71960 -19618
+ Misses 35565 32781 -2784
+ Partials 7106 6317 -789
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.98% <66.66%> (+0.05%)` | :arrow_up: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.32% <0.00%> (+<0.01%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.95% <66.66%> (-3.76%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.24% <0.00%> (+0.01%)` | :arrow_up: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `38.38% <66.66%> (+<0.01%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...hudi/table/upgrade/SevenToEightUpgradeHandler.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Ftable%2Fupgrade%2FSevenToEightUpgradeHandler.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL3RhYmxlL3VwZ3JhZGUvU2V2ZW5Ub0VpZ2h0VXBncmFkZUhhbmRsZXIuamF2YQ==) | `77.85% <66.66%> (-9.47%)` | :arrow_down: |
... and [916 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18411/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-18410] Fix Flink upgrade blocking dispatcher thread causing heartbeat timeout [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18411:
URL: https://github.com/apache/hudi/pull/18411#issuecomment-4149220997
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `66.66667%` with `1 line` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 54.44%. Comparing base ([`1eb97b3`](https://app.codecov.io/gh/apache/hudi/commit/1eb97b31826eabee4fb9c4bf78dc44d2bdb2dad0?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`10ccb3d`](https://app.codecov.io/gh/apache/hudi/commit/10ccb3d818b0e218b08db4e0777b531e494a9708?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...hudi/table/upgrade/SevenToEightUpgradeHandler.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Ftable%2Fupgrade%2FSevenToEightUpgradeHandler.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL3RhYmxlL3VwZ3JhZGUvU2V2ZW5Ub0VpZ2h0VXBncmFkZUhhbmRsZXIuamF2YQ==) | 66.66% | [1 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
> :exclamation: There is a different number of reports uploaded between BASE (1eb97b3) and HEAD (10ccb3d). Click for more details.
>
> <details><summary>HEAD has 17 uploads less than BASE</summary>
>
>| Flag | BASE (1eb97b3) | HEAD (10ccb3d) |
>|------|------|------|
>|spark-scala-tests|10|5|
>|spark-java-tests|15|4|
>|common-and-other-modules|1|0|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18411 +/- ##
=============================================
- Coverage 68.21% 54.44% -13.78%
+ Complexity 27709 18272 -9437
=============================================
Files 2440 1996 -444
Lines 134249 111058 -23191
Branches 16179 14012 -2167
=============================================
- Hits 91578 60462 -31116
- Misses 35565 44421 +8856
+ Partials 7106 6175 -931
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.98% <66.66%> (+0.05%)` | :arrow_up: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.32% <0.00%> (+<0.01%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `32.87% <66.66%> (-15.84%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `32.59% <0.00%> (-12.65%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `38.38% <66.66%> (+<0.01%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...hudi/table/upgrade/SevenToEightUpgradeHandler.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Ftable%2Fupgrade%2FSevenToEightUpgradeHandler.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL3RhYmxlL3VwZ3JhZGUvU2V2ZW5Ub0VpZ2h0VXBncmFkZUhhbmRsZXIuamF2YQ==) | `63.57% <66.66%> (-23.76%)` | :arrow_down: |
... and [1375 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18411/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-18410] Optimize LSM timeline migration batch size and compaction during upgrade [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18411:
URL: https://github.com/apache/hudi/pull/18411#issuecomment-4157460060
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `66.66667%` with `1 line` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 48.15%. Comparing base ([`1eb97b3`](https://app.codecov.io/gh/apache/hudi/commit/1eb97b31826eabee4fb9c4bf78dc44d2bdb2dad0?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`cd5f54b`](https://app.codecov.io/gh/apache/hudi/commit/cd5f54b2de81575b98c515152c82fd71f950b2b4?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 6 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...hudi/table/upgrade/SevenToEightUpgradeHandler.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Ftable%2Fupgrade%2FSevenToEightUpgradeHandler.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL3RhYmxlL3VwZ3JhZGUvU2V2ZW5Ub0VpZ2h0VXBncmFkZUhhbmRsZXIuamF2YQ==) | 66.66% | [1 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
> :exclamation: There is a different number of reports uploaded between BASE (1eb97b3) and HEAD (cd5f54b). Click for more details.
>
> <details><summary>HEAD has 20 uploads less than BASE</summary>
>
>| Flag | BASE (1eb97b3) | HEAD (cd5f54b) |
>|------|------|------|
>|spark-scala-tests|10|5|
>|spark-java-tests|15|2|
>|common-and-other-modules|1|0|
>|utilities|1|0|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18411 +/- ##
=============================================
- Coverage 68.21% 48.15% -20.07%
+ Complexity 27709 16355 -11354
=============================================
Files 2440 1995 -445
Lines 134249 111150 -23099
Branches 16179 14042 -2137
=============================================
- Hits 91578 53523 -38055
- Misses 35565 52028 +16463
+ Partials 7106 5599 -1507
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.97% <66.66%> (+0.04%)` | :arrow_up: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.40% <0.00%> (+0.08%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `33.32% <66.66%> (-15.39%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `32.69% <0.00%> (-12.55%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...hudi/table/upgrade/SevenToEightUpgradeHandler.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Ftable%2Fupgrade%2FSevenToEightUpgradeHandler.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL3RhYmxlL3VwZ3JhZGUvU2V2ZW5Ub0VpZ2h0VXBncmFkZUhhbmRsZXIuamF2YQ==) | `62.14% <66.66%> (-25.19%)` | :arrow_down: |
... and [1512 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18411/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-18410] Optimize LSM timeline migration batch size and compaction during upgrade [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18411:
URL: https://github.com/apache/hudi/pull/18411#issuecomment-4157423042
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `66.66667%` with `1 line` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 45.77%. Comparing base ([`1eb97b3`](https://app.codecov.io/gh/apache/hudi/commit/1eb97b31826eabee4fb9c4bf78dc44d2bdb2dad0?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`cd5f54b`](https://app.codecov.io/gh/apache/hudi/commit/cd5f54b2de81575b98c515152c82fd71f950b2b4?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 6 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...hudi/table/upgrade/SevenToEightUpgradeHandler.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Ftable%2Fupgrade%2FSevenToEightUpgradeHandler.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL3RhYmxlL3VwZ3JhZGUvU2V2ZW5Ub0VpZ2h0VXBncmFkZUhhbmRsZXIuamF2YQ==) | 66.66% | [1 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
> :exclamation: There is a different number of reports uploaded between BASE (1eb97b3) and HEAD (cd5f54b). Click for more details.
>
> <details><summary>HEAD has 24 uploads less than BASE</summary>
>
>| Flag | BASE (1eb97b3) | HEAD (cd5f54b) |
>|------|------|------|
>|spark-scala-tests|10|3|
>|spark-java-tests|15|0|
>|common-and-other-modules|1|0|
>|utilities|1|0|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18411 +/- ##
=============================================
- Coverage 68.21% 45.77% -22.44%
+ Complexity 27709 15226 -12483
=============================================
Files 2440 1966 -474
Lines 134249 108548 -25701
Branches 16179 13483 -2696
=============================================
- Hits 91578 49691 -41887
- Misses 35565 53622 +18057
+ Partials 7106 5235 -1871
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.97% <66.66%> (+0.04%)` | :arrow_up: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.40% <0.00%> (+0.08%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `32.82% <0.00%> (-12.42%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...hudi/table/upgrade/SevenToEightUpgradeHandler.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Ftable%2Fupgrade%2FSevenToEightUpgradeHandler.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL3RhYmxlL3VwZ3JhZGUvU2V2ZW5Ub0VpZ2h0VXBncmFkZUhhbmRsZXIuamF2YQ==) | `57.85% <66.66%> (-29.47%)` | :arrow_down: |
... and [1579 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18411/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-18410] Optimize LSM timeline migration batch size and compaction during upgrade [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18411:
URL: https://github.com/apache/hudi/pull/18411#issuecomment-4157480579
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `87.50000%` with `3 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 58.75%. Comparing base ([`1eb97b3`](https://app.codecov.io/gh/apache/hudi/commit/1eb97b31826eabee4fb9c4bf78dc44d2bdb2dad0?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`cd5f54b`](https://app.codecov.io/gh/apache/hudi/commit/cd5f54b2de81575b98c515152c82fd71f950b2b4?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 6 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/sink/StreamWriteOperatorCoordinator.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-flink-datasource%2Fhudi-flink%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fsink%2FStreamWriteOperatorCoordinator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1mbGluay1kYXRhc291cmNlL2h1ZGktZmxpbmsvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvc2luay9TdHJlYW1Xcml0ZU9wZXJhdG9yQ29vcmRpbmF0b3IuamF2YQ==) | 90.47% | [1 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...hudi/table/upgrade/SevenToEightUpgradeHandler.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Ftable%2Fupgrade%2FSevenToEightUpgradeHandler.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL3RhYmxlL3VwZ3JhZGUvU2V2ZW5Ub0VpZ2h0VXBncmFkZUhhbmRsZXIuamF2YQ==) | 66.66% | [1 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
> :exclamation: There is a different number of reports uploaded between BASE (1eb97b3) and HEAD (cd5f54b). Click for more details.
>
> <details><summary>HEAD has 16 uploads less than BASE</summary>
>
>| Flag | BASE (1eb97b3) | HEAD (cd5f54b) |
>|------|------|------|
>|spark-scala-tests|10|5|
>|spark-java-tests|15|5|
>|utilities|1|0|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18411 +/- ##
============================================
- Coverage 68.21% 58.75% -9.46%
+ Complexity 27709 23915 -3794
============================================
Files 2440 2439 -1
Lines 134249 134358 +109
Branches 16179 16210 +31
============================================
- Hits 91578 78945 -12633
- Misses 35565 48667 +13102
+ Partials 7106 6746 -360
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.35% <87.50%> (+0.03%)` | :arrow_up: |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.97% <66.66%> (+0.04%)` | :arrow_up: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.40% <0.00%> (+0.08%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `32.48% <66.66%> (-16.23%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `32.69% <0.00%> (-12.55%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...hudi/table/upgrade/SevenToEightUpgradeHandler.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Ftable%2Fupgrade%2FSevenToEightUpgradeHandler.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL3RhYmxlL3VwZ3JhZGUvU2V2ZW5Ub0VpZ2h0VXBncmFkZUhhbmRsZXIuamF2YQ==) | `75.71% <66.66%> (-11.61%)` | :arrow_down: |
| [...ache/hudi/sink/StreamWriteOperatorCoordinator.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-flink-datasource%2Fhudi-flink%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fsink%2FStreamWriteOperatorCoordinator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1mbGluay1kYXRhc291cmNlL2h1ZGktZmxpbmsvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvc2luay9TdHJlYW1Xcml0ZU9wZXJhdG9yQ29vcmRpbmF0b3IuamF2YQ==) | `81.38% <90.47%> (+0.34%)` | :arrow_up: |
... and [667 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18411/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-18410] Optimize LSM timeline migration batch size and compaction during upgrade [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18411:
URL: https://github.com/apache/hudi/pull/18411#issuecomment-4157481531
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `87.50000%` with `3 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 61.26%. Comparing base ([`1eb97b3`](https://app.codecov.io/gh/apache/hudi/commit/1eb97b31826eabee4fb9c4bf78dc44d2bdb2dad0?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`cd5f54b`](https://app.codecov.io/gh/apache/hudi/commit/cd5f54b2de81575b98c515152c82fd71f950b2b4?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 6 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/sink/StreamWriteOperatorCoordinator.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-flink-datasource%2Fhudi-flink%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fsink%2FStreamWriteOperatorCoordinator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1mbGluay1kYXRhc291cmNlL2h1ZGktZmxpbmsvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvc2luay9TdHJlYW1Xcml0ZU9wZXJhdG9yQ29vcmRpbmF0b3IuamF2YQ==) | 90.47% | [1 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...hudi/table/upgrade/SevenToEightUpgradeHandler.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Ftable%2Fupgrade%2FSevenToEightUpgradeHandler.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL3RhYmxlL3VwZ3JhZGUvU2V2ZW5Ub0VpZ2h0VXBncmFkZUhhbmRsZXIuamF2YQ==) | 66.66% | [1 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
> :exclamation: There is a different number of reports uploaded between BASE (1eb97b3) and HEAD (cd5f54b). Click for more details.
>
> <details><summary>HEAD has 15 uploads less than BASE</summary>
>
>| Flag | BASE (1eb97b3) | HEAD (cd5f54b) |
>|------|------|------|
>|spark-scala-tests|10|5|
>|spark-java-tests|15|5|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18411 +/- ##
============================================
- Coverage 68.21% 61.26% -6.96%
+ Complexity 27709 24874 -2835
============================================
Files 2440 2439 -1
Lines 134249 134358 +109
Branches 16179 16210 +31
============================================
- Hits 91578 82310 -9268
- Misses 35565 45150 +9585
+ Partials 7106 6898 -208
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.35% <87.50%> (+0.03%)` | :arrow_up: |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.97% <66.66%> (+0.04%)` | :arrow_up: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.40% <0.00%> (+0.08%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `32.48% <66.66%> (-16.23%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `32.69% <0.00%> (-12.55%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `38.37% <66.66%> (+<0.01%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...hudi/table/upgrade/SevenToEightUpgradeHandler.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Ftable%2Fupgrade%2FSevenToEightUpgradeHandler.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL3RhYmxlL3VwZ3JhZGUvU2V2ZW5Ub0VpZ2h0VXBncmFkZUhhbmRsZXIuamF2YQ==) | `76.42% <66.66%> (-10.90%)` | :arrow_down: |
| [...ache/hudi/sink/StreamWriteOperatorCoordinator.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-flink-datasource%2Fhudi-flink%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fsink%2FStreamWriteOperatorCoordinator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1mbGluay1kYXRhc291cmNlL2h1ZGktZmxpbmsvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvc2luay9TdHJlYW1Xcml0ZU9wZXJhdG9yQ29vcmRpbmF0b3IuamF2YQ==) | `81.38% <90.47%> (+0.34%)` | :arrow_up: |
... and [552 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18411/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-18410] Optimize LSM timeline migration batch size and compaction during upgrade [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18411:
URL: https://github.com/apache/hudi/pull/18411#issuecomment-4157640340
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `87.50000%` with `3 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 68.25%. Comparing base ([`1eb97b3`](https://app.codecov.io/gh/apache/hudi/commit/1eb97b31826eabee4fb9c4bf78dc44d2bdb2dad0?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`cd5f54b`](https://app.codecov.io/gh/apache/hudi/commit/cd5f54b2de81575b98c515152c82fd71f950b2b4?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 6 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/sink/StreamWriteOperatorCoordinator.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-flink-datasource%2Fhudi-flink%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fsink%2FStreamWriteOperatorCoordinator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1mbGluay1kYXRhc291cmNlL2h1ZGktZmxpbmsvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvc2luay9TdHJlYW1Xcml0ZU9wZXJhdG9yQ29vcmRpbmF0b3IuamF2YQ==) | 90.47% | [1 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...hudi/table/upgrade/SevenToEightUpgradeHandler.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Ftable%2Fupgrade%2FSevenToEightUpgradeHandler.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL3RhYmxlL3VwZ3JhZGUvU2V2ZW5Ub0VpZ2h0VXBncmFkZUhhbmRsZXIuamF2YQ==) | 66.66% | [0 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18411 +/- ##
============================================
+ Coverage 68.21% 68.25% +0.04%
- Complexity 27709 27760 +51
============================================
Files 2440 2439 -1
Lines 134249 134358 +109
Branches 16179 16210 +31
============================================
+ Hits 91578 91710 +132
+ Misses 35565 35547 -18
+ Partials 7106 7101 -5
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.35% <87.50%> (+0.03%)` | :arrow_up: |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.97% <66.66%> (+0.04%)` | :arrow_up: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.40% <0.00%> (+0.08%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.74% <66.66%> (+0.03%)` | :arrow_up: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.29% <0.00%> (+0.06%)` | :arrow_up: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `38.37% <66.66%> (+<0.01%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...hudi/table/upgrade/SevenToEightUpgradeHandler.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Ftable%2Fupgrade%2FSevenToEightUpgradeHandler.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL3RhYmxlL3VwZ3JhZGUvU2V2ZW5Ub0VpZ2h0VXBncmFkZUhhbmRsZXIuamF2YQ==) | `85.00% <66.66%> (-2.33%)` | :arrow_down: |
| [...ache/hudi/sink/StreamWriteOperatorCoordinator.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-flink-datasource%2Fhudi-flink%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fsink%2FStreamWriteOperatorCoordinator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1mbGluay1kYXRhc291cmNlL2h1ZGktZmxpbmsvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvc2luay9TdHJlYW1Xcml0ZU9wZXJhdG9yQ29vcmRpbmF0b3IuamF2YQ==) | `81.38% <90.47%> (+0.34%)` | :arrow_up: |
... and [41 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18411/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-18410] Optimize LSM timeline migration batch size and compaction during upgrade [hudi]
Posted by "danny0405 (via GitHub)" <gi...@apache.org>.
danny0405 commented on code in PR #18411:
URL: https://github.com/apache/hudi/pull/18411#discussion_r3013409245
##########
hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/sink/StreamWriteOperatorCoordinator.java:
##########
@@ -385,16 +407,23 @@ public void executionAttemptReady(int i, int attemptNumber, SubtaskGateway gatew
@Override
public CompletableFuture<CoordinationResponse> handleCoordinationRequest(CoordinationRequest request) {
- if (request instanceof Correspondent.InstantTimeRequest) {
- return handleInstantRequest((Correspondent.InstantTimeRequest) request);
- }
- if (request instanceof Correspondent.InflightInstantsRequest) {
- return handleInFlightInstantsRequest((Correspondent.InflightInstantsRequest) request);
- }
- if (request instanceof Correspondent.AwaitPendingInstantsRequest) {
- return handleAwaitPendingInstantsRequest((Correspondent.AwaitPendingInstantsRequest) request);
- }
- throw new HoodieException("Unexpected coordination request type: " + request.getClass().getSimpleName());
+ // Gate coordination requests on initialization completion. The upgrade,
+ // metadata table init, and event restoration run asynchronously on the
+ // executor thread (see start()). Coordination requests like startInstant()
+ // run on a separate instantRequestExecutor and must not race ahead of
+ // initialization — e.g., RecordIndexPartitioner depends on MDT init.
+ return initFuture.thenCompose(ignored -> {
Review Comment:
we have some other code snippets that relies on the completion of table upgrade/downgrade(table state ready for write), this makes the dependency more complicated, does it make sense we move these heavy upgrade into a separate spark job?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-18410] Optimize LSM timeline migration batch size and compaction during upgrade [hudi]
Posted by "yihua (via GitHub)" <gi...@apache.org>.
yihua commented on code in PR #18411:
URL: https://github.com/apache/hudi/pull/18411#discussion_r3034562056
##########
hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/sink/StreamWriteOperatorCoordinator.java:
##########
@@ -385,16 +407,23 @@ public void executionAttemptReady(int i, int attemptNumber, SubtaskGateway gatew
@Override
public CompletableFuture<CoordinationResponse> handleCoordinationRequest(CoordinationRequest request) {
- if (request instanceof Correspondent.InstantTimeRequest) {
- return handleInstantRequest((Correspondent.InstantTimeRequest) request);
- }
- if (request instanceof Correspondent.InflightInstantsRequest) {
- return handleInFlightInstantsRequest((Correspondent.InflightInstantsRequest) request);
- }
- if (request instanceof Correspondent.AwaitPendingInstantsRequest) {
- return handleAwaitPendingInstantsRequest((Correspondent.AwaitPendingInstantsRequest) request);
- }
- throw new HoodieException("Unexpected coordination request type: " + request.getClass().getSimpleName());
+ // Gate coordination requests on initialization completion. The upgrade,
+ // metadata table init, and event restoration run asynchronously on the
+ // executor thread (see start()). Coordination requests like startInstant()
+ // run on a separate instantRequestExecutor and must not race ahead of
+ // initialization — e.g., RecordIndexPartitioner depends on MDT init.
+ return initFuture.thenCompose(ignored -> {
Review Comment:
🤖 Good question. A separate pre-upgrade step would keep the coordinator's `start()` simple, but it does change the UX from a transparent on-first-write upgrade to an explicit manual step. One middle ground: keep the in-place upgrade path but add a standalone CLI tool (or Spark action) that users with very large archived timelines can run ahead of time. That way the common case stays zero-touch, and only the pathological case (thousands of archived actions) needs the separate job.
##########
hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/sink/StreamWriteOperatorCoordinator.java:
##########
@@ -244,15 +245,36 @@ public void start() throws Exception {
.threadFactory(getThreadFactory("instant-request"))
.exceptionHook((errMsg, t) -> this.context.failJob(new HoodieException(errMsg, t)))
.build();
- // start the executor if required
- if (tableState.syncHive) {
- initHiveSync();
- }
- // start client id heartbeats for optimistic concurrency control
- if (OptionsResolver.isMultiWriter(conf)) {
- initClientIds(conf);
- }
- restoreEvents();
+
+ // Run upgrade, metadata table init, and event restoration on the executor
+ // thread instead of the Pekko dispatcher thread. Running these synchronously
+ // on the dispatcher thread blocks heartbeat responses when the operations
+ // involve heavy I/O (e.g., LSM timeline migration with hundreds of archived
+ // actions), causing the ResourceManager to disconnect the JobManager.
+ //
+ // Safety guarantees:
+ // - Events via handleEventFromOperator() are submitted to the same
+ // single-threaded executor, so FIFO ordering ensures initialization
Review Comment:
🤖 The outer `catch (Throwable throwable)` block at the end of `start()` also doesn't complete `initFuture`. If an exception is thrown after the executors are created but before the lambda is submitted (or if `executor.execute()` itself fails to enqueue), `initFuture` will remain incomplete and coordination requests will hang. It might be worth adding `initFuture.completeExceptionally(throwable)` in that catch block as well.
##########
hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/sink/StreamWriteOperatorCoordinator.java:
##########
@@ -244,15 +245,36 @@ public void start() throws Exception {
.threadFactory(getThreadFactory("instant-request"))
.exceptionHook((errMsg, t) -> this.context.failJob(new HoodieException(errMsg, t)))
.build();
- // start the executor if required
- if (tableState.syncHive) {
- initHiveSync();
- }
- // start client id heartbeats for optimistic concurrency control
- if (OptionsResolver.isMultiWriter(conf)) {
- initClientIds(conf);
- }
- restoreEvents();
+
+ // Run upgrade, metadata table init, and event restoration on the executor
+ // thread instead of the Pekko dispatcher thread. Running these synchronously
+ // on the dispatcher thread blocks heartbeat responses when the operations
+ // involve heavy I/O (e.g., LSM timeline migration with hundreds of archived
+ // actions), causing the ResourceManager to disconnect the JobManager.
+ //
+ // Safety guarantees:
+ // - Events via handleEventFromOperator() are submitted to the same
+ // single-threaded executor, so FIFO ordering ensures initialization
+ // completes before any event processing.
+ // - Coordination requests via handleCoordinationRequest() run on the
+ // separate instantRequestExecutor and are gated on initFuture to
+ // prevent startInstant() from racing ahead of the upgrade.
+ this.executor.execute(() -> {
+ this.writeClient.tryUpgrade(instant, this.metaClient);
+ initMetadataTable(this.writeClient);
+ if (tableState.scheduleMdtCompaction) {
+ this.metadataWriteClient = StreamerUtil.createMetadataWriteClient(writeClient);
+ }
+ if (tableState.syncHive) {
+ initHiveSync();
+ }
Review Comment:
🤖 If any of the init steps (tryUpgrade, initMetadataTable, etc.) throw an exception, `NonThrownExecutor` swallows it and calls the exception hook, but `initFuture.complete(null)` is never reached. This leaves `initFuture` permanently incomplete, so any `handleCoordinationRequest` caller already waiting on `initFuture.thenCompose(...)` will hang indefinitely. Could you wrap the lambda body in try-finally (or try-catch) to ensure `initFuture.completeExceptionally(t)` on failure?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-18410] Fix Flink upgrade blocking dispatcher thread causing heartbeat timeout [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18411:
URL: https://github.com/apache/hudi/pull/18411#issuecomment-4149235816
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `66.66667%` with `1 line` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 64.82%. Comparing base ([`1eb97b3`](https://app.codecov.io/gh/apache/hudi/commit/1eb97b31826eabee4fb9c4bf78dc44d2bdb2dad0?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`10ccb3d`](https://app.codecov.io/gh/apache/hudi/commit/10ccb3d818b0e218b08db4e0777b531e494a9708?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...hudi/table/upgrade/SevenToEightUpgradeHandler.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Ftable%2Fupgrade%2FSevenToEightUpgradeHandler.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL3RhYmxlL3VwZ3JhZGUvU2V2ZW5Ub0VpZ2h0VXBncmFkZUhhbmRsZXIuamF2YQ==) | 66.66% | [0 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18411 +/- ##
============================================
- Coverage 68.21% 64.82% -3.40%
+ Complexity 27709 22053 -5656
============================================
Files 2440 1996 -444
Lines 134249 111058 -23191
Branches 16179 14012 -2167
============================================
- Hits 91578 71989 -19589
+ Misses 35565 32753 -2812
+ Partials 7106 6316 -790
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.98% <66.66%> (+0.05%)` | :arrow_up: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.32% <0.00%> (+<0.01%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.99% <66.66%> (-3.72%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.24% <0.00%> (+0.01%)` | :arrow_up: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `38.38% <66.66%> (+<0.01%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...hudi/table/upgrade/SevenToEightUpgradeHandler.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Ftable%2Fupgrade%2FSevenToEightUpgradeHandler.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL3RhYmxlL3VwZ3JhZGUvU2V2ZW5Ub0VpZ2h0VXBncmFkZUhhbmRsZXIuamF2YQ==) | `77.85% <66.66%> (-9.47%)` | :arrow_down: |
... and [909 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18411/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-18410] Fix Flink upgrade blocking dispatcher thread causing heartbeat timeout [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18411:
URL: https://github.com/apache/hudi/pull/18411#issuecomment-4149211898
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `66.66667%` with `1 line` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 45.71%. Comparing base ([`1eb97b3`](https://app.codecov.io/gh/apache/hudi/commit/1eb97b31826eabee4fb9c4bf78dc44d2bdb2dad0?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`10ccb3d`](https://app.codecov.io/gh/apache/hudi/commit/10ccb3d818b0e218b08db4e0777b531e494a9708?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...hudi/table/upgrade/SevenToEightUpgradeHandler.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Ftable%2Fupgrade%2FSevenToEightUpgradeHandler.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL3RhYmxlL3VwZ3JhZGUvU2V2ZW5Ub0VpZ2h0VXBncmFkZUhhbmRsZXIuamF2YQ==) | 66.66% | [1 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
> :exclamation: There is a different number of reports uploaded between BASE (1eb97b3) and HEAD (10ccb3d). Click for more details.
>
> <details><summary>HEAD has 24 uploads less than BASE</summary>
>
>| Flag | BASE (1eb97b3) | HEAD (10ccb3d) |
>|------|------|------|
>|spark-scala-tests|10|3|
>|spark-java-tests|15|0|
>|common-and-other-modules|1|0|
>|utilities|1|0|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18411 +/- ##
=============================================
- Coverage 68.21% 45.71% -22.51%
+ Complexity 27709 15209 -12500
=============================================
Files 2440 1967 -473
Lines 134249 108485 -25764
Branches 16179 13461 -2718
=============================================
- Hits 91578 49591 -41987
- Misses 35565 53687 +18122
+ Partials 7106 5207 -1899
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.98% <66.66%> (+0.05%)` | :arrow_up: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.32% <0.00%> (+<0.01%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `32.79% <0.00%> (-12.45%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...hudi/table/upgrade/SevenToEightUpgradeHandler.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Ftable%2Fupgrade%2FSevenToEightUpgradeHandler.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL3RhYmxlL3VwZ3JhZGUvU2V2ZW5Ub0VpZ2h0VXBncmFkZUhhbmRsZXIuamF2YQ==) | `57.85% <66.66%> (-29.47%)` | :arrow_down: |
... and [1577 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18411/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-18410] Fix Flink upgrade blocking dispatcher thread causing heartbeat timeout [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18411:
URL: https://github.com/apache/hudi/pull/18411#issuecomment-4149213444
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `66.66667%` with `1 line` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 45.21%. Comparing base ([`1eb97b3`](https://app.codecov.io/gh/apache/hudi/commit/1eb97b31826eabee4fb9c4bf78dc44d2bdb2dad0?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`10ccb3d`](https://app.codecov.io/gh/apache/hudi/commit/10ccb3d818b0e218b08db4e0777b531e494a9708?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...hudi/table/upgrade/SevenToEightUpgradeHandler.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Ftable%2Fupgrade%2FSevenToEightUpgradeHandler.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL3RhYmxlL3VwZ3JhZGUvU2V2ZW5Ub0VpZ2h0VXBncmFkZUhhbmRsZXIuamF2YQ==) | 66.66% | [1 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
> :exclamation: There is a different number of reports uploaded between BASE (1eb97b3) and HEAD (10ccb3d). Click for more details.
>
> <details><summary>HEAD has 22 uploads less than BASE</summary>
>
>| Flag | BASE (1eb97b3) | HEAD (10ccb3d) |
>|------|------|------|
>|spark-scala-tests|10|5|
>|spark-java-tests|15|0|
>|common-and-other-modules|1|0|
>|utilities|1|0|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18411 +/- ##
=============================================
- Coverage 68.21% 45.21% -23.01%
+ Complexity 27709 15401 -12308
=============================================
Files 2440 1996 -444
Lines 134249 111058 -23191
Branches 16179 14012 -2167
=============================================
- Hits 91578 50211 -41367
- Misses 35565 55486 +19921
+ Partials 7106 5361 -1745
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.98% <66.66%> (+0.05%)` | :arrow_up: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.32% <0.00%> (+<0.01%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `32.59% <0.00%> (-12.65%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...hudi/table/upgrade/SevenToEightUpgradeHandler.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Ftable%2Fupgrade%2FSevenToEightUpgradeHandler.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL3RhYmxlL3VwZ3JhZGUvU2V2ZW5Ub0VpZ2h0VXBncmFkZUhhbmRsZXIuamF2YQ==) | `57.85% <66.66%> (-29.47%)` | :arrow_down: |
... and [1559 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18411/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-18410] Fix Flink upgrade blocking dispatcher thread causing heartbeat timeout [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18411:
URL: https://github.com/apache/hudi/pull/18411#issuecomment-4149204678
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `66.66667%` with `1 line` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 53.88%. Comparing base ([`1eb97b3`](https://app.codecov.io/gh/apache/hudi/commit/1eb97b31826eabee4fb9c4bf78dc44d2bdb2dad0?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`10ccb3d`](https://app.codecov.io/gh/apache/hudi/commit/10ccb3d818b0e218b08db4e0777b531e494a9708?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...hudi/table/upgrade/SevenToEightUpgradeHandler.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Ftable%2Fupgrade%2FSevenToEightUpgradeHandler.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL3RhYmxlL3VwZ3JhZGUvU2V2ZW5Ub0VpZ2h0VXBncmFkZUhhbmRsZXIuamF2YQ==) | 66.66% | [1 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
> :exclamation: There is a different number of reports uploaded between BASE (1eb97b3) and HEAD (10ccb3d). Click for more details.
>
> <details><summary>HEAD has 27 uploads less than BASE</summary>
>
>| Flag | BASE (1eb97b3) | HEAD (10ccb3d) |
>|------|------|------|
>|spark-scala-tests|10|0|
>|spark-java-tests|15|0|
>|common-and-other-modules|1|0|
>|utilities|1|0|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18411 +/- ##
=============================================
- Coverage 68.21% 53.88% -14.34%
+ Complexity 27709 12183 -15526
=============================================
Files 2440 1424 -1016
Lines 134249 70441 -63808
Branches 16179 7898 -8281
=============================================
- Hits 91578 37954 -53624
+ Misses 35565 29117 -6448
+ Partials 7106 3370 -3736
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.98% <66.66%> (+0.05%)` | :arrow_up: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.32% <0.00%> (+<0.01%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...hudi/table/upgrade/SevenToEightUpgradeHandler.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Ftable%2Fupgrade%2FSevenToEightUpgradeHandler.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL3RhYmxlL3VwZ3JhZGUvU2V2ZW5Ub0VpZ2h0VXBncmFkZUhhbmRsZXIuamF2YQ==) | `57.85% <66.66%> (-29.47%)` | :arrow_down: |
... and [1789 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18411/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-18410] Fix Flink upgrade blocking dispatcher thread causing heartbeat timeout [hudi]
Posted by "danny0405 (via GitHub)" <gi...@apache.org>.
danny0405 commented on code in PR #18411:
URL: https://github.com/apache/hudi/pull/18411#discussion_r3006987175
##########
hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/sink/StreamWriteOperatorCoordinator.java:
##########
@@ -244,6 +248,17 @@ public void start() throws Exception {
.threadFactory(getThreadFactory("instant-request"))
.exceptionHook((errMsg, t) -> this.context.failJob(new HoodieException(errMsg, t)))
.build();
+
+ // Run upgrade and post-upgrade initialization on the executor thread.
+ this.executor.execute(() -> {
+ this.writeClient.tryUpgrade(instant, this.metaClient);
+ initMetadataTable(this.writeClient);
+ if (tableState.scheduleMdtCompaction) {
+ this.metadataWriteClient = StreamerUtil.createMetadataWriteClient(writeClient);
Review Comment:
we actually want to block the open of the coordinator before the Hudi table is ready for write, in case there is no inconsistency from the write task sight of view.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-18410] Optimize LSM timeline migration batch size and compaction during upgrade [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18411:
URL: https://github.com/apache/hudi/pull/18411#issuecomment-4157572130
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `87.50000%` with `3 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 67.45%. Comparing base ([`1eb97b3`](https://app.codecov.io/gh/apache/hudi/commit/1eb97b31826eabee4fb9c4bf78dc44d2bdb2dad0?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`cd5f54b`](https://app.codecov.io/gh/apache/hudi/commit/cd5f54b2de81575b98c515152c82fd71f950b2b4?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 6 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/sink/StreamWriteOperatorCoordinator.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-flink-datasource%2Fhudi-flink%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fsink%2FStreamWriteOperatorCoordinator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1mbGluay1kYXRhc291cmNlL2h1ZGktZmxpbmsvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvc2luay9TdHJlYW1Xcml0ZU9wZXJhdG9yQ29vcmRpbmF0b3IuamF2YQ==) | 90.47% | [1 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...hudi/table/upgrade/SevenToEightUpgradeHandler.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Ftable%2Fupgrade%2FSevenToEightUpgradeHandler.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL3RhYmxlL3VwZ3JhZGUvU2V2ZW5Ub0VpZ2h0VXBncmFkZUhhbmRsZXIuamF2YQ==) | 66.66% | [0 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18411 +/- ##
============================================
- Coverage 68.21% 67.45% -0.77%
+ Complexity 27709 27441 -268
============================================
Files 2440 2439 -1
Lines 134249 134358 +109
Branches 16179 16210 +31
============================================
- Hits 91578 90627 -951
- Misses 35565 36570 +1005
- Partials 7106 7161 +55
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.35% <87.50%> (+0.03%)` | :arrow_up: |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.97% <66.66%> (+0.04%)` | :arrow_up: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.40% <0.00%> (+0.08%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.14% <66.66%> (-3.57%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.29% <0.00%> (+0.06%)` | :arrow_up: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `38.37% <66.66%> (+<0.01%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...hudi/table/upgrade/SevenToEightUpgradeHandler.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Ftable%2Fupgrade%2FSevenToEightUpgradeHandler.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL3RhYmxlL3VwZ3JhZGUvU2V2ZW5Ub0VpZ2h0VXBncmFkZUhhbmRsZXIuamF2YQ==) | `85.00% <66.66%> (-2.33%)` | :arrow_down: |
| [...ache/hudi/sink/StreamWriteOperatorCoordinator.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-flink-datasource%2Fhudi-flink%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fsink%2FStreamWriteOperatorCoordinator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1mbGluay1kYXRhc291cmNlL2h1ZGktZmxpbmsvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvc2luay9TdHJlYW1Xcml0ZU9wZXJhdG9yQ29vcmRpbmF0b3IuamF2YQ==) | `81.38% <90.47%> (+0.34%)` | :arrow_up: |
... and [162 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18411/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-18410] Fix Flink upgrade blocking dispatcher thread causing heartbeat timeout [hudi]
Posted by "cshuo (via GitHub)" <gi...@apache.org>.
cshuo commented on code in PR #18411:
URL: https://github.com/apache/hudi/pull/18411#discussion_r3007251875
##########
hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/sink/StreamWriteOperatorCoordinator.java:
##########
@@ -228,14 +228,18 @@ public void start() throws Exception {
this.metaClient = initTableIfNotExists(this.conf);
// the write client must create after the table creation
this.writeClient = FlinkWriteClients.createWriteClient(conf);
- this.writeClient.tryUpgrade(instant, this.metaClient);
- initMetadataTable(this.writeClient);
- if (tableState.scheduleMdtCompaction) {
- this.metadataWriteClient = StreamerUtil.createMetadataWriteClient(writeClient);
- }
-
- // start the executor
+ // Create executors BEFORE the upgrade so that the heavy initialization
+ // (upgrade, metadata table init, event restoration) can run on the executor
+ // thread instead of the Pekko dispatcher thread. Running the upgrade
+ // synchronously on the dispatcher thread blocks heartbeat responses and
+ // causes the ResourceManager to disconnect the JobManager when upgrades
+ // take longer than the heartbeat timeout (e.g., LSM timeline migration
+ // with hundreds of archived actions).
+ //
+ // Since the executor is single-threaded and FIFO, submitting the
Review Comment:
Initialization is submitted asynchronously to executor, while instant-time requests run on another executor `instantRequestExecutor` immediately. That allows `startInstant()` to race ahead of `tryUpgrade()` and metadata init, which leads to unexpected result.
Besides, the initialization/running of the ingestion pipeline also depends on the MDT init when streaming RLI is enabled, e.g., `RecordIndexPartitioner` needs to know the file group count of RLI before shuffling the records.
https://github.com/apache/hudi/blob/d241b0901b271b2ff6b86a03904bbfd25ec300c8/hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/sink/partitioner/RecordIndexPartitioner.java#L53-L57
Overall, it't not safe to start the pipeline and begin processing inputs before the upgrading and MDT initialization is finished.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-18410] Optimize LSM timeline migration batch size and compaction during upgrade [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18411:
URL: https://github.com/apache/hudi/pull/18411#issuecomment-4157420282
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `66.66667%` with `1 line` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 46.34%. Comparing base ([`1eb97b3`](https://app.codecov.io/gh/apache/hudi/commit/1eb97b31826eabee4fb9c4bf78dc44d2bdb2dad0?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`cd5f54b`](https://app.codecov.io/gh/apache/hudi/commit/cd5f54b2de81575b98c515152c82fd71f950b2b4?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 6 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...hudi/table/upgrade/SevenToEightUpgradeHandler.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Ftable%2Fupgrade%2FSevenToEightUpgradeHandler.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL3RhYmxlL3VwZ3JhZGUvU2V2ZW5Ub0VpZ2h0VXBncmFkZUhhbmRsZXIuamF2YQ==) | 66.66% | [1 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
> :exclamation: There is a different number of reports uploaded between BASE (1eb97b3) and HEAD (cd5f54b). Click for more details.
>
> <details><summary>HEAD has 25 uploads less than BASE</summary>
>
>| Flag | BASE (1eb97b3) | HEAD (cd5f54b) |
>|------|------|------|
>|spark-scala-tests|10|2|
>|spark-java-tests|15|0|
>|common-and-other-modules|1|0|
>|utilities|1|0|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18411 +/- ##
=============================================
- Coverage 68.21% 46.34% -21.87%
+ Complexity 27709 15066 -12643
=============================================
Files 2440 1949 -491
Lines 134249 106230 -28019
Branches 16179 12991 -3188
=============================================
- Hits 91578 49237 -42341
- Misses 35565 51894 +16329
+ Partials 7106 5099 -2007
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.97% <66.66%> (+0.04%)` | :arrow_up: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.40% <0.00%> (+0.08%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `33.10% <0.00%> (-12.14%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...hudi/table/upgrade/SevenToEightUpgradeHandler.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Ftable%2Fupgrade%2FSevenToEightUpgradeHandler.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL3RhYmxlL3VwZ3JhZGUvU2V2ZW5Ub0VpZ2h0VXBncmFkZUhhbmRsZXIuamF2YQ==) | `57.85% <66.66%> (-29.47%)` | :arrow_down: |
... and [1588 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18411/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-18410] Fix Flink upgrade blocking dispatcher thread causing heartbeat timeout [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18411:
URL: https://github.com/apache/hudi/pull/18411#issuecomment-4149233870
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `66.66667%` with `1 line` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 64.79%. Comparing base ([`1eb97b3`](https://app.codecov.io/gh/apache/hudi/commit/1eb97b31826eabee4fb9c4bf78dc44d2bdb2dad0?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`10ccb3d`](https://app.codecov.io/gh/apache/hudi/commit/10ccb3d818b0e218b08db4e0777b531e494a9708?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...hudi/table/upgrade/SevenToEightUpgradeHandler.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Ftable%2Fupgrade%2FSevenToEightUpgradeHandler.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL3RhYmxlL3VwZ3JhZGUvU2V2ZW5Ub0VpZ2h0VXBncmFkZUhhbmRsZXIuamF2YQ==) | 66.66% | [0 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18411 +/- ##
============================================
- Coverage 68.21% 64.79% -3.43%
+ Complexity 27709 22046 -5663
============================================
Files 2440 1996 -444
Lines 134249 111058 -23191
Branches 16179 14012 -2167
============================================
- Hits 91578 71960 -19618
+ Misses 35565 32781 -2784
+ Partials 7106 6317 -789
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.98% <66.66%> (+0.05%)` | :arrow_up: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.32% <0.00%> (+<0.01%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.95% <66.66%> (-3.76%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.24% <0.00%> (+0.01%)` | :arrow_up: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `38.38% <66.66%> (+<0.01%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...hudi/table/upgrade/SevenToEightUpgradeHandler.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Ftable%2Fupgrade%2FSevenToEightUpgradeHandler.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL3RhYmxlL3VwZ3JhZGUvU2V2ZW5Ub0VpZ2h0VXBncmFkZUhhbmRsZXIuamF2YQ==) | `77.85% <66.66%> (-9.47%)` | :arrow_down: |
... and [916 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18411/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-18410] Fix Flink upgrade blocking dispatcher thread causing heartbeat timeout [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18411:
URL: https://github.com/apache/hudi/pull/18411#issuecomment-4149237955
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `66.66667%` with `1 line` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 64.85%. Comparing base ([`1eb97b3`](https://app.codecov.io/gh/apache/hudi/commit/1eb97b31826eabee4fb9c4bf78dc44d2bdb2dad0?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`10ccb3d`](https://app.codecov.io/gh/apache/hudi/commit/10ccb3d818b0e218b08db4e0777b531e494a9708?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...hudi/table/upgrade/SevenToEightUpgradeHandler.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Ftable%2Fupgrade%2FSevenToEightUpgradeHandler.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL3RhYmxlL3VwZ3JhZGUvU2V2ZW5Ub0VpZ2h0VXBncmFkZUhhbmRsZXIuamF2YQ==) | 66.66% | [0 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18411 +/- ##
============================================
- Coverage 68.21% 64.85% -3.37%
+ Complexity 27709 22067 -5642
============================================
Files 2440 1996 -444
Lines 134249 111058 -23191
Branches 16179 14012 -2167
============================================
- Hits 91578 72026 -19552
+ Misses 35565 32716 -2849
+ Partials 7106 6316 -790
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.98% <66.66%> (+0.05%)` | :arrow_up: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.32% <0.00%> (+<0.01%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.05% <66.66%> (-3.66%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.24% <0.00%> (+0.01%)` | :arrow_up: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `38.38% <66.66%> (+<0.01%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...hudi/table/upgrade/SevenToEightUpgradeHandler.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Ftable%2Fupgrade%2FSevenToEightUpgradeHandler.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL3RhYmxlL3VwZ3JhZGUvU2V2ZW5Ub0VpZ2h0VXBncmFkZUhhbmRsZXIuamF2YQ==) | `77.85% <66.66%> (-9.47%)` | :arrow_down: |
... and [901 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18411/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-18410] Optimize LSM timeline migration batch size and compaction during upgrade [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #18411:
URL: https://github.com/apache/hudi/pull/18411#issuecomment-4157036474
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "10ccb3d818b0e218b08db4e0777b531e494a9708",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12900",
"triggerID" : "10ccb3d818b0e218b08db4e0777b531e494a9708",
"triggerType" : "PUSH"
}, {
"hash" : "df05521d76e687ac5e4d09c65e3d94a5d03ae5d6",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "df05521d76e687ac5e4d09c65e3d94a5d03ae5d6",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 10ccb3d818b0e218b08db4e0777b531e494a9708 Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12900)
* df05521d76e687ac5e4d09c65e3d94a5d03ae5d6 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-18410] Optimize LSM timeline migration batch size and compaction during upgrade [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18411:
URL: https://github.com/apache/hudi/pull/18411#issuecomment-4157109972
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `66.66667%` with `1 line` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 44.97%. Comparing base ([`1eb97b3`](https://app.codecov.io/gh/apache/hudi/commit/1eb97b31826eabee4fb9c4bf78dc44d2bdb2dad0?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`df05521`](https://app.codecov.io/gh/apache/hudi/commit/df05521d76e687ac5e4d09c65e3d94a5d03ae5d6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 6 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...hudi/table/upgrade/SevenToEightUpgradeHandler.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Ftable%2Fupgrade%2FSevenToEightUpgradeHandler.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL3RhYmxlL3VwZ3JhZGUvU2V2ZW5Ub0VpZ2h0VXBncmFkZUhhbmRsZXIuamF2YQ==) | 66.66% | [1 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
> :exclamation: There is a different number of reports uploaded between BASE (1eb97b3) and HEAD (df05521). Click for more details.
>
> <details><summary>HEAD has 28 uploads less than BASE</summary>
>
>| Flag | BASE (1eb97b3) | HEAD (df05521) |
>|------|------|------|
>|spark-scala-tests|10|0|
>|spark-client-hadoop-common|1|0|
>|spark-java-tests|15|0|
>|common-and-other-modules|1|0|
>|utilities|1|0|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18411 +/- ##
=============================================
- Coverage 68.21% 44.97% -23.25%
+ Complexity 27709 8456 -19253
=============================================
Files 2440 1194 -1246
Lines 134249 61631 -72618
Branches 16179 6609 -9570
=============================================
- Hits 91578 27718 -63860
+ Misses 35565 30895 -4670
+ Partials 7106 3018 -4088
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.97% <66.66%> (+0.04%)` | :arrow_up: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...hudi/table/upgrade/SevenToEightUpgradeHandler.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Ftable%2Fupgrade%2FSevenToEightUpgradeHandler.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL3RhYmxlL3VwZ3JhZGUvU2V2ZW5Ub0VpZ2h0VXBncmFkZUhhbmRsZXIuamF2YQ==) | `57.85% <66.66%> (-29.47%)` | :arrow_down: |
... and [2042 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18411/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-18410] Fix Flink upgrade blocking dispatcher thread causing heartbeat timeout [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18411:
URL: https://github.com/apache/hudi/pull/18411#issuecomment-4149218314
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `66.66667%` with `1 line` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 48.05%. Comparing base ([`1eb97b3`](https://app.codecov.io/gh/apache/hudi/commit/1eb97b31826eabee4fb9c4bf78dc44d2bdb2dad0?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`10ccb3d`](https://app.codecov.io/gh/apache/hudi/commit/10ccb3d818b0e218b08db4e0777b531e494a9708?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...hudi/table/upgrade/SevenToEightUpgradeHandler.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Ftable%2Fupgrade%2FSevenToEightUpgradeHandler.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL3RhYmxlL3VwZ3JhZGUvU2V2ZW5Ub0VpZ2h0VXBncmFkZUhhbmRsZXIuamF2YQ==) | 66.66% | [1 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
> :exclamation: There is a different number of reports uploaded between BASE (1eb97b3) and HEAD (10ccb3d). Click for more details.
>
> <details><summary>HEAD has 19 uploads less than BASE</summary>
>
>| Flag | BASE (1eb97b3) | HEAD (10ccb3d) |
>|------|------|------|
>|spark-scala-tests|10|5|
>|spark-java-tests|15|3|
>|common-and-other-modules|1|0|
>|utilities|1|0|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18411 +/- ##
=============================================
- Coverage 68.21% 48.05% -20.16%
+ Complexity 27709 16316 -11393
=============================================
Files 2440 1996 -444
Lines 134249 111058 -23191
Branches 16179 14012 -2167
=============================================
- Hits 91578 53373 -38205
- Misses 35565 52123 +16558
+ Partials 7106 5562 -1544
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.98% <66.66%> (+0.05%)` | :arrow_up: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.32% <0.00%> (+<0.01%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `33.32% <66.66%> (-15.39%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `32.59% <0.00%> (-12.65%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...hudi/table/upgrade/SevenToEightUpgradeHandler.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Ftable%2Fupgrade%2FSevenToEightUpgradeHandler.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL3RhYmxlL3VwZ3JhZGUvU2V2ZW5Ub0VpZ2h0VXBncmFkZUhhbmRsZXIuamF2YQ==) | `62.14% <66.66%> (-25.19%)` | :arrow_down: |
... and [1511 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18411/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-18410] Fix Flink upgrade blocking dispatcher thread causing heartbeat timeout [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18411:
URL: https://github.com/apache/hudi/pull/18411#issuecomment-4149215229
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `66.66667%` with `1 line` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 47.99%. Comparing base ([`1eb97b3`](https://app.codecov.io/gh/apache/hudi/commit/1eb97b31826eabee4fb9c4bf78dc44d2bdb2dad0?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`10ccb3d`](https://app.codecov.io/gh/apache/hudi/commit/10ccb3d818b0e218b08db4e0777b531e494a9708?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...hudi/table/upgrade/SevenToEightUpgradeHandler.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Ftable%2Fupgrade%2FSevenToEightUpgradeHandler.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL3RhYmxlL3VwZ3JhZGUvU2V2ZW5Ub0VpZ2h0VXBncmFkZUhhbmRsZXIuamF2YQ==) | 66.66% | [1 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
> :exclamation: There is a different number of reports uploaded between BASE (1eb97b3) and HEAD (10ccb3d). Click for more details.
>
> <details><summary>HEAD has 21 uploads less than BASE</summary>
>
>| Flag | BASE (1eb97b3) | HEAD (10ccb3d) |
>|------|------|------|
>|spark-scala-tests|10|5|
>|spark-java-tests|15|1|
>|common-and-other-modules|1|0|
>|utilities|1|0|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18411 +/- ##
=============================================
- Coverage 68.21% 47.99% -20.22%
+ Complexity 27709 16297 -11412
=============================================
Files 2440 1996 -444
Lines 134249 111058 -23191
Branches 16179 14012 -2167
=============================================
- Hits 91578 53303 -38275
- Misses 35565 52179 +16614
+ Partials 7106 5576 -1530
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.98% <66.66%> (+0.05%)` | :arrow_up: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.32% <0.00%> (+<0.01%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `33.69% <66.66%> (-15.02%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `32.59% <0.00%> (-12.65%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...hudi/table/upgrade/SevenToEightUpgradeHandler.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Ftable%2Fupgrade%2FSevenToEightUpgradeHandler.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL3RhYmxlL3VwZ3JhZGUvU2V2ZW5Ub0VpZ2h0VXBncmFkZUhhbmRsZXIuamF2YQ==) | `62.14% <66.66%> (-25.19%)` | :arrow_down: |
... and [1510 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18411/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-18410] Fix Flink upgrade blocking dispatcher thread causing heartbeat timeout [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18411:
URL: https://github.com/apache/hudi/pull/18411#issuecomment-4149212593
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `66.66667%` with `1 line` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 45.71%. Comparing base ([`1eb97b3`](https://app.codecov.io/gh/apache/hudi/commit/1eb97b31826eabee4fb9c4bf78dc44d2bdb2dad0?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`10ccb3d`](https://app.codecov.io/gh/apache/hudi/commit/10ccb3d818b0e218b08db4e0777b531e494a9708?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...hudi/table/upgrade/SevenToEightUpgradeHandler.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Ftable%2Fupgrade%2FSevenToEightUpgradeHandler.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL3RhYmxlL3VwZ3JhZGUvU2V2ZW5Ub0VpZ2h0VXBncmFkZUhhbmRsZXIuamF2YQ==) | 66.66% | [1 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
> :exclamation: There is a different number of reports uploaded between BASE (1eb97b3) and HEAD (10ccb3d). Click for more details.
>
> <details><summary>HEAD has 23 uploads less than BASE</summary>
>
>| Flag | BASE (1eb97b3) | HEAD (10ccb3d) |
>|------|------|------|
>|spark-scala-tests|10|4|
>|spark-java-tests|15|0|
>|common-and-other-modules|1|0|
>|utilities|1|0|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18411 +/- ##
=============================================
- Coverage 68.21% 45.71% -22.51%
+ Complexity 27709 15209 -12500
=============================================
Files 2440 1967 -473
Lines 134249 108490 -25759
Branches 16179 13467 -2712
=============================================
- Hits 91578 49595 -41983
- Misses 35565 53689 +18124
+ Partials 7106 5206 -1900
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.98% <66.66%> (+0.05%)` | :arrow_up: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.32% <0.00%> (+<0.01%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `32.79% <0.00%> (-12.45%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...hudi/table/upgrade/SevenToEightUpgradeHandler.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Ftable%2Fupgrade%2FSevenToEightUpgradeHandler.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL3RhYmxlL3VwZ3JhZGUvU2V2ZW5Ub0VpZ2h0VXBncmFkZUhhbmRsZXIuamF2YQ==) | `57.85% <66.66%> (-29.47%)` | :arrow_down: |
... and [1575 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18411/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-18410] Fix Flink upgrade blocking dispatcher thread causing heartbeat timeout [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18411:
URL: https://github.com/apache/hudi/pull/18411#issuecomment-4149231103
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `66.66667%` with `1 line` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 64.63%. Comparing base ([`1eb97b3`](https://app.codecov.io/gh/apache/hudi/commit/1eb97b31826eabee4fb9c4bf78dc44d2bdb2dad0?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`10ccb3d`](https://app.codecov.io/gh/apache/hudi/commit/10ccb3d818b0e218b08db4e0777b531e494a9708?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...hudi/table/upgrade/SevenToEightUpgradeHandler.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Ftable%2Fupgrade%2FSevenToEightUpgradeHandler.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL3RhYmxlL3VwZ3JhZGUvU2V2ZW5Ub0VpZ2h0VXBncmFkZUhhbmRsZXIuamF2YQ==) | 66.66% | [0 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18411 +/- ##
============================================
- Coverage 68.21% 64.63% -3.59%
+ Complexity 27709 22004 -5705
============================================
Files 2440 1996 -444
Lines 134249 111058 -23191
Branches 16179 14012 -2167
============================================
- Hits 91578 71781 -19797
+ Misses 35565 32937 -2628
+ Partials 7106 6340 -766
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.98% <66.66%> (+0.05%)` | :arrow_up: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.32% <0.00%> (+<0.01%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.88% <66.66%> (-3.83%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.05% <0.00%> (-0.19%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `38.38% <66.66%> (+<0.01%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...hudi/table/upgrade/SevenToEightUpgradeHandler.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Ftable%2Fupgrade%2FSevenToEightUpgradeHandler.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL3RhYmxlL3VwZ3JhZGUvU2V2ZW5Ub0VpZ2h0VXBncmFkZUhhbmRsZXIuamF2YQ==) | `77.85% <66.66%> (-9.47%)` | :arrow_down: |
... and [955 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18411/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-18410] Fix Flink upgrade blocking dispatcher thread causing heartbeat timeout [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #18411:
URL: https://github.com/apache/hudi/pull/18411#issuecomment-4149164272
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "10ccb3d818b0e218b08db4e0777b531e494a9708",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "10ccb3d818b0e218b08db4e0777b531e494a9708",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 10ccb3d818b0e218b08db4e0777b531e494a9708 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-18410] Optimize LSM timeline migration batch size and compaction during upgrade [hudi]
Posted by "prashantwason (via GitHub)" <gi...@apache.org>.
prashantwason commented on code in PR #18411:
URL: https://github.com/apache/hudi/pull/18411#discussion_r3041688950
##########
hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/sink/StreamWriteOperatorCoordinator.java:
##########
@@ -385,16 +407,23 @@ public void executionAttemptReady(int i, int attemptNumber, SubtaskGateway gatew
@Override
public CompletableFuture<CoordinationResponse> handleCoordinationRequest(CoordinationRequest request) {
- if (request instanceof Correspondent.InstantTimeRequest) {
- return handleInstantRequest((Correspondent.InstantTimeRequest) request);
- }
- if (request instanceof Correspondent.InflightInstantsRequest) {
- return handleInFlightInstantsRequest((Correspondent.InflightInstantsRequest) request);
- }
- if (request instanceof Correspondent.AwaitPendingInstantsRequest) {
- return handleAwaitPendingInstantsRequest((Correspondent.AwaitPendingInstantsRequest) request);
- }
- throw new HoodieException("Unexpected coordination request type: " + request.getClass().getSimpleName());
+ // Gate coordination requests on initialization completion. The upgrade,
+ // metadata table init, and event restoration run asynchronously on the
+ // executor thread (see start()). Coordination requests like startInstant()
+ // run on a separate instantRequestExecutor and must not race ahead of
+ // initialization — e.g., RecordIndexPartitioner depends on MDT init.
+ return initFuture.thenCompose(ignored -> {
Review Comment:
Agreed that a separate pre-upgrade job could simplify things for very large tables. For this PR, the async-with-gating approach keeps the zero-touch UX for the common case while unblocking the dispatcher thread. A standalone CLI/Spark upgrade tool could be a good follow-up for users with thousands of archived actions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-18410] Optimize LSM timeline migration batch size and compaction during upgrade [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #18411:
URL: https://github.com/apache/hudi/pull/18411#issuecomment-4195408968
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "10ccb3d818b0e218b08db4e0777b531e494a9708",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12900",
"triggerID" : "10ccb3d818b0e218b08db4e0777b531e494a9708",
"triggerType" : "PUSH"
}, {
"hash" : "df05521d76e687ac5e4d09c65e3d94a5d03ae5d6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12928",
"triggerID" : "df05521d76e687ac5e4d09c65e3d94a5d03ae5d6",
"triggerType" : "PUSH"
}, {
"hash" : "cd5f54b2de81575b98c515152c82fd71f950b2b4",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12930",
"triggerID" : "cd5f54b2de81575b98c515152c82fd71f950b2b4",
"triggerType" : "PUSH"
}, {
"hash" : "5c078d876a231d32e7127c802e2f6197c65bfee7",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=13102",
"triggerID" : "5c078d876a231d32e7127c802e2f6197c65bfee7",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 5c078d876a231d32e7127c802e2f6197c65bfee7 Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=13102)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-18410] Optimize LSM timeline migration batch size and compaction during upgrade [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18411:
URL: https://github.com/apache/hudi/pull/18411#issuecomment-4157477250
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `87.50000%` with `3 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 58.75%. Comparing base ([`1eb97b3`](https://app.codecov.io/gh/apache/hudi/commit/1eb97b31826eabee4fb9c4bf78dc44d2bdb2dad0?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`cd5f54b`](https://app.codecov.io/gh/apache/hudi/commit/cd5f54b2de81575b98c515152c82fd71f950b2b4?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 6 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/sink/StreamWriteOperatorCoordinator.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-flink-datasource%2Fhudi-flink%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fsink%2FStreamWriteOperatorCoordinator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1mbGluay1kYXRhc291cmNlL2h1ZGktZmxpbmsvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvc2luay9TdHJlYW1Xcml0ZU9wZXJhdG9yQ29vcmRpbmF0b3IuamF2YQ==) | 90.47% | [1 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...hudi/table/upgrade/SevenToEightUpgradeHandler.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Ftable%2Fupgrade%2FSevenToEightUpgradeHandler.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL3RhYmxlL3VwZ3JhZGUvU2V2ZW5Ub0VpZ2h0VXBncmFkZUhhbmRsZXIuamF2YQ==) | 66.66% | [1 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
> :exclamation: There is a different number of reports uploaded between BASE (1eb97b3) and HEAD (cd5f54b). Click for more details.
>
> <details><summary>HEAD has 17 uploads less than BASE</summary>
>
>| Flag | BASE (1eb97b3) | HEAD (cd5f54b) |
>|------|------|------|
>|spark-scala-tests|10|5|
>|spark-java-tests|15|4|
>|utilities|1|0|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18411 +/- ##
============================================
- Coverage 68.21% 58.75% -9.47%
+ Complexity 27709 23915 -3794
============================================
Files 2440 2439 -1
Lines 134249 134358 +109
Branches 16179 16210 +31
============================================
- Hits 91578 78940 -12638
- Misses 35565 48673 +13108
+ Partials 7106 6745 -361
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.35% <87.50%> (+0.03%)` | :arrow_up: |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.97% <66.66%> (+0.04%)` | :arrow_up: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.40% <0.00%> (+0.08%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `32.48% <66.66%> (-16.23%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `32.69% <0.00%> (-12.55%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...hudi/table/upgrade/SevenToEightUpgradeHandler.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Ftable%2Fupgrade%2FSevenToEightUpgradeHandler.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL3RhYmxlL3VwZ3JhZGUvU2V2ZW5Ub0VpZ2h0VXBncmFkZUhhbmRsZXIuamF2YQ==) | `75.71% <66.66%> (-11.61%)` | :arrow_down: |
| [...ache/hudi/sink/StreamWriteOperatorCoordinator.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-flink-datasource%2Fhudi-flink%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fsink%2FStreamWriteOperatorCoordinator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1mbGluay1kYXRhc291cmNlL2h1ZGktZmxpbmsvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvc2luay9TdHJlYW1Xcml0ZU9wZXJhdG9yQ29vcmRpbmF0b3IuamF2YQ==) | `81.38% <90.47%> (+0.34%)` | :arrow_up: |
... and [667 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18411/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-18410] Optimize LSM timeline migration batch size and compaction during upgrade [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18411:
URL: https://github.com/apache/hudi/pull/18411#issuecomment-4157562260
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `87.50000%` with `3 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 67.43%. Comparing base ([`1eb97b3`](https://app.codecov.io/gh/apache/hudi/commit/1eb97b31826eabee4fb9c4bf78dc44d2bdb2dad0?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`cd5f54b`](https://app.codecov.io/gh/apache/hudi/commit/cd5f54b2de81575b98c515152c82fd71f950b2b4?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 6 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/sink/StreamWriteOperatorCoordinator.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-flink-datasource%2Fhudi-flink%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fsink%2FStreamWriteOperatorCoordinator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1mbGluay1kYXRhc291cmNlL2h1ZGktZmxpbmsvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvc2luay9TdHJlYW1Xcml0ZU9wZXJhdG9yQ29vcmRpbmF0b3IuamF2YQ==) | 90.47% | [1 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...hudi/table/upgrade/SevenToEightUpgradeHandler.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Ftable%2Fupgrade%2FSevenToEightUpgradeHandler.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL3RhYmxlL3VwZ3JhZGUvU2V2ZW5Ub0VpZ2h0VXBncmFkZUhhbmRsZXIuamF2YQ==) | 66.66% | [0 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18411 +/- ##
============================================
- Coverage 68.21% 67.43% -0.79%
+ Complexity 27709 27435 -274
============================================
Files 2440 2439 -1
Lines 134249 134358 +109
Branches 16179 16210 +31
============================================
- Hits 91578 90602 -976
- Misses 35565 36594 +1029
- Partials 7106 7162 +56
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.35% <87.50%> (+0.03%)` | :arrow_up: |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.97% <66.66%> (+0.04%)` | :arrow_up: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.40% <0.00%> (+0.08%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.12% <66.66%> (-3.59%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.29% <0.00%> (+0.06%)` | :arrow_up: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `38.37% <66.66%> (+<0.01%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...hudi/table/upgrade/SevenToEightUpgradeHandler.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Ftable%2Fupgrade%2FSevenToEightUpgradeHandler.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL3RhYmxlL3VwZ3JhZGUvU2V2ZW5Ub0VpZ2h0VXBncmFkZUhhbmRsZXIuamF2YQ==) | `85.00% <66.66%> (-2.33%)` | :arrow_down: |
| [...ache/hudi/sink/StreamWriteOperatorCoordinator.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-flink-datasource%2Fhudi-flink%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fsink%2FStreamWriteOperatorCoordinator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1mbGluay1kYXRhc291cmNlL2h1ZGktZmxpbmsvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvc2luay9TdHJlYW1Xcml0ZU9wZXJhdG9yQ29vcmRpbmF0b3IuamF2YQ==) | `81.38% <90.47%> (+0.34%)` | :arrow_up: |
... and [167 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18411/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-18410] Optimize LSM timeline migration batch size and compaction during upgrade [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #18411:
URL: https://github.com/apache/hudi/pull/18411#issuecomment-4157964356
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "10ccb3d818b0e218b08db4e0777b531e494a9708",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12900",
"triggerID" : "10ccb3d818b0e218b08db4e0777b531e494a9708",
"triggerType" : "PUSH"
}, {
"hash" : "df05521d76e687ac5e4d09c65e3d94a5d03ae5d6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12928",
"triggerID" : "df05521d76e687ac5e4d09c65e3d94a5d03ae5d6",
"triggerType" : "PUSH"
}, {
"hash" : "cd5f54b2de81575b98c515152c82fd71f950b2b4",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12930",
"triggerID" : "cd5f54b2de81575b98c515152c82fd71f950b2b4",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* cd5f54b2de81575b98c515152c82fd71f950b2b4 Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12930)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-18410] Optimize LSM timeline migration batch size and compaction during upgrade [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18411:
URL: https://github.com/apache/hudi/pull/18411#issuecomment-4157534011
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `87.50000%` with `3 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 67.37%. Comparing base ([`1eb97b3`](https://app.codecov.io/gh/apache/hudi/commit/1eb97b31826eabee4fb9c4bf78dc44d2bdb2dad0?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`cd5f54b`](https://app.codecov.io/gh/apache/hudi/commit/cd5f54b2de81575b98c515152c82fd71f950b2b4?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 6 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/sink/StreamWriteOperatorCoordinator.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-flink-datasource%2Fhudi-flink%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fsink%2FStreamWriteOperatorCoordinator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1mbGluay1kYXRhc291cmNlL2h1ZGktZmxpbmsvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvc2luay9TdHJlYW1Xcml0ZU9wZXJhdG9yQ29vcmRpbmF0b3IuamF2YQ==) | 90.47% | [1 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...hudi/table/upgrade/SevenToEightUpgradeHandler.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Ftable%2Fupgrade%2FSevenToEightUpgradeHandler.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL3RhYmxlL3VwZ3JhZGUvU2V2ZW5Ub0VpZ2h0VXBncmFkZUhhbmRsZXIuamF2YQ==) | 66.66% | [0 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18411 +/- ##
============================================
- Coverage 68.21% 67.37% -0.85%
+ Complexity 27709 27399 -310
============================================
Files 2440 2439 -1
Lines 134249 134358 +109
Branches 16179 16210 +31
============================================
- Hits 91578 90520 -1058
- Misses 35565 36663 +1098
- Partials 7106 7175 +69
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.35% <87.50%> (+0.03%)` | :arrow_up: |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.97% <66.66%> (+0.04%)` | :arrow_up: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.40% <0.00%> (+0.08%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.95% <66.66%> (-3.76%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.29% <0.00%> (+0.06%)` | :arrow_up: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `38.37% <66.66%> (+<0.01%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...hudi/table/upgrade/SevenToEightUpgradeHandler.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Ftable%2Fupgrade%2FSevenToEightUpgradeHandler.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL3RhYmxlL3VwZ3JhZGUvU2V2ZW5Ub0VpZ2h0VXBncmFkZUhhbmRsZXIuamF2YQ==) | `85.00% <66.66%> (-2.33%)` | :arrow_down: |
| [...ache/hudi/sink/StreamWriteOperatorCoordinator.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-flink-datasource%2Fhudi-flink%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fsink%2FStreamWriteOperatorCoordinator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1mbGluay1kYXRhc291cmNlL2h1ZGktZmxpbmsvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvc2luay9TdHJlYW1Xcml0ZU9wZXJhdG9yQ29vcmRpbmF0b3IuamF2YQ==) | `81.38% <90.47%> (+0.34%)` | :arrow_up: |
... and [180 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18411/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-18410] Optimize LSM timeline migration batch size and compaction during upgrade [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18411:
URL: https://github.com/apache/hudi/pull/18411#issuecomment-4157573339
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `87.50000%` with `3 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 68.25%. Comparing base ([`1eb97b3`](https://app.codecov.io/gh/apache/hudi/commit/1eb97b31826eabee4fb9c4bf78dc44d2bdb2dad0?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`cd5f54b`](https://app.codecov.io/gh/apache/hudi/commit/cd5f54b2de81575b98c515152c82fd71f950b2b4?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 6 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/sink/StreamWriteOperatorCoordinator.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-flink-datasource%2Fhudi-flink%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fsink%2FStreamWriteOperatorCoordinator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1mbGluay1kYXRhc291cmNlL2h1ZGktZmxpbmsvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvc2luay9TdHJlYW1Xcml0ZU9wZXJhdG9yQ29vcmRpbmF0b3IuamF2YQ==) | 90.47% | [1 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...hudi/table/upgrade/SevenToEightUpgradeHandler.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Ftable%2Fupgrade%2FSevenToEightUpgradeHandler.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL3RhYmxlL3VwZ3JhZGUvU2V2ZW5Ub0VpZ2h0VXBncmFkZUhhbmRsZXIuamF2YQ==) | 66.66% | [0 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18411 +/- ##
============================================
+ Coverage 68.21% 68.25% +0.03%
- Complexity 27709 27758 +49
============================================
Files 2440 2439 -1
Lines 134249 134358 +109
Branches 16179 16210 +31
============================================
+ Hits 91578 91703 +125
+ Misses 35565 35553 -12
+ Partials 7106 7102 -4
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.35% <87.50%> (+0.03%)` | :arrow_up: |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.97% <66.66%> (+0.04%)` | :arrow_up: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.40% <0.00%> (+0.08%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.67% <66.66%> (-0.04%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.29% <0.00%> (+0.06%)` | :arrow_up: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `38.37% <66.66%> (+<0.01%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...hudi/table/upgrade/SevenToEightUpgradeHandler.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Ftable%2Fupgrade%2FSevenToEightUpgradeHandler.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL3RhYmxlL3VwZ3JhZGUvU2V2ZW5Ub0VpZ2h0VXBncmFkZUhhbmRsZXIuamF2YQ==) | `85.00% <66.66%> (-2.33%)` | :arrow_down: |
| [...ache/hudi/sink/StreamWriteOperatorCoordinator.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-flink-datasource%2Fhudi-flink%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fsink%2FStreamWriteOperatorCoordinator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1mbGluay1kYXRhc291cmNlL2h1ZGktZmxpbmsvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvc2luay9TdHJlYW1Xcml0ZU9wZXJhdG9yQ29vcmRpbmF0b3IuamF2YQ==) | `81.38% <90.47%> (+0.34%)` | :arrow_up: |
... and [43 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18411/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-18410] Optimize LSM timeline migration batch size and compaction during upgrade [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18411:
URL: https://github.com/apache/hudi/pull/18411#issuecomment-4157494319
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `87.50000%` with `3 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 65.77%. Comparing base ([`1eb97b3`](https://app.codecov.io/gh/apache/hudi/commit/1eb97b31826eabee4fb9c4bf78dc44d2bdb2dad0?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`cd5f54b`](https://app.codecov.io/gh/apache/hudi/commit/cd5f54b2de81575b98c515152c82fd71f950b2b4?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 6 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/sink/StreamWriteOperatorCoordinator.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-flink-datasource%2Fhudi-flink%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fsink%2FStreamWriteOperatorCoordinator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1mbGluay1kYXRhc291cmNlL2h1ZGktZmxpbmsvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvc2luay9TdHJlYW1Xcml0ZU9wZXJhdG9yQ29vcmRpbmF0b3IuamF2YQ==) | 90.47% | [1 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...hudi/table/upgrade/SevenToEightUpgradeHandler.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Ftable%2Fupgrade%2FSevenToEightUpgradeHandler.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL3RhYmxlL3VwZ3JhZGUvU2V2ZW5Ub0VpZ2h0VXBncmFkZUhhbmRsZXIuamF2YQ==) | 66.66% | [1 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18411 +/- ##
============================================
- Coverage 68.21% 65.77% -2.45%
+ Complexity 27709 26689 -1020
============================================
Files 2440 2439 -1
Lines 134249 134358 +109
Branches 16179 16210 +31
============================================
- Hits 91578 88371 -3207
- Misses 35565 38679 +3114
- Partials 7106 7308 +202
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.35% <87.50%> (+0.03%)` | :arrow_up: |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.97% <66.66%> (+0.04%)` | :arrow_up: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.40% <0.00%> (+0.08%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `32.48% <66.66%> (-16.23%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.99% <0.00%> (-0.25%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `38.37% <66.66%> (+<0.01%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...hudi/table/upgrade/SevenToEightUpgradeHandler.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Ftable%2Fupgrade%2FSevenToEightUpgradeHandler.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL3RhYmxlL3VwZ3JhZGUvU2V2ZW5Ub0VpZ2h0VXBncmFkZUhhbmRsZXIuamF2YQ==) | `76.42% <66.66%> (-10.90%)` | :arrow_down: |
| [...ache/hudi/sink/StreamWriteOperatorCoordinator.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-flink-datasource%2Fhudi-flink%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fsink%2FStreamWriteOperatorCoordinator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1mbGluay1kYXRhc291cmNlL2h1ZGktZmxpbmsvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvc2luay9TdHJlYW1Xcml0ZU9wZXJhdG9yQ29vcmRpbmF0b3IuamF2YQ==) | `81.38% <90.47%> (+0.34%)` | :arrow_up: |
... and [383 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18411/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-18410] Optimize LSM timeline migration batch size and compaction during upgrade [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18411:
URL: https://github.com/apache/hudi/pull/18411#issuecomment-4157443960
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `66.66667%` with `1 line` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 45.30%. Comparing base ([`1eb97b3`](https://app.codecov.io/gh/apache/hudi/commit/1eb97b31826eabee4fb9c4bf78dc44d2bdb2dad0?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`cd5f54b`](https://app.codecov.io/gh/apache/hudi/commit/cd5f54b2de81575b98c515152c82fd71f950b2b4?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 6 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...hudi/table/upgrade/SevenToEightUpgradeHandler.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Ftable%2Fupgrade%2FSevenToEightUpgradeHandler.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL3RhYmxlL3VwZ3JhZGUvU2V2ZW5Ub0VpZ2h0VXBncmFkZUhhbmRsZXIuamF2YQ==) | 66.66% | [1 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
> :exclamation: There is a different number of reports uploaded between BASE (1eb97b3) and HEAD (cd5f54b). Click for more details.
>
> <details><summary>HEAD has 22 uploads less than BASE</summary>
>
>| Flag | BASE (1eb97b3) | HEAD (cd5f54b) |
>|------|------|------|
>|spark-scala-tests|10|5|
>|spark-java-tests|15|0|
>|common-and-other-modules|1|0|
>|utilities|1|0|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18411 +/- ##
=============================================
- Coverage 68.21% 45.30% -22.92%
+ Complexity 27709 15438 -12271
=============================================
Files 2440 1995 -445
Lines 134249 111150 -23099
Branches 16179 14042 -2137
=============================================
- Hits 91578 50356 -41222
- Misses 35565 55399 +19834
+ Partials 7106 5395 -1711
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.97% <66.66%> (+0.04%)` | :arrow_up: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.40% <0.00%> (+0.08%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `32.69% <0.00%> (-12.55%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...hudi/table/upgrade/SevenToEightUpgradeHandler.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Ftable%2Fupgrade%2FSevenToEightUpgradeHandler.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL3RhYmxlL3VwZ3JhZGUvU2V2ZW5Ub0VpZ2h0VXBncmFkZUhhbmRsZXIuamF2YQ==) | `57.85% <66.66%> (-29.47%)` | :arrow_down: |
... and [1561 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18411/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-18410] Optimize LSM timeline migration batch size and compaction during upgrade [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #18411:
URL: https://github.com/apache/hudi/pull/18411#issuecomment-4195011406
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "10ccb3d818b0e218b08db4e0777b531e494a9708",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12900",
"triggerID" : "10ccb3d818b0e218b08db4e0777b531e494a9708",
"triggerType" : "PUSH"
}, {
"hash" : "df05521d76e687ac5e4d09c65e3d94a5d03ae5d6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12928",
"triggerID" : "df05521d76e687ac5e4d09c65e3d94a5d03ae5d6",
"triggerType" : "PUSH"
}, {
"hash" : "cd5f54b2de81575b98c515152c82fd71f950b2b4",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12930",
"triggerID" : "cd5f54b2de81575b98c515152c82fd71f950b2b4",
"triggerType" : "PUSH"
}, {
"hash" : "5c078d876a231d32e7127c802e2f6197c65bfee7",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "5c078d876a231d32e7127c802e2f6197c65bfee7",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* cd5f54b2de81575b98c515152c82fd71f950b2b4 Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12930)
* 5c078d876a231d32e7127c802e2f6197c65bfee7 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-18410] Fix Flink upgrade blocking dispatcher thread causing heartbeat timeout [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18411:
URL: https://github.com/apache/hudi/pull/18411#issuecomment-4149221579
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `66.66667%` with `1 line` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 54.47%. Comparing base ([`1eb97b3`](https://app.codecov.io/gh/apache/hudi/commit/1eb97b31826eabee4fb9c4bf78dc44d2bdb2dad0?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`10ccb3d`](https://app.codecov.io/gh/apache/hudi/commit/10ccb3d818b0e218b08db4e0777b531e494a9708?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...hudi/table/upgrade/SevenToEightUpgradeHandler.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Ftable%2Fupgrade%2FSevenToEightUpgradeHandler.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL3RhYmxlL3VwZ3JhZGUvU2V2ZW5Ub0VpZ2h0VXBncmFkZUhhbmRsZXIuamF2YQ==) | 66.66% | [1 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
> :exclamation: There is a different number of reports uploaded between BASE (1eb97b3) and HEAD (10ccb3d). Click for more details.
>
> <details><summary>HEAD has 16 uploads less than BASE</summary>
>
>| Flag | BASE (1eb97b3) | HEAD (10ccb3d) |
>|------|------|------|
>|spark-scala-tests|10|5|
>|spark-java-tests|15|5|
>|common-and-other-modules|1|0|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18411 +/- ##
=============================================
- Coverage 68.21% 54.47% -13.75%
+ Complexity 27709 18285 -9424
=============================================
Files 2440 1996 -444
Lines 134249 111058 -23191
Branches 16179 14012 -2167
=============================================
- Hits 91578 60498 -31080
- Misses 35565 44389 +8824
+ Partials 7106 6171 -935
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.98% <66.66%> (+0.05%)` | :arrow_up: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.32% <0.00%> (+<0.01%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `32.49% <66.66%> (-16.22%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `32.59% <0.00%> (-12.65%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `38.38% <66.66%> (+<0.01%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...hudi/table/upgrade/SevenToEightUpgradeHandler.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Ftable%2Fupgrade%2FSevenToEightUpgradeHandler.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL3RhYmxlL3VwZ3JhZGUvU2V2ZW5Ub0VpZ2h0VXBncmFkZUhhbmRsZXIuamF2YQ==) | `63.57% <66.66%> (-23.76%)` | :arrow_down: |
... and [1374 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18411/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-18410] Fix Flink upgrade blocking dispatcher thread causing heartbeat timeout [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18411:
URL: https://github.com/apache/hudi/pull/18411#issuecomment-4149218533
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `66.66667%` with `1 line` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 48.09%. Comparing base ([`1eb97b3`](https://app.codecov.io/gh/apache/hudi/commit/1eb97b31826eabee4fb9c4bf78dc44d2bdb2dad0?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`10ccb3d`](https://app.codecov.io/gh/apache/hudi/commit/10ccb3d818b0e218b08db4e0777b531e494a9708?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...hudi/table/upgrade/SevenToEightUpgradeHandler.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Ftable%2Fupgrade%2FSevenToEightUpgradeHandler.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL3RhYmxlL3VwZ3JhZGUvU2V2ZW5Ub0VpZ2h0VXBncmFkZUhhbmRsZXIuamF2YQ==) | 66.66% | [1 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
> :exclamation: There is a different number of reports uploaded between BASE (1eb97b3) and HEAD (10ccb3d). Click for more details.
>
> <details><summary>HEAD has 18 uploads less than BASE</summary>
>
>| Flag | BASE (1eb97b3) | HEAD (10ccb3d) |
>|------|------|------|
>|spark-scala-tests|10|5|
>|spark-java-tests|15|4|
>|common-and-other-modules|1|0|
>|utilities|1|0|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18411 +/- ##
=============================================
- Coverage 68.21% 48.09% -20.13%
+ Complexity 27709 16327 -11382
=============================================
Files 2440 1996 -444
Lines 134249 111058 -23191
Branches 16179 14012 -2167
=============================================
- Hits 91578 53408 -38170
- Misses 35565 52093 +16528
+ Partials 7106 5557 -1549
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.98% <66.66%> (+0.05%)` | :arrow_up: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.32% <0.00%> (+<0.01%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `32.87% <66.66%> (-15.84%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `32.59% <0.00%> (-12.65%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...hudi/table/upgrade/SevenToEightUpgradeHandler.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Ftable%2Fupgrade%2FSevenToEightUpgradeHandler.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL3RhYmxlL3VwZ3JhZGUvU2V2ZW5Ub0VpZ2h0VXBncmFkZUhhbmRsZXIuamF2YQ==) | `62.14% <66.66%> (-25.19%)` | :arrow_down: |
... and [1511 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18411/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-18410] Fix Flink upgrade blocking dispatcher thread causing heartbeat timeout [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #18411:
URL: https://github.com/apache/hudi/pull/18411#issuecomment-4149277926
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "10ccb3d818b0e218b08db4e0777b531e494a9708",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12900",
"triggerID" : "10ccb3d818b0e218b08db4e0777b531e494a9708",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 10ccb3d818b0e218b08db4e0777b531e494a9708 Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12900)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-18410] Optimize LSM timeline migration batch size and compaction during upgrade [hudi]
Posted by "yihua (via GitHub)" <gi...@apache.org>.
yihua commented on code in PR #18411:
URL: https://github.com/apache/hudi/pull/18411#discussion_r3041746722
##########
hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/sink/StreamWriteOperatorCoordinator.java:
##########
@@ -244,17 +245,44 @@ public void start() throws Exception {
.threadFactory(getThreadFactory("instant-request"))
.exceptionHook((errMsg, t) -> this.context.failJob(new HoodieException(errMsg, t)))
.build();
- // start the executor if required
- if (tableState.syncHive) {
- initHiveSync();
- }
- // start client id heartbeats for optimistic concurrency control
- if (OptionsResolver.isMultiWriter(conf)) {
- initClientIds(conf);
- }
- restoreEvents();
+
+ // Run upgrade, metadata table init, and event restoration on the executor
+ // thread instead of the Pekko dispatcher thread. Running these synchronously
+ // on the dispatcher thread blocks heartbeat responses when the operations
+ // involve heavy I/O (e.g., LSM timeline migration with hundreds of archived
+ // actions), causing the ResourceManager to disconnect the JobManager.
+ //
+ // Safety guarantees:
+ // - Events via handleEventFromOperator() are submitted to the same
+ // single-threaded executor, so FIFO ordering ensures initialization
+ // completes before any event processing.
+ // - Coordination requests via handleCoordinationRequest() run on the
+ // separate instantRequestExecutor and are gated on initFuture to
+ // prevent startInstant() from racing ahead of the upgrade.
+ this.executor.execute(() -> {
+ try {
+ this.writeClient.tryUpgrade(instant, this.metaClient);
+ initMetadataTable(this.writeClient);
+ if (tableState.scheduleMdtCompaction) {
+ this.metadataWriteClient = StreamerUtil.createMetadataWriteClient(writeClient);
+ }
+ if (tableState.syncHive) {
+ initHiveSync();
+ }
+ if (OptionsResolver.isMultiWriter(conf)) {
+ initClientIds(conf);
+ }
+ restoreEvents();
+ initFuture.complete(null);
+ } catch (Throwable t) {
+ initFuture.completeExceptionally(t);
+ throw t;
+ }
Review Comment:
🤖 The `throw t` here is swallowed by `NonThrownExecutor`, so when async init fails (e.g. `tryUpgrade` throws), `initFuture` is completed exceptionally but `context.failJob(t)` is never called — unlike the synchronous failure path in the outer `catch` block which does call it. Could you add `context.failJob(t)` here so the Flink job is proactively terminated on init failure, rather than silently waiting for the first coordination request to discover the broken future?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-18410] Optimize LSM timeline migration batch size and compaction during upgrade [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #18411:
URL: https://github.com/apache/hudi/pull/18411#issuecomment-4195024644
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "10ccb3d818b0e218b08db4e0777b531e494a9708",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12900",
"triggerID" : "10ccb3d818b0e218b08db4e0777b531e494a9708",
"triggerType" : "PUSH"
}, {
"hash" : "df05521d76e687ac5e4d09c65e3d94a5d03ae5d6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12928",
"triggerID" : "df05521d76e687ac5e4d09c65e3d94a5d03ae5d6",
"triggerType" : "PUSH"
}, {
"hash" : "cd5f54b2de81575b98c515152c82fd71f950b2b4",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12930",
"triggerID" : "cd5f54b2de81575b98c515152c82fd71f950b2b4",
"triggerType" : "PUSH"
}, {
"hash" : "5c078d876a231d32e7127c802e2f6197c65bfee7",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=13102",
"triggerID" : "5c078d876a231d32e7127c802e2f6197c65bfee7",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* cd5f54b2de81575b98c515152c82fd71f950b2b4 Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12930)
* 5c078d876a231d32e7127c802e2f6197c65bfee7 Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=13102)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-18410] Optimize LSM timeline migration batch size and compaction during upgrade [hudi]
Posted by "prashantwason (via GitHub)" <gi...@apache.org>.
prashantwason commented on PR #18411:
URL: https://github.com/apache/hudi/pull/18411#issuecomment-4157169218
@danny0405 @cshuo I've pushed a revised async approach that addresses both concerns:
**The race condition is fixed** — `handleCoordinationRequest()` is now gated on `initFuture.thenCompose()`, so `startInstant()` on the `instantRequestExecutor` cannot race ahead of initialization. The `initFuture` only completes after `tryUpgrade()`, `initMetadataTable()`, and `restoreEvents()` finish.
**No inconsistency from the write task's perspective** — both paths are covered:
1. **Events** (`handleEventFromOperator`): submitted to the same single-threaded `executor`, so FIFO ordering guarantees initialization completes before any event processing
2. **Coordination requests** (`handleCoordinationRequest`): gated on `initFuture`, so they block until initialization completes — the `RecordIndexPartitioner` MDT dependency is safe
The key difference from the previous attempt: instead of letting coordination requests run immediately while init is still in progress, they now wait for init to complete via `CompletableFuture` composition. This gives us the benefit of not blocking the dispatcher thread while still ensuring the coordinator is fully initialized before any write operations begin.
All 36 `TestStreamWriteOperatorCoordinator` tests pass.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-18410] Optimize LSM timeline migration batch size and compaction during upgrade [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18411:
URL: https://github.com/apache/hudi/pull/18411#issuecomment-4157461233
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `66.66667%` with `1 line` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 48.18%. Comparing base ([`1eb97b3`](https://app.codecov.io/gh/apache/hudi/commit/1eb97b31826eabee4fb9c4bf78dc44d2bdb2dad0?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`cd5f54b`](https://app.codecov.io/gh/apache/hudi/commit/cd5f54b2de81575b98c515152c82fd71f950b2b4?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 6 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...hudi/table/upgrade/SevenToEightUpgradeHandler.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Ftable%2Fupgrade%2FSevenToEightUpgradeHandler.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL3RhYmxlL3VwZ3JhZGUvU2V2ZW5Ub0VpZ2h0VXBncmFkZUhhbmRsZXIuamF2YQ==) | 66.66% | [1 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
> :exclamation: There is a different number of reports uploaded between BASE (1eb97b3) and HEAD (cd5f54b). Click for more details.
>
> <details><summary>HEAD has 19 uploads less than BASE</summary>
>
>| Flag | BASE (1eb97b3) | HEAD (cd5f54b) |
>|------|------|------|
>|spark-scala-tests|10|5|
>|spark-java-tests|15|3|
>|common-and-other-modules|1|0|
>|utilities|1|0|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18411 +/- ##
=============================================
- Coverage 68.21% 48.18% -20.04%
+ Complexity 27709 16366 -11343
=============================================
Files 2440 1995 -445
Lines 134249 111150 -23099
Branches 16179 14042 -2137
=============================================
- Hits 91578 53557 -38021
- Misses 35565 51999 +16434
+ Partials 7106 5594 -1512
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.97% <66.66%> (+0.04%)` | :arrow_up: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.40% <0.00%> (+0.08%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `32.87% <66.66%> (-15.84%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `32.69% <0.00%> (-12.55%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...hudi/table/upgrade/SevenToEightUpgradeHandler.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Ftable%2Fupgrade%2FSevenToEightUpgradeHandler.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL3RhYmxlL3VwZ3JhZGUvU2V2ZW5Ub0VpZ2h0VXBncmFkZUhhbmRsZXIuamF2YQ==) | `62.14% <66.66%> (-25.19%)` | :arrow_down: |
... and [1512 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18411/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-18410] Optimize LSM timeline migration batch size and compaction during upgrade [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #18411:
URL: https://github.com/apache/hudi/pull/18411#issuecomment-4157582277
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "10ccb3d818b0e218b08db4e0777b531e494a9708",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12900",
"triggerID" : "10ccb3d818b0e218b08db4e0777b531e494a9708",
"triggerType" : "PUSH"
}, {
"hash" : "df05521d76e687ac5e4d09c65e3d94a5d03ae5d6",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12928",
"triggerID" : "df05521d76e687ac5e4d09c65e3d94a5d03ae5d6",
"triggerType" : "PUSH"
}, {
"hash" : "cd5f54b2de81575b98c515152c82fd71f950b2b4",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12930",
"triggerID" : "cd5f54b2de81575b98c515152c82fd71f950b2b4",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* df05521d76e687ac5e4d09c65e3d94a5d03ae5d6 Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12928)
* cd5f54b2de81575b98c515152c82fd71f950b2b4 Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12930)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-18410] Optimize LSM timeline migration batch size and compaction during upgrade [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18411:
URL: https://github.com/apache/hudi/pull/18411#issuecomment-4157696755
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `87.50000%` with `3 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 68.25%. Comparing base ([`1eb97b3`](https://app.codecov.io/gh/apache/hudi/commit/1eb97b31826eabee4fb9c4bf78dc44d2bdb2dad0?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`cd5f54b`](https://app.codecov.io/gh/apache/hudi/commit/cd5f54b2de81575b98c515152c82fd71f950b2b4?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 6 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/sink/StreamWriteOperatorCoordinator.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-flink-datasource%2Fhudi-flink%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fsink%2FStreamWriteOperatorCoordinator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1mbGluay1kYXRhc291cmNlL2h1ZGktZmxpbmsvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvc2luay9TdHJlYW1Xcml0ZU9wZXJhdG9yQ29vcmRpbmF0b3IuamF2YQ==) | 90.47% | [1 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...hudi/table/upgrade/SevenToEightUpgradeHandler.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Ftable%2Fupgrade%2FSevenToEightUpgradeHandler.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL3RhYmxlL3VwZ3JhZGUvU2V2ZW5Ub0VpZ2h0VXBncmFkZUhhbmRsZXIuamF2YQ==) | 66.66% | [0 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18411 +/- ##
============================================
+ Coverage 68.21% 68.25% +0.04%
- Complexity 27709 27760 +51
============================================
Files 2440 2439 -1
Lines 134249 134358 +109
Branches 16179 16210 +31
============================================
+ Hits 91578 91711 +133
+ Misses 35565 35547 -18
+ Partials 7106 7100 -6
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.35% <87.50%> (+0.03%)` | :arrow_up: |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.97% <66.66%> (+0.04%)` | :arrow_up: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.40% <0.00%> (+0.08%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.78% <66.66%> (+0.07%)` | :arrow_up: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.29% <0.00%> (+0.06%)` | :arrow_up: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `38.37% <66.66%> (+<0.01%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...hudi/table/upgrade/SevenToEightUpgradeHandler.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Ftable%2Fupgrade%2FSevenToEightUpgradeHandler.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL3RhYmxlL3VwZ3JhZGUvU2V2ZW5Ub0VpZ2h0VXBncmFkZUhhbmRsZXIuamF2YQ==) | `85.00% <66.66%> (-2.33%)` | :arrow_down: |
| [...ache/hudi/sink/StreamWriteOperatorCoordinator.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-flink-datasource%2Fhudi-flink%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fsink%2FStreamWriteOperatorCoordinator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1mbGluay1kYXRhc291cmNlL2h1ZGktZmxpbmsvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvc2luay9TdHJlYW1Xcml0ZU9wZXJhdG9yQ29vcmRpbmF0b3IuamF2YQ==) | `81.38% <90.47%> (+0.34%)` | :arrow_up: |
... and [42 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18411/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-18410] Optimize LSM timeline migration batch size and compaction during upgrade [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18411:
URL: https://github.com/apache/hudi/pull/18411#issuecomment-4157527758
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `87.50000%` with `3 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 65.82%. Comparing base ([`1eb97b3`](https://app.codecov.io/gh/apache/hudi/commit/1eb97b31826eabee4fb9c4bf78dc44d2bdb2dad0?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`cd5f54b`](https://app.codecov.io/gh/apache/hudi/commit/cd5f54b2de81575b98c515152c82fd71f950b2b4?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 6 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/sink/StreamWriteOperatorCoordinator.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-flink-datasource%2Fhudi-flink%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fsink%2FStreamWriteOperatorCoordinator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1mbGluay1kYXRhc291cmNlL2h1ZGktZmxpbmsvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvc2luay9TdHJlYW1Xcml0ZU9wZXJhdG9yQ29vcmRpbmF0b3IuamF2YQ==) | 90.47% | [1 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...hudi/table/upgrade/SevenToEightUpgradeHandler.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Ftable%2Fupgrade%2FSevenToEightUpgradeHandler.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL3RhYmxlL3VwZ3JhZGUvU2V2ZW5Ub0VpZ2h0VXBncmFkZUhhbmRsZXIuamF2YQ==) | 66.66% | [1 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18411 +/- ##
============================================
- Coverage 68.21% 65.82% -2.40%
+ Complexity 27709 26708 -1001
============================================
Files 2440 2439 -1
Lines 134249 134358 +109
Branches 16179 16210 +31
============================================
- Hits 91578 88436 -3142
- Misses 35565 38608 +3043
- Partials 7106 7314 +208
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.35% <87.50%> (+0.03%)` | :arrow_up: |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.97% <66.66%> (+0.04%)` | :arrow_up: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.40% <0.00%> (+0.08%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `32.48% <66.66%> (-16.23%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.10% <0.00%> (-0.14%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `38.37% <66.66%> (+<0.01%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...hudi/table/upgrade/SevenToEightUpgradeHandler.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Ftable%2Fupgrade%2FSevenToEightUpgradeHandler.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL3RhYmxlL3VwZ3JhZGUvU2V2ZW5Ub0VpZ2h0VXBncmFkZUhhbmRsZXIuamF2YQ==) | `76.42% <66.66%> (-10.90%)` | :arrow_down: |
| [...ache/hudi/sink/StreamWriteOperatorCoordinator.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-flink-datasource%2Fhudi-flink%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fsink%2FStreamWriteOperatorCoordinator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1mbGluay1kYXRhc291cmNlL2h1ZGktZmxpbmsvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvc2luay9TdHJlYW1Xcml0ZU9wZXJhdG9yQ29vcmRpbmF0b3IuamF2YQ==) | `81.38% <90.47%> (+0.34%)` | :arrow_up: |
... and [372 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18411/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-18410] Optimize LSM timeline migration batch size and compaction during upgrade [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18411:
URL: https://github.com/apache/hudi/pull/18411#issuecomment-4157535392
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `87.50000%` with `3 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 67.37%. Comparing base ([`1eb97b3`](https://app.codecov.io/gh/apache/hudi/commit/1eb97b31826eabee4fb9c4bf78dc44d2bdb2dad0?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`cd5f54b`](https://app.codecov.io/gh/apache/hudi/commit/cd5f54b2de81575b98c515152c82fd71f950b2b4?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 6 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/sink/StreamWriteOperatorCoordinator.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-flink-datasource%2Fhudi-flink%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fsink%2FStreamWriteOperatorCoordinator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1mbGluay1kYXRhc291cmNlL2h1ZGktZmxpbmsvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvc2luay9TdHJlYW1Xcml0ZU9wZXJhdG9yQ29vcmRpbmF0b3IuamF2YQ==) | 90.47% | [1 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...hudi/table/upgrade/SevenToEightUpgradeHandler.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Ftable%2Fupgrade%2FSevenToEightUpgradeHandler.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL3RhYmxlL3VwZ3JhZGUvU2V2ZW5Ub0VpZ2h0VXBncmFkZUhhbmRsZXIuamF2YQ==) | 66.66% | [0 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18411 +/- ##
============================================
- Coverage 68.21% 67.37% -0.85%
+ Complexity 27709 27399 -310
============================================
Files 2440 2439 -1
Lines 134249 134358 +109
Branches 16179 16210 +31
============================================
- Hits 91578 90520 -1058
- Misses 35565 36663 +1098
- Partials 7106 7175 +69
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.35% <87.50%> (+0.03%)` | :arrow_up: |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.97% <66.66%> (+0.04%)` | :arrow_up: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.40% <0.00%> (+0.08%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.95% <66.66%> (-3.76%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.29% <0.00%> (+0.06%)` | :arrow_up: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `38.37% <66.66%> (+<0.01%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...hudi/table/upgrade/SevenToEightUpgradeHandler.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Ftable%2Fupgrade%2FSevenToEightUpgradeHandler.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL3RhYmxlL3VwZ3JhZGUvU2V2ZW5Ub0VpZ2h0VXBncmFkZUhhbmRsZXIuamF2YQ==) | `85.00% <66.66%> (-2.33%)` | :arrow_down: |
| [...ache/hudi/sink/StreamWriteOperatorCoordinator.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-flink-datasource%2Fhudi-flink%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fsink%2FStreamWriteOperatorCoordinator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1mbGluay1kYXRhc291cmNlL2h1ZGktZmxpbmsvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvc2luay9TdHJlYW1Xcml0ZU9wZXJhdG9yQ29vcmRpbmF0b3IuamF2YQ==) | `81.38% <90.47%> (+0.34%)` | :arrow_up: |
... and [180 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18411/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-18410] Fix Flink upgrade blocking dispatcher thread causing heartbeat timeout [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18411:
URL: https://github.com/apache/hudi/pull/18411#issuecomment-4149225847
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `66.66667%` with `1 line` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 60.64%. Comparing base ([`1eb97b3`](https://app.codecov.io/gh/apache/hudi/commit/1eb97b31826eabee4fb9c4bf78dc44d2bdb2dad0?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`10ccb3d`](https://app.codecov.io/gh/apache/hudi/commit/10ccb3d818b0e218b08db4e0777b531e494a9708?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...hudi/table/upgrade/SevenToEightUpgradeHandler.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Ftable%2Fupgrade%2FSevenToEightUpgradeHandler.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL3RhYmxlL3VwZ3JhZGUvU2V2ZW5Ub0VpZ2h0VXBncmFkZUhhbmRsZXIuamF2YQ==) | 66.66% | [1 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
> :exclamation: There is a different number of reports uploaded between BASE (1eb97b3) and HEAD (10ccb3d). Click for more details.
>
> <details><summary>HEAD has 15 uploads less than BASE</summary>
>
>| Flag | BASE (1eb97b3) | HEAD (10ccb3d) |
>|------|------|------|
>|spark-scala-tests|10|6|
>|spark-java-tests|15|5|
>|common-and-other-modules|1|0|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18411 +/- ##
============================================
- Coverage 68.21% 60.64% -7.58%
+ Complexity 27709 20364 -7345
============================================
Files 2440 1996 -444
Lines 134249 111058 -23191
Branches 16179 14012 -2167
============================================
- Hits 91578 67351 -24227
- Misses 35565 37104 +1539
+ Partials 7106 6603 -503
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.98% <66.66%> (+0.05%)` | :arrow_up: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.32% <0.00%> (+<0.01%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `32.49% <66.66%> (-16.22%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.95% <0.00%> (-0.29%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `38.38% <66.66%> (+<0.01%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...hudi/table/upgrade/SevenToEightUpgradeHandler.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Ftable%2Fupgrade%2FSevenToEightUpgradeHandler.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL3RhYmxlL3VwZ3JhZGUvU2V2ZW5Ub0VpZ2h0VXBncmFkZUhhbmRsZXIuamF2YQ==) | `63.57% <66.66%> (-23.76%)` | :arrow_down: |
... and [1220 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18411/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-18410] Fix Flink upgrade blocking dispatcher thread causing heartbeat timeout [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18411:
URL: https://github.com/apache/hudi/pull/18411#issuecomment-4149228759
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `66.66667%` with `1 line` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 60.70%. Comparing base ([`1eb97b3`](https://app.codecov.io/gh/apache/hudi/commit/1eb97b31826eabee4fb9c4bf78dc44d2bdb2dad0?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`10ccb3d`](https://app.codecov.io/gh/apache/hudi/commit/10ccb3d818b0e218b08db4e0777b531e494a9708?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...hudi/table/upgrade/SevenToEightUpgradeHandler.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Ftable%2Fupgrade%2FSevenToEightUpgradeHandler.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL3RhYmxlL3VwZ3JhZGUvU2V2ZW5Ub0VpZ2h0VXBncmFkZUhhbmRsZXIuamF2YQ==) | 66.66% | [1 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
> :exclamation: There is a different number of reports uploaded between BASE (1eb97b3) and HEAD (10ccb3d). Click for more details.
>
> <details><summary>HEAD has 14 uploads less than BASE</summary>
>
>| Flag | BASE (1eb97b3) | HEAD (10ccb3d) |
>|------|------|------|
>|spark-scala-tests|10|7|
>|spark-java-tests|15|5|
>|common-and-other-modules|1|0|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18411 +/- ##
============================================
- Coverage 68.21% 60.70% -7.51%
+ Complexity 27709 20383 -7326
============================================
Files 2440 1996 -444
Lines 134249 111058 -23191
Branches 16179 14012 -2167
============================================
- Hits 91578 67418 -24160
- Misses 35565 37032 +1467
+ Partials 7106 6608 -498
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.98% <66.66%> (+0.05%)` | :arrow_up: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.32% <0.00%> (+<0.01%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `32.49% <66.66%> (-16.22%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.05% <0.00%> (-0.19%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `38.38% <66.66%> (+<0.01%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...hudi/table/upgrade/SevenToEightUpgradeHandler.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Ftable%2Fupgrade%2FSevenToEightUpgradeHandler.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL3RhYmxlL3VwZ3JhZGUvU2V2ZW5Ub0VpZ2h0VXBncmFkZUhhbmRsZXIuamF2YQ==) | `63.57% <66.66%> (-23.76%)` | :arrow_down: |
... and [1209 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18411/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-18410] Fix Flink upgrade blocking dispatcher thread causing heartbeat timeout [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18411:
URL: https://github.com/apache/hudi/pull/18411#issuecomment-4149262483
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `66.66667%` with `1 line` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 66.02%. Comparing base ([`1eb97b3`](https://app.codecov.io/gh/apache/hudi/commit/1eb97b31826eabee4fb9c4bf78dc44d2bdb2dad0?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`10ccb3d`](https://app.codecov.io/gh/apache/hudi/commit/10ccb3d818b0e218b08db4e0777b531e494a9708?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...hudi/table/upgrade/SevenToEightUpgradeHandler.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Ftable%2Fupgrade%2FSevenToEightUpgradeHandler.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL3RhYmxlL3VwZ3JhZGUvU2V2ZW5Ub0VpZ2h0VXBncmFkZUhhbmRsZXIuamF2YQ==) | 66.66% | [0 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18411 +/- ##
============================================
- Coverage 68.21% 66.02% -2.19%
+ Complexity 27709 22470 -5239
============================================
Files 2440 1996 -444
Lines 134249 111058 -23191
Branches 16179 14012 -2167
============================================
- Hits 91578 73327 -18251
+ Misses 35565 31490 -4075
+ Partials 7106 6241 -865
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.98% <66.66%> (+0.05%)` | :arrow_up: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.32% <0.00%> (+<0.01%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.66% <66.66%> (-0.05%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.25% <0.00%> (+0.01%)` | :arrow_up: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `38.38% <66.66%> (+<0.01%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...hudi/table/upgrade/SevenToEightUpgradeHandler.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Ftable%2Fupgrade%2FSevenToEightUpgradeHandler.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL3RhYmxlL3VwZ3JhZGUvU2V2ZW5Ub0VpZ2h0VXBncmFkZUhhbmRsZXIuamF2YQ==) | `77.85% <66.66%> (-9.47%)` | :arrow_down: |
... and [794 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18411/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-18410] Fix Flink upgrade blocking dispatcher thread causing heartbeat timeout [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #18411:
URL: https://github.com/apache/hudi/pull/18411#issuecomment-4149167314
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "10ccb3d818b0e218b08db4e0777b531e494a9708",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12900",
"triggerID" : "10ccb3d818b0e218b08db4e0777b531e494a9708",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 10ccb3d818b0e218b08db4e0777b531e494a9708 Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12900)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-18410] Fix Flink upgrade blocking dispatcher thread causing heartbeat timeout [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18411:
URL: https://github.com/apache/hudi/pull/18411#issuecomment-4149239393
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `66.66667%` with `1 line` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 66.02%. Comparing base ([`1eb97b3`](https://app.codecov.io/gh/apache/hudi/commit/1eb97b31826eabee4fb9c4bf78dc44d2bdb2dad0?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`10ccb3d`](https://app.codecov.io/gh/apache/hudi/commit/10ccb3d818b0e218b08db4e0777b531e494a9708?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...hudi/table/upgrade/SevenToEightUpgradeHandler.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Ftable%2Fupgrade%2FSevenToEightUpgradeHandler.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL3RhYmxlL3VwZ3JhZGUvU2V2ZW5Ub0VpZ2h0VXBncmFkZUhhbmRsZXIuamF2YQ==) | 66.66% | [0 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18411 +/- ##
============================================
- Coverage 68.21% 66.02% -2.20%
+ Complexity 27709 22470 -5239
============================================
Files 2440 1996 -444
Lines 134249 111058 -23191
Branches 16179 14012 -2167
============================================
- Hits 91578 73324 -18254
+ Misses 35565 31493 -4072
+ Partials 7106 6241 -865
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.98% <66.66%> (+0.05%)` | :arrow_up: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.32% <0.00%> (+<0.01%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.63% <66.66%> (-0.09%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.24% <0.00%> (+0.01%)` | :arrow_up: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `38.38% <66.66%> (+<0.01%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...hudi/table/upgrade/SevenToEightUpgradeHandler.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Ftable%2Fupgrade%2FSevenToEightUpgradeHandler.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL3RhYmxlL3VwZ3JhZGUvU2V2ZW5Ub0VpZ2h0VXBncmFkZUhhbmRsZXIuamF2YQ==) | `77.85% <66.66%> (-9.47%)` | :arrow_down: |
... and [797 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18411/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-18410] Fix Flink upgrade blocking dispatcher thread causing heartbeat timeout [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18411:
URL: https://github.com/apache/hudi/pull/18411#issuecomment-4149239919
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `66.66667%` with `1 line` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 66.02%. Comparing base ([`1eb97b3`](https://app.codecov.io/gh/apache/hudi/commit/1eb97b31826eabee4fb9c4bf78dc44d2bdb2dad0?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`10ccb3d`](https://app.codecov.io/gh/apache/hudi/commit/10ccb3d818b0e218b08db4e0777b531e494a9708?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...hudi/table/upgrade/SevenToEightUpgradeHandler.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Ftable%2Fupgrade%2FSevenToEightUpgradeHandler.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL3RhYmxlL3VwZ3JhZGUvU2V2ZW5Ub0VpZ2h0VXBncmFkZUhhbmRsZXIuamF2YQ==) | 66.66% | [0 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18411 +/- ##
============================================
- Coverage 68.21% 66.02% -2.20%
+ Complexity 27709 22470 -5239
============================================
Files 2440 1996 -444
Lines 134249 111058 -23191
Branches 16179 14012 -2167
============================================
- Hits 91578 73324 -18254
+ Misses 35565 31493 -4072
+ Partials 7106 6241 -865
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.98% <66.66%> (+0.05%)` | :arrow_up: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.32% <0.00%> (+<0.01%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.63% <66.66%> (-0.09%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.25% <0.00%> (+0.01%)` | :arrow_up: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `38.38% <66.66%> (+<0.01%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...hudi/table/upgrade/SevenToEightUpgradeHandler.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Ftable%2Fupgrade%2FSevenToEightUpgradeHandler.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL3RhYmxlL3VwZ3JhZGUvU2V2ZW5Ub0VpZ2h0VXBncmFkZUhhbmRsZXIuamF2YQ==) | `77.85% <66.66%> (-9.47%)` | :arrow_down: |
... and [797 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18411/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-18410] Fix Flink upgrade blocking dispatcher thread causing heartbeat timeout [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18411:
URL: https://github.com/apache/hudi/pull/18411#issuecomment-4149256859
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `66.66667%` with `1 line` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 66.02%. Comparing base ([`1eb97b3`](https://app.codecov.io/gh/apache/hudi/commit/1eb97b31826eabee4fb9c4bf78dc44d2bdb2dad0?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`10ccb3d`](https://app.codecov.io/gh/apache/hudi/commit/10ccb3d818b0e218b08db4e0777b531e494a9708?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...hudi/table/upgrade/SevenToEightUpgradeHandler.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Ftable%2Fupgrade%2FSevenToEightUpgradeHandler.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL3RhYmxlL3VwZ3JhZGUvU2V2ZW5Ub0VpZ2h0VXBncmFkZUhhbmRsZXIuamF2YQ==) | 66.66% | [0 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18411 +/- ##
============================================
- Coverage 68.21% 66.02% -2.19%
+ Complexity 27709 22470 -5239
============================================
Files 2440 1996 -444
Lines 134249 111058 -23191
Branches 16179 14012 -2167
============================================
- Hits 91578 73327 -18251
+ Misses 35565 31490 -4075
+ Partials 7106 6241 -865
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.98% <66.66%> (+0.05%)` | :arrow_up: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.32% <0.00%> (+<0.01%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.66% <66.66%> (-0.05%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.25% <0.00%> (+0.01%)` | :arrow_up: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `38.38% <66.66%> (+<0.01%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...hudi/table/upgrade/SevenToEightUpgradeHandler.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Ftable%2Fupgrade%2FSevenToEightUpgradeHandler.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL3RhYmxlL3VwZ3JhZGUvU2V2ZW5Ub0VpZ2h0VXBncmFkZUhhbmRsZXIuamF2YQ==) | `77.85% <66.66%> (-9.47%)` | :arrow_down: |
... and [794 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18411/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-18410] Fix Flink upgrade blocking dispatcher thread causing heartbeat timeout [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18411:
URL: https://github.com/apache/hudi/pull/18411#issuecomment-4149267599
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `66.66667%` with `1 line` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 66.02%. Comparing base ([`1eb97b3`](https://app.codecov.io/gh/apache/hudi/commit/1eb97b31826eabee4fb9c4bf78dc44d2bdb2dad0?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`10ccb3d`](https://app.codecov.io/gh/apache/hudi/commit/10ccb3d818b0e218b08db4e0777b531e494a9708?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...hudi/table/upgrade/SevenToEightUpgradeHandler.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Ftable%2Fupgrade%2FSevenToEightUpgradeHandler.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL3RhYmxlL3VwZ3JhZGUvU2V2ZW5Ub0VpZ2h0VXBncmFkZUhhbmRsZXIuamF2YQ==) | 66.66% | [0 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18411 +/- ##
============================================
- Coverage 68.21% 66.02% -2.19%
+ Complexity 27709 22470 -5239
============================================
Files 2440 1996 -444
Lines 134249 111058 -23191
Branches 16179 14012 -2167
============================================
- Hits 91578 73327 -18251
+ Misses 35565 31490 -4075
+ Partials 7106 6241 -865
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.98% <66.66%> (+0.05%)` | :arrow_up: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.32% <0.00%> (+<0.01%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.69% <66.66%> (-0.02%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.25% <0.00%> (+0.01%)` | :arrow_up: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `38.38% <66.66%> (+<0.01%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...hudi/table/upgrade/SevenToEightUpgradeHandler.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Ftable%2Fupgrade%2FSevenToEightUpgradeHandler.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL3RhYmxlL3VwZ3JhZGUvU2V2ZW5Ub0VpZ2h0VXBncmFkZUhhbmRsZXIuamF2YQ==) | `77.85% <66.66%> (-9.47%)` | :arrow_down: |
... and [794 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18411/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-18410] Fix Flink upgrade blocking dispatcher thread causing heartbeat timeout [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18411:
URL: https://github.com/apache/hudi/pull/18411#issuecomment-4149210562
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `66.66667%` with `1 line` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 46.77%. Comparing base ([`1eb97b3`](https://app.codecov.io/gh/apache/hudi/commit/1eb97b31826eabee4fb9c4bf78dc44d2bdb2dad0?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`10ccb3d`](https://app.codecov.io/gh/apache/hudi/commit/10ccb3d818b0e218b08db4e0777b531e494a9708?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...hudi/table/upgrade/SevenToEightUpgradeHandler.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Ftable%2Fupgrade%2FSevenToEightUpgradeHandler.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL3RhYmxlL3VwZ3JhZGUvU2V2ZW5Ub0VpZ2h0VXBncmFkZUhhbmRsZXIuamF2YQ==) | 66.66% | [1 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
> :exclamation: There is a different number of reports uploaded between BASE (1eb97b3) and HEAD (10ccb3d). Click for more details.
>
> <details><summary>HEAD has 26 uploads less than BASE</summary>
>
>| Flag | BASE (1eb97b3) | HEAD (10ccb3d) |
>|------|------|------|
>|spark-scala-tests|10|1|
>|spark-java-tests|15|0|
>|common-and-other-modules|1|0|
>|utilities|1|0|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18411 +/- ##
=============================================
- Coverage 68.21% 46.77% -21.45%
+ Complexity 27709 14887 -12822
=============================================
Files 2440 1933 -507
Lines 134249 103678 -30571
Branches 16179 12414 -3765
=============================================
- Hits 91578 48492 -43086
- Misses 35565 50231 +14666
+ Partials 7106 4955 -2151
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.98% <66.66%> (+0.05%)` | :arrow_up: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.32% <0.00%> (+<0.01%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `33.23% <0.00%> (-12.00%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...hudi/table/upgrade/SevenToEightUpgradeHandler.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Ftable%2Fupgrade%2FSevenToEightUpgradeHandler.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL3RhYmxlL3VwZ3JhZGUvU2V2ZW5Ub0VpZ2h0VXBncmFkZUhhbmRsZXIuamF2YQ==) | `57.85% <66.66%> (-29.47%)` | :arrow_down: |
... and [1614 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18411/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-18410] Fix Flink upgrade blocking dispatcher thread causing heartbeat timeout [hudi]
Posted by "danny0405 (via GitHub)" <gi...@apache.org>.
danny0405 commented on code in PR #18411:
URL: https://github.com/apache/hudi/pull/18411#discussion_r3006985663
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/upgrade/SevenToEightUpgradeHandler.java:
##########
@@ -249,6 +249,12 @@ static void upgradeKeyGeneratorType(HoodieTableConfig tableConfig, Map<ConfigPro
}
}
+ // Use a large batch size for migration to minimize the number of parquet files created
+ // on remote storage. Each write() call involves multiple remote storage operations (exists check,
+ // parquet write, manifest update). Using the default archival batch size (10) with hundreds of
+ // actions creates excessive I/O that significantly increases the total migration time.
+ private static final int MIGRATION_BATCH_SIZE = 500;
Review Comment:
wondering how much gains for this large batch, like 30% cost saving?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-18410] Optimize LSM timeline migration batch size and compaction during upgrade [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #18411:
URL: https://github.com/apache/hudi/pull/18411#issuecomment-4157177141
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "10ccb3d818b0e218b08db4e0777b531e494a9708",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12900",
"triggerID" : "10ccb3d818b0e218b08db4e0777b531e494a9708",
"triggerType" : "PUSH"
}, {
"hash" : "df05521d76e687ac5e4d09c65e3d94a5d03ae5d6",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12928",
"triggerID" : "df05521d76e687ac5e4d09c65e3d94a5d03ae5d6",
"triggerType" : "PUSH"
}, {
"hash" : "cd5f54b2de81575b98c515152c82fd71f950b2b4",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "cd5f54b2de81575b98c515152c82fd71f950b2b4",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 10ccb3d818b0e218b08db4e0777b531e494a9708 Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12900)
* df05521d76e687ac5e4d09c65e3d94a5d03ae5d6 Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12928)
* cd5f54b2de81575b98c515152c82fd71f950b2b4 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-18410] Optimize LSM timeline migration batch size and compaction during upgrade [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18411:
URL: https://github.com/apache/hudi/pull/18411#issuecomment-4157424035
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `66.66667%` with `1 line` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 45.77%. Comparing base ([`1eb97b3`](https://app.codecov.io/gh/apache/hudi/commit/1eb97b31826eabee4fb9c4bf78dc44d2bdb2dad0?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`cd5f54b`](https://app.codecov.io/gh/apache/hudi/commit/cd5f54b2de81575b98c515152c82fd71f950b2b4?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 6 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...hudi/table/upgrade/SevenToEightUpgradeHandler.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Ftable%2Fupgrade%2FSevenToEightUpgradeHandler.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL3RhYmxlL3VwZ3JhZGUvU2V2ZW5Ub0VpZ2h0VXBncmFkZUhhbmRsZXIuamF2YQ==) | 66.66% | [1 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
> :exclamation: There is a different number of reports uploaded between BASE (1eb97b3) and HEAD (cd5f54b). Click for more details.
>
> <details><summary>HEAD has 24 uploads less than BASE</summary>
>
>| Flag | BASE (1eb97b3) | HEAD (cd5f54b) |
>|------|------|------|
>|spark-scala-tests|10|3|
>|spark-java-tests|15|0|
>|common-and-other-modules|1|0|
>|utilities|1|0|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18411 +/- ##
=============================================
- Coverage 68.21% 45.77% -22.44%
+ Complexity 27709 15226 -12483
=============================================
Files 2440 1966 -474
Lines 134249 108548 -25701
Branches 16179 13483 -2696
=============================================
- Hits 91578 49691 -41887
- Misses 35565 53622 +18057
+ Partials 7106 5235 -1871
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.97% <66.66%> (+0.04%)` | :arrow_up: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.40% <0.00%> (+0.08%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `32.82% <0.00%> (-12.42%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...hudi/table/upgrade/SevenToEightUpgradeHandler.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Ftable%2Fupgrade%2FSevenToEightUpgradeHandler.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL3RhYmxlL3VwZ3JhZGUvU2V2ZW5Ub0VpZ2h0VXBncmFkZUhhbmRsZXIuamF2YQ==) | `57.85% <66.66%> (-29.47%)` | :arrow_down: |
... and [1579 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18411/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-18410] Optimize LSM timeline migration batch size and compaction during upgrade [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #18411:
URL: https://github.com/apache/hudi/pull/18411#issuecomment-4157463099
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "10ccb3d818b0e218b08db4e0777b531e494a9708",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12900",
"triggerID" : "10ccb3d818b0e218b08db4e0777b531e494a9708",
"triggerType" : "PUSH"
}, {
"hash" : "df05521d76e687ac5e4d09c65e3d94a5d03ae5d6",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12928",
"triggerID" : "df05521d76e687ac5e4d09c65e3d94a5d03ae5d6",
"triggerType" : "PUSH"
}, {
"hash" : "cd5f54b2de81575b98c515152c82fd71f950b2b4",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12930",
"triggerID" : "cd5f54b2de81575b98c515152c82fd71f950b2b4",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 10ccb3d818b0e218b08db4e0777b531e494a9708 Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12900)
* df05521d76e687ac5e4d09c65e3d94a5d03ae5d6 Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12928)
* cd5f54b2de81575b98c515152c82fd71f950b2b4 Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12930)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-18410] Optimize LSM timeline migration batch size and compaction during upgrade [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18411:
URL: https://github.com/apache/hudi/pull/18411#issuecomment-4157539908
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `87.50000%` with `3 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 67.37%. Comparing base ([`1eb97b3`](https://app.codecov.io/gh/apache/hudi/commit/1eb97b31826eabee4fb9c4bf78dc44d2bdb2dad0?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`cd5f54b`](https://app.codecov.io/gh/apache/hudi/commit/cd5f54b2de81575b98c515152c82fd71f950b2b4?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 6 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/sink/StreamWriteOperatorCoordinator.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-flink-datasource%2Fhudi-flink%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fsink%2FStreamWriteOperatorCoordinator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1mbGluay1kYXRhc291cmNlL2h1ZGktZmxpbmsvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvc2luay9TdHJlYW1Xcml0ZU9wZXJhdG9yQ29vcmRpbmF0b3IuamF2YQ==) | 90.47% | [1 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...hudi/table/upgrade/SevenToEightUpgradeHandler.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Ftable%2Fupgrade%2FSevenToEightUpgradeHandler.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL3RhYmxlL3VwZ3JhZGUvU2V2ZW5Ub0VpZ2h0VXBncmFkZUhhbmRsZXIuamF2YQ==) | 66.66% | [0 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18411 +/- ##
============================================
- Coverage 68.21% 67.37% -0.85%
+ Complexity 27709 27399 -310
============================================
Files 2440 2439 -1
Lines 134249 134358 +109
Branches 16179 16210 +31
============================================
- Hits 91578 90520 -1058
- Misses 35565 36663 +1098
- Partials 7106 7175 +69
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.35% <87.50%> (+0.03%)` | :arrow_up: |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.97% <66.66%> (+0.04%)` | :arrow_up: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.40% <0.00%> (+0.08%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.95% <66.66%> (-3.76%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.29% <0.00%> (+0.06%)` | :arrow_up: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `38.37% <66.66%> (+<0.01%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...hudi/table/upgrade/SevenToEightUpgradeHandler.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Ftable%2Fupgrade%2FSevenToEightUpgradeHandler.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL3RhYmxlL3VwZ3JhZGUvU2V2ZW5Ub0VpZ2h0VXBncmFkZUhhbmRsZXIuamF2YQ==) | `85.00% <66.66%> (-2.33%)` | :arrow_down: |
| [...ache/hudi/sink/StreamWriteOperatorCoordinator.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-flink-datasource%2Fhudi-flink%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fsink%2FStreamWriteOperatorCoordinator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1mbGluay1kYXRhc291cmNlL2h1ZGktZmxpbmsvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvc2luay9TdHJlYW1Xcml0ZU9wZXJhdG9yQ29vcmRpbmF0b3IuamF2YQ==) | `81.38% <90.47%> (+0.34%)` | :arrow_up: |
... and [180 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18411/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-18410] Optimize LSM timeline migration batch size and compaction during upgrade [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18411:
URL: https://github.com/apache/hudi/pull/18411#issuecomment-4157659088
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `87.50000%` with `3 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 68.25%. Comparing base ([`1eb97b3`](https://app.codecov.io/gh/apache/hudi/commit/1eb97b31826eabee4fb9c4bf78dc44d2bdb2dad0?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`cd5f54b`](https://app.codecov.io/gh/apache/hudi/commit/cd5f54b2de81575b98c515152c82fd71f950b2b4?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 6 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/sink/StreamWriteOperatorCoordinator.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-flink-datasource%2Fhudi-flink%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fsink%2FStreamWriteOperatorCoordinator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1mbGluay1kYXRhc291cmNlL2h1ZGktZmxpbmsvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvc2luay9TdHJlYW1Xcml0ZU9wZXJhdG9yQ29vcmRpbmF0b3IuamF2YQ==) | 90.47% | [1 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...hudi/table/upgrade/SevenToEightUpgradeHandler.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Ftable%2Fupgrade%2FSevenToEightUpgradeHandler.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL3RhYmxlL3VwZ3JhZGUvU2V2ZW5Ub0VpZ2h0VXBncmFkZUhhbmRsZXIuamF2YQ==) | 66.66% | [0 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18411 +/- ##
============================================
+ Coverage 68.21% 68.25% +0.04%
- Complexity 27709 27760 +51
============================================
Files 2440 2439 -1
Lines 134249 134358 +109
Branches 16179 16210 +31
============================================
+ Hits 91578 91710 +132
+ Misses 35565 35547 -18
+ Partials 7106 7101 -5
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.35% <87.50%> (+0.03%)` | :arrow_up: |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.97% <66.66%> (+0.04%)` | :arrow_up: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.40% <0.00%> (+0.08%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.77% <66.66%> (+0.06%)` | :arrow_up: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.29% <0.00%> (+0.06%)` | :arrow_up: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `38.37% <66.66%> (+<0.01%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...hudi/table/upgrade/SevenToEightUpgradeHandler.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Ftable%2Fupgrade%2FSevenToEightUpgradeHandler.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL3RhYmxlL3VwZ3JhZGUvU2V2ZW5Ub0VpZ2h0VXBncmFkZUhhbmRsZXIuamF2YQ==) | `85.00% <66.66%> (-2.33%)` | :arrow_down: |
| [...ache/hudi/sink/StreamWriteOperatorCoordinator.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-flink-datasource%2Fhudi-flink%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fsink%2FStreamWriteOperatorCoordinator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1mbGluay1kYXRhc291cmNlL2h1ZGktZmxpbmsvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvc2luay9TdHJlYW1Xcml0ZU9wZXJhdG9yQ29vcmRpbmF0b3IuamF2YQ==) | `81.38% <90.47%> (+0.34%)` | :arrow_up: |
... and [41 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18411/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-18410] Optimize LSM timeline migration batch size and compaction during upgrade [hudi]
Posted by "prashantwason (via GitHub)" <gi...@apache.org>.
prashantwason commented on code in PR #18411:
URL: https://github.com/apache/hudi/pull/18411#discussion_r3041687638
##########
hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/sink/StreamWriteOperatorCoordinator.java:
##########
@@ -244,15 +245,36 @@ public void start() throws Exception {
.threadFactory(getThreadFactory("instant-request"))
.exceptionHook((errMsg, t) -> this.context.failJob(new HoodieException(errMsg, t)))
.build();
- // start the executor if required
- if (tableState.syncHive) {
- initHiveSync();
- }
- // start client id heartbeats for optimistic concurrency control
- if (OptionsResolver.isMultiWriter(conf)) {
- initClientIds(conf);
- }
- restoreEvents();
+
+ // Run upgrade, metadata table init, and event restoration on the executor
+ // thread instead of the Pekko dispatcher thread. Running these synchronously
+ // on the dispatcher thread blocks heartbeat responses when the operations
+ // involve heavy I/O (e.g., LSM timeline migration with hundreds of archived
+ // actions), causing the ResourceManager to disconnect the JobManager.
+ //
+ // Safety guarantees:
+ // - Events via handleEventFromOperator() are submitted to the same
+ // single-threaded executor, so FIFO ordering ensures initialization
+ // completes before any event processing.
+ // - Coordination requests via handleCoordinationRequest() run on the
+ // separate instantRequestExecutor and are gated on initFuture to
+ // prevent startInstant() from racing ahead of the upgrade.
+ this.executor.execute(() -> {
+ this.writeClient.tryUpgrade(instant, this.metaClient);
+ initMetadataTable(this.writeClient);
+ if (tableState.scheduleMdtCompaction) {
+ this.metadataWriteClient = StreamerUtil.createMetadataWriteClient(writeClient);
+ }
+ if (tableState.syncHive) {
+ initHiveSync();
+ }
Review Comment:
Good catch. Added try-catch in the lambda body — on failure, `initFuture.completeExceptionally(t)` is called before re-throwing, so `NonThrownExecutor`'s exception hook still fires (calling `context.failJob()`) and coordination requests get the exception instead of hanging.
##########
hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/sink/StreamWriteOperatorCoordinator.java:
##########
@@ -244,15 +245,36 @@ public void start() throws Exception {
.threadFactory(getThreadFactory("instant-request"))
.exceptionHook((errMsg, t) -> this.context.failJob(new HoodieException(errMsg, t)))
.build();
- // start the executor if required
- if (tableState.syncHive) {
- initHiveSync();
- }
- // start client id heartbeats for optimistic concurrency control
- if (OptionsResolver.isMultiWriter(conf)) {
- initClientIds(conf);
- }
- restoreEvents();
+
+ // Run upgrade, metadata table init, and event restoration on the executor
+ // thread instead of the Pekko dispatcher thread. Running these synchronously
+ // on the dispatcher thread blocks heartbeat responses when the operations
+ // involve heavy I/O (e.g., LSM timeline migration with hundreds of archived
+ // actions), causing the ResourceManager to disconnect the JobManager.
+ //
+ // Safety guarantees:
+ // - Events via handleEventFromOperator() are submitted to the same
+ // single-threaded executor, so FIFO ordering ensures initialization
Review Comment:
Fixed — added `initFuture.completeExceptionally(throwable)` in the outer catch block as well.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-18410] Optimize LSM timeline migration batch size and compaction during upgrade [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18411:
URL: https://github.com/apache/hudi/pull/18411#issuecomment-4157457117
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `66.66667%` with `1 line` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 48.09%. Comparing base ([`1eb97b3`](https://app.codecov.io/gh/apache/hudi/commit/1eb97b31826eabee4fb9c4bf78dc44d2bdb2dad0?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`cd5f54b`](https://app.codecov.io/gh/apache/hudi/commit/cd5f54b2de81575b98c515152c82fd71f950b2b4?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 6 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...hudi/table/upgrade/SevenToEightUpgradeHandler.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Ftable%2Fupgrade%2FSevenToEightUpgradeHandler.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL3RhYmxlL3VwZ3JhZGUvU2V2ZW5Ub0VpZ2h0VXBncmFkZUhhbmRsZXIuamF2YQ==) | 66.66% | [1 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
> :exclamation: There is a different number of reports uploaded between BASE (1eb97b3) and HEAD (cd5f54b). Click for more details.
>
> <details><summary>HEAD has 21 uploads less than BASE</summary>
>
>| Flag | BASE (1eb97b3) | HEAD (cd5f54b) |
>|------|------|------|
>|spark-scala-tests|10|5|
>|spark-java-tests|15|1|
>|common-and-other-modules|1|0|
>|utilities|1|0|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18411 +/- ##
=============================================
- Coverage 68.21% 48.09% -20.12%
+ Complexity 27709 16339 -11370
=============================================
Files 2440 1995 -445
Lines 134249 111150 -23099
Branches 16179 14042 -2137
=============================================
- Hits 91578 53460 -38118
- Misses 35565 52082 +16517
+ Partials 7106 5608 -1498
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.97% <66.66%> (+0.04%)` | :arrow_up: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.40% <0.00%> (+0.08%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `33.71% <66.66%> (-15.00%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `32.69% <0.00%> (-12.55%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...hudi/table/upgrade/SevenToEightUpgradeHandler.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Ftable%2Fupgrade%2FSevenToEightUpgradeHandler.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL3RhYmxlL3VwZ3JhZGUvU2V2ZW5Ub0VpZ2h0VXBncmFkZUhhbmRsZXIuamF2YQ==) | `62.14% <66.66%> (-25.19%)` | :arrow_down: |
... and [1512 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18411/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-18410] Optimize LSM timeline migration batch size and compaction during upgrade [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18411:
URL: https://github.com/apache/hudi/pull/18411#issuecomment-4157437166
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `66.66667%` with `1 line` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 45.30%. Comparing base ([`1eb97b3`](https://app.codecov.io/gh/apache/hudi/commit/1eb97b31826eabee4fb9c4bf78dc44d2bdb2dad0?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`cd5f54b`](https://app.codecov.io/gh/apache/hudi/commit/cd5f54b2de81575b98c515152c82fd71f950b2b4?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 6 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...hudi/table/upgrade/SevenToEightUpgradeHandler.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Ftable%2Fupgrade%2FSevenToEightUpgradeHandler.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL3RhYmxlL3VwZ3JhZGUvU2V2ZW5Ub0VpZ2h0VXBncmFkZUhhbmRsZXIuamF2YQ==) | 66.66% | [1 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
> :exclamation: There is a different number of reports uploaded between BASE (1eb97b3) and HEAD (cd5f54b). Click for more details.
>
> <details><summary>HEAD has 23 uploads less than BASE</summary>
>
>| Flag | BASE (1eb97b3) | HEAD (cd5f54b) |
>|------|------|------|
>|spark-scala-tests|10|4|
>|spark-java-tests|15|0|
>|common-and-other-modules|1|0|
>|utilities|1|0|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18411 +/- ##
=============================================
- Coverage 68.21% 45.30% -22.92%
+ Complexity 27709 15438 -12271
=============================================
Files 2440 1995 -445
Lines 134249 111145 -23104
Branches 16179 14036 -2143
=============================================
- Hits 91578 50352 -41226
- Misses 35565 55397 +19832
+ Partials 7106 5396 -1710
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.97% <66.66%> (+0.04%)` | :arrow_up: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.40% <0.00%> (+0.08%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `32.68% <0.00%> (-12.55%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...hudi/table/upgrade/SevenToEightUpgradeHandler.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Ftable%2Fupgrade%2FSevenToEightUpgradeHandler.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL3RhYmxlL3VwZ3JhZGUvU2V2ZW5Ub0VpZ2h0VXBncmFkZUhhbmRsZXIuamF2YQ==) | `57.85% <66.66%> (-29.47%)` | :arrow_down: |
... and [1563 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18411/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-18410] Optimize LSM timeline migration batch size and compaction during upgrade [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18411:
URL: https://github.com/apache/hudi/pull/18411#issuecomment-4157554866
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `87.50000%` with `3 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 67.39%. Comparing base ([`1eb97b3`](https://app.codecov.io/gh/apache/hudi/commit/1eb97b31826eabee4fb9c4bf78dc44d2bdb2dad0?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`cd5f54b`](https://app.codecov.io/gh/apache/hudi/commit/cd5f54b2de81575b98c515152c82fd71f950b2b4?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 6 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/hudi/sink/StreamWriteOperatorCoordinator.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-flink-datasource%2Fhudi-flink%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fsink%2FStreamWriteOperatorCoordinator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1mbGluay1kYXRhc291cmNlL2h1ZGktZmxpbmsvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvc2luay9TdHJlYW1Xcml0ZU9wZXJhdG9yQ29vcmRpbmF0b3IuamF2YQ==) | 90.47% | [1 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...hudi/table/upgrade/SevenToEightUpgradeHandler.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Ftable%2Fupgrade%2FSevenToEightUpgradeHandler.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL3RhYmxlL3VwZ3JhZGUvU2V2ZW5Ub0VpZ2h0VXBncmFkZUhhbmRsZXIuamF2YQ==) | 66.66% | [0 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18411 +/- ##
============================================
- Coverage 68.21% 67.39% -0.82%
+ Complexity 27709 27406 -303
============================================
Files 2440 2439 -1
Lines 134249 134358 +109
Branches 16179 16210 +31
============================================
- Hits 91578 90554 -1024
- Misses 35565 36634 +1069
- Partials 7106 7170 +64
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.35% <87.50%> (+0.03%)` | :arrow_up: |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.97% <66.66%> (+0.04%)` | :arrow_up: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.40% <0.00%> (+0.08%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.02% <66.66%> (-3.69%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.29% <0.00%> (+0.06%)` | :arrow_up: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `38.37% <66.66%> (+<0.01%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...hudi/table/upgrade/SevenToEightUpgradeHandler.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Ftable%2Fupgrade%2FSevenToEightUpgradeHandler.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL3RhYmxlL3VwZ3JhZGUvU2V2ZW5Ub0VpZ2h0VXBncmFkZUhhbmRsZXIuamF2YQ==) | `85.00% <66.66%> (-2.33%)` | :arrow_down: |
| [...ache/hudi/sink/StreamWriteOperatorCoordinator.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-flink-datasource%2Fhudi-flink%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fsink%2FStreamWriteOperatorCoordinator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1mbGluay1kYXRhc291cmNlL2h1ZGktZmxpbmsvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvc2luay9TdHJlYW1Xcml0ZU9wZXJhdG9yQ29vcmRpbmF0b3IuamF2YQ==) | `81.38% <90.47%> (+0.34%)` | :arrow_up: |
... and [172 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18411/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-18410] Fix Flink upgrade blocking dispatcher thread causing heartbeat timeout [hudi]
Posted by "prashantwason (via GitHub)" <gi...@apache.org>.
prashantwason commented on code in PR #18411:
URL: https://github.com/apache/hudi/pull/18411#discussion_r3011357839
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/upgrade/SevenToEightUpgradeHandler.java:
##########
@@ -249,6 +249,12 @@ static void upgradeKeyGeneratorType(HoodieTableConfig tableConfig, Map<ConfigPro
}
}
+ // Use a large batch size for migration to minimize the number of parquet files created
+ // on remote storage. Each write() call involves multiple remote storage operations (exists check,
+ // parquet write, manifest update). Using the default archival batch size (10) with hundreds of
+ // actions creates excessive I/O that significantly increases the total migration time.
+ private static final int MIGRATION_BATCH_SIZE = 500;
Review Comment:
For a table with N archived actions, original code does `N/10` batches, each with 1 write + 1 compactAndClean = ~5 remote storage ops. With `MIGRATION_BATCH_SIZE = 500`, it does `N/500` batches with 1 write each + 1 final compactAndClean.
Concrete numbers:
- **50 actions**: 25 ops → 5 ops (~80% reduction)
- **200 actions**: 100 ops → 5 ops (~95% reduction)
- **500 actions**: 250 ops → 6 ops (~97% reduction)
The biggest gain is from moving `compactAndClean()` to end-only. In the original code, each compaction re-processes all previously written L0 files, so the cost grows quadratically with the number of batches. With end-only compaction, it's a single O(N) pass.
Pushed a revised commit that drops the async executor change (per your and @cshuo's feedback) and keeps only this I/O optimization.
##########
hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/sink/StreamWriteOperatorCoordinator.java:
##########
@@ -244,6 +248,17 @@ public void start() throws Exception {
.threadFactory(getThreadFactory("instant-request"))
.exceptionHook((errMsg, t) -> this.context.failJob(new HoodieException(errMsg, t)))
.build();
+
+ // Run upgrade and post-upgrade initialization on the executor thread.
+ this.executor.execute(() -> {
+ this.writeClient.tryUpgrade(instant, this.metaClient);
+ initMetadataTable(this.writeClient);
+ if (tableState.scheduleMdtCompaction) {
+ this.metadataWriteClient = StreamerUtil.createMetadataWriteClient(writeClient);
Review Comment:
Agreed. Reverted the async change — the coordinator must block start() until upgrade and MDT init are complete. The revised commit keeps only the I/O optimization in SevenToEightUpgradeHandler (larger batch size + single compactAndClean at end).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-18410] Fix Flink upgrade blocking dispatcher thread causing heartbeat timeout [hudi]
Posted by "prashantwason (via GitHub)" <gi...@apache.org>.
prashantwason commented on code in PR #18411:
URL: https://github.com/apache/hudi/pull/18411#discussion_r3011359344
##########
hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/sink/StreamWriteOperatorCoordinator.java:
##########
@@ -228,14 +228,18 @@ public void start() throws Exception {
this.metaClient = initTableIfNotExists(this.conf);
// the write client must create after the table creation
this.writeClient = FlinkWriteClients.createWriteClient(conf);
- this.writeClient.tryUpgrade(instant, this.metaClient);
- initMetadataTable(this.writeClient);
- if (tableState.scheduleMdtCompaction) {
- this.metadataWriteClient = StreamerUtil.createMetadataWriteClient(writeClient);
- }
-
- // start the executor
+ // Create executors BEFORE the upgrade so that the heavy initialization
+ // (upgrade, metadata table init, event restoration) can run on the executor
+ // thread instead of the Pekko dispatcher thread. Running the upgrade
+ // synchronously on the dispatcher thread blocks heartbeat responses and
+ // causes the ResourceManager to disconnect the JobManager when upgrades
+ // take longer than the heartbeat timeout (e.g., LSM timeline migration
+ // with hundreds of archived actions).
+ //
+ // Since the executor is single-threaded and FIFO, submitting the
Review Comment:
Good points — the race between `startInstant()` on `instantRequestExecutor` and the async upgrade on `executor` is a real issue, and the RLI dependency on MDT init makes it unsafe to start the pipeline early.
Reverted the async change completely. The revised commit only optimizes `upgradeToLSMTimeline()` to reduce the I/O cost of the migration itself (batch size 500 + single compactAndClean at end), keeping the synchronous start() flow intact.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-18410] Optimize LSM timeline migration batch size and compaction during upgrade [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #18411:
URL: https://github.com/apache/hudi/pull/18411#issuecomment-4157057032
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "10ccb3d818b0e218b08db4e0777b531e494a9708",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12900",
"triggerID" : "10ccb3d818b0e218b08db4e0777b531e494a9708",
"triggerType" : "PUSH"
}, {
"hash" : "df05521d76e687ac5e4d09c65e3d94a5d03ae5d6",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12928",
"triggerID" : "df05521d76e687ac5e4d09c65e3d94a5d03ae5d6",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 10ccb3d818b0e218b08db4e0777b531e494a9708 Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12900)
* df05521d76e687ac5e4d09c65e3d94a5d03ae5d6 Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12928)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-18410] Optimize LSM timeline migration batch size and compaction during upgrade [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18411:
URL: https://github.com/apache/hudi/pull/18411#issuecomment-4157257925
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `66.66667%` with `1 line` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 44.97%. Comparing base ([`1eb97b3`](https://app.codecov.io/gh/apache/hudi/commit/1eb97b31826eabee4fb9c4bf78dc44d2bdb2dad0?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`cd5f54b`](https://app.codecov.io/gh/apache/hudi/commit/cd5f54b2de81575b98c515152c82fd71f950b2b4?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 6 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...hudi/table/upgrade/SevenToEightUpgradeHandler.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Ftable%2Fupgrade%2FSevenToEightUpgradeHandler.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL3RhYmxlL3VwZ3JhZGUvU2V2ZW5Ub0VpZ2h0VXBncmFkZUhhbmRsZXIuamF2YQ==) | 66.66% | [1 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
> :exclamation: There is a different number of reports uploaded between BASE (1eb97b3) and HEAD (cd5f54b). Click for more details.
>
> <details><summary>HEAD has 28 uploads less than BASE</summary>
>
>| Flag | BASE (1eb97b3) | HEAD (cd5f54b) |
>|------|------|------|
>|spark-scala-tests|10|0|
>|spark-client-hadoop-common|1|0|
>|spark-java-tests|15|0|
>|common-and-other-modules|1|0|
>|utilities|1|0|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18411 +/- ##
=============================================
- Coverage 68.21% 44.97% -23.25%
+ Complexity 27709 8455 -19254
=============================================
Files 2440 1194 -1246
Lines 134249 61631 -72618
Branches 16179 6609 -9570
=============================================
- Hits 91578 27717 -63861
+ Misses 35565 30895 -4670
+ Partials 7106 3019 -4087
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.97% <66.66%> (+0.04%)` | :arrow_up: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...hudi/table/upgrade/SevenToEightUpgradeHandler.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Ftable%2Fupgrade%2FSevenToEightUpgradeHandler.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL3RhYmxlL3VwZ3JhZGUvU2V2ZW5Ub0VpZ2h0VXBncmFkZUhhbmRsZXIuamF2YQ==) | `57.85% <66.66%> (-29.47%)` | :arrow_down: |
... and [2042 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18411/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-18410] Fix Flink upgrade blocking dispatcher thread causing heartbeat timeout [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18411:
URL: https://github.com/apache/hudi/pull/18411#issuecomment-4149212011
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `66.66667%` with `1 line` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 45.71%. Comparing base ([`1eb97b3`](https://app.codecov.io/gh/apache/hudi/commit/1eb97b31826eabee4fb9c4bf78dc44d2bdb2dad0?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`10ccb3d`](https://app.codecov.io/gh/apache/hudi/commit/10ccb3d818b0e218b08db4e0777b531e494a9708?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...hudi/table/upgrade/SevenToEightUpgradeHandler.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Ftable%2Fupgrade%2FSevenToEightUpgradeHandler.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL3RhYmxlL3VwZ3JhZGUvU2V2ZW5Ub0VpZ2h0VXBncmFkZUhhbmRsZXIuamF2YQ==) | 66.66% | [1 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
> :exclamation: There is a different number of reports uploaded between BASE (1eb97b3) and HEAD (10ccb3d). Click for more details.
>
> <details><summary>HEAD has 24 uploads less than BASE</summary>
>
>| Flag | BASE (1eb97b3) | HEAD (10ccb3d) |
>|------|------|------|
>|spark-scala-tests|10|3|
>|spark-java-tests|15|0|
>|common-and-other-modules|1|0|
>|utilities|1|0|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18411 +/- ##
=============================================
- Coverage 68.21% 45.71% -22.51%
+ Complexity 27709 15209 -12500
=============================================
Files 2440 1967 -473
Lines 134249 108485 -25764
Branches 16179 13461 -2718
=============================================
- Hits 91578 49591 -41987
- Misses 35565 53687 +18122
+ Partials 7106 5207 -1899
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.98% <66.66%> (+0.05%)` | :arrow_up: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.32% <0.00%> (+<0.01%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `32.79% <0.00%> (-12.45%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...hudi/table/upgrade/SevenToEightUpgradeHandler.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Ftable%2Fupgrade%2FSevenToEightUpgradeHandler.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL3RhYmxlL3VwZ3JhZGUvU2V2ZW5Ub0VpZ2h0VXBncmFkZUhhbmRsZXIuamF2YQ==) | `57.85% <66.66%> (-29.47%)` | :arrow_down: |
... and [1577 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18411/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-18410] Fix Flink upgrade blocking dispatcher thread causing heartbeat timeout [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18411:
URL: https://github.com/apache/hudi/pull/18411#issuecomment-4149240148
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `66.66667%` with `1 line` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 66.02%. Comparing base ([`1eb97b3`](https://app.codecov.io/gh/apache/hudi/commit/1eb97b31826eabee4fb9c4bf78dc44d2bdb2dad0?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`10ccb3d`](https://app.codecov.io/gh/apache/hudi/commit/10ccb3d818b0e218b08db4e0777b531e494a9708?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...hudi/table/upgrade/SevenToEightUpgradeHandler.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Ftable%2Fupgrade%2FSevenToEightUpgradeHandler.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL3RhYmxlL3VwZ3JhZGUvU2V2ZW5Ub0VpZ2h0VXBncmFkZUhhbmRsZXIuamF2YQ==) | 66.66% | [0 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18411 +/- ##
============================================
- Coverage 68.21% 66.02% -2.20%
+ Complexity 27709 22470 -5239
============================================
Files 2440 1996 -444
Lines 134249 111058 -23191
Branches 16179 14012 -2167
============================================
- Hits 91578 73325 -18253
+ Misses 35565 31492 -4073
+ Partials 7106 6241 -865
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.98% <66.66%> (+0.05%)` | :arrow_up: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.32% <0.00%> (+<0.01%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.63% <66.66%> (-0.08%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.25% <0.00%> (+0.01%)` | :arrow_up: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `38.38% <66.66%> (+<0.01%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...hudi/table/upgrade/SevenToEightUpgradeHandler.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Ftable%2Fupgrade%2FSevenToEightUpgradeHandler.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL3RhYmxlL3VwZ3JhZGUvU2V2ZW5Ub0VpZ2h0VXBncmFkZUhhbmRsZXIuamF2YQ==) | `77.85% <66.66%> (-9.47%)` | :arrow_down: |
... and [796 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18411/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-18410] Fix Flink upgrade blocking dispatcher thread causing heartbeat timeout [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18411:
URL: https://github.com/apache/hudi/pull/18411#issuecomment-4149179967
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `66.66667%` with `1 line` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 44.98%. Comparing base ([`1eb97b3`](https://app.codecov.io/gh/apache/hudi/commit/1eb97b31826eabee4fb9c4bf78dc44d2bdb2dad0?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`10ccb3d`](https://app.codecov.io/gh/apache/hudi/commit/10ccb3d818b0e218b08db4e0777b531e494a9708?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...hudi/table/upgrade/SevenToEightUpgradeHandler.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Ftable%2Fupgrade%2FSevenToEightUpgradeHandler.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL3RhYmxlL3VwZ3JhZGUvU2V2ZW5Ub0VpZ2h0VXBncmFkZUhhbmRsZXIuamF2YQ==) | 66.66% | [1 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
> :exclamation: There is a different number of reports uploaded between BASE (1eb97b3) and HEAD (10ccb3d). Click for more details.
>
> <details><summary>HEAD has 28 uploads less than BASE</summary>
>
>| Flag | BASE (1eb97b3) | HEAD (10ccb3d) |
>|------|------|------|
>|spark-scala-tests|10|0|
>|spark-client-hadoop-common|1|0|
>|spark-java-tests|15|0|
>|common-and-other-modules|1|0|
>|utilities|1|0|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18411 +/- ##
=============================================
- Coverage 68.21% 44.98% -23.24%
+ Complexity 27709 8455 -19254
=============================================
Files 2440 1194 -1246
Lines 134249 61609 -72640
Branches 16179 6602 -9577
=============================================
- Hits 91578 27714 -63864
+ Misses 35565 30878 -4687
+ Partials 7106 3017 -4089
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.98% <66.66%> (+0.05%)` | :arrow_up: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18411/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18411?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...hudi/table/upgrade/SevenToEightUpgradeHandler.java](https://app.codecov.io/gh/apache/hudi/pull/18411?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Ftable%2Fupgrade%2FSevenToEightUpgradeHandler.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL3RhYmxlL3VwZ3JhZGUvU2V2ZW5Ub0VpZ2h0VXBncmFkZUhhbmRsZXIuamF2YQ==) | `57.85% <66.66%> (-29.47%)` | :arrow_down: |
... and [2042 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18411/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org