You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by "shauryachats (via GitHub)" <gi...@apache.org> on 2026/05/29 19:03:24 UTC
[PR] [bugfix] Fix instance partition import rebalance for COMPLETED segments [pinot]
shauryachats opened a new pull request, #18635:
URL: https://github.com/apache/pinot/pull/18635
## Summary
Tables that import instance partitions from another table via `instancePartitionsMap` (without a separate `instanceAssignmentConfigMap` entry for COMPLETED) were broken during rebalance.
`TableRebalancer.getInstancePartitionsMap()` only loads COMPLETED instance partitions when `InstanceAssignmentConfigUtils.shouldRelocateCompletedSegments()` returns true - but that method only looked at `instanceAssignmentConfigMap` and tenant tag overrides, not imported partitions in `instancePartitionsMap`.
As a result, rebalance passed only CONSUMING instance partitions into segment assignment. Completed segments stayed pinned to one server per stream partition instead of spreading across all servers in each instance partition under `ReplicaGroupSegmentAssignmentStrategy` (e.g. 16 servers used instead of 64 in the prod layout).
The fix adds a check for `InstancePartitionsUtils.hasPreConfiguredInstancePartitions(tableConfig, COMPLETED)` so imported COMPLETED partitions trigger relocation the same way as explicitly configured assignment.
## Test
`RealtimeReplicaGroupSegmentAssignmentTest#testImportedInstancePartitionsWithMultipleServersPerPartition` mirrors the TableRebalancer path: it builds the rebalance `instancePartitionsMap` via `shouldRelocateCompletedSegments(tableConfig)` rather than always injecting COMPLETED IPs. It asserts the pre-fix behavior when COMPLETED IPs are omitted, and that with the fix all servers receive completed segments (including bootstrap rebalance).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] [bugfix] Fix instance partition import rebalance for COMPLETED segments [pinot]
Posted by "xiangfu0 (via GitHub)" <gi...@apache.org>.
xiangfu0 merged PR #18635:
URL: https://github.com/apache/pinot/pull/18635
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] [bugfix] Fix instance partition import rebalance for COMPLETED segments [pinot]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18635:
URL: https://github.com/apache/pinot/pull/18635#issuecomment-4579466435
## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/18635?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `0%` with `1 line` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 56.81%. Comparing base ([`aee5a1f`](https://app.codecov.io/gh/apache/pinot/commit/aee5a1f10ae0d82d8c806e0bf2cebd8b99b886b6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`b88c463`](https://app.codecov.io/gh/apache/pinot/commit/b88c463231e3932d493845927c6f6caca8aea6f0?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 4 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/pinot/pull/18635?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...mmon/assignment/InstanceAssignmentConfigUtils.java](https://app.codecov.io/gh/apache/pinot/pull/18635?src=pr&el=tree&filepath=pinot-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fpinot%2Fcommon%2Fassignment%2FInstanceAssignmentConfigUtils.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vYXNzaWdubWVudC9JbnN0YW5jZUFzc2lnbm1lbnRDb25maWdVdGlscy5qYXZh) | 0.00% | [0 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/pinot/pull/18635?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
> :exclamation: There is a different number of reports uploaded between BASE (aee5a1f) and HEAD (b88c463). Click for more details.
>
> <details><summary>HEAD has 16 uploads less than BASE</summary>
>
>| Flag | BASE (aee5a1f) | HEAD (b88c463) |
>|------|------|------|
>|java-21|5|1|
>|unittests|2|1|
>|temurin|5|1|
>|unittests2|1|0|
>|integration|3|0|
>|integration2|1|0|
>|integration1|1|0|
>|custom-integration1|1|0|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18635 +/- ##
============================================
- Coverage 64.45% 56.81% -7.64%
+ Complexity 1137 1 -1136
============================================
Files 3337 2571 -766
Lines 206067 149211 -56856
Branches 32127 24110 -8017
============================================
- Hits 132816 84779 -48037
+ Misses 62599 57260 -5339
+ Partials 10652 7172 -3480
```
| [Flag](https://app.codecov.io/gh/apache/pinot/pull/18635/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [custom-integration1](https://app.codecov.io/gh/apache/pinot/pull/18635/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [integration](https://app.codecov.io/gh/apache/pinot/pull/18635/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [integration1](https://app.codecov.io/gh/apache/pinot/pull/18635/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [integration2](https://app.codecov.io/gh/apache/pinot/pull/18635/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [java-21](https://app.codecov.io/gh/apache/pinot/pull/18635/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `56.81% <0.00%> (-7.64%)` | :arrow_down: |
| [temurin](https://app.codecov.io/gh/apache/pinot/pull/18635/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `56.81% <0.00%> (-7.64%)` | :arrow_down: |
| [unittests](https://app.codecov.io/gh/apache/pinot/pull/18635/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `56.81% <0.00%> (-7.64%)` | :arrow_down: |
| [unittests1](https://app.codecov.io/gh/apache/pinot/pull/18635/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `56.81% <0.00%> (-0.02%)` | :arrow_down: |
| [unittests2](https://app.codecov.io/gh/apache/pinot/pull/18635/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
</details>
[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/apache/pinot/pull/18635?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
<details><summary> :rocket: New features to boost your workflow: </summary>
- :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org