You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by "J-HowHuang (via GitHub)" <gi...@apache.org> on 2025/11/04 18:48:33 UTC
[PR] Fix edge case when rebalanceNonReplicaGroupBasedTable doesn't assign the segment evenly [pinot]
J-HowHuang opened a new pull request, #17138:
URL: https://github.com/apache/pinot/pull/17138
## Description
The greedy algorithm to re-assign segments according to their current assignment was introduced in this PR: https://github.com/apache/pinot/pull/16135
The algorithm didn't account for the following scenario:
```
3 servers, 3 replicas:
+----------+----------+----------+
| Server 1 | Server 2 | Server 3 |
+----------+----------+----------+
| seg_1 | seg_1 | seg_1 |
| seg_2 | seg_2 | seg_2 |
| seg_3 | seg_3 | seg_3 |
| seg_4 | seg_4 | seg_4 |
| seg_5 | seg_5 | seg_5 |
| seg_6 | seg_6 | seg_6 |
+----------+----------+----------+
```
Now the replication is reduced to 2. For each segment 1-6, the algorithm is going to choose the first two servers by the default order (I'm not sure if it's alphabetical but I assume) from the current assignment ideal state, until the server is assigned with 4 segments (6 segments * 2 replicas / 3 servers). The default first two are server 1 and 2, and is identical across all segments.
The algorithm will assign the segment like this initially:
```
+----------+----------+----------+
| Server 1 | Server 2 | Server 3 |
+----------+----------+----------+
| seg_1 | seg_1 | |
| seg_2 | seg_2 | |
| seg_3 | seg_3 | |
| seg_4 | seg_4 | |
| | | |
| | | |
+----------+----------+----------+
```
Then, for segment 5 and 6, because the first two servers in the default order are assigned 4 already, it's added to the third server in the list
```
+----------+----------+----------+
| Server 1 | Server 2 | Server 3 |
+----------+----------+----------+
| seg_1 | seg_1 | seg_5 |
| seg_2 | seg_2 | seg_6 |
| seg_3 | seg_3 | |
| seg_4 | seg_4 | |
| | | |
| | | |
+----------+----------+----------+
```
Now we have one replica of each of segment 5 and 6 left, because they cannot be assigned to server 3 again. The algorithm then assign them to the server hosting least segments
```
+----------+----------+----------+
| Server 1 | Server 2 | Server 3 |
+----------+----------+----------+
| seg_1 | seg_1 | seg_5 |
| seg_2 | seg_2 | seg_6 |
| seg_3 | seg_3 | |
| seg_4 | seg_4 | |
| seg_5 | seg_6 | |
| | | |
+----------+----------+----------+
```
This is why the following rebalance summary was seen:
```
{
"numServersGettingNewSegments": 0,
"numServers": {
"valueBeforeRebalance": 3,
"expectedValueAfterRebalance": 3
},
"serversAdded": [],
"serversRemoved": [],
"serversUnchanged": [
"Server_pinot-server-server-0-5",
"Server_pinot-server-server-0-4",
"Server_pinot-server-server-0-6"
],
"serversGettingNewSegments": [],
"serverSegmentChangeInfo": {
"Server_pinot-server-server-0-5": {
"serverStatus": "UNCHANGED",
"totalSegmentsAfterRebalance": 1025, <----------------- uneven
"totalSegmentsBeforeRebalance": 1230,
"segmentsAdded": 0,
"segmentsDeleted": 205,
"segmentsUnchanged": 1025,
"tagList": [
"DefaultTenant_REALTIME",
"DefaultTenant_OFFLINE"
]
},
"Server_pinot-server-server-0-4": {
"serverStatus": "UNCHANGED",
"totalSegmentsAfterRebalance": 1025, <----------------- uneven
"totalSegmentsBeforeRebalance": 1230,
"segmentsAdded": 0,
"segmentsDeleted": 205,
"segmentsUnchanged": 1025,
"tagList": [
"DefaultTenant_OFFLINE",
"DefaultTenant_REALTIME"
]
},
"Server_pinot-server-server-0-6": {
"serverStatus": "UNCHANGED",
"totalSegmentsAfterRebalance": 410, <----------------- uneven
"totalSegmentsBeforeRebalance": 1230,
"segmentsAdded": 0,
"segmentsDeleted": 820,
"segmentsUnchanged": 410,
"tagList": [
"DefaultTenant_OFFLINE",
"DefaultTenant_REALTIME"
]
}
}
}
{
"totalSegmentsToBeMoved": 0,
"totalSegmentsToBeDeleted": 1230,
"maxSegmentsAddedToASingleServer": 0,
"estimatedAverageSegmentSizeInBytes": 1926625,
"totalEstimatedDataToBeMovedInBytes": 0,
"replicationFactor": {
"valueBeforeRebalance": 3,
"expectedValueAfterRebalance": 2 <----------------- we reduce the RF here
},
"numSegmentsInSingleReplica": {
"valueBeforeRebalance": 1230,
"expectedValueAfterRebalance": 1230
},
"numSegmentsAcrossAllReplicas": {
"valueBeforeRebalance": 3690,
"expectedValueAfterRebalance": 2460
}
}
```
## Change
The algorithm is modified by sorting the candidate servers by their current assigned segments count first, then use this order to assign the segment to give priority to those server who has fewer segments.
## Test
Add a unit test to address the above scenario. The original algorithm would've failed the test.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] Fix edge case when rebalanceNonReplicaGroupBasedTable doesn't assign the segment evenly [pinot]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #17138:
URL: https://github.com/apache/pinot/pull/17138#issuecomment-3487722843
## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/17138?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:white_check_mark: All modified and coverable lines are covered by tests.
:white_check_mark: Project coverage is 56.07%. Comparing base ([`eaaf1ac`](https://app.codecov.io/gh/apache/pinot/commit/eaaf1ac519a2e6e3c19bec523d5ed580af637e57?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`4b9e9dd`](https://app.codecov.io/gh/apache/pinot/commit/4b9e9dd17bfb5f21a68096ed4dcb6bf42330bf3c?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 5 commits behind head on master.
> :exclamation: There is a different number of reports uploaded between BASE (eaaf1ac) and HEAD (4b9e9dd). Click for more details.
>
> <details><summary>HEAD has 36 uploads less than BASE</summary>
>
>| Flag | BASE (eaaf1ac) | HEAD (4b9e9dd) |
>|------|------|------|
>|java-21|5|1|
>|unittests1|2|1|
>|unittests|4|1|
>|temurin|10|1|
>|java-11|5|0|
>|unittests2|2|0|
>|integration|6|0|
>|integration2|2|0|
>|integration1|2|0|
>|custom-integration1|2|0|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #17138 +/- ##
============================================
- Coverage 63.18% 56.07% -7.12%
+ Complexity 1428 701 -727
============================================
Files 3104 2426 -678
Lines 183501 135666 -47835
Branches 28129 21664 -6465
============================================
- Hits 115945 76074 -39871
+ Misses 58597 53210 -5387
+ Partials 8959 6382 -2577
```
| [Flag](https://app.codecov.io/gh/apache/pinot/pull/17138/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [custom-integration1](https://app.codecov.io/gh/apache/pinot/pull/17138/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [integration](https://app.codecov.io/gh/apache/pinot/pull/17138/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [integration1](https://app.codecov.io/gh/apache/pinot/pull/17138/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [integration2](https://app.codecov.io/gh/apache/pinot/pull/17138/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [java-11](https://app.codecov.io/gh/apache/pinot/pull/17138/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [java-21](https://app.codecov.io/gh/apache/pinot/pull/17138/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `56.07% <ø> (-7.09%)` | :arrow_down: |
| [temurin](https://app.codecov.io/gh/apache/pinot/pull/17138/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `56.07% <ø> (-7.12%)` | :arrow_down: |
| [unittests](https://app.codecov.io/gh/apache/pinot/pull/17138/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `56.07% <ø> (-7.11%)` | :arrow_down: |
| [unittests1](https://app.codecov.io/gh/apache/pinot/pull/17138/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `56.07% <ø> (-0.13%)` | :arrow_down: |
| [unittests2](https://app.codecov.io/gh/apache/pinot/pull/17138/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
</details>
[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/apache/pinot/pull/17138?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
<details><summary> :rocket: New features to boost your workflow: </summary>
- :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] Fix edge case when rebalanceNonReplicaGroupBasedTable doesn't assign the segment evenly [pinot]
Posted by "somandal (via GitHub)" <gi...@apache.org>.
somandal merged PR #17138:
URL: https://github.com/apache/pinot/pull/17138
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org