You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by "Jackie-Jiang (via GitHub)" <gi...@apache.org> on 2025/04/22 23:29:07 UTC
[PR] Replace RebalanceConfig.MinimizeDataMovementOptions with Enablement [pinot]
Jackie-Jiang opened a new pull request, #15619:
URL: https://github.com/apache/pinot/pull/15619
(no comment)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] Replace RebalanceConfig.MinimizeDataMovementOptions with Enablement [pinot]
Posted by "Jackie-Jiang (via GitHub)" <gi...@apache.org>.
Jackie-Jiang merged PR #15619:
URL: https://github.com/apache/pinot/pull/15619
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] Replace RebalanceConfig.MinimizeDataMovementOptions with Enablement [pinot]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #15619:
URL: https://github.com/apache/pinot/pull/15619#issuecomment-2822752566
## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/15619?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
All modified and coverable lines are covered by tests :white_check_mark:
> Project coverage is 55.66%. Comparing base [(`9f030bb`)](https://app.codecov.io/gh/apache/pinot/commit/9f030bb0f20de69595156c18188946b2d2876e1c?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) to head [(`0ca4e96`)](https://app.codecov.io/gh/apache/pinot/commit/0ca4e96f654d0d68f09a68166ec00307dd361bc2?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
> :exclamation: There is a different number of reports uploaded between BASE (9f030bb) and HEAD (0ca4e96). Click for more details.
>
> <details><summary>HEAD has 67 uploads less than BASE</summary>
>
>| Flag | BASE (9f030bb) | HEAD (0ca4e96) |
>|------|------|------|
>|java-21|10|1|
>|unittests|6|1|
>|unittests1|3|1|
>|skip-bytebuffers-true|4|1|
>|temurin|15|1|
>|skip-bytebuffers-false|8|0|
>|unittests2|3|0|
>|java-11|5|0|
>|integration|9|0|
>|integration2|3|0|
>|integration1|3|0|
>|custom-integration1|3|0|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #15619 +/- ##
============================================
- Coverage 62.77% 55.66% -7.11%
+ Complexity 1384 675 -709
============================================
Files 2864 2239 -625
Lines 162734 121102 -41632
Branches 24917 19325 -5592
============================================
- Hits 102151 67411 -34740
+ Misses 52875 48139 -4736
+ Partials 7708 5552 -2156
```
| [Flag](https://app.codecov.io/gh/apache/pinot/pull/15619/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [custom-integration1](https://app.codecov.io/gh/apache/pinot/pull/15619/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [integration](https://app.codecov.io/gh/apache/pinot/pull/15619/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [integration1](https://app.codecov.io/gh/apache/pinot/pull/15619/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [integration2](https://app.codecov.io/gh/apache/pinot/pull/15619/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [java-11](https://app.codecov.io/gh/apache/pinot/pull/15619/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [java-21](https://app.codecov.io/gh/apache/pinot/pull/15619/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `55.66% <ø> (-7.08%)` | :arrow_down: |
| [skip-bytebuffers-false](https://app.codecov.io/gh/apache/pinot/pull/15619/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [skip-bytebuffers-true](https://app.codecov.io/gh/apache/pinot/pull/15619/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `55.66% <ø> (-7.06%)` | :arrow_down: |
| [temurin](https://app.codecov.io/gh/apache/pinot/pull/15619/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `55.66% <ø> (-7.11%)` | :arrow_down: |
| [unittests](https://app.codecov.io/gh/apache/pinot/pull/15619/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `55.66% <ø> (-7.11%)` | :arrow_down: |
| [unittests1](https://app.codecov.io/gh/apache/pinot/pull/15619/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `55.66% <ø> (-0.04%)` | :arrow_down: |
| [unittests2](https://app.codecov.io/gh/apache/pinot/pull/15619/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
</details>
[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/apache/pinot/pull/15619?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
<details><summary> :rocket: New features to boost your workflow: </summary>
- :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] Replace RebalanceConfig.MinimizeDataMovementOptions with Enablement [pinot]
Posted by "somandal (via GitHub)" <gi...@apache.org>.
somandal commented on code in PR #15619:
URL: https://github.com/apache/pinot/pull/15619#discussion_r2055077997
##########
pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/assignment/instance/InstanceAssignmentDriver.java:
##########
@@ -104,17 +106,15 @@ public InstancePartitions assignInstances(String tierName, List<InstanceConfig>
private InstancePartitions getInstancePartitions(String instancePartitionsName,
InstanceAssignmentConfig instanceAssignmentConfig, List<InstanceConfig> instanceConfigs,
@Nullable InstancePartitions existingInstancePartitions,
- @Nullable InstancePartitions preConfiguredInstancePartitions, @Nullable Boolean minimizeDataMovementOverride) {
+ @Nullable InstancePartitions preConfiguredInstancePartitions, Enablement minimizeDataMovementEnablement) {
String tableNameWithType = _tableConfig.getTableName();
// minimizeDataMovement might be set back to false within InstanceTagPoolSelector and InstancePartitionSelector
// if existingInstancePartitions is null.
boolean minimizeDataMovement =
- minimizeDataMovementOverride == null ? instanceAssignmentConfig.isMinimizeDataMovement()
- : minimizeDataMovementOverride;
- LOGGER.info("Starting {} instance assignment for table {}, original minimizeDataMovement: {}, "
- + "overriding with minimizeDataMovementOverride: {}", instancePartitionsName, tableNameWithType,
- instanceAssignmentConfig.isMinimizeDataMovement(), minimizeDataMovementOverride);
+ minimizeDataMovementEnablement.isEnabled(instanceAssignmentConfig::isMinimizeDataMovement);
+ LOGGER.info("Starting {} instance assignment for table: {} with minimizeDataMovement: {}", instancePartitionsName,
Review Comment:
nit: shall we also log the `instanceAssignmentConfig.isMinimizeDataMovement()` value for debugging purposes?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] Replace RebalanceConfig.MinimizeDataMovementOptions with Enablement [pinot]
Posted by "Jackie-Jiang (via GitHub)" <gi...@apache.org>.
Jackie-Jiang commented on code in PR #15619:
URL: https://github.com/apache/pinot/pull/15619#discussion_r2056669231
##########
pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/assignment/instance/InstanceAssignmentDriver.java:
##########
@@ -104,17 +106,15 @@ public InstancePartitions assignInstances(String tierName, List<InstanceConfig>
private InstancePartitions getInstancePartitions(String instancePartitionsName,
InstanceAssignmentConfig instanceAssignmentConfig, List<InstanceConfig> instanceConfigs,
@Nullable InstancePartitions existingInstancePartitions,
- @Nullable InstancePartitions preConfiguredInstancePartitions, @Nullable Boolean minimizeDataMovementOverride) {
+ @Nullable InstancePartitions preConfiguredInstancePartitions, Enablement minimizeDataMovementEnablement) {
String tableNameWithType = _tableConfig.getTableName();
// minimizeDataMovement might be set back to false within InstanceTagPoolSelector and InstancePartitionSelector
// if existingInstancePartitions is null.
boolean minimizeDataMovement =
- minimizeDataMovementOverride == null ? instanceAssignmentConfig.isMinimizeDataMovement()
- : minimizeDataMovementOverride;
- LOGGER.info("Starting {} instance assignment for table {}, original minimizeDataMovement: {}, "
- + "overriding with minimizeDataMovementOverride: {}", instancePartitionsName, tableNameWithType,
- instanceAssignmentConfig.isMinimizeDataMovement(), minimizeDataMovementOverride);
+ minimizeDataMovementEnablement.isEnabled(instanceAssignmentConfig::isMinimizeDataMovement);
+ LOGGER.info("Starting {} instance assignment for table: {} with minimizeDataMovement: {}", instancePartitionsName,
Review Comment:
Sure
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org