You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by "deepthi912 (via GitHub)" <gi...@apache.org> on 2025/01/31 06:00:29 UTC
[PR] Make HelixMaxStateTransitions extendable in BaseControllerStarter [pinot]
deepthi912 opened a new pull request, #14960:
URL: https://github.com/apache/pinot/pull/14960
Refactor to a separate method for HelixMaxTransitions and make it extendable
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] Make HelixMaxStateTransitions extendable in BaseControllerStarter [pinot]
Posted by "somandal (via GitHub)" <gi...@apache.org>.
somandal commented on code in PR #14960:
URL: https://github.com/apache/pinot/pull/14960#discussion_r1937499997
##########
pinot-controller/src/main/java/org/apache/pinot/controller/BaseControllerStarter.java:
##########
@@ -300,6 +300,10 @@ private void setupHelixSystemProperties() {
}
private void setupHelixClusterConstraints() {
+ setupInstanceMaxStateTransitions();
+ }
+
+ protected void setupInstanceMaxStateTransitions() {
Review Comment:
Instead of this, can't we just override the following property with a new default from the class that extends this one?:
```
public static final String CONFIG_OF_HELIX_INSTANCE_MAX_STATE_TRANSITIONS =
"pinot.helix.instance.state.maxStateTransitions";
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] Make HelixMaxStateTransitions extendable in BaseControllerStarter [pinot]
Posted by "deepthi912 (via GitHub)" <gi...@apache.org>.
deepthi912 commented on code in PR #14960:
URL: https://github.com/apache/pinot/pull/14960#discussion_r1937571752
##########
pinot-controller/src/main/java/org/apache/pinot/controller/BaseControllerStarter.java:
##########
@@ -300,6 +300,10 @@ private void setupHelixSystemProperties() {
}
private void setupHelixClusterConstraints() {
+ setupInstanceMaxStateTransitions();
+ }
+
+ protected void setupInstanceMaxStateTransitions() {
Review Comment:
Maybe I can also try to override if this property exists perhaps for the extendable class! Will close this.
##########
pinot-controller/src/main/java/org/apache/pinot/controller/BaseControllerStarter.java:
##########
@@ -300,6 +300,10 @@ private void setupHelixSystemProperties() {
}
private void setupHelixClusterConstraints() {
+ setupInstanceMaxStateTransitions();
+ }
+
+ protected void setupInstanceMaxStateTransitions() {
Review Comment:
Maybe I can also try to override if this property exists perhaps for the extendable class!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] Make HelixMaxStateTransitions extendable in BaseControllerStarter [pinot]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #14960:
URL: https://github.com/apache/pinot/pull/14960#issuecomment-2626424703
## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/14960?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
All modified and coverable lines are covered by tests :white_check_mark:
> Project coverage is 56.04%. Comparing base [(`59551e4`)](https://app.codecov.io/gh/apache/pinot/commit/59551e45224f1535c4863fd577622b37366ccc97?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) to head [(`8d9a317`)](https://app.codecov.io/gh/apache/pinot/commit/8d9a31729b34461d9397ea3921437f7d00118354?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
> Report is 1652 commits behind head on master.
> :exclamation: There is a different number of reports uploaded between BASE (59551e4) and HEAD (8d9a317). Click for more details.
>
> <details><summary>HEAD has 53 uploads less than BASE</summary>
>
>| Flag | BASE (59551e4) | HEAD (8d9a317) |
>|------|------|------|
>|integration|7|0|
>|integration2|3|0|
>|temurin|12|1|
>|java-21|7|1|
>|skip-bytebuffers-true|3|1|
>|skip-bytebuffers-false|7|0|
>|unittests|5|1|
>|unittests1|2|1|
>|java-11|5|0|
>|unittests2|3|0|
>|integration1|2|0|
>|custom-integration1|2|0|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #14960 +/- ##
============================================
- Coverage 61.75% 56.04% -5.72%
- Complexity 207 703 +496
============================================
Files 2436 2122 -314
Lines 133233 114552 -18681
Branches 20636 18432 -2204
============================================
- Hits 82274 64197 -18077
- Misses 44911 45102 +191
+ Partials 6048 5253 -795
```
| [Flag](https://app.codecov.io/gh/apache/pinot/pull/14960/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [custom-integration1](https://app.codecov.io/gh/apache/pinot/pull/14960/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [integration](https://app.codecov.io/gh/apache/pinot/pull/14960/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [integration1](https://app.codecov.io/gh/apache/pinot/pull/14960/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [integration2](https://app.codecov.io/gh/apache/pinot/pull/14960/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [java-11](https://app.codecov.io/gh/apache/pinot/pull/14960/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [java-21](https://app.codecov.io/gh/apache/pinot/pull/14960/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `56.04% <ø> (-5.59%)` | :arrow_down: |
| [skip-bytebuffers-false](https://app.codecov.io/gh/apache/pinot/pull/14960/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [skip-bytebuffers-true](https://app.codecov.io/gh/apache/pinot/pull/14960/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `56.04% <ø> (+28.31%)` | :arrow_up: |
| [temurin](https://app.codecov.io/gh/apache/pinot/pull/14960/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `56.04% <ø> (-5.72%)` | :arrow_down: |
| [unittests](https://app.codecov.io/gh/apache/pinot/pull/14960/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `56.04% <ø> (-5.71%)` | :arrow_down: |
| [unittests1](https://app.codecov.io/gh/apache/pinot/pull/14960/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `56.04% <ø> (+9.15%)` | :arrow_up: |
| [unittests2](https://app.codecov.io/gh/apache/pinot/pull/14960/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
</details>
[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/apache/pinot/pull/14960?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] Make HelixMaxStateTransitions extendable in BaseControllerStarter [pinot]
Posted by "deepthi912 (via GitHub)" <gi...@apache.org>.
deepthi912 closed pull request #14960: Make HelixMaxStateTransitions extendable in BaseControllerStarter
URL: https://github.com/apache/pinot/pull/14960
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] Make HelixMaxStateTransitions extendable in BaseControllerStarter [pinot]
Posted by "deepthi912 (via GitHub)" <gi...@apache.org>.
deepthi912 commented on code in PR #14960:
URL: https://github.com/apache/pinot/pull/14960#discussion_r1937548486
##########
pinot-controller/src/main/java/org/apache/pinot/controller/BaseControllerStarter.java:
##########
@@ -300,6 +300,10 @@ private void setupHelixSystemProperties() {
}
private void setupHelixClusterConstraints() {
+ setupInstanceMaxStateTransitions();
+ }
+
+ protected void setupInstanceMaxStateTransitions() {
Review Comment:
We could do that but not for every controller mode this was applicable which is why I tried to make this method extendable for easy navigation:
If you see :
```
switch (_controllerMode) {
case DUAL:
setUpHelixController();
setUpPinotController();
break;
case PINOT_ONLY:
setUpPinotController();
break;
case HELIX_ONLY:
setUpHelixController();
break;
default:
LOGGER.error("Invalid mode: {}", _controllerMode);
break;
}
```
DUAL, HELIX_ONLY --- Yes the other way around overridePropertyIfExists also works!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] Make HelixMaxStateTransitions extendable in BaseControllerStarter [pinot]
Posted by "somandal (via GitHub)" <gi...@apache.org>.
somandal commented on code in PR #14960:
URL: https://github.com/apache/pinot/pull/14960#discussion_r1937559296
##########
pinot-controller/src/main/java/org/apache/pinot/controller/BaseControllerStarter.java:
##########
@@ -300,6 +300,10 @@ private void setupHelixSystemProperties() {
}
private void setupHelixClusterConstraints() {
+ setupInstanceMaxStateTransitions();
+ }
+
+ protected void setupInstanceMaxStateTransitions() {
Review Comment:
Would that matter though, if all the controllers set the same property default? the Dual/Helix only one will pick it up, right? every cluster has to have at least one Pinot controller with Dual/Helix only? but maybe I'm missing something. Good to see if you can try it out with a couple of controllers on your dev machine?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] Make HelixMaxStateTransitions extendable in BaseControllerStarter [pinot]
Posted by "deepthi912 (via GitHub)" <gi...@apache.org>.
deepthi912 commented on code in PR #14960:
URL: https://github.com/apache/pinot/pull/14960#discussion_r1937548486
##########
pinot-controller/src/main/java/org/apache/pinot/controller/BaseControllerStarter.java:
##########
@@ -300,6 +300,10 @@ private void setupHelixSystemProperties() {
}
private void setupHelixClusterConstraints() {
+ setupInstanceMaxStateTransitions();
+ }
+
+ protected void setupInstanceMaxStateTransitions() {
Review Comment:
We could do that but not for every controller mode this was not applicable which is why I tried to make this method extendable:
If you see :
```
switch (_controllerMode) {
case DUAL:
setUpHelixController();
setUpPinotController();
break;
case PINOT_ONLY:
setUpPinotController();
break;
case HELIX_ONLY:
setUpHelixController();
break;
default:
LOGGER.error("Invalid mode: {}", _controllerMode);
break;
}
```
DUAL, HELIX_ONLY --- Yes the other way around overridePropertyIfExists also works!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] Make HelixMaxStateTransitions extendable in BaseControllerStarter [pinot]
Posted by "deepthi912 (via GitHub)" <gi...@apache.org>.
deepthi912 commented on code in PR #14960:
URL: https://github.com/apache/pinot/pull/14960#discussion_r1937548486
##########
pinot-controller/src/main/java/org/apache/pinot/controller/BaseControllerStarter.java:
##########
@@ -300,6 +300,10 @@ private void setupHelixSystemProperties() {
}
private void setupHelixClusterConstraints() {
+ setupInstanceMaxStateTransitions();
+ }
+
+ protected void setupInstanceMaxStateTransitions() {
Review Comment:
We could do that but not for every controller mode this was applicable which is why I tried to make this method extendable:
If you see :
```
switch (_controllerMode) {
case DUAL:
setUpHelixController();
setUpPinotController();
break;
case PINOT_ONLY:
setUpPinotController();
break;
case HELIX_ONLY:
setUpHelixController();
break;
default:
LOGGER.error("Invalid mode: {}", _controllerMode);
break;
}
```
DUAL, HELIX_ONLY --- Yes the other way around overridePropertyIfExists also works!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] Make HelixMaxStateTransitions extendable in BaseControllerStarter [pinot]
Posted by "somandal (via GitHub)" <gi...@apache.org>.
somandal commented on code in PR #14960:
URL: https://github.com/apache/pinot/pull/14960#discussion_r1937499997
##########
pinot-controller/src/main/java/org/apache/pinot/controller/BaseControllerStarter.java:
##########
@@ -300,6 +300,10 @@ private void setupHelixSystemProperties() {
}
private void setupHelixClusterConstraints() {
+ setupInstanceMaxStateTransitions();
+ }
+
+ protected void setupInstanceMaxStateTransitions() {
Review Comment:
Instead of this, can't we just override the following property with a new default from the class that extends this one?:
```
public static final String CONFIG_OF_HELIX_INSTANCE_MAX_STATE_TRANSITIONS =
"pinot.helix.instance.state.maxStateTransitions";
```
You'd need to override the default in the init() function in that case
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] Make HelixMaxStateTransitions extendable in BaseControllerStarter [pinot]
Posted by "deepthi912 (via GitHub)" <gi...@apache.org>.
deepthi912 commented on code in PR #14960:
URL: https://github.com/apache/pinot/pull/14960#discussion_r1937548486
##########
pinot-controller/src/main/java/org/apache/pinot/controller/BaseControllerStarter.java:
##########
@@ -300,6 +300,10 @@ private void setupHelixSystemProperties() {
}
private void setupHelixClusterConstraints() {
+ setupInstanceMaxStateTransitions();
+ }
+
+ protected void setupInstanceMaxStateTransitions() {
Review Comment:
We could do that but not for every controller mode this was not applicable which is why I tried to make this method extendable for easy navigation:
If you see :
```
switch (_controllerMode) {
case DUAL:
setUpHelixController();
setUpPinotController();
break;
case PINOT_ONLY:
setUpPinotController();
break;
case HELIX_ONLY:
setUpHelixController();
break;
default:
LOGGER.error("Invalid mode: {}", _controllerMode);
break;
}
```
DUAL, HELIX_ONLY --- Yes the other way around overridePropertyIfExists also works!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org