You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@helix.apache.org by GitBox <gi...@apache.org> on 2021/03/23 18:20:22 UTC
[GitHub] [helix] jiajunwang opened a new pull request #1676: Improve the GenericHelixController global tracking record to support multiple controller objects for the same cluster in one JVM.
jiajunwang opened a new pull request #1676:
URL: https://github.com/apache/helix/pull/1676
### Issues
- [X] My PR addresses the following Helix issues and references them in the PR description:
Resolves #1617
### Description
- [X] Here are some details about my PR, including screenshots of any UI changes:
This PR refines the GenericHelixController global tracking record so it acts as expected even we have more than one instance in the JVM. This impacts test logic only in general.
### Tests
- [X] The following tests are written for this issue:
The previously added workaround code has been removed. So the new logic can be tested by the original code.
- The following is the result of the "mvn test" command on the appropriate module:
(If CI test fails due to known issue, please specify the issue and test PR locally. Then copy & paste the result of "mvn test" to here.)
### Documentation (Optional)
- In case of new functionality, my PR adds documentation in the following wiki page:
(Link the GitHub wiki you added)
### Commits
- My commits all reference appropriate Apache Helix GitHub issues in their subject lines. In addition, my commits follow the guidelines from "[How to write a good git commit message](http://chris.beams.io/posts/git-commit/)":
1. Subject is separated from body by a blank line
1. Subject is limited to 50 characters (not including Jira issue reference)
1. Subject does not end with a period
1. Subject uses the imperative mood ("add", not "adding")
1. Body wraps at 72 characters
1. Body explains "what" and "why", not "how"
### Code Quality
- My diff has been formatted using helix-style.xml
(helix-style-intellij.xml if IntelliJ IDE is used)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org
[GitHub] [helix] jiajunwang commented on pull request #1676: Improve the GenericHelixController global tracking record to support multiple controller objects for the same cluster in one JVM.
Posted by GitBox <gi...@apache.org>.
jiajunwang commented on pull request #1676:
URL: https://github.com/apache/helix/pull/1676#issuecomment-808442595
Approved by @dasahcc , I will merge the code. Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org
[GitHub] [helix] lei-xia commented on a change in pull request #1676: Improve the GenericHelixController global tracking record to support multiple controller objects for the same cluster in one JVM.
Posted by GitBox <gi...@apache.org>.
lei-xia commented on a change in pull request #1676:
URL: https://github.com/apache/helix/pull/1676#discussion_r600847152
##########
File path: helix-core/src/main/java/org/apache/helix/controller/GenericHelixController.java
##########
@@ -225,10 +227,52 @@ protected StatefulRebalancer createRebalancer(HelixManager helixManager) {
* 1) ZK callback should go to ClusterDataCache and trigger data cache refresh only
* 2) then ClusterDataCache.refresh triggers rebalance pipeline.
*/
- /* Map of cluster->GenrichelixController */
- private static Map<String, GenericHelixController> HelixControllerFactory = new ConcurrentHashMap<>();
- public static GenericHelixController getController(String clusterName) {
- return HelixControllerFactory.get(clusterName);
+ /* Map of cluster->Set of GenericHelixController*/
+ private static Map<String, ImmutableSet<GenericHelixController>> _helixControllerFactory =
Review comment:
minor: for static variables, what is our name convention? I think _wordAnotherWord format is for instance variable?
##########
File path: helix-core/src/main/java/org/apache/helix/controller/GenericHelixController.java
##########
@@ -225,10 +227,52 @@ protected StatefulRebalancer createRebalancer(HelixManager helixManager) {
* 1) ZK callback should go to ClusterDataCache and trigger data cache refresh only
* 2) then ClusterDataCache.refresh triggers rebalance pipeline.
*/
- /* Map of cluster->GenrichelixController */
- private static Map<String, GenericHelixController> HelixControllerFactory = new ConcurrentHashMap<>();
- public static GenericHelixController getController(String clusterName) {
- return HelixControllerFactory.get(clusterName);
+ /* Map of cluster->Set of GenericHelixController*/
+ private static Map<String, ImmutableSet<GenericHelixController>> _helixControllerFactory =
+ new ConcurrentHashMap<>();
+
+ public static GenericHelixController getLeaderController(String clusterName) {
Review comment:
Same question, why we could have two controllers for same cluster? Is this just happening in tests because the controller from other tests were not properly closed?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org
[GitHub] [helix] jiajunwang commented on a change in pull request #1676: Improve the GenericHelixController global tracking record to support multiple controller objects for the same cluster in one JVM.
Posted by GitBox <gi...@apache.org>.
jiajunwang commented on a change in pull request #1676:
URL: https://github.com/apache/helix/pull/1676#discussion_r600978316
##########
File path: helix-core/src/main/java/org/apache/helix/controller/GenericHelixController.java
##########
@@ -225,10 +227,52 @@ protected StatefulRebalancer createRebalancer(HelixManager helixManager) {
* 1) ZK callback should go to ClusterDataCache and trigger data cache refresh only
* 2) then ClusterDataCache.refresh triggers rebalance pipeline.
*/
- /* Map of cluster->GenrichelixController */
- private static Map<String, GenericHelixController> HelixControllerFactory = new ConcurrentHashMap<>();
- public static GenericHelixController getController(String clusterName) {
- return HelixControllerFactory.get(clusterName);
+ /* Map of cluster->Set of GenericHelixController*/
+ private static Map<String, ImmutableSet<GenericHelixController>> _helixControllerFactory =
Review comment:
The constants (static final) should be all upper case + underscore. But this var is not constants. Then, all the other related naming conventions include,
1. Class and instance variables: _fooBar (this is the one that makes sense)
2. Local variables: fooBar
On the other hand, HelixControllerFactory is not a good option. Since it conflicts with the Classes/Interfaces naming convention.
And there is no specific convention for static variables. IMHO, it is not necessary.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org
[GitHub] [helix] dasahcc commented on a change in pull request #1676: Improve the GenericHelixController global tracking record to support multiple controller objects for the same cluster in one JVM.
Posted by GitBox <gi...@apache.org>.
dasahcc commented on a change in pull request #1676:
URL: https://github.com/apache/helix/pull/1676#discussion_r600778959
##########
File path: helix-core/src/main/java/org/apache/helix/controller/GenericHelixController.java
##########
@@ -225,10 +227,52 @@ protected StatefulRebalancer createRebalancer(HelixManager helixManager) {
* 1) ZK callback should go to ClusterDataCache and trigger data cache refresh only
* 2) then ClusterDataCache.refresh triggers rebalance pipeline.
*/
- /* Map of cluster->GenrichelixController */
- private static Map<String, GenericHelixController> HelixControllerFactory = new ConcurrentHashMap<>();
- public static GenericHelixController getController(String clusterName) {
- return HelixControllerFactory.get(clusterName);
+ /* Map of cluster->Set of GenericHelixController*/
+ private static Map<String, ImmutableSet<GenericHelixController>> _helixControllerFactory =
+ new ConcurrentHashMap<>();
+
+ public static GenericHelixController getLeaderController(String clusterName) {
Review comment:
Question: Is this necessary? This is usually in memory triggered rebalance. It only triggers when the node is leader for the cluster. Also you should not see one generic helix controller perform as LEADER and STANDBY at same time for a cluster. Do we need this set?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org
[GitHub] [helix] jiajunwang merged pull request #1676: Improve the GenericHelixController global tracking record to support multiple controller objects for the same cluster in one JVM.
Posted by GitBox <gi...@apache.org>.
jiajunwang merged pull request #1676:
URL: https://github.com/apache/helix/pull/1676
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org
[GitHub] [helix] jiajunwang commented on a change in pull request #1676: Improve the GenericHelixController global tracking record to support multiple controller objects for the same cluster in one JVM.
Posted by GitBox <gi...@apache.org>.
jiajunwang commented on a change in pull request #1676:
URL: https://github.com/apache/helix/pull/1676#discussion_r600783822
##########
File path: helix-core/src/main/java/org/apache/helix/controller/GenericHelixController.java
##########
@@ -225,10 +227,52 @@ protected StatefulRebalancer createRebalancer(HelixManager helixManager) {
* 1) ZK callback should go to ClusterDataCache and trigger data cache refresh only
* 2) then ClusterDataCache.refresh triggers rebalance pipeline.
*/
- /* Map of cluster->GenrichelixController */
- private static Map<String, GenericHelixController> HelixControllerFactory = new ConcurrentHashMap<>();
- public static GenericHelixController getController(String clusterName) {
- return HelixControllerFactory.get(clusterName);
+ /* Map of cluster->Set of GenericHelixController*/
+ private static Map<String, ImmutableSet<GenericHelixController>> _helixControllerFactory =
+ new ConcurrentHashMap<>();
+
+ public static GenericHelixController getLeaderController(String clusterName) {
Review comment:
This problem fails more than 2 tests. In production, and if the application dev is reasonable, then it won't appear.
My take is that 1. the new change is harmless for the production logic. 2. if not doing this, we have to hack our test logic. So it may hide some real problem. Since the test is not doing what production code does.
So it is really necessary.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org
[GitHub] [helix] jiajunwang commented on a change in pull request #1676: Improve the GenericHelixController global tracking record to support multiple controller objects for the same cluster in one JVM.
Posted by GitBox <gi...@apache.org>.
jiajunwang commented on a change in pull request #1676:
URL: https://github.com/apache/helix/pull/1676#discussion_r600978316
##########
File path: helix-core/src/main/java/org/apache/helix/controller/GenericHelixController.java
##########
@@ -225,10 +227,52 @@ protected StatefulRebalancer createRebalancer(HelixManager helixManager) {
* 1) ZK callback should go to ClusterDataCache and trigger data cache refresh only
* 2) then ClusterDataCache.refresh triggers rebalance pipeline.
*/
- /* Map of cluster->GenrichelixController */
- private static Map<String, GenericHelixController> HelixControllerFactory = new ConcurrentHashMap<>();
- public static GenericHelixController getController(String clusterName) {
- return HelixControllerFactory.get(clusterName);
+ /* Map of cluster->Set of GenericHelixController*/
+ private static Map<String, ImmutableSet<GenericHelixController>> _helixControllerFactory =
Review comment:
The constants (static final) should be all upper case + underscore. But this var is not constants. Then, all the other related naming conventions include,
1. Class and instance variables: _fooBar (this is the one that makes sense)
2. Local variables: fooBar
On the other hand, HelixControllerFactory is not a good option. Since it conflicts with the Classes/Interfaces naming convention.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org
[GitHub] [helix] jiajunwang commented on a change in pull request #1676: Improve the GenericHelixController global tracking record to support multiple controller objects for the same cluster in one JVM.
Posted by GitBox <gi...@apache.org>.
jiajunwang commented on a change in pull request #1676:
URL: https://github.com/apache/helix/pull/1676#discussion_r600975996
##########
File path: helix-core/src/main/java/org/apache/helix/controller/GenericHelixController.java
##########
@@ -225,10 +227,52 @@ protected StatefulRebalancer createRebalancer(HelixManager helixManager) {
* 1) ZK callback should go to ClusterDataCache and trigger data cache refresh only
* 2) then ClusterDataCache.refresh triggers rebalance pipeline.
*/
- /* Map of cluster->GenrichelixController */
- private static Map<String, GenericHelixController> HelixControllerFactory = new ConcurrentHashMap<>();
- public static GenericHelixController getController(String clusterName) {
- return HelixControllerFactory.get(clusterName);
+ /* Map of cluster->Set of GenericHelixController*/
+ private static Map<String, ImmutableSet<GenericHelixController>> _helixControllerFactory =
+ new ConcurrentHashMap<>();
+
+ public static GenericHelixController getLeaderController(String clusterName) {
Review comment:
There are more than one test cases need to start multiple controllers. If not doing so, then all these tests have to trigger the rebalance manually, since the auto scheduled ondemand rebalance won't work. This is a potential problem since the workaround hide problems.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org