You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@helix.apache.org by "GrantPSpencer (via GitHub)" <gi...@apache.org> on 2024/08/29 04:33:21 UTC

[PR] Consider fault zones when sorting combinedPreferenceList [helix]

GrantPSpencer opened a new pull request, #2894:
URL: https://github.com/apache/helix/pull/2894

   ### Issues
   
   - [x] My PR addresses the following Helix issues and references them in the PR description:
   Fixes #2885 
   
   ### Description
   
   - [x] Here are some details about my PR, including screenshots of any UI changes:
   
   When determining which replica to drop between 2 replicas that are not in the preference list, controller should consider the fault zone representation. The controller should prioritize dropping a replica in a fault zone that already has another copy of the replica existing in it. If the controller drops a replica from a fault zone that already has another replica in it, then we will not decrease the # of fault zones where replicas for the partition exist.
   
   TODO:
   - [ ] Determine if sorting by MZ should only be done if the replica states are the same. With current change, sorting by MZ takes precedence of sorting by state for replicas that are both not in the preference list.
   
   ### Tests
   
   - [x] The following tests are written for this issue:
   
   `TestPreferenceListNodeComparatorWithTopologyAware.java`
   
   - The following is the result of the "mvn test" command on the appropriate module:
   
   ```
   $ mvn test -o -Dtest=TestPreferenceListNodeComparatorWithTopologyAware -pl=helix-core
   [INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0
   [INFO] 
   [INFO] ------------------------------------------------------------------------
   [INFO] BUILD SUCCESS
   [INFO] ------------------------------------------------------------------------
   [INFO] Total time:  29.011 s
   [INFO] Finished at: 2024-08-28T21:30:03-07:00
   [INFO] ------------------------------------------------------------------------
   ```
   
   ### Changes that Break Backward Compatibility (Optional)
   
   - My PR contains changes that break backward compatibility or previous assumptions for certain methods or API. They include:
   
   N/A
   
   ### Commits
   
   - My commits all reference appropriate Apache Helix GitHub issues in their subject lines. In addition, my commits follow the guidelines from "[How to write a good git commit message](http://chris.beams.io/posts/git-commit/)":
     1. Subject is separated from body by a blank line
     1. Subject is limited to 50 characters (not including Jira issue reference)
     1. Subject does not end with a period
     1. Subject uses the imperative mood ("add", not "adding")
     1. Body wraps at 72 characters
     1. Body explains "what" and "why", not "how"
   
   ### Code Quality
   
   - My diff has been formatted using helix-style.xml 
   (helix-style-intellij.xml if IntelliJ IDE is used)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org


Re: [PR] Consider fault zones when sorting combinedPreferenceList [helix]

Posted by "junkaixue (via GitHub)" <gi...@apache.org>.
junkaixue merged PR #2894:
URL: https://github.com/apache/helix/pull/2894


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org


Re: [PR] Consider fault zones when sorting combinedPreferenceList [helix]

Posted by "junkaixue (via GitHub)" <gi...@apache.org>.
junkaixue commented on code in PR #2894:
URL: https://github.com/apache/helix/pull/2894#discussion_r1736644539


##########
helix-core/src/main/java/org/apache/helix/controller/rebalancer/AbstractRebalancer.java:
##########
@@ -581,6 +585,46 @@ public int compare(String ins1, String ins2) {
       Integer p1 = Integer.MAX_VALUE;
       Integer p2 = Integer.MAX_VALUE;
 
+      // Order by MZ representation (overrepresented should be dropped first)
+      String mz1 = null;
+      String mz2 = null;
+      String faultZoneType = null;
+
+      if (_cache != null && _cache.getInstanceConfigMap() != null &&
+          _cache.getClusterConfig() != null && _cache.getClusterConfig().isTopologyAwareEnabled()) {
+
+        faultZoneType = _cache.getClusterConfig().getFaultZoneType();
+        mz1 = _cache.getInstanceConfigMap().get(ins1) != null ?
+            _cache.getInstanceConfigMap().get(ins1).getDomainAsMap().get(faultZoneType) : null;
+        mz2 = _cache.getInstanceConfigMap().get(ins2) != null ?
+            _cache.getInstanceConfigMap().get(ins2).getDomainAsMap().get(faultZoneType) : null;
+      }
+
+      // If mz is not valid for either instance, skip mz comparison
+      // If instances are in the same MZ, skip mz comparison
+      if (mz1 != null && mz2 != null && !mz1.equals(mz2)) {
+        // Count number of replicas in each MZ
+        int mz1Count = 0;
+        int mz2Count = 0;
+        for (String instance : _currentStateMap.keySet()) {
+            if (ins1.equals(instance) || ins2.equals(instance)) {
+              continue;
+            }
+
+            String currMz = _cache.getInstanceConfigMap().get(instance).getDomainAsMap().get(faultZoneType);
+
+            if (mz1.equals(currMz)) {
+              mz1Count++;
+            } else if (mz2.equals(currMz)) {
+              mz2Count++;
+            }
+        }
+        if (mz1Count != mz2Count) {
+          return mz1Count - mz2Count;
+        }

Review Comment:
   You are counting instances per mz. It is expensive to recompute the counting looping through the current state map.
   
   You can make a compute and loop current state to compute MZ counting mapping in constructor once. And every compare can leverage that map.



##########
helix-core/src/main/java/org/apache/helix/controller/rebalancer/AbstractRebalancer.java:
##########
@@ -557,20 +557,24 @@ protected static class PreferenceListNodeComparator implements Comparator<String
     protected final Map<String, String> _currentStateMap;
     protected final StateModelDefinition _stateModelDef;
     protected final List<String> _preferenceList;
+    ResourceControllerDataProvider _cache;
 
-    public PreferenceListNodeComparator(Map<String, String> currentStateMap,
-        StateModelDefinition stateModelDef, List<String> preferenceList) {
+    public PreferenceListNodeComparator(Map<String, String> currentStateMap, StateModelDefinition stateModelDef,
+        List<String> preferenceList, ResourceControllerDataProvider cache) {

Review Comment:
   We cannot directly change the public constructor in open source.
   
   You can add another constructor and keep the original one. So just pass null for the cache and invoke the new constructor for the old one like:
   this (xxxxxx).
   
   
   This can help you do the compute control as well. Means when cache is null, skip the MZ sorting and keep the old logic.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org


Re: [PR] Consider fault zones when sorting combinedPreferenceList [helix]

Posted by "GrantPSpencer (via GitHub)" <gi...@apache.org>.
GrantPSpencer commented on code in PR #2894:
URL: https://github.com/apache/helix/pull/2894#discussion_r1736791991


##########
helix-core/src/main/java/org/apache/helix/controller/rebalancer/AbstractRebalancer.java:
##########
@@ -581,6 +585,46 @@ public int compare(String ins1, String ins2) {
       Integer p1 = Integer.MAX_VALUE;
       Integer p2 = Integer.MAX_VALUE;
 
+      // Order by MZ representation (overrepresented should be dropped first)
+      String mz1 = null;
+      String mz2 = null;
+      String faultZoneType = null;
+
+      if (_cache != null && _cache.getInstanceConfigMap() != null &&
+          _cache.getClusterConfig() != null && _cache.getClusterConfig().isTopologyAwareEnabled()) {
+
+        faultZoneType = _cache.getClusterConfig().getFaultZoneType();
+        mz1 = _cache.getInstanceConfigMap().get(ins1) != null ?
+            _cache.getInstanceConfigMap().get(ins1).getDomainAsMap().get(faultZoneType) : null;
+        mz2 = _cache.getInstanceConfigMap().get(ins2) != null ?
+            _cache.getInstanceConfigMap().get(ins2).getDomainAsMap().get(faultZoneType) : null;
+      }
+
+      // If mz is not valid for either instance, skip mz comparison
+      // If instances are in the same MZ, skip mz comparison
+      if (mz1 != null && mz2 != null && !mz1.equals(mz2)) {
+        // Count number of replicas in each MZ
+        int mz1Count = 0;
+        int mz2Count = 0;
+        for (String instance : _currentStateMap.keySet()) {
+            if (ins1.equals(instance) || ins2.equals(instance)) {
+              continue;
+            }
+
+            String currMz = _cache.getInstanceConfigMap().get(instance).getDomainAsMap().get(faultZoneType);
+
+            if (mz1.equals(currMz)) {
+              mz1Count++;
+            } else if (mz2.equals(currMz)) {
+              mz2Count++;
+            }
+        }
+        if (mz1Count != mz2Count) {
+          return mz1Count - mz2Count;
+        }

Review Comment:
   Good points on both issues, thank you for the review. I'll address this 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org


Re: [PR] Consider fault zones when sorting combinedPreferenceList [helix]

Posted by "GrantPSpencer (via GitHub)" <gi...@apache.org>.
GrantPSpencer commented on PR #2894:
URL: https://github.com/apache/helix/pull/2894#issuecomment-2327076035

   Pull request approved by: @junkaixue 
   Commit message: Consider fault zones when sorting combinedPreferenceList


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org