You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@helix.apache.org by "MarkGaox (via GitHub)" <gi...@apache.org> on 2024/11/12 01:30:53 UTC

[PR] Support Stoppable Check for Non-Topology-Aware Clusters [helix]

MarkGaox opened a new pull request, #2961:
URL: https://github.com/apache/helix/pull/2961

   ### Issues
   
   - [x] My PR addresses the following Helix issues and references them in the PR description:
   #2893
   
   ### Description
   
   - [X] Here are some details about my PR, including screenshots of any UI changes:
   * Add a new selection base: NON_TOPO_BASED. In this scenario, the order of the zone will not be respected and all instances will be performed stoppable check at once. Additionally, this change supports the stoppable check for non-topo aware clusters.
   
   ### Tests
   
   - [X] The following tests are written for this issue:
   ` mvn test -Dtest=TestInstancesAccessor,TestMaintenanceManagementService,TestInstanceValidationUtilInRest,TestPerInstanceAccessor -pl helix-rest && mvn test -Dtest=TestInstanceValidationUtil -pl helix-core`
   
   
   ### Changes that Break Backward Compatibility (Optional)
   
   - My PR contains changes that break backward compatibility or previous assumptions for certain methods or API. They include:
   
   (Consider including all behavior changes for public methods or API. Also include these changes in merge description so that other developers are aware of these changes. This allows them to make relevant code changes in feature branches accounting for the new method/API behavior.)
   
   ### Documentation (Optional)
   
   - In case of new functionality, my PR adds documentation in the following wiki page:
   
   (Link the GitHub wiki you added)
   
   ### Commits
   
   - My commits all reference appropriate Apache Helix GitHub issues in their subject lines. In addition, my commits follow the guidelines from "[How to write a good git commit message](http://chris.beams.io/posts/git-commit/)":
     1. Subject is separated from body by a blank line
     1. Subject is limited to 50 characters (not including Jira issue reference)
     1. Subject does not end with a period
     1. Subject uses the imperative mood ("add", not "adding")
     1. Body wraps at 72 characters
     1. Body explains "what" and "why", not "how"
   
   ### Code Quality
   
   - My diff has been formatted using helix-style.xml 
   (helix-style-intellij.xml if IntelliJ IDE is used)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org


Re: [PR] Support Stoppable Check for Non-Topology-Aware Clusters [helix]

Posted by "MarkGaox (via GitHub)" <gi...@apache.org>.
MarkGaox commented on code in PR #2961:
URL: https://github.com/apache/helix/pull/2961#discussion_r1850908447


##########
helix-rest/src/main/java/org/apache/helix/rest/clusterMaintenanceService/StoppableInstancesSelector.java:
##########
@@ -136,6 +136,40 @@ public ObjectNode getStoppableInstancesCrossZones(List<String> instances,
     return result;
   }
 
+  /**
+   * Evaluates and collects stoppable instances without respecting the zone order.
+   * The method iterates through instances, performing stoppable checks, and records reasons for
+   * non-stoppability.
+   *
+   * @param instances A list of instance to be evaluated.
+   * @param toBeStoppedInstances A list of instances presumed to be already stopped
+   * @return An ObjectNode containing:
+   *         - 'stoppableNode': List of instances that can be stopped.
+   *         - 'instance_not_stoppable_with_reasons': A map with the instance name as the key and
+   *         a list of reasons for non-stoppability as the value.
+   * @throws IOException
+   */
+  public ObjectNode getStoppableInstancesWithoutTopology(List<String> instances,
+      List<String> toBeStoppedInstances) throws IOException {
+    ObjectNode result = JsonNodeFactory.instance.objectNode();
+    ArrayNode stoppableInstances =
+        result.putArray(InstancesAccessor.InstancesProperties.instance_stoppable_parallel.name());
+    ObjectNode failedStoppableInstances = result.putObject(
+        InstancesAccessor.InstancesProperties.instance_not_stoppable_with_reasons.name());
+    Set<String> toBeStoppedInstancesSet = new HashSet<>(toBeStoppedInstances);
+    collectEvacuatingInstances(toBeStoppedInstancesSet);

Review Comment:
   Yes, that makes sense to me. I’ll create an issue and address it in a separate PR.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org


Re: [PR] Support Stoppable Check for Non-Topology-Aware Clusters [helix]

Posted by "MarkGaox (via GitHub)" <gi...@apache.org>.
MarkGaox commented on code in PR #2961:
URL: https://github.com/apache/helix/pull/2961#discussion_r1849626917


##########
helix-rest/src/main/java/org/apache/helix/rest/server/resources/helix/InstancesAccessor.java:
##########
@@ -285,6 +294,14 @@ private Response batchGetStoppableInstances(String clusterId, JsonNode node, boo
         }
       }
 
+      if (selectionBase != InstanceHealthSelectionBase.non_topo_based && clusterTopology.getZones()

Review Comment:
   This change may potentially break backward compatibility for users who intend to check only a single zone but didn't provide an orderOfZone.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org


Re: [PR] Support Stoppable Check for Non-Topology-Aware Clusters [helix]

Posted by "zpinto (via GitHub)" <gi...@apache.org>.
zpinto commented on code in PR #2961:
URL: https://github.com/apache/helix/pull/2961#discussion_r1842753309


##########
helix-rest/src/main/java/org/apache/helix/rest/clusterMaintenanceService/StoppableInstancesSelector.java:
##########
@@ -136,6 +136,40 @@ public ObjectNode getStoppableInstancesCrossZones(List<String> instances,
     return result;
   }
 
+  /**
+   * Evaluates and collects stoppable instances without respecting the zone order.
+   * The method iterates through instances, performing stoppable checks, and records reasons for
+   * non-stoppability.
+   *
+   * @param instances A list of instance to be evaluated.
+   * @param toBeStoppedInstances A list of instances presumed to be already stopped
+   * @return An ObjectNode containing:
+   *         - 'stoppableNode': List of instances that can be stopped.
+   *         - 'instance_not_stoppable_with_reasons': A map with the instance name as the key and
+   *         a list of reasons for non-stoppability as the value.
+   * @throws IOException
+   */
+  public ObjectNode getStoppableInstancesWithoutTopology(List<String> instances,
+      List<String> toBeStoppedInstances) throws IOException {
+    ObjectNode result = JsonNodeFactory.instance.objectNode();
+    ArrayNode stoppableInstances =
+        result.putArray(InstancesAccessor.InstancesProperties.instance_stoppable_parallel.name());
+    ObjectNode failedStoppableInstances = result.putObject(
+        InstancesAccessor.InstancesProperties.instance_not_stoppable_with_reasons.name());
+    Set<String> toBeStoppedInstancesSet = new HashSet<>(toBeStoppedInstances);
+    collectEvacuatingInstances(toBeStoppedInstancesSet);

Review Comment:
   Should included evacuating, unknown, or swap in because we don't want the case where a replica on an evacuating, unknown, or swap-in instance allows another instance that is enabled to be stoppable. By the time the instance that was reported as stoppable gets taken down, the replica could have been dropped from the evacuating or unknown instance. Also, the swap-in replica is not actually serving if the use case is using routing table provider and it cannot ever become top-state until the swap is finished.
   
   Let's discuss more offline. I think there may be a better way to account for these edge cases/race conditions without explicitly marking instances as toBeStopped based on their InstanceOperation



##########
helix-rest/src/test/java/org/apache/helix/rest/server/AbstractTestClass.java:
##########
@@ -711,6 +713,62 @@ private void preSetupForCrosszoneParallelInstancesStoppableTest(String clusterNa
     _clusters.add(clusterName);
     _workflowMap.put(clusterName, createWorkflows(clusterName, 3));
   }
+
+  private void preSetupForNonTopoAwareInstancesStoppableTest(String clusterName,
+      List<String> instances) throws Exception {
+    _gSetupTool.addCluster(clusterName, true);
+    ClusterConfig clusterConfig = _configAccessor.getClusterConfig(clusterName);
+    clusterConfig.setFaultZoneType("helixZoneId");

Review Comment:
   Based on my reading of the ClusterTopology class, I don't think that the changes in this PR will work in the case that clusterConfig does not have fault zone type set. We need to make sure that stoppable check works for clusters that DON'T have TOPOLOGY, TOPOLOGY_AWARE_ENABLED, and FAULT_ZONE_TYPE configs set.
   
   Please add tests for this case and make the necessary changes to support this.



##########
helix-rest/src/main/java/org/apache/helix/rest/server/resources/helix/InstancesAccessor.java:
##########
@@ -285,6 +294,14 @@ private Response batchGetStoppableInstances(String clusterId, JsonNode node, boo
         }
       }
 
+      if (selectionBase != InstanceHealthSelectionBase.non_topo_based && clusterTopology.getZones()

Review Comment:
   Why do we need the user to explicitly provide a sectionBase? Can't we simplify this API by figuring out the InstanceHealthSelectionBase internally?
   
   if orderOfZone is provided we use zone_based
   if necessary topology configs are set, we use cross_zone_based
   else we use non_topo_based
   
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org


Re: [PR] Support Stoppable Check for Non-Topology-Aware Clusters [helix]

Posted by "MarkGaox (via GitHub)" <gi...@apache.org>.
MarkGaox commented on code in PR #2961:
URL: https://github.com/apache/helix/pull/2961#discussion_r1850908447


##########
helix-rest/src/main/java/org/apache/helix/rest/clusterMaintenanceService/StoppableInstancesSelector.java:
##########
@@ -136,6 +136,40 @@ public ObjectNode getStoppableInstancesCrossZones(List<String> instances,
     return result;
   }
 
+  /**
+   * Evaluates and collects stoppable instances without respecting the zone order.
+   * The method iterates through instances, performing stoppable checks, and records reasons for
+   * non-stoppability.
+   *
+   * @param instances A list of instance to be evaluated.
+   * @param toBeStoppedInstances A list of instances presumed to be already stopped
+   * @return An ObjectNode containing:
+   *         - 'stoppableNode': List of instances that can be stopped.
+   *         - 'instance_not_stoppable_with_reasons': A map with the instance name as the key and
+   *         a list of reasons for non-stoppability as the value.
+   * @throws IOException
+   */
+  public ObjectNode getStoppableInstancesWithoutTopology(List<String> instances,
+      List<String> toBeStoppedInstances) throws IOException {
+    ObjectNode result = JsonNodeFactory.instance.objectNode();
+    ArrayNode stoppableInstances =
+        result.putArray(InstancesAccessor.InstancesProperties.instance_stoppable_parallel.name());
+    ObjectNode failedStoppableInstances = result.putObject(
+        InstancesAccessor.InstancesProperties.instance_not_stoppable_with_reasons.name());
+    Set<String> toBeStoppedInstancesSet = new HashSet<>(toBeStoppedInstances);
+    collectEvacuatingInstances(toBeStoppedInstancesSet);

Review Comment:
   Yes, that makes sense to me. Will address it in this PR.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org


Re: [PR] Support Stoppable Check for Non-Topology-Aware Clusters [helix]

Posted by "xyuanlu (via GitHub)" <gi...@apache.org>.
xyuanlu commented on code in PR #2961:
URL: https://github.com/apache/helix/pull/2961#discussion_r1879225073


##########
helix-rest/src/main/java/org/apache/helix/rest/clusterMaintenanceService/StoppableInstancesSelector.java:
##########
@@ -136,6 +136,40 @@ public ObjectNode getStoppableInstancesCrossZones(List<String> instances,
     return result;
   }
 
+  /**
+   * Evaluates and collects stoppable instances not based on the zone order.
+   * The method iterates through instances, performing stoppable checks, and records reasons for
+   * non-stoppability.
+   *
+   * @param instances A list of instance to be evaluated.
+   * @param toBeStoppedInstances A list of instances presumed to be already stopped
+   * @return An ObjectNode containing:
+   *         - 'stoppableNode': List of instances that can be stopped.
+   *         - 'instance_not_stoppable_with_reasons': A map with the instance name as the key and
+   *         a list of reasons for non-stoppability as the value.
+   * @throws IOException
+   */
+  public ObjectNode getStoppableInstancesNonZoneBased(List<String> instances,
+      List<String> toBeStoppedInstances) throws IOException {
+    ObjectNode result = JsonNodeFactory.instance.objectNode();
+    ArrayNode stoppableInstances =
+        result.putArray(InstancesAccessor.InstancesProperties.instance_stoppable_parallel.name());
+    ObjectNode failedStoppableInstances = result.putObject(
+        InstancesAccessor.InstancesProperties.instance_not_stoppable_with_reasons.name());
+    Set<String> toBeStoppedInstancesSet = new HashSet<>(toBeStoppedInstances);

Review Comment:
   Set<String> toBeStoppedInstancesSet = findToBeStoppedInstances(toBeStoppedInstances);



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org


Re: [PR] Support Stoppable Check for Non-Topology-Aware Clusters [helix]

Posted by "xyuanlu (via GitHub)" <gi...@apache.org>.
xyuanlu commented on code in PR #2961:
URL: https://github.com/apache/helix/pull/2961#discussion_r1879225073


##########
helix-rest/src/main/java/org/apache/helix/rest/clusterMaintenanceService/StoppableInstancesSelector.java:
##########
@@ -136,6 +136,40 @@ public ObjectNode getStoppableInstancesCrossZones(List<String> instances,
     return result;
   }
 
+  /**
+   * Evaluates and collects stoppable instances not based on the zone order.
+   * The method iterates through instances, performing stoppable checks, and records reasons for
+   * non-stoppability.
+   *
+   * @param instances A list of instance to be evaluated.
+   * @param toBeStoppedInstances A list of instances presumed to be already stopped
+   * @return An ObjectNode containing:
+   *         - 'stoppableNode': List of instances that can be stopped.
+   *         - 'instance_not_stoppable_with_reasons': A map with the instance name as the key and
+   *         a list of reasons for non-stoppability as the value.
+   * @throws IOException
+   */
+  public ObjectNode getStoppableInstancesNonZoneBased(List<String> instances,
+      List<String> toBeStoppedInstances) throws IOException {
+    ObjectNode result = JsonNodeFactory.instance.objectNode();
+    ArrayNode stoppableInstances =
+        result.putArray(InstancesAccessor.InstancesProperties.instance_stoppable_parallel.name());
+    ObjectNode failedStoppableInstances = result.putObject(
+        InstancesAccessor.InstancesProperties.instance_not_stoppable_with_reasons.name());
+    Set<String> toBeStoppedInstancesSet = new HashSet<>(toBeStoppedInstances);

Review Comment:
   Set <String> toBeStoppedInstancesSet = findToBeStoppedInstances(toBeStoppedInstances);



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org


Re: [PR] Support Stoppable Check for Non-Topology-Aware Clusters [helix]

Posted by "xyuanlu (via GitHub)" <gi...@apache.org>.
xyuanlu commented on code in PR #2961:
URL: https://github.com/apache/helix/pull/2961#discussion_r1879209342


##########
helix-rest/src/main/java/org/apache/helix/rest/clusterMaintenanceService/StoppableInstancesSelector.java:
##########
@@ -136,6 +136,40 @@ public ObjectNode getStoppableInstancesCrossZones(List<String> instances,
     return result;
   }
 
+  /**
+   * Evaluates and collects stoppable instances not based on the zone order.
+   * The method iterates through instances, performing stoppable checks, and records reasons for
+   * non-stoppability.
+   *
+   * @param instances A list of instance to be evaluated.
+   * @param toBeStoppedInstances A list of instances presumed to be already stopped
+   * @return An ObjectNode containing:
+   *         - 'stoppableNode': List of instances that can be stopped.
+   *         - 'instance_not_stoppable_with_reasons': A map with the instance name as the key and
+   *         a list of reasons for non-stoppability as the value.
+   * @throws IOException
+   */
+  public ObjectNode getStoppableInstancesNonZoneBased(List<String> instances,
+      List<String> toBeStoppedInstances) throws IOException {
+    ObjectNode result = JsonNodeFactory.instance.objectNode();
+    ArrayNode stoppableInstances =
+        result.putArray(InstancesAccessor.InstancesProperties.instance_stoppable_parallel.name());
+    ObjectNode failedStoppableInstances = result.putObject(
+        InstancesAccessor.InstancesProperties.instance_not_stoppable_with_reasons.name());
+    Set<String> toBeStoppedInstancesSet = new HashSet<>(toBeStoppedInstances);

Review Comment:
   159 - 160 can be inline



##########
helix-rest/src/main/java/org/apache/helix/rest/clusterMaintenanceService/StoppableInstancesSelector.java:
##########
@@ -136,6 +136,40 @@ public ObjectNode getStoppableInstancesCrossZones(List<String> instances,
     return result;
   }
 
+  /**
+   * Evaluates and collects stoppable instances not based on the zone order.
+   * The method iterates through instances, performing stoppable checks, and records reasons for
+   * non-stoppability.
+   *
+   * @param instances A list of instance to be evaluated.
+   * @param toBeStoppedInstances A list of instances presumed to be already stopped
+   * @return An ObjectNode containing:
+   *         - 'stoppableNode': List of instances that can be stopped.
+   *         - 'instance_not_stoppable_with_reasons': A map with the instance name as the key and
+   *         a list of reasons for non-stoppability as the value.
+   * @throws IOException
+   */
+  public ObjectNode getStoppableInstancesNonZoneBased(List<String> instances,
+      List<String> toBeStoppedInstances) throws IOException {
+    ObjectNode result = JsonNodeFactory.instance.objectNode();
+    ArrayNode stoppableInstances =
+        result.putArray(InstancesAccessor.InstancesProperties.instance_stoppable_parallel.name());
+    ObjectNode failedStoppableInstances = result.putObject(
+        InstancesAccessor.InstancesProperties.instance_not_stoppable_with_reasons.name());
+    Set<String> toBeStoppedInstancesSet = new HashSet<>(toBeStoppedInstances);

Review Comment:
   Also if every caller of `findToBeStoppedInstances ` need to create a new set, we could have `list` as input var for findToBeStoppedInstances?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org


Re: [PR] Support Stoppable Check for Non-Topology-Aware Clusters [helix]

Posted by "xyuanlu (via GitHub)" <gi...@apache.org>.
xyuanlu merged PR #2961:
URL: https://github.com/apache/helix/pull/2961


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org


Re: [PR] Support Stoppable Check for Non-Topology-Aware Clusters [helix]

Posted by "xyuanlu (via GitHub)" <gi...@apache.org>.
xyuanlu commented on code in PR #2961:
URL: https://github.com/apache/helix/pull/2961#discussion_r1857322076


##########
helix-rest/src/main/java/org/apache/helix/rest/clusterMaintenanceService/StoppableInstancesSelector.java:
##########
@@ -136,6 +136,40 @@ public ObjectNode getStoppableInstancesCrossZones(List<String> instances,
     return result;
   }
 
+  /**
+   * Evaluates and collects stoppable instances without respecting the zone order.
+   * The method iterates through instances, performing stoppable checks, and records reasons for
+   * non-stoppability.
+   *
+   * @param instances A list of instance to be evaluated.
+   * @param toBeStoppedInstances A list of instances presumed to be already stopped
+   * @return An ObjectNode containing:
+   *         - 'stoppableNode': List of instances that can be stopped.
+   *         - 'instance_not_stoppable_with_reasons': A map with the instance name as the key and
+   *         a list of reasons for non-stoppability as the value.
+   * @throws IOException
+   */
+  public ObjectNode getStoppableInstancesWithoutTopology(List<String> instances,

Review Comment:
   getStoppableInstancesNonTopologyBased? Or sth similar? 
   Otherwise it is a bit ambiguous if we are not considering topology, or the cluster dont have topology configured.  



##########
helix-rest/src/main/java/org/apache/helix/rest/server/resources/helix/InstancesAccessor.java:
##########
@@ -82,7 +82,7 @@ public enum InstancesProperties {
   }
 
   public enum InstanceHealthSelectionBase {
-    instance_based,
+    non_topo_based,
     zone_based,

Review Comment:
   non_zone_based? 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org


Re: [PR] Support Stoppable Check for Non-Topology-Aware Clusters [helix]

Posted by "xyuanlu (via GitHub)" <gi...@apache.org>.
xyuanlu commented on code in PR #2961:
URL: https://github.com/apache/helix/pull/2961#discussion_r1879225073


##########
helix-rest/src/main/java/org/apache/helix/rest/clusterMaintenanceService/StoppableInstancesSelector.java:
##########
@@ -136,6 +136,40 @@ public ObjectNode getStoppableInstancesCrossZones(List<String> instances,
     return result;
   }
 
+  /**
+   * Evaluates and collects stoppable instances not based on the zone order.
+   * The method iterates through instances, performing stoppable checks, and records reasons for
+   * non-stoppability.
+   *
+   * @param instances A list of instance to be evaluated.
+   * @param toBeStoppedInstances A list of instances presumed to be already stopped
+   * @return An ObjectNode containing:
+   *         - 'stoppableNode': List of instances that can be stopped.
+   *         - 'instance_not_stoppable_with_reasons': A map with the instance name as the key and
+   *         a list of reasons for non-stoppability as the value.
+   * @throws IOException
+   */
+  public ObjectNode getStoppableInstancesNonZoneBased(List<String> instances,
+      List<String> toBeStoppedInstances) throws IOException {
+    ObjectNode result = JsonNodeFactory.instance.objectNode();
+    ArrayNode stoppableInstances =
+        result.putArray(InstancesAccessor.InstancesProperties.instance_stoppable_parallel.name());
+    ObjectNode failedStoppableInstances = result.putObject(
+        InstancesAccessor.InstancesProperties.instance_not_stoppable_with_reasons.name());
+    Set<String> toBeStoppedInstancesSet = new HashSet<>(toBeStoppedInstances);

Review Comment:
   Set `<String`> toBeStoppedInstancesSet = findToBeStoppedInstances(toBeStoppedInstances);



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org


Re: [PR] Support Stoppable Check for Non-Topology-Aware Clusters [helix]

Posted by "MarkGaox (via GitHub)" <gi...@apache.org>.
MarkGaox commented on code in PR #2961:
URL: https://github.com/apache/helix/pull/2961#discussion_r1850908447


##########
helix-rest/src/main/java/org/apache/helix/rest/clusterMaintenanceService/StoppableInstancesSelector.java:
##########
@@ -136,6 +136,40 @@ public ObjectNode getStoppableInstancesCrossZones(List<String> instances,
     return result;
   }
 
+  /**
+   * Evaluates and collects stoppable instances without respecting the zone order.
+   * The method iterates through instances, performing stoppable checks, and records reasons for
+   * non-stoppability.
+   *
+   * @param instances A list of instance to be evaluated.
+   * @param toBeStoppedInstances A list of instances presumed to be already stopped
+   * @return An ObjectNode containing:
+   *         - 'stoppableNode': List of instances that can be stopped.
+   *         - 'instance_not_stoppable_with_reasons': A map with the instance name as the key and
+   *         a list of reasons for non-stoppability as the value.
+   * @throws IOException
+   */
+  public ObjectNode getStoppableInstancesWithoutTopology(List<String> instances,
+      List<String> toBeStoppedInstances) throws IOException {
+    ObjectNode result = JsonNodeFactory.instance.objectNode();
+    ArrayNode stoppableInstances =
+        result.putArray(InstancesAccessor.InstancesProperties.instance_stoppable_parallel.name());
+    ObjectNode failedStoppableInstances = result.putObject(
+        InstancesAccessor.InstancesProperties.instance_not_stoppable_with_reasons.name());
+    Set<String> toBeStoppedInstancesSet = new HashSet<>(toBeStoppedInstances);
+    collectEvacuatingInstances(toBeStoppedInstancesSet);

Review Comment:
   Yes, that makes sense to me. I’ll create an issue and address it in this PR.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org


Re: [PR] Support Stoppable Check for Non-Topology-Aware Clusters [helix]

Posted by "MarkGaox (via GitHub)" <gi...@apache.org>.
MarkGaox commented on PR #2961:
URL: https://github.com/apache/helix/pull/2961#issuecomment-2533601262

   PR approved by @xyuanlu. TFTR. Final commit message: "Support Stoppable Check for Non-Topology-Aware Clusters"


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org


Re: [PR] Support Stoppable Check for Non-Topology-Aware Clusters [helix]

Posted by "MarkGaox (via GitHub)" <gi...@apache.org>.
MarkGaox commented on code in PR #2961:
URL: https://github.com/apache/helix/pull/2961#discussion_r1851432758


##########
helix-rest/src/main/java/org/apache/helix/rest/server/resources/helix/InstancesAccessor.java:
##########
@@ -285,6 +294,14 @@ private Response batchGetStoppableInstances(String clusterId, JsonNode node, boo
         }
       }
 
+      if (selectionBase != InstanceHealthSelectionBase.non_topo_based && clusterTopology.getZones()

Review Comment:
   Discussed offline. Will make NON_TOPO be the default value if the selection base is not provided. In a separate PR will support automatically detect whether to use NON_TOPO_BASE or CROSS_ZONE_BASE if the user didn't specify the selection base.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org


Re: [PR] Support Stoppable Check for Non-Topology-Aware Clusters [helix]

Posted by "MarkGaox (via GitHub)" <gi...@apache.org>.
MarkGaox commented on code in PR #2961:
URL: https://github.com/apache/helix/pull/2961#discussion_r1879220611


##########
helix-rest/src/main/java/org/apache/helix/rest/clusterMaintenanceService/StoppableInstancesSelector.java:
##########
@@ -136,6 +136,40 @@ public ObjectNode getStoppableInstancesCrossZones(List<String> instances,
     return result;
   }
 
+  /**
+   * Evaluates and collects stoppable instances not based on the zone order.
+   * The method iterates through instances, performing stoppable checks, and records reasons for
+   * non-stoppability.
+   *
+   * @param instances A list of instance to be evaluated.
+   * @param toBeStoppedInstances A list of instances presumed to be already stopped
+   * @return An ObjectNode containing:
+   *         - 'stoppableNode': List of instances that can be stopped.
+   *         - 'instance_not_stoppable_with_reasons': A map with the instance name as the key and
+   *         a list of reasons for non-stoppability as the value.
+   * @throws IOException
+   */
+  public ObjectNode getStoppableInstancesNonZoneBased(List<String> instances,
+      List<String> toBeStoppedInstances) throws IOException {
+    ObjectNode result = JsonNodeFactory.instance.objectNode();
+    ArrayNode stoppableInstances =
+        result.putArray(InstancesAccessor.InstancesProperties.instance_stoppable_parallel.name());
+    ObjectNode failedStoppableInstances = result.putObject(
+        InstancesAccessor.InstancesProperties.instance_not_stoppable_with_reasons.name());
+    Set<String> toBeStoppedInstancesSet = new HashSet<>(toBeStoppedInstances);

Review Comment:
   By using a set, we avoid changing the method signature when calling _maintenanceService.batchGetInstancesStoppableChecks and passing the toBeStoppedInstances.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org


Re: [PR] Support Stoppable Check for Non-Topology-Aware Clusters [helix]

Posted by "xyuanlu (via GitHub)" <gi...@apache.org>.
xyuanlu commented on code in PR #2961:
URL: https://github.com/apache/helix/pull/2961#discussion_r1868471190


##########
helix-rest/src/main/java/org/apache/helix/rest/clusterMaintenanceService/StoppableInstancesSelector.java:
##########
@@ -258,18 +293,21 @@ private Map<String, Set<String>> getOrderedZoneToInstancesMap(
   }
 
   /**
-   * Collect instances marked for evacuation in the current topology and add them into the given set
+   * Collect instances marked for evacuation, swap, or unkown in the current topology and add

Review Comment:
   nit: I would suggest refine on comment here. I think we are referring to all the instances in the cluster, not necessarily the topology aware cluster's topology tree. 



##########
helix-rest/src/main/java/org/apache/helix/rest/server/resources/helix/InstancesAccessor.java:
##########
@@ -285,6 +296,28 @@ private Response batchGetStoppableInstances(String clusterId, JsonNode node, boo
         }
       }
 
+      ClusterTopology clusterTopology = clusterService.getClusterTopology(clusterId);
+      if (selectionBase != InstanceHealthSelectionBase.non_zone_based) {
+        if (!clusterService.isClusterTopologyAware(clusterId)) {
+          String message = "Cluster " + clusterId
+              + " is not topology aware. Please enable the topology in cluster config or set "
+              + "'selection_base' to 'non_zone_based'.";
+          _logger.error(message);
+          return badRequest(message);
+        }
+
+        // Find instances that lack topology information
+        Set<String> topologyUnawareInstances =

Review Comment:
   nit: we can merge the helper function here. 



##########
helix-rest/src/main/java/org/apache/helix/rest/clusterMaintenanceService/StoppableInstancesSelector.java:
##########
@@ -136,6 +136,40 @@ public ObjectNode getStoppableInstancesCrossZones(List<String> instances,
     return result;
   }
 
+  /**
+   * Evaluates and collects stoppable instances not based on the zone order.
+   * The method iterates through instances, performing stoppable checks, and records reasons for
+   * non-stoppability.
+   *
+   * @param instances A list of instance to be evaluated.
+   * @param toBeStoppedInstances A list of instances presumed to be already stopped
+   * @return An ObjectNode containing:
+   *         - 'stoppableNode': List of instances that can be stopped.
+   *         - 'instance_not_stoppable_with_reasons': A map with the instance name as the key and
+   *         a list of reasons for non-stoppability as the value.
+   * @throws IOException
+   */
+  public ObjectNode getStoppableInstancesNonZoneBased(List<String> instances,
+      List<String> toBeStoppedInstances) throws IOException {
+    ObjectNode result = JsonNodeFactory.instance.objectNode();
+    ArrayNode stoppableInstances =
+        result.putArray(InstancesAccessor.InstancesProperties.instance_stoppable_parallel.name());
+    ObjectNode failedStoppableInstances = result.putObject(
+        InstancesAccessor.InstancesProperties.instance_not_stoppable_with_reasons.name());
+    Set<String> toBeStoppedInstancesSet = new HashSet<>(toBeStoppedInstances);
+    findToBeStoppedInstances(toBeStoppedInstancesSet);
+
+    // Because zone order calculation is omitted, we must verify each instance's existence
+    // to ensure we only process valid instances before performing stoppable check.
+    Set<String> nonExistingInstances = processNonexistentInstances(instances, failedStoppableInstances);

Review Comment:
   do we add back `nonExistingInstances ` to result?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org


Re: [PR] Support Stoppable Check for Non-Topology-Aware Clusters [helix]

Posted by "MarkGaox (via GitHub)" <gi...@apache.org>.
MarkGaox commented on code in PR #2961:
URL: https://github.com/apache/helix/pull/2961#discussion_r1870498922


##########
helix-rest/src/main/java/org/apache/helix/rest/clusterMaintenanceService/StoppableInstancesSelector.java:
##########
@@ -136,6 +136,40 @@ public ObjectNode getStoppableInstancesCrossZones(List<String> instances,
     return result;
   }
 
+  /**
+   * Evaluates and collects stoppable instances not based on the zone order.
+   * The method iterates through instances, performing stoppable checks, and records reasons for
+   * non-stoppability.
+   *
+   * @param instances A list of instance to be evaluated.
+   * @param toBeStoppedInstances A list of instances presumed to be already stopped
+   * @return An ObjectNode containing:
+   *         - 'stoppableNode': List of instances that can be stopped.
+   *         - 'instance_not_stoppable_with_reasons': A map with the instance name as the key and
+   *         a list of reasons for non-stoppability as the value.
+   * @throws IOException
+   */
+  public ObjectNode getStoppableInstancesNonZoneBased(List<String> instances,
+      List<String> toBeStoppedInstances) throws IOException {
+    ObjectNode result = JsonNodeFactory.instance.objectNode();
+    ArrayNode stoppableInstances =
+        result.putArray(InstancesAccessor.InstancesProperties.instance_stoppable_parallel.name());
+    ObjectNode failedStoppableInstances = result.putObject(
+        InstancesAccessor.InstancesProperties.instance_not_stoppable_with_reasons.name());
+    Set<String> toBeStoppedInstancesSet = new HashSet<>(toBeStoppedInstances);
+    findToBeStoppedInstances(toBeStoppedInstancesSet);
+
+    // Because zone order calculation is omitted, we must verify each instance's existence
+    // to ensure we only process valid instances before performing stoppable check.
+    Set<String> nonExistingInstances = processNonexistentInstances(instances, failedStoppableInstances);

Review Comment:
   Yes, processNonexistentInstances will add non-existing instances to the response result.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org