You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@ignite.apache.org by "Pochatkin (via GitHub)" <gi...@apache.org> on 2024/10/22 11:33:24 UTC

[PR] IGNITE-23054 Improve cluster status REST endpoint [ignite-3]

Pochatkin opened a new pull request, #4614:
URL: https://github.com/apache/ignite-3/pull/4614

   https://issues.apache.org/jira/browse/IGNITE-23054


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-23054 Improve cluster status REST endpoint [ignite-3]

Posted by "sk0x50 (via GitHub)" <gi...@apache.org>.
sk0x50 commented on PR #4614:
URL: https://github.com/apache/ignite-3/pull/4614#issuecomment-3601432207

   closed due to inactivity


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-23054 Improve cluster status REST endpoint [ignite-3]

Posted by "sk0x50 (via GitHub)" <gi...@apache.org>.
sk0x50 closed pull request #4614: IGNITE-23054 Improve cluster status REST endpoint
URL: https://github.com/apache/ignite-3/pull/4614


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-23054 Improve cluster status REST endpoint [ignite-3]

Posted by "valepakh (via GitHub)" <gi...@apache.org>.
valepakh commented on code in PR #4614:
URL: https://github.com/apache/ignite-3/pull/4614#discussion_r1825660349


##########
modules/cli/src/main/java/org/apache/ignite/internal/cli/call/cluster/status/ClusterStatusCall.java:
##########
@@ -30,13 +30,20 @@
 import org.apache.ignite.rest.client.api.ClusterManagementApi;
 import org.apache.ignite.rest.client.invoker.ApiException;
 import org.apache.ignite.rest.client.model.ClusterNode;
-import org.apache.ignite.rest.client.model.ClusterState;
 
 /**
  * Call to get cluster status.
  */
 @Singleton
-public class ClusterStatusCall implements Call<UrlCallInput, ClusterStatus> {
+public class ClusterStatusCall implements Call<UrlCallInput, ClusterState> {
+    /**
+     * We need to overlap timeout from raft client.
+     * We can't determine timeout value because it's configurable for each node

Review Comment:
   Maybe we can read the node configuration here and just add a couple of seconds to the configured timeout?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-23054 Improve cluster status REST endpoint [ignite-3]

Posted by "Pochatkin (via GitHub)" <gi...@apache.org>.
Pochatkin commented on code in PR #4614:
URL: https://github.com/apache/ignite-3/pull/4614#discussion_r1832438272


##########
modules/rest-api/src/main/java/org/apache/ignite/internal/rest/api/cluster/ClusterState.java:
##########
@@ -64,24 +73,28 @@ public class ClusterState {
     @JsonCreator
     public ClusterState(
             @JsonProperty("cmgNodes") Collection<String> cmgNodes,
-            @JsonProperty("msNodes") Collection<String> msNodes,
+            @JsonProperty("msNodes")  Collection<String> msNodes,

Review Comment:
   Done



##########
modules/rest/src/main/java/org/apache/ignite/internal/rest/cluster/ClusterManagementController.java:
##########
@@ -84,16 +113,31 @@ public CompletableFuture<Void> init(@Body InitCommand initCommand) {
         });
     }
 
-    private static ClusterState mapClusterState(org.apache.ignite.internal.cluster.management.ClusterState clusterState) {
+    private ClusterState mapClusterState(org.apache.ignite.internal.cluster.management.ClusterState clusterState) {
         return new ClusterState(
                 clusterState.cmgNodes(),
                 clusterState.metaStorageNodes(),
                 clusterState.igniteVersion().toString(),
                 new ClusterTag(clusterState.clusterTag().clusterName(), clusterState.clusterTag().clusterId()),
-                clusterState.formerClusterIds()
+                clusterState.formerClusterIds(),
+                mapClusterStatus(clusterState)
         );
     }
 
+    private ClusterStatus mapClusterStatus(org.apache.ignite.internal.cluster.management.ClusterState clusterState) {
+        Set<String> metaStorageNodes = clusterState.metaStorageNodes();
+        long presentedMetaStorageNodes = topologyService.allMembers().stream()

Review Comment:
   Done



##########
modules/rest/src/main/java/org/apache/ignite/internal/rest/cluster/ClusterManagementController.java:
##########
@@ -84,16 +113,31 @@ public CompletableFuture<Void> init(@Body InitCommand initCommand) {
         });
     }
 
-    private static ClusterState mapClusterState(org.apache.ignite.internal.cluster.management.ClusterState clusterState) {
+    private ClusterState mapClusterState(org.apache.ignite.internal.cluster.management.ClusterState clusterState) {
         return new ClusterState(
                 clusterState.cmgNodes(),
                 clusterState.metaStorageNodes(),
                 clusterState.igniteVersion().toString(),
                 new ClusterTag(clusterState.clusterTag().clusterName(), clusterState.clusterTag().clusterId()),
-                clusterState.formerClusterIds()
+                clusterState.formerClusterIds(),
+                mapClusterStatus(clusterState)
         );
     }
 
+    private ClusterStatus mapClusterStatus(org.apache.ignite.internal.cluster.management.ClusterState clusterState) {
+        Set<String> metaStorageNodes = clusterState.metaStorageNodes();
+        long presentedMetaStorageNodes = topologyService.allMembers().stream()
+                .map(ClusterNode::name)
+                .filter(metaStorageNodes::contains)
+                .count();
+
+        if (presentedMetaStorageNodes <= metaStorageNodes.size() / 2) {

Review Comment:
   Done



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-23054 Improve cluster status REST endpoint [ignite-3]

Posted by "PakhomovAlexander (via GitHub)" <gi...@apache.org>.
PakhomovAlexander commented on code in PR #4614:
URL: https://github.com/apache/ignite-3/pull/4614#discussion_r1860781182


##########
modules/rest-api/src/main/java/org/apache/ignite/internal/rest/api/cluster/ClusterStateDtoBuilder.java:
##########
@@ -0,0 +1,84 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.rest.api.cluster;
+
+import java.util.List;
+import java.util.UUID;
+import org.jetbrains.annotations.Nullable;
+
+/**
+ * Builder for {@link ClusterStateDto}.
+ */
+public final class ClusterStateDtoBuilder {
+    private GroupState cmgStatus;
+
+    private GroupState metastoreStatus;
+
+    private String igniteVersion;
+
+    private ClusterTag clusterTag;
+
+    private @Nullable List<UUID> formerClusterIds;
+
+    private ClusterStateDtoBuilder() {
+
+    }
+
+    public static ClusterStateDtoBuilder newInstance() {

Review Comment:
   `newInstance()` can be confused with `Class.newInstance()`. We tend to use `builder()` name. 



##########
modules/rest/src/main/java/org/apache/ignite/internal/rest/cluster/ClusterManagementController.java:
##########
@@ -54,16 +62,46 @@ public class ClusterManagementController implements ClusterManagementApi, Resour
      */
     public ClusterManagementController(
             ClusterInitializer clusterInitializer,
+            MetastoreGroupAvailabilityService metastoreAvailabilityService,
             ClusterManagementGroupManager clusterManagementGroupManager
     ) {
         this.clusterInitializer = clusterInitializer;
         this.clusterManagementGroupManager = clusterManagementGroupManager;
+        this.metastoreAvailabilityService = metastoreAvailabilityService;
     }
 
     /** {@inheritDoc} */
     @Override
-    public CompletableFuture<ClusterState> clusterState() {
-        return clusterManagementGroupManager.clusterState().thenApply(ClusterManagementController::mapClusterState);
+    public CompletableFuture<ClusterStateDto> clusterState() {
+        CompletableFuture<GroupState> metastoreFut = CompletableFutures.combine(
+                metastoreAvailabilityService.isAlive()
+                        .thenApply(isAlive1 ->  isAlive1 ? GroupStatus.HEALTHY : GroupStatus.MAJORITY_LOST),
+                metastoreAvailabilityService.availableMembers(),
+                (groupStatus, strings) -> new GroupState(strings, groupStatus)
+        );
+
+        CompletableFuture<ClusterStateDtoBuilder> builderFut = clusterManagementGroupManager.clusterState()
+                .handle((state, t) -> {
+                    ClusterStateDtoBuilder builder = ClusterStateDtoBuilder.newInstance();
+                    if (state != null) {
+                        builder.igniteVersion(state.version())
+                                .clusterTag(new ClusterTag(state.clusterTag().clusterName(), state.clusterTag().clusterId()))
+                                .formerClusterIds(state.formerClusterIds());
+                    } else {
+                        builder.igniteVersion("N/A").clusterTag(new ClusterTag("N/A", null));

Review Comment:
   `builder.unknownState()` might be useful here.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-23054 Improve cluster status REST endpoint [ignite-3]

Posted by "Pochatkin (via GitHub)" <gi...@apache.org>.
Pochatkin commented on code in PR #4614:
URL: https://github.com/apache/ignite-3/pull/4614#discussion_r1832437912


##########
modules/cli/src/integrationTest/java/org/apache/ignite/internal/cli/commands/cluster/status/ItClusterStatusCommandInitializedTest.java:
##########
@@ -17,37 +17,79 @@
 
 package org.apache.ignite.internal.cli.commands.cluster.status;
 
+import static java.util.function.Function.identity;
 import static java.util.stream.Collectors.joining;
 import static org.junit.jupiter.api.Assertions.assertAll;
 
 import java.util.Arrays;
-import org.apache.ignite.Ignite;
+import java.util.Map;
+import java.util.function.Function;
+import java.util.stream.Collectors;
+import java.util.stream.IntStream;
 import org.apache.ignite.internal.cli.CliIntegrationTest;
+import org.jetbrains.annotations.Nullable;
 import org.junit.jupiter.api.DisplayName;
 import org.junit.jupiter.api.Test;
 
 /**
  * Tests for {@link ClusterStatusCommand} for the cluster that is initialized.
  */
 class ItClusterStatusCommandInitializedTest extends CliIntegrationTest {
+    private Function<int[], String> mapper;
+
+    @Override
+    protected @Nullable int[] metastoreNodes() {
+        return new int[] { 0 };
+    }
+
+    @Override
+    protected @Nullable int[] cmgNodes() {
+        return new int[] { 1 };
+    }
+
     @Test
     @DisplayName("Should print status when valid cluster url is given but cluster is initialized")
-    void printStatus() {
-        String cmgNodes = Arrays.stream(cmgMetastoreNodes())
-                .mapToObj(CLUSTER::node)
-                .map(Ignite::name)
+    void printStatus() throws InterruptedException {
+        String node0Url = NODE_URL;
+        String node1Url = "http://localhost:" + CLUSTER.httpPort(1);
+
+        Map<Integer, String> nodeNames = IntStream.range(0, initialNodes())
+                .boxed()
+                .collect(Collectors.toMap(identity(), i ->  CLUSTER.node(i).name()));
+
+        mapper = nodes -> Arrays.stream(nodes)
+                .mapToObj(nodeNames::get)
                 .collect(joining(", ", "[", "]"));
 
-        execute("cluster", "status", "--url", NODE_URL);
+        CLUSTER.stopNode(0);
+        execute("cluster", "status", "--url", node1Url);
+        assertOutput("cluster", 2, "Metastore majority lost", cmgNodes(), metastoreNodes());
+
+        CLUSTER.startNode(0);
+        Thread.sleep(10000);

Review Comment:
   Removed



##########
modules/cli/src/integrationTest/java/org/apache/ignite/internal/cli/commands/sql/ItSqlCommandTest.java:
##########
@@ -33,6 +33,8 @@ class ItSqlCommandTest extends CliSqlCommandTestBase {
     void nonExistingFile() {
         execute("sql", "--file", "nonexisting", "--jdbc-url", JDBC_URL);
 
+        CLUSTER.stopNode(0);

Review Comment:
   Removed



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-23054 Improve cluster status REST endpoint [ignite-3]

Posted by "PakhomovAlexander (via GitHub)" <gi...@apache.org>.
PakhomovAlexander commented on code in PR #4614:
URL: https://github.com/apache/ignite-3/pull/4614#discussion_r1860784774


##########
modules/rest/src/main/java/org/apache/ignite/internal/rest/cluster/ClusterManagementController.java:
##########
@@ -54,16 +62,46 @@ public class ClusterManagementController implements ClusterManagementApi, Resour
      */
     public ClusterManagementController(
             ClusterInitializer clusterInitializer,
+            MetastoreGroupAvailabilityService metastoreAvailabilityService,
             ClusterManagementGroupManager clusterManagementGroupManager
     ) {
         this.clusterInitializer = clusterInitializer;
         this.clusterManagementGroupManager = clusterManagementGroupManager;
+        this.metastoreAvailabilityService = metastoreAvailabilityService;
     }
 
     /** {@inheritDoc} */
     @Override
-    public CompletableFuture<ClusterState> clusterState() {
-        return clusterManagementGroupManager.clusterState().thenApply(ClusterManagementController::mapClusterState);
+    public CompletableFuture<ClusterStateDto> clusterState() {
+        CompletableFuture<GroupState> metastoreFut = CompletableFutures.combine(
+                metastoreAvailabilityService.isAlive()
+                        .thenApply(isAlive1 ->  isAlive1 ? GroupStatus.HEALTHY : GroupStatus.MAJORITY_LOST),
+                metastoreAvailabilityService.availableMembers(),
+                (groupStatus, strings) -> new GroupState(strings, groupStatus)
+        );
+
+        CompletableFuture<ClusterStateDtoBuilder> builderFut = clusterManagementGroupManager.clusterState()
+                .handle((state, t) -> {
+                    ClusterStateDtoBuilder builder = ClusterStateDtoBuilder.newInstance();
+                    if (state != null) {
+                        builder.igniteVersion(state.version())
+                                .clusterTag(new ClusterTag(state.clusterTag().clusterName(), state.clusterTag().clusterId()))
+                                .formerClusterIds(state.formerClusterIds());
+                    } else {
+                        builder.igniteVersion("N/A").clusterTag(new ClusterTag("N/A", null));

Review Comment:
   `builder.unknownState()` might be useful here.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-23054 Improve cluster status REST endpoint [ignite-3]

Posted by "Pochatkin (via GitHub)" <gi...@apache.org>.
Pochatkin commented on code in PR #4614:
URL: https://github.com/apache/ignite-3/pull/4614#discussion_r1832452310


##########
modules/cli/src/main/java/org/apache/ignite/internal/cli/call/cluster/status/ClusterState.java:
##########
@@ -38,15 +39,26 @@ public class ClusterStatus {
 
     private final List<String> metadataStorageNodes;
 
-    private ClusterStatus(int nodeCount, boolean initialized, String name,
-            boolean connected, String nodeUrl, List<String> cmgNodes, List<String> metadataStorageNodes) {
+    private final ClusterStatus clusterStatus;
+
+    private ClusterState(
+            int nodeCount,
+            boolean initialized,
+            String name,
+            boolean connected,
+            String nodeUrl,
+            List<String> cmgNodes,
+            List<String> metadataStorageNodes,
+            ClusterStatus clusterStatus
+    ) {
         this.nodeCount = nodeCount;
         this.initialized = initialized;
         this.name = name;
         this.connected = connected;
         this.nodeUrl = nodeUrl;
         this.cmgNodes = cmgNodes;
         this.metadataStorageNodes = metadataStorageNodes;

Review Comment:
   We use builder approach in CLI module



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-23054 Improve cluster status REST endpoint [ignite-3]

Posted by "sk0x50 (via GitHub)" <gi...@apache.org>.
sk0x50 commented on PR #4614:
URL: https://github.com/apache/ignite-3/pull/4614#issuecomment-3588641702

   @Pochatkin is this PR still actual? can it be closed due to inactivity?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-23054 Improve cluster status REST endpoint [ignite-3]

Posted by "valepakh (via GitHub)" <gi...@apache.org>.
valepakh commented on code in PR #4614:
URL: https://github.com/apache/ignite-3/pull/4614#discussion_r1824144594


##########
modules/rest-api/src/main/java/org/apache/ignite/internal/rest/api/cluster/ClusterState.java:
##########
@@ -64,24 +73,28 @@ public class ClusterState {
     @JsonCreator
     public ClusterState(
             @JsonProperty("cmgNodes") Collection<String> cmgNodes,
-            @JsonProperty("msNodes") Collection<String> msNodes,
+            @JsonProperty("msNodes")  Collection<String> msNodes,

Review Comment:
   ```suggestion
               @JsonProperty("msNodes") Collection<String> msNodes,
   ```



##########
modules/cli/src/integrationTest/java/org/apache/ignite/internal/cli/commands/sql/ItSqlCommandTest.java:
##########
@@ -33,6 +33,8 @@ class ItSqlCommandTest extends CliSqlCommandTestBase {
     void nonExistingFile() {
         execute("sql", "--file", "nonexisting", "--jdbc-url", JDBC_URL);
 
+        CLUSTER.stopNode(0);

Review Comment:
   Why this is needed?



##########
modules/rest/src/main/java/org/apache/ignite/internal/rest/cluster/ClusterManagementController.java:
##########
@@ -84,16 +113,31 @@ public CompletableFuture<Void> init(@Body InitCommand initCommand) {
         });
     }
 
-    private static ClusterState mapClusterState(org.apache.ignite.internal.cluster.management.ClusterState clusterState) {
+    private ClusterState mapClusterState(org.apache.ignite.internal.cluster.management.ClusterState clusterState) {
         return new ClusterState(
                 clusterState.cmgNodes(),
                 clusterState.metaStorageNodes(),
                 clusterState.igniteVersion().toString(),
                 new ClusterTag(clusterState.clusterTag().clusterName(), clusterState.clusterTag().clusterId()),
-                clusterState.formerClusterIds()
+                clusterState.formerClusterIds(),
+                mapClusterStatus(clusterState)
         );
     }
 
+    private ClusterStatus mapClusterStatus(org.apache.ignite.internal.cluster.management.ClusterState clusterState) {
+        Set<String> metaStorageNodes = clusterState.metaStorageNodes();
+        long presentedMetaStorageNodes = topologyService.allMembers().stream()

Review Comment:
   ```suggestion
           long presentMetaStorageNodes = topologyService.allMembers().stream()
   ```



##########
modules/rest/src/main/java/org/apache/ignite/internal/rest/cluster/ClusterManagementController.java:
##########
@@ -84,16 +113,31 @@ public CompletableFuture<Void> init(@Body InitCommand initCommand) {
         });
     }
 
-    private static ClusterState mapClusterState(org.apache.ignite.internal.cluster.management.ClusterState clusterState) {
+    private ClusterState mapClusterState(org.apache.ignite.internal.cluster.management.ClusterState clusterState) {
         return new ClusterState(
                 clusterState.cmgNodes(),
                 clusterState.metaStorageNodes(),
                 clusterState.igniteVersion().toString(),
                 new ClusterTag(clusterState.clusterTag().clusterName(), clusterState.clusterTag().clusterId()),
-                clusterState.formerClusterIds()
+                clusterState.formerClusterIds(),
+                mapClusterStatus(clusterState)
         );
     }
 
+    private ClusterStatus mapClusterStatus(org.apache.ignite.internal.cluster.management.ClusterState clusterState) {
+        Set<String> metaStorageNodes = clusterState.metaStorageNodes();
+        long presentedMetaStorageNodes = topologyService.allMembers().stream()
+                .map(ClusterNode::name)
+                .filter(metaStorageNodes::contains)
+                .count();
+
+        if (presentedMetaStorageNodes <= metaStorageNodes.size() / 2) {

Review Comment:
   ```suggestion
           if (presentMetaStorageNodes <= metaStorageNodes.size() / 2) {
   ```



##########
modules/cli/src/integrationTest/java/org/apache/ignite/internal/cli/commands/cluster/status/ItClusterStatusCommandInitializedTest.java:
##########
@@ -17,37 +17,79 @@
 
 package org.apache.ignite.internal.cli.commands.cluster.status;
 
+import static java.util.function.Function.identity;
 import static java.util.stream.Collectors.joining;
 import static org.junit.jupiter.api.Assertions.assertAll;
 
 import java.util.Arrays;
-import org.apache.ignite.Ignite;
+import java.util.Map;
+import java.util.function.Function;
+import java.util.stream.Collectors;
+import java.util.stream.IntStream;
 import org.apache.ignite.internal.cli.CliIntegrationTest;
+import org.jetbrains.annotations.Nullable;
 import org.junit.jupiter.api.DisplayName;
 import org.junit.jupiter.api.Test;
 
 /**
  * Tests for {@link ClusterStatusCommand} for the cluster that is initialized.
  */
 class ItClusterStatusCommandInitializedTest extends CliIntegrationTest {
+    private Function<int[], String> mapper;
+
+    @Override
+    protected @Nullable int[] metastoreNodes() {
+        return new int[] { 0 };
+    }
+
+    @Override
+    protected @Nullable int[] cmgNodes() {
+        return new int[] { 1 };
+    }
+
     @Test
     @DisplayName("Should print status when valid cluster url is given but cluster is initialized")
-    void printStatus() {
-        String cmgNodes = Arrays.stream(cmgMetastoreNodes())
-                .mapToObj(CLUSTER::node)
-                .map(Ignite::name)
+    void printStatus() throws InterruptedException {
+        String node0Url = NODE_URL;
+        String node1Url = "http://localhost:" + CLUSTER.httpPort(1);
+
+        Map<Integer, String> nodeNames = IntStream.range(0, initialNodes())
+                .boxed()
+                .collect(Collectors.toMap(identity(), i ->  CLUSTER.node(i).name()));
+
+        mapper = nodes -> Arrays.stream(nodes)
+                .mapToObj(nodeNames::get)
                 .collect(joining(", ", "[", "]"));
 
-        execute("cluster", "status", "--url", NODE_URL);
+        CLUSTER.stopNode(0);
+        execute("cluster", "status", "--url", node1Url);
+        assertOutput("cluster", 2, "Metastore majority lost", cmgNodes(), metastoreNodes());
+
+        CLUSTER.startNode(0);
+        Thread.sleep(10000);

Review Comment:
   Let's change this to `await().untilAsserted()`



##########
modules/cli/src/main/java/org/apache/ignite/internal/cli/call/cluster/status/ClusterStatusCall.java:
##########
@@ -30,13 +30,13 @@
 import org.apache.ignite.rest.client.api.ClusterManagementApi;
 import org.apache.ignite.rest.client.invoker.ApiException;
 import org.apache.ignite.rest.client.model.ClusterNode;
-import org.apache.ignite.rest.client.model.ClusterState;
 
 /**
  * Call to get cluster status.
  */
 @Singleton
-public class ClusterStatusCall implements Call<UrlCallInput, ClusterStatus> {
+public class ClusterStatusCall implements Call<UrlCallInput, ClusterState> {
+    private static final int READ_TIMEOUT = 50_000;

Review Comment:
   Why this needs to be 50 seconds? As I understand this depends on the timeout settings in the cluster state retrieval, can we at least point to the place where this is configured in the CMGManager?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-23054 Improve cluster status REST endpoint [ignite-3]

Posted by "Pochatkin (via GitHub)" <gi...@apache.org>.
Pochatkin commented on code in PR #4614:
URL: https://github.com/apache/ignite-3/pull/4614#discussion_r1832451978


##########
modules/cli/src/main/java/org/apache/ignite/internal/cli/call/cluster/status/ClusterState.java:
##########
@@ -18,11 +18,12 @@
 package org.apache.ignite.internal.cli.call.cluster.status;
 
 import java.util.List;
+import org.apache.ignite.rest.client.model.ClusterStatus;
 
 /**
  * Class that represents the cluster status.
  */
-public class ClusterStatus {
+public class ClusterState {

Review Comment:
   This is CLI output DTO object. So, I have added postfix



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-23054 Improve cluster status REST endpoint [ignite-3]

Posted by "rpuch (via GitHub)" <gi...@apache.org>.
rpuch commented on code in PR #4614:
URL: https://github.com/apache/ignite-3/pull/4614#discussion_r1825728926


##########
modules/cli/src/main/java/org/apache/ignite/internal/cli/decorators/ClusterStatusDecorator.java:
##########
@@ -45,4 +46,17 @@ public TerminalOutput decorate(ClusterStatus data) {
                         data.nodeCount(), fg(Color.RED).mark("not initialized")
                 ));
     }
+
+    private static String status(ClusterStatus status) {
+        switch (status) {
+            case MS_MAJORITY_LOST:
+                return fg(Color.RED).mark("Metastore majority lost");
+            case HEALTHY:
+                return fg(Color.GREEN).mark("active");
+            case CMG_MAJORITY_LOST:
+                return fg(Color.RED).mark("CMG majority lost");

Review Comment:
   The ordering is a bit strange: why is `HEALTHY` in the middle?



##########
modules/rest-api/src/main/java/org/apache/ignite/internal/rest/api/cluster/ClusterState.java:
##########
@@ -34,24 +36,31 @@
  */
 @Schema(description = "Information about current cluster state.")
 public class ClusterState {
-    @Schema(description = "List of cluster management group nodes. These nodes are responsible for maintaining RAFT cluster topology.")
+    @Schema(description = "List of cluster management group nodes. These nodes are responsible for maintaining RAFT cluster topology.",
+            requiredMode = RequiredMode.REQUIRED)
     @IgniteToStringInclude
     private final Collection<String> cmgNodes;
 
-    @Schema(description = "List of metastorage nodes. These nodes are responsible for storing RAFT cluster metadata.")
+    @Schema(description = "List of metastorage nodes. These nodes are responsible for storing RAFT cluster metadata.",
+            requiredMode = RequiredMode.REQUIRED)
     @IgniteToStringInclude
     private final Collection<String> msNodes;
 
-    @Schema(description = "Version of Apache Ignite that the cluster was created on.")
+    @Schema(description = "Version of Apache Ignite that the cluster was created on.", requiredMode = RequiredMode.REQUIRED)
     private final String igniteVersion;
 
-    @Schema(description = "Unique tag that identifies the cluster.")
+    @Schema(description = "Unique tag that identifies the cluster.",
+            requiredMode = RequiredMode.REQUIRED)
     private final ClusterTag clusterTag;
 
     @Schema(description = "IDs the cluster had before.")
     @IgniteToStringInclude
     private final @Nullable List<UUID> formerClusterIds;
 
+    @Schema(description = "Cluster status.",
+            requiredMode = RequiredMode.REQUIRED)
+    private final ClusterStatus clusterStatus;

Review Comment:
   It seems that entities from different domains are mixed here. This `ClusterState` class represents the CMG cluster state that is the state stored in the CMG related to the cluster itself. But `ClusterStatus` is some volatile thing, it might change during runtime however it wants, even if the CMG cluster state does not change.
   
   I believe we should not mix them together. Could a distinct API endpoint (and CLI command) be added to access this volatile information?



##########
modules/cli/src/main/java/org/apache/ignite/internal/cli/call/cluster/status/ClusterState.java:
##########
@@ -38,15 +39,26 @@ public class ClusterStatus {
 
     private final List<String> metadataStorageNodes;
 
-    private ClusterStatus(int nodeCount, boolean initialized, String name,
-            boolean connected, String nodeUrl, List<String> cmgNodes, List<String> metadataStorageNodes) {
+    private final ClusterStatus clusterStatus;
+
+    private ClusterState(
+            int nodeCount,
+            boolean initialized,
+            String name,
+            boolean connected,
+            String nodeUrl,
+            List<String> cmgNodes,
+            List<String> metadataStorageNodes,
+            ClusterStatus clusterStatus
+    ) {
         this.nodeCount = nodeCount;
         this.initialized = initialized;
         this.name = name;
         this.connected = connected;
         this.nodeUrl = nodeUrl;
         this.cmgNodes = cmgNodes;
         this.metadataStorageNodes = metadataStorageNodes;

Review Comment:
   How about making defensive copies? Better safe than sorry, and this class should not have a huge amount of instances



##########
modules/rest-api/src/main/java/org/apache/ignite/internal/rest/api/cluster/ClusterStatus.java:
##########
@@ -0,0 +1,40 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.rest.api.cluster;
+
+/**
+ * Current health status of the cluster.
+ */
+public enum ClusterStatus {
+    /**
+     * The cluster is completely healthy. Minor losses in any of the groups are possible,
+     * but this does not affect the operation of the cluster.
+     */
+    HEALTHY,
+
+    /**
+     * The metastore group has lost its majority. Almost all of the cluster functions are inoperative.
+     * To restore their operation, it is necessary to return the majority to the metastore group.

Review Comment:
   ```suggestion
        * To restore their operation, it is necessary to restore the majority to the metastore group.
   ```



##########
modules/rest/src/main/java/org/apache/ignite/internal/rest/cluster/ClusterManagementController.java:
##########
@@ -84,16 +113,31 @@ public CompletableFuture<Void> init(@Body InitCommand initCommand) {
         });
     }
 
-    private static ClusterState mapClusterState(org.apache.ignite.internal.cluster.management.ClusterState clusterState) {
+    private ClusterState mapClusterState(org.apache.ignite.internal.cluster.management.ClusterState clusterState) {
         return new ClusterState(
                 clusterState.cmgNodes(),
                 clusterState.metaStorageNodes(),
                 clusterState.igniteVersion().toString(),
                 new ClusterTag(clusterState.clusterTag().clusterName(), clusterState.clusterTag().clusterId()),
-                clusterState.formerClusterIds()
+                clusterState.formerClusterIds(),
+                mapClusterStatus(clusterState)
         );
     }
 
+    private ClusterStatus mapClusterStatus(org.apache.ignite.internal.cluster.management.ClusterState clusterState) {
+        Set<String> metaStorageNodes = clusterState.metaStorageNodes();
+        long presentMetaStorageNodes = topologyService.allMembers().stream()
+                .map(ClusterNode::name)
+                .filter(metaStorageNodes::contains)
+                .count();
+
+        if (presentMetaStorageNodes <= metaStorageNodes.size() / 2) {
+            return ClusterStatus.MS_MAJORITY_LOST;

Review Comment:
   The fact that we have enough nodes in the physical topology does not necessarily imply we have an MG majority. For instance, something on levels higher than the network might prevent a leader to be elected. The most accurate way would be to try executing a read command against the MS group (like reading its current revision) and handle a timeout.



##########
modules/cli/src/main/java/org/apache/ignite/internal/cli/call/cluster/status/ClusterState.java:
##########
@@ -18,11 +18,12 @@
 package org.apache.ignite.internal.cli.call.cluster.status;
 
 import java.util.List;
+import org.apache.ignite.rest.client.model.ClusterStatus;
 
 /**
  * Class that represents the cluster status.
  */
-public class ClusterStatus {
+public class ClusterState {

Review Comment:
   We already have an entity called 'cluster state' in the CMG domain. This one is a different entity and yet it's called identically. Could a distinct name be devised? Something like ClusterRuntimeState, maybe.



##########
modules/rest-api/src/main/java/org/apache/ignite/internal/rest/api/cluster/ClusterStatus.java:
##########
@@ -0,0 +1,40 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.rest.api.cluster;
+
+/**
+ * Current health status of the cluster.
+ */
+public enum ClusterStatus {
+    /**
+     * The cluster is completely healthy. Minor losses in any of the groups are possible,
+     * but this does not affect the operation of the cluster.
+     */
+    HEALTHY,
+
+    /**
+     * The metastore group has lost its majority. Almost all of the cluster functions are inoperative.
+     * To restore their operation, it is necessary to return the majority to the metastore group.
+     */
+    MS_MAJORITY_LOST,
+
+    /**
+     * The cluster management group has lost its majority. The cluster is completely inoperative until the majority is returned.

Review Comment:
   ```suggestion
        * The cluster management group has lost its majority. The cluster is completely inoperative until the majority is restored.
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org