You are viewing a plain text version of this content. The canonical link for it is here.

Posted to reviews@iotdb.apache.org by "HxpSerein (via GitHub)" <gi...@apache.org> on 2025/01/13 01:19:04 UTC

[PR] [remove datanode] Add Remove DataNode SQL [iotdb]

HxpSerein opened a new pull request, #14678:
URL: https://github.com/apache/iotdb/pull/14678

   ## Description
   
   same as the title


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] [remove datanode] Add Remove DataNode SQL [iotdb]

Posted by "liyuheng55555 (via GitHub)" <gi...@apache.org>.

liyuheng55555 commented on code in PR #14678:
URL: https://github.com/apache/iotdb/pull/14678#discussion_r1919745323


##########
iotdb-core/relational-grammar/src/main/antlr4/org/apache/iotdb/db/relational/grammar/sql/RelationalSql.g4:
##########
@@ -464,7 +465,9 @@ migrateRegionStatement
     : MIGRATE REGION regionId=INTEGER_VALUE FROM fromId=INTEGER_VALUE TO toId=INTEGER_VALUE
     ;
 
-
+removeDataNodeStatement
+    : REMOVE DATANODE dataNodeId=INTEGER_VALUE (',' dataNodeId=INTEGER_VALUE)*
+    ;

Review Comment:
   remove datanode**s** ?



##########
iotdb-core/antlr/src/main/antlr4/org/apache/iotdb/db/qp/sql/IoTDBSqlParser.g4:
##########
@@ -536,6 +536,11 @@ verifyConnection
     : VERIFY CONNECTION (DETAILS)?
     ;
 
+// ---- Remove DataNode
+removeDataNode
+    : REMOVE DATANODE dataNodeId=INTEGER_LITERAL (COMMA dataNodeId=INTEGER_LITERAL)*

Review Comment:
   remove datanode**s**?



##########
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/execution/config/executor/ClusterConfigTaskExecutor.java:
##########
@@ -2773,6 +2779,62 @@ public SettableFuture<ConfigTaskResult> migrateRegion(
     return future;
   }
 
+  @Override
+  public SettableFuture<ConfigTaskResult> removeDataNode(
+      final RemoveDataNodeStatement removeDataNodeStatement) {
+    final SettableFuture<ConfigTaskResult> future = SettableFuture.create();
+    try (ConfigNodeClient configNodeClient =
+        CONFIG_NODE_CLIENT_MANAGER.borrowClient(ConfigNodeInfo.CONFIG_REGION_ID)) {
+      Set<Integer> nodeIds = removeDataNodeStatement.getNodeIds();
+
+      Set<Integer> validNodeIds =
+          configNodeClient.getDataNodeConfiguration(-1).getDataNodeConfigurationMap().keySet();
+
+      Set<Integer> invalidNodeIds = new HashSet<>(nodeIds);
+      invalidNodeIds.removeAll(validNodeIds);
+
+      if (!invalidNodeIds.isEmpty()) {
+        LOGGER.info("Cannot remove invalid nodeIds:{}", invalidNodeIds);
+        nodeIds.removeAll(invalidNodeIds);
+      }
+
+      LOGGER.info("Starting to remove DataNode with nodeIds: {}", nodeIds);
+
+      final Set<Integer> finalNodeIds = nodeIds;
+      List<TDataNodeLocation> removeDataNodeLocations =
+          configNodeClient
+              .getDataNodeConfiguration(-1)
+              .getDataNodeConfigurationMap()
+              .values()
+              .stream()
+              .map(TDataNodeConfiguration::getLocation)
+              .filter(location -> finalNodeIds.contains(location.getDataNodeId()))
+              .collect(Collectors.toList());
+      LOGGER.info(
+          "Start to remove datanode, removed DataNodes endpoint: {}", removeDataNodeLocations);

Review Comment:
   It's not recommended to print TDataNodeLocation directly, because it's too long.
   
   Please consider RegionMaintainHandler.simplifiedLocation



##########
iotdb-core/antlr/src/main/antlr4/org/apache/iotdb/db/qp/sql/SqlLexer.g4:
##########
@@ -217,6 +217,14 @@ DATABASES
     : D A T A B A S E S
     ;
 
+DATANODE
+    : D A T A N O D E
+    ;
+
+DATANODEADDRESS
+    : D A T A N O D E A D D R E S S
+    ;

Review Comment:
   what's this



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] [remove datanode] Add Remove DataNode SQL [iotdb]

Posted by "OneSizeFitsQuorum (via GitHub)" <gi...@apache.org>.

OneSizeFitsQuorum commented on code in PR #14678:
URL: https://github.com/apache/iotdb/pull/14678#discussion_r1922993265


##########
integration-test/src/test/java/org/apache/iotdb/confignode/it/removedatanode/IoTDBRemoveDataNodeITFramework.java:
##########
@@ -66,6 +69,10 @@ public class IoTDBRemoveDataNodeITFramework {
   private static final String defaultSchemaRegionGroupExtensionPolicy = "CUSTOM";
   private static final String defaultDataRegionGroupExtensionPolicy = "CUSTOM";
 
+  public static final int NOT_USE_SQL = 0;

Review Comment:
   use enum?



##########
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/statement/metadata/RemoveDataNodeStatement.java:
##########
@@ -0,0 +1,60 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.db.queryengine.plan.statement.metadata;
+
+import org.apache.iotdb.commons.path.PartialPath;
+import org.apache.iotdb.db.queryengine.plan.analyze.QueryType;
+import org.apache.iotdb.db.queryengine.plan.statement.IConfigStatement;
+import org.apache.iotdb.db.queryengine.plan.statement.Statement;
+import org.apache.iotdb.db.queryengine.plan.statement.StatementVisitor;
+
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+
+public class RemoveDataNodeStatement extends Statement implements IConfigStatement {

Review Comment:
   override checkPermissionBeforeProcess？The maintain permission can also do this. Please also check region migration related permissions
   
   <img width="886" alt="image" src="https://github.com/user-attachments/assets/132ac303-32fe-4746-97a9-9632be8274fd" />
   



##########
iotdb-core/antlr/src/main/antlr4/org/apache/iotdb/db/qp/sql/IoTDBSqlParser.g4:
##########
@@ -536,6 +536,11 @@ verifyConnection
     : VERIFY CONNECTION (DETAILS)?
     ;
 
+// ---- Remove DataNode
+removeDataNode
+    : REMOVE DATANODE dataNodeId=INTEGER_LITERAL (COMMA dataNodeId=INTEGER_LITERAL)*

Review Comment:
   I'm ok with current implementation because we use `remove-datanode.sh`



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org