You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@iotdb.apache.org by GitBox <gi...@apache.org> on 2022/09/06 03:09:00 UTC

[GitHub] [iotdb] MarcosZyk opened a new pull request, #7242: Delete timeseries

MarcosZyk opened a new pull request, #7242:
URL: https://github.com/apache/iotdb/pull/7242

   ## Description
   
   
   ### Content1 ...
   
   ### Content2 ...
   
   ### Content3 ...
   
   <!--
   In each section, please describe design decisions made, including:
    - Choice of algorithms
    - Behavioral aspects. What configuration values are acceptable? How are corner cases and error 
       conditions handled, such as when there are insufficient resources?
    - Class organization and design (how the logic is split between classes, inheritance, composition, 
       design patterns)
    - Method organization and design (how the logic is split between methods, parameters and return types)
    - Naming (class, method, API, configuration, HTTP endpoint, names of emitted metrics)
   -->
   
   
   <!-- It's good to describe an alternative design (or mention an alternative name) for every design 
   (or naming) decision point and compare the alternatives with the designs that you've implemented 
   (or the names you've chosen) to highlight the advantages of the chosen designs and names. -->
   
   <!-- If there was a discussion of the design of the feature implemented in this PR elsewhere 
   (e. g. a "Proposal" issue, any other issue, or a thread in the development mailing list), 
   link to that discussion from this PR description and explain what have changed in your final design 
   compared to your original proposal or the consensus version in the end of the discussion. 
   If something hasn't changed since the original discussion, you can omit a detailed discussion of 
   those aspects of the design here, perhaps apart from brief mentioning for the sake of readability 
   of this PR description. -->
   
   <!-- Some of the aspects mentioned above may be omitted for simple and small changes. -->
   
   <hr>
   
   This PR has:
   - [ ] been self-reviewed.
       - [ ] concurrent read
       - [ ] concurrent write
       - [ ] concurrent read and write 
   - [ ] added documentation for new or modified features or behaviors.
   - [ ] added Javadocs for most classes and all non-trivial methods. 
   - [ ] added or updated version, __license__, or notice information
   - [ ] added comments explaining the "why" and the intent of the code wherever would not be obvious 
     for an unfamiliar reader.
   - [ ] added unit tests or modified existing tests to cover new code paths, ensuring the threshold 
     for code coverage.
   - [ ] added integration tests.
   - [ ] been tested in a test IoTDB cluster.
   
   <!-- Check the items by putting "x" in the brackets for the done things. Not all of these items 
   apply to every PR. Remove the items which are not done or not relevant to the PR. None of the items 
   from the checklist above are strictly necessary, but it would be very helpful if you at least 
   self-review the PR. -->
   
   <hr>
   
   ##### Key changed/added classes (or packages if there are too many classes) in this PR
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [iotdb] MarcosZyk commented on a diff in pull request #7242: [IOTDB-3184] Implement Delete Timeseries for Cluster

Posted by GitBox <gi...@apache.org>.
MarcosZyk commented on code in PR #7242:
URL: https://github.com/apache/iotdb/pull/7242#discussion_r969512044


##########
confignode/src/main/java/org/apache/iotdb/confignode/procedure/impl/DeleteTimeSeriesProcedure.java:
##########
@@ -0,0 +1,611 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.confignode.procedure.impl;
+
+import org.apache.iotdb.common.rpc.thrift.TConsensusGroupId;
+import org.apache.iotdb.common.rpc.thrift.TDataNodeLocation;
+import org.apache.iotdb.common.rpc.thrift.TRegionReplicaSet;
+import org.apache.iotdb.common.rpc.thrift.TSStatus;
+import org.apache.iotdb.commons.path.PartialPath;
+import org.apache.iotdb.confignode.client.async.datanode.AsyncDataNodeClientPool;
+import org.apache.iotdb.confignode.client.async.handlers.ConstructSchemaBlackListHandler;
+import org.apache.iotdb.confignode.client.async.handlers.DeleteDataForDeleteTimeSeriesHandler;
+import org.apache.iotdb.confignode.client.async.handlers.DeleteTimeSeriesHandler;
+import org.apache.iotdb.confignode.client.async.handlers.FetchSchemaBlackLsitHandler;
+import org.apache.iotdb.confignode.client.async.handlers.InvalidateMatchedSchemaCacheHandler;
+import org.apache.iotdb.confignode.client.async.handlers.RollbackSchemaBlackListHandler;
+import org.apache.iotdb.confignode.procedure.StateMachineProcedure;
+import org.apache.iotdb.confignode.procedure.env.ConfigNodeProcedureEnv;
+import org.apache.iotdb.confignode.procedure.exception.ProcedureException;
+import org.apache.iotdb.confignode.procedure.exception.ProcedureSuspendedException;
+import org.apache.iotdb.confignode.procedure.exception.ProcedureYieldException;
+import org.apache.iotdb.confignode.procedure.state.DeleteTimeSeriesState;
+import org.apache.iotdb.confignode.procedure.store.ProcedureFactory;
+import org.apache.iotdb.db.exception.metadata.PathNotExistException;
+import org.apache.iotdb.db.mpp.common.schematree.PathPatternTree;
+import org.apache.iotdb.mpp.rpc.thrift.TConstructSchemaBlackListReq;
+import org.apache.iotdb.mpp.rpc.thrift.TDeleteDataForDeleteTimeSeriesReq;
+import org.apache.iotdb.mpp.rpc.thrift.TDeleteTimeSeriesReq;
+import org.apache.iotdb.mpp.rpc.thrift.TFetchSchemaBlackListReq;
+import org.apache.iotdb.mpp.rpc.thrift.TFetchSchemaBlackListResp;
+import org.apache.iotdb.mpp.rpc.thrift.TInvalidateMatchedSchemaCacheReq;
+import org.apache.iotdb.mpp.rpc.thrift.TRollbackSchemaBlackListReq;
+import org.apache.iotdb.rpc.TSStatusCode;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.ByteArrayOutputStream;
+import java.io.DataOutputStream;
+import java.io.IOException;
+import java.nio.ByteBuffer;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.stream.Collectors;
+
+public class DeleteTimeSeriesProcedure
+    extends StateMachineProcedure<ConfigNodeProcedureEnv, DeleteTimeSeriesState> {
+
+  private static final Logger LOGGER = LoggerFactory.getLogger(DeleteTimeSeriesProcedure.class);
+
+  private PathPatternTree patternTree;
+  private ByteBuffer patternTreeBytes;
+
+  private String requestMessage;
+
+  public DeleteTimeSeriesProcedure() {
+    super();
+  }
+
+  public DeleteTimeSeriesProcedure(PathPatternTree patternTree) {
+    super();
+    setPatternTree(patternTree);
+  }
+
+  @Override
+  protected Flow executeFromState(ConfigNodeProcedureEnv env, DeleteTimeSeriesState state)
+      throws ProcedureSuspendedException, ProcedureYieldException, InterruptedException {
+    switch (state) {
+      case CONSTRUCT_BLACK_LIST:
+        LOGGER.info("Construct schema black list of timeseries {}", requestMessage);
+        if (constructBlackList(env) > 0) {
+          setNextState(DeleteTimeSeriesState.CLEAN_DATANODE_SCHEMA_CACHE);
+          break;
+        } else {
+          setFailure(
+              new ProcedureException(
+                  new PathNotExistException(
+                      patternTree.getAllPathPatterns().stream()
+                          .map(PartialPath::getFullPath)
+                          .collect(Collectors.toList()))));
+          return Flow.NO_MORE_STATE;
+        }
+      case CLEAN_DATANODE_SCHEMA_CACHE:
+        LOGGER.info("Invalidate cache of timeseries {}", requestMessage);
+        invalidateCache(env);
+        break;
+      case DELETE_DATA:
+        LOGGER.info("Delete data of timeseries {}", requestMessage);
+        deleteData(env);
+        break;
+      case DELETE_TIMESERIES:
+        LOGGER.info("Delete timeseries schema of {}", requestMessage);
+        deleteTimeSeries(env);
+        return Flow.NO_MORE_STATE;
+      default:
+        setFailure(new ProcedureException("Unrecognized state " + state.toString()));
+        return Flow.NO_MORE_STATE;
+    }
+    return Flow.HAS_MORE_STATE;
+  }
+
+  private int constructBlackList(ConfigNodeProcedureEnv env) {

Review Comment:
   Annotations added.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [iotdb] MarcosZyk commented on a diff in pull request #7242: [IOTDB-3184] Implement Delete Timeseries for Cluster

Posted by GitBox <gi...@apache.org>.
MarcosZyk commented on code in PR #7242:
URL: https://github.com/apache/iotdb/pull/7242#discussion_r969513069


##########
confignode/src/main/java/org/apache/iotdb/confignode/client/async/handlers/FetchSchemaBlackLsitHandler.java:
##########
@@ -0,0 +1,101 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.confignode.client.async.handlers;
+
+import org.apache.iotdb.common.rpc.thrift.TDataNodeLocation;
+import org.apache.iotdb.common.rpc.thrift.TSStatus;
+import org.apache.iotdb.confignode.client.DataNodeRequestType;
+import org.apache.iotdb.mpp.rpc.thrift.TFetchSchemaBlackListResp;
+import org.apache.iotdb.rpc.RpcUtils;
+import org.apache.iotdb.rpc.TSStatusCode;
+
+import org.apache.thrift.async.AsyncMethodCallback;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.util.HashMap;
+import java.util.Map;
+import java.util.concurrent.CountDownLatch;
+
+public class FetchSchemaBlackLsitHandler extends AbstractRetryHandler
+    implements AsyncMethodCallback<TFetchSchemaBlackListResp> {
+
+  private static final Logger LOGGER = LoggerFactory.getLogger(FetchSchemaBlackLsitHandler.class);
+
+  private Map<Integer, TFetchSchemaBlackListResp> dataNodeResponseMap = new HashMap<>();
+
+  public FetchSchemaBlackLsitHandler(Map<Integer, TDataNodeLocation> dataNodeLocationMap) {
+    super(DataNodeRequestType.FETCH_SCHEMA_BLACK_LIST, dataNodeLocationMap);
+  }
+
+  public FetchSchemaBlackLsitHandler(
+      CountDownLatch countDownLatch,
+      TDataNodeLocation targetDataNode,
+      Map<Integer, TDataNodeLocation> dataNodeLocationMap) {
+    super(
+        countDownLatch,
+        DataNodeRequestType.FETCH_SCHEMA_BLACK_LIST,
+        targetDataNode,
+        dataNodeLocationMap);
+  }
+
+  public Map<Integer, TFetchSchemaBlackListResp> getDataNodeResponseMap() {
+    return dataNodeResponseMap;
+  }
+
+  @Override
+  public void onComplete(TFetchSchemaBlackListResp tFetchSchemaBlackListResp) {
+    TSStatus tsStatus = tFetchSchemaBlackListResp.getStatus();
+    dataNodeResponseMap.put(targetDataNode.getDataNodeId(), tFetchSchemaBlackListResp);
+    if (tsStatus.getCode() == TSStatusCode.SUCCESS_STATUS.getStatusCode()) {
+      dataNodeLocationMap.remove(targetDataNode.getDataNodeId());
+      LOGGER.info("Successfully fetch schema black list on DataNode: {}", targetDataNode);
+    } else if (tsStatus.getCode() == TSStatusCode.MULTIPLE_ERROR.getStatusCode()) {
+      dataNodeLocationMap.remove(targetDataNode.getDataNodeId());
+      LOGGER.error(
+          "Failed to fetch schema black list on DataNode {}, {}",
+          dataNodeLocationMap.get(targetDataNode.getDataNodeId()),
+          tsStatus);

Review Comment:
   Fixed.



##########
confignode/src/main/java/org/apache/iotdb/confignode/client/async/handlers/RollbackSchemaBlackListHandler.java:
##########
@@ -0,0 +1,99 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.confignode.client.async.handlers;
+
+import org.apache.iotdb.common.rpc.thrift.TDataNodeLocation;
+import org.apache.iotdb.common.rpc.thrift.TSStatus;
+import org.apache.iotdb.confignode.client.DataNodeRequestType;
+import org.apache.iotdb.rpc.RpcUtils;
+import org.apache.iotdb.rpc.TSStatusCode;
+
+import org.apache.thrift.async.AsyncMethodCallback;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.util.HashMap;
+import java.util.Map;
+import java.util.concurrent.CountDownLatch;
+
+public class RollbackSchemaBlackListHandler extends AbstractRetryHandler
+    implements AsyncMethodCallback<TSStatus> {
+
+  private static final Logger LOGGER =
+      LoggerFactory.getLogger(RollbackSchemaBlackListHandler.class);
+
+  private Map<Integer, TSStatus> dataNodeResponseStatusMap = new HashMap<>();
+
+  public RollbackSchemaBlackListHandler(Map<Integer, TDataNodeLocation> dataNodeLocationMap) {
+    super(DataNodeRequestType.ROLLBACK_SCHEMA_BLACK_LIST, dataNodeLocationMap);
+  }
+
+  public RollbackSchemaBlackListHandler(
+      CountDownLatch countDownLatch,
+      TDataNodeLocation targetDataNode,
+      Map<Integer, TDataNodeLocation> dataNodeLocationMap) {
+    super(
+        countDownLatch,
+        DataNodeRequestType.ROLLBACK_SCHEMA_BLACK_LIST,
+        targetDataNode,
+        dataNodeLocationMap);
+  }
+
+  public Map<Integer, TSStatus> getDataNodeResponseStatusMap() {
+    return dataNodeResponseStatusMap;
+  }
+
+  @Override
+  public void onComplete(TSStatus tsStatus) {
+    dataNodeResponseStatusMap.put(targetDataNode.getDataNodeId(), tsStatus);
+    if (tsStatus.getCode() == TSStatusCode.SUCCESS_STATUS.getStatusCode()) {
+      dataNodeLocationMap.remove(targetDataNode.getDataNodeId());
+      LOGGER.info("Successfully rollback schema black list on DataNode: {}", targetDataNode);
+    } else if (tsStatus.getCode() == TSStatusCode.MULTIPLE_ERROR.getStatusCode()) {
+      dataNodeLocationMap.remove(targetDataNode.getDataNodeId());
+      LOGGER.error(
+          "Failed to rollback schema black list on DataNode {}, {}",
+          dataNodeLocationMap.get(targetDataNode.getDataNodeId()),
+          tsStatus);

Review Comment:
   Fixed.



##########
server/src/main/java/org/apache/iotdb/db/metadata/mnode/MeasurementMNode.java:
##########
@@ -46,6 +46,8 @@ public class MeasurementMNode extends MNode implements IMeasurementMNode {
   private long offset = -1;
   /** measurement's Schema for one timeseries represented by current leaf node */
   private IMeasurementSchema schema;
+  /** whether this measurement is been pre deleted and considered in black list */

Review Comment:
   Fixed.



##########
confignode/src/main/java/org/apache/iotdb/confignode/client/async/datanode/AsyncDataNodeClientPool.java:
##########
@@ -183,6 +224,31 @@ public void sendAsyncRequestToDataNodeWithRetry(
     }
   }
 
+  public void sendAsyncRequestToDataNodeWithRetry(Object req, AbstractRetryHandler handler) {
+    Map<Integer, TDataNodeLocation> dataNodeLocationMap = handler.getDataNodeLocationMap();
+    if (dataNodeLocationMap.isEmpty()) {
+      return;
+    }
+    for (int retry = 0; retry < MAX_RETRY_NUM; retry++) {
+      CountDownLatch countDownLatch = new CountDownLatch(dataNodeLocationMap.size());
+      handler.setCountDownLatch(countDownLatch);
+      for (TDataNodeLocation targetDataNode : dataNodeLocationMap.values()) {
+        handler.setTargetDataNode(targetDataNode);
+        sendAsyncRequestToDataNode(targetDataNode, req, handler, retry);

Review Comment:
   Fixed. PTAL



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [iotdb] wangchao316 commented on a diff in pull request #7242: [IOTDB-3184] Implement Delete Timeseries for Cluster

Posted by GitBox <gi...@apache.org>.
wangchao316 commented on code in PR #7242:
URL: https://github.com/apache/iotdb/pull/7242#discussion_r969066406


##########
confignode/src/main/java/org/apache/iotdb/confignode/procedure/impl/DeleteTimeSeriesProcedure.java:
##########
@@ -0,0 +1,611 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.confignode.procedure.impl;
+
+import org.apache.iotdb.common.rpc.thrift.TConsensusGroupId;
+import org.apache.iotdb.common.rpc.thrift.TDataNodeLocation;
+import org.apache.iotdb.common.rpc.thrift.TRegionReplicaSet;
+import org.apache.iotdb.common.rpc.thrift.TSStatus;
+import org.apache.iotdb.commons.path.PartialPath;
+import org.apache.iotdb.confignode.client.async.datanode.AsyncDataNodeClientPool;
+import org.apache.iotdb.confignode.client.async.handlers.ConstructSchemaBlackListHandler;
+import org.apache.iotdb.confignode.client.async.handlers.DeleteDataForDeleteTimeSeriesHandler;
+import org.apache.iotdb.confignode.client.async.handlers.DeleteTimeSeriesHandler;
+import org.apache.iotdb.confignode.client.async.handlers.FetchSchemaBlackLsitHandler;
+import org.apache.iotdb.confignode.client.async.handlers.InvalidateMatchedSchemaCacheHandler;
+import org.apache.iotdb.confignode.client.async.handlers.RollbackSchemaBlackListHandler;
+import org.apache.iotdb.confignode.procedure.StateMachineProcedure;
+import org.apache.iotdb.confignode.procedure.env.ConfigNodeProcedureEnv;
+import org.apache.iotdb.confignode.procedure.exception.ProcedureException;
+import org.apache.iotdb.confignode.procedure.exception.ProcedureSuspendedException;
+import org.apache.iotdb.confignode.procedure.exception.ProcedureYieldException;
+import org.apache.iotdb.confignode.procedure.state.DeleteTimeSeriesState;
+import org.apache.iotdb.confignode.procedure.store.ProcedureFactory;
+import org.apache.iotdb.db.exception.metadata.PathNotExistException;
+import org.apache.iotdb.db.mpp.common.schematree.PathPatternTree;
+import org.apache.iotdb.mpp.rpc.thrift.TConstructSchemaBlackListReq;
+import org.apache.iotdb.mpp.rpc.thrift.TDeleteDataForDeleteTimeSeriesReq;
+import org.apache.iotdb.mpp.rpc.thrift.TDeleteTimeSeriesReq;
+import org.apache.iotdb.mpp.rpc.thrift.TFetchSchemaBlackListReq;
+import org.apache.iotdb.mpp.rpc.thrift.TFetchSchemaBlackListResp;
+import org.apache.iotdb.mpp.rpc.thrift.TInvalidateMatchedSchemaCacheReq;
+import org.apache.iotdb.mpp.rpc.thrift.TRollbackSchemaBlackListReq;
+import org.apache.iotdb.rpc.TSStatusCode;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.ByteArrayOutputStream;
+import java.io.DataOutputStream;
+import java.io.IOException;
+import java.nio.ByteBuffer;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.stream.Collectors;
+
+public class DeleteTimeSeriesProcedure
+    extends StateMachineProcedure<ConfigNodeProcedureEnv, DeleteTimeSeriesState> {
+
+  private static final Logger LOGGER = LoggerFactory.getLogger(DeleteTimeSeriesProcedure.class);
+
+  private PathPatternTree patternTree;
+  private ByteBuffer patternTreeBytes;
+
+  private String requestMessage;
+
+  public DeleteTimeSeriesProcedure() {
+    super();
+  }
+
+  public DeleteTimeSeriesProcedure(PathPatternTree patternTree) {
+    super();
+    setPatternTree(patternTree);
+  }
+
+  @Override
+  protected Flow executeFromState(ConfigNodeProcedureEnv env, DeleteTimeSeriesState state)
+      throws ProcedureSuspendedException, ProcedureYieldException, InterruptedException {
+    switch (state) {
+      case CONSTRUCT_BLACK_LIST:
+        LOGGER.info("Construct schema black list of timeseries {}", requestMessage);
+        if (constructBlackList(env) > 0) {
+          setNextState(DeleteTimeSeriesState.CLEAN_DATANODE_SCHEMA_CACHE);
+          break;
+        } else {
+          setFailure(
+              new ProcedureException(
+                  new PathNotExistException(
+                      patternTree.getAllPathPatterns().stream()
+                          .map(PartialPath::getFullPath)
+                          .collect(Collectors.toList()))));
+          return Flow.NO_MORE_STATE;
+        }
+      case CLEAN_DATANODE_SCHEMA_CACHE:
+        LOGGER.info("Invalidate cache of timeseries {}", requestMessage);
+        invalidateCache(env);
+        break;
+      case DELETE_DATA:
+        LOGGER.info("Delete data of timeseries {}", requestMessage);
+        deleteData(env);
+        break;
+      case DELETE_TIMESERIES:
+        LOGGER.info("Delete timeseries schema of {}", requestMessage);
+        deleteTimeSeries(env);
+        return Flow.NO_MORE_STATE;
+      default:
+        setFailure(new ProcedureException("Unrecognized state " + state.toString()));
+        return Flow.NO_MORE_STATE;
+    }
+    return Flow.HAS_MORE_STATE;
+  }
+
+  private int constructBlackList(ConfigNodeProcedureEnv env) {
+    Map<TConsensusGroupId, TRegionReplicaSet> targetSchemaRegionGroup =
+        env.getConfigManager().getRelatedSchemaRegionGroup(patternTree);
+    if (targetSchemaRegionGroup.isEmpty()) {
+      return 0;
+    }
+    RegionTask<TSStatus> constructBlackListTask =
+        new RegionTask<TSStatus>("construct schema black list", env, targetSchemaRegionGroup) {
+          @Override
+          Map<Integer, TSStatus> sendRequest(
+              TDataNodeLocation dataNodeLocation, List<TConsensusGroupId> consensusGroupIdList) {
+            // construct request and send
+            Map<Integer, TDataNodeLocation> dataNodeLocationMap = new HashMap<>();
+            dataNodeLocationMap.put(dataNodeLocation.getDataNodeId(), dataNodeLocation);
+            ConstructSchemaBlackListHandler handler =
+                new ConstructSchemaBlackListHandler(dataNodeLocationMap);
+            AsyncDataNodeClientPool.getInstance()
+                .sendAsyncRequestToDataNodeWithRetry(
+                    new TConstructSchemaBlackListReq(consensusGroupIdList, patternTreeBytes),
+                    handler);
+            handler
+                .getDataNodeResponseStatusMap()
+                .forEach(
+                    (k, v) -> {
+                      if (v.getCode() == TSStatusCode.SUCCESS_STATUS.getStatusCode()) {
+                        responseMap.put(k, v);
+                      }
+                    });
+            return handler.getDataNodeResponseStatusMap();
+          }
+        };
+    constructBlackListTask.execute();
+
+    if (isFailed()) {
+      return 0;
+    }
+
+    int preDeletedNum = 0;
+    for (TSStatus resp : constructBlackListTask.getResponseMap().values()) {
+      preDeletedNum += Integer.parseInt(resp.getMessage());
+    }
+    return preDeletedNum;
+  }
+
+  private void invalidateCache(ConfigNodeProcedureEnv env) {
+    Map<Integer, TDataNodeLocation> dataNodeLocationMap =
+        env.getConfigManager().getNodeManager().getRegisteredDataNodeLocations();
+    InvalidateMatchedSchemaCacheHandler handler =
+        new InvalidateMatchedSchemaCacheHandler(dataNodeLocationMap);
+    AsyncDataNodeClientPool.getInstance()
+        .sendAsyncRequestToDataNodeWithRetry(
+            new TInvalidateMatchedSchemaCacheReq(patternTreeBytes), handler);
+    Map<Integer, TSStatus> statusMap = handler.getDataNodeResponseStatusMap();
+    for (TSStatus status : statusMap.values()) {
+      // all dataNodes must clear the related schema cache
+      if (status.getCode() != TSStatusCode.SUCCESS_STATUS.getStatusCode()) {
+        LOGGER.error("Failed to invalidate schema cache of timeseries {}", requestMessage);
+        setFailure(new ProcedureException("Invalidate schema cache failed"));
+        return;
+      }
+    }
+
+    setNextState(DeleteTimeSeriesState.DELETE_DATA);
+  }
+
+  private void deleteData(ConfigNodeProcedureEnv env) {
+    Map<TConsensusGroupId, TRegionReplicaSet> relatedSchemaRegionGroup =
+        env.getConfigManager().getRelatedSchemaRegionGroup(patternTree);
+    Map<TDataNodeLocation, List<TConsensusGroupId>> dataNodeSchemaRegionGroupGroupIdMap =
+        getLeaderDataNodeRegionGroupMap(
+            env.getConfigManager().getPartitionManager().getAllLeadership(),
+            relatedSchemaRegionGroup);
+
+    // fetch schema black list by dataNode
+    for (Map.Entry<TDataNodeLocation, List<TConsensusGroupId>> entry :
+        dataNodeSchemaRegionGroupGroupIdMap.entrySet()) {
+      Map<TConsensusGroupId, TRegionReplicaSet> targetSchemaRegionGroup = new HashMap<>();
+      entry
+          .getValue()
+          .forEach(
+              consensusGroupId ->
+                  targetSchemaRegionGroup.put(
+                      consensusGroupId, relatedSchemaRegionGroup.get(consensusGroupId)));
+      PathPatternTree patternTree =
+          fetchSchemaBlackListOnTargetDataNode(env, targetSchemaRegionGroup);
+      if (isFailed()) {
+        return;
+      }
+      if (patternTree == null) {
+        LOGGER.error(
+            "Failed to fetch schema black list for delete data of timeseries {} on {}",
+            requestMessage,
+            entry.getKey());
+        setFailure(new ProcedureException("Fetch schema black list forDelete data failed"));
+        return;
+      }
+
+      if (patternTree.isEmpty()) {
+        continue;
+      }
+
+      Map<TConsensusGroupId, TRegionReplicaSet> relatedDataRegionGroup =
+          env.getConfigManager().getRelatedDataRegionGroup(patternTree);
+
+      // target timeseries has no data
+      if (relatedDataRegionGroup.isEmpty()) {
+        continue;
+      }
+
+      RegionTask<TSStatus> deleteDataTask =
+          new RegionTask<TSStatus>("delete data", env, relatedDataRegionGroup) {
+            @Override
+            Map<Integer, TSStatus> sendRequest(
+                TDataNodeLocation dataNodeLocation, List<TConsensusGroupId> consensusGroupIdList) {
+              Map<Integer, TDataNodeLocation> dataNodeLocationMap = new HashMap<>();
+              dataNodeLocationMap.put(dataNodeLocation.getDataNodeId(), dataNodeLocation);
+              DeleteDataForDeleteTimeSeriesHandler handler =
+                  new DeleteDataForDeleteTimeSeriesHandler(dataNodeLocationMap);
+              AsyncDataNodeClientPool.getInstance()
+                  .sendAsyncRequestToDataNodeWithRetry(
+                      new TDeleteDataForDeleteTimeSeriesReq(
+                          new ArrayList<>(consensusGroupIdList),
+                          preparePatternTreeBytesData(patternTree)),
+                      handler);
+              return handler.getDataNodeResponseStatusMap();
+            }
+          };
+      deleteDataTask.setExecuteOnAllReplicaset(true);
+      deleteDataTask.execute();
+      if (isFailed()) {
+        return;
+      }
+    }
+    setNextState(DeleteTimeSeriesState.DELETE_TIMESERIES);
+  }
+
+  private PathPatternTree fetchSchemaBlackListOnTargetDataNode(
+      ConfigNodeProcedureEnv env,
+      Map<TConsensusGroupId, TRegionReplicaSet> targetSchemaRegionGroup) {
+    RegionTask<TFetchSchemaBlackListResp> fetchSchemaBlackListTask =
+        new RegionTask<TFetchSchemaBlackListResp>(
+            "fetch schema black list", env, targetSchemaRegionGroup) {
+          @Override
+          Map<Integer, TSStatus> sendRequest(
+              TDataNodeLocation dataNodeLocation, List<TConsensusGroupId> consensusGroupIdList) {
+            Map<Integer, TDataNodeLocation> dataNodeLocationMap = new HashMap<>();
+            dataNodeLocationMap.put(dataNodeLocation.getDataNodeId(), dataNodeLocation);
+            FetchSchemaBlackLsitHandler handler =
+                new FetchSchemaBlackLsitHandler(dataNodeLocationMap);
+            AsyncDataNodeClientPool.getInstance()
+                .sendAsyncRequestToDataNodeWithRetry(
+                    new TFetchSchemaBlackListReq(consensusGroupIdList, patternTreeBytes), handler);
+            Map<Integer, TFetchSchemaBlackListResp> respMap = handler.getDataNodeResponseMap();
+            Map<Integer, TSStatus> statusMap = new HashMap<>();
+            respMap.forEach(
+                (k, v) -> {
+                  if (v.getStatus().getCode() == TSStatusCode.SUCCESS_STATUS.getStatusCode()) {
+                    responseMap.put(k, v);
+                  }
+                  statusMap.put(k, v.getStatus());
+                });
+            return statusMap;
+          }
+        };
+    fetchSchemaBlackListTask.execute();
+    if (isFailed()) {
+      return null;
+    }
+
+    Map<Integer, TFetchSchemaBlackListResp> respMap = fetchSchemaBlackListTask.getResponseMap();
+    PathPatternTree patternTree = new PathPatternTree();
+    for (TFetchSchemaBlackListResp resp : respMap.values()) {
+      for (PartialPath path :
+          PathPatternTree.deserialize(ByteBuffer.wrap(resp.getPathPatternTree()))
+              .getAllPathPatterns()) {
+        patternTree.appendPathPattern(path);
+      }
+    }
+    patternTree.constructTree();
+    return patternTree;
+  }
+
+  private void deleteTimeSeries(ConfigNodeProcedureEnv env) {
+    RegionTask<TSStatus> deleteTimeSeriesTask =
+        new RegionTask<TSStatus>(
+            "delete timeseries",
+            env,
+            env.getConfigManager().getRelatedSchemaRegionGroup(patternTree)) {
+          @Override
+          Map<Integer, TSStatus> sendRequest(
+              TDataNodeLocation dataNodeLocation, List<TConsensusGroupId> consensusGroupIdList) {
+            Map<Integer, TDataNodeLocation> dataNodeLocationMap = new HashMap<>();
+            dataNodeLocationMap.put(dataNodeLocation.getDataNodeId(), dataNodeLocation);
+            DeleteTimeSeriesHandler handler = new DeleteTimeSeriesHandler(dataNodeLocationMap);
+            AsyncDataNodeClientPool.getInstance()
+                .sendAsyncRequestToDataNodeWithRetry(
+                    new TDeleteTimeSeriesReq(consensusGroupIdList, patternTreeBytes), handler);
+            return handler.getDataNodeResponseStatusMap();
+          }
+        };
+    deleteTimeSeriesTask.execute();
+  }
+
+  private Map<TDataNodeLocation, List<TConsensusGroupId>> getLeaderDataNodeRegionGroupMap(
+      Map<TConsensusGroupId, Integer> leaderMap,
+      Map<TConsensusGroupId, TRegionReplicaSet> regionReplicaSetMap) {
+    Map<TDataNodeLocation, List<TConsensusGroupId>> dataNodeConsensusGroupIdMap = new HashMap<>();
+    regionReplicaSetMap.forEach(
+        (consensusGroupId, regionReplicaSet) -> {
+          Integer leaderId = leaderMap.get(consensusGroupId);
+          TDataNodeLocation leaderDataNodeLocation = null;
+          if (leaderId == null || leaderId == -1) {
+            leaderDataNodeLocation = regionReplicaSet.getDataNodeLocations().get(0);
+          } else {
+            for (TDataNodeLocation dataNodeLocation : regionReplicaSet.getDataNodeLocations()) {
+              if (dataNodeLocation.getDataNodeId() == leaderId) {
+                leaderDataNodeLocation = dataNodeLocation;
+                break;
+              }
+            }
+          }
+          dataNodeConsensusGroupIdMap
+              .computeIfAbsent(leaderDataNodeLocation, k -> new ArrayList<>())
+              .add(regionReplicaSet.getRegionId());
+        });
+    return dataNodeConsensusGroupIdMap;
+  }
+
+  private Map<TDataNodeLocation, List<TConsensusGroupId>> getAllReplicaDataNodeRegionGroupMap(
+      Map<TConsensusGroupId, TRegionReplicaSet> regionReplicaSetMap) {
+    Map<TDataNodeLocation, List<TConsensusGroupId>> dataNodeConsensusGroupIdMap = new HashMap<>();
+    regionReplicaSetMap.forEach(
+        (consensusGroupId, regionReplicaSet) -> {
+          for (TDataNodeLocation dataNodeLocation : regionReplicaSet.getDataNodeLocations()) {
+            dataNodeConsensusGroupIdMap
+                .computeIfAbsent(dataNodeLocation, k -> new ArrayList<>())
+                .add(regionReplicaSet.getRegionId());
+          }
+        });
+    return dataNodeConsensusGroupIdMap;
+  }
+
+  @Override
+  protected void rollbackState(
+      ConfigNodeProcedureEnv env, DeleteTimeSeriesState deleteTimeSeriesState)
+      throws IOException, InterruptedException, ProcedureException {
+    RegionTask<TSStatus> rollbackStateTask =
+        new RegionTask<TSStatus>(
+            "roll back schema black list",
+            env,

Review Comment:
   In which state is the rollback performed?



##########
confignode/src/main/java/org/apache/iotdb/confignode/procedure/impl/DeleteTimeSeriesProcedure.java:
##########
@@ -0,0 +1,611 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.confignode.procedure.impl;
+
+import org.apache.iotdb.common.rpc.thrift.TConsensusGroupId;
+import org.apache.iotdb.common.rpc.thrift.TDataNodeLocation;
+import org.apache.iotdb.common.rpc.thrift.TRegionReplicaSet;
+import org.apache.iotdb.common.rpc.thrift.TSStatus;
+import org.apache.iotdb.commons.path.PartialPath;
+import org.apache.iotdb.confignode.client.async.datanode.AsyncDataNodeClientPool;
+import org.apache.iotdb.confignode.client.async.handlers.ConstructSchemaBlackListHandler;
+import org.apache.iotdb.confignode.client.async.handlers.DeleteDataForDeleteTimeSeriesHandler;
+import org.apache.iotdb.confignode.client.async.handlers.DeleteTimeSeriesHandler;
+import org.apache.iotdb.confignode.client.async.handlers.FetchSchemaBlackLsitHandler;
+import org.apache.iotdb.confignode.client.async.handlers.InvalidateMatchedSchemaCacheHandler;
+import org.apache.iotdb.confignode.client.async.handlers.RollbackSchemaBlackListHandler;
+import org.apache.iotdb.confignode.procedure.StateMachineProcedure;
+import org.apache.iotdb.confignode.procedure.env.ConfigNodeProcedureEnv;
+import org.apache.iotdb.confignode.procedure.exception.ProcedureException;
+import org.apache.iotdb.confignode.procedure.exception.ProcedureSuspendedException;
+import org.apache.iotdb.confignode.procedure.exception.ProcedureYieldException;
+import org.apache.iotdb.confignode.procedure.state.DeleteTimeSeriesState;
+import org.apache.iotdb.confignode.procedure.store.ProcedureFactory;
+import org.apache.iotdb.db.exception.metadata.PathNotExistException;
+import org.apache.iotdb.db.mpp.common.schematree.PathPatternTree;
+import org.apache.iotdb.mpp.rpc.thrift.TConstructSchemaBlackListReq;
+import org.apache.iotdb.mpp.rpc.thrift.TDeleteDataForDeleteTimeSeriesReq;
+import org.apache.iotdb.mpp.rpc.thrift.TDeleteTimeSeriesReq;
+import org.apache.iotdb.mpp.rpc.thrift.TFetchSchemaBlackListReq;
+import org.apache.iotdb.mpp.rpc.thrift.TFetchSchemaBlackListResp;
+import org.apache.iotdb.mpp.rpc.thrift.TInvalidateMatchedSchemaCacheReq;
+import org.apache.iotdb.mpp.rpc.thrift.TRollbackSchemaBlackListReq;
+import org.apache.iotdb.rpc.TSStatusCode;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.ByteArrayOutputStream;
+import java.io.DataOutputStream;
+import java.io.IOException;
+import java.nio.ByteBuffer;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.stream.Collectors;
+
+public class DeleteTimeSeriesProcedure
+    extends StateMachineProcedure<ConfigNodeProcedureEnv, DeleteTimeSeriesState> {
+
+  private static final Logger LOGGER = LoggerFactory.getLogger(DeleteTimeSeriesProcedure.class);
+
+  private PathPatternTree patternTree;
+  private ByteBuffer patternTreeBytes;
+
+  private String requestMessage;
+
+  public DeleteTimeSeriesProcedure() {
+    super();
+  }
+
+  public DeleteTimeSeriesProcedure(PathPatternTree patternTree) {
+    super();
+    setPatternTree(patternTree);
+  }
+
+  @Override
+  protected Flow executeFromState(ConfigNodeProcedureEnv env, DeleteTimeSeriesState state)
+      throws ProcedureSuspendedException, ProcedureYieldException, InterruptedException {
+    switch (state) {
+      case CONSTRUCT_BLACK_LIST:
+        LOGGER.info("Construct schema black list of timeseries {}", requestMessage);
+        if (constructBlackList(env) > 0) {
+          setNextState(DeleteTimeSeriesState.CLEAN_DATANODE_SCHEMA_CACHE);
+          break;
+        } else {
+          setFailure(
+              new ProcedureException(
+                  new PathNotExistException(
+                      patternTree.getAllPathPatterns().stream()
+                          .map(PartialPath::getFullPath)
+                          .collect(Collectors.toList()))));
+          return Flow.NO_MORE_STATE;
+        }
+      case CLEAN_DATANODE_SCHEMA_CACHE:
+        LOGGER.info("Invalidate cache of timeseries {}", requestMessage);
+        invalidateCache(env);
+        break;
+      case DELETE_DATA:
+        LOGGER.info("Delete data of timeseries {}", requestMessage);
+        deleteData(env);
+        break;
+      case DELETE_TIMESERIES:
+        LOGGER.info("Delete timeseries schema of {}", requestMessage);
+        deleteTimeSeries(env);
+        return Flow.NO_MORE_STATE;
+      default:
+        setFailure(new ProcedureException("Unrecognized state " + state.toString()));
+        return Flow.NO_MORE_STATE;
+    }
+    return Flow.HAS_MORE_STATE;
+  }
+
+  private int constructBlackList(ConfigNodeProcedureEnv env) {
+    Map<TConsensusGroupId, TRegionReplicaSet> targetSchemaRegionGroup =
+        env.getConfigManager().getRelatedSchemaRegionGroup(patternTree);
+    if (targetSchemaRegionGroup.isEmpty()) {
+      return 0;
+    }
+    RegionTask<TSStatus> constructBlackListTask =
+        new RegionTask<TSStatus>("construct schema black list", env, targetSchemaRegionGroup) {
+          @Override
+          Map<Integer, TSStatus> sendRequest(
+              TDataNodeLocation dataNodeLocation, List<TConsensusGroupId> consensusGroupIdList) {
+            // construct request and send
+            Map<Integer, TDataNodeLocation> dataNodeLocationMap = new HashMap<>();
+            dataNodeLocationMap.put(dataNodeLocation.getDataNodeId(), dataNodeLocation);
+            ConstructSchemaBlackListHandler handler =
+                new ConstructSchemaBlackListHandler(dataNodeLocationMap);
+            AsyncDataNodeClientPool.getInstance()
+                .sendAsyncRequestToDataNodeWithRetry(
+                    new TConstructSchemaBlackListReq(consensusGroupIdList, patternTreeBytes),
+                    handler);
+            handler
+                .getDataNodeResponseStatusMap()
+                .forEach(
+                    (k, v) -> {
+                      if (v.getCode() == TSStatusCode.SUCCESS_STATUS.getStatusCode()) {
+                        responseMap.put(k, v);
+                      }
+                    });
+            return handler.getDataNodeResponseStatusMap();
+          }
+        };
+    constructBlackListTask.execute();
+
+    if (isFailed()) {
+      return 0;
+    }
+
+    int preDeletedNum = 0;
+    for (TSStatus resp : constructBlackListTask.getResponseMap().values()) {
+      preDeletedNum += Integer.parseInt(resp.getMessage());
+    }
+    return preDeletedNum;
+  }
+
+  private void invalidateCache(ConfigNodeProcedureEnv env) {
+    Map<Integer, TDataNodeLocation> dataNodeLocationMap =
+        env.getConfigManager().getNodeManager().getRegisteredDataNodeLocations();
+    InvalidateMatchedSchemaCacheHandler handler =
+        new InvalidateMatchedSchemaCacheHandler(dataNodeLocationMap);
+    AsyncDataNodeClientPool.getInstance()
+        .sendAsyncRequestToDataNodeWithRetry(
+            new TInvalidateMatchedSchemaCacheReq(patternTreeBytes), handler);
+    Map<Integer, TSStatus> statusMap = handler.getDataNodeResponseStatusMap();
+    for (TSStatus status : statusMap.values()) {
+      // all dataNodes must clear the related schema cache
+      if (status.getCode() != TSStatusCode.SUCCESS_STATUS.getStatusCode()) {
+        LOGGER.error("Failed to invalidate schema cache of timeseries {}", requestMessage);
+        setFailure(new ProcedureException("Invalidate schema cache failed"));
+        return;
+      }
+    }
+
+    setNextState(DeleteTimeSeriesState.DELETE_DATA);
+  }
+
+  private void deleteData(ConfigNodeProcedureEnv env) {
+    Map<TConsensusGroupId, TRegionReplicaSet> relatedSchemaRegionGroup =
+        env.getConfigManager().getRelatedSchemaRegionGroup(patternTree);
+    Map<TDataNodeLocation, List<TConsensusGroupId>> dataNodeSchemaRegionGroupGroupIdMap =
+        getLeaderDataNodeRegionGroupMap(
+            env.getConfigManager().getPartitionManager().getAllLeadership(),
+            relatedSchemaRegionGroup);
+
+    // fetch schema black list by dataNode
+    for (Map.Entry<TDataNodeLocation, List<TConsensusGroupId>> entry :
+        dataNodeSchemaRegionGroupGroupIdMap.entrySet()) {
+      Map<TConsensusGroupId, TRegionReplicaSet> targetSchemaRegionGroup = new HashMap<>();
+      entry
+          .getValue()
+          .forEach(
+              consensusGroupId ->
+                  targetSchemaRegionGroup.put(
+                      consensusGroupId, relatedSchemaRegionGroup.get(consensusGroupId)));
+      PathPatternTree patternTree =
+          fetchSchemaBlackListOnTargetDataNode(env, targetSchemaRegionGroup);
+      if (isFailed()) {
+        return;
+      }
+      if (patternTree == null) {
+        LOGGER.error(
+            "Failed to fetch schema black list for delete data of timeseries {} on {}",
+            requestMessage,
+            entry.getKey());
+        setFailure(new ProcedureException("Fetch schema black list forDelete data failed"));
+        return;
+      }
+
+      if (patternTree.isEmpty()) {
+        continue;
+      }
+
+      Map<TConsensusGroupId, TRegionReplicaSet> relatedDataRegionGroup =
+          env.getConfigManager().getRelatedDataRegionGroup(patternTree);
+
+      // target timeseries has no data
+      if (relatedDataRegionGroup.isEmpty()) {
+        continue;
+      }
+
+      RegionTask<TSStatus> deleteDataTask =
+          new RegionTask<TSStatus>("delete data", env, relatedDataRegionGroup) {
+            @Override
+            Map<Integer, TSStatus> sendRequest(
+                TDataNodeLocation dataNodeLocation, List<TConsensusGroupId> consensusGroupIdList) {
+              Map<Integer, TDataNodeLocation> dataNodeLocationMap = new HashMap<>();
+              dataNodeLocationMap.put(dataNodeLocation.getDataNodeId(), dataNodeLocation);
+              DeleteDataForDeleteTimeSeriesHandler handler =
+                  new DeleteDataForDeleteTimeSeriesHandler(dataNodeLocationMap);
+              AsyncDataNodeClientPool.getInstance()
+                  .sendAsyncRequestToDataNodeWithRetry(
+                      new TDeleteDataForDeleteTimeSeriesReq(
+                          new ArrayList<>(consensusGroupIdList),
+                          preparePatternTreeBytesData(patternTree)),
+                      handler);
+              return handler.getDataNodeResponseStatusMap();
+            }
+          };
+      deleteDataTask.setExecuteOnAllReplicaset(true);
+      deleteDataTask.execute();
+      if (isFailed()) {
+        return;
+      }
+    }
+    setNextState(DeleteTimeSeriesState.DELETE_TIMESERIES);
+  }
+
+  private PathPatternTree fetchSchemaBlackListOnTargetDataNode(
+      ConfigNodeProcedureEnv env,
+      Map<TConsensusGroupId, TRegionReplicaSet> targetSchemaRegionGroup) {
+    RegionTask<TFetchSchemaBlackListResp> fetchSchemaBlackListTask =
+        new RegionTask<TFetchSchemaBlackListResp>(
+            "fetch schema black list", env, targetSchemaRegionGroup) {
+          @Override
+          Map<Integer, TSStatus> sendRequest(
+              TDataNodeLocation dataNodeLocation, List<TConsensusGroupId> consensusGroupIdList) {
+            Map<Integer, TDataNodeLocation> dataNodeLocationMap = new HashMap<>();
+            dataNodeLocationMap.put(dataNodeLocation.getDataNodeId(), dataNodeLocation);
+            FetchSchemaBlackLsitHandler handler =
+                new FetchSchemaBlackLsitHandler(dataNodeLocationMap);
+            AsyncDataNodeClientPool.getInstance()
+                .sendAsyncRequestToDataNodeWithRetry(
+                    new TFetchSchemaBlackListReq(consensusGroupIdList, patternTreeBytes), handler);
+            Map<Integer, TFetchSchemaBlackListResp> respMap = handler.getDataNodeResponseMap();
+            Map<Integer, TSStatus> statusMap = new HashMap<>();
+            respMap.forEach(
+                (k, v) -> {
+                  if (v.getStatus().getCode() == TSStatusCode.SUCCESS_STATUS.getStatusCode()) {
+                    responseMap.put(k, v);
+                  }
+                  statusMap.put(k, v.getStatus());
+                });
+            return statusMap;
+          }
+        };
+    fetchSchemaBlackListTask.execute();
+    if (isFailed()) {
+      return null;
+    }
+
+    Map<Integer, TFetchSchemaBlackListResp> respMap = fetchSchemaBlackListTask.getResponseMap();
+    PathPatternTree patternTree = new PathPatternTree();
+    for (TFetchSchemaBlackListResp resp : respMap.values()) {
+      for (PartialPath path :
+          PathPatternTree.deserialize(ByteBuffer.wrap(resp.getPathPatternTree()))
+              .getAllPathPatterns()) {
+        patternTree.appendPathPattern(path);
+      }
+    }
+    patternTree.constructTree();
+    return patternTree;
+  }
+
+  private void deleteTimeSeries(ConfigNodeProcedureEnv env) {
+    RegionTask<TSStatus> deleteTimeSeriesTask =
+        new RegionTask<TSStatus>(
+            "delete timeseries",
+            env,
+            env.getConfigManager().getRelatedSchemaRegionGroup(patternTree)) {
+          @Override
+          Map<Integer, TSStatus> sendRequest(
+              TDataNodeLocation dataNodeLocation, List<TConsensusGroupId> consensusGroupIdList) {
+            Map<Integer, TDataNodeLocation> dataNodeLocationMap = new HashMap<>();
+            dataNodeLocationMap.put(dataNodeLocation.getDataNodeId(), dataNodeLocation);
+            DeleteTimeSeriesHandler handler = new DeleteTimeSeriesHandler(dataNodeLocationMap);
+            AsyncDataNodeClientPool.getInstance()
+                .sendAsyncRequestToDataNodeWithRetry(
+                    new TDeleteTimeSeriesReq(consensusGroupIdList, patternTreeBytes), handler);
+            return handler.getDataNodeResponseStatusMap();
+          }
+        };
+    deleteTimeSeriesTask.execute();
+  }
+
+  private Map<TDataNodeLocation, List<TConsensusGroupId>> getLeaderDataNodeRegionGroupMap(
+      Map<TConsensusGroupId, Integer> leaderMap,
+      Map<TConsensusGroupId, TRegionReplicaSet> regionReplicaSetMap) {
+    Map<TDataNodeLocation, List<TConsensusGroupId>> dataNodeConsensusGroupIdMap = new HashMap<>();
+    regionReplicaSetMap.forEach(
+        (consensusGroupId, regionReplicaSet) -> {
+          Integer leaderId = leaderMap.get(consensusGroupId);
+          TDataNodeLocation leaderDataNodeLocation = null;
+          if (leaderId == null || leaderId == -1) {
+            leaderDataNodeLocation = regionReplicaSet.getDataNodeLocations().get(0);
+          } else {
+            for (TDataNodeLocation dataNodeLocation : regionReplicaSet.getDataNodeLocations()) {
+              if (dataNodeLocation.getDataNodeId() == leaderId) {
+                leaderDataNodeLocation = dataNodeLocation;
+                break;
+              }
+            }
+          }
+          dataNodeConsensusGroupIdMap
+              .computeIfAbsent(leaderDataNodeLocation, k -> new ArrayList<>())
+              .add(regionReplicaSet.getRegionId());
+        });
+    return dataNodeConsensusGroupIdMap;
+  }
+
+  private Map<TDataNodeLocation, List<TConsensusGroupId>> getAllReplicaDataNodeRegionGroupMap(
+      Map<TConsensusGroupId, TRegionReplicaSet> regionReplicaSetMap) {
+    Map<TDataNodeLocation, List<TConsensusGroupId>> dataNodeConsensusGroupIdMap = new HashMap<>();
+    regionReplicaSetMap.forEach(
+        (consensusGroupId, regionReplicaSet) -> {
+          for (TDataNodeLocation dataNodeLocation : regionReplicaSet.getDataNodeLocations()) {
+            dataNodeConsensusGroupIdMap
+                .computeIfAbsent(dataNodeLocation, k -> new ArrayList<>())
+                .add(regionReplicaSet.getRegionId());
+          }
+        });
+    return dataNodeConsensusGroupIdMap;
+  }
+
+  @Override
+  protected void rollbackState(
+      ConfigNodeProcedureEnv env, DeleteTimeSeriesState deleteTimeSeriesState)
+      throws IOException, InterruptedException, ProcedureException {
+    RegionTask<TSStatus> rollbackStateTask =
+        new RegionTask<TSStatus>(
+            "roll back schema black list",
+            env,
+            env.getConfigManager().getRelatedSchemaRegionGroup(patternTree)) {
+          @Override
+          Map<Integer, TSStatus> sendRequest(
+              TDataNodeLocation dataNodeLocation, List<TConsensusGroupId> consensusGroupIdList) {
+            Map<Integer, TDataNodeLocation> dataNodeLocationMap = new HashMap<>();
+            dataNodeLocationMap.put(dataNodeLocation.getDataNodeId(), dataNodeLocation);
+            RollbackSchemaBlackListHandler handler =
+                new RollbackSchemaBlackListHandler(dataNodeLocationMap);
+            AsyncDataNodeClientPool.getInstance()
+                .sendAsyncRequestToDataNodeWithRetry(
+                    new TRollbackSchemaBlackListReq(consensusGroupIdList, patternTreeBytes),
+                    handler);
+            return handler.getDataNodeResponseStatusMap();
+          }
+        };
+    rollbackStateTask.execute();
+  }
+
+  @Override
+  protected boolean isRollbackSupported(DeleteTimeSeriesState deleteTimeSeriesState) {
+    return true;
+  }
+
+  @Override
+  protected DeleteTimeSeriesState getState(int stateId) {
+    return DeleteTimeSeriesState.values()[stateId];
+  }
+
+  @Override
+  protected int getStateId(DeleteTimeSeriesState deleteTimeSeriesState) {
+    return deleteTimeSeriesState.ordinal();
+  }
+
+  @Override
+  protected DeleteTimeSeriesState getInitialState() {
+    return DeleteTimeSeriesState.CONSTRUCT_BLACK_LIST;
+  }
+
+  public PathPatternTree getPatternTree() {
+    return patternTree;
+  }
+
+  public void setPatternTree(PathPatternTree patternTree) {
+    this.patternTree = patternTree;
+    requestMessage = patternTree.getAllPathPatterns().toString();
+    patternTreeBytes = preparePatternTreeBytesData(patternTree);
+  }
+
+  private ByteBuffer preparePatternTreeBytesData(PathPatternTree patternTree) {
+    ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream();
+    DataOutputStream dataOutputStream = new DataOutputStream(byteArrayOutputStream);
+    try {
+      patternTree.serialize(dataOutputStream);
+    } catch (IOException ignored) {
+
+    }
+    return ByteBuffer.wrap(byteArrayOutputStream.toByteArray());
+  }
+
+  @Override
+  public void serialize(DataOutputStream stream) throws IOException {
+    stream.writeInt(ProcedureFactory.ProcedureType.DELETE_TIMESERIES_PROCEDURE.ordinal());
+    super.serialize(stream);
+    patternTree.serialize(stream);

Review Comment:
   need serialize and deserialize requestMessage, patternTreeBytes .  If the leader breaks down, other follower nodes need to replay and execute the procedure after receiving the procedure. The attributes required by these states may have been initialized.



##########
confignode/src/main/java/org/apache/iotdb/confignode/procedure/impl/DeleteTimeSeriesProcedure.java:
##########
@@ -0,0 +1,611 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.confignode.procedure.impl;
+
+import org.apache.iotdb.common.rpc.thrift.TConsensusGroupId;
+import org.apache.iotdb.common.rpc.thrift.TDataNodeLocation;
+import org.apache.iotdb.common.rpc.thrift.TRegionReplicaSet;
+import org.apache.iotdb.common.rpc.thrift.TSStatus;
+import org.apache.iotdb.commons.path.PartialPath;
+import org.apache.iotdb.confignode.client.async.datanode.AsyncDataNodeClientPool;
+import org.apache.iotdb.confignode.client.async.handlers.ConstructSchemaBlackListHandler;
+import org.apache.iotdb.confignode.client.async.handlers.DeleteDataForDeleteTimeSeriesHandler;
+import org.apache.iotdb.confignode.client.async.handlers.DeleteTimeSeriesHandler;
+import org.apache.iotdb.confignode.client.async.handlers.FetchSchemaBlackLsitHandler;
+import org.apache.iotdb.confignode.client.async.handlers.InvalidateMatchedSchemaCacheHandler;
+import org.apache.iotdb.confignode.client.async.handlers.RollbackSchemaBlackListHandler;
+import org.apache.iotdb.confignode.procedure.StateMachineProcedure;
+import org.apache.iotdb.confignode.procedure.env.ConfigNodeProcedureEnv;
+import org.apache.iotdb.confignode.procedure.exception.ProcedureException;
+import org.apache.iotdb.confignode.procedure.exception.ProcedureSuspendedException;
+import org.apache.iotdb.confignode.procedure.exception.ProcedureYieldException;
+import org.apache.iotdb.confignode.procedure.state.DeleteTimeSeriesState;
+import org.apache.iotdb.confignode.procedure.store.ProcedureFactory;
+import org.apache.iotdb.db.exception.metadata.PathNotExistException;
+import org.apache.iotdb.db.mpp.common.schematree.PathPatternTree;
+import org.apache.iotdb.mpp.rpc.thrift.TConstructSchemaBlackListReq;
+import org.apache.iotdb.mpp.rpc.thrift.TDeleteDataForDeleteTimeSeriesReq;
+import org.apache.iotdb.mpp.rpc.thrift.TDeleteTimeSeriesReq;
+import org.apache.iotdb.mpp.rpc.thrift.TFetchSchemaBlackListReq;
+import org.apache.iotdb.mpp.rpc.thrift.TFetchSchemaBlackListResp;
+import org.apache.iotdb.mpp.rpc.thrift.TInvalidateMatchedSchemaCacheReq;
+import org.apache.iotdb.mpp.rpc.thrift.TRollbackSchemaBlackListReq;
+import org.apache.iotdb.rpc.TSStatusCode;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.ByteArrayOutputStream;
+import java.io.DataOutputStream;
+import java.io.IOException;
+import java.nio.ByteBuffer;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.stream.Collectors;
+
+public class DeleteTimeSeriesProcedure
+    extends StateMachineProcedure<ConfigNodeProcedureEnv, DeleteTimeSeriesState> {
+
+  private static final Logger LOGGER = LoggerFactory.getLogger(DeleteTimeSeriesProcedure.class);
+
+  private PathPatternTree patternTree;
+  private ByteBuffer patternTreeBytes;
+
+  private String requestMessage;
+
+  public DeleteTimeSeriesProcedure() {
+    super();
+  }
+
+  public DeleteTimeSeriesProcedure(PathPatternTree patternTree) {
+    super();
+    setPatternTree(patternTree);
+  }
+
+  @Override
+  protected Flow executeFromState(ConfigNodeProcedureEnv env, DeleteTimeSeriesState state)
+      throws ProcedureSuspendedException, ProcedureYieldException, InterruptedException {
+    switch (state) {
+      case CONSTRUCT_BLACK_LIST:
+        LOGGER.info("Construct schema black list of timeseries {}", requestMessage);
+        if (constructBlackList(env) > 0) {
+          setNextState(DeleteTimeSeriesState.CLEAN_DATANODE_SCHEMA_CACHE);
+          break;
+        } else {
+          setFailure(
+              new ProcedureException(
+                  new PathNotExistException(
+                      patternTree.getAllPathPatterns().stream()
+                          .map(PartialPath::getFullPath)
+                          .collect(Collectors.toList()))));
+          return Flow.NO_MORE_STATE;
+        }
+      case CLEAN_DATANODE_SCHEMA_CACHE:
+        LOGGER.info("Invalidate cache of timeseries {}", requestMessage);
+        invalidateCache(env);
+        break;
+      case DELETE_DATA:
+        LOGGER.info("Delete data of timeseries {}", requestMessage);
+        deleteData(env);
+        break;
+      case DELETE_TIMESERIES:
+        LOGGER.info("Delete timeseries schema of {}", requestMessage);
+        deleteTimeSeries(env);
+        return Flow.NO_MORE_STATE;
+      default:
+        setFailure(new ProcedureException("Unrecognized state " + state.toString()));
+        return Flow.NO_MORE_STATE;
+    }
+    return Flow.HAS_MORE_STATE;
+  }
+
+  private int constructBlackList(ConfigNodeProcedureEnv env) {
+    Map<TConsensusGroupId, TRegionReplicaSet> targetSchemaRegionGroup =
+        env.getConfigManager().getRelatedSchemaRegionGroup(patternTree);
+    if (targetSchemaRegionGroup.isEmpty()) {
+      return 0;
+    }
+    RegionTask<TSStatus> constructBlackListTask =
+        new RegionTask<TSStatus>("construct schema black list", env, targetSchemaRegionGroup) {
+          @Override
+          Map<Integer, TSStatus> sendRequest(
+              TDataNodeLocation dataNodeLocation, List<TConsensusGroupId> consensusGroupIdList) {
+            // construct request and send
+            Map<Integer, TDataNodeLocation> dataNodeLocationMap = new HashMap<>();
+            dataNodeLocationMap.put(dataNodeLocation.getDataNodeId(), dataNodeLocation);
+            ConstructSchemaBlackListHandler handler =
+                new ConstructSchemaBlackListHandler(dataNodeLocationMap);
+            AsyncDataNodeClientPool.getInstance()
+                .sendAsyncRequestToDataNodeWithRetry(
+                    new TConstructSchemaBlackListReq(consensusGroupIdList, patternTreeBytes),
+                    handler);
+            handler
+                .getDataNodeResponseStatusMap()
+                .forEach(
+                    (k, v) -> {
+                      if (v.getCode() == TSStatusCode.SUCCESS_STATUS.getStatusCode()) {
+                        responseMap.put(k, v);
+                      }
+                    });
+            return handler.getDataNodeResponseStatusMap();
+          }
+        };
+    constructBlackListTask.execute();
+
+    if (isFailed()) {
+      return 0;
+    }
+
+    int preDeletedNum = 0;
+    for (TSStatus resp : constructBlackListTask.getResponseMap().values()) {
+      preDeletedNum += Integer.parseInt(resp.getMessage());
+    }
+    return preDeletedNum;
+  }
+
+  private void invalidateCache(ConfigNodeProcedureEnv env) {
+    Map<Integer, TDataNodeLocation> dataNodeLocationMap =
+        env.getConfigManager().getNodeManager().getRegisteredDataNodeLocations();
+    InvalidateMatchedSchemaCacheHandler handler =
+        new InvalidateMatchedSchemaCacheHandler(dataNodeLocationMap);
+    AsyncDataNodeClientPool.getInstance()
+        .sendAsyncRequestToDataNodeWithRetry(
+            new TInvalidateMatchedSchemaCacheReq(patternTreeBytes), handler);
+    Map<Integer, TSStatus> statusMap = handler.getDataNodeResponseStatusMap();
+    for (TSStatus status : statusMap.values()) {
+      // all dataNodes must clear the related schema cache
+      if (status.getCode() != TSStatusCode.SUCCESS_STATUS.getStatusCode()) {
+        LOGGER.error("Failed to invalidate schema cache of timeseries {}", requestMessage);
+        setFailure(new ProcedureException("Invalidate schema cache failed"));
+        return;
+      }
+    }
+
+    setNextState(DeleteTimeSeriesState.DELETE_DATA);
+  }
+
+  private void deleteData(ConfigNodeProcedureEnv env) {
+    Map<TConsensusGroupId, TRegionReplicaSet> relatedSchemaRegionGroup =
+        env.getConfigManager().getRelatedSchemaRegionGroup(patternTree);
+    Map<TDataNodeLocation, List<TConsensusGroupId>> dataNodeSchemaRegionGroupGroupIdMap =
+        getLeaderDataNodeRegionGroupMap(
+            env.getConfigManager().getPartitionManager().getAllLeadership(),
+            relatedSchemaRegionGroup);
+
+    // fetch schema black list by dataNode
+    for (Map.Entry<TDataNodeLocation, List<TConsensusGroupId>> entry :
+        dataNodeSchemaRegionGroupGroupIdMap.entrySet()) {
+      Map<TConsensusGroupId, TRegionReplicaSet> targetSchemaRegionGroup = new HashMap<>();
+      entry
+          .getValue()
+          .forEach(
+              consensusGroupId ->
+                  targetSchemaRegionGroup.put(
+                      consensusGroupId, relatedSchemaRegionGroup.get(consensusGroupId)));
+      PathPatternTree patternTree =
+          fetchSchemaBlackListOnTargetDataNode(env, targetSchemaRegionGroup);
+      if (isFailed()) {
+        return;
+      }
+      if (patternTree == null) {
+        LOGGER.error(
+            "Failed to fetch schema black list for delete data of timeseries {} on {}",
+            requestMessage,
+            entry.getKey());
+        setFailure(new ProcedureException("Fetch schema black list forDelete data failed"));
+        return;
+      }
+
+      if (patternTree.isEmpty()) {
+        continue;
+      }
+
+      Map<TConsensusGroupId, TRegionReplicaSet> relatedDataRegionGroup =
+          env.getConfigManager().getRelatedDataRegionGroup(patternTree);
+
+      // target timeseries has no data
+      if (relatedDataRegionGroup.isEmpty()) {
+        continue;
+      }
+
+      RegionTask<TSStatus> deleteDataTask =
+          new RegionTask<TSStatus>("delete data", env, relatedDataRegionGroup) {
+            @Override
+            Map<Integer, TSStatus> sendRequest(
+                TDataNodeLocation dataNodeLocation, List<TConsensusGroupId> consensusGroupIdList) {
+              Map<Integer, TDataNodeLocation> dataNodeLocationMap = new HashMap<>();
+              dataNodeLocationMap.put(dataNodeLocation.getDataNodeId(), dataNodeLocation);
+              DeleteDataForDeleteTimeSeriesHandler handler =
+                  new DeleteDataForDeleteTimeSeriesHandler(dataNodeLocationMap);
+              AsyncDataNodeClientPool.getInstance()
+                  .sendAsyncRequestToDataNodeWithRetry(
+                      new TDeleteDataForDeleteTimeSeriesReq(
+                          new ArrayList<>(consensusGroupIdList),
+                          preparePatternTreeBytesData(patternTree)),
+                      handler);
+              return handler.getDataNodeResponseStatusMap();
+            }
+          };
+      deleteDataTask.setExecuteOnAllReplicaset(true);
+      deleteDataTask.execute();
+      if (isFailed()) {
+        return;
+      }
+    }
+    setNextState(DeleteTimeSeriesState.DELETE_TIMESERIES);
+  }
+
+  private PathPatternTree fetchSchemaBlackListOnTargetDataNode(
+      ConfigNodeProcedureEnv env,
+      Map<TConsensusGroupId, TRegionReplicaSet> targetSchemaRegionGroup) {
+    RegionTask<TFetchSchemaBlackListResp> fetchSchemaBlackListTask =
+        new RegionTask<TFetchSchemaBlackListResp>(
+            "fetch schema black list", env, targetSchemaRegionGroup) {
+          @Override
+          Map<Integer, TSStatus> sendRequest(
+              TDataNodeLocation dataNodeLocation, List<TConsensusGroupId> consensusGroupIdList) {
+            Map<Integer, TDataNodeLocation> dataNodeLocationMap = new HashMap<>();
+            dataNodeLocationMap.put(dataNodeLocation.getDataNodeId(), dataNodeLocation);
+            FetchSchemaBlackLsitHandler handler =
+                new FetchSchemaBlackLsitHandler(dataNodeLocationMap);
+            AsyncDataNodeClientPool.getInstance()
+                .sendAsyncRequestToDataNodeWithRetry(
+                    new TFetchSchemaBlackListReq(consensusGroupIdList, patternTreeBytes), handler);
+            Map<Integer, TFetchSchemaBlackListResp> respMap = handler.getDataNodeResponseMap();
+            Map<Integer, TSStatus> statusMap = new HashMap<>();
+            respMap.forEach(
+                (k, v) -> {
+                  if (v.getStatus().getCode() == TSStatusCode.SUCCESS_STATUS.getStatusCode()) {
+                    responseMap.put(k, v);
+                  }
+                  statusMap.put(k, v.getStatus());
+                });
+            return statusMap;
+          }
+        };
+    fetchSchemaBlackListTask.execute();
+    if (isFailed()) {
+      return null;
+    }
+
+    Map<Integer, TFetchSchemaBlackListResp> respMap = fetchSchemaBlackListTask.getResponseMap();
+    PathPatternTree patternTree = new PathPatternTree();
+    for (TFetchSchemaBlackListResp resp : respMap.values()) {
+      for (PartialPath path :
+          PathPatternTree.deserialize(ByteBuffer.wrap(resp.getPathPatternTree()))
+              .getAllPathPatterns()) {
+        patternTree.appendPathPattern(path);
+      }
+    }
+    patternTree.constructTree();
+    return patternTree;
+  }
+
+  private void deleteTimeSeries(ConfigNodeProcedureEnv env) {
+    RegionTask<TSStatus> deleteTimeSeriesTask =
+        new RegionTask<TSStatus>(
+            "delete timeseries",
+            env,
+            env.getConfigManager().getRelatedSchemaRegionGroup(patternTree)) {
+          @Override
+          Map<Integer, TSStatus> sendRequest(
+              TDataNodeLocation dataNodeLocation, List<TConsensusGroupId> consensusGroupIdList) {
+            Map<Integer, TDataNodeLocation> dataNodeLocationMap = new HashMap<>();
+            dataNodeLocationMap.put(dataNodeLocation.getDataNodeId(), dataNodeLocation);
+            DeleteTimeSeriesHandler handler = new DeleteTimeSeriesHandler(dataNodeLocationMap);
+            AsyncDataNodeClientPool.getInstance()
+                .sendAsyncRequestToDataNodeWithRetry(
+                    new TDeleteTimeSeriesReq(consensusGroupIdList, patternTreeBytes), handler);
+            return handler.getDataNodeResponseStatusMap();
+          }
+        };
+    deleteTimeSeriesTask.execute();
+  }
+
+  private Map<TDataNodeLocation, List<TConsensusGroupId>> getLeaderDataNodeRegionGroupMap(
+      Map<TConsensusGroupId, Integer> leaderMap,
+      Map<TConsensusGroupId, TRegionReplicaSet> regionReplicaSetMap) {
+    Map<TDataNodeLocation, List<TConsensusGroupId>> dataNodeConsensusGroupIdMap = new HashMap<>();
+    regionReplicaSetMap.forEach(
+        (consensusGroupId, regionReplicaSet) -> {
+          Integer leaderId = leaderMap.get(consensusGroupId);
+          TDataNodeLocation leaderDataNodeLocation = null;
+          if (leaderId == null || leaderId == -1) {
+            leaderDataNodeLocation = regionReplicaSet.getDataNodeLocations().get(0);
+          } else {
+            for (TDataNodeLocation dataNodeLocation : regionReplicaSet.getDataNodeLocations()) {
+              if (dataNodeLocation.getDataNodeId() == leaderId) {
+                leaderDataNodeLocation = dataNodeLocation;
+                break;
+              }
+            }
+          }
+          dataNodeConsensusGroupIdMap
+              .computeIfAbsent(leaderDataNodeLocation, k -> new ArrayList<>())
+              .add(regionReplicaSet.getRegionId());
+        });
+    return dataNodeConsensusGroupIdMap;
+  }
+
+  private Map<TDataNodeLocation, List<TConsensusGroupId>> getAllReplicaDataNodeRegionGroupMap(
+      Map<TConsensusGroupId, TRegionReplicaSet> regionReplicaSetMap) {
+    Map<TDataNodeLocation, List<TConsensusGroupId>> dataNodeConsensusGroupIdMap = new HashMap<>();
+    regionReplicaSetMap.forEach(
+        (consensusGroupId, regionReplicaSet) -> {
+          for (TDataNodeLocation dataNodeLocation : regionReplicaSet.getDataNodeLocations()) {
+            dataNodeConsensusGroupIdMap
+                .computeIfAbsent(dataNodeLocation, k -> new ArrayList<>())
+                .add(regionReplicaSet.getRegionId());
+          }
+        });
+    return dataNodeConsensusGroupIdMap;
+  }
+
+  @Override
+  protected void rollbackState(
+      ConfigNodeProcedureEnv env, DeleteTimeSeriesState deleteTimeSeriesState)
+      throws IOException, InterruptedException, ProcedureException {
+    RegionTask<TSStatus> rollbackStateTask =
+        new RegionTask<TSStatus>(
+            "roll back schema black list",
+            env,
+            env.getConfigManager().getRelatedSchemaRegionGroup(patternTree)) {
+          @Override
+          Map<Integer, TSStatus> sendRequest(
+              TDataNodeLocation dataNodeLocation, List<TConsensusGroupId> consensusGroupIdList) {
+            Map<Integer, TDataNodeLocation> dataNodeLocationMap = new HashMap<>();
+            dataNodeLocationMap.put(dataNodeLocation.getDataNodeId(), dataNodeLocation);
+            RollbackSchemaBlackListHandler handler =
+                new RollbackSchemaBlackListHandler(dataNodeLocationMap);
+            AsyncDataNodeClientPool.getInstance()
+                .sendAsyncRequestToDataNodeWithRetry(
+                    new TRollbackSchemaBlackListReq(consensusGroupIdList, patternTreeBytes),
+                    handler);
+            return handler.getDataNodeResponseStatusMap();
+          }
+        };
+    rollbackStateTask.execute();
+  }
+
+  @Override
+  protected boolean isRollbackSupported(DeleteTimeSeriesState deleteTimeSeriesState) {
+    return true;

Review Comment:
   all status support rollback, will call rollbackState



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [iotdb] MarcosZyk commented on a diff in pull request #7242: [IOTDB-3184] Implement Delete Timeseries for Cluster

Posted by GitBox <gi...@apache.org>.
MarcosZyk commented on code in PR #7242:
URL: https://github.com/apache/iotdb/pull/7242#discussion_r969512593


##########
confignode/src/main/java/org/apache/iotdb/confignode/procedure/impl/DeleteTimeSeriesProcedure.java:
##########
@@ -0,0 +1,611 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.confignode.procedure.impl;
+
+import org.apache.iotdb.common.rpc.thrift.TConsensusGroupId;
+import org.apache.iotdb.common.rpc.thrift.TDataNodeLocation;
+import org.apache.iotdb.common.rpc.thrift.TRegionReplicaSet;
+import org.apache.iotdb.common.rpc.thrift.TSStatus;
+import org.apache.iotdb.commons.path.PartialPath;
+import org.apache.iotdb.confignode.client.async.datanode.AsyncDataNodeClientPool;
+import org.apache.iotdb.confignode.client.async.handlers.ConstructSchemaBlackListHandler;
+import org.apache.iotdb.confignode.client.async.handlers.DeleteDataForDeleteTimeSeriesHandler;
+import org.apache.iotdb.confignode.client.async.handlers.DeleteTimeSeriesHandler;
+import org.apache.iotdb.confignode.client.async.handlers.FetchSchemaBlackLsitHandler;
+import org.apache.iotdb.confignode.client.async.handlers.InvalidateMatchedSchemaCacheHandler;
+import org.apache.iotdb.confignode.client.async.handlers.RollbackSchemaBlackListHandler;
+import org.apache.iotdb.confignode.procedure.StateMachineProcedure;
+import org.apache.iotdb.confignode.procedure.env.ConfigNodeProcedureEnv;
+import org.apache.iotdb.confignode.procedure.exception.ProcedureException;
+import org.apache.iotdb.confignode.procedure.exception.ProcedureSuspendedException;
+import org.apache.iotdb.confignode.procedure.exception.ProcedureYieldException;
+import org.apache.iotdb.confignode.procedure.state.DeleteTimeSeriesState;
+import org.apache.iotdb.confignode.procedure.store.ProcedureFactory;
+import org.apache.iotdb.db.exception.metadata.PathNotExistException;
+import org.apache.iotdb.db.mpp.common.schematree.PathPatternTree;
+import org.apache.iotdb.mpp.rpc.thrift.TConstructSchemaBlackListReq;
+import org.apache.iotdb.mpp.rpc.thrift.TDeleteDataForDeleteTimeSeriesReq;
+import org.apache.iotdb.mpp.rpc.thrift.TDeleteTimeSeriesReq;
+import org.apache.iotdb.mpp.rpc.thrift.TFetchSchemaBlackListReq;
+import org.apache.iotdb.mpp.rpc.thrift.TFetchSchemaBlackListResp;
+import org.apache.iotdb.mpp.rpc.thrift.TInvalidateMatchedSchemaCacheReq;
+import org.apache.iotdb.mpp.rpc.thrift.TRollbackSchemaBlackListReq;
+import org.apache.iotdb.rpc.TSStatusCode;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.ByteArrayOutputStream;
+import java.io.DataOutputStream;
+import java.io.IOException;
+import java.nio.ByteBuffer;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.stream.Collectors;
+
+public class DeleteTimeSeriesProcedure
+    extends StateMachineProcedure<ConfigNodeProcedureEnv, DeleteTimeSeriesState> {
+
+  private static final Logger LOGGER = LoggerFactory.getLogger(DeleteTimeSeriesProcedure.class);
+
+  private PathPatternTree patternTree;
+  private ByteBuffer patternTreeBytes;
+
+  private String requestMessage;
+
+  public DeleteTimeSeriesProcedure() {
+    super();
+  }
+
+  public DeleteTimeSeriesProcedure(PathPatternTree patternTree) {
+    super();
+    setPatternTree(patternTree);
+  }
+
+  @Override
+  protected Flow executeFromState(ConfigNodeProcedureEnv env, DeleteTimeSeriesState state)
+      throws ProcedureSuspendedException, ProcedureYieldException, InterruptedException {
+    switch (state) {
+      case CONSTRUCT_BLACK_LIST:
+        LOGGER.info("Construct schema black list of timeseries {}", requestMessage);
+        if (constructBlackList(env) > 0) {
+          setNextState(DeleteTimeSeriesState.CLEAN_DATANODE_SCHEMA_CACHE);
+          break;
+        } else {
+          setFailure(
+              new ProcedureException(
+                  new PathNotExistException(
+                      patternTree.getAllPathPatterns().stream()
+                          .map(PartialPath::getFullPath)
+                          .collect(Collectors.toList()))));
+          return Flow.NO_MORE_STATE;
+        }
+      case CLEAN_DATANODE_SCHEMA_CACHE:
+        LOGGER.info("Invalidate cache of timeseries {}", requestMessage);
+        invalidateCache(env);
+        break;
+      case DELETE_DATA:
+        LOGGER.info("Delete data of timeseries {}", requestMessage);
+        deleteData(env);
+        break;
+      case DELETE_TIMESERIES:
+        LOGGER.info("Delete timeseries schema of {}", requestMessage);
+        deleteTimeSeries(env);
+        return Flow.NO_MORE_STATE;
+      default:
+        setFailure(new ProcedureException("Unrecognized state " + state.toString()));
+        return Flow.NO_MORE_STATE;
+    }
+    return Flow.HAS_MORE_STATE;
+  }
+
+  private int constructBlackList(ConfigNodeProcedureEnv env) {
+    Map<TConsensusGroupId, TRegionReplicaSet> targetSchemaRegionGroup =
+        env.getConfigManager().getRelatedSchemaRegionGroup(patternTree);
+    if (targetSchemaRegionGroup.isEmpty()) {
+      return 0;
+    }
+    RegionTask<TSStatus> constructBlackListTask =
+        new RegionTask<TSStatus>("construct schema black list", env, targetSchemaRegionGroup) {
+          @Override
+          Map<Integer, TSStatus> sendRequest(
+              TDataNodeLocation dataNodeLocation, List<TConsensusGroupId> consensusGroupIdList) {
+            // construct request and send
+            Map<Integer, TDataNodeLocation> dataNodeLocationMap = new HashMap<>();
+            dataNodeLocationMap.put(dataNodeLocation.getDataNodeId(), dataNodeLocation);
+            ConstructSchemaBlackListHandler handler =
+                new ConstructSchemaBlackListHandler(dataNodeLocationMap);
+            AsyncDataNodeClientPool.getInstance()
+                .sendAsyncRequestToDataNodeWithRetry(
+                    new TConstructSchemaBlackListReq(consensusGroupIdList, patternTreeBytes),
+                    handler);
+            handler
+                .getDataNodeResponseStatusMap()
+                .forEach(
+                    (k, v) -> {
+                      if (v.getCode() == TSStatusCode.SUCCESS_STATUS.getStatusCode()) {
+                        responseMap.put(k, v);
+                      }
+                    });
+            return handler.getDataNodeResponseStatusMap();
+          }
+        };
+    constructBlackListTask.execute();
+
+    if (isFailed()) {
+      return 0;
+    }
+
+    int preDeletedNum = 0;
+    for (TSStatus resp : constructBlackListTask.getResponseMap().values()) {
+      preDeletedNum += Integer.parseInt(resp.getMessage());
+    }
+    return preDeletedNum;
+  }
+
+  private void invalidateCache(ConfigNodeProcedureEnv env) {
+    Map<Integer, TDataNodeLocation> dataNodeLocationMap =
+        env.getConfigManager().getNodeManager().getRegisteredDataNodeLocations();
+    InvalidateMatchedSchemaCacheHandler handler =
+        new InvalidateMatchedSchemaCacheHandler(dataNodeLocationMap);
+    AsyncDataNodeClientPool.getInstance()
+        .sendAsyncRequestToDataNodeWithRetry(
+            new TInvalidateMatchedSchemaCacheReq(patternTreeBytes), handler);
+    Map<Integer, TSStatus> statusMap = handler.getDataNodeResponseStatusMap();
+    for (TSStatus status : statusMap.values()) {
+      // all dataNodes must clear the related schema cache
+      if (status.getCode() != TSStatusCode.SUCCESS_STATUS.getStatusCode()) {
+        LOGGER.error("Failed to invalidate schema cache of timeseries {}", requestMessage);
+        setFailure(new ProcedureException("Invalidate schema cache failed"));
+        return;
+      }
+    }
+
+    setNextState(DeleteTimeSeriesState.DELETE_DATA);
+  }
+
+  private void deleteData(ConfigNodeProcedureEnv env) {
+    Map<TConsensusGroupId, TRegionReplicaSet> relatedSchemaRegionGroup =
+        env.getConfigManager().getRelatedSchemaRegionGroup(patternTree);
+    Map<TDataNodeLocation, List<TConsensusGroupId>> dataNodeSchemaRegionGroupGroupIdMap =
+        getLeaderDataNodeRegionGroupMap(
+            env.getConfigManager().getPartitionManager().getAllLeadership(),
+            relatedSchemaRegionGroup);
+
+    // fetch schema black list by dataNode
+    for (Map.Entry<TDataNodeLocation, List<TConsensusGroupId>> entry :
+        dataNodeSchemaRegionGroupGroupIdMap.entrySet()) {
+      Map<TConsensusGroupId, TRegionReplicaSet> targetSchemaRegionGroup = new HashMap<>();
+      entry
+          .getValue()
+          .forEach(
+              consensusGroupId ->
+                  targetSchemaRegionGroup.put(
+                      consensusGroupId, relatedSchemaRegionGroup.get(consensusGroupId)));
+      PathPatternTree patternTree =
+          fetchSchemaBlackListOnTargetDataNode(env, targetSchemaRegionGroup);
+      if (isFailed()) {
+        return;
+      }
+      if (patternTree == null) {
+        LOGGER.error(
+            "Failed to fetch schema black list for delete data of timeseries {} on {}",
+            requestMessage,
+            entry.getKey());
+        setFailure(new ProcedureException("Fetch schema black list forDelete data failed"));
+        return;
+      }
+
+      if (patternTree.isEmpty()) {
+        continue;
+      }
+
+      Map<TConsensusGroupId, TRegionReplicaSet> relatedDataRegionGroup =
+          env.getConfigManager().getRelatedDataRegionGroup(patternTree);
+
+      // target timeseries has no data
+      if (relatedDataRegionGroup.isEmpty()) {
+        continue;
+      }
+
+      RegionTask<TSStatus> deleteDataTask =
+          new RegionTask<TSStatus>("delete data", env, relatedDataRegionGroup) {
+            @Override
+            Map<Integer, TSStatus> sendRequest(
+                TDataNodeLocation dataNodeLocation, List<TConsensusGroupId> consensusGroupIdList) {
+              Map<Integer, TDataNodeLocation> dataNodeLocationMap = new HashMap<>();
+              dataNodeLocationMap.put(dataNodeLocation.getDataNodeId(), dataNodeLocation);
+              DeleteDataForDeleteTimeSeriesHandler handler =
+                  new DeleteDataForDeleteTimeSeriesHandler(dataNodeLocationMap);
+              AsyncDataNodeClientPool.getInstance()
+                  .sendAsyncRequestToDataNodeWithRetry(
+                      new TDeleteDataForDeleteTimeSeriesReq(
+                          new ArrayList<>(consensusGroupIdList),
+                          preparePatternTreeBytesData(patternTree)),
+                      handler);
+              return handler.getDataNodeResponseStatusMap();
+            }
+          };
+      deleteDataTask.setExecuteOnAllReplicaset(true);
+      deleteDataTask.execute();
+      if (isFailed()) {
+        return;
+      }
+    }
+    setNextState(DeleteTimeSeriesState.DELETE_TIMESERIES);
+  }
+
+  private PathPatternTree fetchSchemaBlackListOnTargetDataNode(
+      ConfigNodeProcedureEnv env,
+      Map<TConsensusGroupId, TRegionReplicaSet> targetSchemaRegionGroup) {
+    RegionTask<TFetchSchemaBlackListResp> fetchSchemaBlackListTask =
+        new RegionTask<TFetchSchemaBlackListResp>(
+            "fetch schema black list", env, targetSchemaRegionGroup) {
+          @Override
+          Map<Integer, TSStatus> sendRequest(
+              TDataNodeLocation dataNodeLocation, List<TConsensusGroupId> consensusGroupIdList) {
+            Map<Integer, TDataNodeLocation> dataNodeLocationMap = new HashMap<>();
+            dataNodeLocationMap.put(dataNodeLocation.getDataNodeId(), dataNodeLocation);
+            FetchSchemaBlackLsitHandler handler =
+                new FetchSchemaBlackLsitHandler(dataNodeLocationMap);
+            AsyncDataNodeClientPool.getInstance()
+                .sendAsyncRequestToDataNodeWithRetry(
+                    new TFetchSchemaBlackListReq(consensusGroupIdList, patternTreeBytes), handler);
+            Map<Integer, TFetchSchemaBlackListResp> respMap = handler.getDataNodeResponseMap();
+            Map<Integer, TSStatus> statusMap = new HashMap<>();
+            respMap.forEach(
+                (k, v) -> {
+                  if (v.getStatus().getCode() == TSStatusCode.SUCCESS_STATUS.getStatusCode()) {
+                    responseMap.put(k, v);
+                  }
+                  statusMap.put(k, v.getStatus());
+                });
+            return statusMap;
+          }
+        };
+    fetchSchemaBlackListTask.execute();
+    if (isFailed()) {
+      return null;
+    }
+
+    Map<Integer, TFetchSchemaBlackListResp> respMap = fetchSchemaBlackListTask.getResponseMap();
+    PathPatternTree patternTree = new PathPatternTree();
+    for (TFetchSchemaBlackListResp resp : respMap.values()) {
+      for (PartialPath path :
+          PathPatternTree.deserialize(ByteBuffer.wrap(resp.getPathPatternTree()))
+              .getAllPathPatterns()) {
+        patternTree.appendPathPattern(path);
+      }
+    }
+    patternTree.constructTree();
+    return patternTree;
+  }
+
+  private void deleteTimeSeries(ConfigNodeProcedureEnv env) {
+    RegionTask<TSStatus> deleteTimeSeriesTask =
+        new RegionTask<TSStatus>(
+            "delete timeseries",
+            env,
+            env.getConfigManager().getRelatedSchemaRegionGroup(patternTree)) {
+          @Override
+          Map<Integer, TSStatus> sendRequest(
+              TDataNodeLocation dataNodeLocation, List<TConsensusGroupId> consensusGroupIdList) {
+            Map<Integer, TDataNodeLocation> dataNodeLocationMap = new HashMap<>();
+            dataNodeLocationMap.put(dataNodeLocation.getDataNodeId(), dataNodeLocation);
+            DeleteTimeSeriesHandler handler = new DeleteTimeSeriesHandler(dataNodeLocationMap);
+            AsyncDataNodeClientPool.getInstance()
+                .sendAsyncRequestToDataNodeWithRetry(
+                    new TDeleteTimeSeriesReq(consensusGroupIdList, patternTreeBytes), handler);
+            return handler.getDataNodeResponseStatusMap();
+          }
+        };
+    deleteTimeSeriesTask.execute();
+  }
+
+  private Map<TDataNodeLocation, List<TConsensusGroupId>> getLeaderDataNodeRegionGroupMap(
+      Map<TConsensusGroupId, Integer> leaderMap,
+      Map<TConsensusGroupId, TRegionReplicaSet> regionReplicaSetMap) {
+    Map<TDataNodeLocation, List<TConsensusGroupId>> dataNodeConsensusGroupIdMap = new HashMap<>();
+    regionReplicaSetMap.forEach(
+        (consensusGroupId, regionReplicaSet) -> {
+          Integer leaderId = leaderMap.get(consensusGroupId);
+          TDataNodeLocation leaderDataNodeLocation = null;
+          if (leaderId == null || leaderId == -1) {
+            leaderDataNodeLocation = regionReplicaSet.getDataNodeLocations().get(0);
+          } else {
+            for (TDataNodeLocation dataNodeLocation : regionReplicaSet.getDataNodeLocations()) {
+              if (dataNodeLocation.getDataNodeId() == leaderId) {
+                leaderDataNodeLocation = dataNodeLocation;
+                break;
+              }
+            }
+          }
+          dataNodeConsensusGroupIdMap
+              .computeIfAbsent(leaderDataNodeLocation, k -> new ArrayList<>())
+              .add(regionReplicaSet.getRegionId());
+        });
+    return dataNodeConsensusGroupIdMap;
+  }
+
+  private Map<TDataNodeLocation, List<TConsensusGroupId>> getAllReplicaDataNodeRegionGroupMap(
+      Map<TConsensusGroupId, TRegionReplicaSet> regionReplicaSetMap) {
+    Map<TDataNodeLocation, List<TConsensusGroupId>> dataNodeConsensusGroupIdMap = new HashMap<>();
+    regionReplicaSetMap.forEach(
+        (consensusGroupId, regionReplicaSet) -> {
+          for (TDataNodeLocation dataNodeLocation : regionReplicaSet.getDataNodeLocations()) {
+            dataNodeConsensusGroupIdMap
+                .computeIfAbsent(dataNodeLocation, k -> new ArrayList<>())
+                .add(regionReplicaSet.getRegionId());
+          }
+        });
+    return dataNodeConsensusGroupIdMap;
+  }
+
+  @Override
+  protected void rollbackState(
+      ConfigNodeProcedureEnv env, DeleteTimeSeriesState deleteTimeSeriesState)
+      throws IOException, InterruptedException, ProcedureException {
+    RegionTask<TSStatus> rollbackStateTask =
+        new RegionTask<TSStatus>(
+            "roll back schema black list",
+            env,
+            env.getConfigManager().getRelatedSchemaRegionGroup(patternTree)) {
+          @Override
+          Map<Integer, TSStatus> sendRequest(
+              TDataNodeLocation dataNodeLocation, List<TConsensusGroupId> consensusGroupIdList) {
+            Map<Integer, TDataNodeLocation> dataNodeLocationMap = new HashMap<>();
+            dataNodeLocationMap.put(dataNodeLocation.getDataNodeId(), dataNodeLocation);
+            RollbackSchemaBlackListHandler handler =
+                new RollbackSchemaBlackListHandler(dataNodeLocationMap);
+            AsyncDataNodeClientPool.getInstance()
+                .sendAsyncRequestToDataNodeWithRetry(
+                    new TRollbackSchemaBlackListReq(consensusGroupIdList, patternTreeBytes),
+                    handler);
+            return handler.getDataNodeResponseStatusMap();
+          }
+        };
+    rollbackStateTask.execute();
+  }
+
+  @Override
+  protected boolean isRollbackSupported(DeleteTimeSeriesState deleteTimeSeriesState) {
+    return true;
+  }
+
+  @Override
+  protected DeleteTimeSeriesState getState(int stateId) {
+    return DeleteTimeSeriesState.values()[stateId];
+  }
+
+  @Override
+  protected int getStateId(DeleteTimeSeriesState deleteTimeSeriesState) {
+    return deleteTimeSeriesState.ordinal();
+  }
+
+  @Override
+  protected DeleteTimeSeriesState getInitialState() {
+    return DeleteTimeSeriesState.CONSTRUCT_BLACK_LIST;
+  }
+
+  public PathPatternTree getPatternTree() {
+    return patternTree;
+  }
+
+  public void setPatternTree(PathPatternTree patternTree) {
+    this.patternTree = patternTree;
+    requestMessage = patternTree.getAllPathPatterns().toString();
+    patternTreeBytes = preparePatternTreeBytesData(patternTree);
+  }
+
+  private ByteBuffer preparePatternTreeBytesData(PathPatternTree patternTree) {
+    ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream();
+    DataOutputStream dataOutputStream = new DataOutputStream(byteArrayOutputStream);
+    try {
+      patternTree.serialize(dataOutputStream);
+    } catch (IOException ignored) {
+
+    }
+    return ByteBuffer.wrap(byteArrayOutputStream.toByteArray());
+  }
+
+  @Override
+  public void serialize(DataOutputStream stream) throws IOException {
+    stream.writeInt(ProcedureFactory.ProcedureType.DELETE_TIMESERIES_PROCEDURE.ordinal());
+    super.serialize(stream);
+    patternTree.serialize(stream);
+  }
+
+  @Override
+  public void deserialize(ByteBuffer byteBuffer) {
+    super.deserialize(byteBuffer);
+    setPatternTree(PathPatternTree.deserialize(byteBuffer));
+  }
+
+  @Override
+  public boolean equals(Object o) {
+    if (this == o) return true;
+    if (o == null || getClass() != o.getClass()) return false;
+    DeleteTimeSeriesProcedure that = (DeleteTimeSeriesProcedure) o;
+    return this.getProcId() == that.getProcId()
+        && this.getState() == that.getState()
+        && patternTree.equals(that.patternTree);
+  }
+
+  @Override
+  public int hashCode() {
+    return Objects.hash(getProcId(), getState(), patternTree);
+  }
+
+  private abstract class RegionTask<T> {
+
+    private final String taskName;
+    private final ConfigNodeProcedureEnv env;
+    private final Map<TConsensusGroupId, TRegionReplicaSet> targetSchemaRegionGroup;
+
+    private boolean executeOnAllReplicaset = false;
+
+    protected Map<Integer, T> responseMap = new ConcurrentHashMap<>();
+
+    RegionTask(
+        String taskName,
+        ConfigNodeProcedureEnv env,
+        Map<TConsensusGroupId, TRegionReplicaSet> targetSchemaRegionGroup) {
+      this.taskName = taskName;
+      this.env = env;
+      this.targetSchemaRegionGroup = targetSchemaRegionGroup;
+    }
+
+    private void execute() {
+      // organize schema region by dataNode
+      Set<TDataNodeLocation> allFailedDataNodeSet = new HashSet<>();
+      Map<TDataNodeLocation, List<TConsensusGroupId>> dataNodeConsensusGroupIdMap =
+          executeOnAllReplicaset
+              ? getAllReplicaDataNodeRegionGroupMap(targetSchemaRegionGroup)
+              : getLeaderDataNodeRegionGroupMap(
+                  env.getConfigManager().getPartitionManager().getAllLeadership(),
+                  targetSchemaRegionGroup);
+      while (!dataNodeConsensusGroupIdMap.isEmpty()) {
+        Map<TDataNodeLocation, List<TConsensusGroupId>> currentFailedDataNodeMap =
+            sendSchemaRegionRequest(dataNodeConsensusGroupIdMap);
+        // some dataNode execution failure
+        if (isFailed()) {
+          return;
+        }
+
+        if (currentFailedDataNodeMap.isEmpty()) {
+          break;
+        }
+
+        // retry failed dataNode requests caused by unexpected error on other replicates on other
+        // dataNodes
+        currentFailedDataNodeMap.forEach(dataNodeConsensusGroupIdMap::remove);
+        // remove dataNodes that successfully executed request
+        allFailedDataNodeSet.removeAll(dataNodeConsensusGroupIdMap.keySet());
+        dataNodeConsensusGroupIdMap =
+            getAvailableDataNodeLocationForRetry(currentFailedDataNodeMap, allFailedDataNodeSet);
+        // some consensus group has no available dataNode
+        if (isFailed()) {
+          return;
+        }
+      }
+    }
+
+    private Map<TDataNodeLocation, List<TConsensusGroupId>> sendSchemaRegionRequest(
+        Map<TDataNodeLocation, List<TConsensusGroupId>> dataNodeConsensusGroupIdMap) {
+      // send request to each dataNode
+      Map<TDataNodeLocation, List<TConsensusGroupId>> failedDataNodeMap = new HashMap<>();
+      for (Map.Entry<TDataNodeLocation, List<TConsensusGroupId>> entry :
+          dataNodeConsensusGroupIdMap.entrySet()) {
+        // process response
+        Map<Integer, TSStatus> dataNodeResponseMap = sendRequest(entry.getKey(), entry.getValue());
+        TSStatus currentDataNodeResponse = dataNodeResponseMap.get(entry.getKey().getDataNodeId());
+        if (currentDataNodeResponse.getCode() != TSStatusCode.SUCCESS_STATUS.getStatusCode()) {
+          if (currentDataNodeResponse.getCode() == TSStatusCode.MULTIPLE_ERROR.getStatusCode()) {
+            // dataNode execution error
+            LOGGER.error(
+                "Failed to execute [{}] of delete timeseries {} on {}",
+                taskName,
+                requestMessage,
+                entry.getKey());
+            setFailure(
+                new ProcedureException(
+                    String.format(
+                        "Delete timeseries %s failed on when [%s]", requestMessage, taskName)));
+            break;
+          } else {
+            // unexpected error, retry on other replicates on other dataNodes
+            failedDataNodeMap.put(entry.getKey(), entry.getValue());
+          }
+        }
+      }
+      return failedDataNodeMap;
+    }
+
+    private Map<TDataNodeLocation, List<TConsensusGroupId>> getAvailableDataNodeLocationForRetry(
+        Map<TDataNodeLocation, List<TConsensusGroupId>> failedDataNodeConsensusGroupIdMap,
+        Set<TDataNodeLocation> allFailedDataNodeSet) {
+      Map<TConsensusGroupId, Integer> leaderMap =
+          env.getConfigManager().getPartitionManager().getAllLeadership();
+      Map<TDataNodeLocation, List<TConsensusGroupId>> availableDataNodeLocation = new HashMap<>();
+      for (List<TConsensusGroupId> consensusGroupIdList :
+          failedDataNodeConsensusGroupIdMap.values()) {
+        for (TConsensusGroupId consensusGroupId : consensusGroupIdList) {
+          TRegionReplicaSet regionReplicaSet = targetSchemaRegionGroup.get(consensusGroupId);
+          TDataNodeLocation selectedDataNode = null;
+          Integer leaderId = leaderMap.get(consensusGroupId);
+          if (leaderId == null || leaderId == -1) {
+            for (TDataNodeLocation candidateDataNode : regionReplicaSet.getDataNodeLocations()) {
+              if (!allFailedDataNodeSet.contains(candidateDataNode)) {
+                if (selectedDataNode == null) {
+                  selectedDataNode = candidateDataNode;
+                }
+              }

Review Comment:
   Fixed.



##########
confignode/src/main/java/org/apache/iotdb/confignode/procedure/impl/DeleteTimeSeriesProcedure.java:
##########
@@ -0,0 +1,611 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.confignode.procedure.impl;
+
+import org.apache.iotdb.common.rpc.thrift.TConsensusGroupId;
+import org.apache.iotdb.common.rpc.thrift.TDataNodeLocation;
+import org.apache.iotdb.common.rpc.thrift.TRegionReplicaSet;
+import org.apache.iotdb.common.rpc.thrift.TSStatus;
+import org.apache.iotdb.commons.path.PartialPath;
+import org.apache.iotdb.confignode.client.async.datanode.AsyncDataNodeClientPool;
+import org.apache.iotdb.confignode.client.async.handlers.ConstructSchemaBlackListHandler;
+import org.apache.iotdb.confignode.client.async.handlers.DeleteDataForDeleteTimeSeriesHandler;
+import org.apache.iotdb.confignode.client.async.handlers.DeleteTimeSeriesHandler;
+import org.apache.iotdb.confignode.client.async.handlers.FetchSchemaBlackLsitHandler;
+import org.apache.iotdb.confignode.client.async.handlers.InvalidateMatchedSchemaCacheHandler;
+import org.apache.iotdb.confignode.client.async.handlers.RollbackSchemaBlackListHandler;
+import org.apache.iotdb.confignode.procedure.StateMachineProcedure;
+import org.apache.iotdb.confignode.procedure.env.ConfigNodeProcedureEnv;
+import org.apache.iotdb.confignode.procedure.exception.ProcedureException;
+import org.apache.iotdb.confignode.procedure.exception.ProcedureSuspendedException;
+import org.apache.iotdb.confignode.procedure.exception.ProcedureYieldException;
+import org.apache.iotdb.confignode.procedure.state.DeleteTimeSeriesState;
+import org.apache.iotdb.confignode.procedure.store.ProcedureFactory;
+import org.apache.iotdb.db.exception.metadata.PathNotExistException;
+import org.apache.iotdb.db.mpp.common.schematree.PathPatternTree;
+import org.apache.iotdb.mpp.rpc.thrift.TConstructSchemaBlackListReq;
+import org.apache.iotdb.mpp.rpc.thrift.TDeleteDataForDeleteTimeSeriesReq;
+import org.apache.iotdb.mpp.rpc.thrift.TDeleteTimeSeriesReq;
+import org.apache.iotdb.mpp.rpc.thrift.TFetchSchemaBlackListReq;
+import org.apache.iotdb.mpp.rpc.thrift.TFetchSchemaBlackListResp;
+import org.apache.iotdb.mpp.rpc.thrift.TInvalidateMatchedSchemaCacheReq;
+import org.apache.iotdb.mpp.rpc.thrift.TRollbackSchemaBlackListReq;
+import org.apache.iotdb.rpc.TSStatusCode;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.ByteArrayOutputStream;
+import java.io.DataOutputStream;
+import java.io.IOException;
+import java.nio.ByteBuffer;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.stream.Collectors;
+
+public class DeleteTimeSeriesProcedure
+    extends StateMachineProcedure<ConfigNodeProcedureEnv, DeleteTimeSeriesState> {
+
+  private static final Logger LOGGER = LoggerFactory.getLogger(DeleteTimeSeriesProcedure.class);
+
+  private PathPatternTree patternTree;
+  private ByteBuffer patternTreeBytes;
+
+  private String requestMessage;
+
+  public DeleteTimeSeriesProcedure() {
+    super();
+  }
+
+  public DeleteTimeSeriesProcedure(PathPatternTree patternTree) {
+    super();
+    setPatternTree(patternTree);
+  }
+
+  @Override
+  protected Flow executeFromState(ConfigNodeProcedureEnv env, DeleteTimeSeriesState state)
+      throws ProcedureSuspendedException, ProcedureYieldException, InterruptedException {
+    switch (state) {
+      case CONSTRUCT_BLACK_LIST:
+        LOGGER.info("Construct schema black list of timeseries {}", requestMessage);
+        if (constructBlackList(env) > 0) {
+          setNextState(DeleteTimeSeriesState.CLEAN_DATANODE_SCHEMA_CACHE);
+          break;
+        } else {
+          setFailure(
+              new ProcedureException(
+                  new PathNotExistException(
+                      patternTree.getAllPathPatterns().stream()
+                          .map(PartialPath::getFullPath)
+                          .collect(Collectors.toList()))));
+          return Flow.NO_MORE_STATE;
+        }
+      case CLEAN_DATANODE_SCHEMA_CACHE:
+        LOGGER.info("Invalidate cache of timeseries {}", requestMessage);
+        invalidateCache(env);
+        break;
+      case DELETE_DATA:
+        LOGGER.info("Delete data of timeseries {}", requestMessage);
+        deleteData(env);
+        break;
+      case DELETE_TIMESERIES:
+        LOGGER.info("Delete timeseries schema of {}", requestMessage);
+        deleteTimeSeries(env);
+        return Flow.NO_MORE_STATE;
+      default:
+        setFailure(new ProcedureException("Unrecognized state " + state.toString()));
+        return Flow.NO_MORE_STATE;
+    }
+    return Flow.HAS_MORE_STATE;
+  }
+
+  private int constructBlackList(ConfigNodeProcedureEnv env) {
+    Map<TConsensusGroupId, TRegionReplicaSet> targetSchemaRegionGroup =
+        env.getConfigManager().getRelatedSchemaRegionGroup(patternTree);
+    if (targetSchemaRegionGroup.isEmpty()) {
+      return 0;
+    }
+    RegionTask<TSStatus> constructBlackListTask =
+        new RegionTask<TSStatus>("construct schema black list", env, targetSchemaRegionGroup) {
+          @Override
+          Map<Integer, TSStatus> sendRequest(
+              TDataNodeLocation dataNodeLocation, List<TConsensusGroupId> consensusGroupIdList) {
+            // construct request and send
+            Map<Integer, TDataNodeLocation> dataNodeLocationMap = new HashMap<>();
+            dataNodeLocationMap.put(dataNodeLocation.getDataNodeId(), dataNodeLocation);
+            ConstructSchemaBlackListHandler handler =
+                new ConstructSchemaBlackListHandler(dataNodeLocationMap);
+            AsyncDataNodeClientPool.getInstance()
+                .sendAsyncRequestToDataNodeWithRetry(
+                    new TConstructSchemaBlackListReq(consensusGroupIdList, patternTreeBytes),
+                    handler);
+            handler
+                .getDataNodeResponseStatusMap()
+                .forEach(
+                    (k, v) -> {
+                      if (v.getCode() == TSStatusCode.SUCCESS_STATUS.getStatusCode()) {
+                        responseMap.put(k, v);
+                      }
+                    });
+            return handler.getDataNodeResponseStatusMap();
+          }
+        };
+    constructBlackListTask.execute();
+
+    if (isFailed()) {
+      return 0;
+    }
+
+    int preDeletedNum = 0;
+    for (TSStatus resp : constructBlackListTask.getResponseMap().values()) {
+      preDeletedNum += Integer.parseInt(resp.getMessage());
+    }
+    return preDeletedNum;
+  }
+
+  private void invalidateCache(ConfigNodeProcedureEnv env) {
+    Map<Integer, TDataNodeLocation> dataNodeLocationMap =
+        env.getConfigManager().getNodeManager().getRegisteredDataNodeLocations();
+    InvalidateMatchedSchemaCacheHandler handler =
+        new InvalidateMatchedSchemaCacheHandler(dataNodeLocationMap);
+    AsyncDataNodeClientPool.getInstance()
+        .sendAsyncRequestToDataNodeWithRetry(
+            new TInvalidateMatchedSchemaCacheReq(patternTreeBytes), handler);
+    Map<Integer, TSStatus> statusMap = handler.getDataNodeResponseStatusMap();
+    for (TSStatus status : statusMap.values()) {
+      // all dataNodes must clear the related schema cache
+      if (status.getCode() != TSStatusCode.SUCCESS_STATUS.getStatusCode()) {
+        LOGGER.error("Failed to invalidate schema cache of timeseries {}", requestMessage);
+        setFailure(new ProcedureException("Invalidate schema cache failed"));
+        return;
+      }
+    }
+
+    setNextState(DeleteTimeSeriesState.DELETE_DATA);
+  }
+
+  private void deleteData(ConfigNodeProcedureEnv env) {
+    Map<TConsensusGroupId, TRegionReplicaSet> relatedSchemaRegionGroup =
+        env.getConfigManager().getRelatedSchemaRegionGroup(patternTree);
+    Map<TDataNodeLocation, List<TConsensusGroupId>> dataNodeSchemaRegionGroupGroupIdMap =
+        getLeaderDataNodeRegionGroupMap(
+            env.getConfigManager().getPartitionManager().getAllLeadership(),
+            relatedSchemaRegionGroup);
+
+    // fetch schema black list by dataNode
+    for (Map.Entry<TDataNodeLocation, List<TConsensusGroupId>> entry :
+        dataNodeSchemaRegionGroupGroupIdMap.entrySet()) {
+      Map<TConsensusGroupId, TRegionReplicaSet> targetSchemaRegionGroup = new HashMap<>();
+      entry
+          .getValue()
+          .forEach(
+              consensusGroupId ->
+                  targetSchemaRegionGroup.put(
+                      consensusGroupId, relatedSchemaRegionGroup.get(consensusGroupId)));
+      PathPatternTree patternTree =
+          fetchSchemaBlackListOnTargetDataNode(env, targetSchemaRegionGroup);
+      if (isFailed()) {
+        return;
+      }
+      if (patternTree == null) {
+        LOGGER.error(
+            "Failed to fetch schema black list for delete data of timeseries {} on {}",
+            requestMessage,
+            entry.getKey());
+        setFailure(new ProcedureException("Fetch schema black list forDelete data failed"));
+        return;
+      }
+
+      if (patternTree.isEmpty()) {
+        continue;
+      }
+
+      Map<TConsensusGroupId, TRegionReplicaSet> relatedDataRegionGroup =
+          env.getConfigManager().getRelatedDataRegionGroup(patternTree);
+
+      // target timeseries has no data
+      if (relatedDataRegionGroup.isEmpty()) {
+        continue;
+      }
+
+      RegionTask<TSStatus> deleteDataTask =
+          new RegionTask<TSStatus>("delete data", env, relatedDataRegionGroup) {
+            @Override
+            Map<Integer, TSStatus> sendRequest(
+                TDataNodeLocation dataNodeLocation, List<TConsensusGroupId> consensusGroupIdList) {
+              Map<Integer, TDataNodeLocation> dataNodeLocationMap = new HashMap<>();
+              dataNodeLocationMap.put(dataNodeLocation.getDataNodeId(), dataNodeLocation);
+              DeleteDataForDeleteTimeSeriesHandler handler =
+                  new DeleteDataForDeleteTimeSeriesHandler(dataNodeLocationMap);
+              AsyncDataNodeClientPool.getInstance()
+                  .sendAsyncRequestToDataNodeWithRetry(
+                      new TDeleteDataForDeleteTimeSeriesReq(
+                          new ArrayList<>(consensusGroupIdList),
+                          preparePatternTreeBytesData(patternTree)),
+                      handler);
+              return handler.getDataNodeResponseStatusMap();
+            }
+          };
+      deleteDataTask.setExecuteOnAllReplicaset(true);
+      deleteDataTask.execute();
+      if (isFailed()) {
+        return;
+      }
+    }
+    setNextState(DeleteTimeSeriesState.DELETE_TIMESERIES);
+  }
+
+  private PathPatternTree fetchSchemaBlackListOnTargetDataNode(
+      ConfigNodeProcedureEnv env,
+      Map<TConsensusGroupId, TRegionReplicaSet> targetSchemaRegionGroup) {
+    RegionTask<TFetchSchemaBlackListResp> fetchSchemaBlackListTask =
+        new RegionTask<TFetchSchemaBlackListResp>(
+            "fetch schema black list", env, targetSchemaRegionGroup) {
+          @Override
+          Map<Integer, TSStatus> sendRequest(
+              TDataNodeLocation dataNodeLocation, List<TConsensusGroupId> consensusGroupIdList) {
+            Map<Integer, TDataNodeLocation> dataNodeLocationMap = new HashMap<>();
+            dataNodeLocationMap.put(dataNodeLocation.getDataNodeId(), dataNodeLocation);
+            FetchSchemaBlackLsitHandler handler =
+                new FetchSchemaBlackLsitHandler(dataNodeLocationMap);
+            AsyncDataNodeClientPool.getInstance()
+                .sendAsyncRequestToDataNodeWithRetry(
+                    new TFetchSchemaBlackListReq(consensusGroupIdList, patternTreeBytes), handler);
+            Map<Integer, TFetchSchemaBlackListResp> respMap = handler.getDataNodeResponseMap();
+            Map<Integer, TSStatus> statusMap = new HashMap<>();
+            respMap.forEach(
+                (k, v) -> {
+                  if (v.getStatus().getCode() == TSStatusCode.SUCCESS_STATUS.getStatusCode()) {
+                    responseMap.put(k, v);
+                  }
+                  statusMap.put(k, v.getStatus());
+                });
+            return statusMap;
+          }
+        };
+    fetchSchemaBlackListTask.execute();
+    if (isFailed()) {
+      return null;
+    }
+
+    Map<Integer, TFetchSchemaBlackListResp> respMap = fetchSchemaBlackListTask.getResponseMap();
+    PathPatternTree patternTree = new PathPatternTree();
+    for (TFetchSchemaBlackListResp resp : respMap.values()) {
+      for (PartialPath path :
+          PathPatternTree.deserialize(ByteBuffer.wrap(resp.getPathPatternTree()))
+              .getAllPathPatterns()) {
+        patternTree.appendPathPattern(path);
+      }
+    }
+    patternTree.constructTree();
+    return patternTree;
+  }
+
+  private void deleteTimeSeries(ConfigNodeProcedureEnv env) {
+    RegionTask<TSStatus> deleteTimeSeriesTask =
+        new RegionTask<TSStatus>(
+            "delete timeseries",
+            env,
+            env.getConfigManager().getRelatedSchemaRegionGroup(patternTree)) {
+          @Override
+          Map<Integer, TSStatus> sendRequest(
+              TDataNodeLocation dataNodeLocation, List<TConsensusGroupId> consensusGroupIdList) {
+            Map<Integer, TDataNodeLocation> dataNodeLocationMap = new HashMap<>();
+            dataNodeLocationMap.put(dataNodeLocation.getDataNodeId(), dataNodeLocation);
+            DeleteTimeSeriesHandler handler = new DeleteTimeSeriesHandler(dataNodeLocationMap);
+            AsyncDataNodeClientPool.getInstance()
+                .sendAsyncRequestToDataNodeWithRetry(
+                    new TDeleteTimeSeriesReq(consensusGroupIdList, patternTreeBytes), handler);
+            return handler.getDataNodeResponseStatusMap();
+          }
+        };
+    deleteTimeSeriesTask.execute();
+  }
+
+  private Map<TDataNodeLocation, List<TConsensusGroupId>> getLeaderDataNodeRegionGroupMap(
+      Map<TConsensusGroupId, Integer> leaderMap,
+      Map<TConsensusGroupId, TRegionReplicaSet> regionReplicaSetMap) {
+    Map<TDataNodeLocation, List<TConsensusGroupId>> dataNodeConsensusGroupIdMap = new HashMap<>();
+    regionReplicaSetMap.forEach(
+        (consensusGroupId, regionReplicaSet) -> {
+          Integer leaderId = leaderMap.get(consensusGroupId);
+          TDataNodeLocation leaderDataNodeLocation = null;
+          if (leaderId == null || leaderId == -1) {
+            leaderDataNodeLocation = regionReplicaSet.getDataNodeLocations().get(0);
+          } else {
+            for (TDataNodeLocation dataNodeLocation : regionReplicaSet.getDataNodeLocations()) {
+              if (dataNodeLocation.getDataNodeId() == leaderId) {
+                leaderDataNodeLocation = dataNodeLocation;
+                break;
+              }
+            }
+          }
+          dataNodeConsensusGroupIdMap
+              .computeIfAbsent(leaderDataNodeLocation, k -> new ArrayList<>())
+              .add(regionReplicaSet.getRegionId());
+        });
+    return dataNodeConsensusGroupIdMap;
+  }
+
+  private Map<TDataNodeLocation, List<TConsensusGroupId>> getAllReplicaDataNodeRegionGroupMap(
+      Map<TConsensusGroupId, TRegionReplicaSet> regionReplicaSetMap) {
+    Map<TDataNodeLocation, List<TConsensusGroupId>> dataNodeConsensusGroupIdMap = new HashMap<>();
+    regionReplicaSetMap.forEach(
+        (consensusGroupId, regionReplicaSet) -> {
+          for (TDataNodeLocation dataNodeLocation : regionReplicaSet.getDataNodeLocations()) {
+            dataNodeConsensusGroupIdMap
+                .computeIfAbsent(dataNodeLocation, k -> new ArrayList<>())
+                .add(regionReplicaSet.getRegionId());
+          }
+        });
+    return dataNodeConsensusGroupIdMap;
+  }
+
+  @Override
+  protected void rollbackState(
+      ConfigNodeProcedureEnv env, DeleteTimeSeriesState deleteTimeSeriesState)
+      throws IOException, InterruptedException, ProcedureException {
+    RegionTask<TSStatus> rollbackStateTask =
+        new RegionTask<TSStatus>(
+            "roll back schema black list",
+            env,
+            env.getConfigManager().getRelatedSchemaRegionGroup(patternTree)) {
+          @Override
+          Map<Integer, TSStatus> sendRequest(
+              TDataNodeLocation dataNodeLocation, List<TConsensusGroupId> consensusGroupIdList) {
+            Map<Integer, TDataNodeLocation> dataNodeLocationMap = new HashMap<>();
+            dataNodeLocationMap.put(dataNodeLocation.getDataNodeId(), dataNodeLocation);
+            RollbackSchemaBlackListHandler handler =
+                new RollbackSchemaBlackListHandler(dataNodeLocationMap);
+            AsyncDataNodeClientPool.getInstance()
+                .sendAsyncRequestToDataNodeWithRetry(
+                    new TRollbackSchemaBlackListReq(consensusGroupIdList, patternTreeBytes),
+                    handler);
+            return handler.getDataNodeResponseStatusMap();
+          }
+        };
+    rollbackStateTask.execute();
+  }
+
+  @Override
+  protected boolean isRollbackSupported(DeleteTimeSeriesState deleteTimeSeriesState) {
+    return true;
+  }
+
+  @Override
+  protected DeleteTimeSeriesState getState(int stateId) {
+    return DeleteTimeSeriesState.values()[stateId];
+  }
+
+  @Override
+  protected int getStateId(DeleteTimeSeriesState deleteTimeSeriesState) {
+    return deleteTimeSeriesState.ordinal();
+  }
+
+  @Override
+  protected DeleteTimeSeriesState getInitialState() {
+    return DeleteTimeSeriesState.CONSTRUCT_BLACK_LIST;
+  }
+
+  public PathPatternTree getPatternTree() {
+    return patternTree;
+  }
+
+  public void setPatternTree(PathPatternTree patternTree) {
+    this.patternTree = patternTree;
+    requestMessage = patternTree.getAllPathPatterns().toString();
+    patternTreeBytes = preparePatternTreeBytesData(patternTree);
+  }
+
+  private ByteBuffer preparePatternTreeBytesData(PathPatternTree patternTree) {
+    ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream();
+    DataOutputStream dataOutputStream = new DataOutputStream(byteArrayOutputStream);
+    try {
+      patternTree.serialize(dataOutputStream);
+    } catch (IOException ignored) {
+
+    }
+    return ByteBuffer.wrap(byteArrayOutputStream.toByteArray());
+  }
+
+  @Override
+  public void serialize(DataOutputStream stream) throws IOException {
+    stream.writeInt(ProcedureFactory.ProcedureType.DELETE_TIMESERIES_PROCEDURE.ordinal());
+    super.serialize(stream);
+    patternTree.serialize(stream);
+  }
+
+  @Override
+  public void deserialize(ByteBuffer byteBuffer) {
+    super.deserialize(byteBuffer);
+    setPatternTree(PathPatternTree.deserialize(byteBuffer));
+  }
+
+  @Override
+  public boolean equals(Object o) {
+    if (this == o) return true;
+    if (o == null || getClass() != o.getClass()) return false;
+    DeleteTimeSeriesProcedure that = (DeleteTimeSeriesProcedure) o;
+    return this.getProcId() == that.getProcId()
+        && this.getState() == that.getState()
+        && patternTree.equals(that.patternTree);
+  }
+
+  @Override
+  public int hashCode() {
+    return Objects.hash(getProcId(), getState(), patternTree);
+  }
+
+  private abstract class RegionTask<T> {
+
+    private final String taskName;
+    private final ConfigNodeProcedureEnv env;
+    private final Map<TConsensusGroupId, TRegionReplicaSet> targetSchemaRegionGroup;
+
+    private boolean executeOnAllReplicaset = false;
+
+    protected Map<Integer, T> responseMap = new ConcurrentHashMap<>();
+
+    RegionTask(
+        String taskName,
+        ConfigNodeProcedureEnv env,
+        Map<TConsensusGroupId, TRegionReplicaSet> targetSchemaRegionGroup) {
+      this.taskName = taskName;
+      this.env = env;
+      this.targetSchemaRegionGroup = targetSchemaRegionGroup;
+    }
+
+    private void execute() {
+      // organize schema region by dataNode
+      Set<TDataNodeLocation> allFailedDataNodeSet = new HashSet<>();
+      Map<TDataNodeLocation, List<TConsensusGroupId>> dataNodeConsensusGroupIdMap =
+          executeOnAllReplicaset
+              ? getAllReplicaDataNodeRegionGroupMap(targetSchemaRegionGroup)
+              : getLeaderDataNodeRegionGroupMap(
+                  env.getConfigManager().getPartitionManager().getAllLeadership(),
+                  targetSchemaRegionGroup);
+      while (!dataNodeConsensusGroupIdMap.isEmpty()) {
+        Map<TDataNodeLocation, List<TConsensusGroupId>> currentFailedDataNodeMap =
+            sendSchemaRegionRequest(dataNodeConsensusGroupIdMap);
+        // some dataNode execution failure
+        if (isFailed()) {
+          return;
+        }
+
+        if (currentFailedDataNodeMap.isEmpty()) {
+          break;
+        }
+
+        // retry failed dataNode requests caused by unexpected error on other replicates on other
+        // dataNodes
+        currentFailedDataNodeMap.forEach(dataNodeConsensusGroupIdMap::remove);
+        // remove dataNodes that successfully executed request
+        allFailedDataNodeSet.removeAll(dataNodeConsensusGroupIdMap.keySet());
+        dataNodeConsensusGroupIdMap =
+            getAvailableDataNodeLocationForRetry(currentFailedDataNodeMap, allFailedDataNodeSet);
+        // some consensus group has no available dataNode
+        if (isFailed()) {
+          return;
+        }
+      }
+    }
+
+    private Map<TDataNodeLocation, List<TConsensusGroupId>> sendSchemaRegionRequest(
+        Map<TDataNodeLocation, List<TConsensusGroupId>> dataNodeConsensusGroupIdMap) {
+      // send request to each dataNode
+      Map<TDataNodeLocation, List<TConsensusGroupId>> failedDataNodeMap = new HashMap<>();
+      for (Map.Entry<TDataNodeLocation, List<TConsensusGroupId>> entry :
+          dataNodeConsensusGroupIdMap.entrySet()) {
+        // process response
+        Map<Integer, TSStatus> dataNodeResponseMap = sendRequest(entry.getKey(), entry.getValue());
+        TSStatus currentDataNodeResponse = dataNodeResponseMap.get(entry.getKey().getDataNodeId());
+        if (currentDataNodeResponse.getCode() != TSStatusCode.SUCCESS_STATUS.getStatusCode()) {
+          if (currentDataNodeResponse.getCode() == TSStatusCode.MULTIPLE_ERROR.getStatusCode()) {
+            // dataNode execution error
+            LOGGER.error(
+                "Failed to execute [{}] of delete timeseries {} on {}",
+                taskName,
+                requestMessage,
+                entry.getKey());
+            setFailure(
+                new ProcedureException(
+                    String.format(
+                        "Delete timeseries %s failed on when [%s]", requestMessage, taskName)));
+            break;
+          } else {
+            // unexpected error, retry on other replicates on other dataNodes
+            failedDataNodeMap.put(entry.getKey(), entry.getValue());
+          }
+        }
+      }
+      return failedDataNodeMap;
+    }
+
+    private Map<TDataNodeLocation, List<TConsensusGroupId>> getAvailableDataNodeLocationForRetry(
+        Map<TDataNodeLocation, List<TConsensusGroupId>> failedDataNodeConsensusGroupIdMap,
+        Set<TDataNodeLocation> allFailedDataNodeSet) {
+      Map<TConsensusGroupId, Integer> leaderMap =
+          env.getConfigManager().getPartitionManager().getAllLeadership();
+      Map<TDataNodeLocation, List<TConsensusGroupId>> availableDataNodeLocation = new HashMap<>();
+      for (List<TConsensusGroupId> consensusGroupIdList :
+          failedDataNodeConsensusGroupIdMap.values()) {
+        for (TConsensusGroupId consensusGroupId : consensusGroupIdList) {
+          TRegionReplicaSet regionReplicaSet = targetSchemaRegionGroup.get(consensusGroupId);
+          TDataNodeLocation selectedDataNode = null;
+          Integer leaderId = leaderMap.get(consensusGroupId);
+          if (leaderId == null || leaderId == -1) {

Review Comment:
   Added.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [iotdb] xingtanzjr commented on a diff in pull request #7242: [IOTDB-3184] Implement Delete Timeseries for Cluster

Posted by GitBox <gi...@apache.org>.
xingtanzjr commented on code in PR #7242:
URL: https://github.com/apache/iotdb/pull/7242#discussion_r969322573


##########
confignode/src/main/java/org/apache/iotdb/confignode/client/async/datanode/AsyncDataNodeClientPool.java:
##########
@@ -183,6 +224,31 @@ public void sendAsyncRequestToDataNodeWithRetry(
     }
   }
 
+  public void sendAsyncRequestToDataNodeWithRetry(Object req, AbstractRetryHandler handler) {
+    Map<Integer, TDataNodeLocation> dataNodeLocationMap = handler.getDataNodeLocationMap();
+    if (dataNodeLocationMap.isEmpty()) {
+      return;
+    }
+    for (int retry = 0; retry < MAX_RETRY_NUM; retry++) {
+      CountDownLatch countDownLatch = new CountDownLatch(dataNodeLocationMap.size());
+      handler.setCountDownLatch(countDownLatch);
+      for (TDataNodeLocation targetDataNode : dataNodeLocationMap.values()) {
+        handler.setTargetDataNode(targetDataNode);
+        sendAsyncRequestToDataNode(targetDataNode, req, handler, retry);

Review Comment:
   There is concurrent issue here. 
   
   The `targetDataNode` may be changed before every request. When one request's `onComplete` is invoked asynchronously, its `targetDataNode` may not be the real one because it may be changed by following request 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [iotdb] MarcosZyk commented on a diff in pull request #7242: [IOTDB-3184] Implement Delete Timeseries for Cluster

Posted by GitBox <gi...@apache.org>.
MarcosZyk commented on code in PR #7242:
URL: https://github.com/apache/iotdb/pull/7242#discussion_r970206088


##########
confignode/src/main/java/org/apache/iotdb/confignode/manager/ConfigManager.java:
##########
@@ -966,4 +969,80 @@ public TGetPathsSetTemplatesResp getPathsSetTemplate(String req) {
       return new TGetPathsSetTemplatesResp(status);
     }
   }
+
+  @Override
+  public TSStatus deleteTimeSeries(PathPatternTree req) {
+    TSStatus status = confirmLeader();
+    if (status.getCode() == TSStatusCode.SUCCESS_STATUS.getStatusCode()) {
+      return procedureManager.deleteTimeSeries(req);
+    } else {
+      return status;
+    }
+  }
+
+  /** Get all related schemaRegion which may contains the timeseries matched by given patternTree */
+  public Map<TConsensusGroupId, TRegionReplicaSet> getRelatedSchemaRegionGroup(
+      PathPatternTree patternTree) {
+    Map<String, Map<TSeriesPartitionSlot, TConsensusGroupId>> schemaPartitionTable =
+        getSchemaPartition(patternTree).getSchemaPartitionTable();
+
+    List<TRegionReplicaSet> allRegionReplicaSets = getPartitionManager().getAllReplicaSets();
+    Set<TConsensusGroupId> groupIdSet =
+        schemaPartitionTable.values().stream()
+            .flatMap(m -> m.values().stream())
+            .collect(Collectors.toSet());
+    Map<TConsensusGroupId, TRegionReplicaSet> filteredRegionReplicaSets = new HashMap<>();
+    for (TRegionReplicaSet regionReplicaSet : allRegionReplicaSets) {
+      if (groupIdSet.contains(regionReplicaSet.getRegionId())) {
+        filteredRegionReplicaSets.put(regionReplicaSet.getRegionId(), regionReplicaSet);
+      }
+    }
+    return filteredRegionReplicaSets;
+  }
+
+  /**
+   * Get all related dataRegion which may contains the data of specific timeseries matched by given
+   * patternTree
+   */
+  public Map<TConsensusGroupId, TRegionReplicaSet> getRelatedDataRegionGroup(
+      PathPatternTree patternTree) {
+    // get all storage group and slot by getting schema partition
+    Map<String, Map<TSeriesPartitionSlot, TConsensusGroupId>> schemaPartitionTable =
+        getSchemaPartition(patternTree).getSchemaPartitionTable();
+
+    // construct request for getting data partition
+    Map<String, Map<TSeriesPartitionSlot, List<TTimePartitionSlot>>> partitionSlotsMap =
+        new HashMap<>();
+    schemaPartitionTable.forEach(
+        (key, value) -> {
+          Map<TSeriesPartitionSlot, List<TTimePartitionSlot>> slotListMap = new HashMap<>();
+          value.keySet().forEach(slot -> slotListMap.put(slot, Collections.emptyList()));
+          partitionSlotsMap.put(key, slotListMap);
+        });
+
+    // get all data partitions
+    GetDataPartitionPlan getDataPartitionPlan = new GetDataPartitionPlan(partitionSlotsMap);
+    Map<String, Map<TSeriesPartitionSlot, Map<TTimePartitionSlot, List<TConsensusGroupId>>>>
+        dataPartitionTable = getDataPartition(getDataPartitionPlan).getDataPartitionTable();
+
+    // get all region replicaset of target data partitions
+    List<TRegionReplicaSet> allRegionReplicaSets = getPartitionManager().getAllReplicaSets();
+    Set<TConsensusGroupId> groupIdSet =
+        dataPartitionTable.values().stream()
+            .flatMap(
+                tSeriesPartitionSlotMapMap ->
+                    tSeriesPartitionSlotMapMap.values().stream()
+                        .flatMap(
+                            tTimePartitionSlotListMap ->
+                                tTimePartitionSlotListMap.values().stream()
+                                    .flatMap(Collection::stream)))
+            .collect(Collectors.toSet());
+    Map<TConsensusGroupId, TRegionReplicaSet> filteredRegionReplicaSets = new HashMap<>();
+    for (TRegionReplicaSet regionReplicaSet : allRegionReplicaSets) {
+      if (groupIdSet.contains(regionReplicaSet.getRegionId())) {
+        filteredRegionReplicaSets.put(regionReplicaSet.getRegionId(), regionReplicaSet);
+      }
+    }
+    return filteredRegionReplicaSets;
+  }

Review Comment:
   These two functions are implemented across different manager: ```ClusterSchemaManager``` and ```PartitionManager```.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [iotdb] MarcosZyk merged pull request #7242: [IOTDB-3184] Implement Delete Timeseries for Cluster

Posted by GitBox <gi...@apache.org>.
MarcosZyk merged PR #7242:
URL: https://github.com/apache/iotdb/pull/7242


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [iotdb] MarcosZyk commented on a diff in pull request #7242: [IOTDB-3184] Implement Delete Timeseries for Cluster

Posted by GitBox <gi...@apache.org>.
MarcosZyk commented on code in PR #7242:
URL: https://github.com/apache/iotdb/pull/7242#discussion_r969071717


##########
confignode/src/main/java/org/apache/iotdb/confignode/procedure/impl/DeleteTimeSeriesProcedure.java:
##########
@@ -0,0 +1,611 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.confignode.procedure.impl;
+
+import org.apache.iotdb.common.rpc.thrift.TConsensusGroupId;
+import org.apache.iotdb.common.rpc.thrift.TDataNodeLocation;
+import org.apache.iotdb.common.rpc.thrift.TRegionReplicaSet;
+import org.apache.iotdb.common.rpc.thrift.TSStatus;
+import org.apache.iotdb.commons.path.PartialPath;
+import org.apache.iotdb.confignode.client.async.datanode.AsyncDataNodeClientPool;
+import org.apache.iotdb.confignode.client.async.handlers.ConstructSchemaBlackListHandler;
+import org.apache.iotdb.confignode.client.async.handlers.DeleteDataForDeleteTimeSeriesHandler;
+import org.apache.iotdb.confignode.client.async.handlers.DeleteTimeSeriesHandler;
+import org.apache.iotdb.confignode.client.async.handlers.FetchSchemaBlackLsitHandler;
+import org.apache.iotdb.confignode.client.async.handlers.InvalidateMatchedSchemaCacheHandler;
+import org.apache.iotdb.confignode.client.async.handlers.RollbackSchemaBlackListHandler;
+import org.apache.iotdb.confignode.procedure.StateMachineProcedure;
+import org.apache.iotdb.confignode.procedure.env.ConfigNodeProcedureEnv;
+import org.apache.iotdb.confignode.procedure.exception.ProcedureException;
+import org.apache.iotdb.confignode.procedure.exception.ProcedureSuspendedException;
+import org.apache.iotdb.confignode.procedure.exception.ProcedureYieldException;
+import org.apache.iotdb.confignode.procedure.state.DeleteTimeSeriesState;
+import org.apache.iotdb.confignode.procedure.store.ProcedureFactory;
+import org.apache.iotdb.db.exception.metadata.PathNotExistException;
+import org.apache.iotdb.db.mpp.common.schematree.PathPatternTree;
+import org.apache.iotdb.mpp.rpc.thrift.TConstructSchemaBlackListReq;
+import org.apache.iotdb.mpp.rpc.thrift.TDeleteDataForDeleteTimeSeriesReq;
+import org.apache.iotdb.mpp.rpc.thrift.TDeleteTimeSeriesReq;
+import org.apache.iotdb.mpp.rpc.thrift.TFetchSchemaBlackListReq;
+import org.apache.iotdb.mpp.rpc.thrift.TFetchSchemaBlackListResp;
+import org.apache.iotdb.mpp.rpc.thrift.TInvalidateMatchedSchemaCacheReq;
+import org.apache.iotdb.mpp.rpc.thrift.TRollbackSchemaBlackListReq;
+import org.apache.iotdb.rpc.TSStatusCode;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.ByteArrayOutputStream;
+import java.io.DataOutputStream;
+import java.io.IOException;
+import java.nio.ByteBuffer;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.stream.Collectors;
+
+public class DeleteTimeSeriesProcedure
+    extends StateMachineProcedure<ConfigNodeProcedureEnv, DeleteTimeSeriesState> {
+
+  private static final Logger LOGGER = LoggerFactory.getLogger(DeleteTimeSeriesProcedure.class);
+
+  private PathPatternTree patternTree;
+  private ByteBuffer patternTreeBytes;
+
+  private String requestMessage;
+
+  public DeleteTimeSeriesProcedure() {
+    super();
+  }
+
+  public DeleteTimeSeriesProcedure(PathPatternTree patternTree) {
+    super();
+    setPatternTree(patternTree);
+  }
+
+  @Override
+  protected Flow executeFromState(ConfigNodeProcedureEnv env, DeleteTimeSeriesState state)
+      throws ProcedureSuspendedException, ProcedureYieldException, InterruptedException {
+    switch (state) {
+      case CONSTRUCT_BLACK_LIST:
+        LOGGER.info("Construct schema black list of timeseries {}", requestMessage);
+        if (constructBlackList(env) > 0) {
+          setNextState(DeleteTimeSeriesState.CLEAN_DATANODE_SCHEMA_CACHE);
+          break;
+        } else {
+          setFailure(
+              new ProcedureException(
+                  new PathNotExistException(
+                      patternTree.getAllPathPatterns().stream()
+                          .map(PartialPath::getFullPath)
+                          .collect(Collectors.toList()))));
+          return Flow.NO_MORE_STATE;
+        }
+      case CLEAN_DATANODE_SCHEMA_CACHE:
+        LOGGER.info("Invalidate cache of timeseries {}", requestMessage);
+        invalidateCache(env);
+        break;
+      case DELETE_DATA:
+        LOGGER.info("Delete data of timeseries {}", requestMessage);
+        deleteData(env);
+        break;
+      case DELETE_TIMESERIES:
+        LOGGER.info("Delete timeseries schema of {}", requestMessage);
+        deleteTimeSeries(env);
+        return Flow.NO_MORE_STATE;
+      default:
+        setFailure(new ProcedureException("Unrecognized state " + state.toString()));
+        return Flow.NO_MORE_STATE;
+    }
+    return Flow.HAS_MORE_STATE;
+  }
+
+  private int constructBlackList(ConfigNodeProcedureEnv env) {
+    Map<TConsensusGroupId, TRegionReplicaSet> targetSchemaRegionGroup =
+        env.getConfigManager().getRelatedSchemaRegionGroup(patternTree);
+    if (targetSchemaRegionGroup.isEmpty()) {
+      return 0;
+    }
+    RegionTask<TSStatus> constructBlackListTask =
+        new RegionTask<TSStatus>("construct schema black list", env, targetSchemaRegionGroup) {
+          @Override
+          Map<Integer, TSStatus> sendRequest(
+              TDataNodeLocation dataNodeLocation, List<TConsensusGroupId> consensusGroupIdList) {
+            // construct request and send
+            Map<Integer, TDataNodeLocation> dataNodeLocationMap = new HashMap<>();
+            dataNodeLocationMap.put(dataNodeLocation.getDataNodeId(), dataNodeLocation);
+            ConstructSchemaBlackListHandler handler =
+                new ConstructSchemaBlackListHandler(dataNodeLocationMap);
+            AsyncDataNodeClientPool.getInstance()
+                .sendAsyncRequestToDataNodeWithRetry(
+                    new TConstructSchemaBlackListReq(consensusGroupIdList, patternTreeBytes),
+                    handler);
+            handler
+                .getDataNodeResponseStatusMap()
+                .forEach(
+                    (k, v) -> {
+                      if (v.getCode() == TSStatusCode.SUCCESS_STATUS.getStatusCode()) {
+                        responseMap.put(k, v);
+                      }
+                    });
+            return handler.getDataNodeResponseStatusMap();
+          }
+        };
+    constructBlackListTask.execute();
+
+    if (isFailed()) {
+      return 0;
+    }
+
+    int preDeletedNum = 0;
+    for (TSStatus resp : constructBlackListTask.getResponseMap().values()) {
+      preDeletedNum += Integer.parseInt(resp.getMessage());
+    }
+    return preDeletedNum;
+  }
+
+  private void invalidateCache(ConfigNodeProcedureEnv env) {
+    Map<Integer, TDataNodeLocation> dataNodeLocationMap =
+        env.getConfigManager().getNodeManager().getRegisteredDataNodeLocations();
+    InvalidateMatchedSchemaCacheHandler handler =
+        new InvalidateMatchedSchemaCacheHandler(dataNodeLocationMap);
+    AsyncDataNodeClientPool.getInstance()
+        .sendAsyncRequestToDataNodeWithRetry(
+            new TInvalidateMatchedSchemaCacheReq(patternTreeBytes), handler);
+    Map<Integer, TSStatus> statusMap = handler.getDataNodeResponseStatusMap();
+    for (TSStatus status : statusMap.values()) {
+      // all dataNodes must clear the related schema cache
+      if (status.getCode() != TSStatusCode.SUCCESS_STATUS.getStatusCode()) {
+        LOGGER.error("Failed to invalidate schema cache of timeseries {}", requestMessage);
+        setFailure(new ProcedureException("Invalidate schema cache failed"));
+        return;
+      }
+    }
+
+    setNextState(DeleteTimeSeriesState.DELETE_DATA);
+  }
+
+  private void deleteData(ConfigNodeProcedureEnv env) {
+    Map<TConsensusGroupId, TRegionReplicaSet> relatedSchemaRegionGroup =
+        env.getConfigManager().getRelatedSchemaRegionGroup(patternTree);
+    Map<TDataNodeLocation, List<TConsensusGroupId>> dataNodeSchemaRegionGroupGroupIdMap =
+        getLeaderDataNodeRegionGroupMap(
+            env.getConfigManager().getPartitionManager().getAllLeadership(),
+            relatedSchemaRegionGroup);
+
+    // fetch schema black list by dataNode
+    for (Map.Entry<TDataNodeLocation, List<TConsensusGroupId>> entry :
+        dataNodeSchemaRegionGroupGroupIdMap.entrySet()) {
+      Map<TConsensusGroupId, TRegionReplicaSet> targetSchemaRegionGroup = new HashMap<>();
+      entry
+          .getValue()
+          .forEach(
+              consensusGroupId ->
+                  targetSchemaRegionGroup.put(
+                      consensusGroupId, relatedSchemaRegionGroup.get(consensusGroupId)));
+      PathPatternTree patternTree =
+          fetchSchemaBlackListOnTargetDataNode(env, targetSchemaRegionGroup);
+      if (isFailed()) {
+        return;
+      }
+      if (patternTree == null) {
+        LOGGER.error(
+            "Failed to fetch schema black list for delete data of timeseries {} on {}",
+            requestMessage,
+            entry.getKey());
+        setFailure(new ProcedureException("Fetch schema black list forDelete data failed"));
+        return;
+      }
+
+      if (patternTree.isEmpty()) {
+        continue;
+      }
+
+      Map<TConsensusGroupId, TRegionReplicaSet> relatedDataRegionGroup =
+          env.getConfigManager().getRelatedDataRegionGroup(patternTree);
+
+      // target timeseries has no data
+      if (relatedDataRegionGroup.isEmpty()) {
+        continue;
+      }
+
+      RegionTask<TSStatus> deleteDataTask =
+          new RegionTask<TSStatus>("delete data", env, relatedDataRegionGroup) {
+            @Override
+            Map<Integer, TSStatus> sendRequest(
+                TDataNodeLocation dataNodeLocation, List<TConsensusGroupId> consensusGroupIdList) {
+              Map<Integer, TDataNodeLocation> dataNodeLocationMap = new HashMap<>();
+              dataNodeLocationMap.put(dataNodeLocation.getDataNodeId(), dataNodeLocation);
+              DeleteDataForDeleteTimeSeriesHandler handler =
+                  new DeleteDataForDeleteTimeSeriesHandler(dataNodeLocationMap);
+              AsyncDataNodeClientPool.getInstance()
+                  .sendAsyncRequestToDataNodeWithRetry(
+                      new TDeleteDataForDeleteTimeSeriesReq(
+                          new ArrayList<>(consensusGroupIdList),
+                          preparePatternTreeBytesData(patternTree)),
+                      handler);
+              return handler.getDataNodeResponseStatusMap();
+            }
+          };
+      deleteDataTask.setExecuteOnAllReplicaset(true);
+      deleteDataTask.execute();
+      if (isFailed()) {
+        return;
+      }
+    }
+    setNextState(DeleteTimeSeriesState.DELETE_TIMESERIES);
+  }
+
+  private PathPatternTree fetchSchemaBlackListOnTargetDataNode(
+      ConfigNodeProcedureEnv env,
+      Map<TConsensusGroupId, TRegionReplicaSet> targetSchemaRegionGroup) {
+    RegionTask<TFetchSchemaBlackListResp> fetchSchemaBlackListTask =
+        new RegionTask<TFetchSchemaBlackListResp>(
+            "fetch schema black list", env, targetSchemaRegionGroup) {
+          @Override
+          Map<Integer, TSStatus> sendRequest(
+              TDataNodeLocation dataNodeLocation, List<TConsensusGroupId> consensusGroupIdList) {
+            Map<Integer, TDataNodeLocation> dataNodeLocationMap = new HashMap<>();
+            dataNodeLocationMap.put(dataNodeLocation.getDataNodeId(), dataNodeLocation);
+            FetchSchemaBlackLsitHandler handler =
+                new FetchSchemaBlackLsitHandler(dataNodeLocationMap);
+            AsyncDataNodeClientPool.getInstance()
+                .sendAsyncRequestToDataNodeWithRetry(
+                    new TFetchSchemaBlackListReq(consensusGroupIdList, patternTreeBytes), handler);
+            Map<Integer, TFetchSchemaBlackListResp> respMap = handler.getDataNodeResponseMap();
+            Map<Integer, TSStatus> statusMap = new HashMap<>();
+            respMap.forEach(
+                (k, v) -> {
+                  if (v.getStatus().getCode() == TSStatusCode.SUCCESS_STATUS.getStatusCode()) {
+                    responseMap.put(k, v);
+                  }
+                  statusMap.put(k, v.getStatus());
+                });
+            return statusMap;
+          }
+        };
+    fetchSchemaBlackListTask.execute();
+    if (isFailed()) {
+      return null;
+    }
+
+    Map<Integer, TFetchSchemaBlackListResp> respMap = fetchSchemaBlackListTask.getResponseMap();
+    PathPatternTree patternTree = new PathPatternTree();
+    for (TFetchSchemaBlackListResp resp : respMap.values()) {
+      for (PartialPath path :
+          PathPatternTree.deserialize(ByteBuffer.wrap(resp.getPathPatternTree()))
+              .getAllPathPatterns()) {
+        patternTree.appendPathPattern(path);
+      }
+    }
+    patternTree.constructTree();
+    return patternTree;
+  }
+
+  private void deleteTimeSeries(ConfigNodeProcedureEnv env) {
+    RegionTask<TSStatus> deleteTimeSeriesTask =
+        new RegionTask<TSStatus>(
+            "delete timeseries",
+            env,
+            env.getConfigManager().getRelatedSchemaRegionGroup(patternTree)) {
+          @Override
+          Map<Integer, TSStatus> sendRequest(
+              TDataNodeLocation dataNodeLocation, List<TConsensusGroupId> consensusGroupIdList) {
+            Map<Integer, TDataNodeLocation> dataNodeLocationMap = new HashMap<>();
+            dataNodeLocationMap.put(dataNodeLocation.getDataNodeId(), dataNodeLocation);
+            DeleteTimeSeriesHandler handler = new DeleteTimeSeriesHandler(dataNodeLocationMap);
+            AsyncDataNodeClientPool.getInstance()
+                .sendAsyncRequestToDataNodeWithRetry(
+                    new TDeleteTimeSeriesReq(consensusGroupIdList, patternTreeBytes), handler);
+            return handler.getDataNodeResponseStatusMap();
+          }
+        };
+    deleteTimeSeriesTask.execute();
+  }
+
+  private Map<TDataNodeLocation, List<TConsensusGroupId>> getLeaderDataNodeRegionGroupMap(
+      Map<TConsensusGroupId, Integer> leaderMap,
+      Map<TConsensusGroupId, TRegionReplicaSet> regionReplicaSetMap) {
+    Map<TDataNodeLocation, List<TConsensusGroupId>> dataNodeConsensusGroupIdMap = new HashMap<>();
+    regionReplicaSetMap.forEach(
+        (consensusGroupId, regionReplicaSet) -> {
+          Integer leaderId = leaderMap.get(consensusGroupId);
+          TDataNodeLocation leaderDataNodeLocation = null;
+          if (leaderId == null || leaderId == -1) {
+            leaderDataNodeLocation = regionReplicaSet.getDataNodeLocations().get(0);
+          } else {
+            for (TDataNodeLocation dataNodeLocation : regionReplicaSet.getDataNodeLocations()) {
+              if (dataNodeLocation.getDataNodeId() == leaderId) {
+                leaderDataNodeLocation = dataNodeLocation;
+                break;
+              }
+            }
+          }
+          dataNodeConsensusGroupIdMap
+              .computeIfAbsent(leaderDataNodeLocation, k -> new ArrayList<>())
+              .add(regionReplicaSet.getRegionId());
+        });
+    return dataNodeConsensusGroupIdMap;
+  }
+
+  private Map<TDataNodeLocation, List<TConsensusGroupId>> getAllReplicaDataNodeRegionGroupMap(
+      Map<TConsensusGroupId, TRegionReplicaSet> regionReplicaSetMap) {
+    Map<TDataNodeLocation, List<TConsensusGroupId>> dataNodeConsensusGroupIdMap = new HashMap<>();
+    regionReplicaSetMap.forEach(
+        (consensusGroupId, regionReplicaSet) -> {
+          for (TDataNodeLocation dataNodeLocation : regionReplicaSet.getDataNodeLocations()) {
+            dataNodeConsensusGroupIdMap
+                .computeIfAbsent(dataNodeLocation, k -> new ArrayList<>())
+                .add(regionReplicaSet.getRegionId());
+          }
+        });
+    return dataNodeConsensusGroupIdMap;
+  }
+
+  @Override
+  protected void rollbackState(
+      ConfigNodeProcedureEnv env, DeleteTimeSeriesState deleteTimeSeriesState)
+      throws IOException, InterruptedException, ProcedureException {
+    RegionTask<TSStatus> rollbackStateTask =
+        new RegionTask<TSStatus>(
+            "roll back schema black list",
+            env,
+            env.getConfigManager().getRelatedSchemaRegionGroup(patternTree)) {
+          @Override
+          Map<Integer, TSStatus> sendRequest(
+              TDataNodeLocation dataNodeLocation, List<TConsensusGroupId> consensusGroupIdList) {
+            Map<Integer, TDataNodeLocation> dataNodeLocationMap = new HashMap<>();
+            dataNodeLocationMap.put(dataNodeLocation.getDataNodeId(), dataNodeLocation);
+            RollbackSchemaBlackListHandler handler =
+                new RollbackSchemaBlackListHandler(dataNodeLocationMap);
+            AsyncDataNodeClientPool.getInstance()
+                .sendAsyncRequestToDataNodeWithRetry(
+                    new TRollbackSchemaBlackListReq(consensusGroupIdList, patternTreeBytes),
+                    handler);
+            return handler.getDataNodeResponseStatusMap();
+          }
+        };
+    rollbackStateTask.execute();
+  }
+
+  @Override
+  protected boolean isRollbackSupported(DeleteTimeSeriesState deleteTimeSeriesState) {
+    return true;
+  }
+
+  @Override
+  protected DeleteTimeSeriesState getState(int stateId) {
+    return DeleteTimeSeriesState.values()[stateId];
+  }
+
+  @Override
+  protected int getStateId(DeleteTimeSeriesState deleteTimeSeriesState) {
+    return deleteTimeSeriesState.ordinal();
+  }
+
+  @Override
+  protected DeleteTimeSeriesState getInitialState() {
+    return DeleteTimeSeriesState.CONSTRUCT_BLACK_LIST;
+  }
+
+  public PathPatternTree getPatternTree() {
+    return patternTree;
+  }
+
+  public void setPatternTree(PathPatternTree patternTree) {
+    this.patternTree = patternTree;
+    requestMessage = patternTree.getAllPathPatterns().toString();
+    patternTreeBytes = preparePatternTreeBytesData(patternTree);
+  }
+
+  private ByteBuffer preparePatternTreeBytesData(PathPatternTree patternTree) {
+    ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream();
+    DataOutputStream dataOutputStream = new DataOutputStream(byteArrayOutputStream);
+    try {
+      patternTree.serialize(dataOutputStream);
+    } catch (IOException ignored) {
+
+    }
+    return ByteBuffer.wrap(byteArrayOutputStream.toByteArray());
+  }
+
+  @Override
+  public void serialize(DataOutputStream stream) throws IOException {
+    stream.writeInt(ProcedureFactory.ProcedureType.DELETE_TIMESERIES_PROCEDURE.ordinal());
+    super.serialize(stream);
+    patternTree.serialize(stream);

Review Comment:
   These two attributes is generated from patternTree for state usage, which avoids redundant generation



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [iotdb] xingtanzjr commented on a diff in pull request #7242: [IOTDB-3184] Implement Delete Timeseries for Cluster

Posted by GitBox <gi...@apache.org>.
xingtanzjr commented on code in PR #7242:
URL: https://github.com/apache/iotdb/pull/7242#discussion_r969442222


##########
server/src/main/java/org/apache/iotdb/db/service/thrift/impl/DataNodeInternalRPCServiceImpl.java:
##########
@@ -137,8 +146,11 @@ public class DataNodeInternalRPCServiceImpl implements IDataNodeRPCService.Iface
   private final SchemaEngine schemaEngine = SchemaEngine.getInstance();
   private final StorageEngineV2 storageEngine = StorageEngineV2.getInstance();
 
+  private final RegionManager regionManager;
+
   public DataNodeInternalRPCServiceImpl() {
     super();
+    regionManager = new RegionManager(schemaEngine, storageEngine);

Review Comment:
   We need to confirm that the RegionManager is shared in different instances of DataNodeInternalRPCServiceImpl



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [iotdb] xingtanzjr commented on a diff in pull request #7242: [IOTDB-3184] Implement Delete Timeseries for Cluster

Posted by GitBox <gi...@apache.org>.
xingtanzjr commented on code in PR #7242:
URL: https://github.com/apache/iotdb/pull/7242#discussion_r969322573


##########
confignode/src/main/java/org/apache/iotdb/confignode/client/async/datanode/AsyncDataNodeClientPool.java:
##########
@@ -183,6 +224,31 @@ public void sendAsyncRequestToDataNodeWithRetry(
     }
   }
 
+  public void sendAsyncRequestToDataNodeWithRetry(Object req, AbstractRetryHandler handler) {
+    Map<Integer, TDataNodeLocation> dataNodeLocationMap = handler.getDataNodeLocationMap();
+    if (dataNodeLocationMap.isEmpty()) {
+      return;
+    }
+    for (int retry = 0; retry < MAX_RETRY_NUM; retry++) {
+      CountDownLatch countDownLatch = new CountDownLatch(dataNodeLocationMap.size());
+      handler.setCountDownLatch(countDownLatch);
+      for (TDataNodeLocation targetDataNode : dataNodeLocationMap.values()) {
+        handler.setTargetDataNode(targetDataNode);
+        sendAsyncRequestToDataNode(targetDataNode, req, handler, retry);

Review Comment:
   There is concurrent issue here. 
   
   The `targetDataNode` may be changed before every request. When one request's `onComplete` is invoked asynchronously, its `targetDataNode` may not be the real one because it may be changed by subsequent request 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [iotdb] MarcosZyk commented on a diff in pull request #7242: [IOTDB-3184] Implement Delete Timeseries for Cluster

Posted by GitBox <gi...@apache.org>.
MarcosZyk commented on code in PR #7242:
URL: https://github.com/apache/iotdb/pull/7242#discussion_r969511776


##########
confignode/src/main/java/org/apache/iotdb/confignode/procedure/impl/DeleteTimeSeriesProcedure.java:
##########
@@ -0,0 +1,611 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.confignode.procedure.impl;
+
+import org.apache.iotdb.common.rpc.thrift.TConsensusGroupId;
+import org.apache.iotdb.common.rpc.thrift.TDataNodeLocation;
+import org.apache.iotdb.common.rpc.thrift.TRegionReplicaSet;
+import org.apache.iotdb.common.rpc.thrift.TSStatus;
+import org.apache.iotdb.commons.path.PartialPath;
+import org.apache.iotdb.confignode.client.async.datanode.AsyncDataNodeClientPool;
+import org.apache.iotdb.confignode.client.async.handlers.ConstructSchemaBlackListHandler;
+import org.apache.iotdb.confignode.client.async.handlers.DeleteDataForDeleteTimeSeriesHandler;
+import org.apache.iotdb.confignode.client.async.handlers.DeleteTimeSeriesHandler;
+import org.apache.iotdb.confignode.client.async.handlers.FetchSchemaBlackLsitHandler;
+import org.apache.iotdb.confignode.client.async.handlers.InvalidateMatchedSchemaCacheHandler;
+import org.apache.iotdb.confignode.client.async.handlers.RollbackSchemaBlackListHandler;
+import org.apache.iotdb.confignode.procedure.StateMachineProcedure;
+import org.apache.iotdb.confignode.procedure.env.ConfigNodeProcedureEnv;
+import org.apache.iotdb.confignode.procedure.exception.ProcedureException;
+import org.apache.iotdb.confignode.procedure.exception.ProcedureSuspendedException;
+import org.apache.iotdb.confignode.procedure.exception.ProcedureYieldException;
+import org.apache.iotdb.confignode.procedure.state.DeleteTimeSeriesState;
+import org.apache.iotdb.confignode.procedure.store.ProcedureFactory;
+import org.apache.iotdb.db.exception.metadata.PathNotExistException;
+import org.apache.iotdb.db.mpp.common.schematree.PathPatternTree;
+import org.apache.iotdb.mpp.rpc.thrift.TConstructSchemaBlackListReq;
+import org.apache.iotdb.mpp.rpc.thrift.TDeleteDataForDeleteTimeSeriesReq;
+import org.apache.iotdb.mpp.rpc.thrift.TDeleteTimeSeriesReq;
+import org.apache.iotdb.mpp.rpc.thrift.TFetchSchemaBlackListReq;
+import org.apache.iotdb.mpp.rpc.thrift.TFetchSchemaBlackListResp;
+import org.apache.iotdb.mpp.rpc.thrift.TInvalidateMatchedSchemaCacheReq;
+import org.apache.iotdb.mpp.rpc.thrift.TRollbackSchemaBlackListReq;
+import org.apache.iotdb.rpc.TSStatusCode;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.ByteArrayOutputStream;
+import java.io.DataOutputStream;
+import java.io.IOException;
+import java.nio.ByteBuffer;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.stream.Collectors;
+
+public class DeleteTimeSeriesProcedure
+    extends StateMachineProcedure<ConfigNodeProcedureEnv, DeleteTimeSeriesState> {
+
+  private static final Logger LOGGER = LoggerFactory.getLogger(DeleteTimeSeriesProcedure.class);
+
+  private PathPatternTree patternTree;
+  private ByteBuffer patternTreeBytes;
+
+  private String requestMessage;
+
+  public DeleteTimeSeriesProcedure() {
+    super();
+  }
+
+  public DeleteTimeSeriesProcedure(PathPatternTree patternTree) {
+    super();
+    setPatternTree(patternTree);
+  }
+
+  @Override
+  protected Flow executeFromState(ConfigNodeProcedureEnv env, DeleteTimeSeriesState state)
+      throws ProcedureSuspendedException, ProcedureYieldException, InterruptedException {
+    switch (state) {
+      case CONSTRUCT_BLACK_LIST:
+        LOGGER.info("Construct schema black list of timeseries {}", requestMessage);
+        if (constructBlackList(env) > 0) {
+          setNextState(DeleteTimeSeriesState.CLEAN_DATANODE_SCHEMA_CACHE);
+          break;
+        } else {
+          setFailure(
+              new ProcedureException(
+                  new PathNotExistException(
+                      patternTree.getAllPathPatterns().stream()
+                          .map(PartialPath::getFullPath)
+                          .collect(Collectors.toList()))));
+          return Flow.NO_MORE_STATE;
+        }
+      case CLEAN_DATANODE_SCHEMA_CACHE:
+        LOGGER.info("Invalidate cache of timeseries {}", requestMessage);
+        invalidateCache(env);
+        break;
+      case DELETE_DATA:
+        LOGGER.info("Delete data of timeseries {}", requestMessage);
+        deleteData(env);
+        break;
+      case DELETE_TIMESERIES:
+        LOGGER.info("Delete timeseries schema of {}", requestMessage);
+        deleteTimeSeries(env);
+        return Flow.NO_MORE_STATE;
+      default:
+        setFailure(new ProcedureException("Unrecognized state " + state.toString()));
+        return Flow.NO_MORE_STATE;
+    }
+    return Flow.HAS_MORE_STATE;
+  }
+
+  private int constructBlackList(ConfigNodeProcedureEnv env) {
+    Map<TConsensusGroupId, TRegionReplicaSet> targetSchemaRegionGroup =
+        env.getConfigManager().getRelatedSchemaRegionGroup(patternTree);
+    if (targetSchemaRegionGroup.isEmpty()) {
+      return 0;
+    }
+    RegionTask<TSStatus> constructBlackListTask =
+        new RegionTask<TSStatus>("construct schema black list", env, targetSchemaRegionGroup) {
+          @Override
+          Map<Integer, TSStatus> sendRequest(
+              TDataNodeLocation dataNodeLocation, List<TConsensusGroupId> consensusGroupIdList) {
+            // construct request and send
+            Map<Integer, TDataNodeLocation> dataNodeLocationMap = new HashMap<>();
+            dataNodeLocationMap.put(dataNodeLocation.getDataNodeId(), dataNodeLocation);
+            ConstructSchemaBlackListHandler handler =
+                new ConstructSchemaBlackListHandler(dataNodeLocationMap);
+            AsyncDataNodeClientPool.getInstance()
+                .sendAsyncRequestToDataNodeWithRetry(
+                    new TConstructSchemaBlackListReq(consensusGroupIdList, patternTreeBytes),
+                    handler);
+            handler
+                .getDataNodeResponseStatusMap()
+                .forEach(
+                    (k, v) -> {
+                      if (v.getCode() == TSStatusCode.SUCCESS_STATUS.getStatusCode()) {
+                        responseMap.put(k, v);
+                      }
+                    });
+            return handler.getDataNodeResponseStatusMap();
+          }
+        };
+    constructBlackListTask.execute();
+
+    if (isFailed()) {
+      return 0;
+    }
+
+    int preDeletedNum = 0;
+    for (TSStatus resp : constructBlackListTask.getResponseMap().values()) {
+      preDeletedNum += Integer.parseInt(resp.getMessage());
+    }
+    return preDeletedNum;
+  }
+
+  private void invalidateCache(ConfigNodeProcedureEnv env) {
+    Map<Integer, TDataNodeLocation> dataNodeLocationMap =
+        env.getConfigManager().getNodeManager().getRegisteredDataNodeLocations();
+    InvalidateMatchedSchemaCacheHandler handler =
+        new InvalidateMatchedSchemaCacheHandler(dataNodeLocationMap);
+    AsyncDataNodeClientPool.getInstance()
+        .sendAsyncRequestToDataNodeWithRetry(
+            new TInvalidateMatchedSchemaCacheReq(patternTreeBytes), handler);
+    Map<Integer, TSStatus> statusMap = handler.getDataNodeResponseStatusMap();
+    for (TSStatus status : statusMap.values()) {
+      // all dataNodes must clear the related schema cache
+      if (status.getCode() != TSStatusCode.SUCCESS_STATUS.getStatusCode()) {
+        LOGGER.error("Failed to invalidate schema cache of timeseries {}", requestMessage);
+        setFailure(new ProcedureException("Invalidate schema cache failed"));
+        return;
+      }
+    }
+
+    setNextState(DeleteTimeSeriesState.DELETE_DATA);
+  }
+
+  private void deleteData(ConfigNodeProcedureEnv env) {
+    Map<TConsensusGroupId, TRegionReplicaSet> relatedSchemaRegionGroup =
+        env.getConfigManager().getRelatedSchemaRegionGroup(patternTree);
+    Map<TDataNodeLocation, List<TConsensusGroupId>> dataNodeSchemaRegionGroupGroupIdMap =
+        getLeaderDataNodeRegionGroupMap(
+            env.getConfigManager().getPartitionManager().getAllLeadership(),
+            relatedSchemaRegionGroup);
+
+    // fetch schema black list by dataNode
+    for (Map.Entry<TDataNodeLocation, List<TConsensusGroupId>> entry :
+        dataNodeSchemaRegionGroupGroupIdMap.entrySet()) {
+      Map<TConsensusGroupId, TRegionReplicaSet> targetSchemaRegionGroup = new HashMap<>();
+      entry
+          .getValue()
+          .forEach(
+              consensusGroupId ->
+                  targetSchemaRegionGroup.put(
+                      consensusGroupId, relatedSchemaRegionGroup.get(consensusGroupId)));
+      PathPatternTree patternTree =
+          fetchSchemaBlackListOnTargetDataNode(env, targetSchemaRegionGroup);
+      if (isFailed()) {
+        return;
+      }
+      if (patternTree == null) {
+        LOGGER.error(
+            "Failed to fetch schema black list for delete data of timeseries {} on {}",
+            requestMessage,
+            entry.getKey());
+        setFailure(new ProcedureException("Fetch schema black list forDelete data failed"));
+        return;
+      }
+
+      if (patternTree.isEmpty()) {
+        continue;
+      }
+
+      Map<TConsensusGroupId, TRegionReplicaSet> relatedDataRegionGroup =
+          env.getConfigManager().getRelatedDataRegionGroup(patternTree);
+
+      // target timeseries has no data
+      if (relatedDataRegionGroup.isEmpty()) {
+        continue;
+      }
+
+      RegionTask<TSStatus> deleteDataTask =
+          new RegionTask<TSStatus>("delete data", env, relatedDataRegionGroup) {
+            @Override
+            Map<Integer, TSStatus> sendRequest(
+                TDataNodeLocation dataNodeLocation, List<TConsensusGroupId> consensusGroupIdList) {
+              Map<Integer, TDataNodeLocation> dataNodeLocationMap = new HashMap<>();
+              dataNodeLocationMap.put(dataNodeLocation.getDataNodeId(), dataNodeLocation);
+              DeleteDataForDeleteTimeSeriesHandler handler =
+                  new DeleteDataForDeleteTimeSeriesHandler(dataNodeLocationMap);
+              AsyncDataNodeClientPool.getInstance()
+                  .sendAsyncRequestToDataNodeWithRetry(
+                      new TDeleteDataForDeleteTimeSeriesReq(
+                          new ArrayList<>(consensusGroupIdList),
+                          preparePatternTreeBytesData(patternTree)),
+                      handler);
+              return handler.getDataNodeResponseStatusMap();
+            }
+          };
+      deleteDataTask.setExecuteOnAllReplicaset(true);
+      deleteDataTask.execute();
+      if (isFailed()) {
+        return;
+      }
+    }
+    setNextState(DeleteTimeSeriesState.DELETE_TIMESERIES);
+  }
+
+  private PathPatternTree fetchSchemaBlackListOnTargetDataNode(
+      ConfigNodeProcedureEnv env,
+      Map<TConsensusGroupId, TRegionReplicaSet> targetSchemaRegionGroup) {
+    RegionTask<TFetchSchemaBlackListResp> fetchSchemaBlackListTask =
+        new RegionTask<TFetchSchemaBlackListResp>(
+            "fetch schema black list", env, targetSchemaRegionGroup) {
+          @Override
+          Map<Integer, TSStatus> sendRequest(
+              TDataNodeLocation dataNodeLocation, List<TConsensusGroupId> consensusGroupIdList) {
+            Map<Integer, TDataNodeLocation> dataNodeLocationMap = new HashMap<>();
+            dataNodeLocationMap.put(dataNodeLocation.getDataNodeId(), dataNodeLocation);
+            FetchSchemaBlackLsitHandler handler =
+                new FetchSchemaBlackLsitHandler(dataNodeLocationMap);
+            AsyncDataNodeClientPool.getInstance()
+                .sendAsyncRequestToDataNodeWithRetry(
+                    new TFetchSchemaBlackListReq(consensusGroupIdList, patternTreeBytes), handler);
+            Map<Integer, TFetchSchemaBlackListResp> respMap = handler.getDataNodeResponseMap();
+            Map<Integer, TSStatus> statusMap = new HashMap<>();
+            respMap.forEach(
+                (k, v) -> {
+                  if (v.getStatus().getCode() == TSStatusCode.SUCCESS_STATUS.getStatusCode()) {
+                    responseMap.put(k, v);
+                  }
+                  statusMap.put(k, v.getStatus());
+                });
+            return statusMap;
+          }
+        };
+    fetchSchemaBlackListTask.execute();
+    if (isFailed()) {
+      return null;
+    }
+
+    Map<Integer, TFetchSchemaBlackListResp> respMap = fetchSchemaBlackListTask.getResponseMap();
+    PathPatternTree patternTree = new PathPatternTree();
+    for (TFetchSchemaBlackListResp resp : respMap.values()) {
+      for (PartialPath path :
+          PathPatternTree.deserialize(ByteBuffer.wrap(resp.getPathPatternTree()))
+              .getAllPathPatterns()) {
+        patternTree.appendPathPattern(path);
+      }
+    }
+    patternTree.constructTree();
+    return patternTree;
+  }
+
+  private void deleteTimeSeries(ConfigNodeProcedureEnv env) {
+    RegionTask<TSStatus> deleteTimeSeriesTask =
+        new RegionTask<TSStatus>(
+            "delete timeseries",
+            env,
+            env.getConfigManager().getRelatedSchemaRegionGroup(patternTree)) {
+          @Override
+          Map<Integer, TSStatus> sendRequest(
+              TDataNodeLocation dataNodeLocation, List<TConsensusGroupId> consensusGroupIdList) {
+            Map<Integer, TDataNodeLocation> dataNodeLocationMap = new HashMap<>();
+            dataNodeLocationMap.put(dataNodeLocation.getDataNodeId(), dataNodeLocation);
+            DeleteTimeSeriesHandler handler = new DeleteTimeSeriesHandler(dataNodeLocationMap);
+            AsyncDataNodeClientPool.getInstance()
+                .sendAsyncRequestToDataNodeWithRetry(
+                    new TDeleteTimeSeriesReq(consensusGroupIdList, patternTreeBytes), handler);
+            return handler.getDataNodeResponseStatusMap();
+          }
+        };
+    deleteTimeSeriesTask.execute();
+  }
+
+  private Map<TDataNodeLocation, List<TConsensusGroupId>> getLeaderDataNodeRegionGroupMap(
+      Map<TConsensusGroupId, Integer> leaderMap,
+      Map<TConsensusGroupId, TRegionReplicaSet> regionReplicaSetMap) {
+    Map<TDataNodeLocation, List<TConsensusGroupId>> dataNodeConsensusGroupIdMap = new HashMap<>();
+    regionReplicaSetMap.forEach(
+        (consensusGroupId, regionReplicaSet) -> {
+          Integer leaderId = leaderMap.get(consensusGroupId);
+          TDataNodeLocation leaderDataNodeLocation = null;
+          if (leaderId == null || leaderId == -1) {
+            leaderDataNodeLocation = regionReplicaSet.getDataNodeLocations().get(0);
+          } else {
+            for (TDataNodeLocation dataNodeLocation : regionReplicaSet.getDataNodeLocations()) {
+              if (dataNodeLocation.getDataNodeId() == leaderId) {
+                leaderDataNodeLocation = dataNodeLocation;
+                break;
+              }
+            }
+          }
+          dataNodeConsensusGroupIdMap
+              .computeIfAbsent(leaderDataNodeLocation, k -> new ArrayList<>())
+              .add(regionReplicaSet.getRegionId());
+        });
+    return dataNodeConsensusGroupIdMap;
+  }
+
+  private Map<TDataNodeLocation, List<TConsensusGroupId>> getAllReplicaDataNodeRegionGroupMap(
+      Map<TConsensusGroupId, TRegionReplicaSet> regionReplicaSetMap) {
+    Map<TDataNodeLocation, List<TConsensusGroupId>> dataNodeConsensusGroupIdMap = new HashMap<>();
+    regionReplicaSetMap.forEach(
+        (consensusGroupId, regionReplicaSet) -> {
+          for (TDataNodeLocation dataNodeLocation : regionReplicaSet.getDataNodeLocations()) {
+            dataNodeConsensusGroupIdMap
+                .computeIfAbsent(dataNodeLocation, k -> new ArrayList<>())
+                .add(regionReplicaSet.getRegionId());
+          }
+        });
+    return dataNodeConsensusGroupIdMap;
+  }
+
+  @Override
+  protected void rollbackState(
+      ConfigNodeProcedureEnv env, DeleteTimeSeriesState deleteTimeSeriesState)
+      throws IOException, InterruptedException, ProcedureException {
+    RegionTask<TSStatus> rollbackStateTask =
+        new RegionTask<TSStatus>(
+            "roll back schema black list",
+            env,
+            env.getConfigManager().getRelatedSchemaRegionGroup(patternTree)) {
+          @Override
+          Map<Integer, TSStatus> sendRequest(
+              TDataNodeLocation dataNodeLocation, List<TConsensusGroupId> consensusGroupIdList) {
+            Map<Integer, TDataNodeLocation> dataNodeLocationMap = new HashMap<>();
+            dataNodeLocationMap.put(dataNodeLocation.getDataNodeId(), dataNodeLocation);
+            RollbackSchemaBlackListHandler handler =
+                new RollbackSchemaBlackListHandler(dataNodeLocationMap);
+            AsyncDataNodeClientPool.getInstance()
+                .sendAsyncRequestToDataNodeWithRetry(
+                    new TRollbackSchemaBlackListReq(consensusGroupIdList, patternTreeBytes),
+                    handler);
+            return handler.getDataNodeResponseStatusMap();
+          }
+        };
+    rollbackStateTask.execute();
+  }
+
+  @Override
+  protected boolean isRollbackSupported(DeleteTimeSeriesState deleteTimeSeriesState) {
+    return true;
+  }
+
+  @Override
+  protected DeleteTimeSeriesState getState(int stateId) {
+    return DeleteTimeSeriesState.values()[stateId];
+  }
+
+  @Override
+  protected int getStateId(DeleteTimeSeriesState deleteTimeSeriesState) {
+    return deleteTimeSeriesState.ordinal();
+  }
+
+  @Override
+  protected DeleteTimeSeriesState getInitialState() {
+    return DeleteTimeSeriesState.CONSTRUCT_BLACK_LIST;
+  }
+
+  public PathPatternTree getPatternTree() {
+    return patternTree;
+  }
+
+  public void setPatternTree(PathPatternTree patternTree) {
+    this.patternTree = patternTree;
+    requestMessage = patternTree.getAllPathPatterns().toString();
+    patternTreeBytes = preparePatternTreeBytesData(patternTree);
+  }
+
+  private ByteBuffer preparePatternTreeBytesData(PathPatternTree patternTree) {
+    ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream();
+    DataOutputStream dataOutputStream = new DataOutputStream(byteArrayOutputStream);
+    try {
+      patternTree.serialize(dataOutputStream);
+    } catch (IOException ignored) {
+
+    }
+    return ByteBuffer.wrap(byteArrayOutputStream.toByteArray());
+  }
+
+  @Override
+  public void serialize(DataOutputStream stream) throws IOException {
+    stream.writeInt(ProcedureFactory.ProcedureType.DELETE_TIMESERIES_PROCEDURE.ordinal());
+    super.serialize(stream);
+    patternTree.serialize(stream);
+  }
+
+  @Override
+  public void deserialize(ByteBuffer byteBuffer) {
+    super.deserialize(byteBuffer);
+    setPatternTree(PathPatternTree.deserialize(byteBuffer));
+  }

Review Comment:
   Test added.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [iotdb] MarcosZyk commented on a diff in pull request #7242: [IOTDB-3184] Implement Delete Timeseries for Cluster

Posted by GitBox <gi...@apache.org>.
MarcosZyk commented on code in PR #7242:
URL: https://github.com/apache/iotdb/pull/7242#discussion_r970206290


##########
confignode/src/main/java/org/apache/iotdb/confignode/manager/ConfigManager.java:
##########
@@ -966,4 +969,80 @@ public TGetPathsSetTemplatesResp getPathsSetTemplate(String req) {
       return new TGetPathsSetTemplatesResp(status);
     }
   }
+
+  @Override
+  public TSStatus deleteTimeSeries(PathPatternTree req) {
+    TSStatus status = confirmLeader();
+    if (status.getCode() == TSStatusCode.SUCCESS_STATUS.getStatusCode()) {
+      return procedureManager.deleteTimeSeries(req);
+    } else {
+      return status;
+    }
+  }
+
+  /** Get all related schemaRegion which may contains the timeseries matched by given patternTree */
+  public Map<TConsensusGroupId, TRegionReplicaSet> getRelatedSchemaRegionGroup(
+      PathPatternTree patternTree) {
+    Map<String, Map<TSeriesPartitionSlot, TConsensusGroupId>> schemaPartitionTable =
+        getSchemaPartition(patternTree).getSchemaPartitionTable();
+
+    List<TRegionReplicaSet> allRegionReplicaSets = getPartitionManager().getAllReplicaSets();
+    Set<TConsensusGroupId> groupIdSet =
+        schemaPartitionTable.values().stream()
+            .flatMap(m -> m.values().stream())
+            .collect(Collectors.toSet());
+    Map<TConsensusGroupId, TRegionReplicaSet> filteredRegionReplicaSets = new HashMap<>();
+    for (TRegionReplicaSet regionReplicaSet : allRegionReplicaSets) {
+      if (groupIdSet.contains(regionReplicaSet.getRegionId())) {
+        filteredRegionReplicaSets.put(regionReplicaSet.getRegionId(), regionReplicaSet);
+      }
+    }
+    return filteredRegionReplicaSets;
+  }
+
+  /**
+   * Get all related dataRegion which may contains the data of specific timeseries matched by given
+   * patternTree
+   */
+  public Map<TConsensusGroupId, TRegionReplicaSet> getRelatedDataRegionGroup(
+      PathPatternTree patternTree) {
+    // get all storage group and slot by getting schema partition
+    Map<String, Map<TSeriesPartitionSlot, TConsensusGroupId>> schemaPartitionTable =
+        getSchemaPartition(patternTree).getSchemaPartitionTable();
+
+    // construct request for getting data partition
+    Map<String, Map<TSeriesPartitionSlot, List<TTimePartitionSlot>>> partitionSlotsMap =
+        new HashMap<>();
+    schemaPartitionTable.forEach(
+        (key, value) -> {
+          Map<TSeriesPartitionSlot, List<TTimePartitionSlot>> slotListMap = new HashMap<>();
+          value.keySet().forEach(slot -> slotListMap.put(slot, Collections.emptyList()));
+          partitionSlotsMap.put(key, slotListMap);
+        });
+
+    // get all data partitions
+    GetDataPartitionPlan getDataPartitionPlan = new GetDataPartitionPlan(partitionSlotsMap);
+    Map<String, Map<TSeriesPartitionSlot, Map<TTimePartitionSlot, List<TConsensusGroupId>>>>
+        dataPartitionTable = getDataPartition(getDataPartitionPlan).getDataPartitionTable();
+
+    // get all region replicaset of target data partitions
+    List<TRegionReplicaSet> allRegionReplicaSets = getPartitionManager().getAllReplicaSets();
+    Set<TConsensusGroupId> groupIdSet =
+        dataPartitionTable.values().stream()
+            .flatMap(
+                tSeriesPartitionSlotMapMap ->
+                    tSeriesPartitionSlotMapMap.values().stream()
+                        .flatMap(
+                            tTimePartitionSlotListMap ->
+                                tTimePartitionSlotListMap.values().stream()
+                                    .flatMap(Collection::stream)))
+            .collect(Collectors.toSet());
+    Map<TConsensusGroupId, TRegionReplicaSet> filteredRegionReplicaSets = new HashMap<>();
+    for (TRegionReplicaSet regionReplicaSet : allRegionReplicaSets) {
+      if (groupIdSet.contains(regionReplicaSet.getRegionId())) {
+        filteredRegionReplicaSets.put(regionReplicaSet.getRegionId(), regionReplicaSet);
+      }
+    }
+    return filteredRegionReplicaSets;
+  }

Review Comment:
   These two functions are implemented across different manager: ```ClusterSchemaManager``` and ```PartitionManager```.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [iotdb] wangchao316 commented on a diff in pull request #7242: [IOTDB-3184] Implement Delete Timeseries for Cluster

Posted by GitBox <gi...@apache.org>.
wangchao316 commented on code in PR #7242:
URL: https://github.com/apache/iotdb/pull/7242#discussion_r969068865


##########
confignode/src/main/java/org/apache/iotdb/confignode/procedure/impl/DeleteTimeSeriesProcedure.java:
##########
@@ -0,0 +1,611 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.confignode.procedure.impl;
+
+import org.apache.iotdb.common.rpc.thrift.TConsensusGroupId;
+import org.apache.iotdb.common.rpc.thrift.TDataNodeLocation;
+import org.apache.iotdb.common.rpc.thrift.TRegionReplicaSet;
+import org.apache.iotdb.common.rpc.thrift.TSStatus;
+import org.apache.iotdb.commons.path.PartialPath;
+import org.apache.iotdb.confignode.client.async.datanode.AsyncDataNodeClientPool;
+import org.apache.iotdb.confignode.client.async.handlers.ConstructSchemaBlackListHandler;
+import org.apache.iotdb.confignode.client.async.handlers.DeleteDataForDeleteTimeSeriesHandler;
+import org.apache.iotdb.confignode.client.async.handlers.DeleteTimeSeriesHandler;
+import org.apache.iotdb.confignode.client.async.handlers.FetchSchemaBlackLsitHandler;
+import org.apache.iotdb.confignode.client.async.handlers.InvalidateMatchedSchemaCacheHandler;
+import org.apache.iotdb.confignode.client.async.handlers.RollbackSchemaBlackListHandler;
+import org.apache.iotdb.confignode.procedure.StateMachineProcedure;
+import org.apache.iotdb.confignode.procedure.env.ConfigNodeProcedureEnv;
+import org.apache.iotdb.confignode.procedure.exception.ProcedureException;
+import org.apache.iotdb.confignode.procedure.exception.ProcedureSuspendedException;
+import org.apache.iotdb.confignode.procedure.exception.ProcedureYieldException;
+import org.apache.iotdb.confignode.procedure.state.DeleteTimeSeriesState;
+import org.apache.iotdb.confignode.procedure.store.ProcedureFactory;
+import org.apache.iotdb.db.exception.metadata.PathNotExistException;
+import org.apache.iotdb.db.mpp.common.schematree.PathPatternTree;
+import org.apache.iotdb.mpp.rpc.thrift.TConstructSchemaBlackListReq;
+import org.apache.iotdb.mpp.rpc.thrift.TDeleteDataForDeleteTimeSeriesReq;
+import org.apache.iotdb.mpp.rpc.thrift.TDeleteTimeSeriesReq;
+import org.apache.iotdb.mpp.rpc.thrift.TFetchSchemaBlackListReq;
+import org.apache.iotdb.mpp.rpc.thrift.TFetchSchemaBlackListResp;
+import org.apache.iotdb.mpp.rpc.thrift.TInvalidateMatchedSchemaCacheReq;
+import org.apache.iotdb.mpp.rpc.thrift.TRollbackSchemaBlackListReq;
+import org.apache.iotdb.rpc.TSStatusCode;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.ByteArrayOutputStream;
+import java.io.DataOutputStream;
+import java.io.IOException;
+import java.nio.ByteBuffer;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.stream.Collectors;
+
+public class DeleteTimeSeriesProcedure
+    extends StateMachineProcedure<ConfigNodeProcedureEnv, DeleteTimeSeriesState> {
+
+  private static final Logger LOGGER = LoggerFactory.getLogger(DeleteTimeSeriesProcedure.class);
+
+  private PathPatternTree patternTree;
+  private ByteBuffer patternTreeBytes;
+
+  private String requestMessage;
+
+  public DeleteTimeSeriesProcedure() {
+    super();
+  }
+
+  public DeleteTimeSeriesProcedure(PathPatternTree patternTree) {
+    super();
+    setPatternTree(patternTree);
+  }
+
+  @Override
+  protected Flow executeFromState(ConfigNodeProcedureEnv env, DeleteTimeSeriesState state)
+      throws ProcedureSuspendedException, ProcedureYieldException, InterruptedException {
+    switch (state) {
+      case CONSTRUCT_BLACK_LIST:
+        LOGGER.info("Construct schema black list of timeseries {}", requestMessage);
+        if (constructBlackList(env) > 0) {
+          setNextState(DeleteTimeSeriesState.CLEAN_DATANODE_SCHEMA_CACHE);
+          break;
+        } else {
+          setFailure(
+              new ProcedureException(
+                  new PathNotExistException(
+                      patternTree.getAllPathPatterns().stream()
+                          .map(PartialPath::getFullPath)
+                          .collect(Collectors.toList()))));
+          return Flow.NO_MORE_STATE;
+        }
+      case CLEAN_DATANODE_SCHEMA_CACHE:
+        LOGGER.info("Invalidate cache of timeseries {}", requestMessage);
+        invalidateCache(env);
+        break;
+      case DELETE_DATA:
+        LOGGER.info("Delete data of timeseries {}", requestMessage);
+        deleteData(env);
+        break;
+      case DELETE_TIMESERIES:
+        LOGGER.info("Delete timeseries schema of {}", requestMessage);
+        deleteTimeSeries(env);
+        return Flow.NO_MORE_STATE;
+      default:
+        setFailure(new ProcedureException("Unrecognized state " + state.toString()));
+        return Flow.NO_MORE_STATE;
+    }
+    return Flow.HAS_MORE_STATE;
+  }
+
+  private int constructBlackList(ConfigNodeProcedureEnv env) {
+    Map<TConsensusGroupId, TRegionReplicaSet> targetSchemaRegionGroup =
+        env.getConfigManager().getRelatedSchemaRegionGroup(patternTree);
+    if (targetSchemaRegionGroup.isEmpty()) {
+      return 0;
+    }
+    RegionTask<TSStatus> constructBlackListTask =
+        new RegionTask<TSStatus>("construct schema black list", env, targetSchemaRegionGroup) {
+          @Override
+          Map<Integer, TSStatus> sendRequest(
+              TDataNodeLocation dataNodeLocation, List<TConsensusGroupId> consensusGroupIdList) {
+            // construct request and send
+            Map<Integer, TDataNodeLocation> dataNodeLocationMap = new HashMap<>();
+            dataNodeLocationMap.put(dataNodeLocation.getDataNodeId(), dataNodeLocation);
+            ConstructSchemaBlackListHandler handler =
+                new ConstructSchemaBlackListHandler(dataNodeLocationMap);
+            AsyncDataNodeClientPool.getInstance()
+                .sendAsyncRequestToDataNodeWithRetry(
+                    new TConstructSchemaBlackListReq(consensusGroupIdList, patternTreeBytes),
+                    handler);
+            handler
+                .getDataNodeResponseStatusMap()
+                .forEach(
+                    (k, v) -> {
+                      if (v.getCode() == TSStatusCode.SUCCESS_STATUS.getStatusCode()) {
+                        responseMap.put(k, v);
+                      }
+                    });
+            return handler.getDataNodeResponseStatusMap();
+          }
+        };
+    constructBlackListTask.execute();
+
+    if (isFailed()) {
+      return 0;
+    }
+
+    int preDeletedNum = 0;
+    for (TSStatus resp : constructBlackListTask.getResponseMap().values()) {
+      preDeletedNum += Integer.parseInt(resp.getMessage());
+    }
+    return preDeletedNum;
+  }
+
+  private void invalidateCache(ConfigNodeProcedureEnv env) {
+    Map<Integer, TDataNodeLocation> dataNodeLocationMap =
+        env.getConfigManager().getNodeManager().getRegisteredDataNodeLocations();
+    InvalidateMatchedSchemaCacheHandler handler =
+        new InvalidateMatchedSchemaCacheHandler(dataNodeLocationMap);
+    AsyncDataNodeClientPool.getInstance()
+        .sendAsyncRequestToDataNodeWithRetry(
+            new TInvalidateMatchedSchemaCacheReq(patternTreeBytes), handler);
+    Map<Integer, TSStatus> statusMap = handler.getDataNodeResponseStatusMap();
+    for (TSStatus status : statusMap.values()) {
+      // all dataNodes must clear the related schema cache
+      if (status.getCode() != TSStatusCode.SUCCESS_STATUS.getStatusCode()) {
+        LOGGER.error("Failed to invalidate schema cache of timeseries {}", requestMessage);
+        setFailure(new ProcedureException("Invalidate schema cache failed"));
+        return;
+      }
+    }
+
+    setNextState(DeleteTimeSeriesState.DELETE_DATA);
+  }
+
+  private void deleteData(ConfigNodeProcedureEnv env) {
+    Map<TConsensusGroupId, TRegionReplicaSet> relatedSchemaRegionGroup =
+        env.getConfigManager().getRelatedSchemaRegionGroup(patternTree);
+    Map<TDataNodeLocation, List<TConsensusGroupId>> dataNodeSchemaRegionGroupGroupIdMap =
+        getLeaderDataNodeRegionGroupMap(
+            env.getConfigManager().getPartitionManager().getAllLeadership(),
+            relatedSchemaRegionGroup);
+
+    // fetch schema black list by dataNode
+    for (Map.Entry<TDataNodeLocation, List<TConsensusGroupId>> entry :
+        dataNodeSchemaRegionGroupGroupIdMap.entrySet()) {
+      Map<TConsensusGroupId, TRegionReplicaSet> targetSchemaRegionGroup = new HashMap<>();
+      entry
+          .getValue()
+          .forEach(
+              consensusGroupId ->
+                  targetSchemaRegionGroup.put(
+                      consensusGroupId, relatedSchemaRegionGroup.get(consensusGroupId)));
+      PathPatternTree patternTree =
+          fetchSchemaBlackListOnTargetDataNode(env, targetSchemaRegionGroup);
+      if (isFailed()) {
+        return;
+      }
+      if (patternTree == null) {
+        LOGGER.error(
+            "Failed to fetch schema black list for delete data of timeseries {} on {}",
+            requestMessage,
+            entry.getKey());
+        setFailure(new ProcedureException("Fetch schema black list forDelete data failed"));
+        return;
+      }
+
+      if (patternTree.isEmpty()) {
+        continue;
+      }
+
+      Map<TConsensusGroupId, TRegionReplicaSet> relatedDataRegionGroup =
+          env.getConfigManager().getRelatedDataRegionGroup(patternTree);
+
+      // target timeseries has no data
+      if (relatedDataRegionGroup.isEmpty()) {
+        continue;
+      }
+
+      RegionTask<TSStatus> deleteDataTask =
+          new RegionTask<TSStatus>("delete data", env, relatedDataRegionGroup) {
+            @Override
+            Map<Integer, TSStatus> sendRequest(
+                TDataNodeLocation dataNodeLocation, List<TConsensusGroupId> consensusGroupIdList) {
+              Map<Integer, TDataNodeLocation> dataNodeLocationMap = new HashMap<>();
+              dataNodeLocationMap.put(dataNodeLocation.getDataNodeId(), dataNodeLocation);
+              DeleteDataForDeleteTimeSeriesHandler handler =
+                  new DeleteDataForDeleteTimeSeriesHandler(dataNodeLocationMap);
+              AsyncDataNodeClientPool.getInstance()
+                  .sendAsyncRequestToDataNodeWithRetry(
+                      new TDeleteDataForDeleteTimeSeriesReq(
+                          new ArrayList<>(consensusGroupIdList),
+                          preparePatternTreeBytesData(patternTree)),
+                      handler);
+              return handler.getDataNodeResponseStatusMap();
+            }
+          };
+      deleteDataTask.setExecuteOnAllReplicaset(true);
+      deleteDataTask.execute();
+      if (isFailed()) {
+        return;
+      }
+    }
+    setNextState(DeleteTimeSeriesState.DELETE_TIMESERIES);
+  }
+
+  private PathPatternTree fetchSchemaBlackListOnTargetDataNode(
+      ConfigNodeProcedureEnv env,
+      Map<TConsensusGroupId, TRegionReplicaSet> targetSchemaRegionGroup) {
+    RegionTask<TFetchSchemaBlackListResp> fetchSchemaBlackListTask =
+        new RegionTask<TFetchSchemaBlackListResp>(
+            "fetch schema black list", env, targetSchemaRegionGroup) {
+          @Override
+          Map<Integer, TSStatus> sendRequest(
+              TDataNodeLocation dataNodeLocation, List<TConsensusGroupId> consensusGroupIdList) {
+            Map<Integer, TDataNodeLocation> dataNodeLocationMap = new HashMap<>();
+            dataNodeLocationMap.put(dataNodeLocation.getDataNodeId(), dataNodeLocation);
+            FetchSchemaBlackLsitHandler handler =
+                new FetchSchemaBlackLsitHandler(dataNodeLocationMap);
+            AsyncDataNodeClientPool.getInstance()
+                .sendAsyncRequestToDataNodeWithRetry(
+                    new TFetchSchemaBlackListReq(consensusGroupIdList, patternTreeBytes), handler);
+            Map<Integer, TFetchSchemaBlackListResp> respMap = handler.getDataNodeResponseMap();
+            Map<Integer, TSStatus> statusMap = new HashMap<>();
+            respMap.forEach(
+                (k, v) -> {
+                  if (v.getStatus().getCode() == TSStatusCode.SUCCESS_STATUS.getStatusCode()) {
+                    responseMap.put(k, v);
+                  }
+                  statusMap.put(k, v.getStatus());
+                });
+            return statusMap;
+          }
+        };
+    fetchSchemaBlackListTask.execute();
+    if (isFailed()) {
+      return null;
+    }
+
+    Map<Integer, TFetchSchemaBlackListResp> respMap = fetchSchemaBlackListTask.getResponseMap();
+    PathPatternTree patternTree = new PathPatternTree();
+    for (TFetchSchemaBlackListResp resp : respMap.values()) {
+      for (PartialPath path :
+          PathPatternTree.deserialize(ByteBuffer.wrap(resp.getPathPatternTree()))
+              .getAllPathPatterns()) {
+        patternTree.appendPathPattern(path);
+      }
+    }
+    patternTree.constructTree();
+    return patternTree;
+  }
+
+  private void deleteTimeSeries(ConfigNodeProcedureEnv env) {
+    RegionTask<TSStatus> deleteTimeSeriesTask =
+        new RegionTask<TSStatus>(
+            "delete timeseries",
+            env,
+            env.getConfigManager().getRelatedSchemaRegionGroup(patternTree)) {
+          @Override
+          Map<Integer, TSStatus> sendRequest(
+              TDataNodeLocation dataNodeLocation, List<TConsensusGroupId> consensusGroupIdList) {
+            Map<Integer, TDataNodeLocation> dataNodeLocationMap = new HashMap<>();
+            dataNodeLocationMap.put(dataNodeLocation.getDataNodeId(), dataNodeLocation);
+            DeleteTimeSeriesHandler handler = new DeleteTimeSeriesHandler(dataNodeLocationMap);
+            AsyncDataNodeClientPool.getInstance()
+                .sendAsyncRequestToDataNodeWithRetry(
+                    new TDeleteTimeSeriesReq(consensusGroupIdList, patternTreeBytes), handler);
+            return handler.getDataNodeResponseStatusMap();
+          }
+        };
+    deleteTimeSeriesTask.execute();
+  }
+
+  private Map<TDataNodeLocation, List<TConsensusGroupId>> getLeaderDataNodeRegionGroupMap(
+      Map<TConsensusGroupId, Integer> leaderMap,
+      Map<TConsensusGroupId, TRegionReplicaSet> regionReplicaSetMap) {
+    Map<TDataNodeLocation, List<TConsensusGroupId>> dataNodeConsensusGroupIdMap = new HashMap<>();
+    regionReplicaSetMap.forEach(
+        (consensusGroupId, regionReplicaSet) -> {
+          Integer leaderId = leaderMap.get(consensusGroupId);
+          TDataNodeLocation leaderDataNodeLocation = null;
+          if (leaderId == null || leaderId == -1) {
+            leaderDataNodeLocation = regionReplicaSet.getDataNodeLocations().get(0);
+          } else {
+            for (TDataNodeLocation dataNodeLocation : regionReplicaSet.getDataNodeLocations()) {
+              if (dataNodeLocation.getDataNodeId() == leaderId) {
+                leaderDataNodeLocation = dataNodeLocation;
+                break;
+              }
+            }
+          }
+          dataNodeConsensusGroupIdMap
+              .computeIfAbsent(leaderDataNodeLocation, k -> new ArrayList<>())
+              .add(regionReplicaSet.getRegionId());
+        });
+    return dataNodeConsensusGroupIdMap;
+  }
+
+  private Map<TDataNodeLocation, List<TConsensusGroupId>> getAllReplicaDataNodeRegionGroupMap(
+      Map<TConsensusGroupId, TRegionReplicaSet> regionReplicaSetMap) {
+    Map<TDataNodeLocation, List<TConsensusGroupId>> dataNodeConsensusGroupIdMap = new HashMap<>();
+    regionReplicaSetMap.forEach(
+        (consensusGroupId, regionReplicaSet) -> {
+          for (TDataNodeLocation dataNodeLocation : regionReplicaSet.getDataNodeLocations()) {
+            dataNodeConsensusGroupIdMap
+                .computeIfAbsent(dataNodeLocation, k -> new ArrayList<>())
+                .add(regionReplicaSet.getRegionId());
+          }
+        });
+    return dataNodeConsensusGroupIdMap;
+  }
+
+  @Override
+  protected void rollbackState(
+      ConfigNodeProcedureEnv env, DeleteTimeSeriesState deleteTimeSeriesState)
+      throws IOException, InterruptedException, ProcedureException {
+    RegionTask<TSStatus> rollbackStateTask =
+        new RegionTask<TSStatus>(
+            "roll back schema black list",
+            env,
+            env.getConfigManager().getRelatedSchemaRegionGroup(patternTree)) {
+          @Override
+          Map<Integer, TSStatus> sendRequest(
+              TDataNodeLocation dataNodeLocation, List<TConsensusGroupId> consensusGroupIdList) {
+            Map<Integer, TDataNodeLocation> dataNodeLocationMap = new HashMap<>();
+            dataNodeLocationMap.put(dataNodeLocation.getDataNodeId(), dataNodeLocation);
+            RollbackSchemaBlackListHandler handler =
+                new RollbackSchemaBlackListHandler(dataNodeLocationMap);
+            AsyncDataNodeClientPool.getInstance()
+                .sendAsyncRequestToDataNodeWithRetry(
+                    new TRollbackSchemaBlackListReq(consensusGroupIdList, patternTreeBytes),
+                    handler);
+            return handler.getDataNodeResponseStatusMap();
+          }
+        };
+    rollbackStateTask.execute();
+  }
+
+  @Override
+  protected boolean isRollbackSupported(DeleteTimeSeriesState deleteTimeSeriesState) {
+    return true;
+  }
+
+  @Override
+  protected DeleteTimeSeriesState getState(int stateId) {
+    return DeleteTimeSeriesState.values()[stateId];
+  }
+
+  @Override
+  protected int getStateId(DeleteTimeSeriesState deleteTimeSeriesState) {
+    return deleteTimeSeriesState.ordinal();
+  }
+
+  @Override
+  protected DeleteTimeSeriesState getInitialState() {
+    return DeleteTimeSeriesState.CONSTRUCT_BLACK_LIST;
+  }
+
+  public PathPatternTree getPatternTree() {
+    return patternTree;
+  }
+
+  public void setPatternTree(PathPatternTree patternTree) {
+    this.patternTree = patternTree;
+    requestMessage = patternTree.getAllPathPatterns().toString();
+    patternTreeBytes = preparePatternTreeBytesData(patternTree);
+  }
+
+  private ByteBuffer preparePatternTreeBytesData(PathPatternTree patternTree) {
+    ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream();
+    DataOutputStream dataOutputStream = new DataOutputStream(byteArrayOutputStream);
+    try {
+      patternTree.serialize(dataOutputStream);
+    } catch (IOException ignored) {
+
+    }
+    return ByteBuffer.wrap(byteArrayOutputStream.toByteArray());
+  }
+
+  @Override
+  public void serialize(DataOutputStream stream) throws IOException {
+    stream.writeInt(ProcedureFactory.ProcedureType.DELETE_TIMESERIES_PROCEDURE.ordinal());
+    super.serialize(stream);
+    patternTree.serialize(stream);

Review Comment:
   need serialize and deserialize requestMessage, patternTreeBytes .  If the leader breaks down, other follower nodes need to replay and execute the procedure after receiving the procedure. The attributes required by these states may have been initialized.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [iotdb] MarcosZyk commented on a diff in pull request #7242: [IOTDB-3184] Implement Delete Timeseries for Cluster

Posted by GitBox <gi...@apache.org>.
MarcosZyk commented on code in PR #7242:
URL: https://github.com/apache/iotdb/pull/7242#discussion_r969070883


##########
confignode/src/main/java/org/apache/iotdb/confignode/procedure/impl/DeleteTimeSeriesProcedure.java:
##########
@@ -0,0 +1,611 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.confignode.procedure.impl;
+
+import org.apache.iotdb.common.rpc.thrift.TConsensusGroupId;
+import org.apache.iotdb.common.rpc.thrift.TDataNodeLocation;
+import org.apache.iotdb.common.rpc.thrift.TRegionReplicaSet;
+import org.apache.iotdb.common.rpc.thrift.TSStatus;
+import org.apache.iotdb.commons.path.PartialPath;
+import org.apache.iotdb.confignode.client.async.datanode.AsyncDataNodeClientPool;
+import org.apache.iotdb.confignode.client.async.handlers.ConstructSchemaBlackListHandler;
+import org.apache.iotdb.confignode.client.async.handlers.DeleteDataForDeleteTimeSeriesHandler;
+import org.apache.iotdb.confignode.client.async.handlers.DeleteTimeSeriesHandler;
+import org.apache.iotdb.confignode.client.async.handlers.FetchSchemaBlackLsitHandler;
+import org.apache.iotdb.confignode.client.async.handlers.InvalidateMatchedSchemaCacheHandler;
+import org.apache.iotdb.confignode.client.async.handlers.RollbackSchemaBlackListHandler;
+import org.apache.iotdb.confignode.procedure.StateMachineProcedure;
+import org.apache.iotdb.confignode.procedure.env.ConfigNodeProcedureEnv;
+import org.apache.iotdb.confignode.procedure.exception.ProcedureException;
+import org.apache.iotdb.confignode.procedure.exception.ProcedureSuspendedException;
+import org.apache.iotdb.confignode.procedure.exception.ProcedureYieldException;
+import org.apache.iotdb.confignode.procedure.state.DeleteTimeSeriesState;
+import org.apache.iotdb.confignode.procedure.store.ProcedureFactory;
+import org.apache.iotdb.db.exception.metadata.PathNotExistException;
+import org.apache.iotdb.db.mpp.common.schematree.PathPatternTree;
+import org.apache.iotdb.mpp.rpc.thrift.TConstructSchemaBlackListReq;
+import org.apache.iotdb.mpp.rpc.thrift.TDeleteDataForDeleteTimeSeriesReq;
+import org.apache.iotdb.mpp.rpc.thrift.TDeleteTimeSeriesReq;
+import org.apache.iotdb.mpp.rpc.thrift.TFetchSchemaBlackListReq;
+import org.apache.iotdb.mpp.rpc.thrift.TFetchSchemaBlackListResp;
+import org.apache.iotdb.mpp.rpc.thrift.TInvalidateMatchedSchemaCacheReq;
+import org.apache.iotdb.mpp.rpc.thrift.TRollbackSchemaBlackListReq;
+import org.apache.iotdb.rpc.TSStatusCode;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.ByteArrayOutputStream;
+import java.io.DataOutputStream;
+import java.io.IOException;
+import java.nio.ByteBuffer;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.stream.Collectors;
+
+public class DeleteTimeSeriesProcedure
+    extends StateMachineProcedure<ConfigNodeProcedureEnv, DeleteTimeSeriesState> {
+
+  private static final Logger LOGGER = LoggerFactory.getLogger(DeleteTimeSeriesProcedure.class);
+
+  private PathPatternTree patternTree;
+  private ByteBuffer patternTreeBytes;
+
+  private String requestMessage;
+
+  public DeleteTimeSeriesProcedure() {
+    super();
+  }
+
+  public DeleteTimeSeriesProcedure(PathPatternTree patternTree) {
+    super();
+    setPatternTree(patternTree);
+  }
+
+  @Override
+  protected Flow executeFromState(ConfigNodeProcedureEnv env, DeleteTimeSeriesState state)
+      throws ProcedureSuspendedException, ProcedureYieldException, InterruptedException {
+    switch (state) {
+      case CONSTRUCT_BLACK_LIST:
+        LOGGER.info("Construct schema black list of timeseries {}", requestMessage);
+        if (constructBlackList(env) > 0) {
+          setNextState(DeleteTimeSeriesState.CLEAN_DATANODE_SCHEMA_CACHE);
+          break;
+        } else {
+          setFailure(
+              new ProcedureException(
+                  new PathNotExistException(
+                      patternTree.getAllPathPatterns().stream()
+                          .map(PartialPath::getFullPath)
+                          .collect(Collectors.toList()))));
+          return Flow.NO_MORE_STATE;
+        }
+      case CLEAN_DATANODE_SCHEMA_CACHE:
+        LOGGER.info("Invalidate cache of timeseries {}", requestMessage);
+        invalidateCache(env);
+        break;
+      case DELETE_DATA:
+        LOGGER.info("Delete data of timeseries {}", requestMessage);
+        deleteData(env);
+        break;
+      case DELETE_TIMESERIES:
+        LOGGER.info("Delete timeseries schema of {}", requestMessage);
+        deleteTimeSeries(env);
+        return Flow.NO_MORE_STATE;
+      default:
+        setFailure(new ProcedureException("Unrecognized state " + state.toString()));
+        return Flow.NO_MORE_STATE;
+    }
+    return Flow.HAS_MORE_STATE;
+  }
+
+  private int constructBlackList(ConfigNodeProcedureEnv env) {
+    Map<TConsensusGroupId, TRegionReplicaSet> targetSchemaRegionGroup =
+        env.getConfigManager().getRelatedSchemaRegionGroup(patternTree);
+    if (targetSchemaRegionGroup.isEmpty()) {
+      return 0;
+    }
+    RegionTask<TSStatus> constructBlackListTask =
+        new RegionTask<TSStatus>("construct schema black list", env, targetSchemaRegionGroup) {
+          @Override
+          Map<Integer, TSStatus> sendRequest(
+              TDataNodeLocation dataNodeLocation, List<TConsensusGroupId> consensusGroupIdList) {
+            // construct request and send
+            Map<Integer, TDataNodeLocation> dataNodeLocationMap = new HashMap<>();
+            dataNodeLocationMap.put(dataNodeLocation.getDataNodeId(), dataNodeLocation);
+            ConstructSchemaBlackListHandler handler =
+                new ConstructSchemaBlackListHandler(dataNodeLocationMap);
+            AsyncDataNodeClientPool.getInstance()
+                .sendAsyncRequestToDataNodeWithRetry(
+                    new TConstructSchemaBlackListReq(consensusGroupIdList, patternTreeBytes),
+                    handler);
+            handler
+                .getDataNodeResponseStatusMap()
+                .forEach(
+                    (k, v) -> {
+                      if (v.getCode() == TSStatusCode.SUCCESS_STATUS.getStatusCode()) {
+                        responseMap.put(k, v);
+                      }
+                    });
+            return handler.getDataNodeResponseStatusMap();
+          }
+        };
+    constructBlackListTask.execute();
+
+    if (isFailed()) {
+      return 0;
+    }
+
+    int preDeletedNum = 0;
+    for (TSStatus resp : constructBlackListTask.getResponseMap().values()) {
+      preDeletedNum += Integer.parseInt(resp.getMessage());
+    }
+    return preDeletedNum;
+  }
+
+  private void invalidateCache(ConfigNodeProcedureEnv env) {
+    Map<Integer, TDataNodeLocation> dataNodeLocationMap =
+        env.getConfigManager().getNodeManager().getRegisteredDataNodeLocations();
+    InvalidateMatchedSchemaCacheHandler handler =
+        new InvalidateMatchedSchemaCacheHandler(dataNodeLocationMap);
+    AsyncDataNodeClientPool.getInstance()
+        .sendAsyncRequestToDataNodeWithRetry(
+            new TInvalidateMatchedSchemaCacheReq(patternTreeBytes), handler);
+    Map<Integer, TSStatus> statusMap = handler.getDataNodeResponseStatusMap();
+    for (TSStatus status : statusMap.values()) {
+      // all dataNodes must clear the related schema cache
+      if (status.getCode() != TSStatusCode.SUCCESS_STATUS.getStatusCode()) {
+        LOGGER.error("Failed to invalidate schema cache of timeseries {}", requestMessage);
+        setFailure(new ProcedureException("Invalidate schema cache failed"));
+        return;
+      }
+    }
+
+    setNextState(DeleteTimeSeriesState.DELETE_DATA);
+  }
+
+  private void deleteData(ConfigNodeProcedureEnv env) {
+    Map<TConsensusGroupId, TRegionReplicaSet> relatedSchemaRegionGroup =
+        env.getConfigManager().getRelatedSchemaRegionGroup(patternTree);
+    Map<TDataNodeLocation, List<TConsensusGroupId>> dataNodeSchemaRegionGroupGroupIdMap =
+        getLeaderDataNodeRegionGroupMap(
+            env.getConfigManager().getPartitionManager().getAllLeadership(),
+            relatedSchemaRegionGroup);
+
+    // fetch schema black list by dataNode
+    for (Map.Entry<TDataNodeLocation, List<TConsensusGroupId>> entry :
+        dataNodeSchemaRegionGroupGroupIdMap.entrySet()) {
+      Map<TConsensusGroupId, TRegionReplicaSet> targetSchemaRegionGroup = new HashMap<>();
+      entry
+          .getValue()
+          .forEach(
+              consensusGroupId ->
+                  targetSchemaRegionGroup.put(
+                      consensusGroupId, relatedSchemaRegionGroup.get(consensusGroupId)));
+      PathPatternTree patternTree =
+          fetchSchemaBlackListOnTargetDataNode(env, targetSchemaRegionGroup);
+      if (isFailed()) {
+        return;
+      }
+      if (patternTree == null) {
+        LOGGER.error(
+            "Failed to fetch schema black list for delete data of timeseries {} on {}",
+            requestMessage,
+            entry.getKey());
+        setFailure(new ProcedureException("Fetch schema black list forDelete data failed"));
+        return;
+      }
+
+      if (patternTree.isEmpty()) {
+        continue;
+      }
+
+      Map<TConsensusGroupId, TRegionReplicaSet> relatedDataRegionGroup =
+          env.getConfigManager().getRelatedDataRegionGroup(patternTree);
+
+      // target timeseries has no data
+      if (relatedDataRegionGroup.isEmpty()) {
+        continue;
+      }
+
+      RegionTask<TSStatus> deleteDataTask =
+          new RegionTask<TSStatus>("delete data", env, relatedDataRegionGroup) {
+            @Override
+            Map<Integer, TSStatus> sendRequest(
+                TDataNodeLocation dataNodeLocation, List<TConsensusGroupId> consensusGroupIdList) {
+              Map<Integer, TDataNodeLocation> dataNodeLocationMap = new HashMap<>();
+              dataNodeLocationMap.put(dataNodeLocation.getDataNodeId(), dataNodeLocation);
+              DeleteDataForDeleteTimeSeriesHandler handler =
+                  new DeleteDataForDeleteTimeSeriesHandler(dataNodeLocationMap);
+              AsyncDataNodeClientPool.getInstance()
+                  .sendAsyncRequestToDataNodeWithRetry(
+                      new TDeleteDataForDeleteTimeSeriesReq(
+                          new ArrayList<>(consensusGroupIdList),
+                          preparePatternTreeBytesData(patternTree)),
+                      handler);
+              return handler.getDataNodeResponseStatusMap();
+            }
+          };
+      deleteDataTask.setExecuteOnAllReplicaset(true);
+      deleteDataTask.execute();
+      if (isFailed()) {
+        return;
+      }
+    }
+    setNextState(DeleteTimeSeriesState.DELETE_TIMESERIES);
+  }
+
+  private PathPatternTree fetchSchemaBlackListOnTargetDataNode(
+      ConfigNodeProcedureEnv env,
+      Map<TConsensusGroupId, TRegionReplicaSet> targetSchemaRegionGroup) {
+    RegionTask<TFetchSchemaBlackListResp> fetchSchemaBlackListTask =
+        new RegionTask<TFetchSchemaBlackListResp>(
+            "fetch schema black list", env, targetSchemaRegionGroup) {
+          @Override
+          Map<Integer, TSStatus> sendRequest(
+              TDataNodeLocation dataNodeLocation, List<TConsensusGroupId> consensusGroupIdList) {
+            Map<Integer, TDataNodeLocation> dataNodeLocationMap = new HashMap<>();
+            dataNodeLocationMap.put(dataNodeLocation.getDataNodeId(), dataNodeLocation);
+            FetchSchemaBlackLsitHandler handler =
+                new FetchSchemaBlackLsitHandler(dataNodeLocationMap);
+            AsyncDataNodeClientPool.getInstance()
+                .sendAsyncRequestToDataNodeWithRetry(
+                    new TFetchSchemaBlackListReq(consensusGroupIdList, patternTreeBytes), handler);
+            Map<Integer, TFetchSchemaBlackListResp> respMap = handler.getDataNodeResponseMap();
+            Map<Integer, TSStatus> statusMap = new HashMap<>();
+            respMap.forEach(
+                (k, v) -> {
+                  if (v.getStatus().getCode() == TSStatusCode.SUCCESS_STATUS.getStatusCode()) {
+                    responseMap.put(k, v);
+                  }
+                  statusMap.put(k, v.getStatus());
+                });
+            return statusMap;
+          }
+        };
+    fetchSchemaBlackListTask.execute();
+    if (isFailed()) {
+      return null;
+    }
+
+    Map<Integer, TFetchSchemaBlackListResp> respMap = fetchSchemaBlackListTask.getResponseMap();
+    PathPatternTree patternTree = new PathPatternTree();
+    for (TFetchSchemaBlackListResp resp : respMap.values()) {
+      for (PartialPath path :
+          PathPatternTree.deserialize(ByteBuffer.wrap(resp.getPathPatternTree()))
+              .getAllPathPatterns()) {
+        patternTree.appendPathPattern(path);
+      }
+    }
+    patternTree.constructTree();
+    return patternTree;
+  }
+
+  private void deleteTimeSeries(ConfigNodeProcedureEnv env) {
+    RegionTask<TSStatus> deleteTimeSeriesTask =
+        new RegionTask<TSStatus>(
+            "delete timeseries",
+            env,
+            env.getConfigManager().getRelatedSchemaRegionGroup(patternTree)) {
+          @Override
+          Map<Integer, TSStatus> sendRequest(
+              TDataNodeLocation dataNodeLocation, List<TConsensusGroupId> consensusGroupIdList) {
+            Map<Integer, TDataNodeLocation> dataNodeLocationMap = new HashMap<>();
+            dataNodeLocationMap.put(dataNodeLocation.getDataNodeId(), dataNodeLocation);
+            DeleteTimeSeriesHandler handler = new DeleteTimeSeriesHandler(dataNodeLocationMap);
+            AsyncDataNodeClientPool.getInstance()
+                .sendAsyncRequestToDataNodeWithRetry(
+                    new TDeleteTimeSeriesReq(consensusGroupIdList, patternTreeBytes), handler);
+            return handler.getDataNodeResponseStatusMap();
+          }
+        };
+    deleteTimeSeriesTask.execute();
+  }
+
+  private Map<TDataNodeLocation, List<TConsensusGroupId>> getLeaderDataNodeRegionGroupMap(
+      Map<TConsensusGroupId, Integer> leaderMap,
+      Map<TConsensusGroupId, TRegionReplicaSet> regionReplicaSetMap) {
+    Map<TDataNodeLocation, List<TConsensusGroupId>> dataNodeConsensusGroupIdMap = new HashMap<>();
+    regionReplicaSetMap.forEach(
+        (consensusGroupId, regionReplicaSet) -> {
+          Integer leaderId = leaderMap.get(consensusGroupId);
+          TDataNodeLocation leaderDataNodeLocation = null;
+          if (leaderId == null || leaderId == -1) {
+            leaderDataNodeLocation = regionReplicaSet.getDataNodeLocations().get(0);
+          } else {
+            for (TDataNodeLocation dataNodeLocation : regionReplicaSet.getDataNodeLocations()) {
+              if (dataNodeLocation.getDataNodeId() == leaderId) {
+                leaderDataNodeLocation = dataNodeLocation;
+                break;
+              }
+            }
+          }
+          dataNodeConsensusGroupIdMap
+              .computeIfAbsent(leaderDataNodeLocation, k -> new ArrayList<>())
+              .add(regionReplicaSet.getRegionId());
+        });
+    return dataNodeConsensusGroupIdMap;
+  }
+
+  private Map<TDataNodeLocation, List<TConsensusGroupId>> getAllReplicaDataNodeRegionGroupMap(
+      Map<TConsensusGroupId, TRegionReplicaSet> regionReplicaSetMap) {
+    Map<TDataNodeLocation, List<TConsensusGroupId>> dataNodeConsensusGroupIdMap = new HashMap<>();
+    regionReplicaSetMap.forEach(
+        (consensusGroupId, regionReplicaSet) -> {
+          for (TDataNodeLocation dataNodeLocation : regionReplicaSet.getDataNodeLocations()) {
+            dataNodeConsensusGroupIdMap
+                .computeIfAbsent(dataNodeLocation, k -> new ArrayList<>())
+                .add(regionReplicaSet.getRegionId());
+          }
+        });
+    return dataNodeConsensusGroupIdMap;
+  }
+
+  @Override
+  protected void rollbackState(
+      ConfigNodeProcedureEnv env, DeleteTimeSeriesState deleteTimeSeriesState)
+      throws IOException, InterruptedException, ProcedureException {
+    RegionTask<TSStatus> rollbackStateTask =
+        new RegionTask<TSStatus>(
+            "roll back schema black list",
+            env,

Review Comment:
   Since schema black list is constructed before all states, it must be rollback to make related timeseries usable again, when any state failed.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [iotdb] MarcosZyk commented on a diff in pull request #7242: [IOTDB-3184] Implement Delete Timeseries for Cluster

Posted by GitBox <gi...@apache.org>.
MarcosZyk commented on code in PR #7242:
URL: https://github.com/apache/iotdb/pull/7242#discussion_r969525686


##########
confignode/src/main/java/org/apache/iotdb/confignode/procedure/impl/DeleteTimeSeriesProcedure.java:
##########
@@ -0,0 +1,611 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.confignode.procedure.impl;
+
+import org.apache.iotdb.common.rpc.thrift.TConsensusGroupId;
+import org.apache.iotdb.common.rpc.thrift.TDataNodeLocation;
+import org.apache.iotdb.common.rpc.thrift.TRegionReplicaSet;
+import org.apache.iotdb.common.rpc.thrift.TSStatus;
+import org.apache.iotdb.commons.path.PartialPath;
+import org.apache.iotdb.confignode.client.async.datanode.AsyncDataNodeClientPool;
+import org.apache.iotdb.confignode.client.async.handlers.ConstructSchemaBlackListHandler;
+import org.apache.iotdb.confignode.client.async.handlers.DeleteDataForDeleteTimeSeriesHandler;
+import org.apache.iotdb.confignode.client.async.handlers.DeleteTimeSeriesHandler;
+import org.apache.iotdb.confignode.client.async.handlers.FetchSchemaBlackLsitHandler;
+import org.apache.iotdb.confignode.client.async.handlers.InvalidateMatchedSchemaCacheHandler;
+import org.apache.iotdb.confignode.client.async.handlers.RollbackSchemaBlackListHandler;
+import org.apache.iotdb.confignode.procedure.StateMachineProcedure;
+import org.apache.iotdb.confignode.procedure.env.ConfigNodeProcedureEnv;
+import org.apache.iotdb.confignode.procedure.exception.ProcedureException;
+import org.apache.iotdb.confignode.procedure.exception.ProcedureSuspendedException;
+import org.apache.iotdb.confignode.procedure.exception.ProcedureYieldException;
+import org.apache.iotdb.confignode.procedure.state.DeleteTimeSeriesState;
+import org.apache.iotdb.confignode.procedure.store.ProcedureFactory;
+import org.apache.iotdb.db.exception.metadata.PathNotExistException;
+import org.apache.iotdb.db.mpp.common.schematree.PathPatternTree;
+import org.apache.iotdb.mpp.rpc.thrift.TConstructSchemaBlackListReq;
+import org.apache.iotdb.mpp.rpc.thrift.TDeleteDataForDeleteTimeSeriesReq;
+import org.apache.iotdb.mpp.rpc.thrift.TDeleteTimeSeriesReq;
+import org.apache.iotdb.mpp.rpc.thrift.TFetchSchemaBlackListReq;
+import org.apache.iotdb.mpp.rpc.thrift.TFetchSchemaBlackListResp;
+import org.apache.iotdb.mpp.rpc.thrift.TInvalidateMatchedSchemaCacheReq;
+import org.apache.iotdb.mpp.rpc.thrift.TRollbackSchemaBlackListReq;
+import org.apache.iotdb.rpc.TSStatusCode;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.ByteArrayOutputStream;
+import java.io.DataOutputStream;
+import java.io.IOException;
+import java.nio.ByteBuffer;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.stream.Collectors;
+
+public class DeleteTimeSeriesProcedure
+    extends StateMachineProcedure<ConfigNodeProcedureEnv, DeleteTimeSeriesState> {
+
+  private static final Logger LOGGER = LoggerFactory.getLogger(DeleteTimeSeriesProcedure.class);
+
+  private PathPatternTree patternTree;
+  private ByteBuffer patternTreeBytes;
+
+  private String requestMessage;
+
+  public DeleteTimeSeriesProcedure() {
+    super();
+  }
+
+  public DeleteTimeSeriesProcedure(PathPatternTree patternTree) {
+    super();
+    setPatternTree(patternTree);
+  }
+
+  @Override
+  protected Flow executeFromState(ConfigNodeProcedureEnv env, DeleteTimeSeriesState state)
+      throws ProcedureSuspendedException, ProcedureYieldException, InterruptedException {
+    switch (state) {
+      case CONSTRUCT_BLACK_LIST:
+        LOGGER.info("Construct schema black list of timeseries {}", requestMessage);
+        if (constructBlackList(env) > 0) {
+          setNextState(DeleteTimeSeriesState.CLEAN_DATANODE_SCHEMA_CACHE);
+          break;
+        } else {
+          setFailure(
+              new ProcedureException(
+                  new PathNotExistException(
+                      patternTree.getAllPathPatterns().stream()
+                          .map(PartialPath::getFullPath)
+                          .collect(Collectors.toList()))));
+          return Flow.NO_MORE_STATE;
+        }
+      case CLEAN_DATANODE_SCHEMA_CACHE:
+        LOGGER.info("Invalidate cache of timeseries {}", requestMessage);
+        invalidateCache(env);
+        break;
+      case DELETE_DATA:
+        LOGGER.info("Delete data of timeseries {}", requestMessage);
+        deleteData(env);
+        break;
+      case DELETE_TIMESERIES:
+        LOGGER.info("Delete timeseries schema of {}", requestMessage);
+        deleteTimeSeries(env);
+        return Flow.NO_MORE_STATE;
+      default:
+        setFailure(new ProcedureException("Unrecognized state " + state.toString()));
+        return Flow.NO_MORE_STATE;
+    }
+    return Flow.HAS_MORE_STATE;
+  }
+
+  private int constructBlackList(ConfigNodeProcedureEnv env) {
+    Map<TConsensusGroupId, TRegionReplicaSet> targetSchemaRegionGroup =
+        env.getConfigManager().getRelatedSchemaRegionGroup(patternTree);
+    if (targetSchemaRegionGroup.isEmpty()) {
+      return 0;
+    }
+    RegionTask<TSStatus> constructBlackListTask =
+        new RegionTask<TSStatus>("construct schema black list", env, targetSchemaRegionGroup) {
+          @Override
+          Map<Integer, TSStatus> sendRequest(
+              TDataNodeLocation dataNodeLocation, List<TConsensusGroupId> consensusGroupIdList) {
+            // construct request and send
+            Map<Integer, TDataNodeLocation> dataNodeLocationMap = new HashMap<>();
+            dataNodeLocationMap.put(dataNodeLocation.getDataNodeId(), dataNodeLocation);
+            ConstructSchemaBlackListHandler handler =
+                new ConstructSchemaBlackListHandler(dataNodeLocationMap);
+            AsyncDataNodeClientPool.getInstance()
+                .sendAsyncRequestToDataNodeWithRetry(
+                    new TConstructSchemaBlackListReq(consensusGroupIdList, patternTreeBytes),
+                    handler);
+            handler
+                .getDataNodeResponseStatusMap()
+                .forEach(
+                    (k, v) -> {
+                      if (v.getCode() == TSStatusCode.SUCCESS_STATUS.getStatusCode()) {
+                        responseMap.put(k, v);
+                      }
+                    });
+            return handler.getDataNodeResponseStatusMap();
+          }
+        };
+    constructBlackListTask.execute();
+
+    if (isFailed()) {
+      return 0;
+    }
+
+    int preDeletedNum = 0;
+    for (TSStatus resp : constructBlackListTask.getResponseMap().values()) {
+      preDeletedNum += Integer.parseInt(resp.getMessage());
+    }
+    return preDeletedNum;
+  }
+
+  private void invalidateCache(ConfigNodeProcedureEnv env) {
+    Map<Integer, TDataNodeLocation> dataNodeLocationMap =
+        env.getConfigManager().getNodeManager().getRegisteredDataNodeLocations();
+    InvalidateMatchedSchemaCacheHandler handler =
+        new InvalidateMatchedSchemaCacheHandler(dataNodeLocationMap);
+    AsyncDataNodeClientPool.getInstance()
+        .sendAsyncRequestToDataNodeWithRetry(
+            new TInvalidateMatchedSchemaCacheReq(patternTreeBytes), handler);
+    Map<Integer, TSStatus> statusMap = handler.getDataNodeResponseStatusMap();
+    for (TSStatus status : statusMap.values()) {
+      // all dataNodes must clear the related schema cache
+      if (status.getCode() != TSStatusCode.SUCCESS_STATUS.getStatusCode()) {
+        LOGGER.error("Failed to invalidate schema cache of timeseries {}", requestMessage);
+        setFailure(new ProcedureException("Invalidate schema cache failed"));
+        return;
+      }
+    }
+
+    setNextState(DeleteTimeSeriesState.DELETE_DATA);
+  }
+
+  private void deleteData(ConfigNodeProcedureEnv env) {
+    Map<TConsensusGroupId, TRegionReplicaSet> relatedSchemaRegionGroup =
+        env.getConfigManager().getRelatedSchemaRegionGroup(patternTree);
+    Map<TDataNodeLocation, List<TConsensusGroupId>> dataNodeSchemaRegionGroupGroupIdMap =
+        getLeaderDataNodeRegionGroupMap(
+            env.getConfigManager().getPartitionManager().getAllLeadership(),
+            relatedSchemaRegionGroup);
+
+    // fetch schema black list by dataNode
+    for (Map.Entry<TDataNodeLocation, List<TConsensusGroupId>> entry :
+        dataNodeSchemaRegionGroupGroupIdMap.entrySet()) {
+      Map<TConsensusGroupId, TRegionReplicaSet> targetSchemaRegionGroup = new HashMap<>();
+      entry
+          .getValue()
+          .forEach(
+              consensusGroupId ->
+                  targetSchemaRegionGroup.put(
+                      consensusGroupId, relatedSchemaRegionGroup.get(consensusGroupId)));
+      PathPatternTree patternTree =
+          fetchSchemaBlackListOnTargetDataNode(env, targetSchemaRegionGroup);

Review Comment:
   We need to resolve the pathPattern into full path representing timeseries, or at list device path based pathPattern. The storage engine need the accurate path to release memory resources, since they are organized by device path.  
   
   Current implementation is to resolve pathPattern to accurate timeseries directly, which protect the concurrent new created timeseries from being affected by current delete timeseries task.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [iotdb] MarcosZyk commented on a diff in pull request #7242: [IOTDB-3184] Implement Delete Timeseries for Cluster

Posted by GitBox <gi...@apache.org>.
MarcosZyk commented on code in PR #7242:
URL: https://github.com/apache/iotdb/pull/7242#discussion_r969512869


##########
confignode/src/main/java/org/apache/iotdb/confignode/client/async/datanode/AsyncDataNodeClientPool.java:
##########
@@ -183,6 +224,31 @@ public void sendAsyncRequestToDataNodeWithRetry(
     }
   }
 
+  public void sendAsyncRequestToDataNodeWithRetry(Object req, AbstractRetryHandler handler) {
+    Map<Integer, TDataNodeLocation> dataNodeLocationMap = handler.getDataNodeLocationMap();
+    if (dataNodeLocationMap.isEmpty()) {
+      return;
+    }
+    for (int retry = 0; retry < MAX_RETRY_NUM; retry++) {
+      CountDownLatch countDownLatch = new CountDownLatch(dataNodeLocationMap.size());
+      handler.setCountDownLatch(countDownLatch);
+      for (TDataNodeLocation targetDataNode : dataNodeLocationMap.values()) {
+        handler.setTargetDataNode(targetDataNode);
+        sendAsyncRequestToDataNode(targetDataNode, req, handler, retry);
+      }
+
+      try {
+        countDownLatch.await();
+      } catch (InterruptedException e) {
+        LOGGER.error("Interrupted during {} on ConfigNode", handler.getDataNodeRequestType());
+      }
+      // Check if there is a node that fails to send the request, and retry if there is one
+      if (dataNodeLocationMap.isEmpty()) {
+        break;
+      }
+    }

Review Comment:
   Fixed. PTAL



##########
confignode/src/main/java/org/apache/iotdb/confignode/client/async/handlers/ConstructSchemaBlackListHandler.java:
##########
@@ -0,0 +1,99 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.confignode.client.async.handlers;
+
+import org.apache.iotdb.common.rpc.thrift.TDataNodeLocation;
+import org.apache.iotdb.common.rpc.thrift.TSStatus;
+import org.apache.iotdb.confignode.client.DataNodeRequestType;
+import org.apache.iotdb.rpc.RpcUtils;
+import org.apache.iotdb.rpc.TSStatusCode;
+
+import org.apache.thrift.async.AsyncMethodCallback;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.util.HashMap;
+import java.util.Map;
+import java.util.concurrent.CountDownLatch;
+
+public class ConstructSchemaBlackListHandler extends AbstractRetryHandler
+    implements AsyncMethodCallback<TSStatus> {
+
+  private static final Logger LOGGER =
+      LoggerFactory.getLogger(ConstructSchemaBlackListHandler.class);
+
+  private Map<Integer, TSStatus> dataNodeResponseStatusMap = new HashMap<>();
+
+  public ConstructSchemaBlackListHandler(Map<Integer, TDataNodeLocation> dataNodeLocationMap) {
+    super(DataNodeRequestType.CONSTRUCT_SCHEMA_BLACK_LIST, dataNodeLocationMap);
+  }
+
+  public ConstructSchemaBlackListHandler(
+      CountDownLatch countDownLatch,
+      TDataNodeLocation targetDataNode,
+      Map<Integer, TDataNodeLocation> dataNodeLocationMap) {
+    super(
+        countDownLatch,
+        DataNodeRequestType.CONSTRUCT_SCHEMA_BLACK_LIST,
+        targetDataNode,
+        dataNodeLocationMap);
+  }
+
+  public Map<Integer, TSStatus> getDataNodeResponseStatusMap() {
+    return dataNodeResponseStatusMap;
+  }
+
+  @Override
+  public void onComplete(TSStatus tsStatus) {
+    dataNodeResponseStatusMap.put(targetDataNode.getDataNodeId(), tsStatus);
+    if (tsStatus.getCode() == TSStatusCode.SUCCESS_STATUS.getStatusCode()) {
+      dataNodeLocationMap.remove(targetDataNode.getDataNodeId());
+      LOGGER.info("Successfully construct schema black list on DataNode: {}", targetDataNode);
+    } else if (tsStatus.getCode() == TSStatusCode.MULTIPLE_ERROR.getStatusCode()) {
+      dataNodeLocationMap.remove(targetDataNode.getDataNodeId());
+      LOGGER.error(
+          "Failed to construct schema black list on DataNode {}, {}",
+          dataNodeLocationMap.get(targetDataNode.getDataNodeId()),
+          tsStatus);

Review Comment:
   Fixed.



##########
confignode/src/main/java/org/apache/iotdb/confignode/client/async/handlers/DeleteDataForDeleteTimeSeriesHandler.java:
##########
@@ -0,0 +1,99 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.confignode.client.async.handlers;
+
+import org.apache.iotdb.common.rpc.thrift.TDataNodeLocation;
+import org.apache.iotdb.common.rpc.thrift.TSStatus;
+import org.apache.iotdb.confignode.client.DataNodeRequestType;
+import org.apache.iotdb.rpc.RpcUtils;
+import org.apache.iotdb.rpc.TSStatusCode;
+
+import org.apache.thrift.async.AsyncMethodCallback;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.util.HashMap;
+import java.util.Map;
+import java.util.concurrent.CountDownLatch;
+
+public class DeleteDataForDeleteTimeSeriesHandler extends AbstractRetryHandler
+    implements AsyncMethodCallback<TSStatus> {
+
+  private static final Logger LOGGER =
+      LoggerFactory.getLogger(DeleteDataForDeleteTimeSeriesHandler.class);
+
+  private Map<Integer, TSStatus> dataNodeResponseStatusMap = new HashMap<>();
+
+  public DeleteDataForDeleteTimeSeriesHandler(Map<Integer, TDataNodeLocation> dataNodeLocationMap) {
+    super(DataNodeRequestType.DELETE_DATA_FOR_DELETE_TIMESERIES, dataNodeLocationMap);
+  }
+
+  public DeleteDataForDeleteTimeSeriesHandler(
+      CountDownLatch countDownLatch,
+      TDataNodeLocation targetDataNode,
+      Map<Integer, TDataNodeLocation> dataNodeLocationMap) {
+    super(
+        countDownLatch,
+        DataNodeRequestType.DELETE_DATA_FOR_DELETE_TIMESERIES,
+        targetDataNode,
+        dataNodeLocationMap);
+  }
+
+  public Map<Integer, TSStatus> getDataNodeResponseStatusMap() {
+    return dataNodeResponseStatusMap;
+  }
+
+  @Override
+  public void onComplete(TSStatus tsStatus) {
+    dataNodeResponseStatusMap.put(targetDataNode.getDataNodeId(), tsStatus);
+    if (tsStatus.getCode() == TSStatusCode.SUCCESS_STATUS.getStatusCode()) {
+      dataNodeLocationMap.remove(targetDataNode.getDataNodeId());
+      LOGGER.info("Successfully delete data for delete timeseries on DataNode: {}", targetDataNode);
+    } else if (tsStatus.getCode() == TSStatusCode.MULTIPLE_ERROR.getStatusCode()) {
+      dataNodeLocationMap.remove(targetDataNode.getDataNodeId());
+      LOGGER.error(
+          "Failed to delete data for delete timeseries on DataNode {}, {}",
+          dataNodeLocationMap.get(targetDataNode.getDataNodeId()),
+          tsStatus);

Review Comment:
   Fixed.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [iotdb] MarcosZyk commented on a diff in pull request #7242: [IOTDB-3184] Implement Delete Timeseries for Cluster

Posted by GitBox <gi...@apache.org>.
MarcosZyk commented on code in PR #7242:
URL: https://github.com/apache/iotdb/pull/7242#discussion_r970204953


##########
server/src/main/java/org/apache/iotdb/db/service/thrift/impl/DataNodeInternalRPCServiceImpl.java:
##########
@@ -137,8 +146,11 @@ public class DataNodeInternalRPCServiceImpl implements IDataNodeRPCService.Iface
   private final SchemaEngine schemaEngine = SchemaEngine.getInstance();
   private final StorageEngineV2 storageEngine = StorageEngineV2.getInstance();
 
+  private final RegionManager regionManager;
+
   public DataNodeInternalRPCServiceImpl() {
     super();
+    regionManager = new RegionManager(schemaEngine, storageEngine);

Review Comment:
   DataNodeInternalRPCServiceImpl is a global unique instance and has the same life cycle as IoTDB.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [iotdb] CRZbulabula commented on a diff in pull request #7242: [IOTDB-3184] Implement Delete Timeseries for Cluster

Posted by GitBox <gi...@apache.org>.
CRZbulabula commented on code in PR #7242:
URL: https://github.com/apache/iotdb/pull/7242#discussion_r969072640


##########
confignode/src/main/java/org/apache/iotdb/confignode/manager/ConfigManager.java:
##########
@@ -966,4 +969,80 @@ public TGetPathsSetTemplatesResp getPathsSetTemplate(String req) {
       return new TGetPathsSetTemplatesResp(status);
     }
   }
+
+  @Override
+  public TSStatus deleteTimeSeries(PathPatternTree req) {
+    TSStatus status = confirmLeader();
+    if (status.getCode() == TSStatusCode.SUCCESS_STATUS.getStatusCode()) {
+      return procedureManager.deleteTimeSeries(req);
+    } else {
+      return status;
+    }
+  }
+
+  /** Get all related schemaRegion which may contains the timeseries matched by given patternTree */
+  public Map<TConsensusGroupId, TRegionReplicaSet> getRelatedSchemaRegionGroup(
+      PathPatternTree patternTree) {
+    Map<String, Map<TSeriesPartitionSlot, TConsensusGroupId>> schemaPartitionTable =
+        getSchemaPartition(patternTree).getSchemaPartitionTable();
+
+    List<TRegionReplicaSet> allRegionReplicaSets = getPartitionManager().getAllReplicaSets();
+    Set<TConsensusGroupId> groupIdSet =
+        schemaPartitionTable.values().stream()
+            .flatMap(m -> m.values().stream())
+            .collect(Collectors.toSet());
+    Map<TConsensusGroupId, TRegionReplicaSet> filteredRegionReplicaSets = new HashMap<>();
+    for (TRegionReplicaSet regionReplicaSet : allRegionReplicaSets) {
+      if (groupIdSet.contains(regionReplicaSet.getRegionId())) {
+        filteredRegionReplicaSets.put(regionReplicaSet.getRegionId(), regionReplicaSet);
+      }
+    }
+    return filteredRegionReplicaSets;
+  }
+
+  /**
+   * Get all related dataRegion which may contains the data of specific timeseries matched by given
+   * patternTree
+   */
+  public Map<TConsensusGroupId, TRegionReplicaSet> getRelatedDataRegionGroup(
+      PathPatternTree patternTree) {
+    // get all storage group and slot by getting schema partition
+    Map<String, Map<TSeriesPartitionSlot, TConsensusGroupId>> schemaPartitionTable =
+        getSchemaPartition(patternTree).getSchemaPartitionTable();
+
+    // construct request for getting data partition
+    Map<String, Map<TSeriesPartitionSlot, List<TTimePartitionSlot>>> partitionSlotsMap =
+        new HashMap<>();
+    schemaPartitionTable.forEach(
+        (key, value) -> {
+          Map<TSeriesPartitionSlot, List<TTimePartitionSlot>> slotListMap = new HashMap<>();
+          value.keySet().forEach(slot -> slotListMap.put(slot, Collections.emptyList()));
+          partitionSlotsMap.put(key, slotListMap);
+        });
+
+    // get all data partitions
+    GetDataPartitionPlan getDataPartitionPlan = new GetDataPartitionPlan(partitionSlotsMap);
+    Map<String, Map<TSeriesPartitionSlot, Map<TTimePartitionSlot, List<TConsensusGroupId>>>>
+        dataPartitionTable = getDataPartition(getDataPartitionPlan).getDataPartitionTable();
+
+    // get all region replicaset of target data partitions
+    List<TRegionReplicaSet> allRegionReplicaSets = getPartitionManager().getAllReplicaSets();
+    Set<TConsensusGroupId> groupIdSet =
+        dataPartitionTable.values().stream()
+            .flatMap(
+                tSeriesPartitionSlotMapMap ->
+                    tSeriesPartitionSlotMapMap.values().stream()
+                        .flatMap(
+                            tTimePartitionSlotListMap ->
+                                tTimePartitionSlotListMap.values().stream()
+                                    .flatMap(Collection::stream)))
+            .collect(Collectors.toSet());
+    Map<TConsensusGroupId, TRegionReplicaSet> filteredRegionReplicaSets = new HashMap<>();
+    for (TRegionReplicaSet regionReplicaSet : allRegionReplicaSets) {
+      if (groupIdSet.contains(regionReplicaSet.getRegionId())) {
+        filteredRegionReplicaSets.put(regionReplicaSet.getRegionId(), regionReplicaSet);
+      }
+    }
+    return filteredRegionReplicaSets;
+  }

Review Comment:
   It's better to move these two interfaces into the PartitionManager



##########
confignode/src/main/java/org/apache/iotdb/confignode/client/async/handlers/ConstructSchemaBlackListHandler.java:
##########
@@ -0,0 +1,99 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.confignode.client.async.handlers;
+
+import org.apache.iotdb.common.rpc.thrift.TDataNodeLocation;
+import org.apache.iotdb.common.rpc.thrift.TSStatus;
+import org.apache.iotdb.confignode.client.DataNodeRequestType;
+import org.apache.iotdb.rpc.RpcUtils;
+import org.apache.iotdb.rpc.TSStatusCode;
+
+import org.apache.thrift.async.AsyncMethodCallback;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.util.HashMap;
+import java.util.Map;
+import java.util.concurrent.CountDownLatch;
+
+public class ConstructSchemaBlackListHandler extends AbstractRetryHandler
+    implements AsyncMethodCallback<TSStatus> {
+
+  private static final Logger LOGGER =
+      LoggerFactory.getLogger(ConstructSchemaBlackListHandler.class);
+
+  private Map<Integer, TSStatus> dataNodeResponseStatusMap = new HashMap<>();
+
+  public ConstructSchemaBlackListHandler(Map<Integer, TDataNodeLocation> dataNodeLocationMap) {
+    super(DataNodeRequestType.CONSTRUCT_SCHEMA_BLACK_LIST, dataNodeLocationMap);
+  }
+
+  public ConstructSchemaBlackListHandler(
+      CountDownLatch countDownLatch,
+      TDataNodeLocation targetDataNode,
+      Map<Integer, TDataNodeLocation> dataNodeLocationMap) {
+    super(
+        countDownLatch,
+        DataNodeRequestType.CONSTRUCT_SCHEMA_BLACK_LIST,
+        targetDataNode,
+        dataNodeLocationMap);
+  }
+
+  public Map<Integer, TSStatus> getDataNodeResponseStatusMap() {
+    return dataNodeResponseStatusMap;
+  }
+
+  @Override
+  public void onComplete(TSStatus tsStatus) {
+    dataNodeResponseStatusMap.put(targetDataNode.getDataNodeId(), tsStatus);
+    if (tsStatus.getCode() == TSStatusCode.SUCCESS_STATUS.getStatusCode()) {
+      dataNodeLocationMap.remove(targetDataNode.getDataNodeId());
+      LOGGER.info("Successfully construct schema black list on DataNode: {}", targetDataNode);
+    } else if (tsStatus.getCode() == TSStatusCode.MULTIPLE_ERROR.getStatusCode()) {
+      dataNodeLocationMap.remove(targetDataNode.getDataNodeId());
+      LOGGER.error(
+          "Failed to construct schema black list on DataNode {}, {}",
+          dataNodeLocationMap.get(targetDataNode.getDataNodeId()),
+          tsStatus);

Review Comment:
   You'll get a NPE if you remove before get T^T



##########
confignode/src/main/java/org/apache/iotdb/confignode/client/async/datanode/AsyncDataNodeClientPool.java:
##########
@@ -183,6 +224,31 @@ public void sendAsyncRequestToDataNodeWithRetry(
     }
   }
 
+  public void sendAsyncRequestToDataNodeWithRetry(Object req, AbstractRetryHandler handler) {
+    Map<Integer, TDataNodeLocation> dataNodeLocationMap = handler.getDataNodeLocationMap();
+    if (dataNodeLocationMap.isEmpty()) {
+      return;
+    }
+    for (int retry = 0; retry < MAX_RETRY_NUM; retry++) {
+      CountDownLatch countDownLatch = new CountDownLatch(dataNodeLocationMap.size());
+      handler.setCountDownLatch(countDownLatch);
+      for (TDataNodeLocation targetDataNode : dataNodeLocationMap.values()) {
+        handler.setTargetDataNode(targetDataNode);
+        sendAsyncRequestToDataNode(targetDataNode, req, handler, retry);
+      }
+
+      try {
+        countDownLatch.await();
+      } catch (InterruptedException e) {
+        LOGGER.error("Interrupted during {} on ConfigNode", handler.getDataNodeRequestType());
+      }
+      // Check if there is a node that fails to send the request, and retry if there is one
+      if (dataNodeLocationMap.isEmpty()) {
+        break;
+      }
+    }

Review Comment:
   Normally, we construct a new handler for every asynchronize request instead of using the same one. Otherwise, you should add concurrency control in your handler~



##########
confignode/src/main/java/org/apache/iotdb/confignode/procedure/impl/DeleteTimeSeriesProcedure.java:
##########
@@ -0,0 +1,611 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.confignode.procedure.impl;
+
+import org.apache.iotdb.common.rpc.thrift.TConsensusGroupId;
+import org.apache.iotdb.common.rpc.thrift.TDataNodeLocation;
+import org.apache.iotdb.common.rpc.thrift.TRegionReplicaSet;
+import org.apache.iotdb.common.rpc.thrift.TSStatus;
+import org.apache.iotdb.commons.path.PartialPath;
+import org.apache.iotdb.confignode.client.async.datanode.AsyncDataNodeClientPool;
+import org.apache.iotdb.confignode.client.async.handlers.ConstructSchemaBlackListHandler;
+import org.apache.iotdb.confignode.client.async.handlers.DeleteDataForDeleteTimeSeriesHandler;
+import org.apache.iotdb.confignode.client.async.handlers.DeleteTimeSeriesHandler;
+import org.apache.iotdb.confignode.client.async.handlers.FetchSchemaBlackLsitHandler;
+import org.apache.iotdb.confignode.client.async.handlers.InvalidateMatchedSchemaCacheHandler;
+import org.apache.iotdb.confignode.client.async.handlers.RollbackSchemaBlackListHandler;
+import org.apache.iotdb.confignode.procedure.StateMachineProcedure;
+import org.apache.iotdb.confignode.procedure.env.ConfigNodeProcedureEnv;
+import org.apache.iotdb.confignode.procedure.exception.ProcedureException;
+import org.apache.iotdb.confignode.procedure.exception.ProcedureSuspendedException;
+import org.apache.iotdb.confignode.procedure.exception.ProcedureYieldException;
+import org.apache.iotdb.confignode.procedure.state.DeleteTimeSeriesState;
+import org.apache.iotdb.confignode.procedure.store.ProcedureFactory;
+import org.apache.iotdb.db.exception.metadata.PathNotExistException;
+import org.apache.iotdb.db.mpp.common.schematree.PathPatternTree;
+import org.apache.iotdb.mpp.rpc.thrift.TConstructSchemaBlackListReq;
+import org.apache.iotdb.mpp.rpc.thrift.TDeleteDataForDeleteTimeSeriesReq;
+import org.apache.iotdb.mpp.rpc.thrift.TDeleteTimeSeriesReq;
+import org.apache.iotdb.mpp.rpc.thrift.TFetchSchemaBlackListReq;
+import org.apache.iotdb.mpp.rpc.thrift.TFetchSchemaBlackListResp;
+import org.apache.iotdb.mpp.rpc.thrift.TInvalidateMatchedSchemaCacheReq;
+import org.apache.iotdb.mpp.rpc.thrift.TRollbackSchemaBlackListReq;
+import org.apache.iotdb.rpc.TSStatusCode;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.ByteArrayOutputStream;
+import java.io.DataOutputStream;
+import java.io.IOException;
+import java.nio.ByteBuffer;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.stream.Collectors;
+
+public class DeleteTimeSeriesProcedure
+    extends StateMachineProcedure<ConfigNodeProcedureEnv, DeleteTimeSeriesState> {
+
+  private static final Logger LOGGER = LoggerFactory.getLogger(DeleteTimeSeriesProcedure.class);
+
+  private PathPatternTree patternTree;
+  private ByteBuffer patternTreeBytes;
+
+  private String requestMessage;
+
+  public DeleteTimeSeriesProcedure() {
+    super();
+  }
+
+  public DeleteTimeSeriesProcedure(PathPatternTree patternTree) {
+    super();
+    setPatternTree(patternTree);
+  }
+
+  @Override
+  protected Flow executeFromState(ConfigNodeProcedureEnv env, DeleteTimeSeriesState state)
+      throws ProcedureSuspendedException, ProcedureYieldException, InterruptedException {
+    switch (state) {
+      case CONSTRUCT_BLACK_LIST:
+        LOGGER.info("Construct schema black list of timeseries {}", requestMessage);
+        if (constructBlackList(env) > 0) {
+          setNextState(DeleteTimeSeriesState.CLEAN_DATANODE_SCHEMA_CACHE);
+          break;
+        } else {
+          setFailure(
+              new ProcedureException(
+                  new PathNotExistException(
+                      patternTree.getAllPathPatterns().stream()
+                          .map(PartialPath::getFullPath)
+                          .collect(Collectors.toList()))));
+          return Flow.NO_MORE_STATE;
+        }
+      case CLEAN_DATANODE_SCHEMA_CACHE:
+        LOGGER.info("Invalidate cache of timeseries {}", requestMessage);
+        invalidateCache(env);
+        break;
+      case DELETE_DATA:
+        LOGGER.info("Delete data of timeseries {}", requestMessage);
+        deleteData(env);
+        break;
+      case DELETE_TIMESERIES:
+        LOGGER.info("Delete timeseries schema of {}", requestMessage);
+        deleteTimeSeries(env);
+        return Flow.NO_MORE_STATE;
+      default:
+        setFailure(new ProcedureException("Unrecognized state " + state.toString()));
+        return Flow.NO_MORE_STATE;
+    }
+    return Flow.HAS_MORE_STATE;
+  }
+
+  private int constructBlackList(ConfigNodeProcedureEnv env) {
+    Map<TConsensusGroupId, TRegionReplicaSet> targetSchemaRegionGroup =
+        env.getConfigManager().getRelatedSchemaRegionGroup(patternTree);
+    if (targetSchemaRegionGroup.isEmpty()) {
+      return 0;
+    }
+    RegionTask<TSStatus> constructBlackListTask =
+        new RegionTask<TSStatus>("construct schema black list", env, targetSchemaRegionGroup) {
+          @Override
+          Map<Integer, TSStatus> sendRequest(
+              TDataNodeLocation dataNodeLocation, List<TConsensusGroupId> consensusGroupIdList) {
+            // construct request and send
+            Map<Integer, TDataNodeLocation> dataNodeLocationMap = new HashMap<>();
+            dataNodeLocationMap.put(dataNodeLocation.getDataNodeId(), dataNodeLocation);
+            ConstructSchemaBlackListHandler handler =
+                new ConstructSchemaBlackListHandler(dataNodeLocationMap);
+            AsyncDataNodeClientPool.getInstance()
+                .sendAsyncRequestToDataNodeWithRetry(
+                    new TConstructSchemaBlackListReq(consensusGroupIdList, patternTreeBytes),
+                    handler);
+            handler
+                .getDataNodeResponseStatusMap()
+                .forEach(
+                    (k, v) -> {
+                      if (v.getCode() == TSStatusCode.SUCCESS_STATUS.getStatusCode()) {
+                        responseMap.put(k, v);
+                      }
+                    });
+            return handler.getDataNodeResponseStatusMap();
+          }
+        };
+    constructBlackListTask.execute();
+
+    if (isFailed()) {
+      return 0;
+    }
+
+    int preDeletedNum = 0;
+    for (TSStatus resp : constructBlackListTask.getResponseMap().values()) {
+      preDeletedNum += Integer.parseInt(resp.getMessage());
+    }
+    return preDeletedNum;
+  }
+
+  private void invalidateCache(ConfigNodeProcedureEnv env) {
+    Map<Integer, TDataNodeLocation> dataNodeLocationMap =
+        env.getConfigManager().getNodeManager().getRegisteredDataNodeLocations();
+    InvalidateMatchedSchemaCacheHandler handler =
+        new InvalidateMatchedSchemaCacheHandler(dataNodeLocationMap);
+    AsyncDataNodeClientPool.getInstance()
+        .sendAsyncRequestToDataNodeWithRetry(
+            new TInvalidateMatchedSchemaCacheReq(patternTreeBytes), handler);
+    Map<Integer, TSStatus> statusMap = handler.getDataNodeResponseStatusMap();
+    for (TSStatus status : statusMap.values()) {
+      // all dataNodes must clear the related schema cache
+      if (status.getCode() != TSStatusCode.SUCCESS_STATUS.getStatusCode()) {
+        LOGGER.error("Failed to invalidate schema cache of timeseries {}", requestMessage);
+        setFailure(new ProcedureException("Invalidate schema cache failed"));
+        return;
+      }
+    }
+
+    setNextState(DeleteTimeSeriesState.DELETE_DATA);
+  }
+
+  private void deleteData(ConfigNodeProcedureEnv env) {
+    Map<TConsensusGroupId, TRegionReplicaSet> relatedSchemaRegionGroup =
+        env.getConfigManager().getRelatedSchemaRegionGroup(patternTree);
+    Map<TDataNodeLocation, List<TConsensusGroupId>> dataNodeSchemaRegionGroupGroupIdMap =
+        getLeaderDataNodeRegionGroupMap(
+            env.getConfigManager().getPartitionManager().getAllLeadership(),
+            relatedSchemaRegionGroup);
+
+    // fetch schema black list by dataNode
+    for (Map.Entry<TDataNodeLocation, List<TConsensusGroupId>> entry :
+        dataNodeSchemaRegionGroupGroupIdMap.entrySet()) {
+      Map<TConsensusGroupId, TRegionReplicaSet> targetSchemaRegionGroup = new HashMap<>();
+      entry
+          .getValue()
+          .forEach(
+              consensusGroupId ->
+                  targetSchemaRegionGroup.put(
+                      consensusGroupId, relatedSchemaRegionGroup.get(consensusGroupId)));
+      PathPatternTree patternTree =
+          fetchSchemaBlackListOnTargetDataNode(env, targetSchemaRegionGroup);
+      if (isFailed()) {
+        return;
+      }
+      if (patternTree == null) {
+        LOGGER.error(
+            "Failed to fetch schema black list for delete data of timeseries {} on {}",
+            requestMessage,
+            entry.getKey());
+        setFailure(new ProcedureException("Fetch schema black list forDelete data failed"));
+        return;
+      }
+
+      if (patternTree.isEmpty()) {
+        continue;
+      }
+
+      Map<TConsensusGroupId, TRegionReplicaSet> relatedDataRegionGroup =
+          env.getConfigManager().getRelatedDataRegionGroup(patternTree);
+
+      // target timeseries has no data
+      if (relatedDataRegionGroup.isEmpty()) {
+        continue;
+      }
+
+      RegionTask<TSStatus> deleteDataTask =
+          new RegionTask<TSStatus>("delete data", env, relatedDataRegionGroup) {
+            @Override
+            Map<Integer, TSStatus> sendRequest(
+                TDataNodeLocation dataNodeLocation, List<TConsensusGroupId> consensusGroupIdList) {
+              Map<Integer, TDataNodeLocation> dataNodeLocationMap = new HashMap<>();
+              dataNodeLocationMap.put(dataNodeLocation.getDataNodeId(), dataNodeLocation);
+              DeleteDataForDeleteTimeSeriesHandler handler =
+                  new DeleteDataForDeleteTimeSeriesHandler(dataNodeLocationMap);
+              AsyncDataNodeClientPool.getInstance()
+                  .sendAsyncRequestToDataNodeWithRetry(
+                      new TDeleteDataForDeleteTimeSeriesReq(
+                          new ArrayList<>(consensusGroupIdList),
+                          preparePatternTreeBytesData(patternTree)),
+                      handler);
+              return handler.getDataNodeResponseStatusMap();
+            }
+          };
+      deleteDataTask.setExecuteOnAllReplicaset(true);
+      deleteDataTask.execute();
+      if (isFailed()) {
+        return;
+      }
+    }
+    setNextState(DeleteTimeSeriesState.DELETE_TIMESERIES);
+  }
+
+  private PathPatternTree fetchSchemaBlackListOnTargetDataNode(
+      ConfigNodeProcedureEnv env,
+      Map<TConsensusGroupId, TRegionReplicaSet> targetSchemaRegionGroup) {
+    RegionTask<TFetchSchemaBlackListResp> fetchSchemaBlackListTask =
+        new RegionTask<TFetchSchemaBlackListResp>(
+            "fetch schema black list", env, targetSchemaRegionGroup) {
+          @Override
+          Map<Integer, TSStatus> sendRequest(
+              TDataNodeLocation dataNodeLocation, List<TConsensusGroupId> consensusGroupIdList) {
+            Map<Integer, TDataNodeLocation> dataNodeLocationMap = new HashMap<>();
+            dataNodeLocationMap.put(dataNodeLocation.getDataNodeId(), dataNodeLocation);
+            FetchSchemaBlackLsitHandler handler =
+                new FetchSchemaBlackLsitHandler(dataNodeLocationMap);
+            AsyncDataNodeClientPool.getInstance()
+                .sendAsyncRequestToDataNodeWithRetry(
+                    new TFetchSchemaBlackListReq(consensusGroupIdList, patternTreeBytes), handler);
+            Map<Integer, TFetchSchemaBlackListResp> respMap = handler.getDataNodeResponseMap();
+            Map<Integer, TSStatus> statusMap = new HashMap<>();
+            respMap.forEach(
+                (k, v) -> {
+                  if (v.getStatus().getCode() == TSStatusCode.SUCCESS_STATUS.getStatusCode()) {
+                    responseMap.put(k, v);
+                  }
+                  statusMap.put(k, v.getStatus());
+                });
+            return statusMap;
+          }
+        };
+    fetchSchemaBlackListTask.execute();
+    if (isFailed()) {
+      return null;
+    }
+
+    Map<Integer, TFetchSchemaBlackListResp> respMap = fetchSchemaBlackListTask.getResponseMap();
+    PathPatternTree patternTree = new PathPatternTree();
+    for (TFetchSchemaBlackListResp resp : respMap.values()) {
+      for (PartialPath path :
+          PathPatternTree.deserialize(ByteBuffer.wrap(resp.getPathPatternTree()))
+              .getAllPathPatterns()) {
+        patternTree.appendPathPattern(path);
+      }
+    }
+    patternTree.constructTree();
+    return patternTree;
+  }
+
+  private void deleteTimeSeries(ConfigNodeProcedureEnv env) {
+    RegionTask<TSStatus> deleteTimeSeriesTask =
+        new RegionTask<TSStatus>(
+            "delete timeseries",
+            env,
+            env.getConfigManager().getRelatedSchemaRegionGroup(patternTree)) {
+          @Override
+          Map<Integer, TSStatus> sendRequest(
+              TDataNodeLocation dataNodeLocation, List<TConsensusGroupId> consensusGroupIdList) {
+            Map<Integer, TDataNodeLocation> dataNodeLocationMap = new HashMap<>();
+            dataNodeLocationMap.put(dataNodeLocation.getDataNodeId(), dataNodeLocation);
+            DeleteTimeSeriesHandler handler = new DeleteTimeSeriesHandler(dataNodeLocationMap);
+            AsyncDataNodeClientPool.getInstance()
+                .sendAsyncRequestToDataNodeWithRetry(
+                    new TDeleteTimeSeriesReq(consensusGroupIdList, patternTreeBytes), handler);
+            return handler.getDataNodeResponseStatusMap();
+          }
+        };
+    deleteTimeSeriesTask.execute();
+  }
+
+  private Map<TDataNodeLocation, List<TConsensusGroupId>> getLeaderDataNodeRegionGroupMap(
+      Map<TConsensusGroupId, Integer> leaderMap,
+      Map<TConsensusGroupId, TRegionReplicaSet> regionReplicaSetMap) {
+    Map<TDataNodeLocation, List<TConsensusGroupId>> dataNodeConsensusGroupIdMap = new HashMap<>();
+    regionReplicaSetMap.forEach(
+        (consensusGroupId, regionReplicaSet) -> {
+          Integer leaderId = leaderMap.get(consensusGroupId);
+          TDataNodeLocation leaderDataNodeLocation = null;
+          if (leaderId == null || leaderId == -1) {
+            leaderDataNodeLocation = regionReplicaSet.getDataNodeLocations().get(0);
+          } else {
+            for (TDataNodeLocation dataNodeLocation : regionReplicaSet.getDataNodeLocations()) {
+              if (dataNodeLocation.getDataNodeId() == leaderId) {
+                leaderDataNodeLocation = dataNodeLocation;
+                break;
+              }
+            }
+          }
+          dataNodeConsensusGroupIdMap
+              .computeIfAbsent(leaderDataNodeLocation, k -> new ArrayList<>())
+              .add(regionReplicaSet.getRegionId());
+        });
+    return dataNodeConsensusGroupIdMap;
+  }
+
+  private Map<TDataNodeLocation, List<TConsensusGroupId>> getAllReplicaDataNodeRegionGroupMap(
+      Map<TConsensusGroupId, TRegionReplicaSet> regionReplicaSetMap) {
+    Map<TDataNodeLocation, List<TConsensusGroupId>> dataNodeConsensusGroupIdMap = new HashMap<>();
+    regionReplicaSetMap.forEach(
+        (consensusGroupId, regionReplicaSet) -> {
+          for (TDataNodeLocation dataNodeLocation : regionReplicaSet.getDataNodeLocations()) {
+            dataNodeConsensusGroupIdMap
+                .computeIfAbsent(dataNodeLocation, k -> new ArrayList<>())
+                .add(regionReplicaSet.getRegionId());
+          }
+        });
+    return dataNodeConsensusGroupIdMap;
+  }
+
+  @Override
+  protected void rollbackState(
+      ConfigNodeProcedureEnv env, DeleteTimeSeriesState deleteTimeSeriesState)
+      throws IOException, InterruptedException, ProcedureException {
+    RegionTask<TSStatus> rollbackStateTask =
+        new RegionTask<TSStatus>(
+            "roll back schema black list",
+            env,
+            env.getConfigManager().getRelatedSchemaRegionGroup(patternTree)) {
+          @Override
+          Map<Integer, TSStatus> sendRequest(
+              TDataNodeLocation dataNodeLocation, List<TConsensusGroupId> consensusGroupIdList) {
+            Map<Integer, TDataNodeLocation> dataNodeLocationMap = new HashMap<>();
+            dataNodeLocationMap.put(dataNodeLocation.getDataNodeId(), dataNodeLocation);
+            RollbackSchemaBlackListHandler handler =
+                new RollbackSchemaBlackListHandler(dataNodeLocationMap);
+            AsyncDataNodeClientPool.getInstance()
+                .sendAsyncRequestToDataNodeWithRetry(
+                    new TRollbackSchemaBlackListReq(consensusGroupIdList, patternTreeBytes),
+                    handler);
+            return handler.getDataNodeResponseStatusMap();
+          }
+        };
+    rollbackStateTask.execute();
+  }
+
+  @Override
+  protected boolean isRollbackSupported(DeleteTimeSeriesState deleteTimeSeriesState) {
+    return true;
+  }
+
+  @Override
+  protected DeleteTimeSeriesState getState(int stateId) {
+    return DeleteTimeSeriesState.values()[stateId];
+  }
+
+  @Override
+  protected int getStateId(DeleteTimeSeriesState deleteTimeSeriesState) {
+    return deleteTimeSeriesState.ordinal();
+  }
+
+  @Override
+  protected DeleteTimeSeriesState getInitialState() {
+    return DeleteTimeSeriesState.CONSTRUCT_BLACK_LIST;
+  }
+
+  public PathPatternTree getPatternTree() {
+    return patternTree;
+  }
+
+  public void setPatternTree(PathPatternTree patternTree) {
+    this.patternTree = patternTree;
+    requestMessage = patternTree.getAllPathPatterns().toString();
+    patternTreeBytes = preparePatternTreeBytesData(patternTree);
+  }
+
+  private ByteBuffer preparePatternTreeBytesData(PathPatternTree patternTree) {
+    ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream();
+    DataOutputStream dataOutputStream = new DataOutputStream(byteArrayOutputStream);
+    try {
+      patternTree.serialize(dataOutputStream);
+    } catch (IOException ignored) {
+
+    }
+    return ByteBuffer.wrap(byteArrayOutputStream.toByteArray());
+  }
+
+  @Override
+  public void serialize(DataOutputStream stream) throws IOException {
+    stream.writeInt(ProcedureFactory.ProcedureType.DELETE_TIMESERIES_PROCEDURE.ordinal());
+    super.serialize(stream);
+    patternTree.serialize(stream);
+  }
+
+  @Override
+  public void deserialize(ByteBuffer byteBuffer) {
+    super.deserialize(byteBuffer);
+    setPatternTree(PathPatternTree.deserialize(byteBuffer));
+  }
+
+  @Override
+  public boolean equals(Object o) {
+    if (this == o) return true;
+    if (o == null || getClass() != o.getClass()) return false;
+    DeleteTimeSeriesProcedure that = (DeleteTimeSeriesProcedure) o;
+    return this.getProcId() == that.getProcId()
+        && this.getState() == that.getState()
+        && patternTree.equals(that.patternTree);
+  }
+
+  @Override
+  public int hashCode() {
+    return Objects.hash(getProcId(), getState(), patternTree);
+  }
+
+  private abstract class RegionTask<T> {
+
+    private final String taskName;
+    private final ConfigNodeProcedureEnv env;
+    private final Map<TConsensusGroupId, TRegionReplicaSet> targetSchemaRegionGroup;
+
+    private boolean executeOnAllReplicaset = false;
+
+    protected Map<Integer, T> responseMap = new ConcurrentHashMap<>();
+
+    RegionTask(
+        String taskName,
+        ConfigNodeProcedureEnv env,
+        Map<TConsensusGroupId, TRegionReplicaSet> targetSchemaRegionGroup) {
+      this.taskName = taskName;
+      this.env = env;
+      this.targetSchemaRegionGroup = targetSchemaRegionGroup;
+    }
+
+    private void execute() {
+      // organize schema region by dataNode
+      Set<TDataNodeLocation> allFailedDataNodeSet = new HashSet<>();
+      Map<TDataNodeLocation, List<TConsensusGroupId>> dataNodeConsensusGroupIdMap =
+          executeOnAllReplicaset
+              ? getAllReplicaDataNodeRegionGroupMap(targetSchemaRegionGroup)
+              : getLeaderDataNodeRegionGroupMap(
+                  env.getConfigManager().getPartitionManager().getAllLeadership(),
+                  targetSchemaRegionGroup);
+      while (!dataNodeConsensusGroupIdMap.isEmpty()) {
+        Map<TDataNodeLocation, List<TConsensusGroupId>> currentFailedDataNodeMap =
+            sendSchemaRegionRequest(dataNodeConsensusGroupIdMap);
+        // some dataNode execution failure
+        if (isFailed()) {
+          return;
+        }
+
+        if (currentFailedDataNodeMap.isEmpty()) {
+          break;
+        }
+
+        // retry failed dataNode requests caused by unexpected error on other replicates on other
+        // dataNodes
+        currentFailedDataNodeMap.forEach(dataNodeConsensusGroupIdMap::remove);
+        // remove dataNodes that successfully executed request
+        allFailedDataNodeSet.removeAll(dataNodeConsensusGroupIdMap.keySet());
+        dataNodeConsensusGroupIdMap =
+            getAvailableDataNodeLocationForRetry(currentFailedDataNodeMap, allFailedDataNodeSet);
+        // some consensus group has no available dataNode
+        if (isFailed()) {
+          return;
+        }
+      }
+    }
+
+    private Map<TDataNodeLocation, List<TConsensusGroupId>> sendSchemaRegionRequest(
+        Map<TDataNodeLocation, List<TConsensusGroupId>> dataNodeConsensusGroupIdMap) {
+      // send request to each dataNode
+      Map<TDataNodeLocation, List<TConsensusGroupId>> failedDataNodeMap = new HashMap<>();
+      for (Map.Entry<TDataNodeLocation, List<TConsensusGroupId>> entry :
+          dataNodeConsensusGroupIdMap.entrySet()) {
+        // process response
+        Map<Integer, TSStatus> dataNodeResponseMap = sendRequest(entry.getKey(), entry.getValue());
+        TSStatus currentDataNodeResponse = dataNodeResponseMap.get(entry.getKey().getDataNodeId());
+        if (currentDataNodeResponse.getCode() != TSStatusCode.SUCCESS_STATUS.getStatusCode()) {
+          if (currentDataNodeResponse.getCode() == TSStatusCode.MULTIPLE_ERROR.getStatusCode()) {
+            // dataNode execution error
+            LOGGER.error(
+                "Failed to execute [{}] of delete timeseries {} on {}",
+                taskName,
+                requestMessage,
+                entry.getKey());
+            setFailure(
+                new ProcedureException(
+                    String.format(
+                        "Delete timeseries %s failed on when [%s]", requestMessage, taskName)));
+            break;
+          } else {
+            // unexpected error, retry on other replicates on other dataNodes
+            failedDataNodeMap.put(entry.getKey(), entry.getValue());
+          }
+        }
+      }
+      return failedDataNodeMap;
+    }
+
+    private Map<TDataNodeLocation, List<TConsensusGroupId>> getAvailableDataNodeLocationForRetry(
+        Map<TDataNodeLocation, List<TConsensusGroupId>> failedDataNodeConsensusGroupIdMap,
+        Set<TDataNodeLocation> allFailedDataNodeSet) {
+      Map<TConsensusGroupId, Integer> leaderMap =
+          env.getConfigManager().getPartitionManager().getAllLeadership();
+      Map<TDataNodeLocation, List<TConsensusGroupId>> availableDataNodeLocation = new HashMap<>();
+      for (List<TConsensusGroupId> consensusGroupIdList :
+          failedDataNodeConsensusGroupIdMap.values()) {
+        for (TConsensusGroupId consensusGroupId : consensusGroupIdList) {
+          TRegionReplicaSet regionReplicaSet = targetSchemaRegionGroup.get(consensusGroupId);
+          TDataNodeLocation selectedDataNode = null;
+          Integer leaderId = leaderMap.get(consensusGroupId);
+          if (leaderId == null || leaderId == -1) {

Review Comment:
   Better add annotation to response to the next annotation "retry on the new leader as possible".



##########
confignode/src/main/java/org/apache/iotdb/confignode/client/async/handlers/RollbackSchemaBlackListHandler.java:
##########
@@ -0,0 +1,99 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.confignode.client.async.handlers;
+
+import org.apache.iotdb.common.rpc.thrift.TDataNodeLocation;
+import org.apache.iotdb.common.rpc.thrift.TSStatus;
+import org.apache.iotdb.confignode.client.DataNodeRequestType;
+import org.apache.iotdb.rpc.RpcUtils;
+import org.apache.iotdb.rpc.TSStatusCode;
+
+import org.apache.thrift.async.AsyncMethodCallback;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.util.HashMap;
+import java.util.Map;
+import java.util.concurrent.CountDownLatch;
+
+public class RollbackSchemaBlackListHandler extends AbstractRetryHandler
+    implements AsyncMethodCallback<TSStatus> {
+
+  private static final Logger LOGGER =
+      LoggerFactory.getLogger(RollbackSchemaBlackListHandler.class);
+
+  private Map<Integer, TSStatus> dataNodeResponseStatusMap = new HashMap<>();
+
+  public RollbackSchemaBlackListHandler(Map<Integer, TDataNodeLocation> dataNodeLocationMap) {
+    super(DataNodeRequestType.ROLLBACK_SCHEMA_BLACK_LIST, dataNodeLocationMap);
+  }
+
+  public RollbackSchemaBlackListHandler(
+      CountDownLatch countDownLatch,
+      TDataNodeLocation targetDataNode,
+      Map<Integer, TDataNodeLocation> dataNodeLocationMap) {
+    super(
+        countDownLatch,
+        DataNodeRequestType.ROLLBACK_SCHEMA_BLACK_LIST,
+        targetDataNode,
+        dataNodeLocationMap);
+  }
+
+  public Map<Integer, TSStatus> getDataNodeResponseStatusMap() {
+    return dataNodeResponseStatusMap;
+  }
+
+  @Override
+  public void onComplete(TSStatus tsStatus) {
+    dataNodeResponseStatusMap.put(targetDataNode.getDataNodeId(), tsStatus);
+    if (tsStatus.getCode() == TSStatusCode.SUCCESS_STATUS.getStatusCode()) {
+      dataNodeLocationMap.remove(targetDataNode.getDataNodeId());
+      LOGGER.info("Successfully rollback schema black list on DataNode: {}", targetDataNode);
+    } else if (tsStatus.getCode() == TSStatusCode.MULTIPLE_ERROR.getStatusCode()) {
+      dataNodeLocationMap.remove(targetDataNode.getDataNodeId());
+      LOGGER.error(
+          "Failed to rollback schema black list on DataNode {}, {}",
+          dataNodeLocationMap.get(targetDataNode.getDataNodeId()),
+          tsStatus);

Review Comment:
   You'll get a NPE if you remove before get T^T



##########
confignode/src/main/java/org/apache/iotdb/confignode/client/async/handlers/FetchSchemaBlackLsitHandler.java:
##########
@@ -0,0 +1,101 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.confignode.client.async.handlers;
+
+import org.apache.iotdb.common.rpc.thrift.TDataNodeLocation;
+import org.apache.iotdb.common.rpc.thrift.TSStatus;
+import org.apache.iotdb.confignode.client.DataNodeRequestType;
+import org.apache.iotdb.mpp.rpc.thrift.TFetchSchemaBlackListResp;
+import org.apache.iotdb.rpc.RpcUtils;
+import org.apache.iotdb.rpc.TSStatusCode;
+
+import org.apache.thrift.async.AsyncMethodCallback;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.util.HashMap;
+import java.util.Map;
+import java.util.concurrent.CountDownLatch;
+
+public class FetchSchemaBlackLsitHandler extends AbstractRetryHandler
+    implements AsyncMethodCallback<TFetchSchemaBlackListResp> {
+
+  private static final Logger LOGGER = LoggerFactory.getLogger(FetchSchemaBlackLsitHandler.class);
+
+  private Map<Integer, TFetchSchemaBlackListResp> dataNodeResponseMap = new HashMap<>();
+
+  public FetchSchemaBlackLsitHandler(Map<Integer, TDataNodeLocation> dataNodeLocationMap) {
+    super(DataNodeRequestType.FETCH_SCHEMA_BLACK_LIST, dataNodeLocationMap);
+  }
+
+  public FetchSchemaBlackLsitHandler(
+      CountDownLatch countDownLatch,
+      TDataNodeLocation targetDataNode,
+      Map<Integer, TDataNodeLocation> dataNodeLocationMap) {
+    super(
+        countDownLatch,
+        DataNodeRequestType.FETCH_SCHEMA_BLACK_LIST,
+        targetDataNode,
+        dataNodeLocationMap);
+  }
+
+  public Map<Integer, TFetchSchemaBlackListResp> getDataNodeResponseMap() {
+    return dataNodeResponseMap;
+  }
+
+  @Override
+  public void onComplete(TFetchSchemaBlackListResp tFetchSchemaBlackListResp) {
+    TSStatus tsStatus = tFetchSchemaBlackListResp.getStatus();
+    dataNodeResponseMap.put(targetDataNode.getDataNodeId(), tFetchSchemaBlackListResp);
+    if (tsStatus.getCode() == TSStatusCode.SUCCESS_STATUS.getStatusCode()) {
+      dataNodeLocationMap.remove(targetDataNode.getDataNodeId());
+      LOGGER.info("Successfully fetch schema black list on DataNode: {}", targetDataNode);
+    } else if (tsStatus.getCode() == TSStatusCode.MULTIPLE_ERROR.getStatusCode()) {
+      dataNodeLocationMap.remove(targetDataNode.getDataNodeId());
+      LOGGER.error(
+          "Failed to fetch schema black list on DataNode {}, {}",
+          dataNodeLocationMap.get(targetDataNode.getDataNodeId()),
+          tsStatus);

Review Comment:
   You'll get a NPE if you remove before get T^T



##########
confignode/src/main/java/org/apache/iotdb/confignode/procedure/impl/DeleteTimeSeriesProcedure.java:
##########
@@ -0,0 +1,611 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.confignode.procedure.impl;
+
+import org.apache.iotdb.common.rpc.thrift.TConsensusGroupId;
+import org.apache.iotdb.common.rpc.thrift.TDataNodeLocation;
+import org.apache.iotdb.common.rpc.thrift.TRegionReplicaSet;
+import org.apache.iotdb.common.rpc.thrift.TSStatus;
+import org.apache.iotdb.commons.path.PartialPath;
+import org.apache.iotdb.confignode.client.async.datanode.AsyncDataNodeClientPool;
+import org.apache.iotdb.confignode.client.async.handlers.ConstructSchemaBlackListHandler;
+import org.apache.iotdb.confignode.client.async.handlers.DeleteDataForDeleteTimeSeriesHandler;
+import org.apache.iotdb.confignode.client.async.handlers.DeleteTimeSeriesHandler;
+import org.apache.iotdb.confignode.client.async.handlers.FetchSchemaBlackLsitHandler;
+import org.apache.iotdb.confignode.client.async.handlers.InvalidateMatchedSchemaCacheHandler;
+import org.apache.iotdb.confignode.client.async.handlers.RollbackSchemaBlackListHandler;
+import org.apache.iotdb.confignode.procedure.StateMachineProcedure;
+import org.apache.iotdb.confignode.procedure.env.ConfigNodeProcedureEnv;
+import org.apache.iotdb.confignode.procedure.exception.ProcedureException;
+import org.apache.iotdb.confignode.procedure.exception.ProcedureSuspendedException;
+import org.apache.iotdb.confignode.procedure.exception.ProcedureYieldException;
+import org.apache.iotdb.confignode.procedure.state.DeleteTimeSeriesState;
+import org.apache.iotdb.confignode.procedure.store.ProcedureFactory;
+import org.apache.iotdb.db.exception.metadata.PathNotExistException;
+import org.apache.iotdb.db.mpp.common.schematree.PathPatternTree;
+import org.apache.iotdb.mpp.rpc.thrift.TConstructSchemaBlackListReq;
+import org.apache.iotdb.mpp.rpc.thrift.TDeleteDataForDeleteTimeSeriesReq;
+import org.apache.iotdb.mpp.rpc.thrift.TDeleteTimeSeriesReq;
+import org.apache.iotdb.mpp.rpc.thrift.TFetchSchemaBlackListReq;
+import org.apache.iotdb.mpp.rpc.thrift.TFetchSchemaBlackListResp;
+import org.apache.iotdb.mpp.rpc.thrift.TInvalidateMatchedSchemaCacheReq;
+import org.apache.iotdb.mpp.rpc.thrift.TRollbackSchemaBlackListReq;
+import org.apache.iotdb.rpc.TSStatusCode;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.ByteArrayOutputStream;
+import java.io.DataOutputStream;
+import java.io.IOException;
+import java.nio.ByteBuffer;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.stream.Collectors;
+
+public class DeleteTimeSeriesProcedure
+    extends StateMachineProcedure<ConfigNodeProcedureEnv, DeleteTimeSeriesState> {
+
+  private static final Logger LOGGER = LoggerFactory.getLogger(DeleteTimeSeriesProcedure.class);
+
+  private PathPatternTree patternTree;
+  private ByteBuffer patternTreeBytes;
+
+  private String requestMessage;
+
+  public DeleteTimeSeriesProcedure() {
+    super();
+  }
+
+  public DeleteTimeSeriesProcedure(PathPatternTree patternTree) {
+    super();
+    setPatternTree(patternTree);
+  }
+
+  @Override
+  protected Flow executeFromState(ConfigNodeProcedureEnv env, DeleteTimeSeriesState state)
+      throws ProcedureSuspendedException, ProcedureYieldException, InterruptedException {
+    switch (state) {
+      case CONSTRUCT_BLACK_LIST:
+        LOGGER.info("Construct schema black list of timeseries {}", requestMessage);
+        if (constructBlackList(env) > 0) {
+          setNextState(DeleteTimeSeriesState.CLEAN_DATANODE_SCHEMA_CACHE);
+          break;
+        } else {
+          setFailure(
+              new ProcedureException(
+                  new PathNotExistException(
+                      patternTree.getAllPathPatterns().stream()
+                          .map(PartialPath::getFullPath)
+                          .collect(Collectors.toList()))));
+          return Flow.NO_MORE_STATE;
+        }
+      case CLEAN_DATANODE_SCHEMA_CACHE:
+        LOGGER.info("Invalidate cache of timeseries {}", requestMessage);
+        invalidateCache(env);
+        break;
+      case DELETE_DATA:
+        LOGGER.info("Delete data of timeseries {}", requestMessage);
+        deleteData(env);
+        break;
+      case DELETE_TIMESERIES:
+        LOGGER.info("Delete timeseries schema of {}", requestMessage);
+        deleteTimeSeries(env);
+        return Flow.NO_MORE_STATE;
+      default:
+        setFailure(new ProcedureException("Unrecognized state " + state.toString()));
+        return Flow.NO_MORE_STATE;
+    }
+    return Flow.HAS_MORE_STATE;
+  }
+
+  private int constructBlackList(ConfigNodeProcedureEnv env) {
+    Map<TConsensusGroupId, TRegionReplicaSet> targetSchemaRegionGroup =
+        env.getConfigManager().getRelatedSchemaRegionGroup(patternTree);
+    if (targetSchemaRegionGroup.isEmpty()) {
+      return 0;
+    }
+    RegionTask<TSStatus> constructBlackListTask =
+        new RegionTask<TSStatus>("construct schema black list", env, targetSchemaRegionGroup) {
+          @Override
+          Map<Integer, TSStatus> sendRequest(
+              TDataNodeLocation dataNodeLocation, List<TConsensusGroupId> consensusGroupIdList) {
+            // construct request and send
+            Map<Integer, TDataNodeLocation> dataNodeLocationMap = new HashMap<>();
+            dataNodeLocationMap.put(dataNodeLocation.getDataNodeId(), dataNodeLocation);
+            ConstructSchemaBlackListHandler handler =
+                new ConstructSchemaBlackListHandler(dataNodeLocationMap);
+            AsyncDataNodeClientPool.getInstance()
+                .sendAsyncRequestToDataNodeWithRetry(
+                    new TConstructSchemaBlackListReq(consensusGroupIdList, patternTreeBytes),
+                    handler);
+            handler
+                .getDataNodeResponseStatusMap()
+                .forEach(
+                    (k, v) -> {
+                      if (v.getCode() == TSStatusCode.SUCCESS_STATUS.getStatusCode()) {
+                        responseMap.put(k, v);
+                      }
+                    });
+            return handler.getDataNodeResponseStatusMap();
+          }
+        };
+    constructBlackListTask.execute();
+
+    if (isFailed()) {
+      return 0;
+    }
+
+    int preDeletedNum = 0;
+    for (TSStatus resp : constructBlackListTask.getResponseMap().values()) {
+      preDeletedNum += Integer.parseInt(resp.getMessage());
+    }
+    return preDeletedNum;
+  }
+
+  private void invalidateCache(ConfigNodeProcedureEnv env) {
+    Map<Integer, TDataNodeLocation> dataNodeLocationMap =
+        env.getConfigManager().getNodeManager().getRegisteredDataNodeLocations();
+    InvalidateMatchedSchemaCacheHandler handler =
+        new InvalidateMatchedSchemaCacheHandler(dataNodeLocationMap);
+    AsyncDataNodeClientPool.getInstance()
+        .sendAsyncRequestToDataNodeWithRetry(
+            new TInvalidateMatchedSchemaCacheReq(patternTreeBytes), handler);
+    Map<Integer, TSStatus> statusMap = handler.getDataNodeResponseStatusMap();
+    for (TSStatus status : statusMap.values()) {
+      // all dataNodes must clear the related schema cache
+      if (status.getCode() != TSStatusCode.SUCCESS_STATUS.getStatusCode()) {
+        LOGGER.error("Failed to invalidate schema cache of timeseries {}", requestMessage);
+        setFailure(new ProcedureException("Invalidate schema cache failed"));
+        return;
+      }
+    }
+
+    setNextState(DeleteTimeSeriesState.DELETE_DATA);
+  }
+
+  private void deleteData(ConfigNodeProcedureEnv env) {
+    Map<TConsensusGroupId, TRegionReplicaSet> relatedSchemaRegionGroup =
+        env.getConfigManager().getRelatedSchemaRegionGroup(patternTree);
+    Map<TDataNodeLocation, List<TConsensusGroupId>> dataNodeSchemaRegionGroupGroupIdMap =
+        getLeaderDataNodeRegionGroupMap(
+            env.getConfigManager().getPartitionManager().getAllLeadership(),
+            relatedSchemaRegionGroup);
+
+    // fetch schema black list by dataNode
+    for (Map.Entry<TDataNodeLocation, List<TConsensusGroupId>> entry :
+        dataNodeSchemaRegionGroupGroupIdMap.entrySet()) {
+      Map<TConsensusGroupId, TRegionReplicaSet> targetSchemaRegionGroup = new HashMap<>();
+      entry
+          .getValue()
+          .forEach(
+              consensusGroupId ->
+                  targetSchemaRegionGroup.put(
+                      consensusGroupId, relatedSchemaRegionGroup.get(consensusGroupId)));
+      PathPatternTree patternTree =
+          fetchSchemaBlackListOnTargetDataNode(env, targetSchemaRegionGroup);
+      if (isFailed()) {
+        return;
+      }
+      if (patternTree == null) {
+        LOGGER.error(
+            "Failed to fetch schema black list for delete data of timeseries {} on {}",
+            requestMessage,
+            entry.getKey());
+        setFailure(new ProcedureException("Fetch schema black list forDelete data failed"));
+        return;
+      }
+
+      if (patternTree.isEmpty()) {
+        continue;
+      }
+
+      Map<TConsensusGroupId, TRegionReplicaSet> relatedDataRegionGroup =
+          env.getConfigManager().getRelatedDataRegionGroup(patternTree);
+
+      // target timeseries has no data
+      if (relatedDataRegionGroup.isEmpty()) {
+        continue;
+      }
+
+      RegionTask<TSStatus> deleteDataTask =
+          new RegionTask<TSStatus>("delete data", env, relatedDataRegionGroup) {
+            @Override
+            Map<Integer, TSStatus> sendRequest(
+                TDataNodeLocation dataNodeLocation, List<TConsensusGroupId> consensusGroupIdList) {
+              Map<Integer, TDataNodeLocation> dataNodeLocationMap = new HashMap<>();
+              dataNodeLocationMap.put(dataNodeLocation.getDataNodeId(), dataNodeLocation);
+              DeleteDataForDeleteTimeSeriesHandler handler =
+                  new DeleteDataForDeleteTimeSeriesHandler(dataNodeLocationMap);
+              AsyncDataNodeClientPool.getInstance()
+                  .sendAsyncRequestToDataNodeWithRetry(
+                      new TDeleteDataForDeleteTimeSeriesReq(
+                          new ArrayList<>(consensusGroupIdList),
+                          preparePatternTreeBytesData(patternTree)),
+                      handler);
+              return handler.getDataNodeResponseStatusMap();
+            }
+          };
+      deleteDataTask.setExecuteOnAllReplicaset(true);
+      deleteDataTask.execute();
+      if (isFailed()) {
+        return;
+      }
+    }
+    setNextState(DeleteTimeSeriesState.DELETE_TIMESERIES);
+  }
+
+  private PathPatternTree fetchSchemaBlackListOnTargetDataNode(
+      ConfigNodeProcedureEnv env,
+      Map<TConsensusGroupId, TRegionReplicaSet> targetSchemaRegionGroup) {
+    RegionTask<TFetchSchemaBlackListResp> fetchSchemaBlackListTask =
+        new RegionTask<TFetchSchemaBlackListResp>(
+            "fetch schema black list", env, targetSchemaRegionGroup) {
+          @Override
+          Map<Integer, TSStatus> sendRequest(
+              TDataNodeLocation dataNodeLocation, List<TConsensusGroupId> consensusGroupIdList) {
+            Map<Integer, TDataNodeLocation> dataNodeLocationMap = new HashMap<>();
+            dataNodeLocationMap.put(dataNodeLocation.getDataNodeId(), dataNodeLocation);
+            FetchSchemaBlackLsitHandler handler =
+                new FetchSchemaBlackLsitHandler(dataNodeLocationMap);
+            AsyncDataNodeClientPool.getInstance()
+                .sendAsyncRequestToDataNodeWithRetry(
+                    new TFetchSchemaBlackListReq(consensusGroupIdList, patternTreeBytes), handler);
+            Map<Integer, TFetchSchemaBlackListResp> respMap = handler.getDataNodeResponseMap();
+            Map<Integer, TSStatus> statusMap = new HashMap<>();
+            respMap.forEach(
+                (k, v) -> {
+                  if (v.getStatus().getCode() == TSStatusCode.SUCCESS_STATUS.getStatusCode()) {
+                    responseMap.put(k, v);
+                  }
+                  statusMap.put(k, v.getStatus());
+                });
+            return statusMap;
+          }
+        };
+    fetchSchemaBlackListTask.execute();
+    if (isFailed()) {
+      return null;
+    }
+
+    Map<Integer, TFetchSchemaBlackListResp> respMap = fetchSchemaBlackListTask.getResponseMap();
+    PathPatternTree patternTree = new PathPatternTree();
+    for (TFetchSchemaBlackListResp resp : respMap.values()) {
+      for (PartialPath path :
+          PathPatternTree.deserialize(ByteBuffer.wrap(resp.getPathPatternTree()))
+              .getAllPathPatterns()) {
+        patternTree.appendPathPattern(path);
+      }
+    }
+    patternTree.constructTree();
+    return patternTree;
+  }
+
+  private void deleteTimeSeries(ConfigNodeProcedureEnv env) {
+    RegionTask<TSStatus> deleteTimeSeriesTask =
+        new RegionTask<TSStatus>(
+            "delete timeseries",
+            env,
+            env.getConfigManager().getRelatedSchemaRegionGroup(patternTree)) {
+          @Override
+          Map<Integer, TSStatus> sendRequest(
+              TDataNodeLocation dataNodeLocation, List<TConsensusGroupId> consensusGroupIdList) {
+            Map<Integer, TDataNodeLocation> dataNodeLocationMap = new HashMap<>();
+            dataNodeLocationMap.put(dataNodeLocation.getDataNodeId(), dataNodeLocation);
+            DeleteTimeSeriesHandler handler = new DeleteTimeSeriesHandler(dataNodeLocationMap);
+            AsyncDataNodeClientPool.getInstance()
+                .sendAsyncRequestToDataNodeWithRetry(
+                    new TDeleteTimeSeriesReq(consensusGroupIdList, patternTreeBytes), handler);
+            return handler.getDataNodeResponseStatusMap();
+          }
+        };
+    deleteTimeSeriesTask.execute();
+  }
+
+  private Map<TDataNodeLocation, List<TConsensusGroupId>> getLeaderDataNodeRegionGroupMap(
+      Map<TConsensusGroupId, Integer> leaderMap,
+      Map<TConsensusGroupId, TRegionReplicaSet> regionReplicaSetMap) {
+    Map<TDataNodeLocation, List<TConsensusGroupId>> dataNodeConsensusGroupIdMap = new HashMap<>();
+    regionReplicaSetMap.forEach(
+        (consensusGroupId, regionReplicaSet) -> {
+          Integer leaderId = leaderMap.get(consensusGroupId);
+          TDataNodeLocation leaderDataNodeLocation = null;
+          if (leaderId == null || leaderId == -1) {
+            leaderDataNodeLocation = regionReplicaSet.getDataNodeLocations().get(0);
+          } else {
+            for (TDataNodeLocation dataNodeLocation : regionReplicaSet.getDataNodeLocations()) {
+              if (dataNodeLocation.getDataNodeId() == leaderId) {
+                leaderDataNodeLocation = dataNodeLocation;
+                break;
+              }
+            }
+          }
+          dataNodeConsensusGroupIdMap
+              .computeIfAbsent(leaderDataNodeLocation, k -> new ArrayList<>())
+              .add(regionReplicaSet.getRegionId());
+        });
+    return dataNodeConsensusGroupIdMap;
+  }
+
+  private Map<TDataNodeLocation, List<TConsensusGroupId>> getAllReplicaDataNodeRegionGroupMap(
+      Map<TConsensusGroupId, TRegionReplicaSet> regionReplicaSetMap) {
+    Map<TDataNodeLocation, List<TConsensusGroupId>> dataNodeConsensusGroupIdMap = new HashMap<>();
+    regionReplicaSetMap.forEach(
+        (consensusGroupId, regionReplicaSet) -> {
+          for (TDataNodeLocation dataNodeLocation : regionReplicaSet.getDataNodeLocations()) {
+            dataNodeConsensusGroupIdMap
+                .computeIfAbsent(dataNodeLocation, k -> new ArrayList<>())
+                .add(regionReplicaSet.getRegionId());
+          }
+        });
+    return dataNodeConsensusGroupIdMap;
+  }
+
+  @Override
+  protected void rollbackState(
+      ConfigNodeProcedureEnv env, DeleteTimeSeriesState deleteTimeSeriesState)
+      throws IOException, InterruptedException, ProcedureException {
+    RegionTask<TSStatus> rollbackStateTask =
+        new RegionTask<TSStatus>(
+            "roll back schema black list",
+            env,
+            env.getConfigManager().getRelatedSchemaRegionGroup(patternTree)) {
+          @Override
+          Map<Integer, TSStatus> sendRequest(
+              TDataNodeLocation dataNodeLocation, List<TConsensusGroupId> consensusGroupIdList) {
+            Map<Integer, TDataNodeLocation> dataNodeLocationMap = new HashMap<>();
+            dataNodeLocationMap.put(dataNodeLocation.getDataNodeId(), dataNodeLocation);
+            RollbackSchemaBlackListHandler handler =
+                new RollbackSchemaBlackListHandler(dataNodeLocationMap);
+            AsyncDataNodeClientPool.getInstance()
+                .sendAsyncRequestToDataNodeWithRetry(
+                    new TRollbackSchemaBlackListReq(consensusGroupIdList, patternTreeBytes),
+                    handler);
+            return handler.getDataNodeResponseStatusMap();
+          }
+        };
+    rollbackStateTask.execute();
+  }
+
+  @Override
+  protected boolean isRollbackSupported(DeleteTimeSeriesState deleteTimeSeriesState) {
+    return true;
+  }
+
+  @Override
+  protected DeleteTimeSeriesState getState(int stateId) {
+    return DeleteTimeSeriesState.values()[stateId];
+  }
+
+  @Override
+  protected int getStateId(DeleteTimeSeriesState deleteTimeSeriesState) {
+    return deleteTimeSeriesState.ordinal();
+  }
+
+  @Override
+  protected DeleteTimeSeriesState getInitialState() {
+    return DeleteTimeSeriesState.CONSTRUCT_BLACK_LIST;
+  }
+
+  public PathPatternTree getPatternTree() {
+    return patternTree;
+  }
+
+  public void setPatternTree(PathPatternTree patternTree) {
+    this.patternTree = patternTree;
+    requestMessage = patternTree.getAllPathPatterns().toString();
+    patternTreeBytes = preparePatternTreeBytesData(patternTree);
+  }
+
+  private ByteBuffer preparePatternTreeBytesData(PathPatternTree patternTree) {
+    ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream();
+    DataOutputStream dataOutputStream = new DataOutputStream(byteArrayOutputStream);
+    try {
+      patternTree.serialize(dataOutputStream);
+    } catch (IOException ignored) {
+
+    }
+    return ByteBuffer.wrap(byteArrayOutputStream.toByteArray());
+  }
+
+  @Override
+  public void serialize(DataOutputStream stream) throws IOException {
+    stream.writeInt(ProcedureFactory.ProcedureType.DELETE_TIMESERIES_PROCEDURE.ordinal());
+    super.serialize(stream);
+    patternTree.serialize(stream);
+  }
+
+  @Override
+  public void deserialize(ByteBuffer byteBuffer) {
+    super.deserialize(byteBuffer);
+    setPatternTree(PathPatternTree.deserialize(byteBuffer));
+  }

Review Comment:
   We'd better add serialize and deserialize tests for every new Procedure. You can refer to org.apache.iotdb.confignode.procedure.impl.CreateRegionGroupsProcedureTest



##########
confignode/src/main/java/org/apache/iotdb/confignode/client/async/handlers/DeleteDataForDeleteTimeSeriesHandler.java:
##########
@@ -0,0 +1,99 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.confignode.client.async.handlers;
+
+import org.apache.iotdb.common.rpc.thrift.TDataNodeLocation;
+import org.apache.iotdb.common.rpc.thrift.TSStatus;
+import org.apache.iotdb.confignode.client.DataNodeRequestType;
+import org.apache.iotdb.rpc.RpcUtils;
+import org.apache.iotdb.rpc.TSStatusCode;
+
+import org.apache.thrift.async.AsyncMethodCallback;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.util.HashMap;
+import java.util.Map;
+import java.util.concurrent.CountDownLatch;
+
+public class DeleteDataForDeleteTimeSeriesHandler extends AbstractRetryHandler
+    implements AsyncMethodCallback<TSStatus> {
+
+  private static final Logger LOGGER =
+      LoggerFactory.getLogger(DeleteDataForDeleteTimeSeriesHandler.class);
+
+  private Map<Integer, TSStatus> dataNodeResponseStatusMap = new HashMap<>();
+
+  public DeleteDataForDeleteTimeSeriesHandler(Map<Integer, TDataNodeLocation> dataNodeLocationMap) {
+    super(DataNodeRequestType.DELETE_DATA_FOR_DELETE_TIMESERIES, dataNodeLocationMap);
+  }
+
+  public DeleteDataForDeleteTimeSeriesHandler(
+      CountDownLatch countDownLatch,
+      TDataNodeLocation targetDataNode,
+      Map<Integer, TDataNodeLocation> dataNodeLocationMap) {
+    super(
+        countDownLatch,
+        DataNodeRequestType.DELETE_DATA_FOR_DELETE_TIMESERIES,
+        targetDataNode,
+        dataNodeLocationMap);
+  }
+
+  public Map<Integer, TSStatus> getDataNodeResponseStatusMap() {
+    return dataNodeResponseStatusMap;
+  }
+
+  @Override
+  public void onComplete(TSStatus tsStatus) {
+    dataNodeResponseStatusMap.put(targetDataNode.getDataNodeId(), tsStatus);
+    if (tsStatus.getCode() == TSStatusCode.SUCCESS_STATUS.getStatusCode()) {
+      dataNodeLocationMap.remove(targetDataNode.getDataNodeId());
+      LOGGER.info("Successfully delete data for delete timeseries on DataNode: {}", targetDataNode);
+    } else if (tsStatus.getCode() == TSStatusCode.MULTIPLE_ERROR.getStatusCode()) {
+      dataNodeLocationMap.remove(targetDataNode.getDataNodeId());
+      LOGGER.error(
+          "Failed to delete data for delete timeseries on DataNode {}, {}",
+          dataNodeLocationMap.get(targetDataNode.getDataNodeId()),
+          tsStatus);

Review Comment:
   You'll get a NPE if you remove before get T^T



##########
confignode/src/main/java/org/apache/iotdb/confignode/procedure/impl/DeleteTimeSeriesProcedure.java:
##########
@@ -0,0 +1,611 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.confignode.procedure.impl;
+
+import org.apache.iotdb.common.rpc.thrift.TConsensusGroupId;
+import org.apache.iotdb.common.rpc.thrift.TDataNodeLocation;
+import org.apache.iotdb.common.rpc.thrift.TRegionReplicaSet;
+import org.apache.iotdb.common.rpc.thrift.TSStatus;
+import org.apache.iotdb.commons.path.PartialPath;
+import org.apache.iotdb.confignode.client.async.datanode.AsyncDataNodeClientPool;
+import org.apache.iotdb.confignode.client.async.handlers.ConstructSchemaBlackListHandler;
+import org.apache.iotdb.confignode.client.async.handlers.DeleteDataForDeleteTimeSeriesHandler;
+import org.apache.iotdb.confignode.client.async.handlers.DeleteTimeSeriesHandler;
+import org.apache.iotdb.confignode.client.async.handlers.FetchSchemaBlackLsitHandler;
+import org.apache.iotdb.confignode.client.async.handlers.InvalidateMatchedSchemaCacheHandler;
+import org.apache.iotdb.confignode.client.async.handlers.RollbackSchemaBlackListHandler;
+import org.apache.iotdb.confignode.procedure.StateMachineProcedure;
+import org.apache.iotdb.confignode.procedure.env.ConfigNodeProcedureEnv;
+import org.apache.iotdb.confignode.procedure.exception.ProcedureException;
+import org.apache.iotdb.confignode.procedure.exception.ProcedureSuspendedException;
+import org.apache.iotdb.confignode.procedure.exception.ProcedureYieldException;
+import org.apache.iotdb.confignode.procedure.state.DeleteTimeSeriesState;
+import org.apache.iotdb.confignode.procedure.store.ProcedureFactory;
+import org.apache.iotdb.db.exception.metadata.PathNotExistException;
+import org.apache.iotdb.db.mpp.common.schematree.PathPatternTree;
+import org.apache.iotdb.mpp.rpc.thrift.TConstructSchemaBlackListReq;
+import org.apache.iotdb.mpp.rpc.thrift.TDeleteDataForDeleteTimeSeriesReq;
+import org.apache.iotdb.mpp.rpc.thrift.TDeleteTimeSeriesReq;
+import org.apache.iotdb.mpp.rpc.thrift.TFetchSchemaBlackListReq;
+import org.apache.iotdb.mpp.rpc.thrift.TFetchSchemaBlackListResp;
+import org.apache.iotdb.mpp.rpc.thrift.TInvalidateMatchedSchemaCacheReq;
+import org.apache.iotdb.mpp.rpc.thrift.TRollbackSchemaBlackListReq;
+import org.apache.iotdb.rpc.TSStatusCode;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.ByteArrayOutputStream;
+import java.io.DataOutputStream;
+import java.io.IOException;
+import java.nio.ByteBuffer;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.stream.Collectors;
+
+public class DeleteTimeSeriesProcedure
+    extends StateMachineProcedure<ConfigNodeProcedureEnv, DeleteTimeSeriesState> {
+
+  private static final Logger LOGGER = LoggerFactory.getLogger(DeleteTimeSeriesProcedure.class);
+
+  private PathPatternTree patternTree;
+  private ByteBuffer patternTreeBytes;
+
+  private String requestMessage;
+
+  public DeleteTimeSeriesProcedure() {
+    super();
+  }
+
+  public DeleteTimeSeriesProcedure(PathPatternTree patternTree) {
+    super();
+    setPatternTree(patternTree);
+  }
+
+  @Override
+  protected Flow executeFromState(ConfigNodeProcedureEnv env, DeleteTimeSeriesState state)
+      throws ProcedureSuspendedException, ProcedureYieldException, InterruptedException {
+    switch (state) {
+      case CONSTRUCT_BLACK_LIST:
+        LOGGER.info("Construct schema black list of timeseries {}", requestMessage);
+        if (constructBlackList(env) > 0) {
+          setNextState(DeleteTimeSeriesState.CLEAN_DATANODE_SCHEMA_CACHE);
+          break;
+        } else {
+          setFailure(
+              new ProcedureException(
+                  new PathNotExistException(
+                      patternTree.getAllPathPatterns().stream()
+                          .map(PartialPath::getFullPath)
+                          .collect(Collectors.toList()))));
+          return Flow.NO_MORE_STATE;
+        }
+      case CLEAN_DATANODE_SCHEMA_CACHE:
+        LOGGER.info("Invalidate cache of timeseries {}", requestMessage);
+        invalidateCache(env);
+        break;
+      case DELETE_DATA:
+        LOGGER.info("Delete data of timeseries {}", requestMessage);
+        deleteData(env);
+        break;
+      case DELETE_TIMESERIES:
+        LOGGER.info("Delete timeseries schema of {}", requestMessage);
+        deleteTimeSeries(env);
+        return Flow.NO_MORE_STATE;
+      default:
+        setFailure(new ProcedureException("Unrecognized state " + state.toString()));
+        return Flow.NO_MORE_STATE;
+    }
+    return Flow.HAS_MORE_STATE;
+  }
+
+  private int constructBlackList(ConfigNodeProcedureEnv env) {
+    Map<TConsensusGroupId, TRegionReplicaSet> targetSchemaRegionGroup =
+        env.getConfigManager().getRelatedSchemaRegionGroup(patternTree);
+    if (targetSchemaRegionGroup.isEmpty()) {
+      return 0;
+    }
+    RegionTask<TSStatus> constructBlackListTask =
+        new RegionTask<TSStatus>("construct schema black list", env, targetSchemaRegionGroup) {
+          @Override
+          Map<Integer, TSStatus> sendRequest(
+              TDataNodeLocation dataNodeLocation, List<TConsensusGroupId> consensusGroupIdList) {
+            // construct request and send
+            Map<Integer, TDataNodeLocation> dataNodeLocationMap = new HashMap<>();
+            dataNodeLocationMap.put(dataNodeLocation.getDataNodeId(), dataNodeLocation);
+            ConstructSchemaBlackListHandler handler =
+                new ConstructSchemaBlackListHandler(dataNodeLocationMap);
+            AsyncDataNodeClientPool.getInstance()
+                .sendAsyncRequestToDataNodeWithRetry(
+                    new TConstructSchemaBlackListReq(consensusGroupIdList, patternTreeBytes),
+                    handler);
+            handler
+                .getDataNodeResponseStatusMap()
+                .forEach(
+                    (k, v) -> {
+                      if (v.getCode() == TSStatusCode.SUCCESS_STATUS.getStatusCode()) {
+                        responseMap.put(k, v);
+                      }
+                    });
+            return handler.getDataNodeResponseStatusMap();
+          }
+        };
+    constructBlackListTask.execute();
+
+    if (isFailed()) {
+      return 0;
+    }
+
+    int preDeletedNum = 0;
+    for (TSStatus resp : constructBlackListTask.getResponseMap().values()) {
+      preDeletedNum += Integer.parseInt(resp.getMessage());
+    }
+    return preDeletedNum;
+  }
+
+  private void invalidateCache(ConfigNodeProcedureEnv env) {
+    Map<Integer, TDataNodeLocation> dataNodeLocationMap =
+        env.getConfigManager().getNodeManager().getRegisteredDataNodeLocations();
+    InvalidateMatchedSchemaCacheHandler handler =
+        new InvalidateMatchedSchemaCacheHandler(dataNodeLocationMap);
+    AsyncDataNodeClientPool.getInstance()
+        .sendAsyncRequestToDataNodeWithRetry(
+            new TInvalidateMatchedSchemaCacheReq(patternTreeBytes), handler);
+    Map<Integer, TSStatus> statusMap = handler.getDataNodeResponseStatusMap();
+    for (TSStatus status : statusMap.values()) {
+      // all dataNodes must clear the related schema cache
+      if (status.getCode() != TSStatusCode.SUCCESS_STATUS.getStatusCode()) {
+        LOGGER.error("Failed to invalidate schema cache of timeseries {}", requestMessage);
+        setFailure(new ProcedureException("Invalidate schema cache failed"));
+        return;
+      }
+    }
+
+    setNextState(DeleteTimeSeriesState.DELETE_DATA);
+  }
+
+  private void deleteData(ConfigNodeProcedureEnv env) {
+    Map<TConsensusGroupId, TRegionReplicaSet> relatedSchemaRegionGroup =
+        env.getConfigManager().getRelatedSchemaRegionGroup(patternTree);
+    Map<TDataNodeLocation, List<TConsensusGroupId>> dataNodeSchemaRegionGroupGroupIdMap =
+        getLeaderDataNodeRegionGroupMap(
+            env.getConfigManager().getPartitionManager().getAllLeadership(),
+            relatedSchemaRegionGroup);
+
+    // fetch schema black list by dataNode
+    for (Map.Entry<TDataNodeLocation, List<TConsensusGroupId>> entry :
+        dataNodeSchemaRegionGroupGroupIdMap.entrySet()) {
+      Map<TConsensusGroupId, TRegionReplicaSet> targetSchemaRegionGroup = new HashMap<>();
+      entry
+          .getValue()
+          .forEach(
+              consensusGroupId ->
+                  targetSchemaRegionGroup.put(
+                      consensusGroupId, relatedSchemaRegionGroup.get(consensusGroupId)));
+      PathPatternTree patternTree =
+          fetchSchemaBlackListOnTargetDataNode(env, targetSchemaRegionGroup);
+      if (isFailed()) {
+        return;
+      }
+      if (patternTree == null) {
+        LOGGER.error(
+            "Failed to fetch schema black list for delete data of timeseries {} on {}",
+            requestMessage,
+            entry.getKey());
+        setFailure(new ProcedureException("Fetch schema black list forDelete data failed"));
+        return;
+      }
+
+      if (patternTree.isEmpty()) {
+        continue;
+      }
+
+      Map<TConsensusGroupId, TRegionReplicaSet> relatedDataRegionGroup =
+          env.getConfigManager().getRelatedDataRegionGroup(patternTree);
+
+      // target timeseries has no data
+      if (relatedDataRegionGroup.isEmpty()) {
+        continue;
+      }
+
+      RegionTask<TSStatus> deleteDataTask =
+          new RegionTask<TSStatus>("delete data", env, relatedDataRegionGroup) {
+            @Override
+            Map<Integer, TSStatus> sendRequest(
+                TDataNodeLocation dataNodeLocation, List<TConsensusGroupId> consensusGroupIdList) {
+              Map<Integer, TDataNodeLocation> dataNodeLocationMap = new HashMap<>();
+              dataNodeLocationMap.put(dataNodeLocation.getDataNodeId(), dataNodeLocation);
+              DeleteDataForDeleteTimeSeriesHandler handler =
+                  new DeleteDataForDeleteTimeSeriesHandler(dataNodeLocationMap);
+              AsyncDataNodeClientPool.getInstance()
+                  .sendAsyncRequestToDataNodeWithRetry(
+                      new TDeleteDataForDeleteTimeSeriesReq(
+                          new ArrayList<>(consensusGroupIdList),
+                          preparePatternTreeBytesData(patternTree)),
+                      handler);
+              return handler.getDataNodeResponseStatusMap();
+            }
+          };
+      deleteDataTask.setExecuteOnAllReplicaset(true);
+      deleteDataTask.execute();
+      if (isFailed()) {
+        return;
+      }
+    }
+    setNextState(DeleteTimeSeriesState.DELETE_TIMESERIES);
+  }
+
+  private PathPatternTree fetchSchemaBlackListOnTargetDataNode(
+      ConfigNodeProcedureEnv env,
+      Map<TConsensusGroupId, TRegionReplicaSet> targetSchemaRegionGroup) {
+    RegionTask<TFetchSchemaBlackListResp> fetchSchemaBlackListTask =
+        new RegionTask<TFetchSchemaBlackListResp>(
+            "fetch schema black list", env, targetSchemaRegionGroup) {
+          @Override
+          Map<Integer, TSStatus> sendRequest(
+              TDataNodeLocation dataNodeLocation, List<TConsensusGroupId> consensusGroupIdList) {
+            Map<Integer, TDataNodeLocation> dataNodeLocationMap = new HashMap<>();
+            dataNodeLocationMap.put(dataNodeLocation.getDataNodeId(), dataNodeLocation);
+            FetchSchemaBlackLsitHandler handler =
+                new FetchSchemaBlackLsitHandler(dataNodeLocationMap);
+            AsyncDataNodeClientPool.getInstance()
+                .sendAsyncRequestToDataNodeWithRetry(
+                    new TFetchSchemaBlackListReq(consensusGroupIdList, patternTreeBytes), handler);
+            Map<Integer, TFetchSchemaBlackListResp> respMap = handler.getDataNodeResponseMap();
+            Map<Integer, TSStatus> statusMap = new HashMap<>();
+            respMap.forEach(
+                (k, v) -> {
+                  if (v.getStatus().getCode() == TSStatusCode.SUCCESS_STATUS.getStatusCode()) {
+                    responseMap.put(k, v);
+                  }
+                  statusMap.put(k, v.getStatus());
+                });
+            return statusMap;
+          }
+        };
+    fetchSchemaBlackListTask.execute();
+    if (isFailed()) {
+      return null;
+    }
+
+    Map<Integer, TFetchSchemaBlackListResp> respMap = fetchSchemaBlackListTask.getResponseMap();
+    PathPatternTree patternTree = new PathPatternTree();
+    for (TFetchSchemaBlackListResp resp : respMap.values()) {
+      for (PartialPath path :
+          PathPatternTree.deserialize(ByteBuffer.wrap(resp.getPathPatternTree()))
+              .getAllPathPatterns()) {
+        patternTree.appendPathPattern(path);
+      }
+    }
+    patternTree.constructTree();
+    return patternTree;
+  }
+
+  private void deleteTimeSeries(ConfigNodeProcedureEnv env) {
+    RegionTask<TSStatus> deleteTimeSeriesTask =
+        new RegionTask<TSStatus>(
+            "delete timeseries",
+            env,
+            env.getConfigManager().getRelatedSchemaRegionGroup(patternTree)) {
+          @Override
+          Map<Integer, TSStatus> sendRequest(
+              TDataNodeLocation dataNodeLocation, List<TConsensusGroupId> consensusGroupIdList) {
+            Map<Integer, TDataNodeLocation> dataNodeLocationMap = new HashMap<>();
+            dataNodeLocationMap.put(dataNodeLocation.getDataNodeId(), dataNodeLocation);
+            DeleteTimeSeriesHandler handler = new DeleteTimeSeriesHandler(dataNodeLocationMap);
+            AsyncDataNodeClientPool.getInstance()
+                .sendAsyncRequestToDataNodeWithRetry(
+                    new TDeleteTimeSeriesReq(consensusGroupIdList, patternTreeBytes), handler);
+            return handler.getDataNodeResponseStatusMap();
+          }
+        };
+    deleteTimeSeriesTask.execute();
+  }
+
+  private Map<TDataNodeLocation, List<TConsensusGroupId>> getLeaderDataNodeRegionGroupMap(
+      Map<TConsensusGroupId, Integer> leaderMap,
+      Map<TConsensusGroupId, TRegionReplicaSet> regionReplicaSetMap) {
+    Map<TDataNodeLocation, List<TConsensusGroupId>> dataNodeConsensusGroupIdMap = new HashMap<>();
+    regionReplicaSetMap.forEach(
+        (consensusGroupId, regionReplicaSet) -> {
+          Integer leaderId = leaderMap.get(consensusGroupId);
+          TDataNodeLocation leaderDataNodeLocation = null;
+          if (leaderId == null || leaderId == -1) {
+            leaderDataNodeLocation = regionReplicaSet.getDataNodeLocations().get(0);
+          } else {
+            for (TDataNodeLocation dataNodeLocation : regionReplicaSet.getDataNodeLocations()) {
+              if (dataNodeLocation.getDataNodeId() == leaderId) {
+                leaderDataNodeLocation = dataNodeLocation;
+                break;
+              }
+            }
+          }
+          dataNodeConsensusGroupIdMap
+              .computeIfAbsent(leaderDataNodeLocation, k -> new ArrayList<>())
+              .add(regionReplicaSet.getRegionId());
+        });
+    return dataNodeConsensusGroupIdMap;
+  }
+
+  private Map<TDataNodeLocation, List<TConsensusGroupId>> getAllReplicaDataNodeRegionGroupMap(

Review Comment:
   Add function annotations to show the difference between getLeaderDataNodeRegionGroupMap.



##########
confignode/src/main/java/org/apache/iotdb/confignode/client/async/handlers/DeleteTimeSeriesHandler.java:
##########
@@ -0,0 +1,95 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.confignode.client.async.handlers;
+
+import org.apache.iotdb.common.rpc.thrift.TDataNodeLocation;
+import org.apache.iotdb.common.rpc.thrift.TSStatus;
+import org.apache.iotdb.confignode.client.DataNodeRequestType;
+import org.apache.iotdb.rpc.RpcUtils;
+import org.apache.iotdb.rpc.TSStatusCode;
+
+import org.apache.thrift.async.AsyncMethodCallback;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.util.HashMap;
+import java.util.Map;
+import java.util.concurrent.CountDownLatch;
+
+public class DeleteTimeSeriesHandler extends AbstractRetryHandler
+    implements AsyncMethodCallback<TSStatus> {
+
+  private static final Logger LOGGER = LoggerFactory.getLogger(DeleteTimeSeriesHandler.class);
+
+  private Map<Integer, TSStatus> dataNodeResponseStatusMap = new HashMap<>();
+
+  public DeleteTimeSeriesHandler(Map<Integer, TDataNodeLocation> dataNodeLocationMap) {
+    super(DataNodeRequestType.DELETE_TIMESERIES, dataNodeLocationMap);
+  }
+
+  public DeleteTimeSeriesHandler(
+      CountDownLatch countDownLatch,
+      TDataNodeLocation targetDataNode,
+      Map<Integer, TDataNodeLocation> dataNodeLocationMap) {
+    super(
+        countDownLatch, DataNodeRequestType.DELETE_TIMESERIES, targetDataNode, dataNodeLocationMap);
+  }
+
+  public Map<Integer, TSStatus> getDataNodeResponseStatusMap() {
+    return dataNodeResponseStatusMap;
+  }
+
+  @Override
+  public void onComplete(TSStatus tsStatus) {
+    dataNodeResponseStatusMap.put(targetDataNode.getDataNodeId(), tsStatus);
+    if (tsStatus.getCode() == TSStatusCode.SUCCESS_STATUS.getStatusCode()) {
+      dataNodeLocationMap.remove(targetDataNode.getDataNodeId());
+      LOGGER.info("Successfully delete timeseries on DataNode: {}", targetDataNode);
+    } else if (tsStatus.getCode() == TSStatusCode.MULTIPLE_ERROR.getStatusCode()) {
+      dataNodeLocationMap.remove(targetDataNode.getDataNodeId());
+      LOGGER.error(
+          "Failed to delete timeseries on DataNode {}, {}",
+          dataNodeLocationMap.get(targetDataNode.getDataNodeId()),
+          tsStatus);

Review Comment:
   You'll get a NPE if you remove before get T^T



##########
confignode/src/main/java/org/apache/iotdb/confignode/procedure/impl/DeleteTimeSeriesProcedure.java:
##########
@@ -0,0 +1,611 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.confignode.procedure.impl;
+
+import org.apache.iotdb.common.rpc.thrift.TConsensusGroupId;
+import org.apache.iotdb.common.rpc.thrift.TDataNodeLocation;
+import org.apache.iotdb.common.rpc.thrift.TRegionReplicaSet;
+import org.apache.iotdb.common.rpc.thrift.TSStatus;
+import org.apache.iotdb.commons.path.PartialPath;
+import org.apache.iotdb.confignode.client.async.datanode.AsyncDataNodeClientPool;
+import org.apache.iotdb.confignode.client.async.handlers.ConstructSchemaBlackListHandler;
+import org.apache.iotdb.confignode.client.async.handlers.DeleteDataForDeleteTimeSeriesHandler;
+import org.apache.iotdb.confignode.client.async.handlers.DeleteTimeSeriesHandler;
+import org.apache.iotdb.confignode.client.async.handlers.FetchSchemaBlackLsitHandler;
+import org.apache.iotdb.confignode.client.async.handlers.InvalidateMatchedSchemaCacheHandler;
+import org.apache.iotdb.confignode.client.async.handlers.RollbackSchemaBlackListHandler;
+import org.apache.iotdb.confignode.procedure.StateMachineProcedure;
+import org.apache.iotdb.confignode.procedure.env.ConfigNodeProcedureEnv;
+import org.apache.iotdb.confignode.procedure.exception.ProcedureException;
+import org.apache.iotdb.confignode.procedure.exception.ProcedureSuspendedException;
+import org.apache.iotdb.confignode.procedure.exception.ProcedureYieldException;
+import org.apache.iotdb.confignode.procedure.state.DeleteTimeSeriesState;
+import org.apache.iotdb.confignode.procedure.store.ProcedureFactory;
+import org.apache.iotdb.db.exception.metadata.PathNotExistException;
+import org.apache.iotdb.db.mpp.common.schematree.PathPatternTree;
+import org.apache.iotdb.mpp.rpc.thrift.TConstructSchemaBlackListReq;
+import org.apache.iotdb.mpp.rpc.thrift.TDeleteDataForDeleteTimeSeriesReq;
+import org.apache.iotdb.mpp.rpc.thrift.TDeleteTimeSeriesReq;
+import org.apache.iotdb.mpp.rpc.thrift.TFetchSchemaBlackListReq;
+import org.apache.iotdb.mpp.rpc.thrift.TFetchSchemaBlackListResp;
+import org.apache.iotdb.mpp.rpc.thrift.TInvalidateMatchedSchemaCacheReq;
+import org.apache.iotdb.mpp.rpc.thrift.TRollbackSchemaBlackListReq;
+import org.apache.iotdb.rpc.TSStatusCode;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.ByteArrayOutputStream;
+import java.io.DataOutputStream;
+import java.io.IOException;
+import java.nio.ByteBuffer;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.stream.Collectors;
+
+public class DeleteTimeSeriesProcedure
+    extends StateMachineProcedure<ConfigNodeProcedureEnv, DeleteTimeSeriesState> {
+
+  private static final Logger LOGGER = LoggerFactory.getLogger(DeleteTimeSeriesProcedure.class);
+
+  private PathPatternTree patternTree;
+  private ByteBuffer patternTreeBytes;
+
+  private String requestMessage;
+
+  public DeleteTimeSeriesProcedure() {
+    super();
+  }
+
+  public DeleteTimeSeriesProcedure(PathPatternTree patternTree) {
+    super();
+    setPatternTree(patternTree);
+  }
+
+  @Override
+  protected Flow executeFromState(ConfigNodeProcedureEnv env, DeleteTimeSeriesState state)
+      throws ProcedureSuspendedException, ProcedureYieldException, InterruptedException {
+    switch (state) {
+      case CONSTRUCT_BLACK_LIST:
+        LOGGER.info("Construct schema black list of timeseries {}", requestMessage);
+        if (constructBlackList(env) > 0) {
+          setNextState(DeleteTimeSeriesState.CLEAN_DATANODE_SCHEMA_CACHE);
+          break;
+        } else {
+          setFailure(
+              new ProcedureException(
+                  new PathNotExistException(
+                      patternTree.getAllPathPatterns().stream()
+                          .map(PartialPath::getFullPath)
+                          .collect(Collectors.toList()))));
+          return Flow.NO_MORE_STATE;
+        }
+      case CLEAN_DATANODE_SCHEMA_CACHE:
+        LOGGER.info("Invalidate cache of timeseries {}", requestMessage);
+        invalidateCache(env);
+        break;
+      case DELETE_DATA:
+        LOGGER.info("Delete data of timeseries {}", requestMessage);
+        deleteData(env);
+        break;
+      case DELETE_TIMESERIES:
+        LOGGER.info("Delete timeseries schema of {}", requestMessage);
+        deleteTimeSeries(env);
+        return Flow.NO_MORE_STATE;
+      default:
+        setFailure(new ProcedureException("Unrecognized state " + state.toString()));
+        return Flow.NO_MORE_STATE;
+    }
+    return Flow.HAS_MORE_STATE;
+  }
+
+  private int constructBlackList(ConfigNodeProcedureEnv env) {
+    Map<TConsensusGroupId, TRegionReplicaSet> targetSchemaRegionGroup =
+        env.getConfigManager().getRelatedSchemaRegionGroup(patternTree);
+    if (targetSchemaRegionGroup.isEmpty()) {
+      return 0;
+    }
+    RegionTask<TSStatus> constructBlackListTask =
+        new RegionTask<TSStatus>("construct schema black list", env, targetSchemaRegionGroup) {
+          @Override
+          Map<Integer, TSStatus> sendRequest(
+              TDataNodeLocation dataNodeLocation, List<TConsensusGroupId> consensusGroupIdList) {
+            // construct request and send
+            Map<Integer, TDataNodeLocation> dataNodeLocationMap = new HashMap<>();
+            dataNodeLocationMap.put(dataNodeLocation.getDataNodeId(), dataNodeLocation);
+            ConstructSchemaBlackListHandler handler =
+                new ConstructSchemaBlackListHandler(dataNodeLocationMap);
+            AsyncDataNodeClientPool.getInstance()
+                .sendAsyncRequestToDataNodeWithRetry(
+                    new TConstructSchemaBlackListReq(consensusGroupIdList, patternTreeBytes),
+                    handler);
+            handler
+                .getDataNodeResponseStatusMap()
+                .forEach(
+                    (k, v) -> {
+                      if (v.getCode() == TSStatusCode.SUCCESS_STATUS.getStatusCode()) {
+                        responseMap.put(k, v);
+                      }
+                    });
+            return handler.getDataNodeResponseStatusMap();
+          }
+        };
+    constructBlackListTask.execute();
+
+    if (isFailed()) {
+      return 0;
+    }
+
+    int preDeletedNum = 0;
+    for (TSStatus resp : constructBlackListTask.getResponseMap().values()) {
+      preDeletedNum += Integer.parseInt(resp.getMessage());
+    }
+    return preDeletedNum;
+  }
+
+  private void invalidateCache(ConfigNodeProcedureEnv env) {
+    Map<Integer, TDataNodeLocation> dataNodeLocationMap =
+        env.getConfigManager().getNodeManager().getRegisteredDataNodeLocations();
+    InvalidateMatchedSchemaCacheHandler handler =
+        new InvalidateMatchedSchemaCacheHandler(dataNodeLocationMap);
+    AsyncDataNodeClientPool.getInstance()
+        .sendAsyncRequestToDataNodeWithRetry(
+            new TInvalidateMatchedSchemaCacheReq(patternTreeBytes), handler);
+    Map<Integer, TSStatus> statusMap = handler.getDataNodeResponseStatusMap();
+    for (TSStatus status : statusMap.values()) {
+      // all dataNodes must clear the related schema cache
+      if (status.getCode() != TSStatusCode.SUCCESS_STATUS.getStatusCode()) {
+        LOGGER.error("Failed to invalidate schema cache of timeseries {}", requestMessage);
+        setFailure(new ProcedureException("Invalidate schema cache failed"));
+        return;
+      }
+    }
+
+    setNextState(DeleteTimeSeriesState.DELETE_DATA);
+  }
+
+  private void deleteData(ConfigNodeProcedureEnv env) {
+    Map<TConsensusGroupId, TRegionReplicaSet> relatedSchemaRegionGroup =
+        env.getConfigManager().getRelatedSchemaRegionGroup(patternTree);
+    Map<TDataNodeLocation, List<TConsensusGroupId>> dataNodeSchemaRegionGroupGroupIdMap =
+        getLeaderDataNodeRegionGroupMap(
+            env.getConfigManager().getPartitionManager().getAllLeadership(),
+            relatedSchemaRegionGroup);
+
+    // fetch schema black list by dataNode
+    for (Map.Entry<TDataNodeLocation, List<TConsensusGroupId>> entry :
+        dataNodeSchemaRegionGroupGroupIdMap.entrySet()) {
+      Map<TConsensusGroupId, TRegionReplicaSet> targetSchemaRegionGroup = new HashMap<>();
+      entry
+          .getValue()
+          .forEach(
+              consensusGroupId ->
+                  targetSchemaRegionGroup.put(
+                      consensusGroupId, relatedSchemaRegionGroup.get(consensusGroupId)));
+      PathPatternTree patternTree =
+          fetchSchemaBlackListOnTargetDataNode(env, targetSchemaRegionGroup);
+      if (isFailed()) {
+        return;
+      }
+      if (patternTree == null) {
+        LOGGER.error(
+            "Failed to fetch schema black list for delete data of timeseries {} on {}",
+            requestMessage,
+            entry.getKey());
+        setFailure(new ProcedureException("Fetch schema black list forDelete data failed"));
+        return;
+      }
+
+      if (patternTree.isEmpty()) {
+        continue;
+      }
+
+      Map<TConsensusGroupId, TRegionReplicaSet> relatedDataRegionGroup =
+          env.getConfigManager().getRelatedDataRegionGroup(patternTree);
+
+      // target timeseries has no data
+      if (relatedDataRegionGroup.isEmpty()) {
+        continue;
+      }
+
+      RegionTask<TSStatus> deleteDataTask =
+          new RegionTask<TSStatus>("delete data", env, relatedDataRegionGroup) {
+            @Override
+            Map<Integer, TSStatus> sendRequest(
+                TDataNodeLocation dataNodeLocation, List<TConsensusGroupId> consensusGroupIdList) {
+              Map<Integer, TDataNodeLocation> dataNodeLocationMap = new HashMap<>();
+              dataNodeLocationMap.put(dataNodeLocation.getDataNodeId(), dataNodeLocation);
+              DeleteDataForDeleteTimeSeriesHandler handler =
+                  new DeleteDataForDeleteTimeSeriesHandler(dataNodeLocationMap);
+              AsyncDataNodeClientPool.getInstance()
+                  .sendAsyncRequestToDataNodeWithRetry(
+                      new TDeleteDataForDeleteTimeSeriesReq(
+                          new ArrayList<>(consensusGroupIdList),
+                          preparePatternTreeBytesData(patternTree)),
+                      handler);
+              return handler.getDataNodeResponseStatusMap();
+            }
+          };
+      deleteDataTask.setExecuteOnAllReplicaset(true);
+      deleteDataTask.execute();
+      if (isFailed()) {
+        return;
+      }
+    }
+    setNextState(DeleteTimeSeriesState.DELETE_TIMESERIES);
+  }
+
+  private PathPatternTree fetchSchemaBlackListOnTargetDataNode(
+      ConfigNodeProcedureEnv env,
+      Map<TConsensusGroupId, TRegionReplicaSet> targetSchemaRegionGroup) {
+    RegionTask<TFetchSchemaBlackListResp> fetchSchemaBlackListTask =
+        new RegionTask<TFetchSchemaBlackListResp>(
+            "fetch schema black list", env, targetSchemaRegionGroup) {
+          @Override
+          Map<Integer, TSStatus> sendRequest(
+              TDataNodeLocation dataNodeLocation, List<TConsensusGroupId> consensusGroupIdList) {
+            Map<Integer, TDataNodeLocation> dataNodeLocationMap = new HashMap<>();
+            dataNodeLocationMap.put(dataNodeLocation.getDataNodeId(), dataNodeLocation);
+            FetchSchemaBlackLsitHandler handler =
+                new FetchSchemaBlackLsitHandler(dataNodeLocationMap);
+            AsyncDataNodeClientPool.getInstance()
+                .sendAsyncRequestToDataNodeWithRetry(
+                    new TFetchSchemaBlackListReq(consensusGroupIdList, patternTreeBytes), handler);
+            Map<Integer, TFetchSchemaBlackListResp> respMap = handler.getDataNodeResponseMap();
+            Map<Integer, TSStatus> statusMap = new HashMap<>();
+            respMap.forEach(
+                (k, v) -> {
+                  if (v.getStatus().getCode() == TSStatusCode.SUCCESS_STATUS.getStatusCode()) {
+                    responseMap.put(k, v);
+                  }
+                  statusMap.put(k, v.getStatus());
+                });
+            return statusMap;
+          }
+        };
+    fetchSchemaBlackListTask.execute();
+    if (isFailed()) {
+      return null;
+    }
+
+    Map<Integer, TFetchSchemaBlackListResp> respMap = fetchSchemaBlackListTask.getResponseMap();
+    PathPatternTree patternTree = new PathPatternTree();
+    for (TFetchSchemaBlackListResp resp : respMap.values()) {
+      for (PartialPath path :
+          PathPatternTree.deserialize(ByteBuffer.wrap(resp.getPathPatternTree()))
+              .getAllPathPatterns()) {
+        patternTree.appendPathPattern(path);
+      }
+    }
+    patternTree.constructTree();
+    return patternTree;
+  }
+
+  private void deleteTimeSeries(ConfigNodeProcedureEnv env) {
+    RegionTask<TSStatus> deleteTimeSeriesTask =
+        new RegionTask<TSStatus>(
+            "delete timeseries",
+            env,
+            env.getConfigManager().getRelatedSchemaRegionGroup(patternTree)) {
+          @Override
+          Map<Integer, TSStatus> sendRequest(
+              TDataNodeLocation dataNodeLocation, List<TConsensusGroupId> consensusGroupIdList) {
+            Map<Integer, TDataNodeLocation> dataNodeLocationMap = new HashMap<>();
+            dataNodeLocationMap.put(dataNodeLocation.getDataNodeId(), dataNodeLocation);
+            DeleteTimeSeriesHandler handler = new DeleteTimeSeriesHandler(dataNodeLocationMap);
+            AsyncDataNodeClientPool.getInstance()
+                .sendAsyncRequestToDataNodeWithRetry(
+                    new TDeleteTimeSeriesReq(consensusGroupIdList, patternTreeBytes), handler);
+            return handler.getDataNodeResponseStatusMap();
+          }
+        };
+    deleteTimeSeriesTask.execute();
+  }
+
+  private Map<TDataNodeLocation, List<TConsensusGroupId>> getLeaderDataNodeRegionGroupMap(
+      Map<TConsensusGroupId, Integer> leaderMap,
+      Map<TConsensusGroupId, TRegionReplicaSet> regionReplicaSetMap) {
+    Map<TDataNodeLocation, List<TConsensusGroupId>> dataNodeConsensusGroupIdMap = new HashMap<>();
+    regionReplicaSetMap.forEach(
+        (consensusGroupId, regionReplicaSet) -> {
+          Integer leaderId = leaderMap.get(consensusGroupId);
+          TDataNodeLocation leaderDataNodeLocation = null;
+          if (leaderId == null || leaderId == -1) {
+            leaderDataNodeLocation = regionReplicaSet.getDataNodeLocations().get(0);
+          } else {
+            for (TDataNodeLocation dataNodeLocation : regionReplicaSet.getDataNodeLocations()) {
+              if (dataNodeLocation.getDataNodeId() == leaderId) {
+                leaderDataNodeLocation = dataNodeLocation;
+                break;
+              }
+            }
+          }
+          dataNodeConsensusGroupIdMap
+              .computeIfAbsent(leaderDataNodeLocation, k -> new ArrayList<>())
+              .add(regionReplicaSet.getRegionId());
+        });
+    return dataNodeConsensusGroupIdMap;
+  }
+
+  private Map<TDataNodeLocation, List<TConsensusGroupId>> getAllReplicaDataNodeRegionGroupMap(
+      Map<TConsensusGroupId, TRegionReplicaSet> regionReplicaSetMap) {
+    Map<TDataNodeLocation, List<TConsensusGroupId>> dataNodeConsensusGroupIdMap = new HashMap<>();
+    regionReplicaSetMap.forEach(
+        (consensusGroupId, regionReplicaSet) -> {
+          for (TDataNodeLocation dataNodeLocation : regionReplicaSet.getDataNodeLocations()) {
+            dataNodeConsensusGroupIdMap
+                .computeIfAbsent(dataNodeLocation, k -> new ArrayList<>())
+                .add(regionReplicaSet.getRegionId());
+          }
+        });
+    return dataNodeConsensusGroupIdMap;
+  }
+
+  @Override
+  protected void rollbackState(
+      ConfigNodeProcedureEnv env, DeleteTimeSeriesState deleteTimeSeriesState)
+      throws IOException, InterruptedException, ProcedureException {
+    RegionTask<TSStatus> rollbackStateTask =
+        new RegionTask<TSStatus>(
+            "roll back schema black list",
+            env,
+            env.getConfigManager().getRelatedSchemaRegionGroup(patternTree)) {
+          @Override
+          Map<Integer, TSStatus> sendRequest(
+              TDataNodeLocation dataNodeLocation, List<TConsensusGroupId> consensusGroupIdList) {
+            Map<Integer, TDataNodeLocation> dataNodeLocationMap = new HashMap<>();
+            dataNodeLocationMap.put(dataNodeLocation.getDataNodeId(), dataNodeLocation);
+            RollbackSchemaBlackListHandler handler =
+                new RollbackSchemaBlackListHandler(dataNodeLocationMap);
+            AsyncDataNodeClientPool.getInstance()
+                .sendAsyncRequestToDataNodeWithRetry(
+                    new TRollbackSchemaBlackListReq(consensusGroupIdList, patternTreeBytes),
+                    handler);
+            return handler.getDataNodeResponseStatusMap();
+          }
+        };
+    rollbackStateTask.execute();
+  }
+
+  @Override
+  protected boolean isRollbackSupported(DeleteTimeSeriesState deleteTimeSeriesState) {
+    return true;
+  }
+
+  @Override
+  protected DeleteTimeSeriesState getState(int stateId) {
+    return DeleteTimeSeriesState.values()[stateId];
+  }
+
+  @Override
+  protected int getStateId(DeleteTimeSeriesState deleteTimeSeriesState) {
+    return deleteTimeSeriesState.ordinal();
+  }
+
+  @Override
+  protected DeleteTimeSeriesState getInitialState() {
+    return DeleteTimeSeriesState.CONSTRUCT_BLACK_LIST;
+  }
+
+  public PathPatternTree getPatternTree() {
+    return patternTree;
+  }
+
+  public void setPatternTree(PathPatternTree patternTree) {
+    this.patternTree = patternTree;
+    requestMessage = patternTree.getAllPathPatterns().toString();
+    patternTreeBytes = preparePatternTreeBytesData(patternTree);
+  }
+
+  private ByteBuffer preparePatternTreeBytesData(PathPatternTree patternTree) {
+    ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream();
+    DataOutputStream dataOutputStream = new DataOutputStream(byteArrayOutputStream);
+    try {
+      patternTree.serialize(dataOutputStream);
+    } catch (IOException ignored) {
+
+    }
+    return ByteBuffer.wrap(byteArrayOutputStream.toByteArray());
+  }
+
+  @Override
+  public void serialize(DataOutputStream stream) throws IOException {
+    stream.writeInt(ProcedureFactory.ProcedureType.DELETE_TIMESERIES_PROCEDURE.ordinal());
+    super.serialize(stream);
+    patternTree.serialize(stream);
+  }
+
+  @Override
+  public void deserialize(ByteBuffer byteBuffer) {
+    super.deserialize(byteBuffer);
+    setPatternTree(PathPatternTree.deserialize(byteBuffer));
+  }
+
+  @Override
+  public boolean equals(Object o) {
+    if (this == o) return true;
+    if (o == null || getClass() != o.getClass()) return false;
+    DeleteTimeSeriesProcedure that = (DeleteTimeSeriesProcedure) o;
+    return this.getProcId() == that.getProcId()
+        && this.getState() == that.getState()
+        && patternTree.equals(that.patternTree);
+  }
+
+  @Override
+  public int hashCode() {
+    return Objects.hash(getProcId(), getState(), patternTree);
+  }
+
+  private abstract class RegionTask<T> {
+
+    private final String taskName;
+    private final ConfigNodeProcedureEnv env;
+    private final Map<TConsensusGroupId, TRegionReplicaSet> targetSchemaRegionGroup;
+
+    private boolean executeOnAllReplicaset = false;
+
+    protected Map<Integer, T> responseMap = new ConcurrentHashMap<>();
+
+    RegionTask(
+        String taskName,
+        ConfigNodeProcedureEnv env,
+        Map<TConsensusGroupId, TRegionReplicaSet> targetSchemaRegionGroup) {
+      this.taskName = taskName;
+      this.env = env;
+      this.targetSchemaRegionGroup = targetSchemaRegionGroup;
+    }
+
+    private void execute() {
+      // organize schema region by dataNode
+      Set<TDataNodeLocation> allFailedDataNodeSet = new HashSet<>();
+      Map<TDataNodeLocation, List<TConsensusGroupId>> dataNodeConsensusGroupIdMap =
+          executeOnAllReplicaset
+              ? getAllReplicaDataNodeRegionGroupMap(targetSchemaRegionGroup)
+              : getLeaderDataNodeRegionGroupMap(
+                  env.getConfigManager().getPartitionManager().getAllLeadership(),
+                  targetSchemaRegionGroup);
+      while (!dataNodeConsensusGroupIdMap.isEmpty()) {
+        Map<TDataNodeLocation, List<TConsensusGroupId>> currentFailedDataNodeMap =
+            sendSchemaRegionRequest(dataNodeConsensusGroupIdMap);
+        // some dataNode execution failure
+        if (isFailed()) {
+          return;
+        }
+
+        if (currentFailedDataNodeMap.isEmpty()) {
+          break;
+        }
+
+        // retry failed dataNode requests caused by unexpected error on other replicates on other
+        // dataNodes
+        currentFailedDataNodeMap.forEach(dataNodeConsensusGroupIdMap::remove);
+        // remove dataNodes that successfully executed request
+        allFailedDataNodeSet.removeAll(dataNodeConsensusGroupIdMap.keySet());
+        dataNodeConsensusGroupIdMap =
+            getAvailableDataNodeLocationForRetry(currentFailedDataNodeMap, allFailedDataNodeSet);
+        // some consensus group has no available dataNode
+        if (isFailed()) {
+          return;
+        }
+      }
+    }
+
+    private Map<TDataNodeLocation, List<TConsensusGroupId>> sendSchemaRegionRequest(
+        Map<TDataNodeLocation, List<TConsensusGroupId>> dataNodeConsensusGroupIdMap) {
+      // send request to each dataNode
+      Map<TDataNodeLocation, List<TConsensusGroupId>> failedDataNodeMap = new HashMap<>();
+      for (Map.Entry<TDataNodeLocation, List<TConsensusGroupId>> entry :
+          dataNodeConsensusGroupIdMap.entrySet()) {
+        // process response
+        Map<Integer, TSStatus> dataNodeResponseMap = sendRequest(entry.getKey(), entry.getValue());
+        TSStatus currentDataNodeResponse = dataNodeResponseMap.get(entry.getKey().getDataNodeId());
+        if (currentDataNodeResponse.getCode() != TSStatusCode.SUCCESS_STATUS.getStatusCode()) {
+          if (currentDataNodeResponse.getCode() == TSStatusCode.MULTIPLE_ERROR.getStatusCode()) {
+            // dataNode execution error
+            LOGGER.error(
+                "Failed to execute [{}] of delete timeseries {} on {}",
+                taskName,
+                requestMessage,
+                entry.getKey());
+            setFailure(
+                new ProcedureException(
+                    String.format(
+                        "Delete timeseries %s failed on when [%s]", requestMessage, taskName)));
+            break;
+          } else {
+            // unexpected error, retry on other replicates on other dataNodes
+            failedDataNodeMap.put(entry.getKey(), entry.getValue());
+          }
+        }
+      }
+      return failedDataNodeMap;
+    }
+
+    private Map<TDataNodeLocation, List<TConsensusGroupId>> getAvailableDataNodeLocationForRetry(
+        Map<TDataNodeLocation, List<TConsensusGroupId>> failedDataNodeConsensusGroupIdMap,
+        Set<TDataNodeLocation> allFailedDataNodeSet) {
+      Map<TConsensusGroupId, Integer> leaderMap =
+          env.getConfigManager().getPartitionManager().getAllLeadership();
+      Map<TDataNodeLocation, List<TConsensusGroupId>> availableDataNodeLocation = new HashMap<>();
+      for (List<TConsensusGroupId> consensusGroupIdList :
+          failedDataNodeConsensusGroupIdMap.values()) {
+        for (TConsensusGroupId consensusGroupId : consensusGroupIdList) {
+          TRegionReplicaSet regionReplicaSet = targetSchemaRegionGroup.get(consensusGroupId);
+          TDataNodeLocation selectedDataNode = null;
+          Integer leaderId = leaderMap.get(consensusGroupId);
+          if (leaderId == null || leaderId == -1) {
+            for (TDataNodeLocation candidateDataNode : regionReplicaSet.getDataNodeLocations()) {
+              if (!allFailedDataNodeSet.contains(candidateDataNode)) {
+                if (selectedDataNode == null) {
+                  selectedDataNode = candidateDataNode;
+                }
+              }

Review Comment:
   ```suggestion
                 if (!allFailedDataNodeSet.contains(candidateDataNode)) {
                   selectedDataNode = candidateDataNode;
                   break;
                 }
   ```



##########
confignode/src/main/java/org/apache/iotdb/confignode/procedure/impl/DeleteTimeSeriesProcedure.java:
##########
@@ -0,0 +1,611 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.confignode.procedure.impl;
+
+import org.apache.iotdb.common.rpc.thrift.TConsensusGroupId;
+import org.apache.iotdb.common.rpc.thrift.TDataNodeLocation;
+import org.apache.iotdb.common.rpc.thrift.TRegionReplicaSet;
+import org.apache.iotdb.common.rpc.thrift.TSStatus;
+import org.apache.iotdb.commons.path.PartialPath;
+import org.apache.iotdb.confignode.client.async.datanode.AsyncDataNodeClientPool;
+import org.apache.iotdb.confignode.client.async.handlers.ConstructSchemaBlackListHandler;
+import org.apache.iotdb.confignode.client.async.handlers.DeleteDataForDeleteTimeSeriesHandler;
+import org.apache.iotdb.confignode.client.async.handlers.DeleteTimeSeriesHandler;
+import org.apache.iotdb.confignode.client.async.handlers.FetchSchemaBlackLsitHandler;
+import org.apache.iotdb.confignode.client.async.handlers.InvalidateMatchedSchemaCacheHandler;
+import org.apache.iotdb.confignode.client.async.handlers.RollbackSchemaBlackListHandler;
+import org.apache.iotdb.confignode.procedure.StateMachineProcedure;
+import org.apache.iotdb.confignode.procedure.env.ConfigNodeProcedureEnv;
+import org.apache.iotdb.confignode.procedure.exception.ProcedureException;
+import org.apache.iotdb.confignode.procedure.exception.ProcedureSuspendedException;
+import org.apache.iotdb.confignode.procedure.exception.ProcedureYieldException;
+import org.apache.iotdb.confignode.procedure.state.DeleteTimeSeriesState;
+import org.apache.iotdb.confignode.procedure.store.ProcedureFactory;
+import org.apache.iotdb.db.exception.metadata.PathNotExistException;
+import org.apache.iotdb.db.mpp.common.schematree.PathPatternTree;
+import org.apache.iotdb.mpp.rpc.thrift.TConstructSchemaBlackListReq;
+import org.apache.iotdb.mpp.rpc.thrift.TDeleteDataForDeleteTimeSeriesReq;
+import org.apache.iotdb.mpp.rpc.thrift.TDeleteTimeSeriesReq;
+import org.apache.iotdb.mpp.rpc.thrift.TFetchSchemaBlackListReq;
+import org.apache.iotdb.mpp.rpc.thrift.TFetchSchemaBlackListResp;
+import org.apache.iotdb.mpp.rpc.thrift.TInvalidateMatchedSchemaCacheReq;
+import org.apache.iotdb.mpp.rpc.thrift.TRollbackSchemaBlackListReq;
+import org.apache.iotdb.rpc.TSStatusCode;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.ByteArrayOutputStream;
+import java.io.DataOutputStream;
+import java.io.IOException;
+import java.nio.ByteBuffer;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.stream.Collectors;
+
+public class DeleteTimeSeriesProcedure
+    extends StateMachineProcedure<ConfigNodeProcedureEnv, DeleteTimeSeriesState> {
+
+  private static final Logger LOGGER = LoggerFactory.getLogger(DeleteTimeSeriesProcedure.class);
+
+  private PathPatternTree patternTree;
+  private ByteBuffer patternTreeBytes;
+
+  private String requestMessage;
+
+  public DeleteTimeSeriesProcedure() {
+    super();
+  }
+
+  public DeleteTimeSeriesProcedure(PathPatternTree patternTree) {
+    super();
+    setPatternTree(patternTree);
+  }
+
+  @Override
+  protected Flow executeFromState(ConfigNodeProcedureEnv env, DeleteTimeSeriesState state)
+      throws ProcedureSuspendedException, ProcedureYieldException, InterruptedException {
+    switch (state) {
+      case CONSTRUCT_BLACK_LIST:
+        LOGGER.info("Construct schema black list of timeseries {}", requestMessage);
+        if (constructBlackList(env) > 0) {
+          setNextState(DeleteTimeSeriesState.CLEAN_DATANODE_SCHEMA_CACHE);
+          break;
+        } else {
+          setFailure(
+              new ProcedureException(
+                  new PathNotExistException(
+                      patternTree.getAllPathPatterns().stream()
+                          .map(PartialPath::getFullPath)
+                          .collect(Collectors.toList()))));
+          return Flow.NO_MORE_STATE;
+        }
+      case CLEAN_DATANODE_SCHEMA_CACHE:
+        LOGGER.info("Invalidate cache of timeseries {}", requestMessage);
+        invalidateCache(env);
+        break;
+      case DELETE_DATA:
+        LOGGER.info("Delete data of timeseries {}", requestMessage);
+        deleteData(env);
+        break;
+      case DELETE_TIMESERIES:
+        LOGGER.info("Delete timeseries schema of {}", requestMessage);
+        deleteTimeSeries(env);
+        return Flow.NO_MORE_STATE;
+      default:
+        setFailure(new ProcedureException("Unrecognized state " + state.toString()));
+        return Flow.NO_MORE_STATE;
+    }
+    return Flow.HAS_MORE_STATE;
+  }
+
+  private int constructBlackList(ConfigNodeProcedureEnv env) {

Review Comment:
   Add function annotations to show what the return value stands for.



##########
confignode/src/main/java/org/apache/iotdb/confignode/procedure/impl/DeleteTimeSeriesProcedure.java:
##########
@@ -0,0 +1,611 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.confignode.procedure.impl;
+
+import org.apache.iotdb.common.rpc.thrift.TConsensusGroupId;
+import org.apache.iotdb.common.rpc.thrift.TDataNodeLocation;
+import org.apache.iotdb.common.rpc.thrift.TRegionReplicaSet;
+import org.apache.iotdb.common.rpc.thrift.TSStatus;
+import org.apache.iotdb.commons.path.PartialPath;
+import org.apache.iotdb.confignode.client.async.datanode.AsyncDataNodeClientPool;
+import org.apache.iotdb.confignode.client.async.handlers.ConstructSchemaBlackListHandler;
+import org.apache.iotdb.confignode.client.async.handlers.DeleteDataForDeleteTimeSeriesHandler;
+import org.apache.iotdb.confignode.client.async.handlers.DeleteTimeSeriesHandler;
+import org.apache.iotdb.confignode.client.async.handlers.FetchSchemaBlackLsitHandler;
+import org.apache.iotdb.confignode.client.async.handlers.InvalidateMatchedSchemaCacheHandler;
+import org.apache.iotdb.confignode.client.async.handlers.RollbackSchemaBlackListHandler;
+import org.apache.iotdb.confignode.procedure.StateMachineProcedure;
+import org.apache.iotdb.confignode.procedure.env.ConfigNodeProcedureEnv;
+import org.apache.iotdb.confignode.procedure.exception.ProcedureException;
+import org.apache.iotdb.confignode.procedure.exception.ProcedureSuspendedException;
+import org.apache.iotdb.confignode.procedure.exception.ProcedureYieldException;
+import org.apache.iotdb.confignode.procedure.state.DeleteTimeSeriesState;
+import org.apache.iotdb.confignode.procedure.store.ProcedureFactory;
+import org.apache.iotdb.db.exception.metadata.PathNotExistException;
+import org.apache.iotdb.db.mpp.common.schematree.PathPatternTree;
+import org.apache.iotdb.mpp.rpc.thrift.TConstructSchemaBlackListReq;
+import org.apache.iotdb.mpp.rpc.thrift.TDeleteDataForDeleteTimeSeriesReq;
+import org.apache.iotdb.mpp.rpc.thrift.TDeleteTimeSeriesReq;
+import org.apache.iotdb.mpp.rpc.thrift.TFetchSchemaBlackListReq;
+import org.apache.iotdb.mpp.rpc.thrift.TFetchSchemaBlackListResp;
+import org.apache.iotdb.mpp.rpc.thrift.TInvalidateMatchedSchemaCacheReq;
+import org.apache.iotdb.mpp.rpc.thrift.TRollbackSchemaBlackListReq;
+import org.apache.iotdb.rpc.TSStatusCode;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.ByteArrayOutputStream;
+import java.io.DataOutputStream;
+import java.io.IOException;
+import java.nio.ByteBuffer;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.stream.Collectors;
+
+public class DeleteTimeSeriesProcedure
+    extends StateMachineProcedure<ConfigNodeProcedureEnv, DeleteTimeSeriesState> {
+
+  private static final Logger LOGGER = LoggerFactory.getLogger(DeleteTimeSeriesProcedure.class);
+
+  private PathPatternTree patternTree;
+  private ByteBuffer patternTreeBytes;
+
+  private String requestMessage;
+
+  public DeleteTimeSeriesProcedure() {
+    super();
+  }
+
+  public DeleteTimeSeriesProcedure(PathPatternTree patternTree) {
+    super();
+    setPatternTree(patternTree);
+  }
+
+  @Override
+  protected Flow executeFromState(ConfigNodeProcedureEnv env, DeleteTimeSeriesState state)
+      throws ProcedureSuspendedException, ProcedureYieldException, InterruptedException {
+    switch (state) {
+      case CONSTRUCT_BLACK_LIST:
+        LOGGER.info("Construct schema black list of timeseries {}", requestMessage);
+        if (constructBlackList(env) > 0) {
+          setNextState(DeleteTimeSeriesState.CLEAN_DATANODE_SCHEMA_CACHE);
+          break;
+        } else {
+          setFailure(
+              new ProcedureException(
+                  new PathNotExistException(
+                      patternTree.getAllPathPatterns().stream()
+                          .map(PartialPath::getFullPath)
+                          .collect(Collectors.toList()))));
+          return Flow.NO_MORE_STATE;
+        }
+      case CLEAN_DATANODE_SCHEMA_CACHE:
+        LOGGER.info("Invalidate cache of timeseries {}", requestMessage);
+        invalidateCache(env);
+        break;
+      case DELETE_DATA:
+        LOGGER.info("Delete data of timeseries {}", requestMessage);
+        deleteData(env);
+        break;
+      case DELETE_TIMESERIES:
+        LOGGER.info("Delete timeseries schema of {}", requestMessage);
+        deleteTimeSeries(env);
+        return Flow.NO_MORE_STATE;
+      default:
+        setFailure(new ProcedureException("Unrecognized state " + state.toString()));
+        return Flow.NO_MORE_STATE;
+    }
+    return Flow.HAS_MORE_STATE;
+  }
+
+  private int constructBlackList(ConfigNodeProcedureEnv env) {
+    Map<TConsensusGroupId, TRegionReplicaSet> targetSchemaRegionGroup =
+        env.getConfigManager().getRelatedSchemaRegionGroup(patternTree);
+    if (targetSchemaRegionGroup.isEmpty()) {
+      return 0;
+    }
+    RegionTask<TSStatus> constructBlackListTask =
+        new RegionTask<TSStatus>("construct schema black list", env, targetSchemaRegionGroup) {
+          @Override
+          Map<Integer, TSStatus> sendRequest(
+              TDataNodeLocation dataNodeLocation, List<TConsensusGroupId> consensusGroupIdList) {
+            // construct request and send
+            Map<Integer, TDataNodeLocation> dataNodeLocationMap = new HashMap<>();
+            dataNodeLocationMap.put(dataNodeLocation.getDataNodeId(), dataNodeLocation);
+            ConstructSchemaBlackListHandler handler =
+                new ConstructSchemaBlackListHandler(dataNodeLocationMap);
+            AsyncDataNodeClientPool.getInstance()
+                .sendAsyncRequestToDataNodeWithRetry(
+                    new TConstructSchemaBlackListReq(consensusGroupIdList, patternTreeBytes),
+                    handler);
+            handler
+                .getDataNodeResponseStatusMap()
+                .forEach(
+                    (k, v) -> {
+                      if (v.getCode() == TSStatusCode.SUCCESS_STATUS.getStatusCode()) {
+                        responseMap.put(k, v);
+                      }
+                    });
+            return handler.getDataNodeResponseStatusMap();
+          }
+        };
+    constructBlackListTask.execute();
+
+    if (isFailed()) {
+      return 0;
+    }
+
+    int preDeletedNum = 0;
+    for (TSStatus resp : constructBlackListTask.getResponseMap().values()) {
+      preDeletedNum += Integer.parseInt(resp.getMessage());
+    }
+    return preDeletedNum;
+  }
+
+  private void invalidateCache(ConfigNodeProcedureEnv env) {
+    Map<Integer, TDataNodeLocation> dataNodeLocationMap =
+        env.getConfigManager().getNodeManager().getRegisteredDataNodeLocations();
+    InvalidateMatchedSchemaCacheHandler handler =
+        new InvalidateMatchedSchemaCacheHandler(dataNodeLocationMap);
+    AsyncDataNodeClientPool.getInstance()
+        .sendAsyncRequestToDataNodeWithRetry(
+            new TInvalidateMatchedSchemaCacheReq(patternTreeBytes), handler);
+    Map<Integer, TSStatus> statusMap = handler.getDataNodeResponseStatusMap();
+    for (TSStatus status : statusMap.values()) {
+      // all dataNodes must clear the related schema cache
+      if (status.getCode() != TSStatusCode.SUCCESS_STATUS.getStatusCode()) {
+        LOGGER.error("Failed to invalidate schema cache of timeseries {}", requestMessage);
+        setFailure(new ProcedureException("Invalidate schema cache failed"));
+        return;
+      }
+    }
+
+    setNextState(DeleteTimeSeriesState.DELETE_DATA);
+  }
+
+  private void deleteData(ConfigNodeProcedureEnv env) {
+    Map<TConsensusGroupId, TRegionReplicaSet> relatedSchemaRegionGroup =
+        env.getConfigManager().getRelatedSchemaRegionGroup(patternTree);
+    Map<TDataNodeLocation, List<TConsensusGroupId>> dataNodeSchemaRegionGroupGroupIdMap =
+        getLeaderDataNodeRegionGroupMap(
+            env.getConfigManager().getPartitionManager().getAllLeadership(),
+            relatedSchemaRegionGroup);
+
+    // fetch schema black list by dataNode
+    for (Map.Entry<TDataNodeLocation, List<TConsensusGroupId>> entry :
+        dataNodeSchemaRegionGroupGroupIdMap.entrySet()) {
+      Map<TConsensusGroupId, TRegionReplicaSet> targetSchemaRegionGroup = new HashMap<>();
+      entry
+          .getValue()
+          .forEach(
+              consensusGroupId ->
+                  targetSchemaRegionGroup.put(
+                      consensusGroupId, relatedSchemaRegionGroup.get(consensusGroupId)));
+      PathPatternTree patternTree =
+          fetchSchemaBlackListOnTargetDataNode(env, targetSchemaRegionGroup);
+      if (isFailed()) {
+        return;
+      }
+      if (patternTree == null) {
+        LOGGER.error(
+            "Failed to fetch schema black list for delete data of timeseries {} on {}",
+            requestMessage,
+            entry.getKey());
+        setFailure(new ProcedureException("Fetch schema black list forDelete data failed"));
+        return;
+      }
+
+      if (patternTree.isEmpty()) {
+        continue;
+      }
+
+      Map<TConsensusGroupId, TRegionReplicaSet> relatedDataRegionGroup =
+          env.getConfigManager().getRelatedDataRegionGroup(patternTree);
+
+      // target timeseries has no data
+      if (relatedDataRegionGroup.isEmpty()) {
+        continue;
+      }
+
+      RegionTask<TSStatus> deleteDataTask =
+          new RegionTask<TSStatus>("delete data", env, relatedDataRegionGroup) {
+            @Override
+            Map<Integer, TSStatus> sendRequest(
+                TDataNodeLocation dataNodeLocation, List<TConsensusGroupId> consensusGroupIdList) {
+              Map<Integer, TDataNodeLocation> dataNodeLocationMap = new HashMap<>();
+              dataNodeLocationMap.put(dataNodeLocation.getDataNodeId(), dataNodeLocation);
+              DeleteDataForDeleteTimeSeriesHandler handler =
+                  new DeleteDataForDeleteTimeSeriesHandler(dataNodeLocationMap);
+              AsyncDataNodeClientPool.getInstance()
+                  .sendAsyncRequestToDataNodeWithRetry(
+                      new TDeleteDataForDeleteTimeSeriesReq(
+                          new ArrayList<>(consensusGroupIdList),
+                          preparePatternTreeBytesData(patternTree)),
+                      handler);
+              return handler.getDataNodeResponseStatusMap();
+            }
+          };
+      deleteDataTask.setExecuteOnAllReplicaset(true);
+      deleteDataTask.execute();
+      if (isFailed()) {
+        return;
+      }
+    }
+    setNextState(DeleteTimeSeriesState.DELETE_TIMESERIES);
+  }
+
+  private PathPatternTree fetchSchemaBlackListOnTargetDataNode(
+      ConfigNodeProcedureEnv env,
+      Map<TConsensusGroupId, TRegionReplicaSet> targetSchemaRegionGroup) {
+    RegionTask<TFetchSchemaBlackListResp> fetchSchemaBlackListTask =
+        new RegionTask<TFetchSchemaBlackListResp>(
+            "fetch schema black list", env, targetSchemaRegionGroup) {
+          @Override
+          Map<Integer, TSStatus> sendRequest(
+              TDataNodeLocation dataNodeLocation, List<TConsensusGroupId> consensusGroupIdList) {
+            Map<Integer, TDataNodeLocation> dataNodeLocationMap = new HashMap<>();
+            dataNodeLocationMap.put(dataNodeLocation.getDataNodeId(), dataNodeLocation);
+            FetchSchemaBlackLsitHandler handler =
+                new FetchSchemaBlackLsitHandler(dataNodeLocationMap);
+            AsyncDataNodeClientPool.getInstance()
+                .sendAsyncRequestToDataNodeWithRetry(
+                    new TFetchSchemaBlackListReq(consensusGroupIdList, patternTreeBytes), handler);
+            Map<Integer, TFetchSchemaBlackListResp> respMap = handler.getDataNodeResponseMap();
+            Map<Integer, TSStatus> statusMap = new HashMap<>();
+            respMap.forEach(
+                (k, v) -> {
+                  if (v.getStatus().getCode() == TSStatusCode.SUCCESS_STATUS.getStatusCode()) {
+                    responseMap.put(k, v);
+                  }
+                  statusMap.put(k, v.getStatus());
+                });
+            return statusMap;
+          }
+        };
+    fetchSchemaBlackListTask.execute();
+    if (isFailed()) {
+      return null;
+    }
+
+    Map<Integer, TFetchSchemaBlackListResp> respMap = fetchSchemaBlackListTask.getResponseMap();
+    PathPatternTree patternTree = new PathPatternTree();
+    for (TFetchSchemaBlackListResp resp : respMap.values()) {
+      for (PartialPath path :
+          PathPatternTree.deserialize(ByteBuffer.wrap(resp.getPathPatternTree()))
+              .getAllPathPatterns()) {
+        patternTree.appendPathPattern(path);
+      }
+    }
+    patternTree.constructTree();
+    return patternTree;
+  }
+
+  private void deleteTimeSeries(ConfigNodeProcedureEnv env) {
+    RegionTask<TSStatus> deleteTimeSeriesTask =
+        new RegionTask<TSStatus>(
+            "delete timeseries",
+            env,
+            env.getConfigManager().getRelatedSchemaRegionGroup(patternTree)) {
+          @Override
+          Map<Integer, TSStatus> sendRequest(
+              TDataNodeLocation dataNodeLocation, List<TConsensusGroupId> consensusGroupIdList) {
+            Map<Integer, TDataNodeLocation> dataNodeLocationMap = new HashMap<>();
+            dataNodeLocationMap.put(dataNodeLocation.getDataNodeId(), dataNodeLocation);
+            DeleteTimeSeriesHandler handler = new DeleteTimeSeriesHandler(dataNodeLocationMap);
+            AsyncDataNodeClientPool.getInstance()
+                .sendAsyncRequestToDataNodeWithRetry(
+                    new TDeleteTimeSeriesReq(consensusGroupIdList, patternTreeBytes), handler);
+            return handler.getDataNodeResponseStatusMap();
+          }
+        };
+    deleteTimeSeriesTask.execute();
+  }
+
+  private Map<TDataNodeLocation, List<TConsensusGroupId>> getLeaderDataNodeRegionGroupMap(

Review Comment:
   Add function annotations to show the difference between getAllReplicaDataNodeRegionGroupMap.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [iotdb] xingtanzjr commented on a diff in pull request #7242: [IOTDB-3184] Implement Delete Timeseries for Cluster

Posted by GitBox <gi...@apache.org>.
xingtanzjr commented on code in PR #7242:
URL: https://github.com/apache/iotdb/pull/7242#discussion_r969207947


##########
server/src/main/java/org/apache/iotdb/db/metadata/mnode/MeasurementMNode.java:
##########
@@ -46,6 +46,8 @@ public class MeasurementMNode extends MNode implements IMeasurementMNode {
   private long offset = -1;
   /** measurement's Schema for one timeseries represented by current leaf node */
   private IMeasurementSchema schema;
+  /** whether this measurement is been pre deleted and considered in black list */

Review Comment:
   Typo `is been`



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [iotdb] MarcosZyk commented on a diff in pull request #7242: [IOTDB-3184] Implement Delete Timeseries for Cluster

Posted by GitBox <gi...@apache.org>.
MarcosZyk commented on code in PR #7242:
URL: https://github.com/apache/iotdb/pull/7242#discussion_r970204279


##########
confignode/src/main/java/org/apache/iotdb/confignode/procedure/impl/DeleteTimeSeriesProcedure.java:
##########
@@ -0,0 +1,611 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.confignode.procedure.impl;
+
+import org.apache.iotdb.common.rpc.thrift.TConsensusGroupId;
+import org.apache.iotdb.common.rpc.thrift.TDataNodeLocation;
+import org.apache.iotdb.common.rpc.thrift.TRegionReplicaSet;
+import org.apache.iotdb.common.rpc.thrift.TSStatus;
+import org.apache.iotdb.commons.path.PartialPath;
+import org.apache.iotdb.confignode.client.async.datanode.AsyncDataNodeClientPool;
+import org.apache.iotdb.confignode.client.async.handlers.ConstructSchemaBlackListHandler;
+import org.apache.iotdb.confignode.client.async.handlers.DeleteDataForDeleteTimeSeriesHandler;
+import org.apache.iotdb.confignode.client.async.handlers.DeleteTimeSeriesHandler;
+import org.apache.iotdb.confignode.client.async.handlers.FetchSchemaBlackLsitHandler;
+import org.apache.iotdb.confignode.client.async.handlers.InvalidateMatchedSchemaCacheHandler;
+import org.apache.iotdb.confignode.client.async.handlers.RollbackSchemaBlackListHandler;
+import org.apache.iotdb.confignode.procedure.StateMachineProcedure;
+import org.apache.iotdb.confignode.procedure.env.ConfigNodeProcedureEnv;
+import org.apache.iotdb.confignode.procedure.exception.ProcedureException;
+import org.apache.iotdb.confignode.procedure.exception.ProcedureSuspendedException;
+import org.apache.iotdb.confignode.procedure.exception.ProcedureYieldException;
+import org.apache.iotdb.confignode.procedure.state.DeleteTimeSeriesState;
+import org.apache.iotdb.confignode.procedure.store.ProcedureFactory;
+import org.apache.iotdb.db.exception.metadata.PathNotExistException;
+import org.apache.iotdb.db.mpp.common.schematree.PathPatternTree;
+import org.apache.iotdb.mpp.rpc.thrift.TConstructSchemaBlackListReq;
+import org.apache.iotdb.mpp.rpc.thrift.TDeleteDataForDeleteTimeSeriesReq;
+import org.apache.iotdb.mpp.rpc.thrift.TDeleteTimeSeriesReq;
+import org.apache.iotdb.mpp.rpc.thrift.TFetchSchemaBlackListReq;
+import org.apache.iotdb.mpp.rpc.thrift.TFetchSchemaBlackListResp;
+import org.apache.iotdb.mpp.rpc.thrift.TInvalidateMatchedSchemaCacheReq;
+import org.apache.iotdb.mpp.rpc.thrift.TRollbackSchemaBlackListReq;
+import org.apache.iotdb.rpc.TSStatusCode;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.ByteArrayOutputStream;
+import java.io.DataOutputStream;
+import java.io.IOException;
+import java.nio.ByteBuffer;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.stream.Collectors;
+
+public class DeleteTimeSeriesProcedure
+    extends StateMachineProcedure<ConfigNodeProcedureEnv, DeleteTimeSeriesState> {
+
+  private static final Logger LOGGER = LoggerFactory.getLogger(DeleteTimeSeriesProcedure.class);
+
+  private PathPatternTree patternTree;
+  private ByteBuffer patternTreeBytes;
+
+  private String requestMessage;
+
+  public DeleteTimeSeriesProcedure() {
+    super();
+  }
+
+  public DeleteTimeSeriesProcedure(PathPatternTree patternTree) {
+    super();
+    setPatternTree(patternTree);
+  }
+
+  @Override
+  protected Flow executeFromState(ConfigNodeProcedureEnv env, DeleteTimeSeriesState state)
+      throws ProcedureSuspendedException, ProcedureYieldException, InterruptedException {
+    switch (state) {
+      case CONSTRUCT_BLACK_LIST:
+        LOGGER.info("Construct schema black list of timeseries {}", requestMessage);
+        if (constructBlackList(env) > 0) {
+          setNextState(DeleteTimeSeriesState.CLEAN_DATANODE_SCHEMA_CACHE);
+          break;
+        } else {
+          setFailure(
+              new ProcedureException(
+                  new PathNotExistException(
+                      patternTree.getAllPathPatterns().stream()
+                          .map(PartialPath::getFullPath)
+                          .collect(Collectors.toList()))));
+          return Flow.NO_MORE_STATE;
+        }
+      case CLEAN_DATANODE_SCHEMA_CACHE:
+        LOGGER.info("Invalidate cache of timeseries {}", requestMessage);
+        invalidateCache(env);
+        break;
+      case DELETE_DATA:
+        LOGGER.info("Delete data of timeseries {}", requestMessage);
+        deleteData(env);
+        break;
+      case DELETE_TIMESERIES:
+        LOGGER.info("Delete timeseries schema of {}", requestMessage);
+        deleteTimeSeries(env);
+        return Flow.NO_MORE_STATE;
+      default:
+        setFailure(new ProcedureException("Unrecognized state " + state.toString()));
+        return Flow.NO_MORE_STATE;
+    }
+    return Flow.HAS_MORE_STATE;
+  }
+
+  private int constructBlackList(ConfigNodeProcedureEnv env) {
+    Map<TConsensusGroupId, TRegionReplicaSet> targetSchemaRegionGroup =
+        env.getConfigManager().getRelatedSchemaRegionGroup(patternTree);
+    if (targetSchemaRegionGroup.isEmpty()) {
+      return 0;
+    }
+    RegionTask<TSStatus> constructBlackListTask =
+        new RegionTask<TSStatus>("construct schema black list", env, targetSchemaRegionGroup) {
+          @Override
+          Map<Integer, TSStatus> sendRequest(
+              TDataNodeLocation dataNodeLocation, List<TConsensusGroupId> consensusGroupIdList) {
+            // construct request and send
+            Map<Integer, TDataNodeLocation> dataNodeLocationMap = new HashMap<>();
+            dataNodeLocationMap.put(dataNodeLocation.getDataNodeId(), dataNodeLocation);
+            ConstructSchemaBlackListHandler handler =
+                new ConstructSchemaBlackListHandler(dataNodeLocationMap);
+            AsyncDataNodeClientPool.getInstance()
+                .sendAsyncRequestToDataNodeWithRetry(
+                    new TConstructSchemaBlackListReq(consensusGroupIdList, patternTreeBytes),
+                    handler);
+            handler
+                .getDataNodeResponseStatusMap()
+                .forEach(
+                    (k, v) -> {
+                      if (v.getCode() == TSStatusCode.SUCCESS_STATUS.getStatusCode()) {
+                        responseMap.put(k, v);
+                      }
+                    });
+            return handler.getDataNodeResponseStatusMap();
+          }
+        };
+    constructBlackListTask.execute();
+
+    if (isFailed()) {
+      return 0;
+    }
+
+    int preDeletedNum = 0;
+    for (TSStatus resp : constructBlackListTask.getResponseMap().values()) {
+      preDeletedNum += Integer.parseInt(resp.getMessage());
+    }
+    return preDeletedNum;
+  }
+
+  private void invalidateCache(ConfigNodeProcedureEnv env) {
+    Map<Integer, TDataNodeLocation> dataNodeLocationMap =
+        env.getConfigManager().getNodeManager().getRegisteredDataNodeLocations();
+    InvalidateMatchedSchemaCacheHandler handler =
+        new InvalidateMatchedSchemaCacheHandler(dataNodeLocationMap);
+    AsyncDataNodeClientPool.getInstance()
+        .sendAsyncRequestToDataNodeWithRetry(
+            new TInvalidateMatchedSchemaCacheReq(patternTreeBytes), handler);
+    Map<Integer, TSStatus> statusMap = handler.getDataNodeResponseStatusMap();
+    for (TSStatus status : statusMap.values()) {
+      // all dataNodes must clear the related schema cache
+      if (status.getCode() != TSStatusCode.SUCCESS_STATUS.getStatusCode()) {
+        LOGGER.error("Failed to invalidate schema cache of timeseries {}", requestMessage);
+        setFailure(new ProcedureException("Invalidate schema cache failed"));
+        return;
+      }
+    }
+
+    setNextState(DeleteTimeSeriesState.DELETE_DATA);
+  }
+
+  private void deleteData(ConfigNodeProcedureEnv env) {
+    Map<TConsensusGroupId, TRegionReplicaSet> relatedSchemaRegionGroup =
+        env.getConfigManager().getRelatedSchemaRegionGroup(patternTree);
+    Map<TDataNodeLocation, List<TConsensusGroupId>> dataNodeSchemaRegionGroupGroupIdMap =
+        getLeaderDataNodeRegionGroupMap(
+            env.getConfigManager().getPartitionManager().getAllLeadership(),
+            relatedSchemaRegionGroup);
+
+    // fetch schema black list by dataNode
+    for (Map.Entry<TDataNodeLocation, List<TConsensusGroupId>> entry :
+        dataNodeSchemaRegionGroupGroupIdMap.entrySet()) {
+      Map<TConsensusGroupId, TRegionReplicaSet> targetSchemaRegionGroup = new HashMap<>();
+      entry
+          .getValue()
+          .forEach(
+              consensusGroupId ->
+                  targetSchemaRegionGroup.put(
+                      consensusGroupId, relatedSchemaRegionGroup.get(consensusGroupId)));
+      PathPatternTree patternTree =
+          fetchSchemaBlackListOnTargetDataNode(env, targetSchemaRegionGroup);
+      if (isFailed()) {
+        return;
+      }
+      if (patternTree == null) {
+        LOGGER.error(
+            "Failed to fetch schema black list for delete data of timeseries {} on {}",
+            requestMessage,
+            entry.getKey());
+        setFailure(new ProcedureException("Fetch schema black list forDelete data failed"));
+        return;
+      }
+
+      if (patternTree.isEmpty()) {
+        continue;
+      }
+
+      Map<TConsensusGroupId, TRegionReplicaSet> relatedDataRegionGroup =
+          env.getConfigManager().getRelatedDataRegionGroup(patternTree);
+
+      // target timeseries has no data
+      if (relatedDataRegionGroup.isEmpty()) {
+        continue;
+      }
+
+      RegionTask<TSStatus> deleteDataTask =
+          new RegionTask<TSStatus>("delete data", env, relatedDataRegionGroup) {
+            @Override
+            Map<Integer, TSStatus> sendRequest(
+                TDataNodeLocation dataNodeLocation, List<TConsensusGroupId> consensusGroupIdList) {
+              Map<Integer, TDataNodeLocation> dataNodeLocationMap = new HashMap<>();
+              dataNodeLocationMap.put(dataNodeLocation.getDataNodeId(), dataNodeLocation);
+              DeleteDataForDeleteTimeSeriesHandler handler =
+                  new DeleteDataForDeleteTimeSeriesHandler(dataNodeLocationMap);
+              AsyncDataNodeClientPool.getInstance()
+                  .sendAsyncRequestToDataNodeWithRetry(
+                      new TDeleteDataForDeleteTimeSeriesReq(
+                          new ArrayList<>(consensusGroupIdList),
+                          preparePatternTreeBytesData(patternTree)),
+                      handler);
+              return handler.getDataNodeResponseStatusMap();
+            }
+          };
+      deleteDataTask.setExecuteOnAllReplicaset(true);
+      deleteDataTask.execute();
+      if (isFailed()) {
+        return;
+      }
+    }
+    setNextState(DeleteTimeSeriesState.DELETE_TIMESERIES);
+  }
+
+  private PathPatternTree fetchSchemaBlackListOnTargetDataNode(
+      ConfigNodeProcedureEnv env,
+      Map<TConsensusGroupId, TRegionReplicaSet> targetSchemaRegionGroup) {
+    RegionTask<TFetchSchemaBlackListResp> fetchSchemaBlackListTask =
+        new RegionTask<TFetchSchemaBlackListResp>(
+            "fetch schema black list", env, targetSchemaRegionGroup) {
+          @Override
+          Map<Integer, TSStatus> sendRequest(
+              TDataNodeLocation dataNodeLocation, List<TConsensusGroupId> consensusGroupIdList) {
+            Map<Integer, TDataNodeLocation> dataNodeLocationMap = new HashMap<>();
+            dataNodeLocationMap.put(dataNodeLocation.getDataNodeId(), dataNodeLocation);
+            FetchSchemaBlackLsitHandler handler =
+                new FetchSchemaBlackLsitHandler(dataNodeLocationMap);
+            AsyncDataNodeClientPool.getInstance()
+                .sendAsyncRequestToDataNodeWithRetry(
+                    new TFetchSchemaBlackListReq(consensusGroupIdList, patternTreeBytes), handler);
+            Map<Integer, TFetchSchemaBlackListResp> respMap = handler.getDataNodeResponseMap();
+            Map<Integer, TSStatus> statusMap = new HashMap<>();
+            respMap.forEach(
+                (k, v) -> {
+                  if (v.getStatus().getCode() == TSStatusCode.SUCCESS_STATUS.getStatusCode()) {
+                    responseMap.put(k, v);
+                  }
+                  statusMap.put(k, v.getStatus());
+                });
+            return statusMap;
+          }
+        };
+    fetchSchemaBlackListTask.execute();
+    if (isFailed()) {
+      return null;
+    }
+
+    Map<Integer, TFetchSchemaBlackListResp> respMap = fetchSchemaBlackListTask.getResponseMap();
+    PathPatternTree patternTree = new PathPatternTree();
+    for (TFetchSchemaBlackListResp resp : respMap.values()) {
+      for (PartialPath path :
+          PathPatternTree.deserialize(ByteBuffer.wrap(resp.getPathPatternTree()))
+              .getAllPathPatterns()) {
+        patternTree.appendPathPattern(path);
+      }
+    }
+    patternTree.constructTree();
+    return patternTree;
+  }
+
+  private void deleteTimeSeries(ConfigNodeProcedureEnv env) {
+    RegionTask<TSStatus> deleteTimeSeriesTask =
+        new RegionTask<TSStatus>(
+            "delete timeseries",
+            env,
+            env.getConfigManager().getRelatedSchemaRegionGroup(patternTree)) {
+          @Override
+          Map<Integer, TSStatus> sendRequest(
+              TDataNodeLocation dataNodeLocation, List<TConsensusGroupId> consensusGroupIdList) {
+            Map<Integer, TDataNodeLocation> dataNodeLocationMap = new HashMap<>();
+            dataNodeLocationMap.put(dataNodeLocation.getDataNodeId(), dataNodeLocation);
+            DeleteTimeSeriesHandler handler = new DeleteTimeSeriesHandler(dataNodeLocationMap);
+            AsyncDataNodeClientPool.getInstance()
+                .sendAsyncRequestToDataNodeWithRetry(
+                    new TDeleteTimeSeriesReq(consensusGroupIdList, patternTreeBytes), handler);
+            return handler.getDataNodeResponseStatusMap();
+          }
+        };
+    deleteTimeSeriesTask.execute();
+  }
+
+  private Map<TDataNodeLocation, List<TConsensusGroupId>> getLeaderDataNodeRegionGroupMap(

Review Comment:
   Added.



##########
confignode/src/main/java/org/apache/iotdb/confignode/procedure/impl/DeleteTimeSeriesProcedure.java:
##########
@@ -0,0 +1,611 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.confignode.procedure.impl;
+
+import org.apache.iotdb.common.rpc.thrift.TConsensusGroupId;
+import org.apache.iotdb.common.rpc.thrift.TDataNodeLocation;
+import org.apache.iotdb.common.rpc.thrift.TRegionReplicaSet;
+import org.apache.iotdb.common.rpc.thrift.TSStatus;
+import org.apache.iotdb.commons.path.PartialPath;
+import org.apache.iotdb.confignode.client.async.datanode.AsyncDataNodeClientPool;
+import org.apache.iotdb.confignode.client.async.handlers.ConstructSchemaBlackListHandler;
+import org.apache.iotdb.confignode.client.async.handlers.DeleteDataForDeleteTimeSeriesHandler;
+import org.apache.iotdb.confignode.client.async.handlers.DeleteTimeSeriesHandler;
+import org.apache.iotdb.confignode.client.async.handlers.FetchSchemaBlackLsitHandler;
+import org.apache.iotdb.confignode.client.async.handlers.InvalidateMatchedSchemaCacheHandler;
+import org.apache.iotdb.confignode.client.async.handlers.RollbackSchemaBlackListHandler;
+import org.apache.iotdb.confignode.procedure.StateMachineProcedure;
+import org.apache.iotdb.confignode.procedure.env.ConfigNodeProcedureEnv;
+import org.apache.iotdb.confignode.procedure.exception.ProcedureException;
+import org.apache.iotdb.confignode.procedure.exception.ProcedureSuspendedException;
+import org.apache.iotdb.confignode.procedure.exception.ProcedureYieldException;
+import org.apache.iotdb.confignode.procedure.state.DeleteTimeSeriesState;
+import org.apache.iotdb.confignode.procedure.store.ProcedureFactory;
+import org.apache.iotdb.db.exception.metadata.PathNotExistException;
+import org.apache.iotdb.db.mpp.common.schematree.PathPatternTree;
+import org.apache.iotdb.mpp.rpc.thrift.TConstructSchemaBlackListReq;
+import org.apache.iotdb.mpp.rpc.thrift.TDeleteDataForDeleteTimeSeriesReq;
+import org.apache.iotdb.mpp.rpc.thrift.TDeleteTimeSeriesReq;
+import org.apache.iotdb.mpp.rpc.thrift.TFetchSchemaBlackListReq;
+import org.apache.iotdb.mpp.rpc.thrift.TFetchSchemaBlackListResp;
+import org.apache.iotdb.mpp.rpc.thrift.TInvalidateMatchedSchemaCacheReq;
+import org.apache.iotdb.mpp.rpc.thrift.TRollbackSchemaBlackListReq;
+import org.apache.iotdb.rpc.TSStatusCode;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.ByteArrayOutputStream;
+import java.io.DataOutputStream;
+import java.io.IOException;
+import java.nio.ByteBuffer;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.stream.Collectors;
+
+public class DeleteTimeSeriesProcedure
+    extends StateMachineProcedure<ConfigNodeProcedureEnv, DeleteTimeSeriesState> {
+
+  private static final Logger LOGGER = LoggerFactory.getLogger(DeleteTimeSeriesProcedure.class);
+
+  private PathPatternTree patternTree;
+  private ByteBuffer patternTreeBytes;
+
+  private String requestMessage;
+
+  public DeleteTimeSeriesProcedure() {
+    super();
+  }
+
+  public DeleteTimeSeriesProcedure(PathPatternTree patternTree) {
+    super();
+    setPatternTree(patternTree);
+  }
+
+  @Override
+  protected Flow executeFromState(ConfigNodeProcedureEnv env, DeleteTimeSeriesState state)
+      throws ProcedureSuspendedException, ProcedureYieldException, InterruptedException {
+    switch (state) {
+      case CONSTRUCT_BLACK_LIST:
+        LOGGER.info("Construct schema black list of timeseries {}", requestMessage);
+        if (constructBlackList(env) > 0) {
+          setNextState(DeleteTimeSeriesState.CLEAN_DATANODE_SCHEMA_CACHE);
+          break;
+        } else {
+          setFailure(
+              new ProcedureException(
+                  new PathNotExistException(
+                      patternTree.getAllPathPatterns().stream()
+                          .map(PartialPath::getFullPath)
+                          .collect(Collectors.toList()))));
+          return Flow.NO_MORE_STATE;
+        }
+      case CLEAN_DATANODE_SCHEMA_CACHE:
+        LOGGER.info("Invalidate cache of timeseries {}", requestMessage);
+        invalidateCache(env);
+        break;
+      case DELETE_DATA:
+        LOGGER.info("Delete data of timeseries {}", requestMessage);
+        deleteData(env);
+        break;
+      case DELETE_TIMESERIES:
+        LOGGER.info("Delete timeseries schema of {}", requestMessage);
+        deleteTimeSeries(env);
+        return Flow.NO_MORE_STATE;
+      default:
+        setFailure(new ProcedureException("Unrecognized state " + state.toString()));
+        return Flow.NO_MORE_STATE;
+    }
+    return Flow.HAS_MORE_STATE;
+  }
+
+  private int constructBlackList(ConfigNodeProcedureEnv env) {
+    Map<TConsensusGroupId, TRegionReplicaSet> targetSchemaRegionGroup =
+        env.getConfigManager().getRelatedSchemaRegionGroup(patternTree);
+    if (targetSchemaRegionGroup.isEmpty()) {
+      return 0;
+    }
+    RegionTask<TSStatus> constructBlackListTask =
+        new RegionTask<TSStatus>("construct schema black list", env, targetSchemaRegionGroup) {
+          @Override
+          Map<Integer, TSStatus> sendRequest(
+              TDataNodeLocation dataNodeLocation, List<TConsensusGroupId> consensusGroupIdList) {
+            // construct request and send
+            Map<Integer, TDataNodeLocation> dataNodeLocationMap = new HashMap<>();
+            dataNodeLocationMap.put(dataNodeLocation.getDataNodeId(), dataNodeLocation);
+            ConstructSchemaBlackListHandler handler =
+                new ConstructSchemaBlackListHandler(dataNodeLocationMap);
+            AsyncDataNodeClientPool.getInstance()
+                .sendAsyncRequestToDataNodeWithRetry(
+                    new TConstructSchemaBlackListReq(consensusGroupIdList, patternTreeBytes),
+                    handler);
+            handler
+                .getDataNodeResponseStatusMap()
+                .forEach(
+                    (k, v) -> {
+                      if (v.getCode() == TSStatusCode.SUCCESS_STATUS.getStatusCode()) {
+                        responseMap.put(k, v);
+                      }
+                    });
+            return handler.getDataNodeResponseStatusMap();
+          }
+        };
+    constructBlackListTask.execute();
+
+    if (isFailed()) {
+      return 0;
+    }
+
+    int preDeletedNum = 0;
+    for (TSStatus resp : constructBlackListTask.getResponseMap().values()) {
+      preDeletedNum += Integer.parseInt(resp.getMessage());
+    }
+    return preDeletedNum;
+  }
+
+  private void invalidateCache(ConfigNodeProcedureEnv env) {
+    Map<Integer, TDataNodeLocation> dataNodeLocationMap =
+        env.getConfigManager().getNodeManager().getRegisteredDataNodeLocations();
+    InvalidateMatchedSchemaCacheHandler handler =
+        new InvalidateMatchedSchemaCacheHandler(dataNodeLocationMap);
+    AsyncDataNodeClientPool.getInstance()
+        .sendAsyncRequestToDataNodeWithRetry(
+            new TInvalidateMatchedSchemaCacheReq(patternTreeBytes), handler);
+    Map<Integer, TSStatus> statusMap = handler.getDataNodeResponseStatusMap();
+    for (TSStatus status : statusMap.values()) {
+      // all dataNodes must clear the related schema cache
+      if (status.getCode() != TSStatusCode.SUCCESS_STATUS.getStatusCode()) {
+        LOGGER.error("Failed to invalidate schema cache of timeseries {}", requestMessage);
+        setFailure(new ProcedureException("Invalidate schema cache failed"));
+        return;
+      }
+    }
+
+    setNextState(DeleteTimeSeriesState.DELETE_DATA);
+  }
+
+  private void deleteData(ConfigNodeProcedureEnv env) {
+    Map<TConsensusGroupId, TRegionReplicaSet> relatedSchemaRegionGroup =
+        env.getConfigManager().getRelatedSchemaRegionGroup(patternTree);
+    Map<TDataNodeLocation, List<TConsensusGroupId>> dataNodeSchemaRegionGroupGroupIdMap =
+        getLeaderDataNodeRegionGroupMap(
+            env.getConfigManager().getPartitionManager().getAllLeadership(),
+            relatedSchemaRegionGroup);
+
+    // fetch schema black list by dataNode
+    for (Map.Entry<TDataNodeLocation, List<TConsensusGroupId>> entry :
+        dataNodeSchemaRegionGroupGroupIdMap.entrySet()) {
+      Map<TConsensusGroupId, TRegionReplicaSet> targetSchemaRegionGroup = new HashMap<>();
+      entry
+          .getValue()
+          .forEach(
+              consensusGroupId ->
+                  targetSchemaRegionGroup.put(
+                      consensusGroupId, relatedSchemaRegionGroup.get(consensusGroupId)));
+      PathPatternTree patternTree =
+          fetchSchemaBlackListOnTargetDataNode(env, targetSchemaRegionGroup);
+      if (isFailed()) {
+        return;
+      }
+      if (patternTree == null) {
+        LOGGER.error(
+            "Failed to fetch schema black list for delete data of timeseries {} on {}",
+            requestMessage,
+            entry.getKey());
+        setFailure(new ProcedureException("Fetch schema black list forDelete data failed"));
+        return;
+      }
+
+      if (patternTree.isEmpty()) {
+        continue;
+      }
+
+      Map<TConsensusGroupId, TRegionReplicaSet> relatedDataRegionGroup =
+          env.getConfigManager().getRelatedDataRegionGroup(patternTree);
+
+      // target timeseries has no data
+      if (relatedDataRegionGroup.isEmpty()) {
+        continue;
+      }
+
+      RegionTask<TSStatus> deleteDataTask =
+          new RegionTask<TSStatus>("delete data", env, relatedDataRegionGroup) {
+            @Override
+            Map<Integer, TSStatus> sendRequest(
+                TDataNodeLocation dataNodeLocation, List<TConsensusGroupId> consensusGroupIdList) {
+              Map<Integer, TDataNodeLocation> dataNodeLocationMap = new HashMap<>();
+              dataNodeLocationMap.put(dataNodeLocation.getDataNodeId(), dataNodeLocation);
+              DeleteDataForDeleteTimeSeriesHandler handler =
+                  new DeleteDataForDeleteTimeSeriesHandler(dataNodeLocationMap);
+              AsyncDataNodeClientPool.getInstance()
+                  .sendAsyncRequestToDataNodeWithRetry(
+                      new TDeleteDataForDeleteTimeSeriesReq(
+                          new ArrayList<>(consensusGroupIdList),
+                          preparePatternTreeBytesData(patternTree)),
+                      handler);
+              return handler.getDataNodeResponseStatusMap();
+            }
+          };
+      deleteDataTask.setExecuteOnAllReplicaset(true);
+      deleteDataTask.execute();
+      if (isFailed()) {
+        return;
+      }
+    }
+    setNextState(DeleteTimeSeriesState.DELETE_TIMESERIES);
+  }
+
+  private PathPatternTree fetchSchemaBlackListOnTargetDataNode(
+      ConfigNodeProcedureEnv env,
+      Map<TConsensusGroupId, TRegionReplicaSet> targetSchemaRegionGroup) {
+    RegionTask<TFetchSchemaBlackListResp> fetchSchemaBlackListTask =
+        new RegionTask<TFetchSchemaBlackListResp>(
+            "fetch schema black list", env, targetSchemaRegionGroup) {
+          @Override
+          Map<Integer, TSStatus> sendRequest(
+              TDataNodeLocation dataNodeLocation, List<TConsensusGroupId> consensusGroupIdList) {
+            Map<Integer, TDataNodeLocation> dataNodeLocationMap = new HashMap<>();
+            dataNodeLocationMap.put(dataNodeLocation.getDataNodeId(), dataNodeLocation);
+            FetchSchemaBlackLsitHandler handler =
+                new FetchSchemaBlackLsitHandler(dataNodeLocationMap);
+            AsyncDataNodeClientPool.getInstance()
+                .sendAsyncRequestToDataNodeWithRetry(
+                    new TFetchSchemaBlackListReq(consensusGroupIdList, patternTreeBytes), handler);
+            Map<Integer, TFetchSchemaBlackListResp> respMap = handler.getDataNodeResponseMap();
+            Map<Integer, TSStatus> statusMap = new HashMap<>();
+            respMap.forEach(
+                (k, v) -> {
+                  if (v.getStatus().getCode() == TSStatusCode.SUCCESS_STATUS.getStatusCode()) {
+                    responseMap.put(k, v);
+                  }
+                  statusMap.put(k, v.getStatus());
+                });
+            return statusMap;
+          }
+        };
+    fetchSchemaBlackListTask.execute();
+    if (isFailed()) {
+      return null;
+    }
+
+    Map<Integer, TFetchSchemaBlackListResp> respMap = fetchSchemaBlackListTask.getResponseMap();
+    PathPatternTree patternTree = new PathPatternTree();
+    for (TFetchSchemaBlackListResp resp : respMap.values()) {
+      for (PartialPath path :
+          PathPatternTree.deserialize(ByteBuffer.wrap(resp.getPathPatternTree()))
+              .getAllPathPatterns()) {
+        patternTree.appendPathPattern(path);
+      }
+    }
+    patternTree.constructTree();
+    return patternTree;
+  }
+
+  private void deleteTimeSeries(ConfigNodeProcedureEnv env) {
+    RegionTask<TSStatus> deleteTimeSeriesTask =
+        new RegionTask<TSStatus>(
+            "delete timeseries",
+            env,
+            env.getConfigManager().getRelatedSchemaRegionGroup(patternTree)) {
+          @Override
+          Map<Integer, TSStatus> sendRequest(
+              TDataNodeLocation dataNodeLocation, List<TConsensusGroupId> consensusGroupIdList) {
+            Map<Integer, TDataNodeLocation> dataNodeLocationMap = new HashMap<>();
+            dataNodeLocationMap.put(dataNodeLocation.getDataNodeId(), dataNodeLocation);
+            DeleteTimeSeriesHandler handler = new DeleteTimeSeriesHandler(dataNodeLocationMap);
+            AsyncDataNodeClientPool.getInstance()
+                .sendAsyncRequestToDataNodeWithRetry(
+                    new TDeleteTimeSeriesReq(consensusGroupIdList, patternTreeBytes), handler);
+            return handler.getDataNodeResponseStatusMap();
+          }
+        };
+    deleteTimeSeriesTask.execute();
+  }
+
+  private Map<TDataNodeLocation, List<TConsensusGroupId>> getLeaderDataNodeRegionGroupMap(
+      Map<TConsensusGroupId, Integer> leaderMap,
+      Map<TConsensusGroupId, TRegionReplicaSet> regionReplicaSetMap) {
+    Map<TDataNodeLocation, List<TConsensusGroupId>> dataNodeConsensusGroupIdMap = new HashMap<>();
+    regionReplicaSetMap.forEach(
+        (consensusGroupId, regionReplicaSet) -> {
+          Integer leaderId = leaderMap.get(consensusGroupId);
+          TDataNodeLocation leaderDataNodeLocation = null;
+          if (leaderId == null || leaderId == -1) {
+            leaderDataNodeLocation = regionReplicaSet.getDataNodeLocations().get(0);
+          } else {
+            for (TDataNodeLocation dataNodeLocation : regionReplicaSet.getDataNodeLocations()) {
+              if (dataNodeLocation.getDataNodeId() == leaderId) {
+                leaderDataNodeLocation = dataNodeLocation;
+                break;
+              }
+            }
+          }
+          dataNodeConsensusGroupIdMap
+              .computeIfAbsent(leaderDataNodeLocation, k -> new ArrayList<>())
+              .add(regionReplicaSet.getRegionId());
+        });
+    return dataNodeConsensusGroupIdMap;
+  }
+
+  private Map<TDataNodeLocation, List<TConsensusGroupId>> getAllReplicaDataNodeRegionGroupMap(

Review Comment:
   Added.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [iotdb] MarcosZyk commented on a diff in pull request #7242: [IOTDB-3184] Implement Delete Timeseries for Cluster

Posted by GitBox <gi...@apache.org>.
MarcosZyk commented on code in PR #7242:
URL: https://github.com/apache/iotdb/pull/7242#discussion_r970204502


##########
confignode/src/main/java/org/apache/iotdb/confignode/client/async/handlers/DeleteTimeSeriesHandler.java:
##########
@@ -0,0 +1,95 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.confignode.client.async.handlers;
+
+import org.apache.iotdb.common.rpc.thrift.TDataNodeLocation;
+import org.apache.iotdb.common.rpc.thrift.TSStatus;
+import org.apache.iotdb.confignode.client.DataNodeRequestType;
+import org.apache.iotdb.rpc.RpcUtils;
+import org.apache.iotdb.rpc.TSStatusCode;
+
+import org.apache.thrift.async.AsyncMethodCallback;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.util.HashMap;
+import java.util.Map;
+import java.util.concurrent.CountDownLatch;
+
+public class DeleteTimeSeriesHandler extends AbstractRetryHandler
+    implements AsyncMethodCallback<TSStatus> {
+
+  private static final Logger LOGGER = LoggerFactory.getLogger(DeleteTimeSeriesHandler.class);
+
+  private Map<Integer, TSStatus> dataNodeResponseStatusMap = new HashMap<>();
+
+  public DeleteTimeSeriesHandler(Map<Integer, TDataNodeLocation> dataNodeLocationMap) {
+    super(DataNodeRequestType.DELETE_TIMESERIES, dataNodeLocationMap);
+  }
+
+  public DeleteTimeSeriesHandler(
+      CountDownLatch countDownLatch,
+      TDataNodeLocation targetDataNode,
+      Map<Integer, TDataNodeLocation> dataNodeLocationMap) {
+    super(
+        countDownLatch, DataNodeRequestType.DELETE_TIMESERIES, targetDataNode, dataNodeLocationMap);
+  }
+
+  public Map<Integer, TSStatus> getDataNodeResponseStatusMap() {
+    return dataNodeResponseStatusMap;
+  }
+
+  @Override
+  public void onComplete(TSStatus tsStatus) {
+    dataNodeResponseStatusMap.put(targetDataNode.getDataNodeId(), tsStatus);
+    if (tsStatus.getCode() == TSStatusCode.SUCCESS_STATUS.getStatusCode()) {
+      dataNodeLocationMap.remove(targetDataNode.getDataNodeId());
+      LOGGER.info("Successfully delete timeseries on DataNode: {}", targetDataNode);
+    } else if (tsStatus.getCode() == TSStatusCode.MULTIPLE_ERROR.getStatusCode()) {
+      dataNodeLocationMap.remove(targetDataNode.getDataNodeId());
+      LOGGER.error(
+          "Failed to delete timeseries on DataNode {}, {}",
+          dataNodeLocationMap.get(targetDataNode.getDataNodeId()),
+          tsStatus);

Review Comment:
   Fixed.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [iotdb] xingtanzjr commented on a diff in pull request #7242: [IOTDB-3184] Implement Delete Timeseries for Cluster

Posted by GitBox <gi...@apache.org>.
xingtanzjr commented on code in PR #7242:
URL: https://github.com/apache/iotdb/pull/7242#discussion_r969514332


##########
confignode/src/main/java/org/apache/iotdb/confignode/procedure/impl/DeleteTimeSeriesProcedure.java:
##########
@@ -0,0 +1,611 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.confignode.procedure.impl;
+
+import org.apache.iotdb.common.rpc.thrift.TConsensusGroupId;
+import org.apache.iotdb.common.rpc.thrift.TDataNodeLocation;
+import org.apache.iotdb.common.rpc.thrift.TRegionReplicaSet;
+import org.apache.iotdb.common.rpc.thrift.TSStatus;
+import org.apache.iotdb.commons.path.PartialPath;
+import org.apache.iotdb.confignode.client.async.datanode.AsyncDataNodeClientPool;
+import org.apache.iotdb.confignode.client.async.handlers.ConstructSchemaBlackListHandler;
+import org.apache.iotdb.confignode.client.async.handlers.DeleteDataForDeleteTimeSeriesHandler;
+import org.apache.iotdb.confignode.client.async.handlers.DeleteTimeSeriesHandler;
+import org.apache.iotdb.confignode.client.async.handlers.FetchSchemaBlackLsitHandler;
+import org.apache.iotdb.confignode.client.async.handlers.InvalidateMatchedSchemaCacheHandler;
+import org.apache.iotdb.confignode.client.async.handlers.RollbackSchemaBlackListHandler;
+import org.apache.iotdb.confignode.procedure.StateMachineProcedure;
+import org.apache.iotdb.confignode.procedure.env.ConfigNodeProcedureEnv;
+import org.apache.iotdb.confignode.procedure.exception.ProcedureException;
+import org.apache.iotdb.confignode.procedure.exception.ProcedureSuspendedException;
+import org.apache.iotdb.confignode.procedure.exception.ProcedureYieldException;
+import org.apache.iotdb.confignode.procedure.state.DeleteTimeSeriesState;
+import org.apache.iotdb.confignode.procedure.store.ProcedureFactory;
+import org.apache.iotdb.db.exception.metadata.PathNotExistException;
+import org.apache.iotdb.db.mpp.common.schematree.PathPatternTree;
+import org.apache.iotdb.mpp.rpc.thrift.TConstructSchemaBlackListReq;
+import org.apache.iotdb.mpp.rpc.thrift.TDeleteDataForDeleteTimeSeriesReq;
+import org.apache.iotdb.mpp.rpc.thrift.TDeleteTimeSeriesReq;
+import org.apache.iotdb.mpp.rpc.thrift.TFetchSchemaBlackListReq;
+import org.apache.iotdb.mpp.rpc.thrift.TFetchSchemaBlackListResp;
+import org.apache.iotdb.mpp.rpc.thrift.TInvalidateMatchedSchemaCacheReq;
+import org.apache.iotdb.mpp.rpc.thrift.TRollbackSchemaBlackListReq;
+import org.apache.iotdb.rpc.TSStatusCode;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.ByteArrayOutputStream;
+import java.io.DataOutputStream;
+import java.io.IOException;
+import java.nio.ByteBuffer;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.stream.Collectors;
+
+public class DeleteTimeSeriesProcedure
+    extends StateMachineProcedure<ConfigNodeProcedureEnv, DeleteTimeSeriesState> {
+
+  private static final Logger LOGGER = LoggerFactory.getLogger(DeleteTimeSeriesProcedure.class);
+
+  private PathPatternTree patternTree;
+  private ByteBuffer patternTreeBytes;
+
+  private String requestMessage;
+
+  public DeleteTimeSeriesProcedure() {
+    super();
+  }
+
+  public DeleteTimeSeriesProcedure(PathPatternTree patternTree) {
+    super();
+    setPatternTree(patternTree);
+  }
+
+  @Override
+  protected Flow executeFromState(ConfigNodeProcedureEnv env, DeleteTimeSeriesState state)
+      throws ProcedureSuspendedException, ProcedureYieldException, InterruptedException {
+    switch (state) {
+      case CONSTRUCT_BLACK_LIST:
+        LOGGER.info("Construct schema black list of timeseries {}", requestMessage);
+        if (constructBlackList(env) > 0) {
+          setNextState(DeleteTimeSeriesState.CLEAN_DATANODE_SCHEMA_CACHE);
+          break;
+        } else {
+          setFailure(
+              new ProcedureException(
+                  new PathNotExistException(
+                      patternTree.getAllPathPatterns().stream()
+                          .map(PartialPath::getFullPath)
+                          .collect(Collectors.toList()))));
+          return Flow.NO_MORE_STATE;
+        }
+      case CLEAN_DATANODE_SCHEMA_CACHE:
+        LOGGER.info("Invalidate cache of timeseries {}", requestMessage);
+        invalidateCache(env);
+        break;
+      case DELETE_DATA:
+        LOGGER.info("Delete data of timeseries {}", requestMessage);
+        deleteData(env);
+        break;
+      case DELETE_TIMESERIES:
+        LOGGER.info("Delete timeseries schema of {}", requestMessage);
+        deleteTimeSeries(env);
+        return Flow.NO_MORE_STATE;
+      default:
+        setFailure(new ProcedureException("Unrecognized state " + state.toString()));
+        return Flow.NO_MORE_STATE;
+    }
+    return Flow.HAS_MORE_STATE;
+  }
+
+  private int constructBlackList(ConfigNodeProcedureEnv env) {
+    Map<TConsensusGroupId, TRegionReplicaSet> targetSchemaRegionGroup =
+        env.getConfigManager().getRelatedSchemaRegionGroup(patternTree);
+    if (targetSchemaRegionGroup.isEmpty()) {
+      return 0;
+    }
+    RegionTask<TSStatus> constructBlackListTask =
+        new RegionTask<TSStatus>("construct schema black list", env, targetSchemaRegionGroup) {
+          @Override
+          Map<Integer, TSStatus> sendRequest(
+              TDataNodeLocation dataNodeLocation, List<TConsensusGroupId> consensusGroupIdList) {
+            // construct request and send
+            Map<Integer, TDataNodeLocation> dataNodeLocationMap = new HashMap<>();
+            dataNodeLocationMap.put(dataNodeLocation.getDataNodeId(), dataNodeLocation);
+            ConstructSchemaBlackListHandler handler =
+                new ConstructSchemaBlackListHandler(dataNodeLocationMap);
+            AsyncDataNodeClientPool.getInstance()
+                .sendAsyncRequestToDataNodeWithRetry(
+                    new TConstructSchemaBlackListReq(consensusGroupIdList, patternTreeBytes),
+                    handler);
+            handler
+                .getDataNodeResponseStatusMap()
+                .forEach(
+                    (k, v) -> {
+                      if (v.getCode() == TSStatusCode.SUCCESS_STATUS.getStatusCode()) {
+                        responseMap.put(k, v);
+                      }
+                    });
+            return handler.getDataNodeResponseStatusMap();
+          }
+        };
+    constructBlackListTask.execute();
+
+    if (isFailed()) {
+      return 0;
+    }
+
+    int preDeletedNum = 0;
+    for (TSStatus resp : constructBlackListTask.getResponseMap().values()) {
+      preDeletedNum += Integer.parseInt(resp.getMessage());
+    }
+    return preDeletedNum;
+  }
+
+  private void invalidateCache(ConfigNodeProcedureEnv env) {
+    Map<Integer, TDataNodeLocation> dataNodeLocationMap =
+        env.getConfigManager().getNodeManager().getRegisteredDataNodeLocations();
+    InvalidateMatchedSchemaCacheHandler handler =
+        new InvalidateMatchedSchemaCacheHandler(dataNodeLocationMap);
+    AsyncDataNodeClientPool.getInstance()
+        .sendAsyncRequestToDataNodeWithRetry(
+            new TInvalidateMatchedSchemaCacheReq(patternTreeBytes), handler);
+    Map<Integer, TSStatus> statusMap = handler.getDataNodeResponseStatusMap();
+    for (TSStatus status : statusMap.values()) {
+      // all dataNodes must clear the related schema cache
+      if (status.getCode() != TSStatusCode.SUCCESS_STATUS.getStatusCode()) {
+        LOGGER.error("Failed to invalidate schema cache of timeseries {}", requestMessage);
+        setFailure(new ProcedureException("Invalidate schema cache failed"));
+        return;
+      }
+    }
+
+    setNextState(DeleteTimeSeriesState.DELETE_DATA);
+  }
+
+  private void deleteData(ConfigNodeProcedureEnv env) {
+    Map<TConsensusGroupId, TRegionReplicaSet> relatedSchemaRegionGroup =
+        env.getConfigManager().getRelatedSchemaRegionGroup(patternTree);
+    Map<TDataNodeLocation, List<TConsensusGroupId>> dataNodeSchemaRegionGroupGroupIdMap =
+        getLeaderDataNodeRegionGroupMap(
+            env.getConfigManager().getPartitionManager().getAllLeadership(),
+            relatedSchemaRegionGroup);
+
+    // fetch schema black list by dataNode
+    for (Map.Entry<TDataNodeLocation, List<TConsensusGroupId>> entry :
+        dataNodeSchemaRegionGroupGroupIdMap.entrySet()) {
+      Map<TConsensusGroupId, TRegionReplicaSet> targetSchemaRegionGroup = new HashMap<>();
+      entry
+          .getValue()
+          .forEach(
+              consensusGroupId ->
+                  targetSchemaRegionGroup.put(
+                      consensusGroupId, relatedSchemaRegionGroup.get(consensusGroupId)));
+      PathPatternTree patternTree =
+          fetchSchemaBlackListOnTargetDataNode(env, targetSchemaRegionGroup);

Review Comment:
   Why do we fetch patternTree from black list rather than the class variable ?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org