[GitHub] [ozone] amaliujia commented on a change in pull request #1914: HDDS-4761. Implement increment count optimization in DeletedBlockLog V2

[GitBox <gi...@apache.org>]

amaliujia commented on a change in pull request #1914:
URL: https://github.com/apache/ozone/pull/1914#discussion_r584440544



##########
File path: hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/block/DeletedBlockLogImplV2.java
##########
@@ -145,8 +148,25 @@ public DeletedBlockLogImplV2(ConfigurationSource conf,
   public void incrementCount(List<Long> txIDs) throws IOException {
     lock.lock();
     try {
-      deletedBlockLogStateManager
-          .increaseRetryCountOfTransactionInDB(new ArrayList<>(txIDs));
+      ArrayList<Long> toUpdateTxIDs = new ArrayList<>();
+      for (Long txID : txIDs) {
+        int currentCount =
+            transactionRetryCountMap.getOrDefault(txID, 0);
+        if (currentCount > maxRetry) {
+          continue;
+        } else {
+          currentCount += 1;
+          if (currentCount > maxRetry) {
+            toUpdateTxIDs.add(txID);
+          }
+          transactionRetryCountMap.put(txID, currentCount);
+        }
+      }
+
+      if (!toUpdateTxIDs.isEmpty()) {

Review comment:
       Done

##########
File path: hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/block/DeletedBlockLogImplV2.java
##########
@@ -79,6 +79,8 @@
   private final Lock lock;
   // Maps txId to set of DNs which are successful in committing the transaction
   private Map<Long, Set<UUID>> transactionToDNsCommitMap;
+  // Maps txId to its retry counts;
+  private Map<Long, Integer> transactionRetryCountMap;

Review comment:
       Done

##########
File path: hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/block/DeletedBlockLogStateManagerImpl.java
##########
@@ -187,19 +190,13 @@ public void increaseRetryCountOfTransactionInDB(
         }
         continue;
       }
-      DeletedBlocksTransaction.Builder builder = block.toBuilder();
-      int currentCount = block.getCount();
-      if (currentCount > -1) {
-        builder.setCount(++currentCount);
-      }
       // if the retry time exceeds the maxRetry value
       // then set the retry value to -1, stop retrying, admins can
       // analyze those blocks and purge them manually by SCMCli.
-      if (currentCount > maxRetry) {
-        builder.setCount(-1);
-      }
+      DeletedBlocksTransaction.Builder builder = block.toBuilder().setCount(-1);
       deletedTable.putWithBatch(
           transactionBuffer.getCurrentBatchOperation(), txID, builder.build());
+      skippingRetryTxIDs.add(txID);

Review comment:
       Gotcha. thanks!




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org