[GitHub] [ozone] lokeshj1703 commented on a change in pull request #2704: HDDS-3227. Ensure eviction of stateMachineData from cache only when both followers catch up

[GitBox <gi...@apache.org>]

lokeshj1703 commented on a change in pull request #2704:
URL: https://github.com/apache/ozone/pull/2704#discussion_r723949450



##########
File path: hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/transport/server/ratis/ContainerStateMachine.java
##########
@@ -735,17 +737,20 @@ public void notifyTermIndexUpdated(long term, long index) {
   @Override
   public CompletableFuture<Message> applyTransaction(TransactionContext trx) {
     long index = trx.getLogEntry().getIndex();
-    // Since leader and one of the followers has written the data, it can
-    // be removed from the stateMachineDataMap.
-    stateMachineDataCache.remove(index);
-
-    DispatcherContext.Builder builder =
-        new DispatcherContext.Builder()
-            .setTerm(trx.getLogEntry().getTerm())
-            .setLogIndex(index);
-
-    long applyTxnStartTime = Time.monotonicNowNanos();
+   // once both the followers catch up, remove the entry from the cache.
+    // if the Resource limit cache is full, leader will push back new requests
+    // and waits for a slow follower to catch up.
     try {
+      if (ratisServer.getServer().getDivision(gid).getInfo().isLeader()
+          && Arrays.stream(ratisServer.getServer().getDivision(gid).getInfo()
+          .getFollowerNextIndices()).allMatch(i -> i >= index)) {
+        stateMachineDataCache.remove(index);

Review comment:
       We will need to handle the indices where if condition fails. We might need a mechanism to remove that data later.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org