[GitHub] [ozone] bshashikant commented on a change in pull request #2704: HDDS-3227. Ensure eviction of stateMachineData from cache only when both followers catch up

[GitBox <gi...@apache.org>]

bshashikant commented on a change in pull request #2704:
URL: https://github.com/apache/ozone/pull/2704#discussion_r747329311



##########
File path: hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/transport/server/ratis/ContainerStateMachine.java
##########
@@ -735,17 +737,20 @@ public void notifyTermIndexUpdated(long term, long index) {
   @Override
   public CompletableFuture<Message> applyTransaction(TransactionContext trx) {
     long index = trx.getLogEntry().getIndex();
-    // Since leader and one of the followers has written the data, it can
-    // be removed from the stateMachineDataMap.
-    stateMachineDataCache.remove(index);
-
-    DispatcherContext.Builder builder =
-        new DispatcherContext.Builder()
-            .setTerm(trx.getLogEntry().getTerm())
-            .setLogIndex(index);
-
-    long applyTxnStartTime = Time.monotonicNowNanos();
+   // once both the followers catch up, remove the entry from the cache.
+    // if the Resource limit cache is full, leader will push back new requests
+    // and waits for a slow follower to catch up.
     try {
+      if (ratisServer.getServer().getDivision(gid).getInfo().isLeader()
+          && Arrays.stream(ratisServer.getServer().getDivision(gid).getInfo()
+          .getFollowerNextIndices()).allMatch(i -> i >= index)) {
+        stateMachineDataCache.remove(index);

Review comment:
       Adding the same call in notifyIndexUpdate() as well. This will be removed as soon as any index update on leader. 




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org