(incubator-celeborn) branch main updated: [CELEBORN-1036][FOLLOWUP] When inflightBatchesPerAddress clear, totalInflightReqs should reset

[zh...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

zhouky pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/incubator-celeborn.git


The following commit(s) were added to refs/heads/main by this push:
     new f8eb1605a [CELEBORN-1036][FOLLOWUP] When inflightBatchesPerAddress clear, totalInflightReqs should reset
f8eb1605a is described below

commit f8eb1605a11a5dd7d1c491dce30c873bbbd9f5e7
Author: liangyongyuan <li...@xiaomi.com>
AuthorDate: Wed Dec 27 16:10:12 2023 +0800

    [CELEBORN-1036][FOLLOWUP] When inflightBatchesPerAddress clear, totalInflightReqs should reset
    
    ### What changes were proposed in this pull request?
    When   `inflightBatchesPerAddress`  clear in  `InFlightRequestTracker.cleanup `, `totalInflightReqs` should also reset to avoid getting stuck when exiting.
    
    ### Why are the changes needed?
    `inflightBatchesPerAddress` has cleared and be empty,but totalInflightReqs is always bigger than 0.
    ![image](https://github.com/apache/incubator-celeborn/assets/46274164/28223f1e-ac9b-4e0b-a26d-9b529af6bca1)
    
    This occurred during the first attempt of the task, where the request for map end failed, but the driver marked that the map has already ended.
    ![image](https://github.com/apache/incubator-celeborn/assets/46274164/7f43d808-2f9b-4775-b04f-30afe4d31e5a)
    
    ### Does this PR introduce _any_ user-facing change?
    No
    
    ### How was this patch tested?
    through exists uts
    
    Closes #2191 from lyy-pineapple/celebron-1036.
    
    Authored-by: liangyongyuan <li...@xiaomi.com>
    Signed-off-by: zky.zhoukeyong <zk...@alibaba-inc.com>
---
 .../java/org/apache/celeborn/common/write/InFlightRequestTracker.java    | 1 +
 1 file changed, 1 insertion(+)

diff --git a/common/src/main/java/org/apache/celeborn/common/write/InFlightRequestTracker.java b/common/src/main/java/org/apache/celeborn/common/write/InFlightRequestTracker.java
index fbfe124a5..c85cfbc8d 100644
--- a/common/src/main/java/org/apache/celeborn/common/write/InFlightRequestTracker.java
+++ b/common/src/main/java/org/apache/celeborn/common/write/InFlightRequestTracker.java
@@ -188,6 +188,7 @@ public class InFlightRequestTracker {
     if (!inflightBatchesPerAddress.isEmpty()) {
       logger.warn("Clear {}", this.getClass().getSimpleName());
       inflightBatchesPerAddress.clear();
+      totalInflightReqs.reset();
     }
     pushStrategy.clear();
   }