You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@celeborn.apache.org by zh...@apache.org on 2023/12/27 08:10:32 UTC
(incubator-celeborn) branch branch-0.4 updated: [CELEBORN-1036][FOLLOWUP] When inflightBatchesPerAddress clear, totalInflightReqs should reset
This is an automated email from the ASF dual-hosted git repository.
zhouky pushed a commit to branch branch-0.4
in repository https://gitbox.apache.org/repos/asf/incubator-celeborn.git
The following commit(s) were added to refs/heads/branch-0.4 by this push:
new 51d2b60fe [CELEBORN-1036][FOLLOWUP] When inflightBatchesPerAddress clear, totalInflightReqs should reset
51d2b60fe is described below
commit 51d2b60fef79f87138318eac9148993e9e431c1e
Author: liangyongyuan <li...@xiaomi.com>
AuthorDate: Wed Dec 27 16:10:12 2023 +0800
[CELEBORN-1036][FOLLOWUP] When inflightBatchesPerAddress clear, totalInflightReqs should reset
### What changes were proposed in this pull request?
When `inflightBatchesPerAddress` clear in `InFlightRequestTracker.cleanup `, `totalInflightReqs` should also reset to avoid getting stuck when exiting.
### Why are the changes needed?
`inflightBatchesPerAddress` has cleared and be empty,but totalInflightReqs is always bigger than 0.

This occurred during the first attempt of the task, where the request for map end failed, but the driver marked that the map has already ended.

### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
through exists uts
Closes #2191 from lyy-pineapple/celebron-1036.
Authored-by: liangyongyuan <li...@xiaomi.com>
Signed-off-by: zky.zhoukeyong <zk...@alibaba-inc.com>
(cherry picked from commit f8eb1605a11a5dd7d1c491dce30c873bbbd9f5e7)
Signed-off-by: zky.zhoukeyong <zk...@alibaba-inc.com>
---
.../java/org/apache/celeborn/common/write/InFlightRequestTracker.java | 1 +
1 file changed, 1 insertion(+)
diff --git a/common/src/main/java/org/apache/celeborn/common/write/InFlightRequestTracker.java b/common/src/main/java/org/apache/celeborn/common/write/InFlightRequestTracker.java
index fbfe124a5..c85cfbc8d 100644
--- a/common/src/main/java/org/apache/celeborn/common/write/InFlightRequestTracker.java
+++ b/common/src/main/java/org/apache/celeborn/common/write/InFlightRequestTracker.java
@@ -188,6 +188,7 @@ public class InFlightRequestTracker {
if (!inflightBatchesPerAddress.isEmpty()) {
logger.warn("Clear {}", this.getClass().getSimpleName());
inflightBatchesPerAddress.clear();
+ totalInflightReqs.reset();
}
pushStrategy.clear();
}