You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@celeborn.apache.org by "waitinfuture (via GitHub)" <gi...@apache.org> on 2023/06/28 09:26:05 UTC
[GitHub] [incubator-celeborn] waitinfuture commented on a diff in pull request #1637: [CELEBORN-725][MINOR] Refine congestion code
waitinfuture commented on code in PR #1637:
URL: https://github.com/apache/incubator-celeborn/pull/1637#discussion_r1244944240
##########
common/src/main/java/org/apache/celeborn/common/write/SlowStartPushStrategy.java:
##########
Review Comment:
> currentMaxReqsInFlight
Hi @onebox-li , thanks for your interest in congestion control!
For your second question, the reason to introduce sleep is that, we encountered situation where even though ```currentMaxReqsInFlight``` decreases to one, shuffleClient can still push data very fast, which means the ```currentMaxReqsInFlight``` can't always slow down the push speed. Basically, the goal of congestion control is to slow down the push speed, so we introduced the sleep logic. In your test case, I think you can configure ```celeborn.client.push.slowStart.initialSleepTime``` to a smaller value and test again. Also, I think the default value 4 for ```celeborn.client.push.maxReqsInFlight``` is too small, 16 is better. And the default value 500ms for ```celeborn.client.push.slowStart.initialSleepTime``` is a bit too large.
I don't quite understand your first question, ```currentMaxReqsInFlight``` is only set to 1 if it is le to 1
```
public synchronized void decreaseCurrentMaxReqs() {
if (currentMaxReqsInFlight.get() <= 1) {
currentMaxReqsInFlight.set(1);
continueCongestedNumber.incrementAndGet();
} else {
currentMaxReqsInFlight.updateAndGet(pre -> pre / 2);
}
reqsInFlightBlockThreshold = currentMaxReqsInFlight.get();
congestionAvoidanceFlag = 0;
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@celeborn.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org