You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@celeborn.apache.org by zh...@apache.org on 2023/07/05 10:16:51 UTC
[incubator-celeborn] branch main updated: [CELEBORN-768] Change default config values for batch rpcs and netty …
This is an automated email from the ASF dual-hosted git repository.
zhouky pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/incubator-celeborn.git
The following commit(s) were added to refs/heads/main by this push:
new 430083536 [CELEBORN-768] Change default config values for batch rpcs and netty …
430083536 is described below
commit 4300835363a229b887428e3f91026f389dd62291
Author: zky.zhoukeyong <zk...@alibaba-inc.com>
AuthorDate: Wed Jul 5 18:16:41 2023 +0800
[CELEBORN-768] Change default config values for batch rpcs and netty …
…memory allocator
### What changes were proposed in this pull request?
Changes the following configs' default values
| config | previous value | current value |
| ------------- | ------------- | ------------- |
| celeborn.network.memory.allocator.share | false | true |
| celeborn.client.shuffle.batchHandleChangePartition.enabled | false | true |
| celeborn.client.shuffle.batchHandleCommitPartition.enabled | false | true |
### Why are the changes needed?
In my test, when graceful shutdown is enabled but ```celeborn.client.shuffle.batchHandleChangePartition.enabled``` and ```celeborn.client.shuffle.batchHandleCommitPartition.enabled``` disabled, the worker takes much longer to stop than the two configs enabled.
In another test where worker size is quite small(2 cores 4 G) and replication is on, if shared allocator is disabled, the netty's onTrim fails to release memory, and further causes push data timeout.
### Does this PR introduce _any_ user-facing change?
No, these conifgs are introduces from 0.3.0.
### How was this patch tested?
Passes GA.
Closes #1682 from waitinfuture/768.
Authored-by: zky.zhoukeyong <zk...@alibaba-inc.com>
Signed-off-by: zky.zhoukeyong <zk...@alibaba-inc.com>
---
.../main/scala/org/apache/celeborn/common/CelebornConf.scala | 10 +++++++---
docs/configuration/client.md | 3 ---
docs/configuration/network.md | 1 -
3 files changed, 7 insertions(+), 7 deletions(-)
diff --git a/common/src/main/scala/org/apache/celeborn/common/CelebornConf.scala b/common/src/main/scala/org/apache/celeborn/common/CelebornConf.scala
index 210c45682..d3ecc5097 100644
--- a/common/src/main/scala/org/apache/celeborn/common/CelebornConf.scala
+++ b/common/src/main/scala/org/apache/celeborn/common/CelebornConf.scala
@@ -1216,10 +1216,11 @@ object CelebornConf extends Logging {
val NETWORK_MEMORY_ALLOCATOR_SHARE: ConfigEntry[Boolean] =
buildConf("celeborn.network.memory.allocator.share")
.categories("network")
+ .internal
.version("0.3.0")
.doc("Whether to share memory allocator.")
.booleanConf
- .createWithDefault(false)
+ .createWithDefault(true)
val NETWORK_MEMORY_ALLOCATOR_ARENAS: OptionalConfigEntry[Int] =
buildConf("celeborn.network.memory.allocator.numArenas")
@@ -2995,11 +2996,12 @@ object CelebornConf extends Logging {
buildConf("celeborn.client.shuffle.batchHandleChangePartition.enabled")
.withAlternative("celeborn.shuffle.batchHandleChangePartition.enabled")
.categories("client")
+ .internal
.doc("When true, LifecycleManager will handle change partition request in batch. " +
"Otherwise, LifecycleManager will process the requests one by one")
.version("0.3.0")
.booleanConf
- .createWithDefault(false)
+ .createWithDefault(true)
val CLIENT_BATCH_HANDLE_CHANGE_PARTITION_THREADS: ConfigEntry[Int] =
buildConf("celeborn.client.shuffle.batchHandleChangePartition.threads")
@@ -3023,11 +3025,12 @@ object CelebornConf extends Logging {
buildConf("celeborn.client.shuffle.batchHandleCommitPartition.enabled")
.withAlternative("celeborn.shuffle.batchHandleCommitPartition.enabled")
.categories("client")
+ .internal
.doc("When true, LifecycleManager will handle commit partition request in batch. " +
"Otherwise, LifecycleManager won't commit partition before stage end")
.version("0.3.0")
.booleanConf
- .createWithDefault(false)
+ .createWithDefault(true)
val CLIENT_BATCH_HANDLE_COMMIT_PARTITION_THREADS: ConfigEntry[Int] =
buildConf("celeborn.client.shuffle.batchHandleCommitPartition.threads")
@@ -3050,6 +3053,7 @@ object CelebornConf extends Logging {
val CLIENT_BATCH_HANDLE_RELEASE_PARTITION_ENABLED: ConfigEntry[Boolean] =
buildConf("celeborn.client.shuffle.batchHandleReleasePartition.enabled")
.categories("client")
+ .internal
.doc("When true, LifecycleManager will handle release partition request in batch. " +
"Otherwise, LifecycleManager will process release partition request immediately")
.version("0.3.0")
diff --git a/docs/configuration/client.md b/docs/configuration/client.md
index 2fc8fb1d2..381a46c3d 100644
--- a/docs/configuration/client.md
+++ b/docs/configuration/client.md
@@ -72,13 +72,10 @@ license: |
| celeborn.client.rpc.registerShuffle.askTimeout | <value of celeborn.<module>.io.connectionTimeout> | Timeout for ask operations during register shuffle. During this process, there are two times for retry opportunities for requesting slots, one request for establishing a connection with Worker and `celeborn.client.reserveSlots.maxRetries` times for retry opportunities for reserving slots. User can customize this value according to your setting. By default, the value is the m [...]
| celeborn.client.rpc.requestPartition.askTimeout | <value of celeborn.<module>.io.connectionTimeout> | Timeout for ask operations during requesting change partition location, such as reviving or spliting partition. During this process, there are `celeborn.client.reserveSlots.maxRetries` times for retry opportunities for reserving slots. User can customize this value according to your setting. By default, the value is the max timeout value `celeborn.<module>.io.connectionTime [...]
| celeborn.client.rpc.reserveSlots.askTimeout | <value of celeborn.rpc.askTimeout> | Timeout for LifecycleManager request reserve slots. | 0.3.0 |
-| celeborn.client.shuffle.batchHandleChangePartition.enabled | false | When true, LifecycleManager will handle change partition request in batch. Otherwise, LifecycleManager will process the requests one by one | 0.3.0 |
| celeborn.client.shuffle.batchHandleChangePartition.interval | 100ms | Interval for LifecycleManager to schedule handling change partition requests in batch. | 0.3.0 |
| celeborn.client.shuffle.batchHandleChangePartition.threads | 8 | Threads number for LifecycleManager to handle change partition request in batch. | 0.3.0 |
-| celeborn.client.shuffle.batchHandleCommitPartition.enabled | false | When true, LifecycleManager will handle commit partition request in batch. Otherwise, LifecycleManager won't commit partition before stage end | 0.3.0 |
| celeborn.client.shuffle.batchHandleCommitPartition.interval | 5s | Interval for LifecycleManager to schedule handling commit partition requests in batch. | 0.3.0 |
| celeborn.client.shuffle.batchHandleCommitPartition.threads | 8 | Threads number for LifecycleManager to handle commit partition request in batch. | 0.3.0 |
-| celeborn.client.shuffle.batchHandleReleasePartition.enabled | true | When true, LifecycleManager will handle release partition request in batch. Otherwise, LifecycleManager will process release partition request immediately | 0.3.0 |
| celeborn.client.shuffle.batchHandleReleasePartition.interval | 5s | Interval for LifecycleManager to schedule handling release partition requests in batch. | 0.3.0 |
| celeborn.client.shuffle.batchHandleReleasePartition.threads | 8 | Threads number for LifecycleManager to handle release partition request in batch. | 0.3.0 |
| celeborn.client.shuffle.compression.codec | LZ4 | The codec used to compress shuffle data. By default, Celeborn provides three codecs: `lz4`, `zstd`, `none`. | 0.3.0 |
diff --git a/docs/configuration/network.md b/docs/configuration/network.md
index 7cf89aebb..690ad518a 100644
--- a/docs/configuration/network.md
+++ b/docs/configuration/network.md
@@ -41,7 +41,6 @@ license: |
| celeborn.network.bind.preferIpAddress | true | When `ture`, prefer to use IP address, otherwise FQDN. This configuration only takes effects when the bind hostname is not set explicitly, in such case, Celeborn will find the first non-loopback address to bind. | 0.3.0 |
| celeborn.network.connect.timeout | 10s | Default socket connect timeout. | 0.2.0 |
| celeborn.network.memory.allocator.numArenas | <undefined> | Number of arenas for pooled memory allocator. Default value is Runtime.getRuntime.availableProcessors, min value is 2. | 0.3.0 |
-| celeborn.network.memory.allocator.share | false | Whether to share memory allocator. | 0.3.0 |
| celeborn.network.memory.allocator.verbose.metric | false | Weather to enable verbose metric for pooled allocator. | 0.3.0 |
| celeborn.network.timeout | 240s | Default timeout for network operations. | 0.2.0 |
| celeborn.port.maxRetries | 1 | When port is occupied, we will retry for max retry times. | 0.2.0 |