You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by kx...@apache.org on 2023/07/24 14:01:23 UTC
[doris] 21/21: [enhancement](config) Enlarge broker scanner bytes conf to 500G, 5G is still not enough (#22126)
This is an automated email from the ASF dual-hosted git repository.
kxiao pushed a commit to branch branch-2.0
in repository https://gitbox.apache.org/repos/asf/doris.git
commit 11b6dbfab9b370e242e74dd229439f3866e5018e
Author: Siyang Tang <82...@users.noreply.github.com>
AuthorDate: Mon Jul 24 19:49:39 2023 +0800
[enhancement](config) Enlarge broker scanner bytes conf to 500G, 5G is still not enough (#22126)
---
docs/en/docs/admin-manual/config/fe-config.md | 2 +-
docs/en/docs/data-operate/import/import-way/broker-load-manual.md | 2 +-
docs/zh-CN/docs/admin-manual/config/fe-config.md | 2 +-
docs/zh-CN/docs/data-operate/import/import-way/broker-load-manual.md | 2 +-
fe/fe-common/src/main/java/org/apache/doris/common/Config.java | 2 +-
5 files changed, 5 insertions(+), 5 deletions(-)
diff --git a/docs/en/docs/admin-manual/config/fe-config.md b/docs/en/docs/admin-manual/config/fe-config.md
index e911001143..3ea9052c84 100644
--- a/docs/en/docs/admin-manual/config/fe-config.md
+++ b/docs/en/docs/admin-manual/config/fe-config.md
@@ -1141,7 +1141,7 @@ fetch stream load record interval.
#### `max_bytes_per_broker_scanner`
-Default：`5 * 1024 * 1024 * 1024L` （5G）
+Default：`500 * 1024 * 1024 * 1024L` （500G）
IsMutable：true
diff --git a/docs/en/docs/data-operate/import/import-way/broker-load-manual.md b/docs/en/docs/data-operate/import/import-way/broker-load-manual.md
index 49de96087e..1c040d25a0 100644
--- a/docs/en/docs/data-operate/import/import-way/broker-load-manual.md
+++ b/docs/en/docs/data-operate/import/import-way/broker-load-manual.md
@@ -320,7 +320,7 @@ The following configurations belong to the system-level configuration of Broker
````text
Parameter name: min_bytes_per_broker_scanner, the default is 64MB, the unit is bytes.
Parameter name: max_broker_concurrency, default 10.
- Parameter name: max_bytes_per_broker_scanner, the default is 5G, the unit is bytes.
+ Parameter name: max_bytes_per_broker_scanner, the default is 500G, the unit is bytes.
````
## Best Practices
diff --git a/docs/zh-CN/docs/admin-manual/config/fe-config.md b/docs/zh-CN/docs/admin-manual/config/fe-config.md
index 1ef82c0570..b3b933dbbe 100644
--- a/docs/zh-CN/docs/admin-manual/config/fe-config.md
+++ b/docs/zh-CN/docs/admin-manual/config/fe-config.md
@@ -1141,7 +1141,7 @@ current running txns on db xxx is xx, larger than limit xx
#### `max_bytes_per_broker_scanner`
-默认值：5 * 1024 * 1024 * 1024L （5G）
+默认值：500 * 1024 * 1024 * 1024L （500G）
是否可以动态配置：true
diff --git a/docs/zh-CN/docs/data-operate/import/import-way/broker-load-manual.md b/docs/zh-CN/docs/data-operate/import/import-way/broker-load-manual.md
index 56e2e0d852..2dca54baa1 100644
--- a/docs/zh-CN/docs/data-operate/import/import-way/broker-load-manual.md
+++ b/docs/zh-CN/docs/data-operate/import/import-way/broker-load-manual.md
@@ -320,7 +320,7 @@ Broker Load 需要借助 Broker 进程访问远端存储，不同的 Broker 需
```text
参数名：min_bytes_per_broker_scanner， 默认 64MB，单位bytes。
参数名：max_broker_concurrency， 默认 10。
- 参数名：max_bytes_per_broker_scanner，默认 5G，单位bytes。
+ 参数名：max_bytes_per_broker_scanner，默认 500G，单位bytes。
```
## 最佳实践
diff --git a/fe/fe-common/src/main/java/org/apache/doris/common/Config.java b/fe/fe-common/src/main/java/org/apache/doris/common/Config.java
index 19ee0321eb..aa44a9aff3 100644
--- a/fe/fe-common/src/main/java/org/apache/doris/common/Config.java
+++ b/fe/fe-common/src/main/java/org/apache/doris/common/Config.java
@@ -803,7 +803,7 @@ public class Config extends ConfigBase {
* Commonly, each Backends has one broker scanner.
*/
@ConfField(mutable = true, masterOnly = true)
- public static long max_bytes_per_broker_scanner = 5 * 1024 * 1024 * 1024L; // 5G
+ public static long max_bytes_per_broker_scanner = 500 * 1024 * 1024 * 1024L; // 500G
/**
* Max number of load jobs, include PENDING、ETL、LOADING、QUORUM_FINISHED.
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org