You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by yi...@apache.org on 2022/04/13 01:50:51 UTC
[incubator-doris] branch master updated: Modify the maximum and minimum number of threads in jetty (#8960)
This is an automated email from the ASF dual-hosted git repository.
yiguolei pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-doris.git
The following commit(s) were added to refs/heads/master by this push:
new 7e08d3e320 Modify the maximum and minimum number of threads in jetty (#8960)
7e08d3e320 is described below
commit 7e08d3e320c1e863a60324642ea0a765376f4998
Author: smallhibiscus <84...@qq.com>
AuthorDate: Wed Apr 13 09:50:46 2022 +0800
Modify the maximum and minimum number of threads in jetty (#8960)
Co-authored-by: smallhibiscus <844981280>
---
docs/en/administrator-guide/config/fe_config.md | 9 ++++++++-
docs/zh-CN/administrator-guide/config/fe_config.md | 10 +++++++++-
fe/fe-core/src/main/java/org/apache/doris/PaloFe.java | 2 ++
fe/fe-core/src/main/java/org/apache/doris/common/Config.java | 10 ++++++++++
.../src/main/java/org/apache/doris/httpv2/HttpServer.java | 11 +++++++++++
5 files changed, 40 insertions(+), 2 deletions(-)
diff --git a/docs/en/administrator-guide/config/fe_config.md b/docs/en/administrator-guide/config/fe_config.md
index 281d841ff5..0d6382a6c2 100644
--- a/docs/en/administrator-guide/config/fe_config.md
+++ b/docs/en/administrator-guide/config/fe_config.md
@@ -198,10 +198,17 @@ Default：0
With the above three parameters, Jetty's thread architecture model is very simple, divided into acceptors, selectors and workers three thread pools. Acceptors are responsible for accepting new connections, and then hand them over to selectors to process the unpacking of the HTTP message protocol, and finally workers process the request. The first two thread pools adopt a non-blocking model, and one thread can handle the read and write of many sockets, so the number of thread pools is small.
-For most projects, only 1-2 acceptors threads are required, and 2 to 4 selectors threads are sufficient. Workers are obstructive business logic, often have more database operations, and require a large number of threads. The specific number depends on the proportion of QPS and IO events of the application. The higher the QPS, the more threads are required, the higher the proportion of IO, the more threads waiting, and the more total threads required.
+For most projects, only 4 acceptors threads are required, and 2 to 4 selectors threads are sufficient. Workers are obstructive business logic, often have more database operations, and require a large number of threads. The specific number depends on the proportion of QPS and IO events of the application. The higher the QPS, the more threads are required, the higher the proportion of IO, the more threads waiting, and the more total threads required.
Worker thread pool is not set by default, set according to your needs
+### jetty_threadPool_minThreads
+
+The minimum number of threads in the Jetty thread pool, the default is 20
+
+### jetty_threadPool_maxThreads
+
+The maximum number of threads in the Jetty thread pool, the default is 400
### jetty_server_max_http_post_size
diff --git a/docs/zh-CN/administrator-guide/config/fe_config.md b/docs/zh-CN/administrator-guide/config/fe_config.md
index e16e58c365..8b14b7aa18 100644
--- a/docs/zh-CN/administrator-guide/config/fe_config.md
+++ b/docs/zh-CN/administrator-guide/config/fe_config.md
@@ -197,10 +197,18 @@ HTTP Server V2 由 SpringBoot 实现, 并采用前后端分离的架构。只有
Jetty 的线程数量由以上三个参数控制。Jetty的线程架构模型非常简单，分为 acceptors、selectors 和 workers 三个线程池。acceptors 负责接受新连接，然后交给 selectors 处理HTTP消息协议的解包，最后由 workers 处理请求。前两个线程池采用非阻塞模型，一个线程可以处理很多 socket 的读写，所以线程池数量较小。
-大多数项目，acceptors 线程只需要1～2个，selectors 线程配置2～4个足矣。workers 是阻塞性的业务逻辑，往往有较多的数据库操作，需要的线程数量较多，具体数量随应用程序的 QPS 和 IO 事件占比而定。QPS 越高，需要的线程数量越多，IO 占比越高，等待的线程数越多，需要的总线程数也越多。
+大多数项目，acceptors 线程只需要4个，selectors 线程配置4个足矣。workers 是阻塞性的业务逻辑，往往有较多的数据库操作，需要的线程数量较多，具体数量随应用程序的 QPS 和 IO 事件占比而定。QPS 越高，需要的线程数量越多，IO 占比越高，等待的线程数越多，需要的总线程数也越多。
workers 线程池默认不做设置，根据自己需要进行设置
+### `jetty_threadPool_minThreads`
+
+Jetty线程池最小线程数，默认为20
+
+### `jetty_threadPool_maxThreads`
+
+Jetty线程池最大线程数，默认为400
+
### `jetty_server_max_http_post_size`
默认值：100 * 1024 * 1024 （100MB）
diff --git a/fe/fe-core/src/main/java/org/apache/doris/PaloFe.java b/fe/fe-core/src/main/java/org/apache/doris/PaloFe.java
index c8ee60a4e1..13e6f5b22e 100755
--- a/fe/fe-core/src/main/java/org/apache/doris/PaloFe.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/PaloFe.java
@@ -140,6 +140,8 @@ public class PaloFe {
httpServer.setAcceptors(Config.jetty_server_acceptors);
httpServer.setSelectors(Config.jetty_server_selectors);
httpServer.setWorkers(Config.jetty_server_workers);
+ httpServer.setMaxThreads(Config.jetty_threadPool_maxThreads);
+ httpServer.setMaxThreads(Config.jetty_threadPool_minThreads);
httpServer.start(dorisHomeDir);
qeService.start();
diff --git a/fe/fe-core/src/main/java/org/apache/doris/common/Config.java b/fe/fe-core/src/main/java/org/apache/doris/common/Config.java
index 34945f9f99..e225e86122 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/common/Config.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/common/Config.java
@@ -343,6 +343,16 @@ public class Config extends ConfigBase {
@ConfField public static int jetty_server_acceptors = 2;
@ConfField public static int jetty_server_selectors = 4;
@ConfField public static int jetty_server_workers = 0;
+
+ /**
+ * Configure the default minimum and maximum number of threads for jetty.
+ * The default minimum and maximum number of threads for jetty is 10 and the maximum is 200.
+ * If this is relatively small in a high-concurrency import scenario,
+ * users can adjust it according to their own conditions.
+ */
+ @ConfField public static int jetty_threadPool_minThreads = 20;
+ @ConfField public static int jetty_threadPool_maxThreads = 400;
+
/**
* jetty Maximum number of bytes in put or post method,default:100MB
*/
diff --git a/fe/fe-core/src/main/java/org/apache/doris/httpv2/HttpServer.java b/fe/fe-core/src/main/java/org/apache/doris/httpv2/HttpServer.java
index 0904463824..97cd932049 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/httpv2/HttpServer.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/httpv2/HttpServer.java
@@ -41,6 +41,17 @@ public class HttpServer extends SpringBootServletInitializer {
private int maxHttpPostSize ;
private int workers;
+ private int minThreads;
+ private int maxThreads;
+
+ public int getMinThreads() { return minThreads; }
+
+ public void setMinThreads(int minThreads) { this.minThreads = minThreads; }
+
+ public int getMaxThreads() { return maxThreads; }
+
+ public void setMaxThreads(int maxThreads) { this.maxThreads = maxThreads; }
+
public void setWorkers(int workers) {
this.workers = workers;
}
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org