You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "wangbo (via GitHub)" <gi...@apache.org> on 2024/07/09 12:01:09 UTC
[PR] [Improment]Query queued by be memory [doris]
wangbo opened a new pull request, #37559:
URL: https://github.com/apache/doris/pull/37559
## Proposed changes
Add a mechanism of back pressure based on memory usage.
when any BE's mem usage bigger than ```query_queue_by_be_used_memory```, query could queue in FE.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improment]Query queued by be memory [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #37559:
URL: https://github.com/apache/doris/pull/37559#issuecomment-2217458956
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR)
Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website).
See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improment]Query queued by be memory [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #37559:
URL: https://github.com/apache/doris/pull/37559#issuecomment-2224525268
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improment]Query queued by be memory [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #37559:
URL: https://github.com/apache/doris/pull/37559#issuecomment-2232665185
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improment]Query queued by be memory [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #37559:
URL: https://github.com/apache/doris/pull/37559#issuecomment-2239737293
<details>
<summary>ClickBench: <b>Total hot run time: 30.41 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 3fab13d7d32d0e91168ab0055bdc2109f00c00be, data reload: false
query1 0.05 0.03 0.03
query2 0.08 0.04 0.04
query3 0.22 0.04 0.05
query4 1.68 0.07 0.07
query5 0.48 0.48 0.48
query6 1.12 0.74 0.73
query7 0.02 0.02 0.02
query8 0.06 0.05 0.05
query9 0.55 0.50 0.48
query10 0.54 0.55 0.55
query11 0.15 0.12 0.12
query12 0.16 0.12 0.12
query13 0.60 0.60 0.60
query14 0.78 0.78 0.79
query15 0.85 0.83 0.81
query16 0.36 0.37 0.36
query17 1.04 1.03 1.04
query18 0.22 0.22 0.23
query19 1.87 1.70 1.68
query20 0.02 0.01 0.01
query21 15.39 0.77 0.67
query22 4.91 6.87 1.58
query23 18.23 1.40 1.26
query24 2.14 0.23 0.21
query25 0.15 0.08 0.08
query26 0.30 0.21 0.21
query27 0.45 0.23 0.23
query28 13.29 1.03 1.01
query29 12.64 3.31 3.25
query30 0.25 0.06 0.06
query31 2.87 0.40 0.39
query32 3.26 0.47 0.48
query33 2.88 2.99 2.96
query34 17.03 4.34 4.32
query35 4.44 4.42 4.43
query36 0.64 0.49 0.49
query37 0.19 0.16 0.16
query38 0.15 0.14 0.16
query39 0.04 0.03 0.04
query40 0.15 0.12 0.12
query41 0.10 0.05 0.05
query42 0.06 0.04 0.05
query43 0.04 0.04 0.04
Total cold run time: 110.45 s
Total hot run time: 30.41 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improment]Query queued by be memory [doris]
Posted by "yiguolei (via GitHub)" <gi...@apache.org>.
yiguolei commented on code in PR #37559:
URL: https://github.com/apache/doris/pull/37559#discussion_r1677304027
##########
fe/fe-core/src/main/java/org/apache/doris/resource/AdmissionControl.java:
##########
@@ -0,0 +1,156 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.resource;
+
+import org.apache.doris.common.ClientPool;
+import org.apache.doris.common.Config;
+import org.apache.doris.common.util.MasterDaemon;
+import org.apache.doris.resource.workloadgroup.QueueToken;
+import org.apache.doris.system.Backend;
+import org.apache.doris.system.SystemInfoService;
+import org.apache.doris.thrift.BackendService;
+import org.apache.doris.thrift.TGetBeResourceRequest;
+import org.apache.doris.thrift.TGetBeResourceResult;
+import org.apache.doris.thrift.TGlobalResourceUsage;
+import org.apache.doris.thrift.TNetworkAddress;
+
+import org.apache.logging.log4j.LogManager;
+import org.apache.logging.log4j.Logger;
+
+import java.util.Collection;
+import java.util.Iterator;
+import java.util.concurrent.ConcurrentLinkedQueue;
+
+public class AdmissionControl extends MasterDaemon {
Review Comment:
why need master daemon? It should also run on follower or observer FE.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improment]Query queued by be memory [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #37559:
URL: https://github.com/apache/doris/pull/37559#issuecomment-2238553499
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improment]Query queued by be memory [doris]
Posted by "yiguolei (via GitHub)" <gi...@apache.org>.
yiguolei commented on code in PR #37559:
URL: https://github.com/apache/doris/pull/37559#discussion_r1675411516
##########
fe/fe-core/src/main/java/org/apache/doris/resource/AdmissionControl.java:
##########
@@ -0,0 +1,158 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.resource;
+
+import org.apache.doris.common.ClientPool;
+import org.apache.doris.common.Config;
+import org.apache.doris.common.util.MasterDaemon;
+import org.apache.doris.resource.workloadgroup.QueueToken;
+import org.apache.doris.system.Backend;
+import org.apache.doris.system.SystemInfoService;
+import org.apache.doris.thrift.BackendService;
+import org.apache.doris.thrift.TGetBeResourceRequest;
+import org.apache.doris.thrift.TGetBeResourceResult;
+import org.apache.doris.thrift.TGlobalResourceUsage;
+import org.apache.doris.thrift.TNetworkAddress;
+
+import org.apache.logging.log4j.LogManager;
+import org.apache.logging.log4j.Logger;
+
+import java.util.Collection;
+import java.util.Iterator;
+import java.util.concurrent.ConcurrentLinkedQueue;
+
+public class AdmissionControl extends MasterDaemon {
+
+ public static final Logger LOG = LogManager.getLogger(AdmissionControl.class);
+
+ private volatile boolean isAllBeMemoryEnough = true;
+
+ private SystemInfoService clusterInfoService;
+
+ public AdmissionControl(SystemInfoService clusterInfoService) {
+ super("get-be-resource-usage-thread", Config.get_be_resource_usage_interval_ms);
+ this.clusterInfoService = clusterInfoService;
+ }
+
+ private ConcurrentLinkedQueue<QueueToken> queryWaitList = new ConcurrentLinkedQueue<>();
+
+ public void addQueueToken(QueueToken queueToken) {
+ queryWaitList.offer(queueToken);
+ }
+
+ @Override
+ protected void runAfterCatalogReady() {
+ double maxBeMemoryUsagePercent = getBeMemoryUsage();
+ notifyWaitMemoryQuery(maxBeMemoryUsagePercent);
+ }
+
+ public double getBeMemoryUsage() {
+ double maxBeMemoryUsagePercent = 0;
+ if (Config.query_queue_by_be_used_memory < 0) {
+ this.isAllBeMemoryEnough = true;
+ return maxBeMemoryUsagePercent;
+ }
+ Collection<Backend> backends = clusterInfoService.getIdToBackend().values();
+ boolean tmpIsAllBeMemoryEnough = true;
+ for (Backend be : backends) {
+ if (!be.isAlive()) {
+ continue;
+ }
+ TNetworkAddress address = null;
+ BackendService.Client client = null;
+ TGetBeResourceResult result = null;
+ boolean rpcOk = true;
+ try {
+ address = new TNetworkAddress(be.getHost(), be.getBePort());
+ client = ClientPool.backendPool.borrowObject(address, 5000);
+ result = client.getBeResource(new TGetBeResourceRequest());
+ } catch (Throwable t) {
+ rpcOk = false;
+ LOG.warn("get be {} resource failed, ", be.getHost(), t);
+ } finally {
+ try {
+ if (rpcOk) {
+ ClientPool.backendPool.returnObject(address, client);
+ } else {
+ ClientPool.backendPool.invalidateObject(address, client);
+ }
+ } catch (Throwable e) {
+ LOG.warn("return rpc client failed. related backend[{}]", be.getHost(),
+ e);
+ }
+ }
+ if (result != null && result.isSetGlobalResourceUsage()) {
+ TGlobalResourceUsage globalResourceUsage = result.getGlobalResourceUsage();
+ if (globalResourceUsage != null && globalResourceUsage.isSetMemLimit()
+ && globalResourceUsage.isSetMemUsage()) {
+ long memUsageL = globalResourceUsage.getMemUsage();
+ long memLimitL = globalResourceUsage.getMemLimit();
+ double memUsage = Double.valueOf(String.valueOf(memUsageL));
+ double memLimit = Double.valueOf(String.valueOf(memLimitL));
+ double memUsagePercent = memUsage / memLimit;
+ maxBeMemoryUsagePercent = Math.max(memUsagePercent, maxBeMemoryUsagePercent);
+
+ if (memUsagePercent > Config.query_queue_by_be_used_memory) {
+ tmpIsAllBeMemoryEnough = false;
+ }
+ LOG.debug("be ip:{}, mem limit:{}, mem usage:{}, mem usage percent:{}, query queue mem:{}",
+ be.getHost(), memLimitL, memUsageL, memUsagePercent, Config.query_queue_by_be_used_memory);
+ }
+ }
+ }
+ this.isAllBeMemoryEnough = tmpIsAllBeMemoryEnough;
+ return maxBeMemoryUsagePercent;
+ }
+
+ private int getNotifyQueryCountByMemoryUsage(double maxBeMemoryUsagePercent) {
+ int count = Integer.MAX_VALUE;
+ if (maxBeMemoryUsagePercent > 0.9) {
+ count = 1;
+ } else if (maxBeMemoryUsagePercent > 0.8) {
+ count = 2;
+ } else if (maxBeMemoryUsagePercent > 0.7) {
+ count = 4;
+ } else if (maxBeMemoryUsagePercent > 0.6) {
+ count = 8;
+ }
+ return count;
+ }
+
+ public void notifyWaitMemoryQuery(double maxBeMemoryUsagePercent) {
+ if (!isAllBeMemoryEnough()) {
+ return;
+ }
Review Comment:
这里只能计算出一个count 来,然后调用queue.notify 方法就行了。 能run 多少个,后续依靠checkResourceAvailable 来判定。
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improment]Query queued by be memory [doris]
Posted by "wangbo (via GitHub)" <gi...@apache.org>.
wangbo commented on code in PR #37559:
URL: https://github.com/apache/doris/pull/37559#discussion_r1675653391
##########
fe/fe-core/src/main/java/org/apache/doris/resource/workloadgroup/QueryQueue.java:
##########
@@ -126,30 +138,47 @@ public QueueToken getToken() throws UserException {
}
}
- public void releaseAndNotify(QueueToken releaseToken) {
+ public void releaseAndNotify(QueueToken releaseToken, AdmissionControl ac) {
Review Comment:
queryQueue每次更新属性得拷贝一次,要作为成员变量的话拷贝那块也得动,改那个地方比较麻烦。所以就先每次都get吧,
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improment]Query queued by be memory [doris]
Posted by "yiguolei (via GitHub)" <gi...@apache.org>.
yiguolei commented on code in PR #37559:
URL: https://github.com/apache/doris/pull/37559#discussion_r1671808009
##########
fe/fe-core/src/main/java/org/apache/doris/tablefunction/MetadataGenerator.java:
##########
@@ -598,6 +598,8 @@ private static TFetchSchemaTableDataResult queriesMetadataResult(TSchemaTableReq
trow.addToColumnValue(new TCell());
} else if (tokenState == TokenState.READY_TO_RUN) {
trow.addToColumnValue(new TCell().setStringVal("RUNNING"));
+ } else if (tokenState == TokenState.WAIT_FOR_MEMORY) {
Review Comment:
为啥要改这里?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improment]Query queued by be memory [doris]
Posted by "yiguolei (via GitHub)" <gi...@apache.org>.
yiguolei commented on code in PR #37559:
URL: https://github.com/apache/doris/pull/37559#discussion_r1675405258
##########
fe/fe-core/src/main/java/org/apache/doris/resource/workloadgroup/QueryQueue.java:
##########
@@ -126,30 +138,47 @@ public QueueToken getToken() throws UserException {
}
}
- public void releaseAndNotify(QueueToken releaseToken) {
+ public void releaseAndNotify(QueueToken releaseToken, AdmissionControl ac) {
queueLock.lock();
try {
- // NOTE:token's tokenState need to be locked by queueLock
- if (releaseToken.isReadyToRun()) {
- currentRunningQueryNum--;
- } else {
- priorityTokenQueue.remove(releaseToken);
- }
- Preconditions.checkArgument(currentRunningQueryNum >= 0);
- while (currentRunningQueryNum < maxConcurrency) {
- QueueToken queueToken = this.priorityTokenQueue.poll();
+ runningQueryQueue.remove(releaseToken);
+ waitingQueryQueue.remove(releaseToken);
+ releaseToken.release();
+ while (runningQueryQueue.size() < maxConcurrency && ac.isAllBeMemoryEnough()) {
Review Comment:
这里继续调用 ac 的checkResourceAvailable 方法
##########
fe/fe-core/src/main/java/org/apache/doris/resource/workloadgroup/QueryQueue.java:
##########
@@ -105,18 +104,31 @@ public QueueToken getToken() throws UserException {
if (LOG.isDebugEnabled()) {
LOG.info(this.debugString());
}
- if (currentRunningQueryNum < maxConcurrency) {
+ AdmissionControl admissionControl = Env.getCurrentEnv().getAdmissionControl();
Review Comment:
把这堆根据resource 判断是否可以运行的逻辑,移动到adminsion control中。比如增加一个方法,boolean checkResourceAvailable(QueryToken) , 如果不能运行,那么admission controll 直接设置token的状态,同时设置message. 否则,后期我们逻辑越来越复杂,这个代码会很大。
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improment]Query queued by be memory [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #37559:
URL: https://github.com/apache/doris/pull/37559#issuecomment-2225560749
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improment]Query queued by be memory [doris]
Posted by "wangbo (via GitHub)" <gi...@apache.org>.
wangbo commented on code in PR #37559:
URL: https://github.com/apache/doris/pull/37559#discussion_r1683757683
##########
fe/fe-common/src/main/java/org/apache/doris/common/Config.java:
##########
@@ -1759,6 +1759,15 @@ public class Config extends ConfigBase {
@ConfField(mutable = true, varType = VariableAnnotation.EXPERIMENTAL)
public static boolean enable_cpu_hard_limit = false;
+ @ConfField(mutable = true, description = {
+ "当BE内存用量大于该值时,查询会进入排队逻辑,默认值为-1,代表该值不生效。取值范围0~1的小数",
+ "When be memory usage bigger than this value, query could queue, "
+ + "default value is -1, means this value not work. Decimal value range from 0 to 1"})
+ public static double query_queue_by_be_used_memory = -1;
Review Comment:
this is a 0~1 float value, so it's not a mb or bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improment]Query queued by be memory [doris]
Posted by "wangbo (via GitHub)" <gi...@apache.org>.
wangbo commented on code in PR #37559:
URL: https://github.com/apache/doris/pull/37559#discussion_r1677351108
##########
fe/fe-core/src/main/java/org/apache/doris/resource/AdmissionControl.java:
##########
@@ -0,0 +1,156 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.resource;
+
+import org.apache.doris.common.ClientPool;
+import org.apache.doris.common.Config;
+import org.apache.doris.common.util.MasterDaemon;
+import org.apache.doris.resource.workloadgroup.QueueToken;
+import org.apache.doris.system.Backend;
+import org.apache.doris.system.SystemInfoService;
+import org.apache.doris.thrift.BackendService;
+import org.apache.doris.thrift.TGetBeResourceRequest;
+import org.apache.doris.thrift.TGetBeResourceResult;
+import org.apache.doris.thrift.TGlobalResourceUsage;
+import org.apache.doris.thrift.TNetworkAddress;
+
+import org.apache.logging.log4j.LogManager;
+import org.apache.logging.log4j.Logger;
+
+import java.util.Collection;
+import java.util.Iterator;
+import java.util.concurrent.ConcurrentLinkedQueue;
+
+public class AdmissionControl extends MasterDaemon {
Review Comment:
```
* MasterDaemon is a kind of thread that will wait Catalog to be ready before running.
* And it can run in all types of FE.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improment]Query queued by be memory [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #37559:
URL: https://github.com/apache/doris/pull/37559#issuecomment-2232554119
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improment]Query queued by be memory [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #37559:
URL: https://github.com/apache/doris/pull/37559#issuecomment-2239715126
<details>
<summary>TPC-H: <b>Total hot run time: 40399 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 3fab13d7d32d0e91168ab0055bdc2109f00c00be, data reload: false
------ Round 1 ----------------------------------
q1 20035 4666 4475 4475
q2 2892 198 208 198
q3 10866 1187 1137 1137
q4 10564 880 668 668
q5 7805 2758 2736 2736
q6 225 145 144 144
q7 973 598 611 598
q8 9214 2105 2139 2105
q9 8942 6618 6629 6618
q10 8809 3795 3755 3755
q11 450 245 251 245
q12 410 233 230 230
q13 18940 2994 3022 2994
q14 286 231 243 231
q15 537 491 513 491
q16 496 376 384 376
q17 1003 763 681 681
q18 8209 7505 7391 7391
q19 5506 1519 1473 1473
q20 648 313 333 313
q21 4986 3258 3318 3258
q22 349 287 282 282
Total cold run time: 122145 ms
Total hot run time: 40399 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4364 4295 4235 4235
q2 381 264 268 264
q3 3013 2787 2810 2787
q4 1887 1674 1605 1605
q5 5325 5366 5333 5333
q6 222 138 136 136
q7 2153 1776 1762 1762
q8 3224 3362 3322 3322
q9 8496 8433 8463 8433
q10 3933 3759 3680 3680
q11 570 476 501 476
q12 772 614 609 609
q13 16385 3033 2983 2983
q14 309 281 265 265
q15 517 480 484 480
q16 468 416 413 413
q17 1776 1501 1479 1479
q18 7609 7669 7410 7410
q19 1650 1448 1575 1448
q20 2004 1768 1788 1768
q21 4988 4682 4754 4682
q22 552 488 488 488
Total cold run time: 70598 ms
Total hot run time: 54058 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improment]Query queued by be memory [doris]
Posted by "wangbo (via GitHub)" <gi...@apache.org>.
wangbo merged PR #37559:
URL: https://github.com/apache/doris/pull/37559
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improment]Query queued by be memory [doris]
Posted by "morningman (via GitHub)" <gi...@apache.org>.
morningman commented on code in PR #37559:
URL: https://github.com/apache/doris/pull/37559#discussion_r1682632878
##########
fe/fe-common/src/main/java/org/apache/doris/common/Config.java:
##########
@@ -1759,6 +1759,15 @@ public class Config extends ConfigBase {
@ConfField(mutable = true, varType = VariableAnnotation.EXPERIMENTAL)
public static boolean enable_cpu_hard_limit = false;
+ @ConfField(mutable = true, description = {
+ "当BE内存用量大于该值时,查询会进入排队逻辑,默认值为-1,代表该值不生效。取值范围0~1的小数",
+ "When be memory usage bigger than this value, query could queue, "
+ + "default value is -1, means this value not work. Decimal value range from 0 to 1"})
+ public static double query_queue_by_be_used_memory = -1;
Review Comment:
```suggestion
public static double query_queue_by_be_used_memory = -1;
```
Better add unit in config name, like `xxx_mb` or `xxx_bytes`
##########
fe/fe-common/src/main/java/org/apache/doris/common/Config.java:
##########
@@ -1759,6 +1759,15 @@ public class Config extends ConfigBase {
@ConfField(mutable = true, varType = VariableAnnotation.EXPERIMENTAL)
public static boolean enable_cpu_hard_limit = false;
+ @ConfField(mutable = true, description = {
+ "当BE内存用量大于该值时,查询会进入排队逻辑,默认值为-1,代表该值不生效。取值范围0~1的小数",
+ "When be memory usage bigger than this value, query could queue, "
+ + "default value is -1, means this value not work. Decimal value range from 0 to 1"})
+ public static double query_queue_by_be_used_memory = -1;
+
+ @ConfField(mutable = true)
Review Comment:
Missing `description` field
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improment]Query queued by be memory [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #37559:
URL: https://github.com/apache/doris/pull/37559#issuecomment-2239729767
<details>
<summary>TPC-DS: <b>Total hot run time: 173405 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 3fab13d7d32d0e91168ab0055bdc2109f00c00be, data reload: false
query1 923 392 360 360
query2 6455 1893 1815 1815
query3 6670 209 220 209
query4 21156 17811 17353 17353
query5 4206 481 479 479
query6 269 175 174 174
query7 4547 278 293 278
query8 244 193 202 193
query9 8317 2454 2440 2440
query10 440 292 283 283
query11 11738 9979 10035 9979
query12 133 87 80 80
query13 1601 380 384 380
query14 10451 7591 7831 7591
query15 220 167 168 167
query16 7837 495 471 471
query17 1593 566 549 549
query18 1962 289 284 284
query19 199 157 155 155
query20 91 85 80 80
query21 212 133 132 132
query22 4374 4091 3944 3944
query23 33914 33278 33396 33278
query24 11960 2888 2876 2876
query25 686 389 411 389
query26 1778 154 151 151
query27 2867 274 273 273
query28 7627 2008 1997 1997
query29 1151 640 634 634
query30 290 151 151 151
query31 1017 751 749 749
query32 93 55 55 55
query33 789 338 350 338
query34 950 479 494 479
query35 899 760 741 741
query36 1117 954 953 953
query37 287 80 81 80
query38 3001 2745 2758 2745
query39 853 810 800 800
query40 284 118 117 117
query41 43 44 50 44
query42 118 102 105 102
query43 538 455 465 455
query44 1222 752 727 727
query45 189 160 160 160
query46 1089 733 718 718
query47 1864 1743 1757 1743
query48 374 297 288 288
query49 1196 408 433 408
query50 800 392 388 388
query51 6955 6819 6798 6798
query52 105 95 94 94
query53 364 308 301 301
query54 854 452 451 451
query55 76 74 74 74
query56 311 268 266 266
query57 1174 1078 1017 1017
query58 260 249 270 249
query59 2867 2660 2715 2660
query60 298 305 285 285
query61 103 98 96 96
query62 831 646 637 637
query63 319 297 293 293
query64 10462 2214 1681 1681
query65 3185 3134 3118 3118
query66 1373 333 339 333
query67 15675 14876 14844 14844
query68 4750 536 544 536
query69 716 393 363 363
query70 1196 1165 1196 1165
query71 459 277 284 277
query72 9190 5928 5562 5562
query73 749 333 324 324
query74 6124 5674 5708 5674
query75 4500 2649 2715 2649
query76 3410 977 948 948
query77 704 321 326 321
query78 9842 9032 8983 8983
query79 3554 516 534 516
query80 2233 476 470 470
query81 585 219 221 219
query82 805 131 133 131
query83 314 172 175 172
query84 266 86 91 86
query85 1794 308 352 308
query86 448 326 313 313
query87 3290 3138 3073 3073
query88 4374 2369 2365 2365
query89 481 382 382 382
query90 1969 187 191 187
query91 129 99 102 99
query92 68 48 48 48
query93 3704 514 512 512
query94 1384 265 294 265
query95 401 308 317 308
query96 604 265 266 265
query97 3153 2983 3051 2983
query98 221 195 190 190
query99 1483 1279 1288 1279
Total cold run time: 288746 ms
Total hot run time: 173405 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improment]Query queued by be memory [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #37559:
URL: https://github.com/apache/doris/pull/37559#issuecomment-2228073682
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improment]Query queued by be memory [doris]
Posted by "yiguolei (via GitHub)" <gi...@apache.org>.
yiguolei commented on code in PR #37559:
URL: https://github.com/apache/doris/pull/37559#discussion_r1675407840
##########
fe/fe-core/src/main/java/org/apache/doris/resource/workloadgroup/QueryQueue.java:
##########
@@ -126,30 +138,47 @@ public QueueToken getToken() throws UserException {
}
}
- public void releaseAndNotify(QueueToken releaseToken) {
+ public void releaseAndNotify(QueueToken releaseToken, AdmissionControl ac) {
queueLock.lock();
try {
- // NOTE:token's tokenState need to be locked by queueLock
- if (releaseToken.isReadyToRun()) {
- currentRunningQueryNum--;
- } else {
- priorityTokenQueue.remove(releaseToken);
- }
- Preconditions.checkArgument(currentRunningQueryNum >= 0);
- while (currentRunningQueryNum < maxConcurrency) {
- QueueToken queueToken = this.priorityTokenQueue.poll();
+ runningQueryQueue.remove(releaseToken);
+ waitingQueryQueue.remove(releaseToken);
+ releaseToken.release();
+ while (runningQueryQueue.size() < maxConcurrency && ac.isAllBeMemoryEnough()) {
+ QueueToken queueToken = this.waitingQueryQueue.poll();
if (queueToken == null) {
break;
}
queueToken.complete();
- currentRunningQueryNum++;
+ runningQueryQueue.offer(queueToken);
+ }
+ } finally {
+ queueLock.unlock();
+ if (LOG.isDebugEnabled()) {
+ LOG.info(this.debugString());
+ }
+ }
+ }
+
+ public boolean notifyQuery(QueueToken queueToken) {
Review Comment:
这个方法还有后面的那个notifySelf 都不用。
直接用releaseAndNotify 这个,把token 传递成null,就是我们想要的逻辑。
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improment]Query queued by be memory [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #37559:
URL: https://github.com/apache/doris/pull/37559#issuecomment-2228216699
<details>
<summary>TPC-DS: <b>Total hot run time: 174090 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit a2649ae49f60a9f796520944d40b05e19031e0f9, data reload: false
query1 930 372 375 372
query2 6458 1916 1950 1916
query3 6643 204 221 204
query4 24913 17558 17356 17356
query5 3747 483 468 468
query6 269 175 171 171
query7 4589 296 293 293
query8 246 207 205 205
query9 8476 2365 2372 2365
query10 447 291 281 281
query11 11667 10215 10141 10141
query12 121 87 89 87
query13 1661 383 374 374
query14 10202 7780 7794 7780
query15 224 167 164 164
query16 7535 322 308 308
query17 1646 533 530 530
query18 1407 282 270 270
query19 195 147 146 146
query20 89 80 80 80
query21 208 130 130 130
query22 4342 4209 4074 4074
query23 34280 33673 33616 33616
query24 10752 2867 2992 2867
query25 621 404 390 390
query26 729 153 154 153
query27 2220 276 291 276
query28 6144 2024 2032 2024
query29 878 648 649 648
query30 255 163 153 153
query31 960 763 763 763
query32 104 56 61 56
query33 678 301 300 300
query34 889 501 513 501
query35 664 584 609 584
query36 1161 964 996 964
query37 148 83 85 83
query38 2978 2820 2872 2820
query39 910 859 855 855
query40 208 124 130 124
query41 46 46 47 46
query42 117 106 109 106
query43 535 477 460 460
query44 1060 771 730 730
query45 193 170 162 162
query46 1109 730 740 730
query47 1868 1785 1752 1752
query48 366 286 288 286
query49 831 412 409 409
query50 774 395 393 393
query51 6936 6845 6871 6845
query52 107 91 98 91
query53 360 298 285 285
query54 854 445 449 445
query55 81 73 77 73
query56 303 261 263 261
query57 1132 1086 1047 1047
query58 260 256 263 256
query59 2907 2696 2706 2696
query60 309 278 293 278
query61 98 95 97 95
query62 811 642 656 642
query63 320 286 292 286
query64 9192 2227 1661 1661
query65 3166 3099 3080 3080
query66 774 329 336 329
query67 15757 15095 14982 14982
query68 4673 523 527 523
query69 699 422 343 343
query70 1114 1161 1151 1151
query71 421 279 288 279
query72 7501 5639 5362 5362
query73 765 328 324 324
query74 6183 5679 5682 5679
query75 3385 2720 2692 2692
query76 2812 967 928 928
query77 628 295 289 289
query78 11642 9517 8899 8899
query79 3471 568 520 520
query80 1497 479 473 473
query81 590 216 219 216
query82 769 137 136 136
query83 338 164 175 164
query84 274 86 91 86
query85 755 334 309 309
query86 473 328 310 310
query87 3290 3166 3115 3115
query88 4220 2450 2479 2450
query89 498 394 381 381
query90 1803 193 194 193
query91 131 101 104 101
query92 61 50 53 50
query93 4490 521 513 513
query94 1139 221 216 216
query95 418 325 321 321
query96 612 275 271 271
query97 3208 3024 3051 3024
query98 283 204 196 196
query99 1678 1289 1258 1258
Total cold run time: 280681 ms
Total hot run time: 174090 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improment]Query queued by be memory [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #37559:
URL: https://github.com/apache/doris/pull/37559#issuecomment-2232554156
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improment]Query queued by be memory [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #37559:
URL: https://github.com/apache/doris/pull/37559#issuecomment-2228225986
<details>
<summary>ClickBench: <b>Total hot run time: 31.16 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit a2649ae49f60a9f796520944d40b05e19031e0f9, data reload: false
query1 0.04 0.03 0.04
query2 0.08 0.04 0.04
query3 0.22 0.05 0.04
query4 1.68 0.08 0.08
query5 0.49 0.48 0.48
query6 1.14 0.73 0.73
query7 0.02 0.01 0.02
query8 0.05 0.05 0.05
query9 0.57 0.51 0.50
query10 0.56 0.54 0.54
query11 0.15 0.12 0.12
query12 0.15 0.12 0.12
query13 0.60 0.58 0.59
query14 0.78 0.77 0.79
query15 0.85 0.82 0.82
query16 0.37 0.37 0.39
query17 1.05 0.98 0.98
query18 0.23 0.22 0.21
query19 1.92 1.77 1.76
query20 0.02 0.01 0.01
query21 15.42 0.75 0.65
query22 4.47 6.62 2.30
query23 18.28 1.42 1.28
query24 2.09 0.23 0.23
query25 0.16 0.08 0.08
query26 0.30 0.21 0.21
query27 0.45 0.24 0.23
query28 13.31 1.01 0.99
query29 12.64 3.31 3.30
query30 0.24 0.06 0.06
query31 2.88 0.38 0.38
query32 3.26 0.48 0.47
query33 2.95 2.91 2.98
query34 17.08 4.37 4.34
query35 4.47 4.42 4.45
query36 0.65 0.50 0.48
query37 0.20 0.16 0.15
query38 0.16 0.15 0.15
query39 0.04 0.04 0.04
query40 0.15 0.12 0.13
query41 0.10 0.05 0.05
query42 0.06 0.05 0.04
query43 0.04 0.04 0.04
Total cold run time: 110.37 s
Total hot run time: 31.16 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improment]Query queued by be memory [doris]
Posted by "wangbo (via GitHub)" <gi...@apache.org>.
wangbo commented on PR #37559:
URL: https://github.com/apache/doris/pull/37559#issuecomment-2232655697
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improment]Query queued by be memory [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #37559:
URL: https://github.com/apache/doris/pull/37559#issuecomment-2232799992
<details>
<summary>ClickBench: <b>Total hot run time: 29.98 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 220abd586aac1562f28e9e5ca89c5fae34b46d64, data reload: false
query1 0.04 0.04 0.03
query2 0.07 0.04 0.03
query3 0.23 0.06 0.05
query4 1.68 0.07 0.06
query5 0.50 0.50 0.49
query6 1.13 0.73 0.72
query7 0.02 0.02 0.01
query8 0.05 0.04 0.04
query9 0.55 0.50 0.49
query10 0.55 0.53 0.55
query11 0.15 0.12 0.11
query12 0.15 0.12 0.12
query13 0.59 0.59 0.58
query14 0.77 0.76 0.79
query15 0.86 0.82 0.82
query16 0.36 0.36 0.36
query17 0.99 1.00 0.99
query18 0.23 0.22 0.22
query19 1.76 1.84 1.74
query20 0.01 0.01 0.00
query21 15.41 0.75 0.67
query22 4.10 7.88 1.25
query23 18.29 1.36 1.29
query24 2.10 0.25 0.22
query25 0.15 0.08 0.08
query26 0.29 0.21 0.22
query27 0.46 0.24 0.24
query28 13.24 1.01 1.01
query29 12.64 3.30 3.30
query30 0.26 0.06 0.05
query31 2.86 0.38 0.40
query32 3.27 0.49 0.47
query33 2.88 2.94 2.93
query34 17.14 4.29 4.33
query35 4.46 4.44 4.40
query36 0.66 0.46 0.48
query37 0.19 0.15 0.16
query38 0.16 0.15 0.15
query39 0.05 0.03 0.04
query40 0.16 0.12 0.12
query41 0.09 0.04 0.05
query42 0.06 0.05 0.05
query43 0.05 0.05 0.04
Total cold run time: 109.66 s
Total hot run time: 29.98 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improment]Query queued by be memory [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #37559:
URL: https://github.com/apache/doris/pull/37559#issuecomment-2232766147
<details>
<summary>TPC-H: <b>Total hot run time: 40268 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 220abd586aac1562f28e9e5ca89c5fae34b46d64, data reload: false
------ Round 1 ----------------------------------
q1 18014 4762 4433 4433
q2 2366 192 186 186
q3 10524 1261 1151 1151
q4 10196 791 780 780
q5 7618 2715 2699 2699
q6 223 137 137 137
q7 966 604 610 604
q8 9231 2085 2084 2084
q9 8919 6594 6555 6555
q10 8774 3819 3814 3814
q11 452 241 239 239
q12 404 227 232 227
q13 17752 2978 3037 2978
q14 282 242 244 242
q15 536 489 478 478
q16 481 383 390 383
q17 984 697 678 678
q18 8186 7546 7425 7425
q19 7379 1393 1423 1393
q20 681 328 323 323
q21 4972 3172 3242 3172
q22 356 287 288 287
Total cold run time: 119296 ms
Total hot run time: 40268 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4410 4270 4264 4264
q2 368 284 270 270
q3 3038 2796 2742 2742
q4 1903 1605 1636 1605
q5 5310 5334 5347 5334
q6 216 135 131 131
q7 2076 1742 1786 1742
q8 3243 3362 3314 3314
q9 8458 8407 8404 8404
q10 3894 3771 3701 3701
q11 572 498 503 498
q12 807 610 572 572
q13 16735 2979 3033 2979
q14 313 283 276 276
q15 532 476 475 475
q16 474 430 427 427
q17 1807 1490 1502 1490
q18 7864 7560 7602 7560
q19 1692 1644 1496 1496
q20 2031 1813 1800 1800
q21 7117 4677 4645 4645
q22 587 498 552 498
Total cold run time: 73447 ms
Total hot run time: 54223 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improment]Query queued by be memory [doris]
Posted by "yiguolei (via GitHub)" <gi...@apache.org>.
yiguolei commented on PR #37559:
URL: https://github.com/apache/doris/pull/37559#issuecomment-2238247885
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improment]Query queued by be memory [doris]
Posted by "wangbo (via GitHub)" <gi...@apache.org>.
wangbo commented on PR #37559:
URL: https://github.com/apache/doris/pull/37559#issuecomment-2217459691
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improment]Query queued by be memory [doris]
Posted by "yiguolei (via GitHub)" <gi...@apache.org>.
yiguolei commented on code in PR #37559:
URL: https://github.com/apache/doris/pull/37559#discussion_r1675373545
##########
fe/fe-core/src/main/java/org/apache/doris/resource/workloadgroup/QueryQueue.java:
##########
@@ -105,18 +104,31 @@ public QueueToken getToken() throws UserException {
if (LOG.isDebugEnabled()) {
LOG.info(this.debugString());
}
- if (currentRunningQueryNum < maxConcurrency) {
+ AdmissionControl admissionControl = Env.getCurrentEnv().getAdmissionControl();
Review Comment:
把这堆根据resource 判断是否可以运行的逻辑,移动到adminsion control中。比如增加一个方法,boolean isResourceAvailable(QueryToken) , 如果不能运行,那么admission controll 直接设置token的状态,同时设置message. 否则,后期我们逻辑越来越复杂,这个代码会很大。
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improment]Query queued by be memory [doris]
Posted by "yiguolei (via GitHub)" <gi...@apache.org>.
yiguolei commented on code in PR #37559:
URL: https://github.com/apache/doris/pull/37559#discussion_r1675374613
##########
fe/fe-core/src/main/java/org/apache/doris/resource/workloadgroup/QueryQueue.java:
##########
@@ -105,18 +104,31 @@ public QueueToken getToken() throws UserException {
if (LOG.isDebugEnabled()) {
LOG.info(this.debugString());
}
- if (currentRunningQueryNum < maxConcurrency) {
+ AdmissionControl admissionControl = Env.getCurrentEnv().getAdmissionControl();
+ boolean isMemEnough = admissionControl.isAllBeMemoryEnough();
+ boolean isReachMaxCon = runningQueryQueue.size() >= maxConcurrency;
+ if (!isReachMaxCon && isMemEnough) {
QueueToken retToken = new QueueToken(TokenState.READY_TO_RUN, queueTimeout, this);
+ runningQueryQueue.offer(retToken);
retToken.complete();
- currentRunningQueryNum++;
return retToken;
}
- if (priorityTokenQueue.size() >= maxQueueSize) {
+ if (maxQueueSize > 0 && waitingQueryQueue.size() >= maxQueueSize) {
throw new UserException("query waiting queue is full, queue length=" + maxQueueSize);
}
- QueueToken newQueryToken = new QueueToken(TokenState.ENQUEUE_SUCCESS, queueTimeout,
- this);
- this.priorityTokenQueue.offer(newQueryToken);
+
+ QueueToken newQueryToken = new QueueToken(TokenState.ENQUEUE_SUCCESS, queueTimeout, this);
+ String queueMsg = "WAIT_IN_QUEUE";
+ if (!isReachMaxCon && !isMemEnough) {
+ queueMsg = "WAIT_BE_MEMORY";
Review Comment:
这个message 应该是adminssion control 在check resource的时候,同时设置的。
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improment]Query queued by be memory [doris]
Posted by "yiguolei (via GitHub)" <gi...@apache.org>.
yiguolei commented on code in PR #37559:
URL: https://github.com/apache/doris/pull/37559#discussion_r1675404264
##########
fe/fe-core/src/main/java/org/apache/doris/resource/workloadgroup/QueryQueue.java:
##########
@@ -126,30 +138,47 @@ public QueueToken getToken() throws UserException {
}
}
- public void releaseAndNotify(QueueToken releaseToken) {
+ public void releaseAndNotify(QueueToken releaseToken, AdmissionControl ac) {
Review Comment:
如果 107 行和 141 行都依赖这个adminsion control, 那么就把这个东东作为queue的一个属性吧,就不用四处传递获取了。
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improment]Query queued by be memory [doris]
Posted by "wangbo (via GitHub)" <gi...@apache.org>.
wangbo commented on PR #37559:
URL: https://github.com/apache/doris/pull/37559#issuecomment-2225549189
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improment]Query queued by be memory [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #37559:
URL: https://github.com/apache/doris/pull/37559#issuecomment-2228198306
<details>
<summary>TPC-H: <b>Total hot run time: 40375 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit a2649ae49f60a9f796520944d40b05e19031e0f9, data reload: false
------ Round 1 ----------------------------------
q1 18202 4463 4335 4335
q2 2016 193 191 191
q3 10520 1216 1171 1171
q4 10215 790 757 757
q5 7569 2720 2683 2683
q6 226 141 141 141
q7 970 607 599 599
q8 9235 2113 2090 2090
q9 8864 6587 6612 6587
q10 8829 3838 3839 3838
q11 446 236 232 232
q12 395 229 223 223
q13 17770 2989 2983 2983
q14 289 243 234 234
q15 539 485 462 462
q16 495 411 388 388
q17 974 716 720 716
q18 8220 7516 7471 7471
q19 5235 1453 1465 1453
q20 672 322 332 322
q21 5000 3274 3215 3215
q22 356 284 286 284
Total cold run time: 117037 ms
Total hot run time: 40375 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4523 4287 4257 4257
q2 372 275 279 275
q3 3052 2887 2909 2887
q4 2005 1686 1783 1686
q5 5723 5588 5528 5528
q6 237 136 134 134
q7 2201 1909 1861 1861
q8 3277 3391 3468 3391
q9 8829 8790 8919 8790
q10 4112 3890 3770 3770
q11 598 512 516 512
q12 847 646 621 621
q13 17149 3179 3198 3179
q14 319 288 296 288
q15 510 489 470 470
q16 492 447 449 447
q17 1847 1550 1522 1522
q18 8079 7954 7887 7887
q19 1710 1707 1606 1606
q20 2390 1907 1895 1895
q21 5075 4900 4874 4874
q22 579 497 499 497
Total cold run time: 73926 ms
Total hot run time: 56377 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improment]Query queued by be memory [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #37559:
URL: https://github.com/apache/doris/pull/37559#issuecomment-2232788571
<details>
<summary>TPC-DS: <b>Total hot run time: 172958 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 220abd586aac1562f28e9e5ca89c5fae34b46d64, data reload: false
query1 919 384 375 375
query2 6479 1956 1886 1886
query3 6662 211 223 211
query4 28564 17697 17209 17209
query5 4183 484 471 471
query6 264 176 171 171
query7 4612 290 291 290
query8 242 196 197 196
query9 8581 2408 2406 2406
query10 445 290 280 280
query11 12225 10038 10058 10038
query12 134 88 83 83
query13 1640 370 401 370
query14 9955 7912 7822 7822
query15 220 170 170 170
query16 7597 308 303 303
query17 1822 538 528 528
query18 1584 281 276 276
query19 205 168 147 147
query20 91 83 84 83
query21 207 127 130 127
query22 4384 4036 4097 4036
query23 33767 33146 33187 33146
query24 11841 2855 2866 2855
query25 652 369 376 369
query26 1765 149 148 148
query27 2707 271 281 271
query28 7374 2009 1989 1989
query29 1076 612 603 603
query30 280 151 148 148
query31 954 764 745 745
query32 95 52 55 52
query33 766 305 307 305
query34 923 495 505 495
query35 678 574 589 574
query36 1108 947 956 947
query37 187 80 86 80
query38 2873 2777 2743 2743
query39 857 854 823 823
query40 277 123 120 120
query41 49 46 46 46
query42 116 97 102 97
query43 491 454 466 454
query44 1212 740 736 736
query45 196 167 161 161
query46 1086 734 732 732
query47 1823 1768 1758 1758
query48 393 291 298 291
query49 1193 420 416 416
query50 788 407 392 392
query51 6950 6784 6788 6784
query52 101 92 94 92
query53 363 314 296 296
query54 1064 453 453 453
query55 75 74 74 74
query56 287 269 279 269
query57 1186 1057 1048 1048
query58 263 243 250 243
query59 2878 2633 2715 2633
query60 307 283 278 278
query61 97 94 104 94
query62 824 662 635 635
query63 335 300 301 300
query64 10599 2203 1646 1646
query65 3217 3120 3101 3101
query66 1335 342 336 336
query67 15559 14929 15160 14929
query68 4612 550 547 547
query69 572 431 355 355
query70 1168 1127 1147 1127
query71 378 289 279 279
query72 7485 5588 5225 5225
query73 757 329 329 329
query74 6114 5686 5729 5686
query75 3387 2721 2717 2717
query76 2863 903 934 903
query77 614 326 321 321
query78 9664 8996 9239 8996
query79 2218 529 541 529
query80 994 471 473 471
query81 586 222 225 222
query82 772 131 131 131
query83 266 168 171 168
query84 242 89 88 88
query85 1793 325 309 309
query86 460 314 344 314
query87 3265 3097 3155 3097
query88 3849 2390 2406 2390
query89 474 382 382 382
query90 1811 203 204 203
query91 132 103 102 102
query92 70 54 53 53
query93 2476 512 514 512
query94 1126 219 228 219
query95 469 330 332 330
query96 611 271 273 271
query97 3156 3026 3028 3026
query98 230 196 198 196
query99 1692 1275 1297 1275
Total cold run time: 286288 ms
Total hot run time: 172958 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improment]Query queued by be memory [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #37559:
URL: https://github.com/apache/doris/pull/37559#issuecomment-2238086614
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improment]Query queued by be memory [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #37559:
URL: https://github.com/apache/doris/pull/37559#issuecomment-2217469357
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improment]Query queued by be memory [doris]
Posted by "yiguolei (via GitHub)" <gi...@apache.org>.
yiguolei commented on code in PR #37559:
URL: https://github.com/apache/doris/pull/37559#discussion_r1671799610
##########
fe/fe-common/src/main/java/org/apache/doris/common/Config.java:
##########
@@ -1759,6 +1759,14 @@ public class Config extends ConfigBase {
@ConfField(mutable = true, varType = VariableAnnotation.EXPERIMENTAL)
public static boolean enable_cpu_hard_limit = false;
+ @ConfField(mutable = true, description = {
+ "当BE内存用量大于该值时,查询会进入排队逻辑,默认值为-1,代表该值不生效。取值范围0~1的小数",
+ "When be memory usage bigger than this value, query could queue, default value is -1, means this value not work. Decimal value range from 0 to 1"})
+ public static double query_queue_by_be_used_memory = -1;
Review Comment:
为啥是一个config,而不是一个workload group的properties?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improment]Query queued by be memory [doris]
Posted by "yiguolei (via GitHub)" <gi...@apache.org>.
yiguolei commented on code in PR #37559:
URL: https://github.com/apache/doris/pull/37559#discussion_r1675373545
##########
fe/fe-core/src/main/java/org/apache/doris/resource/workloadgroup/QueryQueue.java:
##########
@@ -105,18 +104,31 @@ public QueueToken getToken() throws UserException {
if (LOG.isDebugEnabled()) {
LOG.info(this.debugString());
}
- if (currentRunningQueryNum < maxConcurrency) {
+ AdmissionControl admissionControl = Env.getCurrentEnv().getAdmissionControl();
Review Comment:
把这堆根据resource 判断是否可以运行的逻辑,移动到adminsion control中。比如增加一个方法,boolean isResourceAvailable() . 否则,后期我们逻辑越来越复杂,这个代码会很大。
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improment]Query queued by be memory [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #37559:
URL: https://github.com/apache/doris/pull/37559#issuecomment-2224526633
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improment]Query queued by be memory [doris]
Posted by "wangbo (via GitHub)" <gi...@apache.org>.
wangbo commented on PR #37559:
URL: https://github.com/apache/doris/pull/37559#issuecomment-2225553523
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improment]Query queued by be memory [doris]
Posted by "wangbo (via GitHub)" <gi...@apache.org>.
wangbo commented on PR #37559:
URL: https://github.com/apache/doris/pull/37559#issuecomment-2228062650
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improment]Query queued by be memory [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #37559:
URL: https://github.com/apache/doris/pull/37559#issuecomment-2232661733
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improment]Query queued by be memory [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #37559:
URL: https://github.com/apache/doris/pull/37559#issuecomment-2238089189
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org