You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by GitBox <gi...@apache.org> on 2023/01/18 01:29:41 UTC
[GitHub] [doris] caiconghui opened a new issue, #16038: [Enhancement] Make query scan node more evenly distributed
caiconghui opened a new issue, #16038:
URL: https://github.com/apache/doris/issues/16038
### Search before asking
- [X] I had searched in the [issues](https://github.com/apache/doris/issues?q=is%3Aissue) and found no similar issues.
### Description
we create a table with 50 buckets on doris cluster with 50 be nodes, and do query benchmark, just open 50 connections, and
execute query just like "select count(1) from test", and we found that cpu usage is unbalanced，while high cost node is nearly 100 but low cost node is just 30%，and some nodes don't have fragment to execute， so we need to find a better solution to solve this problem.
Once the location of the replica is determined， the query scan node scheduling problem can be described as following:
Minimum cost and maximum flow
<img width="1237" alt="image" src="https://user-images.githubusercontent.com/55968745/213057900-aa14d395-bd99-4293-90f2-17a267690374.png">
### Solution
to find to most evenly distributed solution for one query，the cost is very high，so we add a new strategy to giving high Priority to nodes with few replicas for scan node scheduling
### Are you willing to submit PR?
- [X] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] caiconghui closed issue #16038: [Enhancement] Make query scan node more evenly distributed
Posted by GitBox <gi...@apache.org>.
caiconghui closed issue #16038: [Enhancement] Make query scan node more evenly distributed
URL: https://github.com/apache/doris/issues/16038
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org