You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "xingyingone (via GitHub)" <gi...@apache.org> on 2024/01/09 02:42:53 UTC
[PR] [improvement](executor) improve readability and efficiency of the checks [doris]
xingyingone opened a new pull request, #29696:
URL: https://github.com/apache/doris/pull/29696
## Proposed changes
Issue Number: close #xxx
<!--Describe your changes.-->
## Further comments
If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](executor) improve readability and efficiency of the checks [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #29696:
URL: https://github.com/apache/doris/pull/29696#issuecomment-1882337150
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](executor) improve readability and efficiency of the checks [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #29696:
URL: https://github.com/apache/doris/pull/29696#issuecomment-1882259878
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](executor) improve readability and efficiency of the checks [doris]
Posted by "adonis0147 (via GitHub)" <gi...@apache.org>.
adonis0147 commented on code in PR #29696:
URL: https://github.com/apache/doris/pull/29696#discussion_r1445578359
##########
be/src/runtime/task_group/task_group_manager.cpp:
##########
@@ -69,18 +69,16 @@ TaskGroupPtr TaskGroupManager::get_task_group_by_id(uint64_t tg_id) {
bool TaskGroupManager::set_cg_task_sche_for_query_ctx(uint64_t tg_id, QueryContext* query_ctx_ptr) {
std::lock_guard<std::shared_mutex> write_lock(_task_scheduler_lock);
- if (_tg_sche_map.find(tg_id) != _tg_sche_map.end()) {
- query_ctx_ptr->set_task_scheduler(_tg_sche_map.at(tg_id).get());
- } else {
- return false;
- }
Review Comment:
The original one can support failing fast and it can be more efficient than the modifications under some scenarios.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](executor) improve readability and efficiency of the checks [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29696:
URL: https://github.com/apache/doris/pull/29696#issuecomment-1882336228
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 46.13 seconds
stream load tsv: 569 seconds loaded 74807831229 Bytes, about 125 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 27.3 seconds inserted 10000000 Rows, about 366K ops/s
storage size: 17184131717 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](executor) improve readability and efficiency of the checks [doris]
Posted by "yiguolei (via GitHub)" <gi...@apache.org>.
yiguolei merged PR #29696:
URL: https://github.com/apache/doris/pull/29696
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](executor) improve readability and efficiency of the checks [doris]
Posted by "xingyingone (via GitHub)" <gi...@apache.org>.
xingyingone commented on PR #29696:
URL: https://github.com/apache/doris/pull/29696#issuecomment-1882245561
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](executor) improve readability and efficiency of the checks [doris]
Posted by "adonis0147 (via GitHub)" <gi...@apache.org>.
adonis0147 commented on code in PR #29696:
URL: https://github.com/apache/doris/pull/29696#discussion_r1445578359
##########
be/src/runtime/task_group/task_group_manager.cpp:
##########
@@ -69,18 +69,16 @@ TaskGroupPtr TaskGroupManager::get_task_group_by_id(uint64_t tg_id) {
bool TaskGroupManager::set_cg_task_sche_for_query_ctx(uint64_t tg_id, QueryContext* query_ctx_ptr) {
std::lock_guard<std::shared_mutex> write_lock(_task_scheduler_lock);
- if (_tg_sche_map.find(tg_id) != _tg_sche_map.end()) {
- query_ctx_ptr->set_task_scheduler(_tg_sche_map.at(tg_id).get());
- } else {
- return false;
- }
Review Comment:
The original one can support failing fast and it can be more efficient than the modifications under some scenarios.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](executor) improve readability and efficiency of the checks [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29696:
URL: https://github.com/apache/doris/pull/29696#issuecomment-1882301954
<details>
<summary>TPC-H: <b>Total hot run time: 38430 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 89887da04e9321a61548abc5c67064507a00d434, data reload: false
------ Round 1 ----------------------------------
q1 17780 4943 4904 4904
q2 2015 156 141 141
q3 10586 1118 1200 1118
q4 10457 819 830 819
q5 7798 2967 2917 2917
q6 202 122 125 122
q7 899 514 496 496
q8 9301 1968 1963 1963
q9 6429 6361 6394 6361
q10 8224 3050 2998 2998
q11 429 209 210 209
q12 380 229 231 229
q13 18003 3374 3381 3374
q14 239 213 207 207
q15 544 504 508 504
q16 434 393 396 393
q17 948 706 588 588
q18 7259 6632 6616 6616
q19 1545 1507 1509 1507
q20 704 297 291 291
q21 2707 2415 2364 2364
q22 359 309 323 309
Total cold run time: 107242 ms
Total hot run time: 38430 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4891 4930 4912 4912
q2 331 250 243 243
q3 3265 3283 3246 3246
q4 2096 2002 1996 1996
q5 5769 5746 5742 5742
q6 204 113 117 113
q7 2304 1884 1869 1869
q8 3360 3425 3452 3425
q9 8787 8716 8743 8716
q10 3779 3835 3842 3835
q11 544 447 458 447
q12 798 626 628 626
q13 6147 3176 3178 3176
q14 295 278 255 255
q15 544 504 494 494
q16 554 496 494 494
q17 1841 1848 1856 1848
q18 8553 8369 8217 8217
q19 1579 1637 1611 1611
q20 2176 1942 1934 1934
q21 5622 5187 5245 5187
q22 502 460 477 460
Total cold run time: 63941 ms
Total hot run time: 58846 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](executor) improve readability and efficiency of the checks [doris]
Posted by "xingyingone (via GitHub)" <gi...@apache.org>.
xingyingone commented on code in PR #29696:
URL: https://github.com/apache/doris/pull/29696#discussion_r1445579831
##########
be/src/runtime/task_group/task_group_manager.cpp:
##########
@@ -69,18 +69,16 @@ TaskGroupPtr TaskGroupManager::get_task_group_by_id(uint64_t tg_id) {
bool TaskGroupManager::set_cg_task_sche_for_query_ctx(uint64_t tg_id, QueryContext* query_ctx_ptr) {
std::lock_guard<std::shared_mutex> write_lock(_task_scheduler_lock);
- if (_tg_sche_map.find(tg_id) != _tg_sche_map.end()) {
- query_ctx_ptr->set_task_scheduler(_tg_sche_map.at(tg_id).get());
- } else {
- return false;
- }
Review Comment:
> The original one can support failing fast and it can be more efficient than the modifications under some scenarios.
The process for failure scenarios is the same
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](executor) improve readability and efficiency of the checks [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29696:
URL: https://github.com/apache/doris/pull/29696#issuecomment-1882284064
TeamCity be ut coverage result:
Function Coverage: 36.66% (8639/23566)
Line Coverage: 28.68% (70226/244842)
Region Coverage: 27.63% (36322/131448)
Branch Coverage: 24.31% (18557/76320)
Coverage Report: http://coverage.selectdb-in.cc/coverage/89887da04e9321a61548abc5c67064507a00d434_89887da04e9321a61548abc5c67064507a00d434/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](executor) improve readability and efficiency of the checks [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #29696:
URL: https://github.com/apache/doris/pull/29696#issuecomment-1882337086
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](executor) improve readability and efficiency of the checks [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29696:
URL: https://github.com/apache/doris/pull/29696#issuecomment-1882325079
<details>
<summary>TPC-DS: <b>Total hot run time: 184763 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 89887da04e9321a61548abc5c67064507a00d434, data reload: false
run tpcds-sf100 query with default conf and session variables
query1 937 339 338 338
query2 6567 2011 1923 1923
query3 6716 234 225 225
query4 26830 22558 22555 22555
query5 5769 603 583 583
query6 334 230 211 211
query7 4655 307 288 288
query8 273 241 231 231
query9 8743 2947 2948 2947
query10 554 336 313 313
query11 16320 15789 15757 15757
query12 170 114 98 98
query13 1693 351 343 343
query14 13543 7757 7529 7529
query15 268 245 216 216
query16 6437 340 334 334
query17 1701 522 535 522
query18 1950 309 300 300
query19 248 165 165 165
query20 105 97 105 97
query21 188 95 95 95
query22 4908 5012 4917 4917
query23 32567 31601 31516 31516
query24 12237 2994 2992 2992
query25 600 368 360 360
query26 1843 183 171 171
query27 3027 312 302 302
query28 6942 2135 2123 2123
query29 2063 421 419 419
query30 297 145 150 145
query31 1021 827 820 820
query32 130 86 88 86
query33 840 344 343 343
query34 846 480 471 471
query35 1001 918 866 866
query36 1377 1245 1295 1245
query37 239 93 88 88
query38 3441 3362 3359 3359
query39 1341 1319 1459 1319
query40 302 92 89 89
query41 38 35 36 35
query42 114 111 110 110
query43 583 532 530 530
query44 1207 808 815 808
query45 215 206 212 206
query46 1063 722 743 722
query47 1767 1683 1610 1610
query48 367 320 284 284
query49 1207 322 322 322
query50 740 339 328 328
query51 5452 5270 5309 5270
query52 123 97 99 97
query53 242 175 170 170
query54 1419 687 699 687
query55 116 106 102 102
query56 304 245 288 245
query57 1033 1024 972 972
query58 328 294 290 290
query59 3087 2777 2793 2777
query60 348 302 305 302
query61 133 146 149 146
query62 571 500 484 484
query63 199 183 188 183
query64 6033 1710 1718 1710
query65 3428 3358 3364 3358
query66 1372 362 415 362
query67 15699 15569 15349 15349
query68 13012 541 547 541
query69 597 302 325 302
query70 1796 1605 1654 1605
query71 606 296 258 258
query72 5304 3400 3412 3400
query73 3045 339 344 339
query74 6963 6439 6560 6439
query75 5020 2328 2292 2292
query76 6303 1059 1178 1059
query77 749 303 311 303
query78 9020 8603 8507 8507
query79 1093 535 531 531
query80 574 347 347 347
query81 462 213 213 213
query82 241 112 106 106
query83 188 143 140 140
query84 250 57 57 57
query85 954 281 259 259
query86 436 445 413 413
query87 3563 3449 3430 3430
query88 3085 2513 2532 2513
query89 375 283 269 269
query90 1890 267 253 253
query91 122 95 97 95
query92 81 89 69 69
query93 1139 503 503 503
query94 913 249 224 224
query95 586 510 468 468
query96 637 341 335 335
query97 4336 4231 4338 4231
query98 238 224 220 220
query99 1171 898 882 882
Total cold run time: 300960 ms
Total hot run time: 184763 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](executor) improve readability and efficiency of the checks [doris]
Posted by "xingyingone (via GitHub)" <gi...@apache.org>.
xingyingone commented on code in PR #29696:
URL: https://github.com/apache/doris/pull/29696#discussion_r1445579831
##########
be/src/runtime/task_group/task_group_manager.cpp:
##########
@@ -69,18 +69,16 @@ TaskGroupPtr TaskGroupManager::get_task_group_by_id(uint64_t tg_id) {
bool TaskGroupManager::set_cg_task_sche_for_query_ctx(uint64_t tg_id, QueryContext* query_ctx_ptr) {
std::lock_guard<std::shared_mutex> write_lock(_task_scheduler_lock);
- if (_tg_sche_map.find(tg_id) != _tg_sche_map.end()) {
- query_ctx_ptr->set_task_scheduler(_tg_sche_map.at(tg_id).get());
- } else {
- return false;
- }
Review Comment:
> The original one can support failing fast and it can be more efficient than the modifications under some scenarios.
The process for failure scenarios is the same
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org