You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "sollhui (via GitHub)" <gi...@apache.org> on 2025/11/14 12:51:30 UTC
[PR] [enhance](job) terminate streaming task execute threads promptly when idle [doris]
sollhui opened a new pull request, #58041:
URL: https://github.com/apache/doris/pull/58041
### What problem does this PR solve?
1. terminate streaming task execute threads promptly when idle.
2. set job_streaming_task_exec_thread_num default value from 10 to 100.
### Release note
None
### Check List (For Author)
- Test <!-- At least one of them must be included. -->
- [ ] Regression test
- [ ] Unit Test
- [ ] Manual test (add detailed scripts or steps below)
- [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
- [ ] Previous test can cover this change.
- [ ] No code files have been changed.
- [ ] Other reason <!-- Add your reason? -->
- Behavior changed:
- [ ] No.
- [ ] Yes. <!-- Explain the behavior change -->
- Does this need documentation?
- [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg: https://github.com/apache/doris-website/pull/1214 -->
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR should merge into -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](job) terminate streaming task execute threads promptly when idle [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #58041:
URL: https://github.com/apache/doris/pull/58041#issuecomment-3533029984
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](job) terminate streaming task execute threads promptly when idle [doris]
Posted by "sollhui (via GitHub)" <gi...@apache.org>.
sollhui commented on PR #58041:
URL: https://github.com/apache/doris/pull/58041#issuecomment-3532593050
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](job) terminate streaming task execute threads promptly when idle [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #58041:
URL: https://github.com/apache/doris/pull/58041#issuecomment-3532937670
<details>
<summary>TPC-DS: <b>Total hot run time: 187680 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit f6d374b455d44c02243f96cd01ad5a31a0151810, data reload: false
query1 1019 406 394 394
query2 6569 1760 1737 1737
query3 6767 222 224 222
query4 26348 23565 23135 23135
query5 4398 631 485 485
query6 354 251 235 235
query7 4650 496 304 304
query8 320 259 279 259
query9 8727 2616 2607 2607
query10 471 334 290 290
query11 15109 14999 14832 14832
query12 176 123 112 112
query13 1697 571 461 461
query14 10620 9358 9486 9358
query15 202 195 180 180
query16 7736 683 527 527
query17 1261 775 638 638
query18 2040 428 348 348
query19 213 210 218 210
query20 134 122 120 120
query21 218 132 109 109
query22 4061 4026 3986 3986
query23 33917 32657 32968 32657
query24 8444 2413 2431 2413
query25 582 523 450 450
query26 1118 273 158 158
query27 2749 486 357 357
query28 4326 2205 2179 2179
query29 847 631 488 488
query30 292 225 191 191
query31 896 786 712 712
query32 79 79 75 75
query33 587 392 331 331
query34 802 838 526 526
query35 804 851 765 765
query36 941 994 882 882
query37 123 111 89 89
query38 3502 3546 3441 3441
query39 1453 1426 1441 1426
query40 225 129 119 119
query41 62 60 61 60
query42 124 115 112 112
query43 487 502 471 471
query44 1245 748 741 741
query45 183 181 172 172
query46 876 981 652 652
query47 1720 1770 1677 1677
query48 399 431 327 327
query49 766 513 406 406
query50 655 680 412 412
query51 4002 3929 3981 3929
query52 104 108 103 103
query53 247 275 198 198
query54 333 296 272 272
query55 85 82 84 82
query56 341 342 322 322
query57 1184 1167 1116 1116
query58 287 276 289 276
query59 2482 2672 2545 2545
query60 360 355 340 340
query61 155 169 162 162
query62 801 717 683 683
query63 230 199 199 199
query64 4451 1166 840 840
query65 4027 3977 3971 3971
query66 1093 443 340 340
query67 15318 14994 15094 14994
query68 8282 938 597 597
query69 520 328 293 293
query70 1298 1233 1286 1233
query71 445 349 312 312
query72 5427 4815 4728 4728
query73 632 560 362 362
query74 8764 9138 8944 8944
query75 3290 3304 2863 2863
query76 3276 1145 753 753
query77 564 414 311 311
query78 9485 9615 8859 8859
query79 1450 842 614 614
query80 639 576 505 505
query81 515 265 240 240
query82 214 165 130 130
query83 274 276 260 260
query84 259 114 90 90
query85 890 483 457 457
query86 375 301 296 296
query87 3677 3732 3632 3632
query88 2879 2262 2269 2262
query89 396 336 297 297
query90 1824 225 231 225
query91 170 168 138 138
query92 76 71 69 69
query93 1139 974 652 652
query94 652 450 338 338
query95 410 334 317 317
query96 489 585 283 283
query97 2890 2947 2903 2903
query98 236 212 210 210
query99 1314 1404 1297 1297
Total cold run time: 269545 ms
Total hot run time: 187680 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](job) terminate streaming task execute threads promptly when idle [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #58041:
URL: https://github.com/apache/doris/pull/58041#issuecomment-3533014620
# FE UT Coverage Report
Increment line coverage ` 100.00% (2/2)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/58041_f6d374b455d44c02243f96cd01ad5a31a0151810/fe_increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/58041_f6d374b455d44c02243f96cd01ad5a31a0151810/fe_report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](job) terminate streaming task execute threads promptly when idle [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #58041:
URL: https://github.com/apache/doris/pull/58041#issuecomment-3533030173
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](job) terminate streaming task execute threads promptly when idle [doris]
Posted by "liaoxin01 (via GitHub)" <gi...@apache.org>.
liaoxin01 merged PR #58041:
URL: https://github.com/apache/doris/pull/58041
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](job) terminate streaming task execute threads promptly when idle [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #58041:
URL: https://github.com/apache/doris/pull/58041#issuecomment-3532959673
<details>
<summary>ClickBench: <b>Total hot run time: 27.75 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit f6d374b455d44c02243f96cd01ad5a31a0151810, data reload: false
query1 0.05 0.06 0.05
query2 0.09 0.05 0.05
query3 0.26 0.08 0.08
query4 1.62 0.11 0.11
query5 0.26 0.25 0.25
query6 1.16 0.66 0.65
query7 0.03 0.03 0.02
query8 0.05 0.04 0.04
query9 0.57 0.52 0.51
query10 0.58 0.56 0.57
query11 0.16 0.11 0.12
query12 0.15 0.12 0.12
query13 0.62 0.61 0.61
query14 1.00 0.99 0.99
query15 0.85 0.83 0.84
query16 0.39 0.39 0.39
query17 1.05 1.06 1.03
query18 0.21 0.20 0.19
query19 1.87 1.88 1.79
query20 0.01 0.01 0.02
query21 15.45 0.22 0.13
query22 5.06 0.07 0.05
query23 15.66 0.26 0.10
query24 2.38 0.70 0.80
query25 0.08 0.05 0.06
query26 0.15 0.14 0.12
query27 0.05 0.06 0.05
query28 5.18 1.15 0.93
query29 12.55 3.90 3.21
query30 0.29 0.15 0.12
query31 2.81 0.59 0.38
query32 3.22 0.56 0.47
query33 3.10 3.04 3.08
query34 15.93 5.20 4.62
query35 4.56 4.59 4.54
query36 0.68 0.50 0.50
query37 0.11 0.07 0.07
query38 0.06 0.04 0.04
query39 0.03 0.03 0.03
query40 0.18 0.14 0.13
query41 0.08 0.03 0.03
query42 0.04 0.03 0.02
query43 0.04 0.04 0.03
Total cold run time: 98.67 s
Total hot run time: 27.75 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](job) terminate streaming task execute threads promptly when idle [doris]
Posted by "Thearas (via GitHub)" <gi...@apache.org>.
Thearas commented on PR #58041:
URL: https://github.com/apache/doris/pull/58041#issuecomment-3532592835
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR).
Please clearly describe your PR:
1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
3. What features were added. Why was this function added?
4. Which code was refactored and why was this part of the code refactored?
5. Which functions were optimized and what is the difference before and after the optimization?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](job) terminate streaming task execute threads promptly when idle [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #58041:
URL: https://github.com/apache/doris/pull/58041#issuecomment-3532892475
<details>
<summary>TPC-H: <b>Total hot run time: 34302 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit f6d374b455d44c02243f96cd01ad5a31a0151810, data reload: false
------ Round 1 ----------------------------------
q1 17612 5165 5072 5072
q2 2051 308 203 203
q3 10288 1289 733 733
q4 10232 889 378 378
q5 7490 2452 2288 2288
q6 189 171 140 140
q7 936 781 632 632
q8 9362 1343 1141 1141
q9 6906 5113 5143 5113
q10 6866 2188 1796 1796
q11 493 315 281 281
q12 348 374 232 232
q13 17786 3681 3094 3094
q14 228 237 221 221
q15 571 521 495 495
q16 1026 996 945 945
q17 577 887 368 368
q18 7299 7107 7104 7104
q19 1103 960 575 575
q20 367 351 229 229
q21 3863 3219 2315 2315
q22 1068 1025 947 947
Total cold run time: 106661 ms
Total hot run time: 34302 ms
----- Round 2, with runtime_filter_mode=off -----
q1 5166 5144 5183 5144
q2 251 318 249 249
q3 2206 2693 2262 2262
q4 1402 1797 1402 1402
q5 4189 4424 4456 4424
q6 217 176 137 137
q7 2048 1971 1853 1853
q8 2724 2575 2525 2525
q9 7261 7311 7234 7234
q10 3002 3335 2813 2813
q11 615 539 518 518
q12 685 758 647 647
q13 3641 3948 3380 3380
q14 286 310 278 278
q15 538 496 492 492
q16 1086 1107 1075 1075
q17 1193 1602 1432 1432
q18 7799 7757 7451 7451
q19 869 961 980 961
q20 2003 2084 1879 1879
q21 4972 4381 4372 4372
q22 1095 1073 1014 1014
Total cold run time: 53248 ms
Total hot run time: 51542 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](job) terminate streaming task execute threads promptly when idle [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #58041:
URL: https://github.com/apache/doris/pull/58041#issuecomment-3533368265
# FE Regression Coverage Report
Increment line coverage ` 100.00% (2/2)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/58041_f6d374b455d44c02243f96cd01ad5a31a0151810_merge_fe/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/58041_f6d374b455d44c02243f96cd01ad5a31a0151810_merge_fe/report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org