You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "Gabriel39 (via GitHub)" <gi...@apache.org> on 2024/07/22 06:30:37 UTC
[PR] [conf](parallel) Reduce parallel tasks for large cluster [doris]
Gabriel39 opened a new pull request, #38196:
URL: https://github.com/apache/doris/pull/38196
## Proposed changes
Issue Number: close #xxx
<!--Describe your changes.-->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [conf](parallel) Reduce parallel tasks for large cluster [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #38196:
URL: https://github.com/apache/doris/pull/38196#issuecomment-2242293199
<details>
<summary>TPC-DS: <b>Total hot run time: 173913 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 03c2d2d18544b8eb5cc45a564dd67829278c2c74, data reload: false
query1 926 373 369 369
query2 6432 1966 1861 1861
query3 6638 210 224 210
query4 21173 17619 17568 17568
query5 3770 479 496 479
query6 256 188 180 180
query7 4594 292 286 286
query8 250 194 195 194
query9 8870 2454 2435 2435
query10 435 304 277 277
query11 11019 10243 10140 10140
query12 114 88 81 81
query13 1648 372 369 369
query14 10389 7512 7773 7512
query15 213 163 165 163
query16 7655 494 458 458
query17 1568 558 529 529
query18 1847 277 283 277
query19 202 150 161 150
query20 91 84 82 82
query21 197 127 126 126
query22 4405 4203 4219 4203
query23 34280 33454 33727 33454
query24 11068 2977 2935 2935
query25 656 410 399 399
query26 1116 157 161 157
query27 2496 273 280 273
query28 7092 2054 2048 2048
query29 910 631 657 631
query30 256 155 149 149
query31 955 771 746 746
query32 102 52 58 52
query33 733 334 335 334
query34 956 491 502 491
query35 885 761 754 754
query36 1148 992 956 956
query37 158 94 80 80
query38 2907 2860 2830 2830
query39 932 897 847 847
query40 195 118 119 118
query41 47 44 43 43
query42 113 98 100 98
query43 494 465 475 465
query44 1156 713 721 713
query45 190 164 165 164
query46 1099 755 732 732
query47 1884 1781 1821 1781
query48 379 295 285 285
query49 836 401 410 401
query50 775 387 390 387
query51 6823 6746 6709 6709
query52 99 93 88 88
query53 363 288 281 281
query54 895 445 439 439
query55 74 73 72 72
query56 293 264 269 264
query57 1165 1068 1025 1025
query58 256 246 258 246
query59 3050 2604 2689 2604
query60 293 280 267 267
query61 128 94 97 94
query62 809 648 648 648
query63 318 282 281 281
query64 9445 2248 1684 1684
query65 3162 3089 3144 3089
query66 704 323 324 323
query67 15516 14961 15137 14961
query68 4444 527 522 522
query69 464 332 342 332
query70 1213 1086 1064 1064
query71 407 280 265 265
query72 7062 5094 5794 5094
query73 752 322 320 320
query74 6166 5668 5649 5649
query75 3366 2698 2727 2698
query76 2725 967 959 959
query77 454 318 316 316
query78 10656 8982 9418 8982
query79 2587 514 509 509
query80 2641 518 467 467
query81 598 221 223 221
query82 794 134 135 134
query83 309 164 168 164
query84 273 85 87 85
query85 2192 322 296 296
query86 495 328 319 319
query87 3310 3139 3101 3101
query88 4115 2449 2472 2449
query89 484 383 389 383
query90 1924 197 193 193
query91 126 102 99 99
query92 62 52 51 51
query93 2447 497 486 486
query94 1292 294 311 294
query95 407 325 324 324
query96 639 281 273 273
query97 3175 3060 3060 3060
query98 216 199 197 197
query99 1547 1265 1260 1260
Total cold run time: 277245 ms
Total hot run time: 173913 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [conf](parallel) Reduce parallel tasks for large cluster [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #38196:
URL: https://github.com/apache/doris/pull/38196#issuecomment-2242302554
<details>
<summary>ClickBench: <b>Total hot run time: 30.69 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 03c2d2d18544b8eb5cc45a564dd67829278c2c74, data reload: false
query1 0.04 0.03 0.03
query2 0.08 0.04 0.04
query3 0.23 0.06 0.06
query4 1.68 0.10 0.09
query5 0.50 0.48 0.49
query6 1.12 0.73 0.73
query7 0.02 0.01 0.02
query8 0.06 0.04 0.04
query9 0.54 0.50 0.48
query10 0.53 0.53 0.54
query11 0.16 0.11 0.11
query12 0.15 0.13 0.13
query13 0.60 0.59 0.59
query14 0.77 0.76 0.78
query15 0.87 0.82 0.81
query16 0.36 0.36 0.36
query17 1.03 0.98 1.03
query18 0.23 0.22 0.22
query19 1.88 1.67 1.84
query20 0.02 0.01 0.01
query21 15.39 0.74 0.65
query22 4.32 6.71 2.11
query23 18.28 1.44 1.20
query24 2.07 0.24 0.22
query25 0.15 0.08 0.08
query26 0.30 0.22 0.21
query27 0.45 0.23 0.22
query28 13.29 1.02 0.99
query29 12.64 3.38 3.36
query30 0.25 0.07 0.06
query31 2.87 0.40 0.38
query32 3.26 0.47 0.47
query33 2.90 2.93 2.86
query34 16.99 4.32 4.33
query35 4.46 4.44 4.39
query36 0.65 0.47 0.46
query37 0.19 0.16 0.17
query38 0.14 0.15 0.14
query39 0.04 0.04 0.03
query40 0.15 0.13 0.12
query41 0.09 0.04 0.05
query42 0.05 0.05 0.05
query43 0.04 0.04 0.04
Total cold run time: 109.84 s
Total hot run time: 30.69 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [conf](parallel) Reduce parallel tasks for large cluster [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #38196:
URL: https://github.com/apache/doris/pull/38196#issuecomment-2242274408
<details>
<summary>TPC-H: <b>Total hot run time: 39940 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 03c2d2d18544b8eb5cc45a564dd67829278c2c74, data reload: false
------ Round 1 ----------------------------------
q1 17623 4534 4259 4259
q2 2005 189 186 186
q3 10459 1250 1139 1139
q4 10187 778 915 778
q5 7617 2803 2628 2628
q6 219 136 137 136
q7 956 597 607 597
q8 9213 2090 2115 2090
q9 8756 6600 6589 6589
q10 8883 3782 3720 3720
q11 473 236 243 236
q12 418 229 229 229
q13 17780 2968 2986 2968
q14 286 226 242 226
q15 522 491 482 482
q16 519 389 377 377
q17 969 681 754 681
q18 8096 7449 7408 7408
q19 7802 1464 1446 1446
q20 696 333 325 325
q21 4847 3155 3172 3155
q22 348 293 285 285
Total cold run time: 118674 ms
Total hot run time: 39940 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4362 4298 4277 4277
q2 373 261 257 257
q3 3035 2912 3002 2912
q4 2009 1673 1726 1673
q5 5578 5527 5480 5480
q6 223 137 141 137
q7 2307 1852 1873 1852
q8 3262 3436 3447 3436
q9 8788 8880 8871 8871
q10 4197 3779 3890 3779
q11 597 490 505 490
q12 827 662 645 645
q13 17004 3110 3204 3110
q14 318 282 282 282
q15 548 494 490 490
q16 498 430 434 430
q17 1837 1528 1493 1493
q18 8107 8012 7879 7879
q19 1816 1511 1523 1511
q20 2130 1887 1885 1885
q21 10067 4876 4905 4876
q22 588 498 504 498
Total cold run time: 78471 ms
Total hot run time: 56263 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [conf](parallel) Reduce parallel tasks for large cluster [doris]
Posted by "Gabriel39 (via GitHub)" <gi...@apache.org>.
Gabriel39 merged PR #38196:
URL: https://github.com/apache/doris/pull/38196
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [conf](parallel) Reduce parallel tasks for large cluster [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #38196:
URL: https://github.com/apache/doris/pull/38196#issuecomment-2242214522
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [conf](parallel) Reduce parallel tasks for large cluster [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #38196:
URL: https://github.com/apache/doris/pull/38196#issuecomment-2242214479
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [conf](parallel) Reduce parallel tasks for large cluster [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #38196:
URL: https://github.com/apache/doris/pull/38196#issuecomment-2242188627
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR)
Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website).
See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [conf](parallel) Reduce parallel tasks for large cluster [doris]
Posted by "Gabriel39 (via GitHub)" <gi...@apache.org>.
Gabriel39 commented on PR #38196:
URL: https://github.com/apache/doris/pull/38196#issuecomment-2242188694
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org