You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "yiguolei (via GitHub)" <gi...@apache.org> on 2024/02/19 06:31:13 UTC
[PR] [bugfix](performance) fix performance problem [doris]
yiguolei opened a new pull request, #31093:
URL: https://github.com/apache/doris/pull/31093
## Proposed changes
Issue Number: close #xxx
<!--Describe your changes.-->
## Further comments
If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [bugfix](performance) fix performance problem [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #31093:
URL: https://github.com/apache/doris/pull/31093#issuecomment-1951900498
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [bugfix](performance) fix performance problem [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31093:
URL: https://github.com/apache/doris/pull/31093#issuecomment-1951900674
<details>
<summary>ClickBench: <b>Total hot run time: 31.85 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 262f049e11238ef4b31c467d1e5c5fe8dffac13d, data reload: false
query1 0.03 0.02 0.02
query2 0.06 0.03 0.03
query3 0.23 0.07 0.07
query4 1.67 0.09 0.09
query5 0.50 0.50 0.50
query6 1.39 0.61 0.62
query7 0.02 0.01 0.01
query8 0.04 0.03 0.03
query9 0.52 0.47 0.46
query10 0.50 0.50 0.50
query11 0.12 0.09 0.10
query12 0.13 0.10 0.10
query13 0.60 0.58 0.59
query14 0.79 0.79 0.80
query15 0.82 0.80 0.81
query16 0.33 0.34 0.33
query17 0.93 0.93 0.91
query18 0.18 0.18 0.17
query19 1.81 1.71 1.72
query20 0.02 0.01 0.01
query21 16.82 0.68 0.65
query22 5.11 6.73 3.00
query23 17.87 1.11 1.03
query24 2.22 0.54 0.32
query25 0.64 0.06 0.04
query26 0.15 0.13 0.13
query27 0.06 0.06 0.06
query28 12.06 0.86 0.84
query29 12.51 3.45 3.32
query30 0.52 0.52 0.48
query31 2.77 0.36 0.36
query32 3.35 0.49 0.48
query33 3.14 3.14 3.20
query34 15.35 4.52 4.49
query35 4.53 4.46 4.49
query36 1.07 0.97 0.96
query37 0.07 0.06 0.05
query38 0.04 0.03 0.04
query39 0.02 0.01 0.02
query40 0.18 0.14 0.14
query41 0.08 0.02 0.02
query42 0.02 0.02 0.01
query43 0.03 0.02 0.02
Total cold run time: 109.3 s
Total hot run time: 31.85 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [bugfix](performance) fix performance problem [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31093:
URL: https://github.com/apache/doris/pull/31093#issuecomment-1951886453
TeamCity be ut coverage result:
Function Coverage: 35.78% (8558/23918)
Line Coverage: 27.72% (69386/250313)
Region Coverage: 26.85% (36014/134145)
Branch Coverage: 23.64% (18412/77878)
Coverage Report: http://coverage.selectdb-in.cc/coverage/262f049e11238ef4b31c467d1e5c5fe8dffac13d_262f049e11238ef4b31c467d1e5c5fe8dffac13d/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [bugfix](performance) fix performance problem [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31093:
URL: https://github.com/apache/doris/pull/31093#issuecomment-1951877052
<details>
<summary>TPC-H: <b>Total hot run time: 41767 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 262f049e11238ef4b31c467d1e5c5fe8dffac13d, data reload: false
------ Round 1 ----------------------------------
q1 5391 4974 4958 4958
q2 411 146 135 135
q3 1431 1037 1014 1014
q4 1184 1001 1004 1001
q5 2962 3281 3261 3261
q6 194 135 135 135
q7 1265 898 875 875
q8 1989 2077 2062 2062
q9 6712 6749 6701 6701
q10 6124 2638 2634 2634
q11 326 224 234 224
q12 514 329 331 329
q13 16796 3665 3675 3665
q14 296 270 255 255
q15 619 508 532 508
q16 480 414 454 414
q17 900 855 821 821
q18 7207 6698 6720 6698
q19 1646 1491 1479 1479
q20 562 337 352 337
q21 4299 3930 3917 3917
q22 396 345 344 344
Total cold run time: 61704 ms
Total hot run time: 41767 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4905 4950 5041 4950
q2 298 196 194 194
q3 3691 3634 3652 3634
q4 2555 2544 2502 2502
q5 5821 5802 5817 5802
q6 212 124 127 124
q7 2264 1655 1682 1655
q8 3046 3089 3124 3089
q9 8729 8702 8713 8702
q10 4173 4242 4227 4227
q11 531 372 371 371
q12 765 541 550 541
q13 4167 3407 3391 3391
q14 264 234 247 234
q15 609 496 495 495
q16 474 430 443 430
q17 1712 1624 1650 1624
q18 8278 7494 7564 7494
q19 1637 1641 1638 1638
q20 2108 1824 1819 1819
q21 6548 6156 6178 6156
q22 582 521 520 520
Total cold run time: 63369 ms
Total hot run time: 59592 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [bugfix](performance) fix performance problem [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31093:
URL: https://github.com/apache/doris/pull/31093#issuecomment-1951789321
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [bugfix](performance) fix performance problem [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31093:
URL: https://github.com/apache/doris/pull/31093#issuecomment-1951904969
Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
```
Load test result on commit 262f049e11238ef4b31c467d1e5c5fe8dffac13d with default session variables
Stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc: 60 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet: 33 seconds loaded 861443392 Bytes, about 24 MB/s
Insert into select: 13.4 seconds inserted 10000000 Rows, about 746K ops/s
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [bugfix](performance) fix performance problem [doris]
Posted by "yiguolei (via GitHub)" <gi...@apache.org>.
yiguolei commented on PR #31093:
URL: https://github.com/apache/doris/pull/31093#issuecomment-1951789383
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [bugfix](performance) fix performance problem [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31093:
URL: https://github.com/apache/doris/pull/31093#issuecomment-1951891933
<details>
<summary>TPC-DS: <b>Total hot run time: 177976 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 262f049e11238ef4b31c467d1e5c5fe8dffac13d, data reload: false
query1 921 356 349 349
query2 6533 1716 1800 1716
query3 6720 215 206 206
query4 22966 21162 21062 21062
query5 4253 384 458 384
query6 268 170 182 170
query7 4602 289 292 289
query8 243 191 198 191
query9 8433 2841 2819 2819
query10 417 224 216 216
query11 14914 14579 14579 14579
query12 144 83 81 81
query13 1691 431 410 410
query14 9086 7758 7730 7730
query15 212 186 190 186
query16 7546 260 260 260
query17 1386 577 542 542
query18 1950 268 266 266
query19 188 148 148 148
query20 85 82 84 82
query21 183 133 118 118
query22 4923 4768 4749 4749
query23 32529 31463 31460 31460
query24 12764 3371 3360 3360
query25 650 390 359 359
query26 1868 153 156 153
query27 3065 323 328 323
query28 6569 1837 1830 1830
query29 1123 621 619 619
query30 285 135 146 135
query31 944 766 775 766
query32 91 62 59 59
query33 732 231 237 231
query34 1072 498 497 497
query35 930 838 845 838
query36 968 894 897 894
query37 137 59 60 59
query38 3318 3160 3195 3160
query39 1370 1320 1322 1320
query40 288 105 106 105
query41 38 36 34 34
query42 108 97 100 97
query43 489 442 447 442
query44 1083 701 699 699
query45 197 187 186 186
query46 1036 770 764 764
query47 1636 1517 1605 1517
query48 424 340 347 340
query49 1214 301 301 301
query50 783 386 381 381
query51 5258 5231 5235 5231
query52 99 97 96 96
query53 389 296 298 296
query54 302 224 234 224
query55 79 84 82 82
query56 218 193 197 193
query57 1077 996 945 945
query58 214 197 205 197
query59 2353 2180 2224 2180
query60 247 223 214 214
query61 89 85 84 84
query62 594 382 374 374
query63 335 291 298 291
query64 6323 3092 3191 3092
query65 3284 3255 3231 3231
query66 1353 326 345 326
query67 14431 14276 14215 14215
query68 5108 537 553 537
query69 519 362 357 357
query70 1237 1248 1264 1248
query71 451 251 253 251
query72 6306 2789 2590 2590
query73 704 318 309 309
query74 6922 6557 6506 6506
query75 3220 2553 2583 2553
query76 3353 1147 1203 1147
query77 367 241 236 236
query78 9445 8768 8836 8768
query79 956 502 502 502
query80 508 357 349 349
query81 439 207 224 207
query82 158 89 83 83
query83 142 120 125 120
query84 227 82 77 77
query85 1033 345 354 345
query86 289 290 307 290
query87 3469 3304 3327 3304
query88 2679 2302 2282 2282
query89 443 360 363 360
query90 1967 165 167 165
query91 158 131 126 126
query92 53 50 48 48
query93 960 483 463 463
query94 1111 176 180 176
query95 481 8579 376 376
query96 570 264 264 264
query97 4440 4262 4277 4262
query98 226 211 195 195
query99 1129 772 727 727
Total cold run time: 269090 ms
Total hot run time: 177976 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [bugfix](performance) fix performance problem [doris]
Posted by "HappenLee (via GitHub)" <gi...@apache.org>.
HappenLee merged PR #31093:
URL: https://github.com/apache/doris/pull/31093
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [bugfix](performance) fix performance problem [doris]
Posted by "yiguolei (via GitHub)" <gi...@apache.org>.
yiguolei commented on PR #31093:
URL: https://github.com/apache/doris/pull/31093#issuecomment-1951845164
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [bugfix](performance) fix performance problem [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #31093:
URL: https://github.com/apache/doris/pull/31093#issuecomment-1951798414
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [bugfix](performance) fix performance problem [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31093:
URL: https://github.com/apache/doris/pull/31093#issuecomment-1951834534
TeamCity be ut coverage result:
Function Coverage: 35.78% (8558/23916)
Line Coverage: 27.73% (69394/250291)
Region Coverage: 26.85% (36010/134129)
Branch Coverage: 23.66% (18423/77878)
Coverage Report: http://coverage.selectdb-in.cc/coverage/98f24351ce62884a20e4c235f8b8f2cc7307b16f_98f24351ce62884a20e4c235f8b8f2cc7307b16f/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [bugfix](performance) fix performance problem [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #31093:
URL: https://github.com/apache/doris/pull/31093#issuecomment-1951857443
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [bugfix](performance) fix performance problem [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #31093:
URL: https://github.com/apache/doris/pull/31093#issuecomment-1951900558
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org