You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "mrhhsg (via GitHub)" <gi...@apache.org> on 2024/01/06 03:56:48 UTC
[PR] [chore](thirdparty) update vectorscan to 5.7.11 [doris]
mrhhsg opened a new pull request, #29623:
URL: https://github.com/apache/doris/pull/29623
## Proposed changes
Issue Number: close #xxx
<!--Describe your changes.-->
## Further comments
If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [chore](thirdparty) update vectorscan to 5.7.11 [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29623:
URL: https://github.com/apache/doris/pull/29623#issuecomment-1879574809
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 47.61 seconds
stream load tsv: 563 seconds loaded 74807831229 Bytes, about 126 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 28.1 seconds inserted 10000000 Rows, about 355K ops/s
storage size: 17183449951 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [chore](thirdparty) update vectorscan to 5.7.11 [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29623:
URL: https://github.com/apache/doris/pull/29623#issuecomment-1879574699
<details>
<summary>TPC-DS: <b>Total hot run time: 184328 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit f75fa18f800e59f82d4370cc0bb867ae5a50cc1c, data reload: false
run tpcds-sf100 query with default conf and session variables
query1 907 345 348 345
query2 5035 1934 1923 1923
query3 5146 230 221 221
query4 25621 22496 22454 22454
query5 5104 614 586 586
query6 342 199 206 199
query7 4351 315 297 297
query8 271 234 216 216
query9 7156 2915 2938 2915
query10 544 350 344 344
query11 16257 15688 15657 15657
query12 166 97 101 97
query13 1509 345 361 345
query14 11930 7541 7722 7541
query15 259 239 210 210
query16 5920 333 329 329
query17 1395 523 519 519
query18 1873 309 293 293
query19 217 155 157 155
query20 111 103 103 103
query21 164 93 96 93
query22 5025 4766 4902 4766
query23 32261 31273 31226 31226
query24 12082 2997 2942 2942
query25 613 366 366 366
query26 1711 168 166 166
query27 3021 318 325 318
query28 6727 2171 2065 2065
query29 1876 428 417 417
query30 279 145 146 145
query31 1062 824 821 821
query32 138 90 87 87
query33 808 357 337 337
query34 850 468 468 468
query35 976 874 895 874
query36 1418 1234 1339 1234
query37 191 93 88 88
query38 3503 3441 3354 3354
query39 1339 1285 1466 1285
query40 288 92 91 91
query41 39 40 38 38
query42 113 109 106 106
query43 578 533 545 533
query44 1280 806 779 779
query45 235 209 201 201
query46 1071 702 692 692
query47 1657 1606 1695 1606
query48 373 281 300 281
query49 1070 311 311 311
query50 783 328 365 328
query51 5386 5362 5268 5268
query52 110 102 98 98
query53 242 173 174 173
query54 1654 679 688 679
query55 104 100 99 99
query56 267 272 259 259
query57 1025 1010 969 969
query58 312 285 284 284
query59 3077 2926 2818 2818
query60 324 290 301 290
query61 132 131 133 131
query62 579 479 476 476
query63 196 172 169 169
query64 5651 1760 1704 1704
query65 3404 3359 3374 3359
query66 1324 393 411 393
query67 15638 15660 15476 15476
query68 12212 565 565 565
query69 559 334 324 324
query70 1686 1621 1663 1621
query71 587 294 306 294
query72 5389 3486 3457 3457
query73 2303 343 348 343
query74 7073 6418 6498 6418
query75 5034 2258 2287 2258
query76 6224 1051 1260 1051
query77 745 355 337 337
query78 9078 8607 8658 8607
query79 1041 532 532 532
query80 541 342 363 342
query81 481 209 212 209
query82 217 108 100 100
query83 211 144 151 144
query84 260 58 61 58
query85 936 291 269 269
query86 441 416 424 416
query87 3623 3484 3452 3452
query88 3157 2478 2524 2478
query89 378 287 285 285
query90 1988 264 264 264
query91 121 103 106 103
query92 90 78 79 78
query93 1471 500 495 495
query94 895 253 246 246
query95 565 517 499 499
query96 643 350 344 344
query97 4349 4295 4267 4267
query98 239 222 213 213
query99 1109 847 848 847
Total cold run time: 288716 ms
Total hot run time: 184328 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [chore](thirdparty) update vectorscan to 5.7.11 [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #29623:
URL: https://github.com/apache/doris/pull/29623#issuecomment-1879588728
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [chore](thirdparty) update vectorscan to 5.7.11 [doris]
Posted by "gong-flying (via GitHub)" <gi...@apache.org>.
gong-flying commented on PR #29623:
URL: https://github.com/apache/doris/pull/29623#issuecomment-3230835668
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [chore](thirdparty) update vectorscan to 5.7.11 [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29623:
URL: https://github.com/apache/doris/pull/29623#issuecomment-1879546612
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 46.74 seconds
stream load tsv: 563 seconds loaded 74807831229 Bytes, about 126 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 27.1 seconds inserted 10000000 Rows, about 369K ops/s
storage size: 17183838325 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [chore](thirdparty) update vectorscan to 5.7.11 [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29623:
URL: https://github.com/apache/doris/pull/29623#issuecomment-1879554582
<details>
<summary>TPC-DS: <b>Total hot run time: 184676 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 51d2fa13c7b25c793ebe04a27d39dccc763c9bc4, data reload: false
run tpcds-sf100 query with default conf and session variables
query1 937 353 333 333
query2 6554 1954 1993 1954
query3 6720 239 223 223
query4 28011 22540 22517 22517
query5 5954 597 622 597
query6 332 205 214 205
query7 4627 316 304 304
query8 253 227 218 218
query9 8685 2919 2928 2919
query10 560 320 322 320
query11 15918 15728 15658 15658
query12 185 106 95 95
query13 1686 347 341 341
query14 13557 7649 7530 7530
query15 263 240 230 230
query16 6403 341 328 328
query17 1706 512 516 512
query18 1949 311 302 302
query19 239 163 155 155
query20 119 99 94 94
query21 189 102 96 96
query22 5102 5024 5042 5024
query23 32236 31246 30812 30812
query24 12174 2898 2968 2898
query25 605 362 361 361
query26 1865 161 167 161
query27 2937 307 326 307
query28 6811 2164 2143 2143
query29 2086 405 410 405
query30 291 144 147 144
query31 1021 843 832 832
query32 127 90 71 71
query33 836 340 347 340
query34 844 494 480 480
query35 984 838 843 838
query36 1382 1215 1256 1215
query37 208 93 87 87
query38 3448 3385 3350 3350
query39 1356 1292 1294 1292
query40 305 92 90 90
query41 39 36 36 36
query42 110 105 110 105
query43 565 539 534 534
query44 1326 811 814 811
query45 221 199 200 199
query46 1061 712 709 709
query47 1789 1736 1676 1676
query48 378 295 289 289
query49 1219 310 309 309
query50 783 334 376 334
query51 5372 5324 5301 5301
query52 102 102 100 100
query53 238 170 169 169
query54 1432 665 697 665
query55 106 99 98 98
query56 255 250 262 250
query57 1127 996 1044 996
query58 302 290 275 275
query59 3017 2838 2928 2838
query60 363 300 297 297
query61 129 153 132 132
query62 566 497 489 489
query63 188 173 186 173
query64 5800 1719 1752 1719
query65 3425 3340 3331 3331
query66 1343 403 415 403
query67 15710 15754 15834 15754
query68 13869 564 554 554
query69 630 321 327 321
query70 1924 1667 1579 1579
query71 598 299 251 251
query72 5250 3446 3429 3429
query73 3275 346 354 346
query74 6966 6469 6552 6469
query75 5334 2247 2249 2247
query76 6403 1129 1153 1129
query77 799 349 312 312
query78 9080 8554 8552 8552
query79 2496 521 543 521
query80 555 346 343 343
query81 465 219 219 219
query82 232 105 110 105
query83 181 158 150 150
query84 258 57 57 57
query85 944 283 267 267
query86 446 442 456 442
query87 3642 3524 3465 3465
query88 2921 2519 2545 2519
query89 358 288 291 288
query90 1929 258 258 258
query91 125 93 99 93
query92 94 74 78 74
query93 1209 495 509 495
query94 900 247 246 246
query95 537 512 493 493
query96 664 346 347 346
query97 4325 4275 4238 4238
query98 238 233 219 219
query99 1153 886 876 876
Total cold run time: 304231 ms
Total hot run time: 184676 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [chore](thirdparty) update vectorscan to 5.7.11 [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #29623:
URL: https://github.com/apache/doris/pull/29623#issuecomment-1879588718
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [chore](thirdparty) update vectorscan to 5.7.11 [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29623:
URL: https://github.com/apache/doris/pull/29623#issuecomment-1879552164
<details>
<summary>TPC-H: <b>Total hot run time: 38635 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 51d2fa13c7b25c793ebe04a27d39dccc763c9bc4, data reload: false
------ Round 1 ----------------------------------
q1 17753 4960 4938 4938
q2 2017 156 143 143
q3 10580 1126 1087 1087
q4 10213 819 930 819
q5 7790 2924 2864 2864
q6 203 122 123 122
q7 919 503 502 502
q8 9281 1996 2022 1996
q9 6464 6406 6374 6374
q10 8251 3084 2992 2992
q11 432 215 221 215
q12 383 231 234 231
q13 17997 3409 3361 3361
q14 245 224 214 214
q15 548 505 508 505
q16 453 399 434 399
q17 945 741 619 619
q18 7397 6695 6681 6681
q19 1609 1531 1531 1531
q20 714 316 337 316
q21 2751 2416 2447 2416
q22 362 310 314 310
Total cold run time: 107307 ms
Total hot run time: 38635 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4924 4901 4869 4869
q2 336 231 243 231
q3 3274 3290 3302 3290
q4 2121 2014 1999 1999
q5 5820 5814 5786 5786
q6 203 116 115 115
q7 2332 1899 1947 1899
q8 3359 3443 3481 3443
q9 8773 8762 8752 8752
q10 3783 3824 3826 3824
q11 543 440 448 440
q12 800 620 640 620
q13 5689 3165 3195 3165
q14 281 264 261 261
q15 545 509 506 506
q16 560 527 499 499
q17 1886 1873 1854 1854
q18 8607 8386 8372 8372
q19 1698 1634 1625 1625
q20 2199 1945 1948 1945
q21 5627 5309 5385 5309
q22 523 478 441 441
Total cold run time: 63883 ms
Total hot run time: 59245 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [chore](thirdparty) update vectorscan to 5.7.11 [doris]
Posted by "yiguolei (via GitHub)" <gi...@apache.org>.
yiguolei merged PR #29623:
URL: https://github.com/apache/doris/pull/29623
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [chore](thirdparty) update vectorscan to 5.7.11 [doris]
Posted by "mrhhsg (via GitHub)" <gi...@apache.org>.
mrhhsg commented on PR #29623:
URL: https://github.com/apache/doris/pull/29623#issuecomment-1879568884
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [chore](thirdparty) update vectorscan to 5.7.11 [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29623:
URL: https://github.com/apache/doris/pull/29623#issuecomment-1879572923
<details>
<summary>TPC-H: <b>Total hot run time: 38485 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit f75fa18f800e59f82d4370cc0bb867ae5a50cc1c, data reload: false
------ Round 1 ----------------------------------
q1 17710 4927 4975 4927
q2 2027 154 142 142
q3 10724 1127 1179 1127
q4 10336 730 838 730
q5 7802 2989 3005 2989
q6 201 126 121 121
q7 936 513 498 498
q8 9228 2000 2006 2000
q9 6488 6392 6383 6383
q10 8280 3048 3046 3046
q11 444 209 211 209
q12 383 234 234 234
q13 18008 3354 3380 3354
q14 239 222 199 199
q15 550 499 510 499
q16 485 401 416 401
q17 956 685 497 497
q18 7394 6778 6676 6676
q19 1590 1518 1525 1518
q20 715 330 307 307
q21 2781 2395 2324 2324
q22 361 304 322 304
Total cold run time: 107638 ms
Total hot run time: 38485 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4920 4898 4908 4898
q2 341 239 273 239
q3 3310 3310 3303 3303
q4 2105 2032 2028 2028
q5 5793 5780 5768 5768
q6 214 118 116 116
q7 2382 1908 1901 1901
q8 3367 3451 3475 3451
q9 8835 8738 8767 8738
q10 3804 3842 3829 3829
q11 566 451 452 451
q12 800 645 630 630
q13 9330 3199 3186 3186
q14 299 271 246 246
q15 548 497 497 497
q16 552 493 499 493
q17 1894 1877 1857 1857
q18 8645 8342 8289 8289
q19 1601 1633 1641 1633
q20 2190 1980 1955 1955
q21 5516 5325 5196 5196
q22 524 469 446 446
Total cold run time: 67536 ms
Total hot run time: 59150 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [chore](thirdparty) update vectorscan to 5.7.11 [doris]
Posted by "mrhhsg (via GitHub)" <gi...@apache.org>.
mrhhsg commented on PR #29623:
URL: https://github.com/apache/doris/pull/29623#issuecomment-1879527131
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [chore](thirdparty) update vectorscan to 5.7.11 [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29623:
URL: https://github.com/apache/doris/pull/29623#issuecomment-1879576738
TeamCity be ut coverage result:
Function Coverage: 36.61% (8618/23539)
Line Coverage: 28.64% (70084/244673)
Region Coverage: 27.62% (36271/131343)
Branch Coverage: 24.32% (18531/76196)
Coverage Report: http://coverage.selectdb-in.cc/coverage/f75fa18f800e59f82d4370cc0bb867ae5a50cc1c_f75fa18f800e59f82d4370cc0bb867ae5a50cc1c/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [chore](thirdparty) update vectorscan to 5.7.11 [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29623:
URL: https://github.com/apache/doris/pull/29623#issuecomment-1879536806
TeamCity be ut coverage result:
Function Coverage: 36.61% (8618/23539)
Line Coverage: 28.65% (70079/244634)
Region Coverage: 27.61% (36258/131304)
Branch Coverage: 24.32% (18527/76170)
Coverage Report: http://coverage.selectdb-in.cc/coverage/51d2fa13c7b25c793ebe04a27d39dccc763c9bc4_51d2fa13c7b25c793ebe04a27d39dccc763c9bc4/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org