You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "Gabriel39 (via GitHub)" <gi...@apache.org> on 2024/01/04 10:16:46 UTC
[PR] [pipelineX](fix) Fix incorrect hash distribution [doris]
Gabriel39 opened a new pull request, #29544:
URL: https://github.com/apache/doris/pull/29544
## Proposed changes
Issue Number: close #xxx
<!--Describe your changes.-->
## Further comments
If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [pipelineX](fix) Fix incorrect hash distribution [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #29544:
URL: https://github.com/apache/doris/pull/29544#issuecomment-1876853909
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [pipelineX](fix) Fix incorrect hash distribution [doris]
Posted by "Gabriel39 (via GitHub)" <gi...@apache.org>.
Gabriel39 closed pull request #29544: [pipelineX](fix) Fix incorrect hash distribution
URL: https://github.com/apache/doris/pull/29544
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [pipelineX](fix) Fix incorrect hash distribution [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29544:
URL: https://github.com/apache/doris/pull/29544#issuecomment-1876952226
TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G', run with scripts in https://github.com/apache/doris/tree/master/tools/tpch-tools
```
Tpch sf100 test result on commit 20c1872a0da95dbee5cb407c3078f24909839444, data reload: false
run tpch-sf100 query with default conf and session variables
q1 5436 5176 5205 5176
q2 384 163 155 155
q3 1470 1179 1148 1148
q4 1093 815 780 780
q5 3109 3063 3180 3063
q6 223 133 135 133
q7 959 530 530 530
q8 2149 2184 2290 2184
q9 6699 6651 6844 6651
q10 3164 3085 3120 3085
q11 344 216 204 204
q12 387 239 231 231
q13 4387 3638 3649 3638
q14 250 233 225 225
q15 585 540 528 528
q16 465 397 397 397
q17 1046 581 559 559
q18 7092 6731 6830 6731
q19 1644 1533 1490 1490
q20 549 328 340 328
q21 2864 2445 2449 2445
q22 402 339 343 339
Total cold run time: 44701 ms
Total hot run time: 40020 ms
run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1 5134 5103 5153 5103
q2 338 239 238 238
q3 3390 3320 3322 3320
q4 2157 2029 2036 2029
q5 5960 5910 5897 5897
q6 227 128 128 128
q7 2385 1908 1911 1908
q8 3545 3678 3659 3659
q9 9000 8970 8963 8963
q10 3871 3894 3911 3894
q11 584 485 466 466
q12 799 625 636 625
q13 3882 3180 3190 3180
q14 304 276 290 276
q15 603 529 535 529
q16 546 490 492 490
q17 2040 1811 1831 1811
q18 8737 8359 8594 8359
q19 1738 1687 1730 1687
q20 2254 2013 2000 2000
q21 5743 5280 5279 5279
q22 557 509 521 509
Total cold run time: 63794 ms
Total hot run time: 60350 ms
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [pipelineX](fix) Fix incorrect hash distribution [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29544:
URL: https://github.com/apache/doris/pull/29544#issuecomment-1876927542
TPC-DS test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G', run with scripts in https://github.com/apache/doris/tree/master/tools/tpcds-tools
```
TPC-DS sf100 test result on commit 20c1872a0da95dbee5cb407c3078f24909839444, data reload: false
run tpcds-sf100 query with default conf and session variables
query1 921 342 332 332
query2 6426 1879 1984 1879
query3 6643 207 202 202
query4 26562 22461 22432 22432
query5 3882 498 519 498
query6 272 185 192 185
query7 4588 274 264 264
query8 225 194 194 194
query9 8229 2588 2600 2588
query10 424 278 227 227
query11 16305 15511 15578 15511
query12 133 79 76 76
query13 1645 340 317 317
query14 11559 7145 7000 7000
query15 222 184 192 184
query16 6393 269 263 263
query17 1877 501 490 490
query18 1958 270 261 261
query19 193 135 132 132
query20 81 71 78 71
query21 180 100 101 100
query22 4909 4607 4799 4607
query23 31983 31425 31084 31084
query24 12056 2834 2795 2795
query25 585 352 340 340
query26 1756 143 144 143
query27 2900 271 279 271
query28 7220 1861 1873 1861
query29 2084 403 405 403
query30 288 140 146 140
query31 964 753 754 753
query32 95 59 58 58
query33 724 271 270 270
query34 893 453 443 443
query35 870 781 797 781
query36 1252 1190 1195 1190
query37 182 69 67 67
query38 3372 3302 3245 3245
query39 1321 1274 1260 1260
query40 301 95 85 85
query41 36 34 34 34
query42 85 82 77 77
query43 496 523 491 491
query44 1050 685 699 685
query45 195 183 186 183
query46 1068 623 652 623
query47 1624 1494 1586 1494
query48 334 257 255 255
query49 1209 318 306 306
query50 782 326 364 326
query51 5365 5308 5311 5308
query52 95 86 85 85
query53 216 155 147 147
query54 1374 564 563 563
query55 93 84 79 79
query56 192 188 193 188
query57 1039 909 928 909
query58 220 199 200 199
query59 2747 2578 2594 2578
query60 226 206 222 206
query61 89 87 87 87
query62 621 422 457 422
query63 171 147 145 145
query64 5909 1737 1712 1712
query65 3334 3288 3292 3288
query66 1384 351 342 342
query67 15704 15330 15129 15129
query68 12210 503 503 503
query69 521 242 240 240
query70 1692 1515 1417 1417
query71 506 216 191 191
query72 5710 3593 3626 3593
query73 2796 310 304 304
query74 7653 6488 6483 6483
query75 5038 2270 2254 2254
query76 6296 1134 1120 1120
query77 669 241 253 241
query78 9057 8720 8813 8720
query79 1025 498 503 498
query80 538 347 350 347
query81 450 208 207 207
query82 209 100 91 91
query83 163 134 134 134
query84 246 52 51 51
query85 930 280 278 278
query86 512 391 394 391
query87 3526 3357 3343 3343
query88 2994 2229 2230 2229
query89 321 258 258 258
query90 1899 192 191 191
query91 115 91 96 91
query92 55 50 51 50
query93 1407 484 473 473
query94 794 174 175 174
query95 457 417 398 398
query96 620 321 310 310
query97 4266 4124 4133 4124
query98 222 203 186 186
query99 1142 852 832 832
Total cold run time: 292170 ms
Total hot run time: 178585 ms
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [pipelineX](fix) Fix incorrect hash distribution [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29544:
URL: https://github.com/apache/doris/pull/29544#issuecomment-1876931280
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 49.76 seconds
stream load tsv: 571 seconds loaded 74807831229 Bytes, about 124 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 33 seconds loaded 861443392 Bytes, about 24 MB/s
insert into select: 28.6 seconds inserted 10000000 Rows, about 349K ops/s
storage size: 17184135799 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [pipelineX](fix) Fix incorrect hash distribution [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29544:
URL: https://github.com/apache/doris/pull/29544#issuecomment-1876959807
TeamCity be ut coverage result:
Function Coverage: 36.64% (8616/23517)
Line Coverage: 28.68% (70015/244165)
Region Coverage: 27.65% (36238/131067)
Branch Coverage: 24.35% (18511/76036)
Coverage Report: http://coverage.selectdb-in.cc/coverage/20c1872a0da95dbee5cb407c3078f24909839444_20c1872a0da95dbee5cb407c3078f24909839444/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [pipelineX](fix) Fix incorrect hash distribution [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29544:
URL: https://github.com/apache/doris/pull/29544#issuecomment-1876913710
TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G', run with scripts in https://github.com/apache/doris/tree/master/tools/tpch-tools
```
Tpch sf100 test result on commit 20c1872a0da95dbee5cb407c3078f24909839444, data reload: false
------ Round 1 ----------------------------------
q1 17673 5342 5140 5140
q2 2018 156 144 144
q3 10535 1125 1106 1106
q4 10196 810 873 810
q5 7798 2964 2913 2913
q6 214 136 136 136
q7 928 521 554 521
q8 9292 2051 2053 2051
q9 6848 6519 6374 6374
q10 8285 3026 3065 3026
q11 417 223 207 207
q12 388 227 241 227
q13 18012 3670 3724 3670
q14 254 221 207 207
q15 569 529 521 521
q16 452 400 418 400
q17 975 471 477 471
q18 7392 6760 6684 6684
q19 1581 1404 1334 1334
q20 692 347 323 323
q21 2865 2430 2408 2408
q22 374 325 355 325
Total cold run time: 107758 ms
Total hot run time: 38998 ms
----- Round 2, with runtime_filter_mode=off -----
q1 5210 5152 5128 5128
q2 338 242 231 231
q3 3310 3305 3275 3275
q4 2103 1985 2007 1985
q5 5846 5825 5794 5794
q6 215 126 130 126
q7 2340 1961 1926 1926
q8 3409 3468 3471 3468
q9 8826 8874 8824 8824
q10 3822 3860 3825 3825
q11 581 490 484 484
q12 803 639 640 639
q13 8974 3228 3221 3221
q14 295 261 262 261
q15 578 527 520 520
q16 548 487 511 487
q17 1956 1784 1813 1784
q18 8741 8256 8369 8256
q19 1632 1597 1573 1573
q20 2202 1988 1934 1934
q21 5764 5375 5248 5248
q22 534 490 475 475
Total cold run time: 68027 ms
Total hot run time: 59464 ms
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [pipelineX](fix) Fix incorrect hash distribution [doris]
Posted by "Gabriel39 (via GitHub)" <gi...@apache.org>.
Gabriel39 commented on PR #29544:
URL: https://github.com/apache/doris/pull/29544#issuecomment-1876843466
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org