You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "Gabriel39 (via GitHub)" <gi...@apache.org> on 2024/01/12 07:13:27 UTC
[PR] [pipelineX](improvement) Adjust local exchange strategy [doris]
Gabriel39 opened a new pull request, #29915:
URL: https://github.com/apache/doris/pull/29915
## Proposed changes
Issue Number: close #xxx
<!--Describe your changes.-->
## Further comments
If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [pipelineX](improvement) Adjust local exchange strategy [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29915:
URL: https://github.com/apache/doris/pull/29915#issuecomment-1888633013
<details>
<summary>TPC-H: <b>Total hot run time: 39186 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 1c9d5a5647c5621ec9891a1742bd823cfc647327, data reload: false
------ Round 1 ----------------------------------
q1 17908 5017 4973 4973
q2 2041 136 138 136
q3 10687 1104 1161 1104
q4 10514 780 821 780
q5 7956 3247 3252 3247
q6 211 129 127 127
q7 909 516 507 507
q8 9767 2035 2057 2035
q9 7844 6651 6595 6595
q10 8269 3120 3106 3106
q11 426 213 205 205
q12 366 191 192 191
q13 18091 3439 3439 3439
q14 248 216 216 216
q15 552 514 511 511
q16 431 377 377 377
q17 948 566 518 518
q18 7342 6854 6753 6753
q19 1608 1439 1404 1404
q20 475 293 274 274
q21 2817 2389 2427 2389
q22 344 303 299 299
Total cold run time: 109754 ms
Total hot run time: 39186 ms
----- Round 2, with runtime_filter_mode=off -----
q1 5015 4955 4900 4900
q2 311 200 201 200
q3 3341 3329 3314 3314
q4 2243 2260 2264 2260
q5 5824 5806 5806 5806
q6 191 119 119 119
q7 2305 1844 1873 1844
q8 3471 3563 3589 3563
q9 8865 8807 8812 8807
q10 3743 3824 3826 3824
q11 553 414 429 414
q12 805 608 599 599
q13 3882 3216 3199 3199
q14 293 254 265 254
q15 570 522 517 517
q16 513 473 456 456
q17 2057 2030 2011 2011
q18 8729 8319 8380 8319
q19 1616 1637 1612 1612
q20 2156 1945 1916 1916
q21 6072 5739 5689 5689
q22 546 475 443 443
Total cold run time: 63101 ms
Total hot run time: 60066 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [pipelineX](improvement) Adjust local exchange strategy [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29915:
URL: https://github.com/apache/doris/pull/29915#issuecomment-1889398323
<details>
<summary>TPC-DS: <b>Total hot run time: 178938 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit ea37431e31c4fecb46e65cede7e3063e3943f0bd, data reload: false
query1 2374 352 340 340
query2 6794 1877 1869 1869
query3 6712 208 208 208
query4 26356 22436 22148 22148
query5 8928 592 555 555
query6 279 181 184 181
query7 4632 282 275 275
query8 230 210 206 206
query9 9164 2745 2825 2745
query10 473 247 246 246
query11 16167 15281 15331 15281
query12 135 76 77 76
query13 1758 392 406 392
query14 12742 7377 7151 7151
query15 237 196 198 196
query16 6297 278 245 245
query17 1911 506 494 494
query18 1967 278 264 264
query19 285 152 148 148
query20 81 76 74 74
query21 192 125 131 125
query22 5027 4837 4739 4739
query23 31985 31131 31010 31010
query24 12058 2781 2811 2781
query25 564 331 330 330
query26 1718 154 155 154
query27 2890 289 283 283
query28 6985 1870 1860 1860
query29 2082 681 688 681
query30 284 137 138 137
query31 960 765 771 765
query32 90 64 58 58
query33 716 244 257 244
query34 1112 468 473 468
query35 902 763 759 759
query36 1422 1281 1322 1281
query37 190 72 68 68
query38 3441 3322 3283 3283
query39 1345 1309 1288 1288
query40 321 104 96 96
query41 40 34 35 34
query42 107 96 95 95
query43 548 477 484 477
query44 1122 706 718 706
query45 214 187 187 187
query46 1047 667 661 661
query47 1649 1483 1498 1483
query48 408 327 325 325
query49 1280 315 315 315
query50 708 314 316 314
query51 5288 5220 5244 5220
query52 101 93 93 93
query53 363 291 295 291
query54 923 460 464 460
query55 91 86 84 84
query56 202 201 204 201
query57 980 948 963 948
query58 223 201 194 194
query59 2740 2505 2521 2505
query60 232 236 228 228
query61 88 87 91 87
query62 652 387 416 387
query63 308 273 286 273
query64 5783 1674 1703 1674
query65 3355 3276 3279 3276
query66 1227 338 344 338
query67 15583 15287 15162 15162
query68 12686 537 526 526
query69 709 391 378 378
query70 1765 1515 1579 1515
query71 576 238 253 238
query72 5009 2891 2936 2891
query73 2815 319 319 319
query74 6972 6411 6379 6379
query75 5092 2316 2341 2316
query76 6349 1122 1097 1097
query77 663 256 285 256
query78 9921 8595 8706 8595
query79 1076 510 495 495
query80 546 363 356 356
query81 465 209 204 204
query82 206 90 88 88
query83 167 124 126 124
query84 283 75 77 75
query85 1072 356 341 341
query86 430 369 372 369
query87 3542 3400 3392 3392
query88 3140 2207 2216 2207
query89 451 410 375 375
query90 1964 205 207 205
query91 170 130 129 129
query92 67 57 56 56
query93 1220 452 423 423
query94 1188 196 180 180
query95 530 494 484 484
query96 620 317 318 317
query97 4265 4204 4162 4162
query98 212 199 187 187
query99 1054 737 743 737
Total cold run time: 302293 ms
Total hot run time: 178938 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [pipelineX](improvement) Adjust local exchange strategy [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #29915:
URL: https://github.com/apache/doris/pull/29915#issuecomment-1888677821
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [pipelineX](improvement) Adjust local exchange strategy [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #29915:
URL: https://github.com/apache/doris/pull/29915#issuecomment-1888722462
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [pipelineX](improvement) Adjust local exchange strategy [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29915:
URL: https://github.com/apache/doris/pull/29915#issuecomment-1888745607
<details>
<summary>TPC-H: <b>Total hot run time: 39402 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 2f51d58a45e57a9f4700a201d0bf2df08975991f, data reload: false
------ Round 1 ----------------------------------
q1 17936 5056 5006 5006
q2 2050 141 134 134
q3 10760 1179 1227 1179
q4 10558 786 829 786
q5 8604 3257 3262 3257
q6 207 126 131 126
q7 912 530 528 528
q8 9958 1996 2014 1996
q9 8068 6726 6632 6632
q10 8265 3136 3106 3106
q11 415 202 219 202
q12 363 203 207 203
q13 18120 3449 3415 3415
q14 240 218 215 215
q15 571 525 523 523
q16 435 387 376 376
q17 970 549 511 511
q18 7538 6786 6757 6757
q19 1609 1382 1487 1382
q20 511 319 308 308
q21 2856 2463 2482 2463
q22 355 309 297 297
Total cold run time: 111301 ms
Total hot run time: 39402 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4990 5055 4977 4977
q2 319 201 200 200
q3 3382 3351 3336 3336
q4 2315 2288 2279 2279
q5 5866 5864 5851 5851
q6 196 121 120 120
q7 2348 1888 1892 1888
q8 3590 3695 3716 3695
q9 9104 9119 9033 9033
q10 3968 4073 4046 4046
q11 570 486 499 486
q12 882 661 656 656
q13 3962 3198 3236 3198
q14 290 258 259 258
q15 584 515 515 515
q16 531 470 480 470
q17 2148 2320 2014 2014
q18 10757 8327 8358 8327
q19 1632 1657 1611 1611
q20 2212 1943 1918 1918
q21 6100 5666 5845 5666
q22 563 477 473 473
Total cold run time: 66309 ms
Total hot run time: 61017 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [pipelineX](improvement) Adjust local exchange strategy [doris]
Posted by "Gabriel39 (via GitHub)" <gi...@apache.org>.
Gabriel39 commented on PR #29915:
URL: https://github.com/apache/doris/pull/29915#issuecomment-1888713829
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [pipelineX](improvement) Adjust local exchange strategy [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29915:
URL: https://github.com/apache/doris/pull/29915#issuecomment-1888637221
TeamCity be ut coverage result:
Function Coverage: 36.71% (8650/23566)
Line Coverage: 28.72% (70444/245294)
Region Coverage: 27.63% (36397/131722)
Branch Coverage: 24.35% (18623/76480)
Coverage Report: http://coverage.selectdb-in.cc/coverage/1c9d5a5647c5621ec9891a1742bd823cfc647327_1c9d5a5647c5621ec9891a1742bd823cfc647327/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [pipelineX](improvement) Adjust local exchange strategy [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29915:
URL: https://github.com/apache/doris/pull/29915#issuecomment-1888659021
Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
```
Load test result on commit 1c9d5a5647c5621ec9891a1742bd823cfc647327 with default session variables
Stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc: 58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
Insert into select: 12.2 seconds inserted 10000000 Rows, about 819K ops/s
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [pipelineX](improvement) Adjust local exchange strategy [doris]
Posted by "Gabriel39 (via GitHub)" <gi...@apache.org>.
Gabriel39 commented on PR #29915:
URL: https://github.com/apache/doris/pull/29915#issuecomment-1889269266
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [pipelineX](improvement) Adjust local exchange strategy [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #29915:
URL: https://github.com/apache/doris/pull/29915#issuecomment-1888614446
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [pipelineX](improvement) Adjust local exchange strategy [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29915:
URL: https://github.com/apache/doris/pull/29915#issuecomment-1889476518
TeamCity be ut coverage result:
Function Coverage: 36.70% (8648/23567)
Line Coverage: 28.71% (70424/245263)
Region Coverage: 27.63% (36402/131731)
Branch Coverage: 24.35% (18624/76470)
Coverage Report: http://coverage.selectdb-in.cc/coverage/ea37431e31c4fecb46e65cede7e3063e3943f0bd_ea37431e31c4fecb46e65cede7e3063e3943f0bd/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [pipelineX](improvement) Adjust local exchange strategy [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29915:
URL: https://github.com/apache/doris/pull/29915#issuecomment-1888933674
<details>
<summary>TPC-DS: <b>Total hot run time: 177993 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit f3a833d79a5e6c5367e1c24df1e664895e6d0b30, data reload: false
query1 941 338 333 333
query2 6707 1882 1989 1882
query3 6717 207 206 206
query4 26006 22146 22143 22143
query5 8217 538 575 538
query6 267 194 198 194
query7 4629 282 279 279
query8 233 193 197 193
query9 9157 2758 2779 2758
query10 555 242 242 242
query11 16135 15349 15332 15332
query12 136 75 74 74
query13 1712 388 387 387
query14 12049 7213 7339 7213
query15 245 181 190 181
query16 5261 251 244 244
query17 1893 489 478 478
query18 1956 269 261 261
query19 288 152 154 152
query20 79 77 74 74
query21 203 131 122 122
query22 5005 4828 4703 4703
query23 31670 30761 30812 30761
query24 12281 2859 2769 2769
query25 572 322 325 322
query26 1790 159 154 154
query27 2905 293 284 284
query28 7056 1868 1856 1856
query29 2099 709 650 650
query30 284 138 147 138
query31 972 798 776 776
query32 93 60 62 60
query33 719 250 248 248
query34 1112 455 469 455
query35 940 744 719 719
query36 1410 1241 1308 1241
query37 187 68 69 68
query38 3377 3254 3283 3254
query39 1349 1307 1269 1269
query40 340 102 94 94
query41 41 36 35 35
query42 105 97 95 95
query43 537 506 487 487
query44 1081 715 715 715
query45 204 196 185 185
query46 1044 661 674 661
query47 1686 1480 1600 1480
query48 398 320 324 320
query49 1266 335 324 324
query50 720 314 307 307
query51 5407 5255 5294 5255
query52 104 83 101 83
query53 365 308 294 294
query54 917 459 465 459
query55 99 87 89 87
query56 214 206 194 194
query57 994 977 909 909
query58 207 193 194 193
query59 2814 2576 2542 2542
query60 244 238 230 230
query61 83 83 85 83
query62 653 374 418 374
query63 310 281 274 274
query64 5891 1653 1669 1653
query65 3320 3266 3267 3266
query66 1365 337 328 328
query67 15723 15408 14967 14967
query68 11023 536 535 535
query69 717 381 381 381
query70 1598 1547 1497 1497
query71 564 239 238 238
query72 4911 2808 2785 2785
query73 2109 316 316 316
query74 6968 6395 6461 6395
query75 4784 2280 2272 2272
query76 6336 1070 1113 1070
query77 664 291 268 268
query78 9757 8513 8630 8513
query79 1034 498 497 497
query80 513 342 359 342
query81 451 217 206 206
query82 209 92 90 90
query83 148 120 128 120
query84 278 75 73 73
query85 1006 333 339 333
query86 363 365 372 365
query87 3571 3421 3377 3377
query88 3145 2210 2217 2210
query89 455 383 394 383
query90 1933 221 197 197
query91 162 140 128 128
query92 64 55 54 54
query93 983 420 419 419
query94 1078 173 187 173
query95 535 465 471 465
query96 637 323 323 323
query97 4276 4160 4194 4160
query98 209 195 192 192
query99 974 709 778 709
Total cold run time: 294794 ms
Total hot run time: 177993 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [pipelineX](improvement) Adjust local exchange strategy [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #29915:
URL: https://github.com/apache/doris/pull/29915#issuecomment-1888722496
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [pipelineX](improvement) Adjust local exchange strategy [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #29915:
URL: https://github.com/apache/doris/pull/29915#issuecomment-1888555553
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [pipelineX](improvement) Adjust local exchange strategy [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29915:
URL: https://github.com/apache/doris/pull/29915#issuecomment-1888601764
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 47.23 seconds
stream load tsv: 559 seconds loaded 74807831229 Bytes, about 127 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 21.5 seconds inserted 10000000 Rows, about 465K ops/s
storage size: 17183905558 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [pipelineX](improvement) Adjust local exchange strategy [doris]
Posted by "Gabriel39 (via GitHub)" <gi...@apache.org>.
Gabriel39 commented on PR #29915:
URL: https://github.com/apache/doris/pull/29915#issuecomment-1888546746
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [pipelineX](improvement) Adjust local exchange strategy [doris]
Posted by "Gabriel39 (via GitHub)" <gi...@apache.org>.
Gabriel39 commented on PR #29915:
URL: https://github.com/apache/doris/pull/29915#issuecomment-1888875532
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [pipelineX](improvement) Adjust local exchange strategy [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29915:
URL: https://github.com/apache/doris/pull/29915#issuecomment-1889367131
<details>
<summary>TPC-H: <b>Total hot run time: 39439 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit ea37431e31c4fecb46e65cede7e3063e3943f0bd, data reload: false
------ Round 1 ----------------------------------
q1 17961 5239 5002 5002
q2 2040 142 133 133
q3 10931 1150 1215 1150
q4 11127 795 867 795
q5 8184 3308 3294 3294
q6 213 131 133 131
q7 923 524 527 524
q8 9778 2058 2054 2054
q9 8724 6698 6663 6663
q10 8930 3096 3122 3096
q11 436 214 207 207
q12 371 197 200 197
q13 18149 3418 3410 3410
q14 240 212 214 212
q15 557 513 526 513
q16 492 379 389 379
q17 976 510 496 496
q18 7060 6717 6716 6716
q19 1596 1463 1462 1462
q20 507 300 301 300
q21 2857 2412 2463 2412
q22 350 306 293 293
Total cold run time: 112402 ms
Total hot run time: 39439 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4967 5046 4995 4995
q2 315 206 199 199
q3 3346 3320 3315 3315
q4 2254 2255 2253 2253
q5 5842 5821 5829 5821
q6 197 118 118 118
q7 2340 1866 1917 1866
q8 3501 3590 3596 3590
q9 8983 8998 8924 8924
q10 3819 3871 3898 3871
q11 557 418 447 418
q12 803 612 634 612
q13 3914 3264 3254 3254
q14 290 269 266 266
q15 573 514 513 513
q16 515 477 463 463
q17 2070 2041 2108 2041
q18 8853 8469 8368 8368
q19 1632 1651 1673 1651
q20 2187 2038 2006 2006
q21 6365 5932 6066 5932
q22 626 533 525 525
Total cold run time: 63949 ms
Total hot run time: 61001 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [pipelineX](improvement) Adjust local exchange strategy [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29915:
URL: https://github.com/apache/doris/pull/29915#issuecomment-1888765755
TeamCity be ut coverage result:
Function Coverage: 36.71% (8649/23563)
Line Coverage: 28.71% (70408/245238)
Region Coverage: 27.64% (36402/131720)
Branch Coverage: 24.34% (18615/76468)
Coverage Report: http://coverage.selectdb-in.cc/coverage/2f51d58a45e57a9f4700a201d0bf2df08975991f_2f51d58a45e57a9f4700a201d0bf2df08975991f/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [pipelineX](improvement) Adjust local exchange strategy [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29915:
URL: https://github.com/apache/doris/pull/29915#issuecomment-1888930479
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 46.1 seconds
stream load tsv: 558 seconds loaded 74807831229 Bytes, about 127 MB/s
stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s
stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 21.1 seconds inserted 10000000 Rows, about 473K ops/s
storage size: 17183829845 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [pipelineX](improvement) Adjust local exchange strategy [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #29915:
URL: https://github.com/apache/doris/pull/29915#issuecomment-1889288333
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [pipelineX](improvement) Adjust local exchange strategy [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29915:
URL: https://github.com/apache/doris/pull/29915#issuecomment-1888647968
<details>
<summary>TPC-DS: <b>Total hot run time: 178018 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 1c9d5a5647c5621ec9891a1742bd823cfc647327, data reload: false
query1 925 334 329 329
query2 6719 1885 1820 1820
query3 6723 212 209 209
query4 26220 22168 22098 22098
query5 8183 535 586 535
query6 276 185 197 185
query7 4619 283 267 267
query8 243 206 200 200
query9 9219 2792 2767 2767
query10 650 250 252 250
query11 16066 15376 15348 15348
query12 126 76 73 73
query13 1699 396 394 394
query14 11762 7128 7198 7128
query15 248 189 196 189
query16 5288 259 245 245
query17 1869 482 481 481
query18 1955 261 260 260
query19 285 146 149 146
query20 80 79 70 70
query21 191 137 125 125
query22 5073 4811 4730 4730
query23 31828 30801 30738 30738
query24 12284 2813 2799 2799
query25 559 328 321 321
query26 1733 146 148 146
query27 2975 285 279 279
query28 7304 1914 1892 1892
query29 2104 703 651 651
query30 288 143 140 140
query31 977 752 777 752
query32 92 61 60 60
query33 726 236 240 236
query34 1157 469 470 469
query35 897 739 763 739
query36 1367 1249 1262 1249
query37 186 69 70 69
query38 3445 3261 3299 3261
query39 1378 1287 1280 1280
query40 337 95 100 95
query41 37 34 34 34
query42 96 97 95 95
query43 536 516 506 506
query44 1061 708 717 708
query45 210 186 177 177
query46 1045 673 638 638
query47 1623 1555 1526 1526
query48 407 330 331 330
query49 1268 322 302 302
query50 701 321 314 314
query51 5359 5176 5240 5176
query52 107 92 99 92
query53 355 287 276 276
query54 902 456 475 456
query55 91 85 87 85
query56 213 197 193 193
query57 958 954 931 931
query58 212 196 197 196
query59 2747 2501 2576 2501
query60 244 216 214 214
query61 86 84 86 84
query62 646 392 387 387
query63 317 284 290 284
query64 5828 1685 1678 1678
query65 3340 3258 3266 3258
query66 1357 330 319 319
query67 15570 15179 15154 15154
query68 12767 503 493 493
query69 688 371 374 371
query70 1675 1497 1505 1497
query71 554 243 241 241
query72 4865 2853 2851 2851
query73 2972 312 318 312
query74 7113 6440 6480 6440
query75 5345 2337 2278 2278
query76 6374 1074 1077 1074
query77 670 280 256 256
query78 9886 8676 8581 8581
query79 1034 500 485 485
query80 565 333 342 333
query81 453 205 209 205
query82 206 87 89 87
query83 136 117 129 117
query84 280 75 73 73
query85 1019 345 337 337
query86 368 368 360 360
query87 3607 3405 3377 3377
query88 3119 2185 2198 2185
query89 463 377 394 377
query90 1861 215 202 202
query91 174 132 127 127
query92 63 57 51 51
query93 1332 430 399 399
query94 1226 192 183 183
query95 510 476 467 467
query96 610 309 316 309
query97 4261 4131 4133 4131
query98 204 191 190 190
query99 982 690 694 690
Total cold run time: 298754 ms
Total hot run time: 178018 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [pipelineX](improvement) Adjust local exchange strategy [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29915:
URL: https://github.com/apache/doris/pull/29915#issuecomment-1889401829
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 46.5 seconds
stream load tsv: 586 seconds loaded 74807831229 Bytes, about 121 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 21.0 seconds inserted 10000000 Rows, about 476K ops/s
storage size: 17184138158 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [pipelineX](improvement) Adjust local exchange strategy [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #29915:
URL: https://github.com/apache/doris/pull/29915#issuecomment-1888727109
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [pipelineX](improvement) Adjust local exchange strategy [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29915:
URL: https://github.com/apache/doris/pull/29915#issuecomment-1888649039
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 46.52 seconds
stream load tsv: 558 seconds loaded 74807831229 Bytes, about 127 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 21.0 seconds inserted 10000000 Rows, about 476K ops/s
storage size: 17183963553 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [pipelineX](improvement) Adjust local exchange strategy [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29915:
URL: https://github.com/apache/doris/pull/29915#issuecomment-1888917381
<details>
<summary>TPC-H: <b>Total hot run time: 38539 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit f3a833d79a5e6c5367e1c24df1e664895e6d0b30, data reload: false
------ Round 1 ----------------------------------
q1 17939 4964 4971 4964
q2 2037 145 132 132
q3 10697 1091 1191 1091
q4 10253 802 841 802
q5 7841 3140 3141 3140
q6 204 122 125 122
q7 881 502 491 491
q8 9291 1941 1987 1941
q9 7320 6428 6399 6399
q10 8257 3060 3058 3058
q11 404 224 202 202
q12 360 190 194 190
q13 18126 3402 3388 3388
q14 249 212 216 212
q15 563 514 505 505
q16 425 382 381 381
q17 939 505 538 505
q18 7272 6661 6687 6661
q19 1566 1379 1421 1379
q20 528 301 281 281
q21 2826 2394 2413 2394
q22 342 301 312 301
Total cold run time: 108320 ms
Total hot run time: 38539 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4921 4942 4903 4903
q2 311 204 206 204
q3 3329 3319 3295 3295
q4 2252 2244 2248 2244
q5 5800 5802 5807 5802
q6 191 117 122 117
q7 2312 1886 1858 1858
q8 3419 3542 3543 3542
q9 8850 8781 8798 8781
q10 3748 3809 3841 3809
q11 542 425 449 425
q12 789 607 614 607
q13 4796 3225 3225 3225
q14 283 259 268 259
q15 573 514 508 508
q16 500 464 471 464
q17 2027 2017 2027 2017
q18 8789 8420 8337 8337
q19 1609 1608 1610 1608
q20 2161 1977 1942 1942
q21 6058 5648 5784 5648
q22 577 461 479 461
Total cold run time: 63837 ms
Total hot run time: 60056 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [pipelineX](improvement) Adjust local exchange strategy [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29915:
URL: https://github.com/apache/doris/pull/29915#issuecomment-1888940969
<details>
<summary>ClickBench: <b>Total hot run time: 30.41 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit f3a833d79a5e6c5367e1c24df1e664895e6d0b30, data reload: false
query1 0.07 0.06 0.05
query2 0.06 0.02 0.02
query3 0.24 0.11 0.11
query4 1.75 0.12 0.11
query5 0.52 0.51 0.52
query6 1.39 0.64 0.64
query7 0.02 0.02 0.01
query8 0.03 0.02 0.02
query9 0.54 0.50 0.51
query10 0.55 0.55 0.55
query11 0.12 0.10 0.09
query12 0.12 0.09 0.09
query13 0.60 0.60 0.60
query14 0.79 0.79 0.77
query15 0.82 0.81 0.80
query16 0.34 0.36 0.35
query17 0.97 0.99 1.01
query18 0.26 0.24 0.25
query19 1.85 1.79 1.74
query20 0.01 0.02 0.02
query21 15.43 0.56 0.54
query22 2.41 2.25 1.72
query23 17.06 0.88 0.79
query24 16.16 0.58 0.61
query25 2.17 0.15 0.14
query26 0.14 0.14 0.13
query27 0.15 0.15 0.14
query28 6.77 0.82 0.81
query29 12.59 3.20 3.26
query30 0.56 0.47 0.49
query31 2.78 0.35 0.35
query32 3.36 0.48 0.48
query33 3.18 3.22 3.25
query34 16.18 4.16 4.16
query35 4.16 4.22 4.15
query36 1.10 1.04 1.05
query37 0.06 0.05 0.05
query38 0.03 0.03 0.03
query39 0.03 0.02 0.01
query40 0.16 0.13 0.14
query41 0.07 0.02 0.02
query42 0.02 0.02 0.02
query43 0.02 0.02 0.02
Total cold run time: 115.64 s
Total hot run time: 30.41 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [pipelineX](improvement) Adjust local exchange strategy [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29915:
URL: https://github.com/apache/doris/pull/29915#issuecomment-1889413410
<details>
<summary>ClickBench: <b>Total hot run time: 30.44 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit ea37431e31c4fecb46e65cede7e3063e3943f0bd, data reload: false
query1 0.06 0.06 0.05
query2 0.06 0.02 0.02
query3 0.26 0.12 0.11
query4 1.74 0.13 0.12
query5 0.52 0.52 0.52
query6 1.38 0.67 0.64
query7 0.02 0.01 0.02
query8 0.04 0.03 0.03
query9 0.55 0.49 0.51
query10 0.53 0.54 0.54
query11 0.13 0.10 0.08
query12 0.12 0.10 0.09
query13 0.61 0.60 0.60
query14 0.77 0.81 0.79
query15 0.81 0.80 0.80
query16 0.35 0.36 0.36
query17 0.96 0.95 0.97
query18 0.26 0.24 0.25
query19 1.79 1.76 1.78
query20 0.02 0.01 0.01
query21 15.40 0.57 0.58
query22 2.48 2.73 1.79
query23 17.35 0.85 0.81
query24 16.28 0.59 0.59
query25 2.17 0.15 0.15
query26 0.14 0.13 0.13
query27 0.15 0.15 0.15
query28 5.74 0.81 0.80
query29 12.79 3.16 3.09
query30 0.55 0.47 0.53
query31 2.79 0.35 0.36
query32 3.36 0.49 0.48
query33 3.22 3.24 3.25
query34 15.83 4.20 4.19
query35 4.16 4.14 4.14
query36 1.11 1.06 1.04
query37 0.07 0.05 0.05
query38 0.03 0.02 0.02
query39 0.03 0.02 0.01
query40 0.17 0.13 0.13
query41 0.07 0.02 0.01
query42 0.03 0.01 0.01
query43 0.02 0.02 0.01
Total cold run time: 114.92 s
Total hot run time: 30.44 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [pipelineX](improvement) Adjust local exchange strategy [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29915:
URL: https://github.com/apache/doris/pull/29915#issuecomment-1888567970
<details>
<summary>TPC-H: <b>Total hot run time: 39722 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 8faeac1580beb4957fcc4da655fb6e72b0bc1cb7, data reload: false
------ Round 1 ----------------------------------
q1 18875 5065 5079 5065
q2 2185 154 145 145
q3 10639 1140 1259 1140
q4 10254 817 872 817
q5 7820 3216 3234 3216
q6 206 129 128 128
q7 993 586 567 567
q8 9431 2019 2082 2019
q9 7587 6739 6701 6701
q10 8856 3776 3754 3754
q11 419 227 233 227
q12 369 197 197 197
q13 18100 3171 3172 3171
q14 248 219 212 212
q15 565 531 506 506
q16 449 380 387 380
q17 966 584 474 474
q18 7372 6705 6637 6637
q19 1598 1297 1449 1297
q20 568 317 286 286
q21 2905 2518 2462 2462
q22 387 326 321 321
Total cold run time: 110792 ms
Total hot run time: 39722 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4874 4921 4902 4902
q2 334 205 206 205
q3 3335 3307 3307 3307
q4 2269 2268 2264 2264
q5 5848 5831 5840 5831
q6 197 120 121 120
q7 2432 1980 1953 1953
q8 3568 3696 3751 3696
q9 9222 9226 9229 9226
q10 4470 4554 4496 4496
q11 564 463 457 457
q12 791 604 611 604
q13 8794 2966 2932 2932
q14 298 254 270 254
q15 566 523 513 513
q16 483 421 422 421
q17 2078 1925 1869 1869
q18 8808 8276 8338 8276
q19 1621 1573 1575 1573
q20 2245 1988 1956 1956
q21 6040 5637 5737 5637
q22 560 457 442 442
Total cold run time: 69397 ms
Total hot run time: 60934 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [pipelineX](improvement) Adjust local exchange strategy [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29915:
URL: https://github.com/apache/doris/pull/29915#issuecomment-1888788056
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 46.56 seconds
stream load tsv: 559 seconds loaded 74807831229 Bytes, about 127 MB/s
stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s
stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 22.9 seconds inserted 10000000 Rows, about 436K ops/s
storage size: 17183838617 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [pipelineX](improvement) Adjust local exchange strategy [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29915:
URL: https://github.com/apache/doris/pull/29915#issuecomment-1888904431
TeamCity be ut coverage result:
Function Coverage: 36.71% (8649/23563)
Line Coverage: 28.72% (70421/245235)
Region Coverage: 27.64% (36406/131719)
Branch Coverage: 24.35% (18620/76468)
Coverage Report: http://coverage.selectdb-in.cc/coverage/f3a833d79a5e6c5367e1c24df1e664895e6d0b30_f3a833d79a5e6c5367e1c24df1e664895e6d0b30/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [pipelineX](improvement) Adjust local exchange strategy [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29915:
URL: https://github.com/apache/doris/pull/29915#issuecomment-1888762162
<details>
<summary>TPC-DS: <b>Total hot run time: 178979 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 2f51d58a45e57a9f4700a201d0bf2df08975991f, data reload: false
query1 925 333 327 327
query2 6737 1913 1899 1899
query3 6709 208 204 204
query4 26658 22205 22106 22106
query5 8242 589 560 560
query6 267 178 186 178
query7 4608 268 272 268
query8 226 203 194 194
query9 9244 2776 2779 2776
query10 572 236 237 236
query11 16178 15445 15347 15347
query12 129 74 73 73
query13 1713 377 383 377
query14 12261 7282 7209 7209
query15 246 193 189 189
query16 5280 250 248 248
query17 1891 489 483 483
query18 1968 269 261 261
query19 295 148 148 148
query20 82 79 73 73
query21 190 131 132 131
query22 5259 5098 4950 4950
query23 31736 30873 30897 30873
query24 12000 2864 2757 2757
query25 562 322 326 322
query26 1754 151 156 151
query27 2884 281 286 281
query28 6921 1878 1866 1866
query29 2115 692 671 671
query30 286 140 142 140
query31 970 761 761 761
query32 98 60 62 60
query33 714 242 248 242
query34 1078 474 470 470
query35 914 743 744 743
query36 1286 1236 1306 1236
query37 187 67 70 67
query38 3390 3298 3272 3272
query39 1327 1292 1286 1286
query40 330 99 98 98
query41 37 35 35 35
query42 103 91 99 91
query43 546 500 494 494
query44 1083 719 724 719
query45 209 184 191 184
query46 1039 662 663 662
query47 1662 1569 1568 1568
query48 403 325 314 314
query49 1271 317 316 316
query50 701 320 317 317
query51 5363 5248 5295 5248
query52 110 89 98 89
query53 363 289 290 289
query54 922 464 486 464
query55 89 88 85 85
query56 226 197 204 197
query57 1021 940 970 940
query58 218 188 200 188
query59 2721 2526 2506 2506
query60 248 207 193 193
query61 85 82 89 82
query62 656 398 402 398
query63 305 285 279 279
query64 5804 1708 1672 1672
query65 3337 3267 3258 3258
query66 1366 332 322 322
query67 15779 15622 15066 15066
query68 10180 526 546 526
query69 688 391 383 383
query70 1695 1516 1527 1516
query71 529 214 258 214
query72 4646 2859 2833 2833
query73 1930 320 314 314
query74 7105 6515 6475 6475
query75 4640 2327 2339 2327
query76 6204 1029 1106 1029
query77 644 289 294 289
query78 9816 8750 8860 8750
query79 1045 497 486 486
query80 593 336 356 336
query81 448 206 207 206
query82 231 93 93 93
query83 154 118 123 118
query84 254 77 75 75
query85 1017 341 339 339
query86 388 375 385 375
query87 3584 3407 3405 3405
query88 2865 2205 2209 2205
query89 438 371 414 371
query90 1918 208 196 196
query91 157 128 127 127
query92 69 55 53 53
query93 939 450 418 418
query94 1101 191 176 176
query95 540 475 466 466
query96 620 310 318 310
query97 4346 4183 4175 4175
query98 215 190 188 188
query99 964 696 718 696
Total cold run time: 293862 ms
Total hot run time: 178979 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [pipelineX](improvement) Adjust local exchange strategy [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29915:
URL: https://github.com/apache/doris/pull/29915#issuecomment-1889421661
Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
```
Load test result on commit ea37431e31c4fecb46e65cede7e3063e3943f0bd with default session variables
Stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc: 58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select: 12.5 seconds inserted 10000000 Rows, about 800K ops/s
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [pipelineX](improvement) Adjust local exchange strategy [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29915:
URL: https://github.com/apache/doris/pull/29915#issuecomment-1888771600
<details>
<summary>ClickBench: <b>Total hot run time: 30.5 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 2f51d58a45e57a9f4700a201d0bf2df08975991f, data reload: false
query1 0.06 0.06 0.05
query2 0.06 0.02 0.02
query3 0.24 0.12 0.11
query4 1.77 0.12 0.11
query5 0.52 0.52 0.51
query6 1.39 0.66 0.64
query7 0.02 0.01 0.01
query8 0.04 0.02 0.02
query9 0.54 0.50 0.50
query10 0.55 0.55 0.56
query11 0.13 0.10 0.09
query12 0.11 0.10 0.09
query13 0.61 0.60 0.61
query14 0.79 0.79 0.77
query15 0.83 0.81 0.79
query16 0.34 0.36 0.35
query17 0.99 0.94 0.98
query18 0.26 0.26 0.24
query19 1.84 1.75 1.81
query20 0.01 0.01 0.01
query21 15.42 0.57 0.56
query22 2.62 2.87 1.97
query23 17.37 0.86 0.78
query24 16.18 0.60 0.59
query25 2.16 0.14 0.14
query26 0.13 0.13 0.13
query27 0.14 0.14 0.14
query28 5.77 0.79 0.79
query29 12.56 3.16 3.19
query30 0.55 0.46 0.50
query31 2.78 0.35 0.37
query32 3.38 0.49 0.48
query33 3.16 3.26 3.24
query34 15.79 4.13 4.14
query35 4.19 4.14 4.16
query36 1.10 1.04 1.04
query37 0.07 0.05 0.05
query38 0.04 0.03 0.02
query39 0.02 0.02 0.01
query40 0.15 0.13 0.15
query41 0.07 0.01 0.02
query42 0.02 0.01 0.02
query43 0.03 0.02 0.02
Total cold run time: 114.8 s
Total hot run time: 30.5 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [pipelineX](improvement) Adjust local exchange strategy [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29915:
URL: https://github.com/apache/doris/pull/29915#issuecomment-1888654848
<details>
<summary>ClickBench: <b>Total hot run time: 30.57 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 1c9d5a5647c5621ec9891a1742bd823cfc647327, data reload: false
query1 0.05 0.06 0.06
query2 0.06 0.03 0.03
query3 0.25 0.11 0.11
query4 1.78 0.12 0.12
query5 0.52 0.51 0.57
query6 1.36 0.64 0.63
query7 0.02 0.01 0.01
query8 0.04 0.02 0.02
query9 0.55 0.49 0.50
query10 0.53 0.54 0.55
query11 0.13 0.09 0.10
query12 0.11 0.09 0.08
query13 0.61 0.60 0.59
query14 0.80 0.78 0.78
query15 0.82 0.81 0.81
query16 0.36 0.35 0.36
query17 0.96 0.95 0.92
query18 0.24 0.24 0.23
query19 1.87 1.74 1.76
query20 0.01 0.01 0.01
query21 15.42 0.57 0.57
query22 2.90 2.48 2.08
query23 17.09 0.71 0.80
query24 16.54 0.59 0.60
query25 2.17 0.14 0.14
query26 0.13 0.13 0.13
query27 0.14 0.14 0.14
query28 5.78 0.82 0.83
query29 12.62 3.12 3.14
query30 0.55 0.48 0.50
query31 2.78 0.35 0.36
query32 3.39 0.49 0.48
query33 3.23 3.22 3.21
query34 15.79 4.18 4.17
query35 4.18 4.17 4.19
query36 1.08 1.05 1.03
query37 0.07 0.05 0.05
query38 0.04 0.03 0.03
query39 0.02 0.02 0.02
query40 0.16 0.13 0.13
query41 0.07 0.01 0.01
query42 0.03 0.02 0.01
query43 0.03 0.02 0.01
Total cold run time: 115.28 s
Total hot run time: 30.57 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [pipelineX](improvement) Adjust local exchange strategy [doris]
Posted by "Gabriel39 (via GitHub)" <gi...@apache.org>.
Gabriel39 commented on PR #29915:
URL: https://github.com/apache/doris/pull/29915#issuecomment-1888606252
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [pipelineX](improvement) Adjust local exchange strategy [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #29915:
URL: https://github.com/apache/doris/pull/29915#issuecomment-1888885522
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [pipelineX](improvement) Adjust local exchange strategy [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #29915:
URL: https://github.com/apache/doris/pull/29915#issuecomment-1889434025
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [pipelineX](improvement) Adjust local exchange strategy [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29915:
URL: https://github.com/apache/doris/pull/29915#issuecomment-1888580161
TeamCity be ut coverage result:
Function Coverage: 36.71% (8652/23566)
Line Coverage: 28.73% (70477/245294)
Region Coverage: 27.64% (36406/131722)
Branch Coverage: 24.37% (18635/76480)
Coverage Report: http://coverage.selectdb-in.cc/coverage/8faeac1580beb4957fcc4da655fb6e72b0bc1cb7_8faeac1580beb4957fcc4da655fb6e72b0bc1cb7/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [pipelineX](improvement) Adjust local exchange strategy [doris]
Posted by "Gabriel39 (via GitHub)" <gi...@apache.org>.
Gabriel39 merged PR #29915:
URL: https://github.com/apache/doris/pull/29915
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org