You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "LiBinfeng-01 (via GitHub)" <gi...@apache.org> on 2024/01/05 14:23:49 UTC
[PR] [Feat](PipelineX) add distribute exprs to accelerate pipelinex [doris]
LiBinfeng-01 opened a new pull request, #29615:
URL: https://github.com/apache/doris/pull/29615
## Proposed changes
Issue Number: close #xxx
<!--Describe your changes.-->
## Further comments
If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feat](PipelineX) add distribute exprs to accelerate pipelinex [doris]
Posted by "LiBinfeng-01 (via GitHub)" <gi...@apache.org>.
LiBinfeng-01 closed pull request #29615: [Feat](PipelineX) add distribute exprs to accelerate pipelinex
URL: https://github.com/apache/doris/pull/29615
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feat](PipelineX) add distribute exprs to accelerate pipelinex [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29615:
URL: https://github.com/apache/doris/pull/29615#issuecomment-1878844497
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 46.28 seconds
stream load tsv: 575 seconds loaded 74807831229 Bytes, about 124 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 26.9 seconds inserted 10000000 Rows, about 371K ops/s
storage size: 17188479936 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feat](PipelineX) add distribute exprs to accelerate pipelinex [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29615:
URL: https://github.com/apache/doris/pull/29615#issuecomment-1878867549
TeamCity be ut coverage result:
Function Coverage: 36.62% (8620/23539)
Line Coverage: 28.66% (70108/244630)
Region Coverage: 27.63% (36284/131304)
Branch Coverage: 24.34% (18537/76170)
Coverage Report: http://coverage.selectdb-in.cc/coverage/497c2d0c41927dc0cc819aca4ae283bf579cbeaa_497c2d0c41927dc0cc819aca4ae283bf579cbeaa/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feat](PipelineX) add distribute exprs to accelerate pipelinex [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29615:
URL: https://github.com/apache/doris/pull/29615#issuecomment-1878840555
TPC-DS test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G', run with scripts in https://github.com/apache/doris/tree/master/tools/tpcds-tools
```
TPC-DS sf100 test result on commit 497c2d0c41927dc0cc819aca4ae283bf579cbeaa, data reload: false
run tpcds-sf100 query with default conf and session variables
query1 944 359 333 333
query2 6408 2076 1939 1939
query3 6649 207 203 203
query4 26606 22475 22400 22400
query5 3889 510 530 510
query6 264 184 188 184
query7 4580 277 259 259
query8 229 200 192 192
query9 8297 2575 2602 2575
query10 424 255 250 250
query11 16219 15530 15482 15482
query12 132 76 78 76
query13 1632 332 317 317
query14 11397 7217 7186 7186
query15 216 182 189 182
query16 6467 280 260 260
query17 1868 494 497 494
query18 1962 286 260 260
query19 195 136 129 129
query20 82 74 76 74
query21 198 98 95 95
query22 4863 4589 4407 4407
query23 31944 31449 31207 31207
query24 12094 2815 2812 2812
query25 571 342 332 332
query26 1737 143 146 143
query27 2882 270 277 270
query28 7103 1909 1922 1909
query29 2058 372 383 372
query30 286 147 145 145
query31 978 755 758 755
query32 91 53 56 53
query33 734 255 270 255
query34 900 449 437 437
query35 888 750 793 750
query36 1304 1243 1193 1193
query37 185 73 71 71
query38 3385 3273 3270 3270
query39 1341 1284 1269 1269
query40 302 93 83 83
query41 38 39 35 35
query42 94 82 83 82
query43 552 487 523 487
query44 1048 696 699 696
query45 196 195 186 186
query46 1074 636 633 633
query47 1743 1534 1498 1498
query48 331 257 255 255
query49 1223 306 306 306
query50 803 363 327 327
query51 5399 5252 5232 5232
query52 95 79 91 79
query53 209 157 145 145
query54 1419 561 565 561
query55 92 85 88 85
query56 194 195 195 195
query57 1066 968 958 958
query58 225 190 197 190
query59 2895 2611 2755 2611
query60 237 217 210 210
query61 85 84 86 84
query62 661 460 430 430
query63 173 145 146 145
query64 5882 1662 1743 1662
query65 3334 3279 3270 3270
query66 1386 349 334 334
query67 15939 15383 15268 15268
query68 9752 527 532 527
query69 501 254 244 244
query70 1779 1498 1584 1498
query71 471 207 208 207
query72 5172 3551 3543 3543
query73 1914 310 314 310
query74 7005 6445 6382 6382
query75 4638 2302 2320 2302
query76 6112 1031 1090 1031
query77 661 250 254 250
query78 9167 8683 8707 8683
query79 1017 507 522 507
query80 735 355 358 355
query81 452 214 211 211
query82 228 94 96 94
query83 192 140 138 138
query84 243 54 52 52
query85 925 280 270 270
query86 387 411 406 406
query87 3546 3382 3352 3352
query88 2855 2258 2247 2247
query89 334 250 249 249
query90 1893 199 189 189
query91 116 89 93 89
query92 57 52 50 50
query93 1048 500 439 439
query94 778 184 188 184
query95 465 413 415 413
query96 614 317 316 316
query97 4316 4184 4164 4164
query98 208 200 201 200
query99 1118 787 831 787
Total cold run time: 287396 ms
Total hot run time: 178786 ms
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feat](PipelineX) add distribute exprs to accelerate pipelinex [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29615:
URL: https://github.com/apache/doris/pull/29615#issuecomment-1878824653
TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G', run with scripts in https://github.com/apache/doris/tree/master/tools/tpch-tools
```
Tpch sf100 test result on commit 497c2d0c41927dc0cc819aca4ae283bf579cbeaa, data reload: false
------ Round 1 ----------------------------------
q1 18824 5293 5207 5207
q2 2039 156 144 144
q3 10609 1130 1143 1130
q4 10225 780 824 780
q5 7824 3051 2969 2969
q6 215 131 131 131
q7 945 525 545 525
q8 9251 2070 2004 2004
q9 6933 6465 6425 6425
q10 8263 3104 3014 3014
q11 420 222 209 209
q12 390 230 227 227
q13 18031 3691 3637 3637
q14 254 211 221 211
q15 559 524 516 516
q16 464 393 393 393
q17 989 491 464 464
q18 7349 6747 6691 6691
q19 1607 1381 1362 1362
q20 720 344 344 344
q21 2857 2378 2434 2378
q22 361 310 328 310
Total cold run time: 109129 ms
Total hot run time: 39071 ms
----- Round 2, with runtime_filter_mode=off -----
q1 5090 5122 5086 5086
q2 343 235 252 235
q3 3334 3313 3303 3303
q4 2132 2021 2028 2021
q5 5848 5823 5841 5823
q6 215 121 126 121
q7 2327 1908 1940 1908
q8 3429 3480 3491 3480
q9 8886 8919 8938 8919
q10 3813 3847 3881 3847
q11 615 466 494 466
q12 807 663 639 639
q13 7041 3208 3181 3181
q14 298 269 270 269
q15 577 524 519 519
q16 580 502 504 502
q17 1948 1810 1767 1767
q18 8766 8384 8279 8279
q19 1648 1608 1633 1608
q20 2176 1986 1966 1966
q21 5597 5430 5335 5335
q22 537 510 485 485
Total cold run time: 66007 ms
Total hot run time: 59759 ms
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feat](PipelineX) add distribute exprs to accelerate pipelinex [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #29615:
URL: https://github.com/apache/doris/pull/29615#issuecomment-1878772285
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feat](PipelineX) add distribute exprs to accelerate pipelinex [doris]
Posted by "LiBinfeng-01 (via GitHub)" <gi...@apache.org>.
LiBinfeng-01 commented on PR #29615:
URL: https://github.com/apache/doris/pull/29615#issuecomment-1878761942
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org