You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "Mryange (via GitHub)" <gi...@apache.org> on 2024/03/08 06:45:31 UTC
[PR] [feature](pipelineX) add mem control in local exchange sink [doris]
Mryange opened a new pull request, #31982:
URL: https://github.com/apache/doris/pull/31982
## Proposed changes
Issue Number: close #xxx
<!--Describe your changes.-->
## Further comments
If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](pipelineX) add mem control in local exchange sink [doris]
Posted by "HappenLee (via GitHub)" <gi...@apache.org>.
HappenLee merged PR #31982:
URL: https://github.com/apache/doris/pull/31982
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](pipelineX) add mem control in local exchange sink [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #31982:
URL: https://github.com/apache/doris/pull/31982#issuecomment-1987510426
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](pipelineX) add mem control in local exchange sink [doris]
Posted by "Mryange (via GitHub)" <gi...@apache.org>.
Mryange commented on PR #31982:
URL: https://github.com/apache/doris/pull/31982#issuecomment-1987254779
run p0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](pipelineX) add mem control in local exchange sink [doris]
Posted by "Mryange (via GitHub)" <gi...@apache.org>.
Mryange commented on PR #31982:
URL: https://github.com/apache/doris/pull/31982#issuecomment-1987137180
run p0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](pipelineX) add mem control in local exchange sink [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31982:
URL: https://github.com/apache/doris/pull/31982#issuecomment-1987105942
<details>
<summary>ClickBench: <b>Total hot run time: 32.18 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 18daa8402f2d1984965e0ed48a6eeae432f606d9, data reload: false
query1 0.05 0.03 0.03
query2 0.06 0.02 0.03
query3 0.23 0.06 0.06
query4 1.67 0.10 0.10
query5 0.50 0.50 0.49
query6 1.31 0.68 0.68
query7 0.02 0.01 0.01
query8 0.04 0.03 0.03
query9 0.54 0.52 0.52
query10 0.57 0.57 0.58
query11 0.13 0.10 0.10
query12 0.12 0.10 0.10
query13 0.58 0.58 0.56
query14 0.74 0.75 0.76
query15 0.83 0.81 0.80
query16 0.37 0.39 0.36
query17 0.97 0.94 0.98
query18 0.28 0.26 0.26
query19 1.82 1.68 1.69
query20 0.01 0.01 0.01
query21 15.40 0.62 0.65
query22 2.89 4.00 3.65
query23 17.62 1.05 0.92
query24 2.11 0.60 0.58
query25 0.24 0.04 0.09
query26 0.19 0.14 0.14
query27 0.03 0.03 0.04
query28 11.96 0.83 0.85
query29 12.71 3.32 3.46
query30 0.65 0.53 0.59
query31 2.79 0.33 0.35
query32 3.40 0.43 0.44
query33 2.84 2.78 2.78
query34 15.50 4.31 4.29
query35 4.30 4.33 4.32
query36 1.08 1.01 1.01
query37 0.07 0.05 0.05
query38 0.05 0.03 0.03
query39 0.03 0.02 0.02
query40 0.18 0.14 0.13
query41 0.07 0.03 0.02
query42 0.03 0.02 0.02
query43 0.02 0.02 0.03
Total cold run time: 105 s
Total hot run time: 32.18 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](pipelineX) add mem control in local exchange sink [doris]
Posted by "Mryange (via GitHub)" <gi...@apache.org>.
Mryange commented on PR #31982:
URL: https://github.com/apache/doris/pull/31982#issuecomment-1986671796
run p0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](pipelineX) add mem control in local exchange sink [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31982:
URL: https://github.com/apache/doris/pull/31982#issuecomment-1985125702
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](pipelineX) add mem control in local exchange sink [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31982:
URL: https://github.com/apache/doris/pull/31982#issuecomment-1985188972
TeamCity be ut coverage result:
Function Coverage: 35.37% (8556/24189)
Line Coverage: 27.15% (69525/256065)
Region Coverage: 26.36% (36073/136835)
Branch Coverage: 23.21% (18440/79446)
Coverage Report: http://coverage.selectdb-in.cc/coverage/bcf36e6f2f672820a4f712e5b4ba18a8646b2b7b_bcf36e6f2f672820a4f712e5b4ba18a8646b2b7b/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](pipelineX) add mem control in local exchange sink [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31982:
URL: https://github.com/apache/doris/pull/31982#issuecomment-1987103000
<details>
<summary>TPC-H: <b>Total hot run time: 38245 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 18daa8402f2d1984965e0ed48a6eeae432f606d9, data reload: false
------ Round 1 ----------------------------------
q1 17630 4049 4009 4009
q2 2025 150 145 145
q3 10551 947 942 942
q4 4664 938 940 938
q5 7608 2870 3000 2870
q6 182 126 124 124
q7 1297 846 814 814
q8 9477 2064 2074 2064
q9 7311 6527 6507 6507
q10 8226 2594 2596 2594
q11 421 227 219 219
q12 750 320 317 317
q13 17976 2983 2967 2967
q14 276 256 260 256
q15 501 470 468 468
q16 470 395 411 395
q17 939 881 861 861
q18 6761 5943 6022 5943
q19 1556 1519 1510 1510
q20 549 291 277 277
q21 7577 3728 3732 3728
q22 837 297 297 297
Total cold run time: 107584 ms
Total hot run time: 38245 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4026 3975 3978 3975
q2 319 220 226 220
q3 2972 2907 2908 2907
q4 1860 1833 1829 1829
q5 5237 5247 5222 5222
q6 211 115 116 115
q7 2271 1829 1826 1826
q8 3202 3258 3262 3258
q9 8525 8518 8563 8518
q10 6122 3715 3674 3674
q11 530 443 438 438
q12 680 536 536 536
q13 13718 2833 2819 2819
q14 269 253 264 253
q15 486 447 450 447
q16 451 395 410 395
q17 1693 1661 1633 1633
q18 7771 7520 7294 7294
q19 1706 1623 1612 1612
q20 1881 1686 1700 1686
q21 5011 4776 4835 4776
q22 516 438 439 438
Total cold run time: 69457 ms
Total hot run time: 53871 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](pipelineX) add mem control in local exchange sink [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31982:
URL: https://github.com/apache/doris/pull/31982#issuecomment-1987102198
TeamCity be ut coverage result:
Function Coverage: 35.36% (8556/24197)
Line Coverage: 27.14% (69529/256140)
Region Coverage: 26.37% (36082/136835)
Branch Coverage: 23.22% (18447/79440)
Coverage Report: http://coverage.selectdb-in.cc/coverage/18daa8402f2d1984965e0ed48a6eeae432f606d9_18daa8402f2d1984965e0ed48a6eeae432f606d9/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](pipelineX) add mem control in local exchange sink [doris]
Posted by "Mryange (via GitHub)" <gi...@apache.org>.
Mryange commented on PR #31982:
URL: https://github.com/apache/doris/pull/31982#issuecomment-1985338366
run p0 10
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](pipelineX) add mem control in local exchange sink [doris]
Posted by "Mryange (via GitHub)" <gi...@apache.org>.
Mryange commented on PR #31982:
URL: https://github.com/apache/doris/pull/31982#issuecomment-1987169417
run p0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](pipelineX) add mem control in local exchange sink [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31982:
URL: https://github.com/apache/doris/pull/31982#issuecomment-1987105101
<details>
<summary>TPC-DS: <b>Total hot run time: 186198 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 18daa8402f2d1984965e0ed48a6eeae432f606d9, data reload: false
query1 922 361 339 339
query2 7297 1923 1990 1923
query3 6705 216 213 213
query4 24280 20909 20746 20746
query5 4325 434 434 434
query6 262 192 192 192
query7 4645 312 319 312
query8 230 174 168 168
query9 8411 2270 2271 2270
query10 443 231 235 231
query11 15034 14439 14480 14439
query12 133 88 85 85
query13 1635 430 454 430
query14 10776 9901 9632 9632
query15 240 196 200 196
query16 7845 282 284 282
query17 1874 599 569 569
query18 2073 292 284 284
query19 204 164 168 164
query20 97 94 90 90
query21 206 128 133 128
query22 4660 4485 4466 4466
query23 31389 30771 30880 30771
query24 11230 3029 3093 3029
query25 658 399 398 398
query26 938 162 170 162
query27 2812 362 364 362
query28 6262 1865 1863 1863
query29 1030 619 628 619
query30 312 151 151 151
query31 914 724 733 724
query32 107 60 60 60
query33 760 264 254 254
query34 1041 473 486 473
query35 834 633 639 633
query36 917 817 863 817
query37 113 75 77 75
query38 3379 3162 3201 3162
query39 1443 1377 1376 1376
query40 211 120 119 119
query41 62 59 56 56
query42 109 102 104 102
query43 424 414 401 401
query44 1096 717 714 714
query45 273 261 262 261
query46 1045 796 776 776
query47 1639 1562 1551 1551
query48 420 356 355 355
query49 1131 351 349 349
query50 789 381 379 379
query51 6845 6706 6691 6691
query52 105 99 93 93
query53 357 284 284 284
query54 288 268 252 252
query55 85 84 84 84
query56 247 243 244 243
query57 1076 1005 987 987
query58 250 223 221 221
query59 2476 2290 2367 2290
query60 282 257 311 257
query61 115 119 112 112
query62 662 416 433 416
query63 306 286 281 281
query64 5257 3023 3053 3023
query65 3006 3018 3012 3012
query66 831 324 321 321
query67 14938 14526 14358 14358
query68 6073 548 565 548
query69 603 391 382 382
query70 1190 1100 1157 1100
query71 504 277 266 266
query72 6449 2710 2551 2551
query73 754 326 320 320
query74 6899 6534 6642 6534
query75 9273 7955 7984 7955
query76 4292 1102 1152 1102
query77 625 262 273 262
query78 10440 9625 9656 9625
query79 7765 513 521 513
query80 1725 392 387 387
query81 547 214 218 214
query82 1402 203 195 195
query83 226 145 147 145
query84 283 81 76 76
query85 1572 369 329 329
query86 477 287 285 285
query87 3464 3282 3310 3282
query88 4214 2403 2342 2342
query89 547 355 364 355
query90 1956 176 172 172
query91 164 158 127 127
query92 63 55 47 47
query93 6437 548 527 527
query94 1197 196 191 191
query95 457 349 345 345
query96 611 265 269 265
query97 4400 4286 4274 4274
query98 235 214 219 214
query99 1144 762 758 758
Total cold run time: 296749 ms
Total hot run time: 186198 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](pipelineX) add mem control in local exchange sink [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #31982:
URL: https://github.com/apache/doris/pull/31982#issuecomment-1987510440
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](pipelineX) add mem control in local exchange sink [doris]
Posted by "Mryange (via GitHub)" <gi...@apache.org>.
Mryange commented on PR #31982:
URL: https://github.com/apache/doris/pull/31982#issuecomment-1985125753
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](pipelineX) add mem control in local exchange sink [doris]
Posted by "Mryange (via GitHub)" <gi...@apache.org>.
Mryange commented on PR #31982:
URL: https://github.com/apache/doris/pull/31982#issuecomment-1985595303
run p0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](pipelineX) add mem control in local exchange sink [doris]
Posted by "Mryange (via GitHub)" <gi...@apache.org>.
Mryange commented on PR #31982:
URL: https://github.com/apache/doris/pull/31982#issuecomment-1986725666
run p0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](pipelineX) add mem control in local exchange sink [doris]
Posted by "Mryange (via GitHub)" <gi...@apache.org>.
Mryange commented on PR #31982:
URL: https://github.com/apache/doris/pull/31982#issuecomment-1987095066
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org