You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "924060929 (via GitHub)" <gi...@apache.org> on 2025/12/12 08:55:57 UTC
[PR] [opt](distribution) support bucket shuffle for set operation [doris]
924060929 opened a new pull request, #59006:
URL: https://github.com/apache/doris/pull/59006
### What problem does this PR solve?
support bucket shuffle for set operation(union/intersect/except)
### Check List (For Author)
- Test <!-- At least one of them must be included. -->
- [ ] Regression test
- [x] Unit Test
- [ ] Manual test (add detailed scripts or steps below)
- [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
- [ ] Previous test can cover this change.
- [ ] No code files have been changed.
- [ ] Other reason <!-- Add your reason? -->
- Behavior changed:
- [x] No.
- [ ] Yes. <!-- Explain the behavior change -->
- Does this need documentation?
- [x] No.
- [ ] Yes. <!-- Add document PR link here. eg: https://github.com/apache/doris-website/pull/1214 -->
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR should merge into -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3698368033
<details>
<summary>ClickBench: <b>Total hot run time: 26.94 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit fc57d6152d27913cfd8bc2196adc9e86dfe43c48, data reload: false
query1 0.06 0.05 0.05
query2 0.11 0.05 0.06
query3 0.26 0.09 0.09
query4 1.60 0.12 0.11
query5 0.28 0.25 0.26
query6 1.15 0.67 0.64
query7 0.03 0.03 0.03
query8 0.06 0.04 0.04
query9 0.55 0.49 0.49
query10 0.55 0.56 0.55
query11 0.14 0.11 0.11
query12 0.16 0.13 0.13
query13 0.63 0.61 0.60
query14 0.98 0.98 0.98
query15 0.81 0.80 0.79
query16 0.38 0.42 0.43
query17 1.04 1.04 1.06
query18 0.22 0.21 0.21
query19 1.91 1.89 1.78
query20 0.02 0.02 0.02
query21 15.45 0.30 0.14
query22 4.58 0.05 0.05
query23 15.92 0.28 0.10
query24 1.27 0.67 0.35
query25 0.12 0.09 0.05
query26 0.15 0.15 0.14
query27 0.07 0.05 0.05
query28 4.58 1.06 0.87
query29 12.61 3.94 3.13
query30 0.28 0.14 0.11
query31 2.81 0.62 0.39
query32 3.24 0.57 0.47
query33 3.00 2.98 3.04
query34 16.78 5.20 4.47
query35 4.53 4.46 4.43
query36 0.66 0.50 0.49
query37 0.12 0.07 0.06
query38 0.08 0.04 0.04
query39 0.04 0.04 0.03
query40 0.17 0.14 0.12
query41 0.09 0.03 0.03
query42 0.05 0.03 0.03
query43 0.04 0.03 0.04
Total cold run time: 97.58 s
Total hot run time: 26.94 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "924060929 (via GitHub)" <gi...@apache.org>.
924060929 commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3701800008
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3664434703
<details>
<summary>TPC-DS: <b>Total hot run time: 177791 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit d35f41b95e5be5f12fba46b647fc1ce7dfebbf6f, data reload: false
query5 4974 614 448 448
query6 347 224 207 207
query7 4219 464 269 269
query8 291 250 236 236
query9 8765 2520 2551 2520
query10 553 380 320 320
query11 15271 14777 14566 14566
query12 183 120 113 113
query13 1251 507 375 375
query14 6219 3069 2787 2787
query14_1 2713 2678 2687 2678
query15 217 194 181 181
query16 941 496 439 439
query17 1131 742 606 606
query18 2497 444 355 355
query19 234 259 211 211
query20 123 116 110 110
query21 225 141 117 117
query22 4044 3958 3847 3847
query23 16670 16124 15874 15874
query23_1 16125 16244 15891 15891
query24 7395 1696 1229 1229
query24_1 1249 1212 1200 1200
query25 567 476 418 418
query26 1265 265 168 168
query27 2759 444 301 301
query28 4495 2135 2100 2100
query29 813 528 436 436
query30 318 242 208 208
query31 801 703 622 622
query32 74 71 69 69
query33 539 343 294 294
query34 902 882 539 539
query35 784 806 720 720
query36 856 897 834 834
query37 132 93 81 81
query38 2858 2873 2791 2791
query39 754 742 705 705
query39_1 703 702 693 693
query40 227 151 121 121
query41 68 65 61 61
query42 116 107 104 104
query43 429 426 406 406
query44 1375 748 744 744
query45 191 184 186 184
query46 870 967 613 613
query47 1638 1711 1635 1635
query48 313 321 252 252
query49 635 435 355 355
query50 661 317 218 218
query51 3828 3893 3874 3874
query52 105 110 97 97
query53 325 350 291 291
query54 288 265 248 248
query55 77 77 72 72
query56 296 306 311 306
query57 1151 1179 1091 1091
query58 274 257 257 257
query59 2372 2430 2434 2430
query60 311 316 291 291
query61 159 151 157 151
query62 717 691 610 610
query63 325 299 301 299
query64 4993 1318 1000 1000
query65 4021 3949 3966 3949
query66 1436 476 317 317
query67 15173 15027 14720 14720
query68 4501 1027 739 739
query69 480 347 310 310
query70 1097 935 951 935
query71 372 310 276 276
query72 6144 4881 4910 4881
query73 661 564 314 314
query74 8923 8747 8505 8505
query75 3242 3163 2810 2810
query76 3860 1132 720 720
query77 516 398 303 303
query78 9410 9547 8912 8912
query79 1067 833 610 610
query80 1145 661 559 559
query81 541 271 237 237
query82 573 135 104 104
query83 375 258 238 238
query84 254 118 101 101
query85 967 521 472 472
query86 389 288 276 276
query87 3010 3042 3003 3003
query88 3215 2306 2280 2280
query89 461 410 397 397
query90 1938 156 153 153
query91 183 165 148 148
query92 71 70 68 68
query93 1054 920 567 567
query94 553 303 280 280
query95 587 380 304 304
query96 590 462 203 203
query97 2269 2342 2288 2288
query98 213 195 186 186
query99 1322 1312 1208 1208
Total cold run time: 256589 ms
Total hot run time: 177791 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3665165605
<details>
<summary>TPC-H: <b>Total hot run time: 35092 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 705b6dde2b67ffdc84e7411a52079faff45009a1, data reload: false
------ Round 1 ----------------------------------
q1 17631 4265 4132 4132
q2 2016 350 235 235
q3 10189 1380 749 749
q4 10219 871 309 309
q5 7496 2196 1893 1893
q6 181 170 139 139
q7 996 861 716 716
q8 9338 1450 1141 1141
q9 7088 5301 5329 5301
q10 6856 2405 1990 1990
q11 511 331 304 304
q12 683 716 584 584
q13 17775 3702 3080 3080
q14 283 299 285 285
q15 614 524 516 516
q16 683 684 630 630
q17 688 794 528 528
q18 7419 7139 7100 7100
q19 1105 962 630 630
q20 404 367 240 240
q21 4326 3883 3642 3642
q22 1055 996 948 948
Total cold run time: 107556 ms
Total hot run time: 35092 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4132 4061 4106 4061
q2 339 395 318 318
q3 2134 2713 2343 2343
q4 1357 1756 1296 1296
q5 4241 4582 4650 4582
q6 229 175 132 132
q7 2117 1967 1860 1860
q8 2765 2481 2613 2481
q9 7438 7543 7547 7543
q10 3044 3313 2811 2811
q11 590 507 481 481
q12 696 757 801 757
q13 3596 3931 3387 3387
q14 311 329 291 291
q15 558 521 525 521
q16 684 695 646 646
q17 1180 1439 1370 1370
q18 7918 7915 7540 7540
q19 858 930 903 903
q20 2145 2135 1919 1919
q21 4778 4411 4231 4231
q22 1080 1063 965 965
Total cold run time: 52190 ms
Total hot run time: 50438 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3660155287
<details>
<summary>TPC-DS: <b>Total hot run time: 178363 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit fee457e70de22f8e9a52857742b04a88d7b44a45, data reload: false
query5 4384 571 438 438
query6 342 241 233 233
query7 4232 490 277 277
query8 305 254 256 254
query9 8772 2569 2573 2569
query10 500 391 348 348
query11 15274 14799 14581 14581
query12 170 119 116 116
query13 1279 515 403 403
query14 5729 3009 2733 2733
query14_1 2679 2632 2683 2632
query15 209 204 180 180
query16 847 466 448 448
query17 1144 738 617 617
query18 2439 447 357 357
query19 241 253 215 215
query20 121 116 115 115
query21 224 148 117 117
query22 4019 4191 3933 3933
query23 16561 16266 15976 15976
query23_1 15958 16116 16002 16002
query24 7374 1648 1271 1271
query24_1 1239 1236 1197 1197
query25 624 472 426 426
query26 1242 270 158 158
query27 2784 455 317 317
query28 4454 2122 2117 2117
query29 796 533 440 440
query30 315 244 218 218
query31 807 704 607 607
query32 77 70 64 64
query33 544 346 291 291
query34 902 898 537 537
query35 781 832 725 725
query36 874 900 830 830
query37 129 93 79 79
query38 2813 2896 2875 2875
query39 757 746 723 723
query39_1 712 706 692 692
query40 227 141 124 124
query41 72 64 66 64
query42 106 107 105 105
query43 435 427 400 400
query44 1334 755 764 755
query45 194 195 182 182
query46 873 986 613 613
query47 1654 1705 1592 1592
query48 320 330 244 244
query49 640 428 352 352
query50 656 291 221 221
query51 3860 3854 3859 3854
query52 104 109 95 95
query53 324 358 302 302
query54 285 257 258 257
query55 89 77 73 73
query56 305 286 298 286
query57 1133 1126 1069 1069
query58 277 255 253 253
query59 2431 2484 2395 2395
query60 306 300 301 300
query61 165 158 163 158
query62 690 699 642 642
query63 329 298 300 298
query64 4969 1327 1035 1035
query65 4031 3927 3947 3927
query66 1471 436 324 324
query67 15102 15020 14848 14848
query68 5173 1021 743 743
query69 514 358 315 315
query70 1081 976 970 970
query71 380 306 282 282
query72 6260 5225 5030 5030
query73 668 583 307 307
query74 8772 8864 8573 8573
query75 3180 3164 2800 2800
query76 3845 1134 722 722
query77 531 393 293 293
query78 9743 9692 8896 8896
query79 1184 856 634 634
query80 1531 674 552 552
query81 553 269 237 237
query82 440 135 105 105
query83 360 255 242 242
query84 262 129 114 114
query85 945 518 465 465
query86 382 298 271 271
query87 3029 3114 3005 3005
query88 3219 2275 2261 2261
query89 470 423 388 388
query90 1951 163 153 153
query91 175 169 145 145
query92 71 68 69 68
query93 1076 892 563 563
query94 531 308 273 273
query95 578 328 314 314
query96 582 460 220 220
query97 2317 2367 2262 2262
query98 211 202 188 188
query99 1276 1283 1175 1175
Total cold run time: 256399 ms
Total hot run time: 178363 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3645998952
<details>
<summary>TPC-H: <b>Total hot run time: 34987 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 48c5f71eb30a8e76717d7f1cfaac067182a0dbcf, data reload: false
------ Round 1 ----------------------------------
q1 17618 4135 4058 4058
q2 2019 354 239 239
q3 10142 1354 730 730
q4 10229 941 319 319
q5 7526 2087 1978 1978
q6 189 168 134 134
q7 999 864 716 716
q8 9350 1439 1099 1099
q9 7086 5387 5350 5350
q10 6844 2373 1985 1985
q11 531 332 297 297
q12 659 729 569 569
q13 17787 3714 3015 3015
q14 291 311 286 286
q15 588 517 516 516
q16 680 664 627 627
q17 695 807 511 511
q18 7538 7098 7099 7098
q19 1118 961 621 621
q20 400 359 254 254
q21 4170 3842 3632 3632
q22 1066 1005 953 953
Total cold run time: 107525 ms
Total hot run time: 34987 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4093 4008 3999 3999
q2 325 410 322 322
q3 2150 2666 2320 2320
q4 1330 1745 1305 1305
q5 4233 4485 4815 4485
q6 230 171 131 131
q7 2114 1903 1855 1855
q8 2663 2554 2470 2470
q9 7547 7510 7589 7510
q10 3155 3289 2810 2810
q11 583 508 504 504
q12 670 743 758 743
q13 3560 3942 3370 3370
q14 309 450 357 357
q15 546 510 503 503
q16 644 674 603 603
q17 1174 1428 1405 1405
q18 7963 7621 7631 7621
q19 891 867 899 867
q20 1980 2170 1910 1910
q21 4852 4267 4139 4139
q22 1055 1030 973 973
Total cold run time: 52067 ms
Total hot run time: 50202 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3646136155
# FE UT Coverage Report
Increment line coverage ` 46.36% (51/110)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/59006_48c5f71eb30a8e76717d7f1cfaac067182a0dbcf/fe_increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/59006_48c5f71eb30a8e76717d7f1cfaac067182a0dbcf/fe_report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3674128804
# FE UT Coverage Report
Increment line coverage ` 40.91% (72/176)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/59006_d9fde14000f127e1c3ec4d3a00fe36a46b215e82/fe_increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/59006_d9fde14000f127e1c3ec4d3a00fe36a46b215e82/fe_report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "924060929 (via GitHub)" <gi...@apache.org>.
924060929 commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3674330700
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3659681327
# FE UT Coverage Report
Increment line coverage ` 52.68% (59/112)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/59006_857a966eb4535c85fb9ea60e8e310e1f890f26e6/fe_increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/59006_857a966eb4535c85fb9ea60e8e310e1f890f26e6/fe_report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3660119528
<details>
<summary>TPC-H: <b>Total hot run time: 35367 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit fee457e70de22f8e9a52857742b04a88d7b44a45, data reload: false
------ Round 1 ----------------------------------
q1 17674 4340 4103 4103
q2 2032 349 241 241
q3 10167 1323 765 765
q4 10211 845 323 323
q5 7524 2157 1895 1895
q6 194 169 142 142
q7 1014 877 723 723
q8 9377 1434 1198 1198
q9 7082 5352 5363 5352
q10 6881 2403 1987 1987
q11 519 332 301 301
q12 652 709 596 596
q13 18355 3739 3077 3077
q14 296 304 284 284
q15 598 534 519 519
q16 723 688 653 653
q17 740 857 563 563
q18 7856 7172 6980 6980
q19 1146 964 613 613
q20 407 371 250 250
q21 4270 4032 3826 3826
q22 1091 989 976 976
Total cold run time: 108809 ms
Total hot run time: 35367 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4175 4188 4117 4117
q2 340 430 328 328
q3 2112 2709 2323 2323
q4 1342 1769 1303 1303
q5 4276 5033 4478 4478
q6 241 190 134 134
q7 2069 1990 1830 1830
q8 2668 2628 2516 2516
q9 7619 7587 7506 7506
q10 3067 3247 2835 2835
q11 614 529 500 500
q12 835 776 648 648
q13 3560 4013 3295 3295
q14 305 324 321 321
q15 591 530 524 524
q16 665 718 685 685
q17 1260 1518 1440 1440
q18 7877 7819 7492 7492
q19 901 821 854 821
q20 1887 1964 1843 1843
q21 4704 4279 4179 4179
q22 1099 1027 991 991
Total cold run time: 52207 ms
Total hot run time: 50109 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3656222853
# FE Regression Coverage Report
Increment line coverage ` 56.90% (66/116)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/59006_3d8029fabf02b4ce727baa508757e689dcbe2d47_merge_fe/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/59006_3d8029fabf02b4ce727baa508757e689dcbe2d47_merge_fe/report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3668720775
<details>
<summary>TPC-H: <b>Total hot run time: 11711 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 6e8ee269eef4749bad44a5ff2aa2b2f19b433cb9, data reload: false
------ Round 1 ----------------------------------
revenue0 View NULL NULL -1 0 0 NULL 0 NULL NULL 2023-12-26 18:27:24 NULL NULL utf-8 NULL NULL
partsupp Doris NULL NULL 0 0 0 NULL 0 NULL NULL 2023-12-26 18:27:23 2023-12-26 18:44:20 NULL utf-8 NULL NULL
part Doris NULL NULL 0 0 0 NULL 0 NULL NULL 2023-12-26 18:27:23 2023-12-26 18:27:57 NULL utf-8 NULL NULL
nation Doris NULL NULL 0 0 0 NULL 0 NULL NULL 2023-12-26 18:27:23 2023-12-26 18:27:24 NULL utf-8 NULL NULL
customer Doris NULL NULL 0 0 0 NULL 0 NULL NULL 2023-12-26 18:27:23 2023-12-26 18:27:43 NULL utf-8 NULL NULL
lineitem Doris NULL NULL 0 0 0 NULL 0 NULL NULL 2023-12-26 18:27:23 2023-12-26 18:38:59 NULL utf-8 NULL NULL
supplier Doris NULL NULL 0 0 0 NULL 0 NULL NULL 2023-12-26 18:27:23 2023-12-26 18:27:25 NULL utf-8 NULL NULL
region Doris NULL NULL 0 0 0 NULL 0 NULL NULL 2023-12-26 18:27:23 2023-12-26 18:27:24 NULL utf-8 NULL NULL
orders Doris NULL NULL 0 0 0 NULL 0 NULL NULL 2023-12-26 18:27:23 2023-12-26 18:42:55 NULL utf-8 NULL NULL
============================================
q1 17612 4243 4055 4055
q2 2042 352 247 247
q3 q4 18702 1186 489 489
q5 q6 171 246 138 138
q7 q8 q9 q10 q11 655 312 q12 674 707 583 583
q13 19808 3707 3028 3028
q14 325 269 254 254
q15 612 515 516 515
q16 q17 9183 922 427 427
q18 q19 898 804 702 702
q20 1240 361 255 255
q21 q22 1291 1066 1018 1018
Total cold run time: 72558 ms
Total hot run time: 11711 ms
----- Round 2, with runtime_filter_mode=off -----
revenue0 View NULL NULL -1 0 0 NULL 0 NULL NULL 2023-12-26 18:27:24 NULL NULL utf-8 NULL NULL
partsupp Doris NULL NULL 80000000 56 4534120086 NULL 44625495 NULL NULL 2023-12-26 18:27:23 2023-12-26 18:44:20 NULL utf-8 NULL NULL
part Doris NULL NULL 20000000 37 748811035 NULL 1935627 NULL NULL 2023-12-26 18:27:23 2023-12-26 18:27:57 NULL utf-8 NULL NULL
nation Doris NULL NULL 25 138 3473 NULL 366 NULL NULL 2023-12-26 18:27:23 2023-12-26 18:27:24 NULL utf-8 NULL NULL
customer Doris NULL NULL 15000000 92 1381653732 NULL 4374759 NULL NULL 2023-12-26 18:27:23 2023-12-26 18:27:43 NULL utf-8 NULL NULL
lineitem Doris NULL NULL 600037902 33 19843441616 NULL 61784740 NULL NULL 2023-12-26 18:27:23 2023-12-26 18:38:59 NULL utf-8 NULL NULL
supplier Doris NULL NULL 1000000 87 87519212 NULL 194931 NULL NULL 2023-12-26 18:27:23 2023-12-26 18:27:25 NULL utf-8 NULL NULL
region Doris NULL NULL 5 240 1201 NULL 147 NULL NULL 2023-12-26 18:27:23 2023-12-26 18:27:24 NULL utf-8 NULL NULL
orders Doris NULL NULL 150000000 42 6422171781 NULL 22778155 NULL NULL 2023-12-26 18:27:23 2023-12-26 18:42:55 NULL utf-8 NULL NULL
============================================
q1 4155 4026 4018 4018
q2 345 392 350 350
q3 q4 1330 1633 1563 1563
q5 q6 152 225 130 130
q7 q8 q9 q10 5334 q11 q12 622 647 533 533
q13 3274 3644 3031 3031
q14 239 266 239 239
q15 547 514 516 514
q16 q17 1179 1363 1402 1363
q18 q19 765 895 980 895
q20 2012 2017 1856 1856
q21 q22 1080 1090 1019 1019
Total cold run time: 15700 ms
Total hot run time: 15511 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3665223844
<details>
<summary>ClickBench: <b>Total hot run time: 27.28 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 705b6dde2b67ffdc84e7411a52079faff45009a1, data reload: false
query1 0.06 0.05 0.04
query2 0.11 0.05 0.05
query3 0.26 0.09 0.09
query4 1.61 0.11 0.11
query5 0.29 0.26 0.27
query6 1.14 0.63 0.63
query7 0.04 0.03 0.03
query8 0.05 0.04 0.04
query9 0.58 0.51 0.50
query10 0.55 0.56 0.55
query11 0.16 0.11 0.11
query12 0.16 0.11 0.11
query13 0.62 0.60 0.61
query14 0.99 0.99 0.99
query15 0.82 0.81 0.80
query16 0.39 0.39 0.40
query17 1.05 1.04 0.99
query18 0.24 0.22 0.21
query19 1.92 1.83 1.84
query20 0.02 0.02 0.01
query21 15.43 0.29 0.14
query22 4.77 0.05 0.04
query23 15.97 0.29 0.10
query24 2.26 0.36 0.29
query25 0.09 0.06 0.07
query26 0.14 0.14 0.14
query27 0.06 0.08 0.06
query28 3.50 1.21 1.03
query29 12.59 4.04 3.21
query30 0.28 0.15 0.12
query31 2.81 0.64 0.40
query32 3.23 0.55 0.45
query33 3.12 3.08 2.99
query34 16.82 5.17 4.55
query35 4.52 4.52 4.58
query36 0.67 0.51 0.49
query37 0.11 0.07 0.06
query38 0.09 0.04 0.04
query39 0.05 0.03 0.04
query40 0.17 0.15 0.13
query41 0.09 0.04 0.03
query42 0.04 0.03 0.03
query43 0.04 0.04 0.03
Total cold run time: 97.91 s
Total hot run time: 27.28 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "924060929 (via GitHub)" <gi...@apache.org>.
924060929 commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3696313518
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3696381040
<details>
<summary>ClickBench: <b>Total hot run time: 26.89 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit fc57d6152d27913cfd8bc2196adc9e86dfe43c48, data reload: false
query1 0.05 0.05 0.05
query2 0.10 0.06 0.05
query3 0.26 0.08 0.08
query4 1.61 0.12 0.11
query5 0.28 0.27 0.24
query6 1.14 0.68 0.65
query7 0.03 0.02 0.03
query8 0.05 0.05 0.04
query9 0.57 0.49 0.50
query10 0.55 0.55 0.55
query11 0.16 0.11 0.11
query12 0.16 0.13 0.13
query13 0.61 0.59 0.59
query14 0.98 1.00 0.97
query15 0.81 0.78 0.80
query16 0.38 0.39 0.40
query17 1.06 0.99 1.03
query18 0.22 0.21 0.22
query19 1.97 1.84 1.85
query20 0.01 0.02 0.01
query21 15.46 0.30 0.13
query22 4.92 0.04 0.04
query23 15.97 0.29 0.11
query24 1.21 0.39 0.24
query25 0.07 0.08 0.05
query26 0.14 0.13 0.13
query27 0.07 0.08 0.06
query28 3.22 1.05 0.88
query29 12.61 3.93 3.19
query30 0.28 0.14 0.13
query31 2.82 0.61 0.37
query32 3.23 0.56 0.46
query33 3.04 3.00 3.02
query34 16.93 5.18 4.47
query35 4.45 4.49 4.55
query36 0.64 0.49 0.49
query37 0.11 0.07 0.07
query38 0.07 0.04 0.03
query39 0.05 0.03 0.04
query40 0.18 0.14 0.14
query41 0.09 0.04 0.03
query42 0.04 0.03 0.02
query43 0.04 0.04 0.03
Total cold run time: 96.64 s
Total hot run time: 26.89 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3696701047
# FE Regression Coverage Report
Increment line coverage ` 70.93% (122/172)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/59006_fc57d6152d27913cfd8bc2196adc9e86dfe43c48_merge_fe/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/59006_fc57d6152d27913cfd8bc2196adc9e86dfe43c48_merge_fe/report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3669392898
<details>
<summary>ClickBench: <b>Total hot run time: 27.6 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit c4909b8e1ca544e498fca4fa84004b4635bfd425, data reload: false
query1 0.06 0.05 0.05
query2 0.11 0.05 0.04
query3 0.27 0.08 0.09
query4 1.62 0.11 0.10
query5 0.26 0.24 0.27
query6 1.17 0.63 0.62
query7 0.03 0.03 0.02
query8 0.06 0.04 0.04
query9 0.58 0.52 0.52
query10 0.57 0.55 0.56
query11 0.17 0.10 0.12
query12 0.15 0.11 0.12
query13 0.63 0.60 0.61
query14 0.98 1.01 0.99
query15 0.81 0.79 0.82
query16 0.42 0.40 0.38
query17 1.05 1.00 1.00
query18 0.23 0.21 0.21
query19 1.85 1.78 1.81
query20 0.02 0.01 0.01
query21 15.51 0.30 0.14
query22 4.86 0.05 0.05
query23 16.05 0.29 0.10
query24 1.46 0.54 1.83
query25 0.13 0.06 0.06
query26 0.13 0.14 0.14
query27 0.06 0.08 0.06
query28 5.91 1.21 1.03
query29 12.63 4.15 3.36
query30 0.28 0.14 0.12
query31 2.82 0.64 0.41
query32 3.24 0.56 0.45
query33 2.96 2.98 3.03
query34 16.91 5.24 4.56
query35 4.65 4.56 4.59
query36 0.66 0.50 0.48
query37 0.11 0.07 0.06
query38 0.07 0.04 0.03
query39 0.04 0.03 0.03
query40 0.17 0.15 0.13
query41 0.10 0.03 0.02
query42 0.04 0.03 0.03
query43 0.03 0.03 0.04
Total cold run time: 99.86 s
Total hot run time: 27.6 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3702435957
# BE Regression && UT Coverage Report
Increment line coverage `71.43% (5/7)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/59006_b2e0198031172db18da6bc9fa4273f0ba3c48f7a_merge/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/59006_b2e0198031172db18da6bc9fa4273f0ba3c48f7a_merge/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 72.27% (25093/34723) |
| Line Coverage | 58.99% (263694/446981) |
| Region Coverage | 53.82% (218863/406642) |
| Branch Coverage | 55.39% (94006/169726) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3707928150
<details>
<summary>TPC-H: <b>Total hot run time: 32053 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit dce6dff1cb764b1b9c6337941c86b2b677ddc561, data reload: false
------ Round 1 ----------------------------------
q1 17604 4259 4056 4056
q2 2072 364 251 251
q3 10113 1330 722 722
q4 10201 835 312 312
q5 7517 2107 1857 1857
q6 183 168 138 138
q7 970 775 645 645
q8 9274 1450 1181 1181
q9 4894 4611 4561 4561
q10 6749 1809 1415 1415
q11 523 314 287 287
q12 686 745 579 579
q13 17803 3834 3095 3095
q14 300 302 267 267
q15 561 508 500 500
q16 699 696 614 614
q17 687 815 496 496
q18 6738 6536 6931 6536
q19 1175 1015 686 686
q20 457 404 267 267
q21 3231 2566 2566 2566
q22 1146 1124 1022 1022
Total cold run time: 103583 ms
Total hot run time: 32053 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4525 4282 4221 4221
q2 347 408 345 345
q3 2504 2736 2350 2350
q4 1453 1856 1440 1440
q5 4449 4334 4379 4334
q6 226 169 127 127
q7 1967 1886 1715 1715
q8 2521 2427 2396 2396
q9 7308 7165 7143 7143
q10 2467 2763 2258 2258
q11 598 512 463 463
q12 682 796 588 588
q13 3696 4033 3440 3440
q14 296 285 295 285
q15 557 501 498 498
q16 643 668 594 594
q17 1074 1307 1338 1307
q18 7524 7247 7176 7176
q19 810 783 811 783
q20 1879 1983 1814 1814
q21 4609 4146 4160 4146
q22 1108 1043 1002 1002
Total cold run time: 51243 ms
Total hot run time: 48425 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3674543493
<details>
<summary>ClickBench: <b>Total hot run time: 27.65 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit edf5358bca2a4e95429b7eeff9df276a3ad347df, data reload: false
query1 0.05 0.05 0.05
query2 0.10 0.04 0.04
query3 0.26 0.09 0.09
query4 1.61 0.11 0.11
query5 0.26 0.25 0.24
query6 1.15 0.64 0.63
query7 0.03 0.02 0.02
query8 0.06 0.04 0.04
query9 0.58 0.51 0.51
query10 0.58 0.54 0.56
query11 0.15 0.11 0.11
query12 0.15 0.12 0.12
query13 0.62 0.61 0.61
query14 0.99 0.99 0.98
query15 0.81 0.80 0.81
query16 0.42 0.43 0.40
query17 1.06 1.03 1.08
query18 0.24 0.21 0.21
query19 1.88 1.77 1.85
query20 0.01 0.02 0.01
query21 15.44 0.30 0.14
query22 4.72 0.05 0.05
query23 15.96 0.27 0.11
query24 0.99 0.50 0.34
query25 0.09 0.09 0.07
query26 0.14 0.13 0.14
query27 0.06 0.06 0.05
query28 4.43 1.22 1.03
query29 12.57 3.95 3.45
query30 0.28 0.14 0.12
query31 2.82 0.62 0.39
query32 3.23 0.54 0.46
query33 2.99 3.02 3.03
query34 16.85 5.21 4.60
query35 4.54 4.57 4.56
query36 0.64 0.49 0.50
query37 0.10 0.06 0.07
query38 0.06 0.04 0.03
query39 0.05 0.03 0.03
query40 0.17 0.14 0.12
query41 0.08 0.04 0.03
query42 0.04 0.04 0.03
query43 0.04 0.03 0.04
Total cold run time: 97.3 s
Total hot run time: 27.65 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3701865651
<details>
<summary>ClickBench: <b>Total hot run time: 26.94 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit b2e0198031172db18da6bc9fa4273f0ba3c48f7a, data reload: false
query1 0.05 0.04 0.04
query2 0.10 0.05 0.04
query3 0.26 0.09 0.08
query4 1.60 0.11 0.11
query5 0.27 0.25 0.26
query6 1.15 0.66 0.66
query7 0.04 0.03 0.02
query8 0.05 0.04 0.04
query9 0.57 0.50 0.50
query10 0.53 0.54 0.55
query11 0.17 0.11 0.11
query12 0.16 0.13 0.13
query13 0.62 0.60 0.62
query14 0.99 1.00 0.99
query15 0.80 0.80 0.80
query16 0.38 0.40 0.39
query17 1.03 1.07 1.05
query18 0.23 0.22 0.21
query19 1.87 1.83 1.88
query20 0.01 0.01 0.02
query21 15.44 0.28 0.13
query22 4.72 0.04 0.04
query23 16.02 0.30 0.10
query24 1.14 0.60 0.24
query25 0.07 0.06 0.05
query26 0.14 0.13 0.14
query27 0.12 0.05 0.06
query28 3.83 1.05 0.88
query29 12.57 3.95 3.17
query30 0.28 0.14 0.12
query31 2.80 0.62 0.39
query32 3.23 0.56 0.45
query33 2.97 3.03 3.05
query34 16.72 5.14 4.46
query35 4.48 4.57 4.47
query36 0.67 0.50 0.49
query37 0.12 0.07 0.06
query38 0.07 0.04 0.04
query39 0.05 0.03 0.03
query40 0.17 0.14 0.13
query41 0.09 0.04 0.03
query42 0.05 0.03 0.02
query43 0.04 0.03 0.03
Total cold run time: 96.67 s
Total hot run time: 26.94 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3646040002
<details>
<summary>TPC-DS: <b>Total hot run time: 178993 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 48c5f71eb30a8e76717d7f1cfaac067182a0dbcf, data reload: false
query5 4900 638 472 472
query6 371 227 240 227
query7 4232 469 279 279
query8 299 257 264 257
query9 8764 2549 2570 2549
query10 510 383 340 340
query11 15468 15187 14912 14912
query12 193 117 115 115
query13 1283 500 397 397
query14 6734 3311 2980 2980
query14_1 2910 2919 2901 2901
query15 218 204 181 181
query16 896 485 499 485
query17 1153 733 607 607
query18 2753 450 354 354
query19 235 229 211 211
query20 122 117 125 117
query21 223 151 120 120
query22 4037 4170 3827 3827
query23 16674 16373 15899 15899
query23_1 16068 16169 16070 16070
query24 7312 1665 1253 1253
query24_1 1262 1233 1268 1233
query25 594 495 443 443
query26 1009 273 162 162
query27 2733 466 311 311
query28 4411 2149 2171 2149
query29 773 542 428 428
query30 327 245 215 215
query31 869 694 617 617
query32 80 67 63 63
query33 538 337 293 293
query34 888 885 524 524
query35 788 826 725 725
query36 847 893 825 825
query37 136 95 79 79
query38 2901 2889 2837 2837
query39 758 743 726 726
query39_1 716 709 714 709
query40 226 135 127 127
query41 68 102 62 62
query42 105 105 109 105
query43 446 436 399 399
query44 1314 743 747 743
query45 194 194 185 185
query46 872 981 625 625
query47 1670 1705 1641 1641
query48 316 323 245 245
query49 649 433 379 379
query50 661 300 225 225
query51 3792 3882 3793 3793
query52 106 108 94 94
query53 320 353 288 288
query54 316 275 262 262
query55 80 77 69 69
query56 290 297 302 297
query57 1121 1144 1069 1069
query58 273 259 248 248
query59 2376 2547 2345 2345
query60 310 309 314 309
query61 162 160 162 160
query62 718 677 633 633
query63 331 296 306 296
query64 4541 1350 1005 1005
query65 4044 3971 3937 3937
query66 1334 451 313 313
query67 15179 15111 14945 14945
query68 8441 1003 726 726
query69 486 351 309 309
query70 1124 987 962 962
query71 372 312 284 284
query72 6071 5233 5078 5078
query73 701 646 313 313
query74 8934 8791 8610 8610
query75 3251 3389 2782 2782
query76 3921 1143 741 741
query77 655 399 307 307
query78 9415 9607 8978 8978
query79 1373 866 620 620
query80 687 638 546 546
query81 514 266 231 231
query82 204 134 110 110
query83 262 251 240 240
query84 257 125 105 105
query85 912 517 484 484
query86 387 301 294 294
query87 3093 3101 2996 2996
query88 3847 2267 2254 2254
query89 470 435 394 394
query90 2191 160 161 160
query91 172 177 146 146
query92 82 70 67 67
query93 1606 927 558 558
query94 464 288 274 274
query95 570 382 299 299
query96 585 484 210 210
query97 2230 2345 2223 2223
query98 225 199 189 189
query99 1289 1306 1269 1269
Total cold run time: 261651 ms
Total hot run time: 178993 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3669983120
<details>
<summary>ClickBench: <b>Total hot run time: 27.33 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 8044532befaa150e270ca4f2f540cbf3f82d0023, data reload: false
query1 0.05 0.05 0.05
query2 0.11 0.06 0.05
query3 0.26 0.08 0.08
query4 1.60 0.11 0.11
query5 0.28 0.27 0.26
query6 1.16 0.65 0.62
query7 0.04 0.02 0.02
query8 0.06 0.04 0.04
query9 0.58 0.51 0.51
query10 0.56 0.56 0.56
query11 0.15 0.11 0.10
query12 0.15 0.12 0.13
query13 0.61 0.60 0.59
query14 0.97 0.99 1.00
query15 0.81 0.80 0.80
query16 0.39 0.40 0.40
query17 1.00 0.97 1.01
query18 0.23 0.21 0.22
query19 1.96 1.86 1.85
query20 0.02 0.01 0.01
query21 15.45 0.30 0.14
query22 4.73 0.06 0.05
query23 15.98 0.29 0.10
query24 1.46 0.37 0.30
query25 0.11 0.07 0.08
query26 0.15 0.14 0.14
query27 0.08 0.06 0.05
query28 4.02 1.21 1.03
query29 12.66 4.08 3.26
query30 0.28 0.15 0.12
query31 2.81 0.62 0.39
query32 3.23 0.55 0.48
query33 2.98 2.97 3.03
query34 16.86 5.13 4.56
query35 4.57 4.48 4.61
query36 0.68 0.49 0.51
query37 0.10 0.06 0.06
query38 0.07 0.04 0.04
query39 0.05 0.03 0.03
query40 0.17 0.14 0.14
query41 0.10 0.03 0.03
query42 0.04 0.03 0.03
query43 0.04 0.04 0.03
Total cold run time: 97.61 s
Total hot run time: 27.33 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "924060929 (via GitHub)" <gi...@apache.org>.
924060929 commented on code in PR #59006:
URL: https://github.com/apache/doris/pull/59006#discussion_r2650498202
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/properties/ChildrenPropertiesRegulator.java:
##########
@@ -649,19 +667,82 @@ public List<List<PhysicalProperties>> visitPhysicalSetOperation(PhysicalSetOpera
} else if (requiredDistributionSpec instanceof DistributionSpecHash) {
// TODO: should use the most common hash spec as basic
DistributionSpecHash basic = (DistributionSpecHash) requiredDistributionSpec;
- for (int i = 0; i < originChildrenProperties.size(); i++) {
- DistributionSpecHash current
- = (DistributionSpecHash) originChildrenProperties.get(i).getDistributionSpec();
- if (current.getShuffleType() != ShuffleType.EXECUTION_BUCKETED
- || !bothSideShuffleKeysAreSameOrder(basic, current,
- (DistributionSpecHash) requiredProperties.get(0).getDistributionSpec(),
- (DistributionSpecHash) requiredProperties.get(i).getDistributionSpec())) {
+ int distributeToChildIndex = -1;
+ double basicBuckets = -1;
+ double basicRowCount = -1;
+
+ try {
+ ImmutableSet<ShuffleType> supportedShuffleTypes = ImmutableSet.of(
+ ShuffleType.NATURAL,
+ ShuffleType.STORAGE_BUCKETED
+ );
+ // find the most (bucket num, rowCount) side as the basic
+ for (int i = 0; i < originChildrenProperties.size(); i++) {
+ PhysicalProperties originChildrenProperty = originChildrenProperties.get(i);
+ DistributionSpec childDistribution = originChildrenProperty.getDistributionSpec();
+ if (childDistribution instanceof DistributionSpecHash
+ && supportedShuffleTypes.contains(
+ ((DistributionSpecHash) childDistribution).getShuffleType())) {
Review Comment:
if not exists NATURAL and STORAGE_BUCKETED, all children must be EXECUTION_BUCKETED, and the distributeToChildIndex will be -1, in this case, we will use EXECUTION_BUCKETED in [this branch](https://github.com/apache/doris/pull/59006/files/634b002f9d3d8bb834725b5c6c8befaa1cbec0e7#diff-79d8abdb9d7855b95a985ffa9b4ddbe1c646b58245fc2d0c02bef33374743777R731-R745)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3698348976
<details>
<summary>TPC-H: <b>Total hot run time: 35844 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit fc57d6152d27913cfd8bc2196adc9e86dfe43c48, data reload: false
------ Round 1 ----------------------------------
q1 17603 4158 4019 4019
q2 2033 349 238 238
q3 10229 1298 736 736
q4 10226 918 337 337
q5 7531 2098 1921 1921
q6 185 167 136 136
q7 955 793 674 674
q8 9270 1407 1148 1148
q9 6856 5196 5191 5191
q10 6831 1817 1430 1430
q11 517 328 291 291
q12 698 754 609 609
q13 17813 3847 3117 3117
q14 283 305 272 272
q15 588 517 510 510
q16 696 680 643 643
q17 706 732 614 614
q18 7539 7712 8031 7712
q19 1302 1161 748 748
q20 441 401 259 259
q21 4490 4230 4162 4162
q22 1153 1138 1077 1077
Total cold run time: 107945 ms
Total hot run time: 35844 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4337 4307 4233 4233
q2 309 419 325 325
q3 2364 2860 2392 2392
q4 1409 1832 1421 1421
q5 4458 4362 4330 4330
q6 217 166 126 126
q7 1985 2044 1738 1738
q8 2504 2354 2243 2243
q9 7253 7192 6959 6959
q10 2317 2566 2123 2123
q11 534 472 433 433
q12 665 693 558 558
q13 3305 3798 3054 3054
q14 256 299 263 263
q15 517 490 491 490
q16 621 656 601 601
q17 1072 1196 1296 1196
q18 7484 7241 7295 7241
q19 850 828 878 828
q20 1880 1976 1816 1816
q21 4453 4304 4177 4177
q22 1076 1021 1014 1014
Total cold run time: 49866 ms
Total hot run time: 47561 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3698495789
# BE UT Coverage Report
Increment line coverage `0.00% (0/2)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/b53cd4a552d1b04e8db7674ee303144185799fa6_b53cd4a552d1b04e8db7674ee303144185799fa6/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/b53cd4a552d1b04e8db7674ee303144185799fa6_b53cd4a552d1b04e8db7674ee303144185799fa6/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 53.37% (18951/35508) |
| Line Coverage | 39.25% (175886/448063) |
| Region Coverage | 33.79% (135967/402428) |
| Branch Coverage | 34.76% (58784/169138) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3669964934
<details>
<summary>TPC-DS: <b>Total hot run time: 163529 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 8044532befaa150e270ca4f2f540cbf3f82d0023, data reload: false
query5 4861 604 452 452
query6 340 231 211 211
query7 4203 462 286 286
query8 287 255 238 238
query9 8769 2530 2559 2530
query10 524 373 327 327
query11 15458 14680 14556 14556
query12 179 120 114 114
query13 1257 492 413 413
query14 6441 3015 2742 2742
query14_1 3891 3911 3824 3824
query15 217 205 186 186
query16 852 452 381 381
query17
query18 2699 437 337 337
query19 219 216 198 198
query20 121 115 109 109
query21 220 135 118 118
query22 3959 3873 3907 3873
query23 17280 16471 16291 16291
query23_1 16260 16774 16423 16423
query24
query24_1
query25
query26 1374 280 161 161
query27 4998 473 325 325
query28 4895 2202 2148 2148
query29
query30 344 265 220 220
query31 880 825 649 649
query32 119 73 69 69
query33 628 359 315 315
query34 4694 894 561 561
query35 852 843 767 767
query36 942 928 838 838
query37 179 94 84 84
query38 2986 2962 3058 2962
query39 847 775 763 763
query39_1 735 742 733 733
query40
query41 71 63 61 61
query42 118 113 104 104
query43 440 455 414 414
query44 1515 781 794 781
query45 208 205 190 190
query46 1018 1023 656 656
query47 1784 1832 1753 1753
query48 352 338 250 250
query49
query50
query51 3986 4054 4108 4054
query52 119 115 101 101
query53 347 316 297 297
query54 336 324 273 273
query55 95 86 78 78
query56 336 353 341 341
query57 1254 1181 1059 1059
query58 289 251 263 251
query59 2363 2377 2414 2377
query60 332 312 300 300
query61 183 183 181 181
query62 700 668 631 631
query63 334 289 303 289
query64 6913 1436 1101 1101
query65 4006 3945 3968 3945
query66 1401 422 307 307
query67 15432 15157 14959 14959
query68 7516 1065 718 718
query69 496 352 301 301
query70 1107 1024 928 928
query71 373 309 273 273
query72 6066 4941 4940 4940
query73 700 575 312 312
query74 8835 8729 8614 8614
query75
query76 5526 1206 798 798
query77 530 393 274 274
query78
query79 939 1068 590 590
query80
query81 529 267 234 234
query82 195 126 103 103
query83 322 258 240 240
query84 297 125 102 102
query85
query86 305 296 274 274
query87 2905 3043 2961 2961
query88 3138 2297 2298 2297
query89 449 426 406 406
query90 1978 155 147 147
query91 169 169 144 144
query92 70 67 64 64
query93 1009 934 558 558
query94 374 294 273 273
query95 563 379 311 311
query96 606 451 210 210
query97 2302 2271 2223 2223
query98 208 200 194 194
query99 1230 1277 1222 1222
Total cold run time: 245443 ms
Total hot run time: 163529 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "924060929 (via GitHub)" <gi...@apache.org>.
924060929 commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3659098454
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3663530877
<details>
<summary>TPC-H: <b>Total hot run time: 35281 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 0b794933568c7e451edfdc183a3960fdf41fce7a, data reload: false
------ Round 1 ----------------------------------
q1 17663 4205 4069 4069
q2 2059 358 238 238
q3 10138 1318 738 738
q4 10218 810 315 315
q5 7518 2111 1943 1943
q6 189 169 136 136
q7 1016 855 720 720
q8 9389 1432 1207 1207
q9 7110 5375 5327 5327
q10 6791 2380 1990 1990
q11 505 333 305 305
q12 683 739 581 581
q13 17804 3634 3049 3049
q14 291 295 277 277
q15 588 543 517 517
q16 673 680 637 637
q17 690 750 583 583
q18 7813 7057 7133 7057
q19 1090 968 620 620
q20 401 365 257 257
q21 4236 3928 3779 3779
q22 1006 1005 936 936
Total cold run time: 107871 ms
Total hot run time: 35281 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4076 4040 4247 4040
q2 314 400 327 327
q3 2160 2687 2291 2291
q4 1404 1777 1304 1304
q5 4227 4685 4636 4636
q6 231 182 134 134
q7 2127 1945 1940 1940
q8 2651 2550 2559 2550
q9 7724 7606 7368 7368
q10 3103 3249 2910 2910
q11 602 522 475 475
q12 705 759 615 615
q13 3605 3979 3342 3342
q14 306 303 292 292
q15 549 511 512 511
q16 663 671 645 645
q17 1233 1617 1393 1393
q18 7933 7554 7598 7554
q19 916 896 890 890
q20 1984 2083 1987 1987
q21 4853 4254 4134 4134
q22 1070 1011 957 957
Total cold run time: 52436 ms
Total hot run time: 50295 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3674076597
<details>
<summary>ClickBench: <b>Total hot run time: 27.31 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit d9fde14000f127e1c3ec4d3a00fe36a46b215e82, data reload: false
query1 0.05 0.05 0.05
query2 0.10 0.06 0.05
query3 0.25 0.08 0.08
query4 1.60 0.11 0.11
query5 0.28 0.25 0.25
query6 1.17 0.64 0.63
query7 0.04 0.02 0.02
query8 0.06 0.04 0.04
query9 0.58 0.52 0.51
query10 0.56 0.55 0.55
query11 0.17 0.11 0.11
query12 0.15 0.12 0.12
query13 0.63 0.61 0.60
query14 1.00 0.99 0.98
query15 0.82 0.79 0.82
query16 0.39 0.41 0.39
query17 1.04 1.07 1.05
query18 0.23 0.22 0.21
query19 1.92 1.85 1.86
query20 0.02 0.01 0.01
query21 15.42 0.28 0.14
query22 4.65 0.05 0.04
query23 16.04 0.30 0.11
query24 1.23 1.27 0.22
query25 0.06 0.06 0.05
query26 0.14 0.13 0.14
query27 0.07 0.04 0.05
query28 5.15 1.24 1.03
query29 12.66 4.03 3.21
query30 0.29 0.14 0.12
query31 2.84 0.63 0.40
query32 3.23 0.55 0.46
query33 3.01 3.05 3.02
query34 16.93 5.19 4.55
query35 4.57 4.62 4.54
query36 0.66 0.51 0.50
query37 0.11 0.07 0.07
query38 0.08 0.04 0.04
query39 0.04 0.03 0.03
query40 0.18 0.14 0.13
query41 0.08 0.03 0.02
query42 0.05 0.03 0.03
query43 0.04 0.03 0.03
Total cold run time: 98.59 s
Total hot run time: 27.31 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3698440997
<details>
<summary>TPC-DS: <b>Total hot run time: 174619 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit b53cd4a552d1b04e8db7674ee303144185799fa6, data reload: false
query5 4848 587 459 459
query6 339 225 209 209
query7 4229 465 278 278
query8 338 250 248 248
query9 8786 2606 2593 2593
query10 519 378 335 335
query11 15225 15286 15053 15053
query12 185 117 113 113
query13 1271 504 399 399
query14 6490 3045 2771 2771
query14_1 2660 2626 2648 2626
query15 202 192 182 182
query16 982 462 457 457
query17 1032 658 548 548
query18 2639 418 333 333
query19 217 211 192 192
query20 131 113 112 112
query21 217 142 123 123
query22 3843 4091 4073 4073
query23 15999 15615 15418 15418
query23_1 15338 15402 15363 15363
query24 7362 1564 1177 1177
query24_1 1218 1184 1206 1184
query25 538 451 398 398
query26 1243 256 171 171
query27 2750 449 292 292
query28 4536 2210 2175 2175
query29 829 594 464 464
query30 310 237 212 212
query31 821 636 554 554
query32 80 82 72 72
query33 575 343 299 299
query34 904 885 535 535
query35 755 795 696 696
query36 901 897 831 831
query37 132 97 85 85
query38 2751 2723 2621 2621
query39 759 753 734 734
query39_1 705 724 716 716
query40 223 138 118 118
query41 77 68 68 68
query42 104 103 111 103
query43 459 462 424 424
query44 1349 771 756 756
query45 189 182 178 178
query46 895 970 608 608
query47 1457 1372 1372 1372
query48 316 333 262 262
query49 623 425 342 342
query50 654 277 210 210
query51 3741 3771 3718 3718
query52 106 107 98 98
query53 301 331 275 275
query54 322 272 262 262
query55 82 80 74 74
query56 301 306 306 306
query57 999 1038 907 907
query58 272 261 299 261
query59 2144 2220 2115 2115
query60 312 305 295 295
query61 169 180 164 164
query62 388 344 347 344
query63 304 267 267 267
query64 5034 1294 991 991
query65 3780 3626 3776 3626
query66 1394 431 314 314
query67 15429 15297 14688 14688
query68 5343 1025 721 721
query69 485 331 311 311
query70 1070 985 875 875
query71 366 295 274 274
query72 6060 4782 4896 4782
query73 697 608 313 313
query74 8788 8688 8554 8554
query75 2878 2908 2503 2503
query76 3931 1057 684 684
query77 510 368 282 282
query78 9698 9913 9206 9206
query79 997 936 610 610
query80 776 566 493 493
query81 503 268 227 227
query82 416 144 108 108
query83 275 256 235 235
query84 260 118 103 103
query85 889 511 457 457
query86 350 334 316 316
query87 2854 2943 2764 2764
query88 3306 2277 2270 2270
query89 392 360 333 333
query90 1957 155 149 149
query91 171 168 148 148
query92 77 69 69 69
query93 1105 909 561 561
query94 673 332 324 324
query95 600 346 382 346
query96 597 468 209 209
query97 2316 2345 2282 2282
query98 208 197 202 197
query99 590 573 499 499
Total cold run time: 252489 ms
Total hot run time: 174619 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "924060929 (via GitHub)" <gi...@apache.org>.
924060929 commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3701794253
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3701856441
<details>
<summary>TPC-DS: <b>Total hot run time: 174426 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit b2e0198031172db18da6bc9fa4273f0ba3c48f7a, data reload: false
query5 5906 582 437 437
query6 325 221 209 209
query7 4207 474 271 271
query8 333 258 240 240
query9 8798 2588 2626 2588
query10 555 378 334 334
query11 15083 15121 14789 14789
query12 185 120 117 117
query13 1259 523 389 389
query14 7743 2963 2706 2706
query14_1 2632 2591 2668 2591
query15 256 192 181 181
query16 997 506 455 455
query17 1294 708 594 594
query18 2730 448 352 352
query19 273 226 207 207
query20 124 120 120 120
query21 217 138 122 122
query22 4123 4121 4208 4121
query23 16078 15931 15511 15511
query23_1 15649 15320 15438 15320
query24 7031 1594 1240 1240
query24_1 1234 1209 1234 1209
query25 584 484 446 446
query26 1226 265 171 171
query27 2699 455 302 302
query28 4550 2207 2201 2201
query29 841 556 457 457
query30 308 245 219 219
query31 766 642 536 536
query32 79 73 72 72
query33 544 355 313 313
query34 878 902 536 536
query35 767 802 688 688
query36 856 927 748 748
query37 126 94 81 81
query38 2672 2663 2733 2663
query39 790 772 723 723
query39_1 722 708 721 708
query40 225 137 126 126
query41 77 114 63 63
query42 105 115 115 115
query43 473 436 425 425
query44 1397 750 749 749
query45 182 177 176 176
query46 873 978 626 626
query47 1459 1485 1344 1344
query48 322 340 257 257
query49 606 409 332 332
query50 645 275 219 219
query51 3731 3764 3724 3724
query52 108 105 97 97
query53 306 329 271 271
query54 289 262 246 246
query55 83 75 74 74
query56 284 289 283 283
query57 1030 1007 904 904
query58 270 261 249 249
query59 2036 2222 1955 1955
query60 319 322 304 304
query61 162 160 160 160
query62 376 349 326 326
query63 298 266 273 266
query64 4924 1321 993 993
query65 3841 3707 3652 3652
query66 1339 429 312 312
query67 15956 15846 14854 14854
query68 6160 1025 740 740
query69 501 343 313 313
query70 1083 968 922 922
query71 410 300 274 274
query72 6130 4742 5128 4742
query73 710 599 316 316
query74 8750 8804 8523 8523
query75 2926 2903 2493 2493
query76 3778 1059 675 675
query77 527 384 282 282
query78 9647 9881 9163 9163
query79 1271 915 618 618
query80 667 605 484 484
query81 509 267 234 234
query82 211 144 107 107
query83 264 254 239 239
query84 257 116 106 106
query85 892 518 469 469
query86 367 290 287 287
query87 2814 2871 2761 2761
query88 3166 2257 2242 2242
query89 378 359 324 324
query90 2204 158 147 147
query91 177 166 141 141
query92 76 70 62 62
query93 1104 917 570 570
query94 569 346 299 299
query95 548 366 307 307
query96 574 451 211 211
query97 2348 2423 2281 2281
query98 241 205 204 204
query99 576 595 510 510
Total cold run time: 256011 ms
Total hot run time: 174426 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "924060929 (via GitHub)" <gi...@apache.org>.
924060929 commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3698393263
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "924060929 (via GitHub)" <gi...@apache.org>.
924060929 commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3707911218
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3707964671
# FE UT Coverage Report
Increment line coverage ` 50.80% (95/187)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/59006_dce6dff1cb764b1b9c6337941c86b2b677ddc561/fe_increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/59006_dce6dff1cb764b1b9c6337941c86b2b677ddc561/fe_report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3702449897
# FE Regression Coverage Report
Increment line coverage ` 74.47% (140/188)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/59006_b2e0198031172db18da6bc9fa4273f0ba3c48f7a_merge_fe/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/59006_b2e0198031172db18da6bc9fa4273f0ba3c48f7a_merge_fe/report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3707986049
# BE UT Coverage Report
Increment line coverage `0.00% (0/12)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/dce6dff1cb764b1b9c6337941c86b2b677ddc561_dce6dff1cb764b1b9c6337941c86b2b677ddc561/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/dce6dff1cb764b1b9c6337941c86b2b677ddc561_dce6dff1cb764b1b9c6337941c86b2b677ddc561/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 53.27% (18968/35605) |
| Line Coverage | 39.24% (176008/448565) |
| Region Coverage | 33.79% (136240/403231) |
| Branch Coverage | 34.74% (58812/169306) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3645548455
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR).
Please clearly describe your PR:
1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
3. What features were added. Why was this function added?
4. Which code was refactored and why was this part of the code refactored?
5. Which functions were optimized and what is the difference before and after the optimization?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3645614644
<details>
<summary>TPC-H: <b>Total hot run time: 35279 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit d0855d6ab9fa3e2950745cce9256b1d472bb5e22, data reload: false
------ Round 1 ----------------------------------
q1 17618 4227 4115 4115
q2 2032 351 237 237
q3 10175 1339 759 759
q4 10234 964 334 334
q5 7568 2192 1911 1911
q6 187 173 139 139
q7 1009 862 709 709
q8 9358 1460 1238 1238
q9 7181 5314 5354 5314
q10 6781 2396 1968 1968
q11 523 320 305 305
q12 678 724 595 595
q13 17775 3710 3049 3049
q14 287 288 278 278
q15 586 519 520 519
q16 678 688 629 629
q17 706 841 496 496
q18 7308 7059 7051 7051
q19 919 953 624 624
q20 397 363 252 252
q21 4242 3951 3778 3778
q22 1036 1000 979 979
Total cold run time: 107278 ms
Total hot run time: 35279 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4108 4043 4037 4037
q2 340 412 332 332
q3 2153 2682 2301 2301
q4 1301 1737 1273 1273
q5 4253 4593 4855 4593
q6 224 180 132 132
q7 2039 2037 1828 1828
q8 2715 2523 2563 2523
q9 7759 7532 7465 7465
q10 3078 3376 2887 2887
q11 610 529 489 489
q12 680 748 624 624
q13 3600 3979 3347 3347
q14 306 304 270 270
q15 550 520 532 520
q16 665 728 658 658
q17 1203 1405 1527 1405
q18 8199 7830 7556 7556
q19 948 862 864 862
q20 2009 2062 1960 1960
q21 4971 4324 4230 4230
q22 1054 989 995 989
Total cold run time: 52765 ms
Total hot run time: 50281 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "924060929 (via GitHub)" <gi...@apache.org>.
924060929 commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3645864824
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "924060929 (via GitHub)" <gi...@apache.org>.
924060929 commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3668450289
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3665144499
# FE UT Coverage Report
Increment line coverage ` 55.91% (71/127)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/59006_705b6dde2b67ffdc84e7411a52079faff45009a1/fe_increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/59006_705b6dde2b67ffdc84e7411a52079faff45009a1/fe_report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3665204044
<details>
<summary>TPC-DS: <b>Total hot run time: 178900 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 705b6dde2b67ffdc84e7411a52079faff45009a1, data reload: false
query5 4388 572 444 444
query6 334 235 226 226
query7 4226 471 290 290
query8 301 255 261 255
query9 8749 2560 2561 2560
query10 510 401 332 332
query11 15211 14733 14745 14733
query12 194 117 118 117
query13 1292 513 401 401
query14 5552 3002 2776 2776
query14_1 2681 2671 2679 2671
query15 204 200 179 179
query16 837 468 425 425
query17 908 717 589 589
query18 2428 429 344 344
query19 251 225 202 202
query20 120 113 114 113
query21 219 141 120 120
query22 3886 3878 4001 3878
query23 16438 16374 16119 16119
query23_1 15957 15903 15906 15903
query24 7390 1660 1232 1232
query24_1 1258 1238 1285 1238
query25 608 508 481 481
query26 1250 275 175 175
query27 2738 465 328 328
query28 4472 2132 2138 2132
query29 827 598 481 481
query30 326 244 225 225
query31 834 726 644 644
query32 85 71 76 71
query33 553 382 307 307
query34 916 905 549 549
query35 800 818 722 722
query36 872 914 824 824
query37 136 102 84 84
query38 2849 2838 2850 2838
query39 755 741 733 733
query39_1 726 709 701 701
query40 228 148 129 129
query41 92 68 68 68
query42 109 109 109 109
query43 444 448 412 412
query44 1375 748 748 748
query45 199 196 185 185
query46 885 988 633 633
query47 1668 1705 1647 1647
query48 316 334 243 243
query49 651 460 374 374
query50 670 313 223 223
query51 3827 3826 3781 3781
query52 109 110 102 102
query53 319 355 304 304
query54 316 289 269 269
query55 85 79 78 78
query56 327 332 318 318
query57 1122 1129 1078 1078
query58 275 278 269 269
query59 2406 2411 2392 2392
query60 333 329 317 317
query61 211 160 158 158
query62 724 673 657 657
query63 329 297 300 297
query64 4970 1289 1000 1000
query65 4068 3998 3942 3942
query66 1474 437 334 334
query67 15151 15018 14614 14614
query68 7972 1010 727 727
query69 518 351 310 310
query70 1085 990 974 974
query71 377 306 289 289
query72 6153 5047 5140 5047
query73 723 655 308 308
query74 8480 8810 8604 8604
query75 3196 3181 2837 2837
query76 3840 1145 747 747
query77 517 408 291 291
query78 9363 9557 8863 8863
query79 1753 836 622 622
query80 724 649 552 552
query81 508 267 233 233
query82 206 133 100 100
query83 265 256 238 238
query84 311 127 102 102
query85 914 515 480 480
query86 382 284 286 284
query87 3099 3019 2982 2982
query88 3301 2289 2267 2267
query89 472 428 409 409
query90 2182 167 165 165
query91 178 160 142 142
query92 82 71 64 64
query93 2257 925 557 557
query94 465 317 273 273
query95 571 381 315 315
query96 610 480 213 213
query97 2282 2310 2196 2196
query98 215 197 192 192
query99 1283 1320 1279 1279
Total cold run time: 259187 ms
Total hot run time: 178900 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3674058915
<details>
<summary>TPC-DS: <b>Total hot run time: 178603 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit d9fde14000f127e1c3ec4d3a00fe36a46b215e82, data reload: false
query5 4842 579 445 445
query6 333 238 210 210
query7 4220 480 267 267
query8 319 257 245 245
query9 8791 2513 2549 2513
query10 538 387 325 325
query11 15511 15492 14638 14638
query12 199 117 120 117
query13 1268 526 377 377
query14 6438 3052 2802 2802
query14_1 2655 2594 2664 2594
query15 210 199 177 177
query16 896 487 455 455
query17 1105 690 575 575
query18 2593 430 341 341
query19 234 233 206 206
query20 119 116 112 112
query21 219 139 120 120
query22 3854 3997 3850 3850
query23 16559 16080 16069 16069
query23_1 16063 16117 16068 16068
query24 7376 1660 1271 1271
query24_1 1294 1257 1258 1257
query25 593 499 457 457
query26 1250 277 172 172
query27 2742 463 313 313
query28 4526 2126 2135 2126
query29 847 585 480 480
query30 328 250 218 218
query31 840 703 652 652
query32 84 75 67 67
query33 578 351 308 308
query34 909 910 556 556
query35 800 815 733 733
query36 869 916 800 800
query37 134 94 79 79
query38 2894 2847 2835 2835
query39 770 728 724 724
query39_1 698 706 694 694
query40 230 146 127 127
query41 74 72 68 68
query42 113 111 110 110
query43 423 453 397 397
query44 1387 765 755 755
query45 200 197 192 192
query46 881 993 635 635
query47 1628 1677 1620 1620
query48 343 338 270 270
query49 661 449 378 378
query50 667 311 240 240
query51 3897 3806 3825 3806
query52 113 109 102 102
query53 324 359 317 317
query54 310 275 274 274
query55 88 78 78 78
query56 304 327 326 326
query57 1141 1147 1075 1075
query58 288 279 262 262
query59 2470 2453 2458 2453
query60 315 311 291 291
query61 162 159 161 159
query62 721 703 620 620
query63 331 302 315 302
query64 4930 1313 1003 1003
query65 4035 3986 3949 3949
query66 1401 433 319 319
query67 15306 14997 14622 14622
query68 8385 1023 733 733
query69 488 346 311 311
query70 1124 992 998 992
query71 384 314 287 287
query72 6087 4994 5094 4994
query73 696 640 312 312
query74 8969 8728 8665 8665
query75 3269 3202 2799 2799
query76 4023 1163 760 760
query77 601 413 302 302
query78 9486 9537 8855 8855
query79 1471 889 615 615
query80 712 659 568 568
query81 506 274 237 237
query82 206 138 105 105
query83 259 268 241 241
query84 251 125 108 108
query85 882 508 468 468
query86 343 301 278 278
query87 3182 3058 2955 2955
query88 4260 2278 2274 2274
query89 469 428 400 400
query90 2230 169 157 157
query91 168 168 154 154
query92 88 69 66 66
query93 1488 914 566 566
query94 471 302 281 281
query95 574 394 312 312
query96 606 482 214 214
query97 2303 2318 2257 2257
query98 233 196 192 192
query99 1269 1303 1253 1253
Total cold run time: 263007 ms
Total hot run time: 178603 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "924060929 (via GitHub)" <gi...@apache.org>.
924060929 commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3673936726
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3663549698
<details>
<summary>TPC-DS: <b>Total hot run time: 178734 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 0b794933568c7e451edfdc183a3960fdf41fce7a, data reload: false
query5 5184 598 434 434
query6 326 250 232 232
query7 4208 464 279 279
query8 303 274 265 265
query9 8796 2538 2502 2502
query10 562 391 336 336
query11 15350 14934 14548 14548
query12 194 114 110 110
query13 1269 510 395 395
query14 6366 3014 2776 2776
query14_1 2661 2660 2678 2660
query15 211 200 181 181
query16 945 474 446 446
query17 1137 727 612 612
query18 2673 452 346 346
query19 232 228 213 213
query20 120 150 111 111
query21 217 136 116 116
query22 3882 4050 3869 3869
query23 16797 16129 16171 16129
query23_1 15977 16094 16102 16094
query24 7318 1664 1210 1210
query24_1 1223 1204 1237 1204
query25 546 466 405 405
query26 1249 266 170 170
query27 2744 462 311 311
query28 4411 2135 2146 2135
query29 799 540 461 461
query30 314 245 214 214
query31 809 707 630 630
query32 77 72 70 70
query33 536 343 283 283
query34 914 912 535 535
query35 771 807 726 726
query36 863 904 829 829
query37 124 88 75 75
query38 2839 2844 2806 2806
query39 760 744 711 711
query39_1 690 690 698 690
query40 233 143 123 123
query41 68 62 63 62
query42 104 106 105 105
query43 436 440 406 406
query44 1330 760 749 749
query45 195 195 183 183
query46 879 994 623 623
query47 1647 1674 1616 1616
query48 332 334 251 251
query49 629 438 350 350
query50 660 305 209 209
query51 3756 3785 3816 3785
query52 107 117 99 99
query53 319 353 303 303
query54 279 253 247 247
query55 84 78 73 73
query56 300 300 299 299
query57 1167 1114 1079 1079
query58 276 252 247 247
query59 2387 2536 2441 2441
query60 315 300 292 292
query61 170 162 157 157
query62 698 665 605 605
query63 329 299 307 299
query64 4990 1360 1124 1124
query65 4038 3943 3968 3943
query66 1414 470 336 336
query67 14921 15189 14790 14790
query68 4875 1036 743 743
query69 506 360 311 311
query70 1097 1002 1012 1002
query71 358 311 286 286
query72 6353 4944 5081 4944
query73 699 627 312 312
query74 8862 8756 8620 8620
query75 3223 3156 2786 2786
query76 3898 1183 733 733
query77 518 389 290 290
query78 9388 9695 8864 8864
query79 1551 874 615 615
query80 736 655 556 556
query81 526 274 231 231
query82 200 136 105 105
query83 265 259 244 244
query84 261 116 112 112
query85 916 523 470 470
query86 388 284 274 274
query87 3013 3043 2967 2967
query88 3257 2279 2254 2254
query89 470 419 387 387
query90 2188 165 153 153
query91 173 171 149 149
query92 82 74 64 64
query93 1454 913 567 567
query94 473 297 274 274
query95 571 329 360 329
query96 605 461 211 211
query97 2299 2306 2216 2216
query98 216 194 188 188
query99 1310 1311 1242 1242
Total cold run time: 257010 ms
Total hot run time: 178734 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3698362286
<details>
<summary>TPC-DS: <b>Total hot run time: 175079 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit fc57d6152d27913cfd8bc2196adc9e86dfe43c48, data reload: false
query5 4660 581 438 438
query6 320 225 208 208
query7 4211 452 263 263
query8 332 247 230 230
query9 8778 2541 2637 2541
query10 540 357 309 309
query11 15196 15164 14969 14969
query12 176 113 109 109
query13 1250 485 367 367
query14 6132 2955 2652 2652
query14_1 2601 2558 2574 2558
query15 201 194 171 171
query16 1005 499 459 459
query17 1088 697 584 584
query18 2430 445 340 340
query19 231 220 202 202
query20 124 118 117 117
query21 221 144 120 120
query22 3978 4021 3985 3985
query23 16060 15605 15333 15333
query23_1 15377 15473 15490 15473
query24 7388 1581 1251 1251
query24_1 1255 1218 1193 1193
query25 553 485 423 423
query26 1254 265 162 162
query27 2769 455 294 294
query28 4582 2182 2181 2181
query29 798 551 453 453
query30 314 243 217 217
query31 817 646 563 563
query32 79 74 73 73
query33 571 349 296 296
query34 903 885 544 544
query35 783 787 703 703
query36 863 935 814 814
query37 130 89 77 77
query38 2730 2633 2658 2633
query39 786 739 735 735
query39_1 728 755 709 709
query40 224 135 119 119
query41 73 68 72 68
query42 105 101 100 100
query43 443 483 421 421
query44 1349 756 755 755
query45 190 184 179 179
query46 867 972 609 609
query47 1456 1485 1381 1381
query48 328 343 252 252
query49 623 428 342 342
query50 654 284 213 213
query51 3792 3781 3694 3694
query52 114 107 97 97
query53 306 327 276 276
query54 309 270 256 256
query55 80 75 73 73
query56 304 352 303 303
query57 1044 1022 954 954
query58 261 244 256 244
query59 2123 2205 2184 2184
query60 317 314 280 280
query61 159 160 154 154
query62 379 351 334 334
query63 294 264 266 264
query64 4936 1320 1020 1020
query65 3825 3737 3643 3643
query66 1456 428 319 319
query67 15131 15605 15849 15605
query68 7619 1003 723 723
query69 498 340 302 302
query70 1042 958 982 958
query71 388 307 277 277
query72 6073 4646 4680 4646
query73 721 576 308 308
query74 8904 8775 8507 8507
query75 2926 2878 2484 2484
query76 3880 1054 666 666
query77 519 355 291 291
query78 9725 10062 9177 9177
query79 1286 922 610 610
query80 629 573 495 495
query81 503 275 237 237
query82 225 146 122 122
query83 277 257 245 245
query84 266 124 103 103
query85 896 506 459 459
query86 388 328 319 319
query87 2871 2876 2789 2789
query88 3260 2287 2298 2287
query89 383 356 331 331
query90 2053 162 158 158
query91 178 162 140 140
query92 74 68 67 67
query93 1106 940 564 564
query94 604 323 292 292
query95 597 327 359 327
query96 608 481 210 210
query97 2338 2367 2264 2264
query98 247 206 198 198
query99 577 582 507 507
Total cold run time: 253918 ms
Total hot run time: 175079 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3707936363
<details>
<summary>TPC-DS: <b>Total hot run time: 173216 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit dce6dff1cb764b1b9c6337941c86b2b677ddc561, data reload: false
query5 5900 597 443 443
query6 336 234 205 205
query7 4231 458 277 277
query8 362 246 232 232
query9 8822 2600 2647 2600
query10 575 353 333 333
query11 15285 15218 14849 14849
query12 189 119 117 117
query13 1246 480 383 383
query14 7789 3088 2870 2870
query14_1 2718 2726 2723 2723
query15 264 193 177 177
query16 981 475 380 380
query17 1265 657 548 548
query18 2707 419 337 337
query19 295 222 211 211
query20 131 119 119 119
query21 219 137 117 117
query22 3816 3972 3786 3786
query23 16012 15539 15360 15360
query23_1 15401 15492 15346 15346
query24 6992 1531 1164 1164
query24_1 1206 1170 1205 1170
query25 526 442 393 393
query26 1221 259 161 161
query27 2708 461 298 298
query28 4567 2143 2152 2143
query29 760 556 425 425
query30 312 239 214 214
query31 779 653 540 540
query32 75 69 69 69
query33 526 330 286 286
query34 877 883 528 528
query35 775 807 695 695
query36 905 913 774 774
query37 129 106 79 79
query38 2690 2729 2692 2692
query39 778 770 728 728
query39_1 708 699 716 699
query40 213 136 118 118
query41 67 65 67 65
query42 108 106 102 102
query43 444 440 391 391
query44 1321 724 736 724
query45 188 184 177 177
query46 877 952 593 593
query47 1330 1404 1396 1396
query48 312 329 245 245
query49 610 445 341 341
query50 634 277 215 215
query51 3773 3757 3828 3757
query52 107 113 103 103
query53 305 328 279 279
query54 285 262 252 252
query55 83 74 72 72
query56 298 291 310 291
query57 1010 983 927 927
query58 272 258 271 258
query59 2089 2244 2192 2192
query60 350 335 314 314
query61 190 191 188 188
query62 376 365 316 316
query63 302 275 278 275
query64 4973 1438 1110 1110
query65 3813 3745 3755 3745
query66 1375 422 322 322
query67 15206 15174 15566 15174
query68 3537 1065 743 743
query69 505 368 320 320
query70 1025 936 956 936
query71 372 321 290 290
query72 6543 3544 3471 3471
query73 755 731 304 304
query74 8774 8779 8621 8621
query75 2827 2793 2468 2468
query76 3634 1084 683 683
query77 528 374 283 283
query78 9718 10050 9236 9236
query79 1075 848 590 590
query80 1221 558 476 476
query81 551 265 237 237
query82 407 145 110 110
query83 371 257 246 246
query84 260 118 105 105
query85 947 507 460 460
query86 468 322 287 287
query87 2919 2969 2759 2759
query88 3173 2234 2240 2234
query89 379 341 324 324
query90 1907 151 149 149
query91 173 168 148 148
query92 74 72 67 67
query93 1041 871 535 535
query94 649 312 281 281
query95 554 384 310 310
query96 580 477 204 204
query97 2323 2378 2291 2291
query98 205 195 197 195
query99 574 603 506 506
Total cold run time: 253449 ms
Total hot run time: 173216 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3708142138
# BE Regression && UT Coverage Report
Increment line coverage `83.33% (10/12)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/59006_dce6dff1cb764b1b9c6337941c86b2b677ddc561_merge/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/59006_dce6dff1cb764b1b9c6337941c86b2b677ddc561_merge/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 72.31% (25171/34810) |
| Line Coverage | 59.02% (264050/447381) |
| Region Coverage | 53.60% (218393/407415) |
| Branch Coverage | 55.38% (94070/169870) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3674524001
<details>
<summary>TPC-DS: <b>Total hot run time: 178451 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit edf5358bca2a4e95429b7eeff9df276a3ad347df, data reload: false
query5 5820 606 456 456
query6 324 233 221 221
query7 4233 469 282 282
query8 317 274 242 242
query9 8768 2562 2578 2562
query10 550 398 334 334
query11 15488 15344 14657 14657
query12 184 118 118 118
query13 1261 532 410 410
query14 7781 3036 2801 2801
query14_1 2705 2719 2694 2694
query15 268 196 182 182
query16 956 494 454 454
query17 1305 747 606 606
query18 2713 459 356 356
query19 294 246 214 214
query20 124 117 113 113
query21 224 137 119 119
query22 3892 4027 3824 3824
query23 16707 16116 15893 15893
query23_1 16057 16085 15957 15957
query24 6993 1643 1225 1225
query24_1 1265 1244 1266 1244
query25 551 474 433 433
query26 1223 272 162 162
query27 2727 455 309 309
query28 4452 2151 2119 2119
query29 805 562 446 446
query30 317 249 216 216
query31 809 717 622 622
query32 77 70 77 70
query33 532 389 289 289
query34 895 920 544 544
query35 807 833 718 718
query36 865 920 838 838
query37 137 93 84 84
query38 2869 2938 2836 2836
query39 781 735 715 715
query39_1 690 693 714 693
query40 222 140 122 122
query41 66 63 62 62
query42 115 104 111 104
query43 442 448 422 422
query44 1379 747 755 747
query45 195 198 188 188
query46 860 1009 644 644
query47 1648 1672 1618 1618
query48 322 339 251 251
query49 631 436 361 361
query50 680 302 221 221
query51 3831 3832 3867 3832
query52 105 111 97 97
query53 327 368 296 296
query54 288 269 255 255
query55 78 75 79 75
query56 324 293 296 293
query57 1141 1142 1070 1070
query58 272 264 254 254
query59 2463 2409 2373 2373
query60 319 325 293 293
query61 162 159 159 159
query62 703 679 671 671
query63 333 302 310 302
query64 4910 1308 1050 1050
query65 4039 3929 3974 3929
query66 1361 428 316 316
query67 15297 15019 14822 14822
query68 3574 1068 754 754
query69 493 354 312 312
query70 1097 994 976 976
query71 385 306 282 282
query72 6069 4913 4891 4891
query73 642 580 314 314
query74 8870 8717 8479 8479
query75 3250 3165 2801 2801
query76 3626 1165 757 757
query77 517 410 313 313
query78 9410 9558 8837 8837
query79 1024 882 609 609
query80 874 648 551 551
query81 503 266 236 236
query82 221 133 107 107
query83 335 260 242 242
query84 253 121 102 102
query85 938 505 526 505
query86 322 310 279 279
query87 3060 3054 3006 3006
query88 3265 2315 2306 2306
query89 477 421 390 390
query90 1799 163 163 163
query91 194 190 163 163
query92 75 76 70 70
query93 995 934 567 567
query94 390 319 296 296
query95 568 403 334 334
query96 613 471 219 219
query97 2297 2331 2246 2246
query98 215 204 191 191
query99 1277 1344 1202 1202
Total cold run time: 256901 ms
Total hot run time: 178451 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3674472832
<details>
<summary>TPC-H: <b>Total hot run time: 36298 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit edf5358bca2a4e95429b7eeff9df276a3ad347df, data reload: false
------ Round 1 ----------------------------------
q1 17586 4388 4092 4092
q2 2046 358 236 236
q3 10152 1304 771 771
q4 10220 846 313 313
q5 7538 2139 1952 1952
q6 201 168 137 137
q7 1037 845 729 729
q8 9371 1425 1156 1156
q9 7062 5329 5364 5329
q10 6779 2385 1989 1989
q11 520 319 301 301
q12 670 753 565 565
q13 17781 3686 3059 3059
q14 295 290 272 272
q15 583 524 525 524
q16 681 701 626 626
q17 680 741 593 593
q18 7590 7460 7989 7460
q19 1000 995 647 647
q20 465 390 270 270
q21 4546 4502 4263 4263
q22 1130 1043 1014 1014
Total cold run time: 107933 ms
Total hot run time: 36298 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4357 4282 4323 4282
q2 351 414 325 325
q3 2359 2768 2526 2526
q4 1480 1959 1412 1412
q5 4551 4756 4476 4476
q6 216 172 127 127
q7 1988 1977 1820 1820
q8 2686 2561 2460 2460
q9 7473 7550 7487 7487
q10 3053 3237 2767 2767
q11 581 496 471 471
q12 657 715 534 534
q13 3304 3695 3057 3057
q14 268 272 256 256
q15 530 497 496 496
q16 622 664 591 591
q17 1116 1299 1359 1299
q18 7337 6995 7081 6995
q19 801 793 821 793
q20 1908 1950 1814 1814
q21 4644 4304 4193 4193
q22 1081 1027 1002 1002
Total cold run time: 51363 ms
Total hot run time: 49183 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "924060929 (via GitHub)" <gi...@apache.org>.
924060929 commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3698273548
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3685106660
# FE UT Coverage Report
Increment line coverage ` 40.91% (72/176)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/59006_634b002f9d3d8bb834725b5c6c8befaa1cbec0e7/fe_increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/59006_634b002f9d3d8bb834725b5c6c8befaa1cbec0e7/fe_report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3666448465
# FE Regression Coverage Report
Increment line coverage ` 64.57% (82/127)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/59006_705b6dde2b67ffdc84e7411a52079faff45009a1_merge_fe/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/59006_705b6dde2b67ffdc84e7411a52079faff45009a1_merge_fe/report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3709171221
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "924060929 (via GitHub)" <gi...@apache.org>.
924060929 merged PR #59006:
URL: https://github.com/apache/doris/pull/59006
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3708237826
# BE Regression && UT Coverage Report
Increment line coverage `83.33% (10/12)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/59006_dce6dff1cb764b1b9c6337941c86b2b677ddc561_merge/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/59006_dce6dff1cb764b1b9c6337941c86b2b677ddc561_merge/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 72.30% (25167/34810) |
| Line Coverage | 59.01% (263991/447381) |
| Region Coverage | 53.59% (218320/407415) |
| Branch Coverage | 55.36% (94040/169870) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3708293926
# BE Regression && UT Coverage Report
Increment line coverage `83.33% (10/12)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/59006_dce6dff1cb764b1b9c6337941c86b2b677ddc561_merge/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/59006_dce6dff1cb764b1b9c6337941c86b2b677ddc561_merge/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 72.30% (25167/34810) |
| Line Coverage | 59.01% (263991/447381) |
| Region Coverage | 53.59% (218320/407415) |
| Branch Coverage | 55.36% (94040/169870) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3668749199
<details>
<summary>TPC-DS: <b>Total hot run time: 162498 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 6e8ee269eef4749bad44a5ff2aa2b2f19b433cb9, data reload: false
query5 4767 628 471 471
query6 380 241 255 241
query7 4225 471 275 275
query8 304 246 259 246
query9 8780 2549 2553 2549
query10 530 376 329 329
query11 15254 15190 15680 15190
query12 187 117 116 116
query13 1394 540 446 446
query14 6516 3186 2973 2973
query14_1 2996 2751 2676 2676
query15 227 208 195 195
query16 1299 510 488 488
query17
query18 2724 518 378 378
query19 228 226 204 204
query20 121 114 113 113
query21 225 142 118 118
query22 4447 4643 4297 4297
query23 17023 16437 16117 16117
query23_1 16580 16252 16602 16252
query24
query24_1
query25
query26 1311 296 156 156
query27 4277 500 350 350
query28 4728 2177 2200 2177
query29 1794
query30 328 237 202 202
query31 1077 687 599 599
query32 73 60 60 60
query33 542 345 277 277
query34 4420 868 530 530
query35 799 801 724 724
query36 925 900 818 818
query37 126 87 77 77
query38 2848 2850 2785 2785
query39 759 750 704 704
query39_1 689 698 700 698
query40
query41 65 57 59 57
query42 99 96 97 96
query43 405 420 394 394
query44 1474 743 727 727
query45 187 191 173 173
query46 961 975 611 611
query47 1661 1727 1581 1581
query48 334 325 245 245
query49
query50
query51 3855 3784 3811 3784
query52 108 105 96 96
query53 317 302 288 288
query54 286 266 243 243
query55 78 78 69 69
query56 296 285 289 285
query57 1130 1130 1077 1077
query58 278 255 249 249
query59 2359 2451 2311 2311
query60 304 308 290 290
query61 157 151 155 151
query62 696 665 618 618
query63 330 293 294 293
query64 6339 1321 1011 1011
query65 4036 3966 3932 3932
query66 1400 435 304 304
query67 15240 15170 14937 14937
query68 5681 1076 729 729
query69 501 340 305 305
query70 1052 999 961 961
query71 399 303 288 288
query72 6238 5040 5097 5040
query73 739 628 305 305
query74 8851 8833 8532 8532
query75
query76 5475 1205 832 832
query77 541 384 297 297
query78
query79 956 1058 586 586
query80
query81 507 257 233 233
query82 206 124 102 102
query83 355 264 253 253
query84 300 123 93 93
query85
query86 322 302 299 299
query87 2918 2946 2878 2878
query88 3116 2301 2269 2269
query89 449 421 386 386
query90 1982 169 149 149
query91 168 168 143 143
query92 71 67 61 61
query93 934 951 554 554
query94 385 303 270 270
query95 561 386 313 313
query96 590 458 204 204
query97 2296 2342 2231 2231
query98 207 189 190 189
query99 1241 1297 1202 1202
Total cold run time: 240422 ms
Total hot run time: 162498 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3669371760
<details>
<summary>TPC-DS: <b>Total hot run time: 163024 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit c4909b8e1ca544e498fca4fa84004b4635bfd425, data reload: false
query5 4397 592 445 445
query6 340 221 200 200
query7 4208 470 287 287
query8 298 259 250 250
query9 8768 2570 2558 2558
query10 533 397 338 338
query11 15256 15201 16260 15201
query12 186 118 118 118
query13 1274 534 428 428
query14 6357 3112 2893 2893
query14_1 2813 2788 2739 2739
query15 217 201 183 183
query16 4090 510 463 463
query17
query18 2471 479 369 369
query19 271 236 224 224
query20 128 120 113 113
query21 242 144 126 126
query22 4556 4453 4249 4249
query23 17005 16544 16515 16515
query23_1 16372 16327 16321 16321
query24
query24_1
query25
query26 1311 285 170 170
query27 5511 469 322 322
query28 4621 2186 2133 2133
query29
query30 317 239 211 211
query31 873 711 616 616
query32 116 72 66 66
query33 571 354 304 304
query34 4492 868 555 555
query35 791 817 733 733
query36 917 902 822 822
query37 153 90 79 79
query38 2880 2951 2854 2854
query39 767 737 716 716
query39_1 695 699 713 699
query40
query41 69 65 63 63
query42 154 96 97 96
query43 419 431 402 402
query44 1490 763 751 751
query45 190 183 179 179
query46 971 970 620 620
query47 1652 1718 1581 1581
query48 323 334 241 241
query49
query50
query51 3745 3768 3765 3765
query52 115 103 92 92
query53 327 303 290 290
query54 285 258 247 247
query55 75 76 75 75
query56 297 295 292 292
query57 1127 1133 1084 1084
query58 269 259 263 259
query59 2375 2376 2336 2336
query60 321 304 284 284
query61 154 152 154 152
query62 707 697 659 659
query63 338 290 301 290
query64 6421 1308 1017 1017
query65 4014 3972 3953 3953
query66 1339 399 325 325
query67 15327 15010 14807 14807
query68 7031 1103 735 735
query69 496 349 312 312
query70 1104 1018 950 950
query71 376 300 296 296
query72 5963 4966 5233 4966
query73 743 663 312 312
query74 8751 8770 8654 8654
query75
query76 4929 1212 784 784
query77 573 383 291 291
query78
query79 929 1058 610 610
query80
query81 526 265 235 235
query82 196 126 105 105
query83 329 262 247 247
query84 298 118 99 99
query85
query86 309 300 279 279
query87 2979 3000 2884 2884
query88 3207 2279 2284 2279
query89 465 414 389 389
query90 1971 160 156 156
query91 181 173 148 148
query92 80 66 63 63
query93 1007 941 561 561
query94 387 300 266 266
query95 568 347 368 347
query96 604 466 208 208
query97 2249 2358 2260 2260
query98 202 195 195 195
query99 1268 1311 1181 1181
Total cold run time: 244185 ms
Total hot run time: 163024 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "924060929 (via GitHub)" <gi...@apache.org>.
924060929 commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3645548693
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3646707901
# FE Regression Coverage Report
Increment line coverage ` 50.91% (56/110)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/59006_48c5f71eb30a8e76717d7f1cfaac067182a0dbcf_merge_fe/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/59006_48c5f71eb30a8e76717d7f1cfaac067182a0dbcf_merge_fe/report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3668768037
<details>
<summary>ClickBench: <b>Total hot run time: 27.7 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 6e8ee269eef4749bad44a5ff2aa2b2f19b433cb9, data reload: false
query1 0.05 0.04 0.04
query2 0.10 0.05 0.05
query3 0.25 0.09 0.09
query4 1.61 0.12 0.11
query5 0.28 0.24 0.25
query6 1.15 0.63 0.64
query7 0.03 0.03 0.02
query8 0.05 0.04 0.04
query9 0.58 0.52 0.52
query10 0.56 0.56 0.56
query11 0.15 0.11 0.11
query12 0.15 0.11 0.13
query13 0.60 0.60 0.60
query14 0.97 0.99 1.00
query15 0.83 0.79 0.80
query16 0.39 0.40 0.40
query17 0.98 1.04 1.07
query18 0.23 0.21 0.22
query19 1.96 1.83 1.87
query20 0.02 0.01 0.01
query21 15.47 0.31 0.15
query22 4.86 0.05 0.04
query23 15.92 0.28 0.10
query24 1.27 0.69 0.62
query25 0.11 0.08 0.05
query26 0.14 0.14 0.13
query27 0.05 0.05 0.05
query28 4.96 1.27 1.02
query29 12.60 4.01 3.26
query30 0.29 0.14 0.12
query31 2.82 0.62 0.39
query32 3.24 0.55 0.45
query33 2.99 3.02 3.00
query34 17.03 5.17 4.59
query35 4.60 4.60 4.51
query36 0.65 0.50 0.49
query37 0.10 0.06 0.07
query38 0.08 0.05 0.04
query39 0.05 0.03 0.03
query40 0.17 0.14 0.13
query41 0.08 0.04 0.03
query42 0.04 0.02 0.02
query43 0.04 0.04 0.03
Total cold run time: 98.5 s
Total hot run time: 27.7 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3669927412
<details>
<summary>TPC-H: <b>Total hot run time: 11837 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 8044532befaa150e270ca4f2f540cbf3f82d0023, data reload: false
------ Round 1 ----------------------------------
revenue0 View NULL NULL -1 0 0 NULL 0 NULL NULL 2023-12-26 18:27:24 NULL NULL utf-8 NULL NULL
partsupp Doris NULL NULL 0 0 0 NULL 0 NULL NULL 2023-12-26 18:27:23 2023-12-26 18:44:20 NULL utf-8 NULL NULL
part Doris NULL NULL 0 0 0 NULL 0 NULL NULL 2023-12-26 18:27:23 2023-12-26 18:27:57 NULL utf-8 NULL NULL
nation Doris NULL NULL 0 0 0 NULL 0 NULL NULL 2023-12-26 18:27:23 2023-12-26 18:27:24 NULL utf-8 NULL NULL
customer Doris NULL NULL 0 0 0 NULL 0 NULL NULL 2023-12-26 18:27:23 2023-12-26 18:27:43 NULL utf-8 NULL NULL
lineitem Doris NULL NULL 0 0 0 NULL 0 NULL NULL 2023-12-26 18:27:23 2023-12-26 18:38:59 NULL utf-8 NULL NULL
supplier Doris NULL NULL 0 0 0 NULL 0 NULL NULL 2023-12-26 18:27:23 2023-12-26 18:27:25 NULL utf-8 NULL NULL
region Doris NULL NULL 0 0 0 NULL 0 NULL NULL 2023-12-26 18:27:23 2023-12-26 18:27:24 NULL utf-8 NULL NULL
orders Doris NULL NULL 0 0 0 NULL 0 NULL NULL 2023-12-26 18:27:23 2023-12-26 18:42:55 NULL utf-8 NULL NULL
============================================
q1 17642 4309 4099 4099
q2 1826 414 318 318
q3 q4 18651 1178 504 504
q5 q6 183 252 141 141
q7 q8 q9 q10 9024 q11 q12 704 640 558 558
q13 17821 3654 3005 3005
q14 340 318 279 279
q15 648 513 525 513
q16 668 q17 9163 919 411 411
q18 q19 884 826 675 675
q20 2248 491 373 373
q21 q22 1120 984 961 961
Total cold run time: 71230 ms
Total hot run time: 11837 ms
----- Round 2, with runtime_filter_mode=off -----
revenue0 View NULL NULL -1 0 0 NULL 0 NULL NULL 2023-12-26 18:27:24 NULL NULL utf-8 NULL NULL
partsupp Doris NULL NULL 80000000 56 4534120086 NULL 44625495 NULL NULL 2023-12-26 18:27:23 2023-12-26 18:44:20 NULL utf-8 NULL NULL
part Doris NULL NULL 20000000 37 748811035 NULL 1935627 NULL NULL 2023-12-26 18:27:23 2023-12-26 18:27:57 NULL utf-8 NULL NULL
nation Doris NULL NULL 25 138 3473 NULL 366 NULL NULL 2023-12-26 18:27:23 2023-12-26 18:27:24 NULL utf-8 NULL NULL
customer Doris NULL NULL 15000000 92 1381653732 NULL 4374759 NULL NULL 2023-12-26 18:27:23 2023-12-26 18:27:43 NULL utf-8 NULL NULL
lineitem Doris NULL NULL 600037902 33 19843441616 NULL 61784740 NULL NULL 2023-12-26 18:27:23 2023-12-26 18:38:59 NULL utf-8 NULL NULL
supplier Doris NULL NULL 1000000 87 87519212 NULL 194931 NULL NULL 2023-12-26 18:27:23 2023-12-26 18:27:25 NULL utf-8 NULL NULL
region Doris NULL NULL 5 240 1201 NULL 147 NULL NULL 2023-12-26 18:27:23 2023-12-26 18:27:24 NULL utf-8 NULL NULL
orders Doris NULL NULL 150000000 42 6422171781 NULL 22778155 NULL NULL 2023-12-26 18:27:23 2023-12-26 18:42:55 NULL utf-8 NULL NULL
============================================
q1 4149 4073 4048 4048
q2 383 423 375 375
q3 q4 1295 1670 1518 1518
q5 q6 157 238 130 130
q7 q8 q9 q10 q11 q12 619 673 552 552
q13 3266 3648 3000 3000
q14 278 289 269 269
q15 564 516 524 516
q16 q17 1160 1382 1353 1353
q18 q19 791 1037 1001 1001
q20 2023 2288 2210 2210
q21 q22 1194 1168 1063 1063
Total cold run time: 15879 ms
Total hot run time: 16035 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3696349598
<details>
<summary>TPC-H: <b>Total hot run time: 35231 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit fc57d6152d27913cfd8bc2196adc9e86dfe43c48, data reload: false
------ Round 1 ----------------------------------
q1 17633 4201 4020 4020
q2 2053 358 235 235
q3 10193 1350 725 725
q4 10227 897 320 320
q5 7548 2120 1918 1918
q6 188 178 139 139
q7 943 784 682 682
q8 9278 1463 1137 1137
q9 6738 5261 5188 5188
q10 6756 1805 1422 1422
q11 521 301 291 291
q12 691 748 630 630
q13 17764 3829 3062 3062
q14 288 323 273 273
q15 574 522 507 507
q16 692 678 615 615
q17 708 719 611 611
q18 7884 7632 8063 7632
q19 1125 1032 646 646
q20 475 385 268 268
q21 4578 4513 3883 3883
q22 1155 1073 1027 1027
Total cold run time: 108012 ms
Total hot run time: 35231 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4225 4232 4239 4232
q2 353 403 330 330
q3 2261 2826 2390 2390
q4 1492 1884 1422 1422
q5 4530 4663 4300 4300
q6 214 171 127 127
q7 2007 1930 1831 1831
q8 2615 2436 2371 2371
q9 7157 7088 7090 7088
q10 2530 2654 2129 2129
q11 520 464 433 433
q12 663 722 593 593
q13 3348 3830 3083 3083
q14 259 275 256 256
q15 517 478 483 478
q16 595 647 597 597
q17 1057 1230 1305 1230
q18 7507 7324 7122 7122
q19 852 852 844 844
q20 1891 1965 1813 1813
q21 4498 4306 4122 4122
q22 1106 1046 980 980
Total cold run time: 50197 ms
Total hot run time: 47771 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3660172977
<details>
<summary>ClickBench: <b>Total hot run time: 27.65 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit fee457e70de22f8e9a52857742b04a88d7b44a45, data reload: false
query1 0.05 0.04 0.04
query2 0.10 0.05 0.05
query3 0.26 0.09 0.09
query4 1.61 0.12 0.11
query5 0.26 0.26 0.25
query6 1.18 0.64 0.64
query7 0.03 0.02 0.03
query8 0.05 0.04 0.04
query9 0.58 0.52 0.50
query10 0.57 0.54 0.55
query11 0.17 0.11 0.12
query12 0.16 0.12 0.12
query13 0.64 0.61 0.60
query14 1.01 1.00 0.99
query15 0.82 0.80 0.82
query16 0.41 0.40 0.40
query17 1.05 1.04 0.98
query18 0.23 0.21 0.22
query19 1.96 1.78 1.89
query20 0.02 0.01 0.01
query21 15.49 0.32 0.14
query22 4.69 0.06 0.05
query23 15.99 0.29 0.11
query24 1.76 0.67 0.70
query25 0.12 0.09 0.09
query26 0.14 0.14 0.13
query27 0.06 0.06 0.07
query28 6.35 1.24 1.04
query29 12.64 4.17 3.22
query30 0.28 0.14 0.12
query31 2.82 0.63 0.40
query32 3.24 0.55 0.45
query33 3.08 2.98 3.05
query34 16.96 5.18 4.53
query35 4.61 4.58 4.56
query36 0.68 0.52 0.49
query37 0.11 0.07 0.06
query38 0.07 0.05 0.04
query39 0.04 0.04 0.03
query40 0.16 0.14 0.13
query41 0.09 0.03 0.02
query42 0.04 0.02 0.02
query43 0.04 0.03 0.03
Total cold run time: 100.62 s
Total hot run time: 27.65 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3664383769
<details>
<summary>TPC-H: <b>Total hot run time: 35905 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit d35f41b95e5be5f12fba46b647fc1ce7dfebbf6f, data reload: false
------ Round 1 ----------------------------------
q1 17639 4191 4052 4052
q2 2044 378 234 234
q3 10162 1383 735 735
q4 10225 836 313 313
q5 7532 2133 1879 1879
q6 186 167 134 134
q7 1001 844 716 716
q8 9365 1417 1123 1123
q9 6990 5358 5368 5358
q10 6783 2387 1980 1980
q11 521 326 300 300
q12 654 717 578 578
q13 17765 3678 3059 3059
q14 297 291 272 272
q15 561 516 512 512
q16 700 700 609 609
q17 696 888 495 495
q18 7311 7517 8113 7517
q19 1156 1024 665 665
q20 454 394 254 254
q21 4640 4293 4080 4080
q22 1138 1091 1040 1040
Total cold run time: 107820 ms
Total hot run time: 35905 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4304 4237 4336 4237
q2 333 385 307 307
q3 2335 2948 2435 2435
q4 1370 1829 1381 1381
q5 4455 4466 4604 4466
q6 208 174 126 126
q7 2007 1923 1789 1789
q8 2771 2549 2520 2520
q9 7591 7448 7490 7448
q10 3035 3315 2829 2829
q11 605 511 500 500
q12 735 683 564 564
q13 3337 3672 3011 3011
q14 266 280 254 254
q15 534 497 503 497
q16 613 648 593 593
q17 1097 1393 1364 1364
q18 7421 7081 6943 6943
q19 858 802 827 802
q20 1890 1946 1812 1812
q21 4611 4341 4256 4256
q22 1101 1028 980 980
Total cold run time: 51477 ms
Total hot run time: 49114 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3663831754
# FE Regression Coverage Report
Increment line coverage ` 64.57% (82/127)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/59006_0b794933568c7e451edfdc183a3960fdf41fce7a_merge_fe/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/59006_0b794933568c7e451edfdc183a3960fdf41fce7a_merge_fe/report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "924060929 (via GitHub)" <gi...@apache.org>.
924060929 commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3669048594
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3675174983
# FE Regression Coverage Report
Increment line coverage ` 54.55% (96/176)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/59006_edf5358bca2a4e95429b7eeff9df276a3ad347df_merge_fe/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/59006_edf5358bca2a4e95429b7eeff9df276a3ad347df_merge_fe/report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3674022422
<details>
<summary>TPC-H: <b>Total hot run time: 35576 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit d9fde14000f127e1c3ec4d3a00fe36a46b215e82, data reload: false
------ Round 1 ----------------------------------
q1 17621 4253 4093 4093
q2 2022 355 239 239
q3 10165 1334 753 753
q4 10210 830 318 318
q5 7512 2117 2015 2015
q6 187 170 141 141
q7 1019 850 717 717
q8 9356 1508 1302 1302
q9 6956 5324 5371 5324
q10 6836 2381 2006 2006
q11 562 351 306 306
q12 681 746 582 582
q13 17821 3667 3048 3048
q14 302 294 273 273
q15 604 517 514 514
q16 708 676 634 634
q17 695 837 568 568
q18 7587 7065 7126 7065
q19 1161 960 624 624
q20 403 376 248 248
q21 4251 4062 3849 3849
q22 1010 999 957 957
Total cold run time: 107669 ms
Total hot run time: 35576 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4152 4089 4103 4089
q2 331 406 327 327
q3 2149 2711 2311 2311
q4 1325 1771 1269 1269
q5 4261 4655 4741 4655
q6 227 183 138 138
q7 2074 2024 1848 1848
q8 2756 2543 2531 2531
q9 7441 7436 7545 7436
q10 3066 3263 2882 2882
q11 605 495 490 490
q12 696 828 682 682
q13 3576 3973 3361 3361
q14 305 326 290 290
q15 556 512 531 512
q16 696 740 649 649
q17 1210 1453 1388 1388
q18 7909 7765 7682 7682
q19 922 853 834 834
q20 1913 2023 1862 1862
q21 4626 4238 4217 4217
q22 1082 1061 982 982
Total cold run time: 51878 ms
Total hot run time: 50435 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "924060929 (via GitHub)" <gi...@apache.org>.
924060929 commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3684970272
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3685031060
<details>
<summary>TPC-H: <b>Total hot run time: 35309 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 634b002f9d3d8bb834725b5c6c8befaa1cbec0e7, data reload: false
------ Round 1 ----------------------------------
q1 17606 4257 4090 4090
q2 2015 355 231 231
q3 10187 1384 743 743
q4 10219 883 318 318
q5 7494 2148 1943 1943
q6 187 175 139 139
q7 1016 858 723 723
q8 9378 1463 1156 1156
q9 7047 5379 5339 5339
q10 6794 2410 1994 1994
q11 539 344 316 316
q12 659 733 563 563
q13 17781 3703 3037 3037
q14 299 295 278 278
q15 578 514 510 510
q16 743 688 659 659
q17 686 791 526 526
q18 7542 7227 7048 7048
q19 1099 973 615 615
q20 399 367 255 255
q21 4271 4006 3873 3873
q22 1074 1002 953 953
Total cold run time: 107613 ms
Total hot run time: 35309 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4161 4126 4137 4126
q2 344 404 310 310
q3 2133 2703 2309 2309
q4 1359 1783 1258 1258
q5 4292 4928 4761 4761
q6 226 175 129 129
q7 2074 1946 1853 1853
q8 2733 2623 2590 2590
q9 7851 7486 7502 7486
q10 3021 3280 2834 2834
q11 635 567 537 537
q12 690 813 602 602
q13 3554 4007 3379 3379
q14 293 315 281 281
q15 548 501 505 501
q16 668 743 704 704
q17 1221 1396 1368 1368
q18 8118 7854 7737 7737
q19 877 875 923 875
q20 2064 2098 1886 1886
q21 4892 4655 4269 4269
q22 1081 1027 1009 1009
Total cold run time: 52835 ms
Total hot run time: 50804 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3685053552
<details>
<summary>TPC-DS: <b>Total hot run time: 180053 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 634b002f9d3d8bb834725b5c6c8befaa1cbec0e7, data reload: false
query5 5094 620 452 452
query6 325 245 219 219
query7 4230 466 279 279
query8 326 261 256 256
query9 8810 2552 2529 2529
query10 530 369 335 335
query11 15481 15300 15446 15300
query12 197 121 120 120
query13 1269 502 391 391
query14 6710 3081 2759 2759
query14_1 2690 2637 2668 2637
query15 218 194 183 183
query16 920 477 460 460
query17 1127 725 609 609
query18 2720 446 355 355
query19 234 233 210 210
query20 121 119 117 117
query21 225 141 123 123
query22 3957 4059 3982 3982
query23 16824 16363 16064 16064
query23_1 16066 16254 16018 16018
query24 7247 1676 1265 1265
query24_1 1261 1217 1254 1217
query25 589 514 461 461
query26 1248 267 179 179
query27 2742 474 311 311
query28 4460 2172 2156 2156
query29 820 581 476 476
query30 328 232 228 228
query31 809 741 611 611
query32 81 68 70 68
query33 541 326 283 283
query34 905 906 534 534
query35 758 817 715 715
query36 873 925 817 817
query37 127 89 80 80
query38 2967 3027 2989 2989
query39 744 761 712 712
query39_1 702 689 693 689
query40 231 140 121 121
query41 64 65 61 61
query42 105 105 105 105
query43 426 440 401 401
query44 1321 762 739 739
query45 198 197 189 189
query46 874 975 604 604
query47 1689 1703 1637 1637
query48 312 322 246 246
query49 614 441 377 377
query50 657 313 222 222
query51 3834 3883 3757 3757
query52 114 116 102 102
query53 323 352 290 290
query54 282 260 269 260
query55 85 77 71 71
query56 298 303 305 303
query57 1163 1158 1116 1116
query58 276 256 259 256
query59 2496 2538 2451 2451
query60 308 315 291 291
query61 173 181 160 160
query62 731 687 654 654
query63 330 295 298 295
query64 4957 1320 1005 1005
query65 4047 3986 3961 3961
query66 1393 434 324 324
query67 15578 15002 14876 14876
query68 6735 1012 728 728
query69 497 350 312 312
query70 1043 987 994 987
query71 372 311 281 281
query72 6100 4949 4914 4914
query73 655 549 308 308
query74 9079 8956 8682 8682
query75 3231 3173 2811 2811
query76 3973 1133 736 736
query77 524 386 311 311
query78 9382 9535 8961 8961
query79 1608 876 603 603
query80 736 656 558 558
query81 531 264 235 235
query82 205 135 102 102
query83 269 252 237 237
query84 258 116 106 106
query85 911 509 481 481
query86 378 312 277 277
query87 3232 3187 3196 3187
query88 3238 2284 2269 2269
query89 457 420 388 388
query90 2138 160 155 155
query91 174 183 147 147
query92 84 77 69 69
query93 1319 897 565 565
query94 483 299 278 278
query95 572 387 314 314
query96 614 453 209 209
query97 2253 2307 2220 2220
query98 219 196 191 191
query99 1314 1317 1274 1274
Total cold run time: 260732 ms
Total hot run time: 180053 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3685062959
<details>
<summary>ClickBench: <b>Total hot run time: 27.31 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 634b002f9d3d8bb834725b5c6c8befaa1cbec0e7, data reload: false
query1 0.05 0.05 0.05
query2 0.10 0.05 0.06
query3 0.26 0.09 0.09
query4 1.62 0.12 0.11
query5 0.27 0.26 0.27
query6 1.18 0.63 0.63
query7 0.04 0.03 0.02
query8 0.06 0.04 0.04
query9 0.56 0.51 0.50
query10 0.56 0.56 0.56
query11 0.15 0.11 0.12
query12 0.16 0.12 0.12
query13 0.63 0.61 0.60
query14 1.02 0.99 1.01
query15 0.81 0.80 0.82
query16 0.43 0.39 0.39
query17 1.06 0.99 1.03
query18 0.23 0.21 0.21
query19 1.88 1.80 1.82
query20 0.02 0.01 0.01
query21 15.46 0.28 0.14
query22 4.83 0.05 0.05
query23 16.11 0.28 0.11
query24 0.97 0.48 0.19
query25 0.08 0.09 0.07
query26 0.15 0.13 0.13
query27 0.07 0.07 0.05
query28 3.57 1.24 1.03
query29 12.61 3.97 3.29
query30 0.28 0.15 0.11
query31 2.82 0.63 0.40
query32 3.24 0.53 0.45
query33 3.04 3.10 3.01
query34 16.98 5.16 4.56
query35 4.57 4.68 4.54
query36 0.66 0.52 0.49
query37 0.11 0.07 0.06
query38 0.07 0.04 0.04
query39 0.05 0.03 0.03
query40 0.17 0.14 0.14
query41 0.09 0.03 0.03
query42 0.04 0.03 0.03
query43 0.04 0.04 0.03
Total cold run time: 97.1 s
Total hot run time: 27.31 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3708305330
# FE Regression Coverage Report
Increment line coverage ` 74.87% (140/187)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/59006_dce6dff1cb764b1b9c6337941c86b2b677ddc561_merge_fe/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/59006_dce6dff1cb764b1b9c6337941c86b2b677ddc561_merge_fe/report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "924060929 (via GitHub)" <gi...@apache.org>.
924060929 commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3659053784
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3660624392
# FE Regression Coverage Report
Increment line coverage ` 66.07% (74/112)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/59006_fee457e70de22f8e9a52857742b04a88d7b44a45_merge_fe/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/59006_fee457e70de22f8e9a52857742b04a88d7b44a45_merge_fe/report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3663559461
<details>
<summary>ClickBench: <b>Total hot run time: 27.93 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 0b794933568c7e451edfdc183a3960fdf41fce7a, data reload: false
query1 0.06 0.05 0.05
query2 0.11 0.05 0.06
query3 0.28 0.09 0.08
query4 1.61 0.12 0.12
query5 0.27 0.25 0.25
query6 1.17 0.66 0.64
query7 0.03 0.03 0.03
query8 0.06 0.05 0.05
query9 0.59 0.51 0.52
query10 0.55 0.56 0.55
query11 0.16 0.12 0.12
query12 0.15 0.12 0.13
query13 0.62 0.61 0.61
query14 0.99 1.00 0.98
query15 0.82 0.80 0.81
query16 0.41 0.38 0.42
query17 1.01 1.01 1.02
query18 0.23 0.22 0.21
query19 1.95 1.81 1.88
query20 0.02 0.01 0.01
query21 15.50 0.28 0.15
query22 4.81 0.05 0.05
query23 15.86 0.27 0.11
query24 1.33 1.41 0.75
query25 0.08 0.08 0.06
query26 0.13 0.14 0.13
query27 0.07 0.07 0.05
query28 4.93 1.20 1.03
query29 12.58 4.01 3.25
query30 0.32 0.17 0.13
query31 2.84 0.63 0.40
query32 3.23 0.55 0.45
query33 3.01 3.07 3.08
query34 16.92 5.14 4.51
query35 4.53 4.55 4.59
query36 0.69 0.50 0.50
query37 0.11 0.07 0.07
query38 0.08 0.04 0.04
query39 0.05 0.03 0.04
query40 0.18 0.14 0.13
query41 0.10 0.03 0.03
query42 0.05 0.03 0.03
query43 0.05 0.04 0.03
Total cold run time: 98.54 s
Total hot run time: 27.93 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3655272196
<details>
<summary>TPC-H: <b>Total hot run time: 36722 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 3d8029fabf02b4ce727baa508757e689dcbe2d47, data reload: false
------ Round 1 ----------------------------------
q1 17660 4242 4161 4161
q2 2059 354 243 243
q3 10136 1306 749 749
q4 10234 938 327 327
q5 7490 2141 1989 1989
q6 185 166 135 135
q7 1012 864 731 731
q8 9363 1513 1144 1144
q9 6931 5344 5398 5344
q10 6798 2415 1976 1976
q11 527 336 300 300
q12 677 713 574 574
q13 17762 3678 3074 3074
q14 300 306 271 271
q15 596 517 520 517
q16 699 674 627 627
q17 682 838 567 567
q18 8098 7848 7988 7848
q19 1101 1019 638 638
q20 428 384 296 296
q21 4820 4194 4259 4194
q22 1136 1017 1054 1017
Total cold run time: 108694 ms
Total hot run time: 36722 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4420 4321 4327 4321
q2 326 425 362 362
q3 2392 2789 2324 2324
q4 1475 1909 1390 1390
q5 4599 4531 4613 4531
q6 221 167 128 128
q7 2045 1958 1827 1827
q8 2636 2567 2554 2554
q9 7522 7478 7598 7478
q10 3109 3262 2803 2803
q11 584 519 491 491
q12 664 744 567 567
q13 3273 3699 3074 3074
q14 263 281 263 263
q15 529 494 492 492
q16 615 659 598 598
q17 1141 1362 1383 1362
q18 7345 7028 6981 6981
q19 866 843 870 843
q20 1973 2027 1794 1794
q21 4645 4253 4170 4170
q22 1053 1023 992 992
Total cold run time: 51696 ms
Total hot run time: 49345 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3659473053
<details>
<summary>TPC-DS: <b>Total hot run time: 178675 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 857a966eb4535c85fb9ea60e8e310e1f890f26e6, data reload: false
query5 4444 588 444 444
query6 335 235 221 221
query7 4226 464 273 273
query8 319 259 243 243
query9 8780 2488 2536 2488
query10 510 378 325 325
query11 15368 15207 14956 14956
query12 179 119 124 119
query13 1262 510 389 389
query14 6141 3020 2727 2727
query14_1 2634 2615 2634 2615
query15 211 197 183 183
query16 884 475 437 437
query17 1146 715 616 616
query18 2496 446 356 356
query19 234 229 206 206
query20 127 118 111 111
query21 221 137 121 121
query22 3999 3974 3873 3873
query23 16662 16193 16124 16124
query23_1 16130 16053 16224 16053
query24 7376 1689 1218 1218
query24_1 1250 1221 1270 1221
query25 600 512 439 439
query26 1261 270 157 157
query27 2774 464 311 311
query28 4486 2125 2117 2117
query29 862 574 466 466
query30 321 255 215 215
query31 867 731 664 664
query32 79 67 73 67
query33 539 329 290 290
query34 931 908 529 529
query35 788 836 714 714
query36 860 909 839 839
query37 126 90 79 79
query38 2884 2885 2866 2866
query39 766 746 724 724
query39_1 722 704 709 704
query40 223 135 116 116
query41 80 65 61 61
query42 105 105 108 105
query43 436 427 409 409
query44 1319 742 745 742
query45 188 195 182 182
query46 877 985 613 613
query47 1679 1720 1638 1638
query48 311 320 245 245
query49 675 421 351 351
query50 652 294 218 218
query51 3823 3797 3827 3797
query52 105 110 96 96
query53 321 348 287 287
query54 281 260 251 251
query55 87 76 72 72
query56 289 294 298 294
query57 1167 1178 1094 1094
query58 278 248 245 245
query59 2425 2445 2394 2394
query60 305 299 286 286
query61 192 158 161 158
query62 723 670 618 618
query63 326 288 293 288
query64 5106 1296 1003 1003
query65 3987 3960 3954 3954
query66 1489 447 318 318
query67 15550 15078 14781 14781
query68 7634 1013 715 715
query69 504 353 313 313
query70 1080 1031 967 967
query71 371 311 276 276
query72 6033 5009 4999 4999
query73 698 595 306 306
query74 8920 8826 8622 8622
query75 3201 3159 2811 2811
query76 3896 1141 728 728
query77 526 386 293 293
query78 9514 9731 8933 8933
query79 1813 841 598 598
query80 1363 658 557 557
query81 543 268 234 234
query82 418 131 103 103
query83 267 253 235 235
query84 258 119 110 110
query85 961 520 475 475
query86 350 282 304 282
query87 2999 3011 2962 2962
query88 3225 2282 2295 2282
query89 473 424 389 389
query90 2008 154 155 154
query91 176 169 152 152
query92 73 65 62 62
query93 1122 885 562 562
query94 547 301 282 282
query95 573 380 309 309
query96 594 464 211 211
query97 2328 2326 2226 2226
query98 212 195 188 188
query99 1305 1298 1182 1182
Total cold run time: 260792 ms
Total hot run time: 178675 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "924060929 (via GitHub)" <gi...@apache.org>.
924060929 commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3669860431
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3670988332
# FE Regression Coverage Report
Increment line coverage ` 54.80% (97/177)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/59006_8044532befaa150e270ca4f2f540cbf3f82d0023_merge_fe/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/59006_8044532befaa150e270ca4f2f540cbf3f82d0023_merge_fe/report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3709171086
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "924060929 (via GitHub)" <gi...@apache.org>.
924060929 commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3707906136
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3707940154
<details>
<summary>ClickBench: <b>Total hot run time: 26.85 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit dce6dff1cb764b1b9c6337941c86b2b677ddc561, data reload: false
query1 0.05 0.04 0.04
query2 0.11 0.05 0.04
query3 0.26 0.09 0.08
query4 1.60 0.12 0.11
query5 0.27 0.27 0.26
query6 1.15 0.66 0.64
query7 0.04 0.03 0.03
query8 0.06 0.04 0.04
query9 0.56 0.50 0.50
query10 0.54 0.52 0.54
query11 0.14 0.10 0.10
query12 0.15 0.11 0.11
query13 0.59 0.58 0.59
query14 0.95 0.95 0.95
query15 0.79 0.78 0.78
query16 0.39 0.38 0.41
query17 1.01 1.01 0.98
query18 0.22 0.21 0.21
query19 1.86 1.84 1.87
query20 0.02 0.01 0.01
query21 15.47 0.25 0.13
query22 5.17 0.05 0.04
query23 15.93 0.27 0.10
query24 0.99 0.23 0.23
query25 0.13 0.06 0.06
query26 0.13 0.14 0.14
query27 0.05 0.06 0.06
query28 3.26 1.06 0.89
query29 12.55 3.94 3.18
query30 0.27 0.13 0.12
query31 2.83 0.64 0.40
query32 3.23 0.57 0.47
query33 2.95 3.02 3.01
query34 16.88 5.13 4.49
query35 4.47 4.53 4.51
query36 0.66 0.50 0.48
query37 0.11 0.07 0.06
query38 0.08 0.04 0.04
query39 0.04 0.02 0.02
query40 0.17 0.15 0.14
query41 0.09 0.03 0.03
query42 0.04 0.02 0.02
query43 0.04 0.04 0.03
Total cold run time: 96.3 s
Total hot run time: 26.85 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3708044140
# BE Regression && UT Coverage Report
Increment line coverage `83.33% (10/12)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/59006_dce6dff1cb764b1b9c6337941c86b2b677ddc561_merge/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/59006_dce6dff1cb764b1b9c6337941c86b2b677ddc561_merge/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 73.99% (25757/34810) |
| Line Coverage | 61.36% (274514/447381) |
| Region Coverage | 56.06% (228390/407415) |
| Branch Coverage | 58.10% (98694/169870) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3645654479
<details>
<summary>TPC-DS: <b>Total hot run time: 174752 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit d0855d6ab9fa3e2950745cce9256b1d472bb5e22, data reload: false
query5
query6 349 229 210 210
query7 4219 421 264 264
query8 316 271 286 271
query9 8914 2563 2568 2563
query10 540 378 336 336
query11 15826 15054 14731 14731
query12 184 120 117 117
query13 1278 514 394 394
query14 6635 3296 3025 3025
query14_1 2892 2915 2906 2906
query15 229 198 186 186
query16 860 488 464 464
query17 1415 751 612 612
query18 2445 450 371 371
query19 259 239 213 213
query20 124 114 121 114
query21 226 135 121 121
query22 3915 3900 3862 3862
query23 16638 16249 15913 15913
query23_1 16219 16125 16088 16088
query24 7500 1671 1201 1201
query24_1 1251 1244 1281 1244
query25 683 509 442 442
query26 1256 289 175 175
query27 2742 479 342 342
query28 4491 2159 2146 2146
query29 840 579 480 480
query30 400 251 217 217
query31 837 709 626 626
query32 79 71 69 69
query33 576 348 341 341
query34 908 904 538 538
query35 792 829 720 720
query36 897 917 847 847
query37 181 88 75 75
query38 2931 2895 2829 2829
query39 765 755 720 720
query39_1 789 696 693 693
query40 226 135 125 125
query41 67 65 62 62
query42 107 103 101 101
query43 437 433 406 406
query44 1326 749 740 740
query45 197 193 183 183
query46 883 999 620 620
query47 1628 1689 1605 1605
query48 333 333 249 249
query49 1050 431 351 351
query50 677 298 217 217
query51 3837 3911 3802 3802
query52 119 111 95 95
query53 313 356 300 300
query54
query55 80 76 73 73
query56 297 299 289 289
query57 1145 1149 1095 1095
query58 280 257 254 254
query59 2396 2584 2425 2425
query60 314 311 298 298
query61 163 155 160 155
query62 751 688 641 641
query63 346 305 301 301
query64 5032 1303 1053 1053
query65 4061 3941 3955 3941
query66 1523 448 336 336
query67 15323 15172 14947 14947
query68 8517 999 741 741
query69 499 359 329 329
query70 1086 985 990 985
query71
query72 6068 4955 5268 4955
query73 724 611 310 310
query74 8545 8871 8582 8582
query75
query76
query77 753 406 290 290
query78 9286 9520 8899 8899
query79 1204 905 622 622
query80 740 659 574 574
query81 490 281 236 236
query82 711 135 103 103
query83 266 265 250 250
query84 251 133 101 101
query85 943 515 459 459
query86 382 288 285 285
query87 3117 3150 3027 3027
query88 3295 2306 2277 2277
query89 478 431 389 389
query90 2144 166 166 166
query91 180 168 147 147
query92 93 65 69 65
query93 1107 955 561 561
query94 602 275 293 275
query95 576 389 311 311
query96 613 469 212 212
query97 2273 2298 2231 2231
query98 213 199 195 195
query99 1313 1371 1232 1232
Total cold run time: 250934 ms
Total hot run time: 174752 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3696474710
# BE UT Coverage Report
Increment line coverage `0.00% (0/2)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/fc57d6152d27913cfd8bc2196adc9e86dfe43c48_fc57d6152d27913cfd8bc2196adc9e86dfe43c48/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/fc57d6152d27913cfd8bc2196adc9e86dfe43c48_fc57d6152d27913cfd8bc2196adc9e86dfe43c48/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 53.38% (18950/35500) |
| Line Coverage | 39.26% (175779/447719) |
| Region Coverage | 33.81% (135984/402157) |
| Branch Coverage | 34.75% (58720/168979) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "924060929 (via GitHub)" <gi...@apache.org>.
924060929 commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3701810416
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "924060929 (via GitHub)" <gi...@apache.org>.
924060929 commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3655199499
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3655339074
<details>
<summary>ClickBench: <b>Total hot run time: 27.21 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 3d8029fabf02b4ce727baa508757e689dcbe2d47, data reload: false
query1 0.06 0.05 0.04
query2 0.09 0.05 0.05
query3 0.26 0.09 0.09
query4 1.60 0.11 0.11
query5 0.28 0.25 0.26
query6 1.17 0.67 0.63
query7 0.03 0.03 0.03
query8 0.06 0.05 0.04
query9 0.58 0.52 0.51
query10 0.56 0.56 0.56
query11 0.17 0.10 0.12
query12 0.14 0.11 0.12
query13 0.64 0.60 0.61
query14 0.99 0.99 0.98
query15 0.80 0.80 0.79
query16 0.38 0.39 0.39
query17 1.07 1.05 1.08
query18 0.24 0.22 0.21
query19 1.88 1.78 1.75
query20 0.02 0.02 0.01
query21 15.45 0.30 0.14
query22 4.74 0.05 0.05
query23 16.24 0.30 0.11
query24 1.41 0.61 0.22
query25 0.08 0.10 0.06
query26 0.14 0.13 0.14
query27 0.06 0.05 0.11
query28 4.22 1.23 1.02
query29 12.63 4.10 3.28
query30 0.28 0.14 0.15
query31 2.81 0.62 0.39
query32 3.24 0.55 0.45
query33 3.00 2.98 3.01
query34 17.00 5.19 4.50
query35 4.61 4.56 4.62
query36 0.66 0.50 0.52
query37 0.11 0.06 0.07
query38 0.07 0.05 0.04
query39 0.05 0.03 0.03
query40 0.17 0.15 0.12
query41 0.09 0.03 0.03
query42 0.04 0.02 0.02
query43 0.04 0.04 0.03
Total cold run time: 98.16 s
Total hot run time: 27.21 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3660507715
# FE UT Coverage Report
Increment line coverage ` 52.68% (59/112)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/59006_fee457e70de22f8e9a52857742b04a88d7b44a45/fe_increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/59006_fee457e70de22f8e9a52857742b04a88d7b44a45/fe_report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "924060929 (via GitHub)" <gi...@apache.org>.
924060929 commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3663498871
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "924060929 (via GitHub)" <gi...@apache.org>.
924060929 commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3664300617
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "924060929 (via GitHub)" <gi...@apache.org>.
924060929 commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3664927495
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3665786531
# FE Regression Coverage Report
Increment line coverage ` 64.57% (82/127)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/59006_705b6dde2b67ffdc84e7411a52079faff45009a1_merge_fe/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/59006_705b6dde2b67ffdc84e7411a52079faff45009a1_merge_fe/report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "924060929 (via GitHub)" <gi...@apache.org>.
924060929 commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3669058554
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "924060929 (via GitHub)" <gi...@apache.org>.
924060929 commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3669872453
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "924060929 (via GitHub)" <gi...@apache.org>.
924060929 commented on code in PR #59006:
URL: https://github.com/apache/doris/pull/59006#discussion_r2650498202
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/properties/ChildrenPropertiesRegulator.java:
##########
@@ -649,19 +667,82 @@ public List<List<PhysicalProperties>> visitPhysicalSetOperation(PhysicalSetOpera
} else if (requiredDistributionSpec instanceof DistributionSpecHash) {
// TODO: should use the most common hash spec as basic
DistributionSpecHash basic = (DistributionSpecHash) requiredDistributionSpec;
- for (int i = 0; i < originChildrenProperties.size(); i++) {
- DistributionSpecHash current
- = (DistributionSpecHash) originChildrenProperties.get(i).getDistributionSpec();
- if (current.getShuffleType() != ShuffleType.EXECUTION_BUCKETED
- || !bothSideShuffleKeysAreSameOrder(basic, current,
- (DistributionSpecHash) requiredProperties.get(0).getDistributionSpec(),
- (DistributionSpecHash) requiredProperties.get(i).getDistributionSpec())) {
+ int distributeToChildIndex = -1;
+ double basicBuckets = -1;
+ double basicRowCount = -1;
+
+ try {
+ ImmutableSet<ShuffleType> supportedShuffleTypes = ImmutableSet.of(
+ ShuffleType.NATURAL,
+ ShuffleType.STORAGE_BUCKETED
+ );
+ // find the most (bucket num, rowCount) side as the basic
+ for (int i = 0; i < originChildrenProperties.size(); i++) {
+ PhysicalProperties originChildrenProperty = originChildrenProperties.get(i);
+ DistributionSpec childDistribution = originChildrenProperty.getDistributionSpec();
+ if (childDistribution instanceof DistributionSpecHash
+ && supportedShuffleTypes.contains(
+ ((DistributionSpecHash) childDistribution).getShuffleType())) {
Review Comment:
if not exists NATURAL and STORAGE_BUCKETED, all children must be EXECUTION_BUCKETED, and the distributeToChildIndex will be -1, in this case, we will use EXECUTION_BUCKETED shuffle in [this branch](https://github.com/apache/doris/pull/59006/files/634b002f9d3d8bb834725b5c6c8befaa1cbec0e7#diff-79d8abdb9d7855b95a985ffa9b4ddbe1c646b58245fc2d0c02bef33374743777R731-R745)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3696370994
<details>
<summary>TPC-DS: <b>Total hot run time: 172917 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit fc57d6152d27913cfd8bc2196adc9e86dfe43c48, data reload: false
query5 4898 583 448 448
query6 324 218 205 205
query7 4211 457 268 268
query8 320 241 230 230
query9 8772 2625 2601 2601
query10 544 350 308 308
query11 15131 15203 14872 14872
query12 179 118 118 118
query13 1255 512 374 374
query14 6470 2946 2753 2753
query14_1 2677 2581 2654 2581
query15 212 187 176 176
query16 1009 463 444 444
query17 1073 702 605 605
query18 2672 454 362 362
query19 229 225 198 198
query20 129 120 117 117
query21 219 147 117 117
query22 4004 4198 4070 4070
query23 16165 15683 15434 15434
query23_1 15449 15353 15452 15353
query24 7449 1612 1215 1215
query24_1 1224 1223 1189 1189
query25 566 482 447 447
query26 1234 270 167 167
query27 2743 440 292 292
query28 4526 2172 2174 2172
query29 751 521 421 421
query30 315 235 212 212
query31 784 624 569 569
query32 76 66 65 65
query33 524 337 270 270
query34 906 930 531 531
query35 745 784 713 713
query36 873 900 829 829
query37 122 85 73 73
query38 2712 2767 2644 2644
query39 779 741 751 741
query39_1 701 694 711 694
query40 220 135 120 120
query41 67 63 65 63
query42 104 121 106 106
query43 435 469 419 419
query44 1315 753 761 753
query45 186 184 176 176
query46 876 977 614 614
query47 1329 1470 1364 1364
query48 309 316 280 280
query49 603 416 321 321
query50 633 278 217 217
query51 4031 3812 3746 3746
query52 106 108 98 98
query53 306 342 270 270
query54 295 257 245 245
query55 78 74 73 73
query56 284 283 289 283
query57 1014 1023 944 944
query58 265 258 247 247
query59 2186 2177 2120 2120
query60 319 309 280 280
query61 161 159 157 157
query62 406 369 324 324
query63 304 263 272 263
query64 4876 1308 975 975
query65 3774 3543 3698 3543
query66 1387 422 316 316
query67 15145 15550 15386 15386
query68 4733 1020 730 730
query69 498 344 302 302
query70 1044 912 938 912
query71 352 297 259 259
query72 6345 5011 2796 2796
query73 727 635 320 320
query74 8780 8771 8625 8625
query75 2890 2885 2523 2523
query76 3896 1047 637 637
query77 510 378 272 272
query78 9776 9796 9153 9153
query79 1485 978 608 608
query80 671 588 501 501
query81 508 266 229 229
query82 353 147 109 109
query83 263 259 235 235
query84 249 124 105 105
query85 894 514 453 453
query86 342 324 290 290
query87 2858 2866 2752 2752
query88 3311 2298 2289 2289
query89 379 356 334 334
query90 1969 162 152 152
query91 184 168 139 139
query92 72 65 63 63
query93 1119 938 559 559
query94 575 321 290 290
query95 578 379 301 301
query96 590 455 203 203
query97 2301 2388 2301 2301
query98 222 199 202 199
query99 571 588 597 588
Total cold run time: 251981 ms
Total hot run time: 172917 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3698447690
<details>
<summary>ClickBench: <b>Total hot run time: 27.19 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit b53cd4a552d1b04e8db7674ee303144185799fa6, data reload: false
query1 0.05 0.04 0.04
query2 0.10 0.05 0.04
query3 0.25 0.08 0.09
query4 1.62 0.12 0.11
query5 0.27 0.26 0.25
query6 1.15 0.66 0.65
query7 0.03 0.03 0.02
query8 0.05 0.05 0.04
query9 0.57 0.52 0.49
query10 0.56 0.55 0.56
query11 0.15 0.11 0.11
query12 0.16 0.12 0.13
query13 0.62 0.60 0.60
query14 0.99 0.98 0.99
query15 0.81 0.79 0.80
query16 0.42 0.38 0.43
query17 1.08 1.05 1.04
query18 0.22 0.21 0.20
query19 1.94 1.80 1.90
query20 0.02 0.02 0.01
query21 15.44 0.30 0.14
query22 4.63 0.05 0.05
query23 16.02 0.27 0.10
query24 1.21 0.57 0.82
query25 0.12 0.19 0.06
query26 0.14 0.14 0.13
query27 0.07 0.06 0.05
query28 5.12 1.04 0.87
query29 12.58 3.99 3.25
query30 0.28 0.14 0.11
query31 2.81 0.64 0.37
query32 3.22 0.55 0.46
query33 2.95 2.99 3.02
query34 16.75 5.11 4.47
query35 4.50 4.48 4.47
query36 0.65 0.49 0.48
query37 0.11 0.06 0.06
query38 0.07 0.04 0.03
query39 0.04 0.02 0.02
query40 0.17 0.14 0.12
query41 0.09 0.04 0.03
query42 0.04 0.03 0.03
query43 0.04 0.04 0.03
Total cold run time: 98.11 s
Total hot run time: 27.19 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3696663664
# BE Regression && UT Coverage Report
Increment line coverage `100.00% (2/2)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/59006_fc57d6152d27913cfd8bc2196adc9e86dfe43c48_merge/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/59006_fc57d6152d27913cfd8bc2196adc9e86dfe43c48_merge/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 72.17% (25045/34705) |
| Line Coverage | 58.91% (263054/446527) |
| Region Coverage | 53.88% (218912/406322) |
| Branch Coverage | 55.37% (93870/169543) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3698800994
# BE Regression && UT Coverage Report
Increment line coverage `100.00% (2/2)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/59006_b53cd4a552d1b04e8db7674ee303144185799fa6_merge/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/59006_b53cd4a552d1b04e8db7674ee303144185799fa6_merge/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 72.17% (25052/34713) |
| Line Coverage | 58.89% (263171/446871) |
| Region Coverage | 53.75% (218527/406591) |
| Branch Coverage | 55.27% (93801/169700) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3685572446
# BE Regression && UT Coverage Report
Increment line coverage `100.00% (2/2)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/59006_634b002f9d3d8bb834725b5c6c8befaa1cbec0e7_merge/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/59006_634b002f9d3d8bb834725b5c6c8befaa1cbec0e7_merge/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 73.95% (25614/34636) |
| Line Coverage | 61.37% (273229/445212) |
| Region Coverage | 56.45% (228800/405325) |
| Branch Coverage | 58.25% (98420/168950) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3701839818
<details>
<summary>TPC-H: <b>Total hot run time: 35258 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit b2e0198031172db18da6bc9fa4273f0ba3c48f7a, data reload: false
------ Round 1 ----------------------------------
q1 17608 4254 4031 4031
q2 2031 361 237 237
q3 10180 1291 762 762
q4 10209 854 319 319
q5 7550 2158 1881 1881
q6 186 168 138 138
q7 951 772 657 657
q8 9300 1433 1157 1157
q9 6769 5185 5241 5185
q10 6760 1824 1421 1421
q11 495 303 282 282
q12 689 725 604 604
q13 17767 3797 3079 3079
q14 286 316 288 288
q15 568 509 510 509
q16 696 684 630 630
q17 715 740 630 630
q18 7334 7497 8060 7497
q19 1111 986 712 712
q20 455 420 263 263
q21 4587 4321 3919 3919
q22 1150 1111 1057 1057
Total cold run time: 107397 ms
Total hot run time: 35258 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4315 4205 4258 4205
q2 333 415 349 349
q3 2511 2828 2442 2442
q4 1415 1884 1418 1418
q5 4402 4357 4527 4357
q6 211 166 124 124
q7 1930 1919 1701 1701
q8 2555 2393 2322 2322
q9 7222 7108 7165 7108
q10 2514 2717 2280 2280
q11 525 487 473 473
q12 770 748 638 638
q13 3395 3762 3051 3051
q14 273 275 255 255
q15 532 486 481 481
q16 622 646 591 591
q17 1051 1209 1240 1209
q18 7317 7415 7222 7222
q19 862 861 873 861
q20 1919 1985 1796 1796
q21 4520 4244 4161 4161
q22 1069 1039 982 982
Total cold run time: 50263 ms
Total hot run time: 48026 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3669326222
<details>
<summary>TPC-H: <b>Total hot run time: 11777 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit c4909b8e1ca544e498fca4fa84004b4635bfd425, data reload: false
------ Round 1 ----------------------------------
revenue0 View NULL NULL -1 0 0 NULL 0 NULL NULL 2023-12-26 18:27:24 NULL NULL utf-8 NULL NULL
partsupp Doris NULL NULL 0 0 0 NULL 0 NULL NULL 2023-12-26 18:27:23 2023-12-26 18:44:20 NULL utf-8 NULL NULL
part Doris NULL NULL 0 0 0 NULL 0 NULL NULL 2023-12-26 18:27:23 2023-12-26 18:27:57 NULL utf-8 NULL NULL
nation Doris NULL NULL 0 0 0 NULL 0 NULL NULL 2023-12-26 18:27:23 2023-12-26 18:27:24 NULL utf-8 NULL NULL
customer Doris NULL NULL 0 0 0 NULL 0 NULL NULL 2023-12-26 18:27:23 2023-12-26 18:27:43 NULL utf-8 NULL NULL
lineitem Doris NULL NULL 0 0 0 NULL 0 NULL NULL 2023-12-26 18:27:23 2023-12-26 18:38:59 NULL utf-8 NULL NULL
supplier Doris NULL NULL 0 0 0 NULL 0 NULL NULL 2023-12-26 18:27:23 2023-12-26 18:27:25 NULL utf-8 NULL NULL
region Doris NULL NULL 0 0 0 NULL 0 NULL NULL 2023-12-26 18:27:23 2023-12-26 18:27:24 NULL utf-8 NULL NULL
orders Doris NULL NULL 0 0 0 NULL 0 NULL NULL 2023-12-26 18:27:23 2023-12-26 18:42:55 NULL utf-8 NULL NULL
============================================
q1 17617 4197 4111 4111
q2 2043 358 240 240
q3 q4 18718 1119 529 529
q5 q6 171 244 137 137
q7 q8 q9 q10 9179 q11 q12 672 642 571 571
q13 17813 3707 3054 3054
q14 337 280 252 252
q15 687 519 516 516
q16 q17 11417 905 406 406
q18 q19 1125 808 679 679
q20 3133 387 250 250
q21 q22 1238 1058 1032 1032
Total cold run time: 74971 ms
Total hot run time: 11777 ms
----- Round 2, with runtime_filter_mode=off -----
revenue0 View NULL NULL -1 0 0 NULL 0 NULL NULL 2023-12-26 18:27:24 NULL NULL utf-8 NULL NULL
partsupp Doris NULL NULL 80000000 56 4534120086 NULL 44625495 NULL NULL 2023-12-26 18:27:23 2023-12-26 18:44:20 NULL utf-8 NULL NULL
part Doris NULL NULL 20000000 37 748811035 NULL 1935627 NULL NULL 2023-12-26 18:27:23 2023-12-26 18:27:57 NULL utf-8 NULL NULL
nation Doris NULL NULL 25 138 3473 NULL 366 NULL NULL 2023-12-26 18:27:23 2023-12-26 18:27:24 NULL utf-8 NULL NULL
customer Doris NULL NULL 15000000 92 1381653732 NULL 4374759 NULL NULL 2023-12-26 18:27:23 2023-12-26 18:27:43 NULL utf-8 NULL NULL
lineitem Doris NULL NULL 600037902 33 19843441616 NULL 61784740 NULL NULL 2023-12-26 18:27:23 2023-12-26 18:38:59 NULL utf-8 NULL NULL
supplier Doris NULL NULL 1000000 87 87519212 NULL 194931 NULL NULL 2023-12-26 18:27:23 2023-12-26 18:27:25 NULL utf-8 NULL NULL
region Doris NULL NULL 5 240 1201 NULL 147 NULL NULL 2023-12-26 18:27:23 2023-12-26 18:27:24 NULL utf-8 NULL NULL
orders Doris NULL NULL 150000000 42 6422171781 NULL 22778155 NULL NULL 2023-12-26 18:27:23 2023-12-26 18:42:55 NULL utf-8 NULL NULL
============================================
q1 4136 4037 4014 4014
q2 339 400 342 342
q3 q4 1359 1703 1516 1516
q5 q6 152 241 129 129
q7 q8 q9 q10 q11 594 q12 626 663 603 603
q13 3274 3668 3051 3051
q14 256 253 252 252
q15 550 521 516 516
q16 636 q17 1159 1361 1392 1361
q18 q19 752 970 1001 970
q20 1874 2013 1828 1828
q21 q22 1118 1078 1020 1020
Total cold run time: 15595 ms
Total hot run time: 15602 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3670803286
# FE UT Coverage Report
Increment line coverage ` 40.68% (72/177)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/59006_8044532befaa150e270ca4f2f540cbf3f82d0023/fe_increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/59006_8044532befaa150e270ca4f2f540cbf3f82d0023/fe_report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3664469570
<details>
<summary>ClickBench: <b>Total hot run time: 27.87 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit d35f41b95e5be5f12fba46b647fc1ce7dfebbf6f, data reload: false
query1 0.06 0.05 0.04
query2 0.10 0.05 0.05
query3 0.27 0.10 0.09
query4 1.61 0.11 0.11
query5 0.26 0.27 0.26
query6 1.17 0.64 0.65
query7 0.03 0.03 0.03
query8 0.05 0.04 0.04
query9 0.59 0.51 0.51
query10 0.55 0.57 0.57
query11 0.16 0.10 0.10
query12 0.15 0.11 0.12
query13 0.62 0.60 0.61
query14 0.99 1.00 0.99
query15 0.82 0.79 0.82
query16 0.42 0.40 0.41
query17 0.98 1.02 1.04
query18 0.23 0.22 0.22
query19 1.93 1.85 1.83
query20 0.02 0.02 0.01
query21 15.48 0.29 0.14
query22 4.86 0.06 0.04
query23 16.06 0.29 0.11
query24 1.20 1.30 0.83
query25 0.08 0.06 0.07
query26 0.14 0.14 0.12
query27 0.07 0.07 0.08
query28 5.19 1.21 1.02
query29 12.66 4.01 3.25
query30 0.27 0.13 0.12
query31 2.82 0.61 0.40
query32 3.23 0.54 0.46
query33 3.01 2.98 3.05
query34 16.76 5.08 4.51
query35 4.57 4.58 4.48
query36 0.68 0.50 0.50
query37 0.11 0.07 0.07
query38 0.06 0.04 0.04
query39 0.04 0.03 0.03
query40 0.17 0.15 0.14
query41 0.09 0.03 0.02
query42 0.04 0.03 0.03
query43 0.04 0.04 0.04
Total cold run time: 98.64 s
Total hot run time: 27.87 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "924060929 (via GitHub)" <gi...@apache.org>.
924060929 commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3668632898
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3655315968
<details>
<summary>TPC-DS: <b>Total hot run time: 166770 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 3d8029fabf02b4ce727baa508757e689dcbe2d47, data reload: false
query5 5881 630 481 481
query6 328 237 218 218
query7 4219 463 276 276
query8
query9 8995 2576 2546 2546
query10 569 398 334 334
query11 15462 14729 14859 14729
query12 185 119 115 115
query13 1271 528 402 402
query14
query14_1
query15 224 211 181 181
query16 1002 527 488 488
query17 2161 680 593 593
query18 2685 430 343 343
query19 346 227 206 206
query20 123 114 111 111
query21 215 137 118 118
query22 3901 3964 3900 3900
query23 16655 16069 15828 15828
query23_1 16032 16016 15853 15853
query24 7152 1653 1254 1254
query24_1 1268 1265 1238 1238
query25 652 475 417 417
query26 1797 268 161 161
query27 2957 458 311 311
query28 5391 2154 2140 2140
query29 942 560 482 482
query30 327 250 213 213
query31 841 730 620 620
query32 79 78 75 75
query33 534 359 314 314
query34 915 919 554 554
query35 800 819 747 747
query36 885 888 824 824
query37 160 100 77 77
query38
query39 794 764 746 746
query39_1 729 718 701 701
query40 230 152 132 132
query41 74 69 67 67
query42 110 105 107 105
query43 436 430 403 403
query44 1349 781 760 760
query45 200 193 180 180
query46 902 1010 633 633
query47 1683 1676 1653 1653
query48 350 357 269 269
query49 657 466 360 360
query50 693 308 229 229
query51 3883 3887 3866 3866
query52 111 120 104 104
query53 324 360 305 305
query54 317 284 276 276
query55 86 73 76 73
query56 312 313 299 299
query57 1133 1173 1088 1088
query58 273 270 264 264
query59 2387 2481 2322 2322
query60 327 325 315 315
query61 188 195 183 183
query62 724 656 641 641
query63 336 318 313 313
query64 4963 1314 1002 1002
query65 4079 3967 3980 3967
query66 1479 427 314 314
query67 15073 14957 14860 14860
query68 7914 1026 742 742
query69 511 351 307 307
query70 1093 1028 970 970
query71 387 350 289 289
query72 6679 4911 5087 4911
query73 672 582 306 306
query74 8669 8818 8592 8592
query75 3169 3159 2784 2784
query76 3762 1160 809 809
query77 525 413 301 301
query78 9438 9418 8784 8784
query79 1714 889 641 641
query80 720 629 550 550
query81 509 272 238 238
query82 201 136 103 103
query83 272 254 246 246
query84 262 123 106 106
query85 896 485 462 462
query86 343 276 292 276
query87
query88 4278 2264 2262 2262
query89 491 434 390 390
query90 2369 158 161 158
query91 175 170 142 142
query92 86 71 67 67
query93 2011 943 555 555
query94 475 298 281 281
query95 555 386 315 315
query96 592 471 218 218
query97 2266 2362 2256 2256
query98 226 211 199 199
query99 1323 1308 1225 1225
Total cold run time: 251588 ms
Total hot run time: 166770 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3659432016
<details>
<summary>TPC-H: <b>Total hot run time: 35327 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 857a966eb4535c85fb9ea60e8e310e1f890f26e6, data reload: false
------ Round 1 ----------------------------------
q1 17610 4227 4025 4025
q2 2033 356 246 246
q3 10156 1305 739 739
q4 10204 870 311 311
q5 7724 2199 1919 1919
q6 229 169 137 137
q7 1007 871 709 709
q8 9366 1477 1164 1164
q9 7263 5317 5397 5317
q10 6854 2403 1970 1970
q11 530 332 299 299
q12 726 723 583 583
q13 17825 3675 3050 3050
q14 288 291 268 268
q15 595 513 515 513
q16 702 673 620 620
q17 707 822 515 515
q18 8237 7294 7221 7221
q19 1110 958 616 616
q20 398 362 245 245
q21 4307 3909 3924 3909
q22 1036 994 951 951
Total cold run time: 108907 ms
Total hot run time: 35327 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4088 4034 4032 4032
q2 329 395 316 316
q3 2137 2685 2315 2315
q4 1330 1753 1294 1294
q5 4248 4772 4657 4657
q6 226 179 130 130
q7 2030 2021 1853 1853
q8 2659 2594 2474 2474
q9 7532 7627 7640 7627
q10 3048 3294 2832 2832
q11 604 512 569 512
q12 710 742 630 630
q13 3644 4141 3189 3189
q14 300 299 283 283
q15 542 501 523 501
q16 676 720 622 622
q17 1239 1455 1436 1436
q18 8060 7565 7705 7565
q19 875 871 913 871
q20 2037 2125 1902 1902
q21 4665 4377 4199 4199
q22 1098 1045 993 993
Total cold run time: 52077 ms
Total hot run time: 50233 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3659493979
<details>
<summary>ClickBench: <b>Total hot run time: 27.18 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 857a966eb4535c85fb9ea60e8e310e1f890f26e6, data reload: false
query1 0.05 0.05 0.05
query2 0.10 0.05 0.04
query3 0.25 0.09 0.08
query4 1.60 0.11 0.12
query5 0.27 0.26 0.25
query6 1.16 0.65 0.62
query7 0.02 0.02 0.02
query8 0.05 0.05 0.04
query9 0.56 0.50 0.51
query10 0.55 0.55 0.55
query11 0.17 0.11 0.10
query12 0.14 0.11 0.12
query13 0.62 0.61 0.61
query14 0.99 1.00 0.97
query15 0.82 0.79 0.81
query16 0.40 0.41 0.42
query17 0.97 1.00 0.99
query18 0.23 0.22 0.22
query19 1.94 1.88 1.80
query20 0.02 0.01 0.01
query21 15.44 0.29 0.15
query22 4.87 0.06 0.05
query23 16.07 0.28 0.10
query24 1.96 0.29 0.64
query25 0.11 0.08 0.05
query26 0.13 0.14 0.13
query27 0.08 0.04 0.06
query28 4.50 1.19 1.02
query29 12.62 3.94 3.18
query30 0.28 0.14 0.11
query31 2.84 0.61 0.39
query32 3.24 0.54 0.47
query33 2.94 3.04 3.11
query34 16.77 5.18 4.52
query35 4.56 4.65 4.53
query36 0.65 0.50 0.48
query37 0.11 0.07 0.06
query38 0.07 0.04 0.04
query39 0.04 0.03 0.04
query40 0.17 0.14 0.13
query41 0.08 0.03 0.03
query42 0.04 0.03 0.03
query43 0.04 0.04 0.04
Total cold run time: 98.52 s
Total hot run time: 27.18 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "924060929 (via GitHub)" <gi...@apache.org>.
924060929 commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3660001198
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3645672205
<details>
<summary>ClickBench: <b>Total hot run time: 27.45 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit d0855d6ab9fa3e2950745cce9256b1d472bb5e22, data reload: false
query1 0.05 0.05 0.04
query2 0.09 0.04 0.04
query3 0.25 0.08 0.08
query4 1.61 0.10 0.12
query5 0.29 0.26 0.28
query6 1.17 0.66 0.65
query7 0.03 0.03 0.02
query8 0.05 0.04 0.04
query9 0.57 0.52 0.50
query10 0.54 0.55 0.56
query11 0.16 0.10 0.12
query12 0.15 0.11 0.12
query13 0.62 0.60 0.60
query14 0.99 0.97 0.98
query15 0.83 0.82 0.81
query16 0.39 0.38 0.39
query17 1.07 1.06 1.03
query18 0.23 0.22 0.22
query19 1.97 1.90 1.81
query20 0.02 0.02 0.01
query21 15.41 0.29 0.15
query22 4.85 0.04 0.04
query23 16.03 0.29 0.11
query24 1.22 0.31 0.41
query25 0.09 0.08 0.08
query26 0.14 0.13 0.14
query27 0.11 0.05 0.04
query28 3.70 1.22 1.02
query29 12.62 4.06 3.39
query30 0.27 0.14 0.11
query31 2.82 0.63 0.41
query32 3.23 0.56 0.45
query33 3.03 3.16 3.02
query34 16.77 5.20 4.51
query35 4.54 4.50 4.57
query36 0.66 0.50 0.51
query37 0.11 0.07 0.06
query38 0.07 0.05 0.04
query39 0.04 0.03 0.03
query40 0.18 0.14 0.14
query41 0.09 0.04 0.03
query42 0.04 0.03 0.03
query43 0.04 0.03 0.03
Total cold run time: 97.14 s
Total hot run time: 27.45 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3646055990
<details>
<summary>ClickBench: <b>Total hot run time: 27.11 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 48c5f71eb30a8e76717d7f1cfaac067182a0dbcf, data reload: false
query1 0.06 0.05 0.05
query2 0.10 0.05 0.05
query3 0.26 0.08 0.08
query4 1.60 0.11 0.11
query5 0.26 0.26 0.26
query6 1.15 0.64 0.63
query7 0.03 0.03 0.03
query8 0.06 0.04 0.04
query9 0.57 0.50 0.50
query10 0.56 0.55 0.56
query11 0.17 0.11 0.11
query12 0.15 0.12 0.11
query13 0.62 0.61 0.60
query14 1.00 0.98 0.99
query15 0.82 0.79 0.81
query16 0.40 0.39 0.41
query17 1.01 0.99 1.01
query18 0.24 0.21 0.21
query19 1.80 1.71 1.76
query20 0.01 0.01 0.01
query21 15.46 0.33 0.14
query22 4.69 0.05 0.05
query23 16.11 0.27 0.10
query24 0.98 0.29 0.24
query25 0.08 0.07 0.06
query26 0.14 0.12 0.14
query27 0.07 0.06 0.06
query28 3.07 1.24 1.03
query29 12.58 4.10 3.21
query30 0.31 0.14 0.13
query31 2.81 0.64 0.39
query32 3.24 0.55 0.46
query33 3.02 3.08 3.03
query34 17.04 5.20 4.51
query35 4.57 4.56 4.58
query36 0.68 0.50 0.49
query37 0.12 0.07 0.06
query38 0.08 0.04 0.04
query39 0.04 0.02 0.03
query40 0.17 0.15 0.12
query41 0.09 0.04 0.03
query42 0.04 0.03 0.03
query43 0.04 0.04 0.03
Total cold run time: 96.3 s
Total hot run time: 27.11 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "morrySnow (via GitHub)" <gi...@apache.org>.
morrySnow commented on code in PR #59006:
URL: https://github.com/apache/doris/pull/59006#discussion_r2645378761
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/algebra/SetOperation.java:
##########
@@ -34,6 +34,14 @@ enum Qualifier {
DISTINCT
}
+ /**
+ * join shuffle type
+ */
+ enum ShuffleType {
+ shuffle,
+ bucketShuffle
Review Comment:
```suggestion
SHUFFLE,
BUCKET_SHUFFLE
```
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/properties/ChildrenPropertiesRegulator.java:
##########
@@ -649,19 +667,82 @@ public List<List<PhysicalProperties>> visitPhysicalSetOperation(PhysicalSetOpera
} else if (requiredDistributionSpec instanceof DistributionSpecHash) {
// TODO: should use the most common hash spec as basic
DistributionSpecHash basic = (DistributionSpecHash) requiredDistributionSpec;
- for (int i = 0; i < originChildrenProperties.size(); i++) {
- DistributionSpecHash current
- = (DistributionSpecHash) originChildrenProperties.get(i).getDistributionSpec();
- if (current.getShuffleType() != ShuffleType.EXECUTION_BUCKETED
- || !bothSideShuffleKeysAreSameOrder(basic, current,
- (DistributionSpecHash) requiredProperties.get(0).getDistributionSpec(),
- (DistributionSpecHash) requiredProperties.get(i).getDistributionSpec())) {
+ int distributeToChildIndex = -1;
+ double basicBuckets = -1;
+ double basicRowCount = -1;
+
+ try {
+ ImmutableSet<ShuffleType> supportedShuffleTypes = ImmutableSet.of(
+ ShuffleType.NATURAL,
+ ShuffleType.STORAGE_BUCKETED
+ );
+ // find the most (bucket num, rowCount) side as the basic
+ for (int i = 0; i < originChildrenProperties.size(); i++) {
+ PhysicalProperties originChildrenProperty = originChildrenProperties.get(i);
+ DistributionSpec childDistribution = originChildrenProperty.getDistributionSpec();
+ if (childDistribution instanceof DistributionSpecHash
+ && supportedShuffleTypes.contains(
+ ((DistributionSpecHash) childDistribution).getShuffleType())) {
+ int bucketNum = getBucketNum(children.get(i).getPlan());
+ Statistics stats = setOperation.child(i).getStats();
+ double rowCount = stats.getRowCount();
Review Comment:
We select the child with the largest number of rows among all children whose bucket count is greater than the BE parallelism.
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/properties/ChildrenPropertiesRegulator.java:
##########
@@ -649,19 +667,82 @@ public List<List<PhysicalProperties>> visitPhysicalSetOperation(PhysicalSetOpera
} else if (requiredDistributionSpec instanceof DistributionSpecHash) {
// TODO: should use the most common hash spec as basic
DistributionSpecHash basic = (DistributionSpecHash) requiredDistributionSpec;
- for (int i = 0; i < originChildrenProperties.size(); i++) {
- DistributionSpecHash current
- = (DistributionSpecHash) originChildrenProperties.get(i).getDistributionSpec();
- if (current.getShuffleType() != ShuffleType.EXECUTION_BUCKETED
- || !bothSideShuffleKeysAreSameOrder(basic, current,
- (DistributionSpecHash) requiredProperties.get(0).getDistributionSpec(),
- (DistributionSpecHash) requiredProperties.get(i).getDistributionSpec())) {
+ int distributeToChildIndex = -1;
+ double basicBuckets = -1;
+ double basicRowCount = -1;
+
+ try {
+ ImmutableSet<ShuffleType> supportedShuffleTypes = ImmutableSet.of(
+ ShuffleType.NATURAL,
+ ShuffleType.STORAGE_BUCKETED
+ );
+ // find the most (bucket num, rowCount) side as the basic
+ for (int i = 0; i < originChildrenProperties.size(); i++) {
+ PhysicalProperties originChildrenProperty = originChildrenProperties.get(i);
+ DistributionSpec childDistribution = originChildrenProperty.getDistributionSpec();
+ if (childDistribution instanceof DistributionSpecHash
+ && supportedShuffleTypes.contains(
+ ((DistributionSpecHash) childDistribution).getShuffleType())) {
Review Comment:
why check this? we should support all shuffle type in this branch
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/properties/RequestPropertyDeriver.java:
##########
@@ -280,13 +280,13 @@ public Void visitPhysicalSetOperation(PhysicalSetOperation setOperation, PlanCon
addRequestPropertyToChildren(createHashRequestAccordingToParent(
setOperation, distributionSpecHash, context));
} else {
- // shuffle all column
- // TODO: for wide table, may be we should add a upper limit of shuffle columns
Review Comment:
why remove this TODO?
##########
fe/fe-core/src/main/java/org/apache/doris/planner/SetOperationNode.java:
##########
@@ -65,6 +65,12 @@ public abstract class SetOperationNode extends PlanNode {
private boolean isColocate = false;
+ private DistributionMode distributionMode = DistributionMode.SHUFFLE;
+
+ public enum DistributionMode {
Review Comment:
should reuse DistributionMode in HashJoinNode, move out DistributionMode from HashJoinNode
##########
fe/fe-core/src/main/java/org/apache/doris/planner/PlanFragment.java:
##########
@@ -177,6 +178,7 @@ public PlanFragment(PlanFragmentId id, PlanNode root, DataPartition partition) {
this.builderRuntimeFilterIds = new HashSet<>();
this.targetRuntimeFilterIds = new HashSet<>();
this.hasBucketShuffleJoin = buildHasBucketShuffleJoin();
+ this.hasBucketShuffleSetOperation = buildHasBucketShuffleSetOperation();
Review Comment:
could we merge them to hasBucketShuffleNode?
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/properties/ChildrenPropertiesRegulator.java:
##########
@@ -649,19 +667,82 @@ public List<List<PhysicalProperties>> visitPhysicalSetOperation(PhysicalSetOpera
} else if (requiredDistributionSpec instanceof DistributionSpecHash) {
// TODO: should use the most common hash spec as basic
DistributionSpecHash basic = (DistributionSpecHash) requiredDistributionSpec;
- for (int i = 0; i < originChildrenProperties.size(); i++) {
- DistributionSpecHash current
- = (DistributionSpecHash) originChildrenProperties.get(i).getDistributionSpec();
- if (current.getShuffleType() != ShuffleType.EXECUTION_BUCKETED
- || !bothSideShuffleKeysAreSameOrder(basic, current,
- (DistributionSpecHash) requiredProperties.get(0).getDistributionSpec(),
- (DistributionSpecHash) requiredProperties.get(i).getDistributionSpec())) {
+ int distributeToChildIndex = -1;
+ double basicBuckets = -1;
+ double basicRowCount = -1;
+
+ try {
+ ImmutableSet<ShuffleType> supportedShuffleTypes = ImmutableSet.of(
+ ShuffleType.NATURAL,
+ ShuffleType.STORAGE_BUCKETED
+ );
+ // find the most (bucket num, rowCount) side as the basic
+ for (int i = 0; i < originChildrenProperties.size(); i++) {
+ PhysicalProperties originChildrenProperty = originChildrenProperties.get(i);
Review Comment:
should check `children.get(i)` is not `PhysicalDistribute`
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/properties/ChildrenPropertiesRegulator.java:
##########
@@ -649,19 +667,82 @@ public List<List<PhysicalProperties>> visitPhysicalSetOperation(PhysicalSetOpera
} else if (requiredDistributionSpec instanceof DistributionSpecHash) {
// TODO: should use the most common hash spec as basic
DistributionSpecHash basic = (DistributionSpecHash) requiredDistributionSpec;
- for (int i = 0; i < originChildrenProperties.size(); i++) {
- DistributionSpecHash current
- = (DistributionSpecHash) originChildrenProperties.get(i).getDistributionSpec();
- if (current.getShuffleType() != ShuffleType.EXECUTION_BUCKETED
- || !bothSideShuffleKeysAreSameOrder(basic, current,
- (DistributionSpecHash) requiredProperties.get(0).getDistributionSpec(),
- (DistributionSpecHash) requiredProperties.get(i).getDistributionSpec())) {
+ int distributeToChildIndex = -1;
+ double basicBuckets = -1;
+ double basicRowCount = -1;
+
+ try {
+ ImmutableSet<ShuffleType> supportedShuffleTypes = ImmutableSet.of(
+ ShuffleType.NATURAL,
+ ShuffleType.STORAGE_BUCKETED
+ );
+ // find the most (bucket num, rowCount) side as the basic
+ for (int i = 0; i < originChildrenProperties.size(); i++) {
+ PhysicalProperties originChildrenProperty = originChildrenProperties.get(i);
+ DistributionSpec childDistribution = originChildrenProperty.getDistributionSpec();
+ if (childDistribution instanceof DistributionSpecHash
+ && supportedShuffleTypes.contains(
+ ((DistributionSpecHash) childDistribution).getShuffleType())) {
+ int bucketNum = getBucketNum(children.get(i).getPlan());
+ Statistics stats = setOperation.child(i).getStats();
+ double rowCount = stats.getRowCount();
+ if (bucketNum > basicBuckets) {
+ basicBuckets = bucketNum;
+ basicRowCount = rowCount;
+ distributeToChildIndex = i;
+ } else if (bucketNum == basicBuckets && rowCount > basicRowCount) {
+ basicRowCount = rowCount;
+ distributeToChildIndex = i;
+ }
+ }
+ }
+ } catch (Throwable t) {
+ // catch stats exception
+ LOG.warn("Can not find the most (bucket num, rowCount): " + t, t);
+ distributeToChildIndex = -1;
+ }
+
+ if (distributeToChildIndex >= 0) {
+ DistributionSpecHash notShuffleSideRequire
+ = (DistributionSpecHash) requiredProperties.get(distributeToChildIndex).getDistributionSpec();
+
+ DistributionSpecHash notNeedShuffleOutput
+ = (DistributionSpecHash) originChildrenProperties.get(distributeToChildIndex)
+ .getDistributionSpec();
+
+ for (int i = 0; i < originChildrenProperties.size(); i++) {
+ DistributionSpecHash current
+ = (DistributionSpecHash) originChildrenProperties.get(i).getDistributionSpec();
+ if (i == distributeToChildIndex) {
+ continue;
+ }
+
+ DistributionSpecHash currentRequire
+ = (DistributionSpecHash) requiredProperties.get(i).getDistributionSpec();
+
PhysicalProperties target = calAnotherSideRequired(
- ShuffleType.EXECUTION_BUCKETED, basic, current,
- (DistributionSpecHash) requiredProperties.get(0).getDistributionSpec(),
- (DistributionSpecHash) requiredProperties.get(i).getDistributionSpec());
+ ShuffleType.STORAGE_BUCKETED,
+ notNeedShuffleOutput, current,
+ notShuffleSideRequire,
+ currentRequire);
updateChildEnforceAndCost(i, target);
}
+ setOperation.setMutableState(PhysicalSetOperation.DISTRIBUTE_TO_CHILD_INDEX, distributeToChildIndex);
Review Comment:
why need this? please add comment to explain it
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/glue/translator/PhysicalPlanTranslator.java:
##########
@@ -2323,6 +2323,13 @@ && findOlapScanNodesByPassExchangeAndJoinNode(setOperationFragment.getPlanRoot()
setOperationNode.setColocate(true);
}
+ for (Plan child : setOperation.children()) {
+ PhysicalPlan childPhysicalPlan = (PhysicalPlan) child;
+ if (JoinUtils.isStorageBucketed(childPhysicalPlan.getPhysicalProperties())) {
+ setOperationNode.setDistributionMode(SetOperationNode.DistributionMode.BUCKET_SHUFFLE);
Review Comment:
should do break here
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3701945938
# BE UT Coverage Report
Increment line coverage `0.00% (0/7)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/b2e0198031172db18da6bc9fa4273f0ba3c48f7a_b2e0198031172db18da6bc9fa4273f0ba3c48f7a/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/b2e0198031172db18da6bc9fa4273f0ba3c48f7a_b2e0198031172db18da6bc9fa4273f0ba3c48f7a/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 53.38% (18955/35510) |
| Line Coverage | 39.27% (175953/448088) |
| Region Coverage | 33.82% (136102/402431) |
| Branch Coverage | 34.76% (58798/169142) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3698424261
<details>
<summary>TPC-H: <b>Total hot run time: 35027 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit b53cd4a552d1b04e8db7674ee303144185799fa6, data reload: false
------ Round 1 ----------------------------------
q1 17623 4385 4111 4111
q2 2034 361 228 228
q3 10173 1279 738 738
q4 10215 849 313 313
q5 7528 2145 1868 1868
q6 193 169 136 136
q7 965 824 656 656
q8 9296 1406 1099 1099
q9 6753 5144 5167 5144
q10 6780 1793 1386 1386
q11 527 310 293 293
q12 689 732 586 586
q13 17777 3750 3099 3099
q14 289 301 279 279
q15 569 510 505 505
q16 705 667 613 613
q17 711 734 614 614
q18 7552 7513 8139 7513
q19 1121 1003 654 654
q20 440 389 264 264
q21 4521 4333 3863 3863
q22 1148 1137 1065 1065
Total cold run time: 107609 ms
Total hot run time: 35027 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4310 4244 4235 4235
q2 325 410 331 331
q3 2416 2776 2420 2420
q4 1341 1901 1452 1452
q5 4436 4375 4301 4301
q6 216 167 126 126
q7 2020 2004 1748 1748
q8 2526 2301 2269 2269
q9 7263 7177 7064 7064
q10 2474 2740 2271 2271
q11 548 465 456 456
q12 704 728 625 625
q13 3669 3857 3081 3081
q14 284 278 266 266
q15 532 485 481 481
q16 615 634 608 608
q17 1081 1257 1302 1257
q18 7462 7409 7356 7356
q19 865 848 883 848
q20 1901 1983 1831 1831
q21 4450 4225 4202 4202
q22 1108 1070 975 975
Total cold run time: 50546 ms
Total hot run time: 48203 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3696468606
# FE UT Coverage Report
Increment line coverage ` 54.07% (93/172)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/59006_fc57d6152d27913cfd8bc2196adc9e86dfe43c48/fe_increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/59006_fc57d6152d27913cfd8bc2196adc9e86dfe43c48/fe_report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3698832543
# FE Regression Coverage Report
Increment line coverage ` 70.93% (122/172)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/59006_b53cd4a552d1b04e8db7674ee303144185799fa6_merge_fe/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/59006_b53cd4a552d1b04e8db7674ee303144185799fa6_merge_fe/report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](distribution) support bucket shuffle for set operation [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59006:
URL: https://github.com/apache/doris/pull/59006#issuecomment-3685142563
# BE UT Coverage Report
Increment line coverage `0.00% (0/2)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/634b002f9d3d8bb834725b5c6c8befaa1cbec0e7_634b002f9d3d8bb834725b5c6c8befaa1cbec0e7/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/634b002f9d3d8bb834725b5c6c8befaa1cbec0e7_634b002f9d3d8bb834725b5c6c8befaa1cbec0e7/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 53.46% (18942/35434) |
| Line Coverage | 39.33% (175614/446489) |
| Region Coverage | 33.86% (135827/401133) |
| Branch Coverage | 34.78% (58566/168372) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org