You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "seawinde (via GitHub)" <gi...@apache.org> on 2024/07/25 12:12:05 UTC
[PR] [improvement](mtmv) Not roll up when aggregate rewrite if roll up group by expr is uniform [doris]
seawinde opened a new pull request, #38387:
URL: https://github.com/apache/doris/pull/38387
## Proposed changes
Not roll up when aggregate rewrite if roll up group by expr is uniform
Such as mv name is mv3_0, and def is:
```sql
CREATE MATERIALIZED VIEW mv3_0
BUILD IMMEDIATE REFRESH COMPLETE ON MANUAL
DISTRIBUTED BY RANDOM BUCKETS 2
PROPERTIES ('replication_num' = '1')
AS
select
o_orderdate,
o_shippriority,
o_comment,
sum(o_totalprice) as sum_total,
max(o_totalprice) as max_total,
min(o_totalprice) as min_total,
count(*) as count_all
from
orders
group by
o_orderdate,
o_shippriority,
o_comment;
```
query sql is as following:
```sql
select
o_comment,
sum(o_totalprice),
max(o_totalprice),
min(o_totalprice),
count(*)
from
orders
where
o_orderdate = '2023-12-09'
and o_shippriority = 1
group by
o_comment;
```
after rewrite the plan is as following, not need to add aggregate
```
PhysicalResultSink
--filter((mv3_0.o_orderdate = '2023-12-09') and (mv3_0.o_shippriority = 1))
----PhysicalOlapScan[mv3_0]
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](mtmv) Not roll up when aggregate rewrite if roll up group by expr is uniform [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #38387:
URL: https://github.com/apache/doris/pull/38387#issuecomment-2264281058
<details>
<summary>TPC-H: <b>Total hot run time: 41321 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit c04a71ac242f88ff77757e9b34880462449ed406, data reload: false
------ Round 1 ----------------------------------
q1 17671 4059 3994 3994
q2 2011 201 192 192
q3 10480 1286 1353 1286
q4 10155 779 956 779
q5 7581 2976 2934 2934
q6 216 139 136 136
q7 1020 605 617 605
q8 9449 1930 1954 1930
q9 8474 6577 6561 6561
q10 8732 3799 3854 3799
q11 430 246 244 244
q12 430 220 222 220
q13 17766 2914 2941 2914
q14 263 236 244 236
q15 532 485 483 483
q16 531 392 380 380
q17 961 933 899 899
q18 7959 7253 7253 7253
q19 1470 1216 1210 1210
q20 554 335 333 333
q21 5273 4731 4648 4648
q22 352 285 292 285
Total cold run time: 112310 ms
Total hot run time: 41321 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4049 4108 4015 4015
q2 326 229 217 217
q3 3025 2958 3092 2958
q4 1972 1989 2005 1989
q5 5539 5491 5475 5475
q6 220 133 130 130
q7 2100 1766 1792 1766
q8 3299 3331 3292 3292
q9 8663 8661 8716 8661
q10 3959 4042 3932 3932
q11 542 440 442 440
q12 767 619 592 592
q13 15290 3118 3080 3080
q14 322 261 282 261
q15 528 484 475 475
q16 454 426 423 423
q17 1764 1735 1733 1733
q18 8142 7828 7734 7734
q19 1701 1705 1719 1705
q20 2035 1866 1826 1826
q21 5716 5429 5254 5254
q22 537 466 468 466
Total cold run time: 70950 ms
Total hot run time: 56424 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](mtmv) Not roll up when aggregate rewrite if roll up group by expr is uniform [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #38387:
URL: https://github.com/apache/doris/pull/38387#issuecomment-2250232827
<details>
<summary>TPC-DS: <b>Total hot run time: 173031 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit e7daa9bd0142b592fa49880030da5202613afe16, data reload: false
query1 905 369 361 361
query2 6450 1826 1787 1787
query3 6642 206 228 206
query4 28578 17748 17323 17323
query5 4170 474 494 474
query6 266 180 160 160
query7 4594 284 286 284
query8 253 203 192 192
query9 8447 2456 2431 2431
query10 443 293 271 271
query11 10951 10137 10110 10110
query12 133 89 82 82
query13 1606 362 360 360
query14 10113 7650 7526 7526
query15 244 162 169 162
query16 7831 501 477 477
query17 1575 557 550 550
query18 2019 277 273 273
query19 192 145 134 134
query20 91 81 82 81
query21 209 100 99 99
query22 4597 4180 4158 4158
query23 33685 33056 32986 32986
query24 11875 2941 2891 2891
query25 623 363 365 363
query26 1699 180 142 142
query27 2921 261 271 261
query28 7691 2014 1997 1997
query29 1030 408 404 404
query30 285 152 148 148
query31 964 757 750 750
query32 91 54 52 52
query33 785 319 350 319
query34 911 463 463 463
query35 857 712 733 712
query36 1106 935 952 935
query37 148 80 75 75
query38 2857 2781 2754 2754
query39 842 785 810 785
query40 270 110 112 110
query41 50 48 46 46
query42 114 94 96 94
query43 497 448 450 448
query44 1190 715 725 715
query45 207 175 175 175
query46 1126 722 727 722
query47 1872 1776 1805 1776
query48 367 293 295 293
query49 1125 415 434 415
query50 793 401 392 392
query51 6815 6682 6642 6642
query52 91 94 85 85
query53 255 176 174 174
query54 873 436 436 436
query55 73 74 73 73
query56 287 266 274 266
query57 1165 1057 1052 1052
query58 252 274 275 274
query59 2831 2580 2398 2398
query60 304 287 276 276
query61 132 91 92 91
query62 823 649 645 645
query63 207 179 173 173
query64 10520 2236 1678 1678
query65 3144 3115 3122 3115
query66 1373 333 332 332
query67 15494 15114 14917 14917
query68 4593 550 557 550
query69 436 299 293 293
query70 1138 1054 1061 1054
query71 396 282 278 278
query72 7069 5691 5913 5691
query73 766 321 324 321
query74 6205 5753 5749 5749
query75 3657 2676 2632 2632
query76 2804 902 922 902
query77 483 303 306 303
query78 9620 9889 9640 9640
query79 2353 531 532 531
query80 1856 475 471 471
query81 588 213 224 213
query82 726 134 133 133
query83 288 169 166 166
query84 261 87 77 77
query85 1404 306 289 289
query86 474 335 328 328
query87 3274 3159 3133 3133
query88 3441 2420 2411 2411
query89 389 289 286 286
query90 1863 191 191 191
query91 138 115 114 114
query92 68 51 49 49
query93 1867 533 545 533
query94 1030 301 291 291
query95 359 276 272 272
query96 589 275 278 275
query97 3190 3049 3076 3049
query98 211 199 192 192
query99 1532 1263 1269 1263
Total cold run time: 283997 ms
Total hot run time: 173031 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](mtmv) Not roll up when aggregate rewrite if roll up group by expr is uniform [doris]
Posted by "morrySnow (via GitHub)" <gi...@apache.org>.
morrySnow commented on code in PR #38387:
URL: https://github.com/apache/doris/pull/38387#discussion_r1697833741
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/exploration/mv/AbstractMaterializedViewAggregateRule.java:
##########
@@ -324,18 +327,20 @@ private boolean isGroupByEquals(Pair<Plan, LogicalAggregate<Plan>> queryTopPlanA
SlotMapping viewToQuerySlotMapping,
StructInfo queryStructInfo,
StructInfo viewStructInfo,
+ Plan tempRewrittenPlan,
MaterializationContext materializationContext,
CascadesContext cascadesContext) {
+
+ if (materializationContext instanceof SyncMaterializationContext) {
+ return false;
+ }
Review Comment:
add comment for this if statement
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](mtmv) Not roll up when aggregate rewrite if roll up group by expr is uniform [doris]
Posted by "starocean999 (via GitHub)" <gi...@apache.org>.
starocean999 merged PR #38387:
URL: https://github.com/apache/doris/pull/38387
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](mtmv) Not roll up when aggregate rewrite if roll up group by expr is uniform [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #38387:
URL: https://github.com/apache/doris/pull/38387#issuecomment-2254885949
<details>
<summary>ClickBench: <b>Total hot run time: 31.17 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 76826d4dca4a4adad654c6e7e227425ee1e8ff87, data reload: false
query1 0.04 0.04 0.04
query2 0.08 0.04 0.04
query3 0.22 0.06 0.06
query4 1.65 0.09 0.10
query5 0.50 0.50 0.48
query6 1.12 0.72 0.74
query7 0.02 0.02 0.01
query8 0.05 0.04 0.04
query9 0.56 0.50 0.50
query10 0.54 0.55 0.55
query11 0.16 0.11 0.12
query12 0.15 0.13 0.13
query13 0.59 0.58 0.58
query14 0.77 0.78 0.77
query15 0.85 0.82 0.82
query16 0.37 0.37 0.37
query17 0.98 1.05 0.98
query18 0.23 0.22 0.22
query19 1.91 1.88 1.81
query20 0.02 0.01 0.01
query21 15.38 0.77 0.66
query22 3.81 7.79 2.00
query23 18.34 1.37 1.28
query24 2.07 0.25 0.24
query25 0.15 0.09 0.09
query26 0.30 0.21 0.22
query27 0.46 0.24 0.24
query28 13.24 1.02 1.03
query29 12.66 3.33 3.35
query30 0.25 0.06 0.05
query31 2.86 0.40 0.39
query32 3.25 0.48 0.48
query33 2.91 2.90 2.95
query34 16.90 4.40 4.43
query35 4.42 4.45 4.45
query36 0.65 0.48 0.50
query37 0.20 0.17 0.17
query38 0.17 0.16 0.16
query39 0.05 0.04 0.03
query40 0.16 0.13 0.13
query41 0.10 0.04 0.04
query42 0.06 0.05 0.06
query43 0.05 0.04 0.04
Total cold run time: 109.25 s
Total hot run time: 31.17 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](mtmv) Not roll up when aggregate rewrite if roll up group by expr is uniform [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #38387:
URL: https://github.com/apache/doris/pull/38387#issuecomment-2263054064
<details>
<summary>TPC-H: <b>Total hot run time: 41561 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 3853d0a833243f599035eda99a6aa3590b02a02e, data reload: false
------ Round 1 ----------------------------------
q1 18269 4243 4140 4140
q2 2022 201 211 201
q3 10477 1298 1314 1298
q4 10163 801 941 801
q5 7607 2946 2934 2934
q6 224 141 139 139
q7 1031 618 626 618
q8 9457 1840 1954 1840
q9 8423 6566 6585 6566
q10 8712 3833 3834 3833
q11 426 252 253 252
q12 406 229 226 226
q13 17754 2931 2938 2931
q14 262 240 245 240
q15 523 482 495 482
q16 491 400 388 388
q17 949 927 903 903
q18 7971 7301 7268 7268
q19 1385 1220 1218 1218
q20 564 323 333 323
q21 5286 4677 4741 4677
q22 356 288 283 283
Total cold run time: 112758 ms
Total hot run time: 41561 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4062 4022 4009 4009
q2 334 224 222 222
q3 2995 2974 2993 2974
q4 1859 1880 1867 1867
q5 5226 5229 5214 5214
q6 214 135 134 134
q7 2048 1659 1681 1659
q8 3155 3265 3252 3252
q9 8295 8280 8260 8260
q10 3745 3871 3841 3841
q11 539 453 471 453
q12 704 531 543 531
q13 16749 2940 2985 2940
q14 282 262 255 255
q15 519 482 478 478
q16 441 397 403 397
q17 1751 1704 1705 1704
q18 7721 7460 7229 7229
q19 1682 1671 1684 1671
q20 1964 1765 1743 1743
q21 5452 5197 5190 5190
q22 500 435 455 435
Total cold run time: 70237 ms
Total hot run time: 54458 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](mtmv) Not roll up when aggregate rewrite if roll up group by expr is uniform [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #38387:
URL: https://github.com/apache/doris/pull/38387#issuecomment-2264294407
<details>
<summary>ClickBench: <b>Total hot run time: 30.05 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit c04a71ac242f88ff77757e9b34880462449ed406, data reload: false
query1 0.04 0.03 0.04
query2 0.07 0.04 0.04
query3 0.22 0.04 0.04
query4 1.68 0.07 0.07
query5 0.48 0.49 0.49
query6 1.13 0.72 0.71
query7 0.02 0.01 0.02
query8 0.05 0.04 0.05
query9 0.57 0.51 0.50
query10 0.56 0.56 0.56
query11 0.15 0.12 0.11
query12 0.15 0.13 0.12
query13 0.61 0.61 0.60
query14 0.76 0.77 0.80
query15 0.91 0.87 0.86
query16 0.36 0.35 0.35
query17 1.01 1.00 1.01
query18 0.22 0.21 0.21
query19 1.82 1.76 1.76
query20 0.02 0.01 0.01
query21 15.39 0.74 0.65
query22 3.94 8.77 1.29
query23 17.87 1.37 1.39
query24 2.26 0.23 0.22
query25 0.18 0.08 0.09
query26 0.32 0.22 0.21
query27 0.46 0.23 0.24
query28 13.16 1.00 1.00
query29 12.54 3.34 3.27
query30 0.26 0.06 0.05
query31 2.87 0.42 0.40
query32 3.25 0.50 0.49
query33 2.98 2.91 2.94
query34 15.44 4.24 4.25
query35 4.31 4.29 4.29
query36 0.67 0.48 0.50
query37 0.19 0.16 0.15
query38 0.17 0.15 0.14
query39 0.04 0.04 0.04
query40 0.15 0.13 0.14
query41 0.09 0.04 0.05
query42 0.06 0.05 0.05
query43 0.05 0.04 0.04
Total cold run time: 107.48 s
Total hot run time: 30.05 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](mtmv) Not roll up when aggregate rewrite if roll up group by expr is uniform [doris]
Posted by "seawinde (via GitHub)" <gi...@apache.org>.
seawinde commented on PR #38387:
URL: https://github.com/apache/doris/pull/38387#issuecomment-2261787204
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](mtmv) Not roll up when aggregate rewrite if roll up group by expr is uniform [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #38387:
URL: https://github.com/apache/doris/pull/38387#issuecomment-2263087493
<details>
<summary>TPC-DS: <b>Total hot run time: 168678 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 3853d0a833243f599035eda99a6aa3590b02a02e, data reload: false
query1 912 380 368 368
query2 6472 1660 1696 1660
query3 6655 215 223 215
query4 19600 17429 17249 17249
query5 4300 515 515 515
query6 283 171 171 171
query7 4609 300 293 293
query8 255 205 199 199
query9 8506 2364 2348 2348
query10 434 277 261 261
query11 10566 10052 10033 10033
query12 140 89 88 88
query13 1635 372 383 372
query14 9811 6929 6914 6914
query15 209 164 172 164
query16 7107 434 451 434
query17 954 561 548 548
query18 1932 284 278 278
query19 191 153 150 150
query20 94 87 86 86
query21 205 111 96 96
query22 4171 4138 3911 3911
query23 33858 32966 33036 32966
query24 10333 3103 3051 3051
query25 711 401 413 401
query26 1776 151 153 151
query27 2894 280 274 274
query28 6942 1966 1951 1951
query29 1336 429 431 429
query30 286 150 153 150
query31 927 760 759 759
query32 103 56 58 56
query33 704 315 329 315
query34 966 472 487 472
query35 869 748 710 710
query36 1009 886 877 877
query37 301 83 81 81
query38 2876 2759 2740 2740
query39 883 806 822 806
query40 291 115 116 115
query41 51 48 49 48
query42 124 102 106 102
query43 453 417 418 417
query44 1163 724 717 717
query45 214 177 179 177
query46 1067 831 773 773
query47 1832 1730 1720 1720
query48 369 292 299 292
query49 1212 449 443 443
query50 892 450 442 442
query51 6784 6809 6662 6662
query52 102 91 93 91
query53 253 182 179 179
query54 638 462 450 450
query55 78 79 80 79
query56 284 279 278 278
query57 1157 1026 1068 1026
query58 275 266 280 266
query59 2541 2385 2374 2374
query60 302 283 294 283
query61 116 112 217 112
query62 880 664 656 656
query63 213 187 183 183
query64 5859 1937 1867 1867
query65 3156 3104 3130 3104
query66 1441 345 334 334
query67 15365 14782 14778 14778
query68 6916 568 579 568
query69 757 406 311 311
query70 1120 1047 1053 1047
query71 538 281 281 281
query72 8050 2635 2498 2498
query73 969 333 331 331
query74 5962 5673 5724 5673
query75 4533 2723 2710 2710
query76 5108 1268 1290 1268
query77 758 308 310 308
query78 9475 8977 8874 8874
query79 6387 528 521 521
query80 862 534 506 506
query81 549 222 226 222
query82 281 127 138 127
query83 321 176 172 172
query84 268 79 80 79
query85 1035 316 311 311
query86 388 286 292 286
query87 3283 3062 3103 3062
query88 3848 2411 2413 2411
query89 493 288 286 286
query90 2100 201 232 201
query91 123 102 102 102
query92 64 52 50 50
query93 5178 599 614 599
query94 1050 312 279 279
query95 383 259 263 259
query96 626 280 281 280
query97 3186 3074 3079 3074
query98 216 200 193 193
query99 1614 1284 1279 1279
Total cold run time: 279368 ms
Total hot run time: 168678 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](mtmv) Not roll up when aggregate rewrite if roll up group by expr is uniform [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #38387:
URL: https://github.com/apache/doris/pull/38387#issuecomment-2250243114
<details>
<summary>ClickBench: <b>Total hot run time: 30.69 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit e7daa9bd0142b592fa49880030da5202613afe16, data reload: false
query1 0.04 0.03 0.03
query2 0.09 0.04 0.04
query3 0.22 0.05 0.05
query4 1.67 0.10 0.07
query5 0.50 0.50 0.50
query6 1.13 0.74 0.72
query7 0.02 0.02 0.02
query8 0.05 0.04 0.04
query9 0.56 0.50 0.49
query10 0.55 0.54 0.54
query11 0.16 0.11 0.11
query12 0.14 0.13 0.13
query13 0.59 0.58 0.58
query14 0.77 0.77 0.79
query15 0.85 0.82 0.81
query16 0.36 0.34 0.36
query17 1.01 0.97 1.02
query18 0.23 0.22 0.21
query19 1.87 1.71 1.82
query20 0.02 0.01 0.01
query21 15.39 0.78 0.66
query22 4.54 7.17 2.00
query23 18.36 1.31 1.23
query24 2.15 0.23 0.22
query25 0.15 0.08 0.09
query26 0.31 0.21 0.21
query27 0.46 0.23 0.23
query28 13.25 1.01 1.00
query29 12.66 3.37 3.31
query30 0.25 0.06 0.05
query31 2.89 0.39 0.38
query32 3.30 0.47 0.46
query33 2.86 2.94 2.89
query34 17.07 4.36 4.34
query35 4.42 4.44 4.44
query36 0.65 0.49 0.47
query37 0.20 0.15 0.15
query38 0.15 0.15 0.15
query39 0.04 0.04 0.03
query40 0.15 0.12 0.12
query41 0.09 0.05 0.05
query42 0.07 0.05 0.05
query43 0.04 0.04 0.03
Total cold run time: 110.28 s
Total hot run time: 30.69 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](mtmv) Not roll up when aggregate rewrite if roll up group by expr is uniform [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #38387:
URL: https://github.com/apache/doris/pull/38387#issuecomment-2250176265
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR)
Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website).
See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](mtmv) Not roll up when aggregate rewrite if roll up group by expr is uniform [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #38387:
URL: https://github.com/apache/doris/pull/38387#issuecomment-2264290000
<details>
<summary>TPC-DS: <b>Total hot run time: 169713 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit c04a71ac242f88ff77757e9b34880462449ed406, data reload: false
query1 912 380 367 367
query2 6452 1703 1655 1655
query3 6659 214 228 214
query4 19534 17623 17221 17221
query5 3634 521 512 512
query6 260 170 165 165
query7 4613 287 289 287
query8 251 198 201 198
query9 8504 2369 2355 2355
query10 466 286 285 285
query11 10509 10221 10157 10157
query12 124 88 91 88
query13 1615 375 378 375
query14 9828 7545 7526 7526
query15 207 163 161 161
query16 6945 443 420 420
query17 936 579 557 557
query18 1829 279 279 279
query19 191 142 139 139
query20 92 87 85 85
query21 201 103 97 97
query22 4179 4077 3962 3962
query23 33743 33788 33501 33501
query24 10282 3116 3061 3061
query25 684 413 399 399
query26 1671 151 154 151
query27 3047 282 283 282
query28 7497 1974 1969 1969
query29 1262 455 439 439
query30 239 153 162 153
query31 933 750 760 750
query32 101 59 56 56
query33 667 344 332 332
query34 930 506 507 506
query35 861 790 749 749
query36 1064 888 884 884
query37 266 80 84 80
query38 2881 2808 2786 2786
query39 895 808 796 796
query40 253 111 111 111
query41 47 65 44 44
query42 119 100 102 100
query43 465 418 437 418
query44 1168 735 723 723
query45 215 173 176 173
query46 1073 795 777 777
query47 1849 1746 1747 1746
query48 366 291 289 289
query49 985 427 428 427
query50 890 427 426 426
query51 6791 6596 6672 6596
query52 103 90 90 90
query53 253 183 178 178
query54 617 447 454 447
query55 78 74 75 74
query56 275 243 259 243
query57 1123 1037 1042 1037
query58 286 277 287 277
query59 2638 2365 2300 2300
query60 293 267 285 267
query61 95 95 91 91
query62 880 644 663 644
query63 220 180 175 175
query64 5744 1904 1864 1864
query65 3169 3070 3114 3070
query66 1301 332 333 332
query67 15396 14958 14663 14663
query68 4329 562 575 562
query69 439 308 314 308
query70 1127 1078 1068 1068
query71 445 277 270 270
query72 7024 2720 2473 2473
query73 760 329 325 325
query74 6002 5757 5640 5640
query75 3353 2738 2744 2738
query76 2370 1203 1282 1203
query77 436 306 306 306
query78 9509 8909 9018 8909
query79 1932 530 535 530
query80 1100 503 512 503
query81 568 227 232 227
query82 1057 130 127 127
query83 235 169 168 168
query84 271 78 80 78
query85 1305 305 297 297
query86 414 313 280 280
query87 3269 3161 3106 3106
query88 2887 2396 2385 2385
query89 393 285 292 285
query90 1735 191 189 189
query91 128 98 96 96
query92 60 49 49 49
query93 1515 611 611 611
query94 814 317 295 295
query95 369 269 267 267
query96 660 279 281 279
query97 3211 3091 3067 3067
query98 218 200 190 190
query99 1634 1305 1301 1301
Total cold run time: 261225 ms
Total hot run time: 169713 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](mtmv) Not roll up when aggregate rewrite if roll up group by expr is uniform [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #38387:
URL: https://github.com/apache/doris/pull/38387#issuecomment-2264669793
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](mtmv) Not roll up when aggregate rewrite if roll up group by expr is uniform [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #38387:
URL: https://github.com/apache/doris/pull/38387#issuecomment-2264669823
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](mtmv) Not roll up when aggregate rewrite if roll up group by expr is uniform [doris]
Posted by "seawinde (via GitHub)" <gi...@apache.org>.
seawinde commented on PR #38387:
URL: https://github.com/apache/doris/pull/38387#issuecomment-2264241609
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](mtmv) Not roll up when aggregate rewrite if roll up group by expr is uniform [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #38387:
URL: https://github.com/apache/doris/pull/38387#issuecomment-2257969504
<details>
<summary>ClickBench: <b>Total hot run time: 30.01 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 85e33b048335f4c7b2c839728f7d1ac92a45ec04, data reload: false
query1 0.04 0.04 0.03
query2 0.07 0.04 0.04
query3 0.22 0.05 0.05
query4 1.69 0.07 0.06
query5 0.48 0.48 0.48
query6 1.16 0.72 0.72
query7 0.02 0.02 0.01
query8 0.05 0.05 0.05
query9 0.57 0.52 0.51
query10 0.58 0.57 0.57
query11 0.16 0.12 0.12
query12 0.15 0.13 0.12
query13 0.61 0.60 0.59
query14 0.78 0.80 0.79
query15 0.92 0.88 0.86
query16 0.36 0.35 0.37
query17 0.98 0.99 0.97
query18 0.23 0.21 0.20
query19 1.85 1.75 1.76
query20 0.01 0.00 0.01
query21 15.39 0.78 0.65
query22 3.62 7.92 1.28
query23 18.48 1.39 1.40
query24 2.29 0.22 0.21
query25 0.19 0.08 0.07
query26 0.31 0.22 0.21
query27 0.46 0.22 0.23
query28 13.16 1.00 0.96
query29 12.51 3.35 3.30
query30 0.25 0.06 0.05
query31 2.88 0.42 0.40
query32 3.23 0.49 0.49
query33 2.99 3.01 2.90
query34 15.44 4.28 4.25
query35 4.26 4.29 4.28
query36 0.68 0.49 0.48
query37 0.19 0.17 0.17
query38 0.17 0.15 0.16
query39 0.04 0.03 0.04
query40 0.16 0.14 0.13
query41 0.10 0.04 0.05
query42 0.05 0.05 0.04
query43 0.05 0.04 0.04
Total cold run time: 107.83 s
Total hot run time: 30.01 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](mtmv) Not roll up when aggregate rewrite if roll up group by expr is uniform [doris]
Posted by "seawinde (via GitHub)" <gi...@apache.org>.
seawinde commented on PR #38387:
URL: https://github.com/apache/doris/pull/38387#issuecomment-2257903779
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](mtmv) Not roll up when aggregate rewrite if roll up group by expr is uniform [doris]
Posted by "starocean999 (via GitHub)" <gi...@apache.org>.
starocean999 commented on code in PR #38387:
URL: https://github.com/apache/doris/pull/38387#discussion_r1697985131
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/exploration/mv/AbstractMaterializedViewAggregateRule.java:
##########
@@ -355,22 +360,116 @@ private boolean isGroupByEquals(Pair<Plan, LogicalAggregate<Plan>> queryTopPlanA
viewGroupExpressionQueryBased
);
}
- if (queryGroupShuttledExpression.equals(viewShuttledExpressionQueryBasedToGroupByExpressionMap.values())) {
+ if (queryGroupShuttledExpression.equals(viewShuttledExpressionQueryBasedToGroupByExpressionMap.keySet())) {
// return true, if equals directly
return true;
}
+
+ boolean isGroupByEquals = false;
+ // check is equals by group by eliminate
+ isGroupByEquals |= isGroupByEqualsAfterGroupByEliminate(queryGroupShuttledExpression,
+ viewShuttledExpressionQueryBasedToGroupByExpressionMap,
+ groupByExpressionToViewShuttledExpressionQueryBasedMap,
+ viewAggregate,
+ cascadesContext);
+ // check is equals by equal filter eliminate
+ Optional<LogicalFilter<Plan>> filterOptional = tempRewrittenPlan.collectFirst(LogicalFilter.class::isInstance);
+ if (!filterOptional.isPresent()) {
+ return isGroupByEquals;
+ }
+ isGroupByEquals |= isGroupByEqualsAfterEqualFilterEliminate(
+ (LogicalPlan) tempRewrittenPlan,
+ queryGroupShuttledExpression,
+ viewShuttledExpressionQueryBasedToGroupByExpressionMap,
+ materializationContext);
+ return isGroupByEquals;
+ }
+
+ /**
+ * Check group by is equals by equal filter eliminate
+ * For example query is select a, b, c from t1 where a = 1 and d = 'xx' group by a, b, c;
+ * mv is select a, b, c, d from t1 group by a, b, c, d;
+ * the group by expression between query and view is equals after equal filter eliminate
+ * should not aggregate roll up
+ * */
+ private static boolean isGroupByEqualsAfterEqualFilterEliminate(
+ LogicalPlan tempRewrittenPlan,
+ Set<Expression> queryGroupShuttledExpression,
+ Map<Expression, Expression> viewShuttledExprQueryBasedToViewGroupByExprMap,
+ MaterializationContext materializationContext) {
+
+ Map<Expression, Expression> viewShuttledExprToScanExprMapping =
+ materializationContext.getShuttledExprToScanExprMapping().flattenMap().get(0);
+ Set<Expression> viewShuttledExprQueryBasedSet = viewShuttledExprQueryBasedToViewGroupByExprMap.keySet();
+ // view group by expr can not cover query group by expr
+ if (!viewShuttledExprQueryBasedSet.containsAll(queryGroupShuttledExpression)) {
+ return false;
+ }
+ Set<Expression> viewShouldUniformExpressionSet = new HashSet<>();
+ // calc the group by expr which is needed to roll up and should be uniform
+ for (Map.Entry<Expression, Expression> expressionEntry :
+ viewShuttledExprQueryBasedToViewGroupByExprMap.entrySet()) {
+ if (queryGroupShuttledExpression.contains(expressionEntry.getKey())) {
+ // the group expr which query has, do not require uniform
+ continue;
+ }
+ viewShouldUniformExpressionSet.add(expressionEntry.getValue());
+ }
+
+ DataTrait dataTrait = tempRewrittenPlan.computeDataTrait();
+ for (Expression shouldUniformExpr : viewShouldUniformExpressionSet) {
+ Expression viewScanExpression = viewShuttledExprToScanExprMapping.get(shouldUniformExpr);
+ if (viewScanExpression == null) {
+ return false;
+ }
+ if (!(viewScanExpression instanceof Slot)) {
+ return false;
+ }
+ if (!dataTrait.isUniform((Slot) viewScanExpression)) {
+ return false;
+ }
+ }
+ return true;
+ }
+
+ /**
+ * Check group by is equal or not after group by eliminate
Review Comment:
the comment of isGroupByEqualsAfterEqualFilterEliminate is easy to understand, but not isGroupByEqualsAfterGroupByEliminate, can we improve that?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](mtmv) Not roll up when aggregate rewrite if roll up group by expr is uniform [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #38387:
URL: https://github.com/apache/doris/pull/38387#issuecomment-2257958067
<details>
<summary>TPC-DS: <b>Total hot run time: 170248 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 85e33b048335f4c7b2c839728f7d1ac92a45ec04, data reload: false
query1 917 383 375 375
query2 6258 1757 1748 1748
query3 6657 212 234 212
query4 20127 17369 17207 17207
query5 3495 507 529 507
query6 269 174 184 174
query7 4603 306 285 285
query8 249 196 188 188
query9 8518 2396 2375 2375
query10 427 280 274 274
query11 10406 10007 10040 10007
query12 134 95 93 93
query13 1631 375 395 375
query14 10254 8394 7755 7755
query15 200 165 161 161
query16 6886 410 437 410
query17 931 573 534 534
query18 1905 288 279 279
query19 189 144 140 140
query20 93 85 87 85
query21 227 103 101 101
query22 4210 3974 4033 3974
query23 33638 33848 33276 33276
query24 10428 3110 3088 3088
query25 701 434 419 419
query26 1672 153 148 148
query27 2846 288 290 288
query28 7343 2025 2012 2012
query29 1306 438 423 423
query30 234 152 160 152
query31 961 765 780 765
query32 105 58 61 58
query33 672 334 329 329
query34 924 483 518 483
query35 890 790 770 770
query36 1069 906 907 906
query37 288 92 94 92
query38 3004 2782 2780 2780
query39 861 824 830 824
query40 263 116 118 116
query41 50 46 47 46
query42 125 104 105 104
query43 484 440 441 440
query44 1189 733 760 733
query45 214 176 177 176
query46 1094 815 790 790
query47 1814 1696 1696 1696
query48 373 334 297 297
query49 936 418 413 413
query50 907 422 427 422
query51 6757 6688 6608 6608
query52 103 88 92 88
query53 258 193 180 180
query54 613 467 459 459
query55 75 78 77 77
query56 281 263 250 250
query57 1153 1037 1004 1004
query58 281 267 284 267
query59 2690 2514 2514 2514
query60 304 263 283 263
query61 99 96 94 94
query62 879 664 669 664
query63 215 183 178 178
query64 5713 1917 1880 1880
query65 3159 3077 3094 3077
query66 1314 333 330 330
query67 15543 15308 14903 14903
query68 4404 569 571 569
query69 719 387 302 302
query70 1070 1065 1070 1065
query71 445 268 275 268
query72 7559 2704 2486 2486
query73 809 317 320 317
query74 6014 5654 5580 5580
query75 3387 2711 2712 2711
query76 3197 1381 1420 1381
query77 550 310 310 310
query78 9360 8966 8834 8834
query79 2641 530 530 530
query80 1011 499 497 497
query81 564 229 226 226
query82 984 135 131 131
query83 265 172 168 168
query84 257 82 82 82
query85 1165 328 313 313
query86 458 284 327 284
query87 3229 3091 3069 3069
query88 3731 2412 2392 2392
query89 381 287 290 287
query90 1661 190 188 188
query91 123 103 104 103
query92 59 49 49 49
query93 1972 610 605 605
query94 784 287 287 287
query95 378 269 270 269
query96 615 279 285 279
query97 3181 3046 3081 3046
query98 218 203 210 203
query99 1696 1276 1313 1276
Total cold run time: 265301 ms
Total hot run time: 170248 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](mtmv) Not roll up when aggregate rewrite if roll up group by expr is uniform [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #38387:
URL: https://github.com/apache/doris/pull/38387#issuecomment-2263106416
<details>
<summary>ClickBench: <b>Total hot run time: 29.88 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 3853d0a833243f599035eda99a6aa3590b02a02e, data reload: false
query1 0.04 0.04 0.04
query2 0.07 0.04 0.03
query3 0.23 0.04 0.05
query4 1.68 0.07 0.07
query5 0.49 0.48 0.49
query6 1.13 0.72 0.71
query7 0.02 0.02 0.01
query8 0.05 0.05 0.04
query9 0.56 0.51 0.51
query10 0.57 0.56 0.57
query11 0.15 0.11 0.12
query12 0.14 0.12 0.13
query13 0.61 0.59 0.60
query14 0.77 0.81 0.79
query15 0.89 0.86 0.86
query16 0.36 0.35 0.35
query17 0.98 1.00 0.99
query18 0.22 0.21 0.21
query19 1.87 1.74 1.80
query20 0.02 0.01 0.01
query21 15.41 0.74 0.64
query22 3.90 8.47 1.28
query23 17.99 1.27 1.24
query24 2.23 0.22 0.21
query25 0.18 0.08 0.08
query26 0.32 0.22 0.22
query27 0.47 0.23 0.22
query28 13.18 1.00 0.96
query29 12.52 3.33 3.30
query30 0.25 0.06 0.06
query31 2.87 0.40 0.40
query32 3.25 0.49 0.49
query33 2.93 2.97 2.95
query34 15.44 4.23 4.24
query35 4.26 4.31 4.28
query36 0.69 0.49 0.48
query37 0.20 0.15 0.16
query38 0.15 0.15 0.14
query39 0.05 0.03 0.04
query40 0.16 0.14 0.14
query41 0.10 0.05 0.04
query42 0.05 0.05 0.04
query43 0.05 0.04 0.05
Total cold run time: 107.5 s
Total hot run time: 29.88 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](mtmv) Not roll up when aggregate rewrite if roll up group by expr is uniform [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #38387:
URL: https://github.com/apache/doris/pull/38387#issuecomment-2254881673
<details>
<summary>TPC-DS: <b>Total hot run time: 174074 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 76826d4dca4a4adad654c6e7e227425ee1e8ff87, data reload: false
query1 918 381 372 372
query2 6438 1953 1844 1844
query3 6635 211 217 211
query4 27962 17520 17378 17378
query5 3629 498 486 486
query6 262 169 184 169
query7 4574 286 285 285
query8 243 223 194 194
query9 8525 2487 2472 2472
query10 442 295 271 271
query11 11240 10205 10014 10014
query12 120 95 88 88
query13 1769 392 380 380
query14 10262 7947 7782 7782
query15 212 165 171 165
query16 7665 496 501 496
query17 1604 569 525 525
query18 1902 276 280 276
query19 191 148 140 140
query20 93 86 89 86
query21 205 100 95 95
query22 4283 4067 3943 3943
query23 34103 33774 33795 33774
query24 10908 2938 2946 2938
query25 585 391 419 391
query26 704 154 154 154
query27 2366 301 284 284
query28 5890 2119 2105 2105
query29 787 431 443 431
query30 250 155 157 155
query31 976 770 765 765
query32 94 53 57 53
query33 704 354 355 354
query34 914 515 503 503
query35 901 753 772 753
query36 1136 974 950 950
query37 143 85 81 81
query38 2982 2765 2747 2747
query39 854 828 851 828
query40 197 114 114 114
query41 46 44 43 43
query42 107 95 95 95
query43 512 468 477 468
query44 1121 724 720 720
query45 213 176 184 176
query46 1078 729 748 729
query47 1867 1765 1775 1765
query48 357 301 297 297
query49 829 409 413 409
query50 793 401 403 401
query51 6722 6671 6647 6647
query52 104 85 95 85
query53 258 185 178 178
query54 855 453 439 439
query55 74 71 75 71
query56 308 275 268 268
query57 1154 1050 1070 1050
query58 265 268 272 268
query59 2999 2757 2768 2757
query60 317 280 291 280
query61 97 92 96 92
query62 786 661 704 661
query63 205 177 182 177
query64 9208 2255 1732 1732
query65 3189 3130 3137 3130
query66 732 347 330 330
query67 15264 15039 15013 15013
query68 4494 563 560 560
query69 435 300 314 300
query70 1140 1148 1133 1133
query71 372 274 266 266
query72 6870 5941 5458 5458
query73 746 332 330 330
query74 6215 5674 5638 5638
query75 3463 2674 2659 2659
query76 2297 951 879 879
query77 436 313 310 310
query78 12534 10198 8962 8962
query79 7038 522 539 522
query80 1394 498 495 495
query81 586 219 216 216
query82 521 142 140 140
query83 274 173 172 172
query84 267 86 77 77
query85 716 376 310 310
query86 482 280 309 280
query87 3266 3136 3081 3081
query88 4600 2491 2491 2491
query89 410 284 282 282
query90 1856 191 196 191
query91 124 101 101 101
query92 63 47 50 47
query93 3269 533 530 530
query94 878 293 282 282
query95 354 258 265 258
query96 607 278 279 278
query97 3218 3038 3060 3038
query98 215 199 198 198
query99 1573 1257 1303 1257
Total cold run time: 283841 ms
Total hot run time: 174074 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](mtmv) Not roll up when aggregate rewrite if roll up group by expr is uniform [doris]
Posted by "morrySnow (via GitHub)" <gi...@apache.org>.
morrySnow commented on PR #38387:
URL: https://github.com/apache/doris/pull/38387#issuecomment-2263011209
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](mtmv) Not roll up when aggregate rewrite if roll up group by expr is uniform [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #38387:
URL: https://github.com/apache/doris/pull/38387#issuecomment-2250212545
<details>
<summary>TPC-H: <b>Total hot run time: 39836 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit e7daa9bd0142b592fa49880030da5202613afe16, data reload: false
------ Round 1 ----------------------------------
q1 18080 4407 4376 4376
q2 2633 253 199 199
q3 11640 1192 1161 1161
q4 11158 692 728 692
q5 7919 2750 2658 2658
q6 224 139 144 139
q7 954 590 585 585
q8 9210 1913 1908 1908
q9 9182 6913 6915 6913
q10 8610 3766 3787 3766
q11 482 246 254 246
q12 399 228 227 227
q13 17758 2990 2974 2974
q14 284 243 236 236
q15 526 488 488 488
q16 502 383 382 382
q17 971 685 766 685
q18 7975 7542 7483 7483
q19 3413 1002 905 905
q20 688 309 321 309
q21 5375 3246 3216 3216
q22 344 288 289 288
Total cold run time: 118327 ms
Total hot run time: 39836 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4389 4248 4256 4248
q2 365 276 272 272
q3 3001 2700 2741 2700
q4 1912 1606 1646 1606
q5 5289 5286 5294 5286
q6 222 128 130 128
q7 2066 1718 1710 1710
q8 3163 3335 3298 3298
q9 8857 8833 8864 8833
q10 3948 3682 3656 3656
q11 599 485 482 482
q12 794 630 632 630
q13 16686 2946 2957 2946
q14 315 271 275 271
q15 509 476 473 473
q16 470 415 411 411
q17 1782 1475 1463 1463
q18 7646 7466 7274 7274
q19 1632 1402 1563 1402
q20 1998 1807 1786 1786
q21 4810 4737 4737 4737
q22 594 534 506 506
Total cold run time: 71047 ms
Total hot run time: 54118 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](mtmv) Not roll up when aggregate rewrite if roll up group by expr is uniform [doris]
Posted by "seawinde (via GitHub)" <gi...@apache.org>.
seawinde commented on PR #38387:
URL: https://github.com/apache/doris/pull/38387#issuecomment-2251879652
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](mtmv) Not roll up when aggregate rewrite if roll up group by expr is uniform [doris]
Posted by "seawinde (via GitHub)" <gi...@apache.org>.
seawinde commented on PR #38387:
URL: https://github.com/apache/doris/pull/38387#issuecomment-2250181700
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](mtmv) Not roll up when aggregate rewrite if roll up group by expr is uniform [doris]
Posted by "seawinde (via GitHub)" <gi...@apache.org>.
seawinde commented on PR #38387:
URL: https://github.com/apache/doris/pull/38387#issuecomment-2254831708
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](mtmv) Not roll up when aggregate rewrite if roll up group by expr is uniform [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #38387:
URL: https://github.com/apache/doris/pull/38387#issuecomment-2254872965
<details>
<summary>TPC-H: <b>Total hot run time: 39631 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 76826d4dca4a4adad654c6e7e227425ee1e8ff87, data reload: false
------ Round 1 ----------------------------------
q1 17867 4429 4300 4300
q2 2017 203 196 196
q3 10491 1183 1095 1095
q4 10268 726 761 726
q5 7538 2734 2680 2680
q6 222 141 140 140
q7 969 601 600 600
q8 9215 1915 1934 1915
q9 8813 6585 6599 6585
q10 8783 3813 3863 3813
q11 453 252 250 250
q12 426 232 228 228
q13 18992 3008 2993 2993
q14 274 244 244 244
q15 518 476 486 476
q16 498 386 383 383
q17 975 693 713 693
q18 8289 7573 7487 7487
q19 4883 1064 1018 1018
q20 665 345 351 345
q21 5040 3192 3417 3192
q22 349 287 272 272
Total cold run time: 117545 ms
Total hot run time: 39631 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4408 4258 4315 4258
q2 364 268 271 268
q3 3034 2949 2928 2928
q4 2016 1694 1788 1694
q5 5663 5545 5543 5543
q6 230 133 138 133
q7 2199 1924 1823 1823
q8 3275 3414 3447 3414
q9 8804 8867 8948 8867
q10 4116 3934 3809 3809
q11 603 510 490 490
q12 825 660 661 660
q13 16119 3212 3234 3212
q14 309 300 298 298
q15 523 494 488 488
q16 506 439 434 434
q17 1836 1540 1496 1496
q18 8175 8012 7805 7805
q19 1744 1489 1701 1489
q20 2119 1859 1896 1859
q21 5207 4998 4841 4841
q22 633 513 513 513
Total cold run time: 72708 ms
Total hot run time: 56322 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](mtmv) Not roll up when aggregate rewrite if roll up group by expr is uniform [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #38387:
URL: https://github.com/apache/doris/pull/38387#issuecomment-2257937046
<details>
<summary>TPC-H: <b>Total hot run time: 42106 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 85e33b048335f4c7b2c839728f7d1ac92a45ec04, data reload: false
------ Round 1 ----------------------------------
q1 7891 4125 4150 4125
q2 423 217 210 210
q3 1533 1438 1408 1408
q4 1147 998 928 928
q5 3055 3019 3012 3012
q6 225 138 137 137
q7 1051 649 624 624
q8 2160 1974 1969 1969
q9 6645 6628 6682 6628
q10 3747 3829 3850 3829
q11 383 257 260 257
q12 417 237 236 236
q13 16821 2963 2972 2963
q14 277 234 243 234
q15 522 474 482 474
q16 497 416 384 384
q17 981 894 867 867
q18 8146 7349 7234 7234
q19 1228 1221 1224 1221
q20 529 346 340 340
q21 5378 4745 4836 4745
q22 355 287 281 281
Total cold run time: 63411 ms
Total hot run time: 42106 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4074 4035 4057 4035
q2 328 221 220 220
q3 3016 2993 3013 2993
q4 1908 1884 1890 1884
q5 5289 5290 5263 5263
q6 219 130 131 130
q7 2073 1669 1700 1669
q8 3248 3312 3352 3312
q9 8391 8801 8638 8638
q10 3974 4057 4050 4050
q11 565 477 455 455
q12 768 593 587 587
q13 3804 3102 3133 3102
q14 321 287 264 264
q15 525 489 490 489
q16 452 403 422 403
q17 1780 1769 1762 1762
q18 8144 7914 7815 7815
q19 1740 1735 1729 1729
q20 2062 1873 1761 1761
q21 5578 5515 5580 5515
q22 524 485 472 472
Total cold run time: 58783 ms
Total hot run time: 56548 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org