You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "seawinde (via GitHub)" <gi...@apache.org> on 2024/07/19 03:01:05 UTC
[PR] [feature](mtmv) Support querying rewrite by materialized view when insert and insert overwrite dml [doris]
seawinde opened a new pull request, #38115:
URL: https://github.com/apache/doris/pull/38115
## Proposed changes
Support querying rewrite by materialized view when DML such as insert and insert overwrite , into outfile etc.
this is default enable, you can disable it by `set enable_dml_materialized_view_rewrite = false`
and the `enable_materialized_view_rewrite` is only control DQL.
It would not rewrite by materialized view when query sql use external table.
If you want rewrite by materialized view when use external table in DML,
you should `set enable_dml_materialized_view_rewrite_when_base_table_unawareness = true;`
this is default false.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](mtmv) Support querying rewrite by materialized view when insert and insert overwrite dml [doris]
Posted by "seawinde (via GitHub)" <gi...@apache.org>.
seawinde commented on PR #38115:
URL: https://github.com/apache/doris/pull/38115#issuecomment-2237996615
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](mtmv) Support querying rewrite by materialized view when insert and insert overwrite dml [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #38115:
URL: https://github.com/apache/doris/pull/38115#issuecomment-2238020213
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](mtmv) Support querying rewrite by materialized view when insert and insert overwrite dml [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #38115:
URL: https://github.com/apache/doris/pull/38115#issuecomment-2238076298
<details>
<summary>TPC-DS: <b>Total hot run time: 174071 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit feb0d60795b843b8180dc2fe86f6358ea0a7630e, data reload: false
query1 928 379 368 368
query2 6451 1808 1833 1808
query3 6664 207 215 207
query4 23566 17438 17378 17378
query5 3727 480 475 475
query6 289 165 160 160
query7 4590 288 283 283
query8 238 187 185 185
query9 8448 2451 2454 2451
query10 436 274 262 262
query11 10812 10169 10101 10101
query12 110 79 83 79
query13 1633 368 363 363
query14 10403 7566 7670 7566
query15 214 165 160 160
query16 7576 343 323 323
query17 1326 578 541 541
query18 1809 273 268 268
query19 199 160 147 147
query20 89 80 85 80
query21 203 132 130 130
query22 4313 4137 4004 4004
query23 33820 33900 33765 33765
query24 10956 2941 2954 2941
query25 609 383 385 383
query26 699 146 144 144
query27 2258 274 276 274
query28 5873 2103 2106 2103
query29 861 637 634 634
query30 264 159 153 153
query31 953 796 748 748
query32 97 53 52 52
query33 647 295 292 292
query34 893 484 503 484
query35 705 587 569 569
query36 1134 979 965 965
query37 155 81 91 81
query38 2984 2961 2838 2838
query39 910 868 868 868
query40 236 122 122 122
query41 47 44 46 44
query42 113 98 94 94
query43 502 458 465 458
query44 1109 756 736 736
query45 210 161 158 158
query46 1092 746 749 746
query47 1868 1816 1782 1782
query48 359 295 296 295
query49 819 405 407 405
query50 783 385 390 385
query51 6887 6815 6701 6701
query52 107 91 94 91
query53 357 286 290 286
query54 845 450 440 440
query55 78 74 71 71
query56 278 256 287 256
query57 1130 1065 1046 1046
query58 240 250 250 250
query59 2808 2584 2576 2576
query60 333 277 270 270
query61 93 93 92 92
query62 812 631 636 631
query63 316 286 284 284
query64 9165 2227 1656 1656
query65 3200 3139 3119 3119
query66 756 343 347 343
query67 15555 14966 15040 14966
query68 4573 545 540 540
query69 668 446 408 408
query70 1206 1135 1128 1128
query71 402 288 273 273
query72 7426 5614 6013 5614
query73 771 327 326 326
query74 6158 5667 5730 5667
query75 3397 2698 2735 2698
query76 2639 909 1005 909
query77 624 299 310 299
query78 9647 10518 9164 9164
query79 3460 520 526 520
query80 949 482 467 467
query81 568 217 220 217
query82 747 142 132 132
query83 190 166 167 166
query84 273 88 85 85
query85 885 305 291 291
query86 452 289 303 289
query87 3281 3095 3208 3095
query88 4123 2368 2389 2368
query89 476 377 384 377
query90 1740 191 189 189
query91 196 102 97 97
query92 60 47 49 47
query93 3622 517 509 509
query94 1014 216 210 210
query95 414 323 321 321
query96 607 272 279 272
query97 3237 3108 3022 3022
query98 223 199 197 197
query99 1566 1251 1238 1238
Total cold run time: 273534 ms
Total hot run time: 174071 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](mtmv) Support querying rewrite by materialized view when insert and insert overwrite dml [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #38115:
URL: https://github.com/apache/doris/pull/38115#issuecomment-2238020590
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](mtmv) Support querying rewrite by materialized view when insert and insert overwrite dml [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #38115:
URL: https://github.com/apache/doris/pull/38115#issuecomment-2238067933
<details>
<summary>TPC-H: <b>Total hot run time: 39676 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit feb0d60795b843b8180dc2fe86f6358ea0a7630e, data reload: false
------ Round 1 ----------------------------------
q1 17633 4754 4266 4266
q2 2008 191 187 187
q3 10457 1270 1094 1094
q4 10190 801 750 750
q5 7526 2704 2681 2681
q6 224 134 136 134
q7 952 591 607 591
q8 9214 2090 2053 2053
q9 8795 6584 6556 6556
q10 8764 3769 3846 3769
q11 512 244 249 244
q12 444 231 231 231
q13 17762 2971 2981 2971
q14 283 239 240 239
q15 540 486 473 473
q16 488 383 373 373
q17 966 681 708 681
q18 8030 7533 7334 7334
q19 6508 1291 1478 1291
q20 683 338 346 338
q21 4987 3141 3239 3141
q22 353 288 279 279
Total cold run time: 117319 ms
Total hot run time: 39676 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4449 4371 4238 4238
q2 375 254 267 254
q3 2991 2928 2940 2928
q4 2022 1641 1784 1641
q5 5594 5560 5479 5479
q6 228 132 129 129
q7 2236 1882 1839 1839
q8 3234 3396 3394 3394
q9 8794 8781 8881 8781
q10 4124 3887 3790 3790
q11 590 506 498 498
q12 827 630 659 630
q13 16034 3205 3190 3190
q14 312 285 279 279
q15 528 498 504 498
q16 502 432 443 432
q17 1844 1518 1508 1508
q18 8166 7849 7911 7849
q19 1715 1529 1655 1529
q20 2198 1883 1854 1854
q21 5083 4915 4712 4712
q22 625 496 493 493
Total cold run time: 72471 ms
Total hot run time: 55945 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](mtmv) Support querying rewrite by materialized view when insert and insert overwrite dml [doris]
Posted by "seawinde (via GitHub)" <gi...@apache.org>.
seawinde commented on PR #38115:
URL: https://github.com/apache/doris/pull/38115#issuecomment-2237989281
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](mtmv) Support querying rewrite by materialized view when insert and insert overwrite dml [doris]
Posted by "morrySnow (via GitHub)" <gi...@apache.org>.
morrySnow merged PR #38115:
URL: https://github.com/apache/doris/pull/38115
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](mtmv) Support querying rewrite by materialized view when insert and insert overwrite dml [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #38115:
URL: https://github.com/apache/doris/pull/38115#issuecomment-2238080436
<details>
<summary>ClickBench: <b>Total hot run time: 30.39 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit feb0d60795b843b8180dc2fe86f6358ea0a7630e, data reload: false
query1 0.04 0.04 0.03
query2 0.08 0.04 0.04
query3 0.22 0.04 0.05
query4 1.67 0.07 0.07
query5 0.50 0.48 0.49
query6 1.13 0.73 0.72
query7 0.02 0.01 0.02
query8 0.05 0.05 0.05
query9 0.54 0.49 0.48
query10 0.54 0.55 0.54
query11 0.15 0.11 0.11
query12 0.14 0.11 0.12
query13 0.59 0.59 0.59
query14 0.76 0.79 0.76
query15 0.85 0.82 0.80
query16 0.34 0.35 0.35
query17 0.99 1.04 1.03
query18 0.23 0.21 0.22
query19 1.85 1.84 1.75
query20 0.01 0.01 0.01
query21 15.39 0.78 0.68
query22 4.58 7.23 1.70
query23 18.24 1.36 1.24
query24 2.16 0.24 0.22
query25 0.15 0.08 0.09
query26 0.29 0.20 0.21
query27 0.45 0.23 0.23
query28 13.17 1.01 1.00
query29 12.63 3.42 3.34
query30 0.25 0.05 0.05
query31 2.87 0.40 0.38
query32 3.28 0.46 0.47
query33 2.92 2.92 2.90
query34 17.03 4.41 4.33
query35 4.48 4.38 4.41
query36 0.65 0.46 0.48
query37 0.18 0.15 0.16
query38 0.15 0.15 0.14
query39 0.04 0.03 0.04
query40 0.15 0.12 0.12
query41 0.09 0.04 0.05
query42 0.05 0.05 0.05
query43 0.05 0.04 0.03
Total cold run time: 109.95 s
Total hot run time: 30.39 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](mtmv) Support querying rewrite by materialized view when insert and insert overwrite dml [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #38115:
URL: https://github.com/apache/doris/pull/38115#issuecomment-2237986829
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR)
Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website).
See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org