You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "englefly (via GitHub)" <gi...@apache.org> on 2026/01/01 03:10:36 UTC
[PR] [fix](CostBasedRewriteJob)restore StatementContext.rewrittenCteProducer in CostBasedRewriteJob [doris]
englefly opened a new pull request, #59517:
URL: https://github.com/apache/doris/pull/59517
### What problem does this PR solve?
cost based rewrite job may contaminate StatementContext.rewrittenCteProducer
fix solution: clone current rewrittenCteProducer, and restore it after each call to getCost(...)
Issue Number: close #xxx
Related PR: #xxx
Problem Summary:
### Release note
None
### Check List (For Author)
- Test <!-- At least one of them must be included. -->
- [ ] Regression test
- [ ] Unit Test
- [ ] Manual test (add detailed scripts or steps below)
- [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
- [ ] Previous test can cover this change.
- [ ] No code files have been changed.
- [ ] Other reason <!-- Add your reason? -->
- Behavior changed:
- [ ] No.
- [ ] Yes. <!-- Explain the behavior change -->
- Does this need documentation?
- [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg: https://github.com/apache/doris-website/pull/1214 -->
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR should merge into -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](CostBasedRewriteJob)restore StatementContext.rewrittenCteProducer in CostBasedRewriteJob [doris]
Posted by "englefly (via GitHub)" <gi...@apache.org>.
englefly commented on PR #59517:
URL: https://github.com/apache/doris/pull/59517#issuecomment-3703204370
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](CostBasedRewriteJob)restore StatementContext.rewrittenCteProducer in CostBasedRewriteJob [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59517:
URL: https://github.com/apache/doris/pull/59517#issuecomment-3743148597
<details>
<summary>TPC-DS: <b>Total hot run time: 173647 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 7e3607af1d53c42cdb7eec85fbc00feddde19256, data reload: false
query5 4455 656 495 495
query6 354 242 220 220
query7 4225 455 279 279
query8 346 246 255 246
query9 8739 2911 2881 2881
query10 527 377 339 339
query11 15352 15053 14819 14819
query12 176 117 121 117
query13 1266 486 375 375
query14 6204 2939 2715 2715
query14_1 2601 2600 2625 2600
query15 198 189 169 169
query16 992 476 456 456
query17 1073 642 571 571
query18 2520 438 318 318
query19 228 219 196 196
query20 125 117 113 113
query21 211 134 120 120
query22 3975 3957 4006 3957
query23 16257 15636 15372 15372
query23_1 15408 15331 15442 15331
query24 7183 1537 1165 1165
query24_1 1163 1159 1161 1159
query25 524 438 385 385
query26 1222 266 154 154
query27 2778 438 260 260
query28 4591 2142 2124 2124
query29 781 523 434 434
query30 311 247 205 205
query31 802 625 557 557
query32 84 76 73 73
query33 516 337 301 301
query34 879 869 537 537
query35 724 763 699 699
query36 899 935 876 876
query37 134 97 81 81
query38 2695 2643 2694 2643
query39 763 777 726 726
query39_1 705 738 702 702
query40 218 130 116 116
query41 64 61 59 59
query42 115 107 109 107
query43 500 425 416 416
query44 1315 735 732 732
query45 186 186 177 177
query46 825 935 575 575
query47 1523 1485 1379 1379
query48 290 322 230 230
query49 604 414 345 345
query50 626 299 211 211
query51 3777 3802 3816 3802
query52 109 108 97 97
query53 296 330 279 279
query54 297 270 259 259
query55 83 80 74 74
query56 316 308 315 308
query57 1003 1029 945 945
query58 280 271 272 271
query59 2150 2194 1915 1915
query60 352 346 332 332
query61 184 187 181 181
query62 391 355 349 349
query63 311 269 274 269
query64 4925 1284 980 980
query65 3790 3789 3757 3757
query66 1450 414 320 320
query67 14839 15070 14941 14941
query68 2712 1005 732 732
query69 460 347 328 328
query70 1006 949 952 949
query71 338 317 291 291
query72 5642 3380 3662 3380
query73 606 710 315 315
query74 8751 8790 8629 8629
query75 2789 2816 2480 2480
query76 2379 1051 689 689
query77 373 382 320 320
query78 9870 10122 9294 9294
query79 1048 889 582 582
query80 1414 602 532 532
query81 545 273 238 238
query82 1358 152 118 118
query83 360 273 258 258
query84 256 125 106 106
query85 1175 489 449 449
query86 370 313 320 313
query87 2855 2888 2747 2747
query88 3026 2145 2107 2107
query89 403 344 322 322
query90 1856 167 160 160
query91 168 168 136 136
query92 80 76 72 72
query93 1007 891 526 526
query94 588 327 291 291
query95 561 337 374 337
query96 557 481 202 202
query97 2364 2350 2338 2338
query98 213 202 197 197
query99 641 623 530 530
Total cold run time: 247677 ms
Total hot run time: 173647 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](CostBasedRewriteJob)restore StatementContext.rewrittenCteProducer in CostBasedRewriteJob [doris]
Posted by "englefly (via GitHub)" <gi...@apache.org>.
englefly commented on PR #59517:
URL: https://github.com/apache/doris/pull/59517#issuecomment-3747411055
run p0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](CostBasedRewriteJob)restore StatementContext.rewrittenCteProducer in CostBasedRewriteJob [doris]
Posted by "englefly (via GitHub)" <gi...@apache.org>.
englefly commented on PR #59517:
URL: https://github.com/apache/doris/pull/59517#issuecomment-3713488063
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](CostBasedRewriteJob)restore StatementContext.rewrittenCteProducer in CostBasedRewriteJob [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #59517:
URL: https://github.com/apache/doris/pull/59517#issuecomment-3782960947
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](CostBasedRewriteJob)restore StatementContext.rewrittenCteProducer in CostBasedRewriteJob [doris]
Posted by "englefly (via GitHub)" <gi...@apache.org>.
englefly commented on code in PR #59517:
URL: https://github.com/apache/doris/pull/59517#discussion_r2663507068
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/jobs/rewrite/CostBasedRewriteJob.java:
##########
@@ -56,6 +61,11 @@ public CostBasedRewriteJob(List<RewriteJob> rewriteJobs) {
// need to generate real rewrite job list
}
+ private void restoreCteProducerMap(StatementContext context, Map<CTEId, LogicalPlan> currentCteProducers) {
+ context.getRewrittenCteProducer().clear();
+ currentCteProducers.forEach(context.getRewrittenCteProducer()::put);
Review Comment:
你的问题是 为什么不put all吗？这里就是put all
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](CostBasedRewriteJob)restore StatementContext.rewrittenCteProducer in CostBasedRewriteJob [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59517:
URL: https://github.com/apache/doris/pull/59517#issuecomment-3703651405
<details>
<summary>TPC-DS: <b>Total hot run time: 172742 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit ef065c65149203a5761c3548d72d363b6ab904c5, data reload: false
query5 4817 584 464 464
query6 335 223 216 216
query7 4221 464 256 256
query8 348 247 241 241
query9 8773 2632 2617 2617
query10 526 385 303 303
query11 15189 15237 14852 14852
query12 171 114 110 110
query13 1248 499 385 385
query14 6257 2960 2783 2783
query14_1 2719 2649 2711 2649
query15 199 196 180 180
query16 981 476 449 449
query17 1098 660 557 557
query18 2494 418 328 328
query19 215 213 184 184
query20 123 113 111 111
query21 213 133 120 120
query22 3858 3956 3782 3782
query23 15727 15485 15235 15235
query23_1 15459 15392 15206 15206
query24 7447 1572 1208 1208
query24_1 1194 1214 1227 1214
query25 525 436 395 395
query26 1238 260 155 155
query27 2763 464 290 290
query28 4593 2168 2169 2168
query29 790 541 444 444
query30 307 240 205 205
query31 796 631 552 552
query32 78 69 67 67
query33 527 328 281 281
query34 923 867 547 547
query35 763 785 716 716
query36 839 865 768 768
query37 123 94 87 87
query38 2770 2670 2626 2626
query39 767 738 750 738
query39_1 728 704 719 704
query40 217 133 116 116
query41 66 61 62 61
query42 102 107 103 103
query43 446 427 425 425
query44 1322 743 758 743
query45 188 188 179 179
query46 882 964 615 615
query47 1447 1445 1305 1305
query48 339 332 252 252
query49 610 408 328 328
query50 636 279 204 204
query51 3757 3773 3708 3708
query52 102 107 97 97
query53 292 327 275 275
query54 278 249 247 247
query55 75 77 72 72
query56 290 300 283 283
query57 1042 1029 932 932
query58 261 245 250 245
query59 2139 2204 1956 1956
query60 342 311 292 292
query61 161 158 170 158
query62 391 346 329 329
query63 302 266 275 266
query64 5052 1408 1143 1143
query65 3798 3745 3605 3605
query66 1440 446 316 316
query67 15109 15147 14983 14983
query68 3606 1044 734 734
query69 493 344 301 301
query70 1063 942 948 942
query71 344 292 276 276
query72 6098 3493 3424 3424
query73 775 709 313 313
query74 8745 8709 8544 8544
query75 2838 2854 2494 2494
query76 3873 1040 652 652
query77 522 355 273 273
query78 9663 9768 9135 9135
query79 960 918 605 605
query80 1144 590 495 495
query81 543 267 241 241
query82 408 142 107 107
query83 388 254 246 246
query84 256 115 105 105
query85 920 538 451 451
query86 390 316 301 301
query87 2796 2859 2853 2853
query88 3213 2284 2272 2272
query89 380 362 322 322
query90 1990 153 140 140
query91 171 166 148 148
query92 69 66 62 62
query93 1019 948 563 563
query94 637 323 278 278
query95 573 323 351 323
query96 586 453 206 206
query97 2298 2372 2315 2315
query98 209 215 211 211
query99 588 614 505 505
Total cold run time: 250043 ms
Total hot run time: 172742 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](CostBasedRewriteJob)restore StatementContext.rewrittenCteProducer in CostBasedRewriteJob [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59517:
URL: https://github.com/apache/doris/pull/59517#issuecomment-3703654723
<details>
<summary>ClickBench: <b>Total hot run time: 27.06 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit ef065c65149203a5761c3548d72d363b6ab904c5, data reload: false
query1 0.05 0.04 0.05
query2 0.09 0.05 0.05
query3 0.26 0.08 0.08
query4 1.60 0.11 0.11
query5 0.27 0.25 0.26
query6 1.15 0.66 0.66
query7 0.03 0.02 0.03
query8 0.06 0.04 0.05
query9 0.58 0.50 0.51
query10 0.56 0.53 0.55
query11 0.16 0.11 0.11
query12 0.17 0.13 0.13
query13 0.62 0.61 0.61
query14 0.99 0.98 0.99
query15 0.80 0.81 0.78
query16 0.38 0.39 0.40
query17 1.04 1.05 1.01
query18 0.23 0.22 0.21
query19 1.91 1.80 1.75
query20 0.02 0.01 0.02
query21 16.41 0.30 0.13
query22 4.66 0.05 0.05
query23 16.05 0.28 0.10
query24 0.97 0.57 0.63
query25 0.12 0.08 0.06
query26 0.14 0.14 0.14
query27 0.09 0.06 0.05
query28 4.20 1.04 0.88
query29 12.60 3.94 3.14
query30 0.28 0.13 0.14
query31 2.81 0.59 0.39
query32 3.23 0.56 0.46
query33 3.05 2.96 3.04
query34 16.49 5.07 4.44
query35 4.49 4.46 4.52
query36 0.67 0.51 0.49
query37 0.11 0.07 0.07
query38 0.07 0.04 0.03
query39 0.04 0.02 0.02
query40 0.16 0.14 0.14
query41 0.08 0.03 0.03
query42 0.05 0.03 0.03
query43 0.04 0.04 0.03
Total cold run time: 97.78 s
Total hot run time: 27.06 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](CostBasedRewriteJob)restore StatementContext.rewrittenCteProducer in CostBasedRewriteJob [doris]
Posted by "englefly (via GitHub)" <gi...@apache.org>.
englefly commented on PR #59517:
URL: https://github.com/apache/doris/pull/59517#issuecomment-3703212016
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](CostBasedRewriteJob)restore StatementContext.rewrittenCteProducer in CostBasedRewriteJob [doris]
Posted by "morrySnow (via GitHub)" <gi...@apache.org>.
morrySnow commented on code in PR #59517:
URL: https://github.com/apache/doris/pull/59517#discussion_r2659501716
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/jobs/rewrite/CostBasedRewriteJob.java:
##########
@@ -56,6 +61,11 @@ public CostBasedRewriteJob(List<RewriteJob> rewriteJobs) {
// need to generate real rewrite job list
}
+ private void restoreCteProducerMap(StatementContext context, Map<CTEId, LogicalPlan> currentCteProducers) {
+ context.getRewrittenCteProducer().clear();
+ currentCteProducers.forEach(context.getRewrittenCteProducer()::put);
Review Comment:
why not put all?
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/jobs/rewrite/CostBasedRewriteJob.java:
##########
@@ -94,8 +111,7 @@ public void execute(JobContext jobContext) {
}
// If the candidate applied cbo rule is better, replace the original plan with it.
if (appliedCboRuleCost.get().first.getValue() < skipCboRuleCost.get().first.getValue()) {
- currentCtx.addPlanProcesses(applyCboRuleCtx.getPlanProcesses());
Review Comment:
why remove `addPlanProcesses`?
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/jobs/rewrite/CostBasedRewriteJob.java:
##########
@@ -69,14 +79,21 @@ public void execute(JobContext jobContext) {
CascadesContext applyCboRuleCtx = CascadesContext.newCurrentTreeContext(currentCtx);
// execute cbo rule on one candidate
Rewriter.getCteChildrenRewriter(applyCboRuleCtx, rewriteJobs).execute();
+ Plan applyCboPlan = applyCboRuleCtx.getRewritePlan();
if (skipCboRuleCtx.getRewritePlan().deepEquals(applyCboRuleCtx.getRewritePlan())) {
// this means rewrite do not do anything
return;
}
+ Map<CTEId, LogicalPlan> currentCteProducers = Maps.newHashMap();
+ // cost based rewrite job may contaminate StatementContext.rewrittenCteProducer
+ // clone current rewrittenCteProducer, and restore it after getCost(.).
+ currentCtx.getStatementContext().getRewrittenCteProducer().forEach(currentCteProducers::put);
Review Comment:
only process `RewrittenCteProducer` is enough? maybe we should process all cte related context just like what we do in `ClearContextStatus`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](CostBasedRewriteJob)restore StatementContext.rewrittenCteProducer in CostBasedRewriteJob [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59517:
URL: https://github.com/apache/doris/pull/59517#issuecomment-3743101360
<details>
<summary>TPC-H: <b>Total hot run time: 31160 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 7e3607af1d53c42cdb7eec85fbc00feddde19256, data reload: false
------ Round 1 ----------------------------------
q1 17657 4311 4072 4072
q2 2040 356 239 239
q3 10146 1263 719 719
q4 10221 897 330 330
q5 7525 2024 1844 1844
q6 184 176 142 142
q7 909 763 637 637
q8 9272 1397 1072 1072
q9 4849 4561 4534 4534
q10 6789 1812 1378 1378
q11 509 284 274 274
q12 663 732 579 579
q13 17784 3787 3065 3065
q14 284 294 276 276
q15 588 512 508 508
q16 682 693 625 625
q17 656 797 476 476
q18 6596 6407 6387 6387
q19 1107 960 579 579
q20 371 357 242 242
q21 2960 2408 2229 2229
q22 1041 1023 953 953
Total cold run time: 102833 ms
Total hot run time: 31160 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4117 4057 4055 4055
q2 340 387 304 304
q3 2076 2590 2217 2217
q4 1323 1735 1335 1335
q5 4095 3963 4103 3963
q6 205 166 132 132
q7 1873 1832 1739 1739
q8 2792 2555 2446 2446
q9 7243 7190 7048 7048
q10 2570 2715 2386 2386
q11 561 476 455 455
q12 739 793 644 644
q13 3550 4008 3334 3334
q14 316 304 300 300
q15 551 576 532 532
q16 660 710 642 642
q17 1158 1406 1455 1406
q18 7918 7838 7967 7838
q19 851 807 833 807
q20 2030 2076 1940 1940
q21 4818 4507 4403 4403
q22 1093 1085 961 961
Total cold run time: 50879 ms
Total hot run time: 48887 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](CostBasedRewriteJob)restore StatementContext.rewrittenCteProducer in CostBasedRewriteJob [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59517:
URL: https://github.com/apache/doris/pull/59517#issuecomment-3713584270
<details>
<summary>TPC-H: <b>Total hot run time: 31683 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 7e3607af1d53c42cdb7eec85fbc00feddde19256, data reload: false
------ Round 1 ----------------------------------
q1 17611 4170 4014 4014
q2 1998 354 235 235
q3 10177 1233 687 687
q4 10216 876 314 314
q5 7546 2100 1811 1811
q6 189 167 135 135
q7 940 804 642 642
q8 9270 1300 1146 1146
q9 4769 4499 4557 4499
q10 6786 1800 1381 1381
q11 518 308 281 281
q12 682 721 574 574
q13 17778 3812 3067 3067
q14 280 289 268 268
q15 570 509 494 494
q16 647 673 616 616
q17 679 828 509 509
q18 6494 6397 6560 6397
q19 1179 1046 646 646
q20 426 402 286 286
q21 3280 2611 2609 2609
q22 1134 1120 1072 1072
Total cold run time: 103169 ms
Total hot run time: 31683 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4357 4274 4253 4253
q2 338 403 321 321
q3 2242 2802 2420 2420
q4 1454 1885 1413 1413
q5 4393 4475 4360 4360
q6 237 171 140 140
q7 1928 1878 1770 1770
q8 2540 2388 2384 2384
q9 7101 7383 6965 6965
q10 2584 2716 2244 2244
q11 560 488 454 454
q12 699 778 645 645
q13 3493 3765 3091 3091
q14 260 296 262 262
q15 532 490 490 490
q16 628 661 605 605
q17 1078 1269 1274 1269
q18 7295 7353 7111 7111
q19 817 784 807 784
q20 1907 1953 1826 1826
q21 4556 4350 4168 4168
q22 1077 1043 962 962
Total cold run time: 50076 ms
Total hot run time: 47937 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](CostBasedRewriteJob)restore StatementContext.rewrittenCteProducer in CostBasedRewriteJob [doris]
Posted by "englefly (via GitHub)" <gi...@apache.org>.
englefly commented on code in PR #59517:
URL: https://github.com/apache/doris/pull/59517#discussion_r2663512980
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/jobs/rewrite/CostBasedRewriteJob.java:
##########
@@ -94,8 +111,7 @@ public void execute(JobContext jobContext) {
}
// If the candidate applied cbo rule is better, replace the original plan with it.
if (appliedCboRuleCost.get().first.getValue() < skipCboRuleCost.get().first.getValue()) {
- currentCtx.addPlanProcesses(applyCboRuleCtx.getPlanProcesses());
Review Comment:
假设plan1 被costBaseRule 改写为 plan2， 然后再被后续rule（不包含其他costBasedRule）改写为plan3
如果选中了costBaseRule的改写，那么退出costBaseRule后，我们应该从plan2继续改写。这里的plan process 记录的是 plan2 到plan3 的过程
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](CostBasedRewriteJob)restore StatementContext.rewrittenCteProducer in CostBasedRewriteJob [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59517:
URL: https://github.com/apache/doris/pull/59517#issuecomment-3703886750
<details>
<summary>TPC-DS: <b>Total hot run time: 173742 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 4438cb164163c15256d4a9a55ecec71ceb694cf8, data reload: false
query5 4871 580 435 435
query6 329 235 219 219
query7 4206 459 271 271
query8 331 249 256 249
query9 8783 2617 2620 2617
query10 528 376 308 308
query11 15165 15125 15006 15006
query12 184 113 114 113
query13 1256 502 391 391
query14 6366 3017 2732 2732
query14_1 2653 2631 2668 2631
query15 205 187 171 171
query16 1022 472 367 367
query17 1066 695 564 564
query18 2615 425 329 329
query19 222 219 189 189
query20 125 117 123 117
query21 215 137 114 114
query22 3879 3886 3814 3814
query23 16198 15634 15378 15378
query23_1 15498 15510 15375 15375
query24 7341 1594 1205 1205
query24_1 1198 1201 1228 1201
query25 518 428 422 422
query26 1242 272 155 155
query27 2766 451 334 334
query28 4502 2197 2199 2197
query29 748 528 424 424
query30 312 242 204 204
query31 785 621 552 552
query32 75 71 67 67
query33 514 338 289 289
query34 891 882 529 529
query35 773 778 700 700
query36 883 882 807 807
query37 132 91 75 75
query38 2762 2717 2662 2662
query39 792 737 739 737
query39_1 714 704 708 704
query40 220 132 113 113
query41 66 64 63 63
query42 110 104 101 101
query43 418 447 430 430
query44 1326 755 743 743
query45 186 182 176 176
query46 873 966 613 613
query47 1389 1457 1320 1320
query48 316 329 241 241
query49 613 418 321 321
query50 650 290 207 207
query51 3715 3792 3797 3792
query52 111 110 93 93
query53 298 326 275 275
query54 289 257 260 257
query55 78 76 75 75
query56 288 291 294 291
query57 1001 969 909 909
query58 266 248 248 248
query59 2060 2141 2095 2095
query60 320 313 309 309
query61 182 189 193 189
query62 405 368 309 309
query63 304 271 274 271
query64 5016 1422 1089 1089
query65 3747 3755 3707 3707
query66 1425 446 330 330
query67 15201 15822 15508 15508
query68 4729 1035 763 763
query69 503 362 320 320
query70 1029 893 979 893
query71 363 313 277 277
query72 6276 3354 3418 3354
query73 779 715 328 328
query74 8747 8805 8609 8609
query75 2837 2832 2464 2464
query76 3950 1062 645 645
query77 516 359 281 281
query78 9836 9986 9237 9237
query79 1145 912 623 623
query80 843 578 480 480
query81 490 270 227 227
query82 411 146 113 113
query83 269 258 236 236
query84 254 123 105 105
query85 893 512 455 455
query86 357 308 305 305
query87 2917 2842 2789 2789
query88 3196 2292 2263 2263
query89 380 357 315 315
query90 1942 155 144 144
query91 172 169 145 145
query92 70 69 63 63
query93 1102 901 565 565
query94 642 332 258 258
query95 578 382 302 302
query96 586 452 208 208
query97 2390 2398 2320 2320
query98 230 195 204 195
query99 583 572 548 548
Total cold run time: 251760 ms
Total hot run time: 173742 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](CostBasedRewriteJob)restore StatementContext.rewrittenCteProducer in CostBasedRewriteJob [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59517:
URL: https://github.com/apache/doris/pull/59517#issuecomment-3703878086
<details>
<summary>TPC-H: <b>Total hot run time: 32582 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 4438cb164163c15256d4a9a55ecec71ceb694cf8, data reload: false
------ Round 1 ----------------------------------
q1 17633 4243 4039 4039
q2 2013 372 244 244
q3 10156 1274 759 759
q4 10229 845 320 320
q5 8107 2100 1963 1963
q6 219 171 139 139
q7 967 814 667 667
q8 9305 1458 1137 1137
q9 5291 4747 4680 4680
q10 6846 1816 1436 1436
q11 510 315 303 303
q12 728 739 583 583
q13 17790 3877 3090 3090
q14 294 294 273 273
q15 587 515 500 500
q16 697 682 627 627
q17 714 738 669 669
q18 6778 6488 7094 6488
q19 1662 1168 649 649
q20 418 372 273 273
q21 3389 2694 2782 2694
q22 1180 1103 1049 1049
Total cold run time: 105513 ms
Total hot run time: 32582 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4396 4267 4300 4267
q2 340 429 329 329
q3 2346 2834 2513 2513
q4 1480 1823 1620 1620
q5 4605 4220 4299 4220
q6 219 176 132 132
q7 1920 1955 1758 1758
q8 2546 2344 2451 2344
q9 7185 7131 7107 7107
q10 2447 2697 2592 2592
q11 560 471 445 445
q12 705 760 598 598
q13 3586 3819 3074 3074
q14 260 281 253 253
q15 520 491 474 474
q16 607 661 589 589
q17 1094 1243 1335 1243
q18 7401 7384 7294 7294
q19 878 849 862 849
q20 1950 1978 1805 1805
q21 4534 4302 4396 4302
q22 1136 1035 1006 1006
Total cold run time: 50715 ms
Total hot run time: 48814 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](CostBasedRewriteJob)restore StatementContext.rewrittenCteProducer in CostBasedRewriteJob [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59517:
URL: https://github.com/apache/doris/pull/59517#issuecomment-3703230297
<details>
<summary>TPC-DS: <b>Total hot run time: 173200 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 073c15115f82ffbe0fe51009f6cfcfc6e8eaa237, data reload: false
query5 4556 607 459 459
query6 338 231 211 211
query7 4215 452 281 281
query8 327 247 255 247
query9 8822 2651 2645 2645
query10 511 357 324 324
query11 15456 15195 14981 14981
query12 178 122 113 113
query13 1263 504 381 381
query14 6247 3028 2776 2776
query14_1 2708 2651 2740 2651
query15 207 201 178 178
query16 1010 474 510 474
query17 1068 673 559 559
query18 2602 440 344 344
query19 228 216 191 191
query20 121 120 113 113
query21 214 142 121 121
query22 3790 4099 3902 3902
query23 16068 15707 15366 15366
query23_1 15595 15367 15454 15367
query24 7443 1580 1211 1211
query24_1 1235 1203 1210 1203
query25 539 439 422 422
query26 1244 269 161 161
query27 2764 474 297 297
query28 4512 2205 2205 2205
query29 804 533 441 441
query30 310 237 219 219
query31 818 631 554 554
query32 75 77 65 65
query33 536 331 283 283
query34 926 879 542 542
query35 771 791 707 707
query36 867 888 800 800
query37 137 95 79 79
query38 2765 2706 2647 2647
query39 779 757 715 715
query39_1 693 703 724 703
query40 220 139 119 119
query41 68 62 62 62
query42 106 106 110 106
query43 442 486 425 425
query44 1400 767 756 756
query45 187 185 175 175
query46 901 993 621 621
query47 1450 1456 1432 1432
query48 318 335 262 262
query49 614 410 328 328
query50 657 278 213 213
query51 3754 3786 3732 3732
query52 104 107 94 94
query53 302 327 275 275
query54 283 270 247 247
query55 80 84 75 75
query56 294 290 295 290
query57 1006 1019 907 907
query58 270 252 254 252
query59 2023 2261 2019 2019
query60 336 328 305 305
query61 196 191 191 191
query62 399 363 343 343
query63 304 272 278 272
query64 5168 1432 1111 1111
query65 3826 3716 3749 3716
query66 1428 448 327 327
query67 15953 14851 14491 14491
query68 4982 1057 751 751
query69 505 353 315 315
query70 1055 884 940 884
query71 362 323 283 283
query72 6231 3507 3521 3507
query73 790 760 323 323
query74 8813 8718 8564 8564
query75 2864 2857 2469 2469
query76 3873 1111 689 689
query77 521 377 282 282
query78 9722 9853 9198 9198
query79 1683 884 622 622
query80 688 582 489 489
query81 504 263 227 227
query82 218 141 109 109
query83 267 257 242 242
query84 263 123 105 105
query85 903 527 466 466
query86 393 319 318 318
query87 2825 2885 2732 2732
query88 3821 2294 2290 2290
query89 405 348 332 332
query90 2225 164 157 157
query91 171 168 142 142
query92 86 70 65 65
query93 1992 944 562 562
query94 573 337 298 298
query95 578 385 312 312
query96 589 484 221 221
query97 2297 2405 2301 2301
query98 247 202 200 200
query99 585 572 520 520
Total cold run time: 255395 ms
Total hot run time: 173200 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](CostBasedRewriteJob)restore StatementContext.rewrittenCteProducer in CostBasedRewriteJob [doris]
Posted by "englefly (via GitHub)" <gi...@apache.org>.
englefly commented on PR #59517:
URL: https://github.com/apache/doris/pull/59517#issuecomment-3747530980
run p0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](CostBasedRewriteJob)restore StatementContext.rewrittenCteProducer in CostBasedRewriteJob [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59517:
URL: https://github.com/apache/doris/pull/59517#issuecomment-3703225040
<details>
<summary>TPC-H: <b>Total hot run time: 32774 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 073c15115f82ffbe0fe51009f6cfcfc6e8eaa237, data reload: false
------ Round 1 ----------------------------------
q1 17681 4284 4053 4053
q2 2052 364 244 244
q3 10168 1280 754 754
q4 10218 902 345 345
q5 7546 2179 1958 1958
q6 189 176 136 136
q7 965 795 673 673
q8 9298 1424 1221 1221
q9 5195 4740 4806 4740
q10 6797 1801 1412 1412
q11 487 318 296 296
q12 696 726 582 582
q13 17795 3804 3162 3162
q14 288 291 273 273
q15 576 510 510 510
q16 713 684 651 651
q17 699 754 652 652
q18 6769 6370 7183 6370
q19 1153 1026 641 641
q20 423 390 274 274
q21 3390 2755 2881 2755
q22 1152 1102 1072 1072
Total cold run time: 104250 ms
Total hot run time: 32774 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4454 4257 4232 4232
q2 328 422 343 343
q3 2386 2746 2334 2334
q4 1417 1863 1461 1461
q5 4560 4257 4291 4257
q6 219 164 127 127
q7 1978 2002 1771 1771
q8 2528 2352 2405 2352
q9 7305 7111 7170 7111
q10 2466 2788 2277 2277
q11 549 506 458 458
q12 768 763 627 627
q13 3644 3985 3440 3440
q14 293 311 276 276
q15 542 497 488 488
q16 630 682 636 636
q17 1066 1278 1265 1265
q18 7486 7408 7060 7060
q19 829 824 852 824
q20 1889 1957 1834 1834
q21 4575 4273 4183 4183
q22 1095 1020 996 996
Total cold run time: 51007 ms
Total hot run time: 48352 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](CostBasedRewriteJob)restore StatementContext.rewrittenCteProducer in CostBasedRewriteJob [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59517:
URL: https://github.com/apache/doris/pull/59517#issuecomment-3713629324
<details>
<summary>TPC-DS: <b>Total hot run time: 172076 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 7e3607af1d53c42cdb7eec85fbc00feddde19256, data reload: false
query5 4730 596 458 458
query6 327 225 220 220
query7 4209 452 267 267
query8 325 236 227 227
query9 8763 2649 2656 2649
query10 512 367 344 344
query11 15213 15194 14797 14797
query12 178 113 126 113
query13 1269 524 391 391
query14 6287 2949 2717 2717
query14_1 2577 2602 2598 2598
query15 197 192 175 175
query16 982 505 469 469
query17 1056 646 551 551
query18 2464 422 337 337
query19 224 221 192 192
query20 126 116 111 111
query21 218 138 118 118
query22 4097 4182 4140 4140
query23 15995 15478 15262 15262
query23_1 15694 15565 15458 15458
query24 7470 1522 1174 1174
query24_1 1188 1188 1166 1166
query25 529 422 378 378
query26 1229 255 149 149
query27 2790 437 289 289
query28 4603 2129 2116 2116
query29 806 545 446 446
query30 296 244 205 205
query31 794 608 525 525
query32 78 66 67 66
query33 526 315 306 306
query34 898 868 512 512
query35 760 791 688 688
query36 815 853 812 812
query37 129 92 75 75
query38 2684 2686 2666 2666
query39 800 742 736 736
query39_1 707 707 714 707
query40 217 126 112 112
query41 65 62 59 59
query42 101 104 106 104
query43 457 426 421 421
query44 1344 726 722 722
query45 186 180 170 170
query46 844 969 603 603
query47 1413 1459 1310 1310
query48 307 327 258 258
query49 605 397 342 342
query50 619 270 206 206
query51 3788 3810 3759 3759
query52 104 105 95 95
query53 294 324 274 274
query54 276 254 255 254
query55 76 74 69 69
query56 280 285 298 285
query57 981 1001 970 970
query58 259 256 269 256
query59 2068 2020 2009 2009
query60 314 303 292 292
query61 195 188 180 180
query62 394 362 328 328
query63 303 266 280 266
query64 5146 1393 972 972
query65 3689 3741 3710 3710
query66 1423 434 308 308
query67 14602 15207 14852 14852
query68 2739 1004 735 735
query69 452 353 309 309
query70 994 944 861 861
query71 313 301 274 274
query72 6144 3359 3464 3359
query73 579 708 298 298
query74 8796 8713 8505 8505
query75 2785 2814 2465 2465
query76 2893 1050 641 641
query77 340 372 287 287
query78 9628 9793 9113 9113
query79 941 938 580 580
query80 601 545 480 480
query81 471 260 233 233
query82 207 145 109 109
query83 262 255 240 240
query84 252 117 102 102
query85 844 531 457 457
query86 316 332 309 309
query87 2829 2854 2759 2759
query88 3194 2242 2217 2217
query89 389 343 338 338
query90 2049 156 154 154
query91 166 169 152 152
query92 71 66 63 63
query93 926 902 535 535
query94 455 321 273 273
query95 583 319 348 319
query96 589 462 210 210
query97 2306 2340 2286 2286
query98 201 200 196 196
query99 598 591 528 528
Total cold run time: 245863 ms
Total hot run time: 172076 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](CostBasedRewriteJob)restore StatementContext.rewrittenCteProducer in CostBasedRewriteJob [doris]
Posted by "englefly (via GitHub)" <gi...@apache.org>.
englefly merged PR #59517:
URL: https://github.com/apache/doris/pull/59517
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](CostBasedRewriteJob)restore StatementContext.rewrittenCteProducer in CostBasedRewriteJob [doris]
Posted by "englefly (via GitHub)" <gi...@apache.org>.
englefly commented on PR #59517:
URL: https://github.com/apache/doris/pull/59517#issuecomment-3703611746
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](CostBasedRewriteJob)restore StatementContext.rewrittenCteProducer in CostBasedRewriteJob [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #59517:
URL: https://github.com/apache/doris/pull/59517#issuecomment-3704012621
# FE Regression Coverage Report
Increment line coverage ` 100.00% (9/9)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/59517_4438cb164163c15256d4a9a55ecec71ceb694cf8_merge_fe/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/59517_4438cb164163c15256d4a9a55ecec71ceb694cf8_merge_fe/report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](CostBasedRewriteJob)restore StatementContext.rewrittenCteProducer in CostBasedRewriteJob [doris]
Posted by "englefly (via GitHub)" <gi...@apache.org>.
englefly commented on PR #59517:
URL: https://github.com/apache/doris/pull/59517#issuecomment-3742851750
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](CostBasedRewriteJob)restore StatementContext.rewrittenCteProducer in CostBasedRewriteJob [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #59517:
URL: https://github.com/apache/doris/pull/59517#issuecomment-3782961086
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](CostBasedRewriteJob)restore StatementContext.rewrittenCteProducer in CostBasedRewriteJob [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59517:
URL: https://github.com/apache/doris/pull/59517#issuecomment-3703643956
<details>
<summary>TPC-H: <b>Total hot run time: 32270 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit ef065c65149203a5761c3548d72d363b6ab904c5, data reload: false
------ Round 1 ----------------------------------
q1 17593 4264 4193 4193
q2 2020 366 234 234
q3 10187 1271 733 733
q4 10215 842 338 338
q5 7550 2171 1907 1907
q6 196 178 146 146
q7 978 821 664 664
q8 9320 1442 1184 1184
q9 5225 4666 4678 4666
q10 6821 1796 1407 1407
q11 506 310 309 309
q12 679 718 612 612
q13 17776 3789 3090 3090
q14 300 295 292 292
q15 596 518 500 500
q16 701 675 646 646
q17 710 727 598 598
q18 6693 6659 6487 6487
q19 1100 968 619 619
q20 440 378 252 252
q21 3139 2541 2414 2414
q22 1068 998 979 979
Total cold run time: 103813 ms
Total hot run time: 32270 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4098 4031 4039 4031
q2 321 403 338 338
q3 2087 2619 2238 2238
q4 1336 1760 1331 1331
q5 4079 3954 4093 3954
q6 207 173 132 132
q7 1823 1774 2028 1774
q8 2573 2454 2399 2399
q9 7193 7163 7133 7133
q10 2607 2737 2330 2330
q11 582 471 471 471
q12 703 802 651 651
q13 3698 3984 3477 3477
q14 304 298 270 270
q15 542 505 509 505
q16 657 687 671 671
q17 1167 1313 1584 1313
q18 8081 8021 7541 7541
q19 941 912 949 912
q20 2082 2073 1941 1941
q21 4878 4547 4166 4166
q22 1094 1029 972 972
Total cold run time: 51053 ms
Total hot run time: 48550 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](CostBasedRewriteJob)restore StatementContext.rewrittenCteProducer in CostBasedRewriteJob [doris]
Posted by "englefly (via GitHub)" <gi...@apache.org>.
englefly commented on PR #59517:
URL: https://github.com/apache/doris/pull/59517#issuecomment-3703848659
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](CostBasedRewriteJob)restore StatementContext.rewrittenCteProducer in CostBasedRewriteJob [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59517:
URL: https://github.com/apache/doris/pull/59517#issuecomment-3703891511
<details>
<summary>ClickBench: <b>Total hot run time: 27.28 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 4438cb164163c15256d4a9a55ecec71ceb694cf8, data reload: false
query1 0.05 0.04 0.04
query2 0.10 0.05 0.05
query3 0.26 0.08 0.08
query4 1.60 0.12 0.11
query5 0.28 0.27 0.25
query6 1.14 0.66 0.65
query7 0.04 0.02 0.03
query8 0.05 0.05 0.04
query9 0.57 0.50 0.49
query10 0.56 0.54 0.55
query11 0.15 0.10 0.11
query12 0.16 0.13 0.13
query13 0.62 0.59 0.59
query14 1.00 0.99 0.98
query15 0.81 0.82 0.79
query16 0.40 0.41 0.42
query17 1.06 0.99 1.00
query18 0.24 0.21 0.22
query19 1.82 1.82 1.83
query20 0.02 0.02 0.01
query21 15.46 0.27 0.15
query22 4.71 0.05 0.05
query23 15.82 0.28 0.11
query24 0.94 0.68 0.79
query25 0.12 0.08 0.06
query26 0.14 0.13 0.14
query27 0.06 0.06 0.06
query28 5.46 1.05 0.88
query29 12.57 3.91 3.12
query30 0.27 0.14 0.11
query31 2.81 0.63 0.37
query32 3.23 0.55 0.49
query33 3.08 3.03 3.05
query34 16.74 5.09 4.47
query35 4.46 4.43 4.55
query36 0.68 0.50 0.49
query37 0.10 0.06 0.06
query38 0.06 0.04 0.04
query39 0.05 0.03 0.03
query40 0.18 0.15 0.13
query41 0.09 0.04 0.03
query42 0.04 0.03 0.03
query43 0.04 0.04 0.03
Total cold run time: 98.04 s
Total hot run time: 27.28 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](CostBasedRewriteJob)restore StatementContext.rewrittenCteProducer in CostBasedRewriteJob [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59517:
URL: https://github.com/apache/doris/pull/59517#issuecomment-3713642859
<details>
<summary>ClickBench: <b>Total hot run time: 26.51 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 7e3607af1d53c42cdb7eec85fbc00feddde19256, data reload: false
query1 0.05 0.04 0.04
query2 0.09 0.04 0.04
query3 0.25 0.08 0.08
query4 1.60 0.12 0.11
query5 0.28 0.26 0.26
query6 1.17 0.66 0.64
query7 0.03 0.03 0.02
query8 0.05 0.03 0.04
query9 0.57 0.51 0.49
query10 0.55 0.55 0.55
query11 0.14 0.10 0.10
query12 0.15 0.10 0.11
query13 0.59 0.60 0.57
query14 0.95 0.94 0.94
query15 0.79 0.76 0.78
query16 0.40 0.41 0.42
query17 1.04 1.04 0.99
query18 0.23 0.21 0.21
query19 1.96 1.78 1.87
query20 0.01 0.01 0.02
query21 15.43 0.23 0.13
query22 5.17 0.05 0.04
query23 15.96 0.28 0.10
query24 1.61 0.35 0.22
query25 0.08 0.10 0.05
query26 0.14 0.13 0.13
query27 0.12 0.06 0.05
query28 3.17 1.05 0.87
query29 12.61 3.88 3.14
query30 0.28 0.14 0.11
query31 2.83 0.64 0.39
query32 3.24 0.57 0.45
query33 2.92 3.10 3.04
query34 16.92 5.09 4.38
query35 4.47 4.49 4.46
query36 0.66 0.50 0.49
query37 0.12 0.06 0.06
query38 0.07 0.04 0.03
query39 0.04 0.02 0.02
query40 0.17 0.14 0.13
query41 0.08 0.03 0.04
query42 0.05 0.03 0.03
query43 0.04 0.04 0.03
Total cold run time: 97.08 s
Total hot run time: 26.51 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](CostBasedRewriteJob)restore StatementContext.rewrittenCteProducer in CostBasedRewriteJob [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #59517:
URL: https://github.com/apache/doris/pull/59517#issuecomment-3743289750
# FE UT Coverage Report
Increment line coverage ` 91.84% (45/49)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/59517_7e3607af1d53c42cdb7eec85fbc00feddde19256/fe_increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/59517_7e3607af1d53c42cdb7eec85fbc00feddde19256/fe_report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](CostBasedRewriteJob)restore StatementContext.rewrittenCteProducer in CostBasedRewriteJob [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59517:
URL: https://github.com/apache/doris/pull/59517#issuecomment-3703233132
<details>
<summary>ClickBench: <b>Total hot run time: 26.79 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 073c15115f82ffbe0fe51009f6cfcfc6e8eaa237, data reload: false
query1 0.05 0.04 0.04
query2 0.10 0.05 0.05
query3 0.26 0.08 0.09
query4 1.61 0.12 0.11
query5 0.27 0.25 0.26
query6 1.14 0.66 0.64
query7 0.04 0.02 0.02
query8 0.05 0.04 0.04
query9 0.58 0.50 0.49
query10 0.56 0.56 0.55
query11 0.16 0.11 0.12
query12 0.16 0.13 0.14
query13 0.61 0.61 0.60
query14 0.99 1.00 0.96
query15 0.80 0.78 0.82
query16 0.42 0.42 0.41
query17 1.06 1.04 1.04
query18 0.22 0.21 0.22
query19 1.86 1.78 1.84
query20 0.02 0.02 0.01
query21 15.47 0.28 0.14
query22 4.86 0.05 0.05
query23 16.25 0.29 0.10
query24 1.34 0.75 0.19
query25 0.12 0.08 0.07
query26 0.14 0.13 0.14
query27 0.10 0.08 0.06
query28 5.18 1.07 0.88
query29 12.59 3.92 3.17
query30 0.30 0.17 0.11
query31 2.82 0.61 0.38
query32 3.23 0.55 0.46
query33 3.08 2.98 2.99
query34 16.65 5.07 4.48
query35 4.48 4.54 4.46
query36 0.66 0.51 0.49
query37 0.10 0.06 0.07
query38 0.07 0.04 0.04
query39 0.05 0.03 0.03
query40 0.17 0.15 0.13
query41 0.09 0.04 0.03
query42 0.04 0.03 0.02
query43 0.05 0.04 0.03
Total cold run time: 98.8 s
Total hot run time: 26.79 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](CostBasedRewriteJob)restore StatementContext.rewrittenCteProducer in CostBasedRewriteJob [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #59517:
URL: https://github.com/apache/doris/pull/59517#issuecomment-3703204318
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR).
Please clearly describe your PR:
1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
3. What features were added. Why was this function added?
4. Which code was refactored and why was this part of the code refactored?
5. Which functions were optimized and what is the difference before and after the optimization?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](CostBasedRewriteJob)restore StatementContext.rewrittenCteProducer in CostBasedRewriteJob [doris]
Posted by "englefly (via GitHub)" <gi...@apache.org>.
englefly commented on code in PR #59517:
URL: https://github.com/apache/doris/pull/59517#discussion_r2663925583
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/jobs/rewrite/CostBasedRewriteJob.java:
##########
@@ -69,14 +79,21 @@ public void execute(JobContext jobContext) {
CascadesContext applyCboRuleCtx = CascadesContext.newCurrentTreeContext(currentCtx);
// execute cbo rule on one candidate
Rewriter.getCteChildrenRewriter(applyCboRuleCtx, rewriteJobs).execute();
+ Plan applyCboPlan = applyCboRuleCtx.getRewritePlan();
if (skipCboRuleCtx.getRewritePlan().deepEquals(applyCboRuleCtx.getRewritePlan())) {
// this means rewrite do not do anything
return;
}
+ Map<CTEId, LogicalPlan> currentCteProducers = Maps.newHashMap();
+ // cost based rewrite job may contaminate StatementContext.rewrittenCteProducer
+ // clone current rewrittenCteProducer, and restore it after getCost(.).
+ currentCtx.getStatementContext().getRewrittenCteProducer().forEach(currentCteProducers::put);
Review Comment:
增加了CteEnvironmentSnapshot, 恢复所有cte相关数据
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](CostBasedRewriteJob)restore StatementContext.rewrittenCteProducer in CostBasedRewriteJob [doris]
Posted by "englefly (via GitHub)" <gi...@apache.org>.
englefly commented on PR #59517:
URL: https://github.com/apache/doris/pull/59517#issuecomment-3783297300
run nonConcurrent
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org