You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "morrySnow (via GitHub)" <gi...@apache.org> on 2025/12/03 08:33:52 UTC
[PR] [clean](planner) remove legacy planner's constant folding [doris]
morrySnow opened a new pull request, #58680:
URL: https://github.com/apache/doris/pull/58680
### What problem does this PR solve?
Problem Summary:
This pull request removes support for evaluating partition expressions defined as functions (e.g., function(column)) during partition pruning and related logic. The changes simplify the handling of binary predicates by eliminating function-based partition expression handling, and remove several related classes and utilities that are no longer needed.
Key changes include:
**Removal of function-based partition expression support:**
* The `invokeFunctionExpr` method and all related logic for handling `FunctionCallExpr` in partition expressions have been removed from `BinaryPredicate`. The `getSlotBinding` method now only supports direct slot references and no longer accepts or processes a list of partition expressions. [[1]](diffhunk://#diff-f491835ce20e42c028131456251d8c55595f933c9cff3fa24cae8f49398c2cc8L193-L245) [[2]](diffhunk://#diff-f491835ce20e42c028131456251d8c55595f933c9cff3fa24cae8f49398c2cc8L256-L274)
* The `ScanNode` class has been updated to stop passing partition expressions to `getSlotBinding`, reflecting the simplified interface. [[1]](diffhunk://#diff-cf5d50dd05d6bc67971a030ce14568d4cef15441387f152b561c888af7449611L262-R262) [[2]](diffhunk://#diff-cf5d50dd05d6bc67971a030ce14568d4cef15441387f152b561c888af7449611L355-R352)
**Code cleanup and removal of unused utilities:**
* The entire `ExpressionFunctions` class, which provided function evaluation utilities for partition expressions, has been deleted.
* The annotation classes `FEFunction` and `FEFunctionList`, which were used for registering and managing function metadata for partition pruning, have also been removed. [[1]](diffhunk://#diff-341fd18912433d2314de90785019094bbaea1fd99346e9eb34b647989d7320f0L1-L33) [[2]](diffhunk://#diff-47b3cfee971787db2e7a971765133b3035a7cef3d5c7572d6e03132576bca135L1-L29)
These changes collectively simplify the codebase and partition pruning logic by removing support and code paths for function-based partition expressions.
### Release note
None
### Check List (For Author)
- Test <!-- At least one of them must be included. -->
- [ ] Regression test
- [ ] Unit Test
- [ ] Manual test (add detailed scripts or steps below)
- [x] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
- [x] Previous test can cover this change.
- [ ] No code files have been changed.
- [ ] Other reason <!-- Add your reason? -->
- Behavior changed:
- [x] No.
- [ ] Yes. <!-- Explain the behavior change -->
- Does this need documentation?
- [x] No.
- [ ] Yes. <!-- Add document PR link here. eg: https://github.com/apache/doris-website/pull/1214 -->
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR should merge into -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [clean](planner) remove legacy planner's constant folding [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #58680:
URL: https://github.com/apache/doris/pull/58680#issuecomment-3606034181
<details>
<summary>ClickBench: <b>Total hot run time: 27.43 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit bf63e613ccf344adfeb40d08e2f97408fe1dc57b, data reload: false
query1 0.05 0.04 0.04
query2 0.10 0.05 0.04
query3 0.26 0.09 0.09
query4 1.61 0.12 0.12
query5 0.27 0.24 0.26
query6 1.16 0.63 0.63
query7 0.03 0.02 0.02
query8 0.05 0.04 0.04
query9 0.57 0.52 0.50
query10 0.56 0.57 0.56
query11 0.16 0.11 0.11
query12 0.15 0.12 0.11
query13 0.64 0.61 0.61
query14 1.00 0.99 0.98
query15 0.81 0.80 0.80
query16 0.40 0.41 0.42
query17 1.01 1.03 1.04
query18 0.23 0.21 0.22
query19 1.93 1.89 1.89
query20 0.02 0.02 0.01
query21 15.46 0.28 0.13
query22 5.01 0.05 0.05
query23 16.05 0.29 0.10
query24 1.46 0.30 0.25
query25 0.08 0.04 0.06
query26 0.14 0.13 0.13
query27 0.07 0.06 0.05
query28 4.20 1.23 1.03
query29 12.66 3.99 3.19
query30 0.27 0.13 0.15
query31 2.82 0.63 0.40
query32 3.23 0.56 0.48
query33 3.04 3.04 3.14
query34 16.74 5.25 4.59
query35 4.63 4.53 4.56
query36 0.66 0.49 0.50
query37 0.11 0.07 0.06
query38 0.07 0.04 0.04
query39 0.05 0.03 0.03
query40 0.19 0.16 0.14
query41 0.08 0.04 0.03
query42 0.04 0.03 0.03
query43 0.05 0.04 0.03
Total cold run time: 98.12 s
Total hot run time: 27.43 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [clean](planner) remove legacy planner's constant folding [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #58680:
URL: https://github.com/apache/doris/pull/58680#issuecomment-3609814300
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [clean](planner) remove legacy planner's constant folding [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #58680:
URL: https://github.com/apache/doris/pull/58680#issuecomment-3606011094
<details>
<summary>TPC-DS: <b>Total hot run time: 183699 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit bf63e613ccf344adfeb40d08e2f97408fe1dc57b, data reload: false
query1 1192 414 398 398
query2 6595 1225 1167 1167
query3 6817 237 224 224
query4 25563 23358 22971 22971
query5 4920 653 521 521
query6 347 249 245 245
query7 4682 535 321 321
query8 295 266 246 246
query9 8724 2679 2662 2662
query10 499 384 322 322
query11 15650 15182 15136 15136
query12 182 119 119 119
query13 1679 585 458 458
query14 9520 6134 6091 6091
query15 212 198 182 182
query16 7488 716 537 537
query17 1190 795 613 613
query18 2037 433 381 381
query19 207 209 181 181
query20 126 127 127 127
query21 221 139 113 113
query22 3984 3932 3846 3846
query23 33043 32076 32087 32076
query24 8506 2416 2464 2416
query25 633 546 452 452
query26 1258 275 168 168
query27 2735 522 355 355
query28 4340 2199 2186 2186
query29 876 629 501 501
query30 322 247 208 208
query31 828 717 636 636
query32 90 78 79 78
query33 593 385 341 341
query34 866 892 549 549
query35 803 852 751 751
query36 892 943 831 831
query37 121 114 95 95
query38 3985 3895 3800 3800
query39 1459 1448 1427 1427
query40 228 140 124 124
query41 68 65 64 64
query42 132 118 112 112
query43 433 461 432 432
query44 1315 764 781 764
query45 204 195 188 188
query46 914 1035 664 664
query47 1685 1737 1655 1655
query48 407 427 317 317
query49 791 509 412 412
query50 697 704 411 411
query51 3953 3953 3917 3917
query52 114 114 107 107
query53 250 272 198 198
query54 324 329 311 311
query55 106 104 99 99
query56 357 366 357 357
query57 1152 1171 1092 1092
query58 310 297 295 295
query59 2331 2391 2382 2382
query60 366 375 367 367
query61 202 194 187 187
query62 811 698 662 662
query63 237 202 210 202
query64 4727 1344 1016 1016
query65 4091 3988 3986 3986
query66 1148 465 353 353
query67 15395 15026 15105 15026
query68 8384 1003 645 645
query69 543 369 323 323
query70 1139 1040 1058 1040
query71 435 356 338 338
query72 5707 4822 4846 4822
query73 678 597 355 355
query74 8750 8774 8805 8774
query75 3054 3027 2529 2529
query76 3255 1153 742 742
query77 550 415 333 333
query78 9569 9684 8967 8967
query79 2579 857 597 597
query80 699 609 494 494
query81 507 274 242 242
query82 489 134 113 113
query83 282 270 249 249
query84 262 123 96 96
query85 962 490 456 456
query86 384 297 303 297
query87 4036 4120 3964 3964
query88 4255 2279 2327 2279
query89 401 332 296 296
query90 1923 223 212 212
query91 175 173 146 146
query92 87 76 70 70
query93 2376 1044 688 688
query94 801 469 345 345
query95 517 426 415 415
query96 550 566 289 289
query97 2594 2676 2603 2603
query98 252 225 221 221
query99 1333 1408 1328 1328
Total cold run time: 273756 ms
Total hot run time: 183699 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [clean](planner) remove legacy planner's constant folding [doris]
Posted by "morrySnow (via GitHub)" <gi...@apache.org>.
morrySnow commented on PR #58680:
URL: https://github.com/apache/doris/pull/58680#issuecomment-3605662931
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [clean](planner) remove legacy planner's constant folding [doris]
Posted by "morrySnow (via GitHub)" <gi...@apache.org>.
morrySnow merged PR #58680:
URL: https://github.com/apache/doris/pull/58680
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [clean](planner) remove legacy planner's constant folding [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #58680:
URL: https://github.com/apache/doris/pull/58680#issuecomment-3605964726
<details>
<summary>TPC-H: <b>Total hot run time: 34479 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit bf63e613ccf344adfeb40d08e2f97408fe1dc57b, data reload: false
------ Round 1 ----------------------------------
q1 17965 5037 4918 4918
q2 2175 314 212 212
q3 10446 1347 770 770
q4 10276 813 320 320
q5 7541 2358 2236 2236
q6 185 169 135 135
q7 957 765 664 664
q8 9397 1405 1122 1122
q9 7103 5331 5370 5331
q10 7216 2183 1803 1803
q11 685 325 298 298
q12 703 386 222 222
q13 18448 3670 3028 3028
q14 271 250 225 225
q15 608 511 522 511
q16 920 851 797 797
q17 758 867 446 446
q18 7781 7110 7165 7110
q19 1148 979 617 617
q20 369 382 246 246
q21 4524 3349 2534 2534
q22 1035 1060 934 934
Total cold run time: 110511 ms
Total hot run time: 34479 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4967 4952 4904 4904
q2 331 415 331 331
q3 2110 2708 2309 2309
q4 1326 1777 1263 1263
q5 4509 4465 4451 4451
q6 226 173 130 130
q7 2051 2001 1797 1797
q8 2706 2487 2526 2487
q9 7491 7629 7532 7532
q10 3086 3259 2827 2827
q11 592 537 493 493
q12 650 772 618 618
q13 3664 3871 3301 3301
q14 308 294 282 282
q15 552 502 513 502
q16 908 923 858 858
q17 1190 1452 1445 1445
q18 7985 7696 7815 7696
q19 993 879 904 879
q20 2023 2094 1829 1829
q21 4674 4274 4123 4123
q22 1072 1010 992 992
Total cold run time: 53414 ms
Total hot run time: 51049 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [clean](planner) remove legacy planner's constant folding [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #58680:
URL: https://github.com/apache/doris/pull/58680#issuecomment-3609814367
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [clean](planner) remove legacy planner's constant folding [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #58680:
URL: https://github.com/apache/doris/pull/58680#issuecomment-3606878617
# FE Regression Coverage Report
Increment line coverage ` 100.00% (2/2)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/58680_bf63e613ccf344adfeb40d08e2f97408fe1dc57b_merge_fe/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/58680_bf63e613ccf344adfeb40d08e2f97408fe1dc57b_merge_fe/report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org