You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "englefly (via GitHub)" <gi...@apache.org> on 2025/10/28 10:32:36 UTC
[PR] [opt](nereids) tpcds 54 missed 2 important runtime filters [doris]
englefly opened a new pull request, #57425:
URL: https://github.com/apache/doris/pull/57425
### What problem does this PR solve?
when we check if there is a target for runtime filter, we should use ctx.probeSlot not ctx.probeExpr
Issue Number: close #xxx
Related PR: #xxx
Problem Summary:
### Release note
None
### Check List (For Author)
- Test <!-- At least one of them must be included. -->
- [ ] Regression test
- [ ] Unit Test
- [ ] Manual test (add detailed scripts or steps below)
- [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
- [ ] Previous test can cover this change.
- [ ] No code files have been changed.
- [ ] Other reason <!-- Add your reason? -->
- Behavior changed:
- [ ] No.
- [ ] Yes. <!-- Explain the behavior change -->
- Does this need documentation?
- [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg: https://github.com/apache/doris-website/pull/1214 -->
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR should merge into -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](nereids) tpcds 54 missed 2 important runtime filters [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #57425:
URL: https://github.com/apache/doris/pull/57425#issuecomment-3459470125
# FE UT Coverage Report
Increment line coverage ` 16.67% (1/6)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/57425_67a08bc0f142fb97427632f687a49c9411c27f61/fe_increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/57425_67a08bc0f142fb97427632f687a49c9411c27f61/fe_report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](nereids) tpcds 54 missed 2 important runtime filters [doris]
Posted by "englefly (via GitHub)" <gi...@apache.org>.
englefly commented on PR #57425:
URL: https://github.com/apache/doris/pull/57425#issuecomment-3465942394
run external
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](nereids) when we check if there is a target for runtime filter, we should use ctx.probeSlot not ctx.probeExpr [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #57425:
URL: https://github.com/apache/doris/pull/57425#issuecomment-3568678635
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](nereids) tpcds 54 missed 2 important runtime filters [doris]
Posted by "englefly (via GitHub)" <gi...@apache.org>.
englefly commented on PR #57425:
URL: https://github.com/apache/doris/pull/57425#issuecomment-3459185203
run feut
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](nereids) when we check if there is a target for runtime filter, we should use ctx.probeSlot not ctx.probeExpr [doris]
Posted by "englefly (via GitHub)" <gi...@apache.org>.
englefly merged PR #57425:
URL: https://github.com/apache/doris/pull/57425
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](nereids) tpcds 54 missed 2 important runtime filters [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #57425:
URL: https://github.com/apache/doris/pull/57425#issuecomment-3456503413
# FE Regression Coverage Report
Increment line coverage ` 66.67% (4/6)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/57425_67a08bc0f142fb97427632f687a49c9411c27f61_merge_fe/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/57425_67a08bc0f142fb97427632f687a49c9411c27f61_merge_fe/report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](nereids) tpcds 54 missed 2 important runtime filters [doris]
Posted by "englefly (via GitHub)" <gi...@apache.org>.
englefly commented on PR #57425:
URL: https://github.com/apache/doris/pull/57425#issuecomment-3459286417
run external
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](nereids) when we check if there is a target for runtime filter, we should use ctx.probeSlot not ctx.probeExpr [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #57425:
URL: https://github.com/apache/doris/pull/57425#issuecomment-3525992874
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](nereids) tpcds 54 missed 2 important runtime filters [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #57425:
URL: https://github.com/apache/doris/pull/57425#issuecomment-3455756305
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR).
Please clearly describe your PR:
1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
3. What features were added. Why was this function added?
4. Which code was refactored and why was this part of the code refactored?
5. Which functions were optimized and what is the difference before and after the optimization?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](nereids) when we check if there is a target for runtime filter, we should use ctx.probeSlot not ctx.probeExpr [doris]
Posted by "englefly (via GitHub)" <gi...@apache.org>.
englefly commented on PR #57425:
URL: https://github.com/apache/doris/pull/57425#issuecomment-3530896625
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](nereids) when we check if there is a target for runtime filter, we should use ctx.probeSlot not ctx.probeExpr [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #57425:
URL: https://github.com/apache/doris/pull/57425#issuecomment-3530975515
<details>
<summary>TPC-DS: <b>Total hot run time: 188478 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 45313c7edd4be91c7dfd7e359402250b8520d0a8, data reload: false
query1 1071 413 408 408
query2 6607 1731 1714 1714
query3 6754 229 219 219
query4 26139 23666 23265 23265
query5 4483 637 485 485
query6 337 230 224 224
query7 4673 488 294 294
query8 305 250 235 235
query9 8706 2620 2627 2620
query10 506 342 285 285
query11 15812 15085 14869 14869
query12 206 119 112 112
query13 1675 543 434 434
query14 11209 9302 9132 9132
query15 196 192 170 170
query16 7336 697 521 521
query17 1236 754 602 602
query18 1994 414 315 315
query19 217 207 186 186
query20 141 131 122 122
query21 219 135 116 116
query22 4060 4133 3985 3985
query23 33965 33108 33003 33003
query24 8519 2373 2416 2373
query25 632 513 473 473
query26 1237 273 162 162
query27 2750 485 352 352
query28 4375 2233 2198 2198
query29 792 601 498 498
query30 305 225 212 212
query31 911 823 749 749
query32 86 81 70 70
query33 606 383 330 330
query34 794 852 524 524
query35 820 852 752 752
query36 963 1008 924 924
query37 126 108 86 86
query38 3523 3535 3512 3512
query39 1525 1433 1423 1423
query40 218 130 121 121
query41 62 60 61 60
query42 123 115 119 115
query43 496 498 469 469
query44 1246 771 749 749
query45 184 176 180 176
query46 873 995 668 668
query47 1768 1770 1705 1705
query48 403 423 336 336
query49 774 487 419 419
query50 642 690 415 415
query51 3850 4048 3966 3966
query52 114 114 104 104
query53 235 270 208 208
query54 314 292 286 286
query55 87 85 84 84
query56 331 337 321 321
query57 1183 1177 1138 1138
query58 292 271 279 271
query59 2508 2707 2630 2630
query60 356 343 324 324
query61 159 168 162 162
query62 804 744 658 658
query63 230 201 204 201
query64 4493 1267 991 991
query65 4007 3982 4003 3982
query66 1162 444 333 333
query67 15271 15110 14980 14980
query68 5097 957 610 610
query69 497 328 297 297
query70 1308 1189 1333 1189
query71 432 343 341 341
query72 5818 4997 5012 4997
query73 635 600 364 364
query74 8859 9154 8917 8917
query75 3337 3319 2816 2816
query76 3263 1140 757 757
query77 523 433 331 331
query78 9474 9851 8876 8876
query79 990 837 591 591
query80 643 589 507 507
query81 496 260 232 232
query82 397 158 131 131
query83 277 268 253 253
query84 250 113 94 94
query85 885 503 446 446
query86 327 309 299 299
query87 3734 3760 3611 3611
query88 2918 2327 2270 2270
query89 389 329 294 294
query90 1803 230 226 226
query91 176 172 137 137
query92 75 66 67 66
query93 1110 988 652 652
query94 653 454 358 358
query95 407 329 316 316
query96 494 565 288 288
query97 2901 2977 2853 2853
query98 233 215 211 211
query99 1353 1399 1322 1322
Total cold run time: 267750 ms
Total hot run time: 188478 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](nereids) when we check if there is a target for runtime filter, we should use ctx.probeSlot not ctx.probeExpr [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #57425:
URL: https://github.com/apache/doris/pull/57425#issuecomment-3530938873
<details>
<summary>TPC-H: <b>Total hot run time: 34159 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 45313c7edd4be91c7dfd7e359402250b8520d0a8, data reload: false
------ Round 1 ----------------------------------
q1 17610 5240 5027 5027
q2 2051 338 209 209
q3 10203 1302 719 719
q4 10226 859 358 358
q5 7505 2379 2329 2329
q6 182 174 137 137
q7 908 762 631 631
q8 9327 1323 1067 1067
q9 6862 5131 5180 5131
q10 6815 2217 1794 1794
q11 493 317 270 270
q12 335 371 231 231
q13 17825 3636 3050 3050
q14 237 228 211 211
q15 573 515 507 507
q16 1019 996 943 943
q17 600 859 357 357
q18 7447 7251 7141 7141
q19 1090 959 558 558
q20 357 340 234 234
q21 3854 3155 2304 2304
q22 1051 1007 951 951
Total cold run time: 106570 ms
Total hot run time: 34159 ms
----- Round 2, with runtime_filter_mode=off -----
q1 5125 5131 5090 5090
q2 296 322 225 225
q3 2166 2720 2327 2327
q4 1358 1796 1328 1328
q5 4177 4436 4602 4436
q6 210 173 130 130
q7 2144 2027 1853 1853
q8 2553 2496 2622 2496
q9 7399 7393 7180 7180
q10 3038 3258 2821 2821
q11 595 533 497 497
q12 683 897 698 698
q13 3418 3885 3405 3405
q14 298 302 309 302
q15 540 523 497 497
q16 1121 1126 1080 1080
q17 1209 1648 1384 1384
q18 8035 7665 7668 7665
q19 856 855 967 855
q20 2041 2077 1933 1933
q21 5119 4563 4285 4285
q22 1089 1066 995 995
Total cold run time: 53470 ms
Total hot run time: 51482 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](nereids) tpcds 54 missed 2 important runtime filters [doris]
Posted by "englefly (via GitHub)" <gi...@apache.org>.
englefly commented on PR #57425:
URL: https://github.com/apache/doris/pull/57425#issuecomment-3455756518
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](nereids) when we check if there is a target for runtime filter, we should use ctx.probeSlot not ctx.probeExpr [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #57425:
URL: https://github.com/apache/doris/pull/57425#issuecomment-3530997787
<details>
<summary>ClickBench: <b>Total hot run time: 27.26 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 45313c7edd4be91c7dfd7e359402250b8520d0a8, data reload: false
query1 0.06 0.05 0.05
query2 0.10 0.05 0.04
query3 0.26 0.08 0.08
query4 1.60 0.11 0.11
query5 0.25 0.25 0.25
query6 1.17 0.65 0.65
query7 0.03 0.02 0.02
query8 0.05 0.04 0.05
query9 0.57 0.52 0.52
query10 0.58 0.57 0.57
query11 0.17 0.11 0.12
query12 0.15 0.12 0.12
query13 0.62 0.60 0.60
query14 1.00 0.99 1.00
query15 0.85 0.84 0.84
query16 0.40 0.40 0.38
query17 1.03 1.02 1.00
query18 0.21 0.20 0.19
query19 1.92 1.88 1.74
query20 0.01 0.01 0.01
query21 15.41 0.20 0.13
query22 5.03 0.07 0.04
query23 15.64 0.27 0.10
query24 2.52 0.71 0.33
query25 0.06 0.06 0.07
query26 0.14 0.13 0.14
query27 0.06 0.06 0.05
query28 4.80 1.15 0.93
query29 12.59 3.85 3.18
query30 0.28 0.13 0.11
query31 2.82 0.60 0.38
query32 3.22 0.56 0.46
query33 3.04 3.11 3.05
query34 15.87 5.14 4.56
query35 4.54 4.59 4.59
query36 0.67 0.50 0.49
query37 0.09 0.07 0.07
query38 0.06 0.04 0.04
query39 0.04 0.03 0.03
query40 0.17 0.14 0.14
query41 0.09 0.04 0.03
query42 0.04 0.02 0.02
query43 0.04 0.04 0.03
Total cold run time: 98.25 s
Total hot run time: 27.26 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](nereids) when we check if there is a target for runtime filter, we should use ctx.probeSlot not ctx.probeExpr [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #57425:
URL: https://github.com/apache/doris/pull/57425#issuecomment-3531547872
# FE Regression Coverage Report
Increment line coverage ` 66.67% (4/6)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/57425_45313c7edd4be91c7dfd7e359402250b8520d0a8_merge_fe/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/57425_45313c7edd4be91c7dfd7e359402250b8520d0a8_merge_fe/report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org