You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "englefly (via GitHub)" <gi...@apache.org> on 2024/02/06 06:27:36 UTC
[PR] [feature](nereids) set wait_time for runtime filter [doris]
englefly opened a new pull request, #30895:
URL: https://github.com/apache/doris/pull/30895
## Proposed changes
Issue Number: close #xxx
<!--Describe your changes.-->
## Further comments
If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](nereids) set wait_time for runtime filter [doris]
Posted by "englefly (via GitHub)" <gi...@apache.org>.
englefly closed pull request #30895: [feature](nereids) set wait_time for runtime filter
URL: https://github.com/apache/doris/pull/30895
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](nereids) set wait_time for runtime filter [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30895:
URL: https://github.com/apache/doris/pull/30895#issuecomment-1952220985
TeamCity be ut coverage result:
Function Coverage: 35.77% (8558/23924)
Line Coverage: 27.71% (69389/250367)
Region Coverage: 26.84% (36010/134174)
Branch Coverage: 23.65% (18421/77886)
Coverage Report: http://coverage.selectdb-in.cc/coverage/d1add0e63bbe1de0b86d07318071cf1061f72b2f_d1add0e63bbe1de0b86d07318071cf1061f72b2f/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](nereids) set wait_time for runtime filter [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #30895:
URL: https://github.com/apache/doris/pull/30895#issuecomment-1928873729
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](nereids) set wait_time for runtime filter [doris]
Posted by "englefly (via GitHub)" <gi...@apache.org>.
englefly commented on code in PR #30895:
URL: https://github.com/apache/doris/pull/30895#discussion_r1479623233
##########
be/src/exprs/runtime_filter.cpp:
##########
@@ -1174,6 +1174,9 @@ Status IRuntimeFilter::init_with_desc(const TRuntimeFilterDesc* desc, const TQue
_has_remote_target = desc->has_remote_targets;
_expr_order = desc->expr_order;
_opt_remote_rf = desc->__isset.opt_remote_rf && desc->opt_remote_rf;
+ if (desc->__isset.wait_time_ms) {
+ _rf_wait_time_ms = desc->wait_time_ms;
Review Comment:
it will be the wait_time in session variable
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](nereids) set wait_time for runtime filter [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #30895:
URL: https://github.com/apache/doris/pull/30895#issuecomment-1956082722
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](nereids) set wait_time for runtime filter [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #30895:
URL: https://github.com/apache/doris/pull/30895#issuecomment-1956039499
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](nereids) set wait_time for runtime filter [doris]
Posted by "englefly (via GitHub)" <gi...@apache.org>.
englefly commented on PR #30895:
URL: https://github.com/apache/doris/pull/30895#issuecomment-1952058031
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](nereids) set wait_time for runtime filter [doris]
Posted by "englefly (via GitHub)" <gi...@apache.org>.
englefly commented on PR #30895:
URL: https://github.com/apache/doris/pull/30895#issuecomment-1956070786
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](nereids) set wait_time for runtime filter [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30895:
URL: https://github.com/apache/doris/pull/30895#issuecomment-1956129814
<details>
<summary>TPC-H: <b>Total hot run time: 41690 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 322ef1f151fab6bcf9a8955c1ad78e1f68f220b5, data reload: false
------ Round 1 ----------------------------------
q1 17632 4995 4930 4930
q2 2057 145 143 143
q3 10602 1018 1048 1018
q4 4748 966 966 966
q5 7736 3187 3305 3187
q6 200 137 145 137
q7 1252 783 779 779
q8 9388 2094 2096 2094
q9 7679 6944 6713 6713
q10 8307 2643 2640 2640
q11 423 221 218 218
q12 786 341 331 331
q13 18001 3654 3676 3654
q14 295 265 259 259
q15 588 509 508 508
q16 481 407 415 407
q17 943 900 900 900
q18 7304 6729 6644 6644
q19 1559 1495 1498 1495
q20 576 351 339 339
q21 6655 4017 3979 3979
q22 853 357 349 349
Total cold run time: 108065 ms
Total hot run time: 41690 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4857 4833 4831 4831
q2 301 188 188 188
q3 3591 3574 3590 3574
q4 2519 2545 2514 2514
q5 5786 5721 5760 5721
q6 209 123 131 123
q7 2262 1670 1629 1629
q8 3016 3076 3125 3076
q9 8732 8726 8686 8686
q10 6775 4273 4230 4230
q11 506 390 389 389
q12 775 543 533 533
q13 4228 3401 3403 3401
q14 263 224 228 224
q15 629 506 497 497
q16 491 434 462 434
q17 1701 1617 1616 1616
q18 8443 7700 7707 7700
q19 1642 1643 1650 1643
q20 2116 1849 1837 1837
q21 6605 6192 6178 6178
q22 561 527 504 504
Total cold run time: 66008 ms
Total hot run time: 59528 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](nereids) set wait_time for runtime filter [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30895:
URL: https://github.com/apache/doris/pull/30895#issuecomment-1928864800
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](nereids) set wait_time for runtime filter [doris]
Posted by "morrySnow (via GitHub)" <gi...@apache.org>.
morrySnow commented on code in PR #30895:
URL: https://github.com/apache/doris/pull/30895#discussion_r1479418711
##########
be/src/exprs/runtime_filter.cpp:
##########
@@ -1174,6 +1174,9 @@ Status IRuntimeFilter::init_with_desc(const TRuntimeFilterDesc* desc, const TQue
_has_remote_target = desc->has_remote_targets;
_expr_order = desc->expr_order;
_opt_remote_rf = desc->__isset.opt_remote_rf && desc->opt_remote_rf;
+ if (desc->__isset.wait_time_ms) {
+ _rf_wait_time_ms = desc->wait_time_ms;
Review Comment:
what value will be assigned to `_rf_wait_time_ms` if `desc->__isset.wait_time_ms` is false?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](nereids) set wait_time for runtime filter [doris]
Posted by "morrySnow (via GitHub)" <gi...@apache.org>.
morrySnow commented on code in PR #30895:
URL: https://github.com/apache/doris/pull/30895#discussion_r1496933429
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/processor/post/RuntimeFilterPrunerForExternalTable.java:
##########
@@ -52,7 +52,8 @@ public Plan processRoot(Plan plan, CascadesContext ctx) {
for (int i = 0; i < rf.getTargetScans().size(); i++) {
PhysicalRelation scan = rf.getTargetScans().get(i);
if (canPrune(scan, joinAncestors)) {
- rfCtx.removeFilter(rf.getTargetSlots().get(i).getExprId(), (PhysicalHashJoin) join);
+ // rfCtx.removeFilter(rf.getTargetSlots().get(i).getExprId(), (PhysicalHashJoin) join);
Review Comment:
remove this comment line
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](nereids) set wait_time for runtime filter [doris]
Posted by "englefly (via GitHub)" <gi...@apache.org>.
englefly commented on code in PR #30895:
URL: https://github.com/apache/doris/pull/30895#discussion_r1497044048
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/processor/post/RuntimeFilterPrunerForExternalTable.java:
##########
@@ -52,7 +52,8 @@ public Plan processRoot(Plan plan, CascadesContext ctx) {
for (int i = 0; i < rf.getTargetScans().size(); i++) {
PhysicalRelation scan = rf.getTargetScans().get(i);
if (canPrune(scan, joinAncestors)) {
- rfCtx.removeFilter(rf.getTargetSlots().get(i).getExprId(), (PhysicalHashJoin) join);
+ // rfCtx.removeFilter(rf.getTargetSlots().get(i).getExprId(), (PhysicalHashJoin) join);
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org