You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "HappenLee (via GitHub)" <gi...@apache.org> on 2024/03/14 01:55:49 UTC
[PR] Lambda [doris]
HappenLee opened a new pull request, #32210:
URL: https://github.com/apache/doris/pull/32210
## Proposed changes
Issue Number: close #xxx
<!--Describe your changes.-->
## Further comments
If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] Lambda [doris]
Posted by "HappenLee (via GitHub)" <gi...@apache.org>.
HappenLee commented on PR #32210:
URL: https://github.com/apache/doris/pull/32210#issuecomment-1996620296
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [RuntimeFilter](exec) support min max runtime filter and do refactor [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #32210:
URL: https://github.com/apache/doris/pull/32210#issuecomment-1998838557
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] Lambda [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32210:
URL: https://github.com/apache/doris/pull/32210#issuecomment-1996769221
TeamCity be ut coverage result:
Function Coverage: 34.99% (8572/24500)
Line Coverage: 26.74% (69457/259758)
Region Coverage: 26.01% (36071/138689)
Branch Coverage: 22.96% (18421/80214)
Coverage Report: http://coverage.selectdb-in.cc/coverage/e1997e2f1819a052ff6f13666689b81215baa23f_e1997e2f1819a052ff6f13666689b81215baa23f/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] Lambda [doris]
Posted by "HappenLee (via GitHub)" <gi...@apache.org>.
HappenLee commented on PR #32210:
URL: https://github.com/apache/doris/pull/32210#issuecomment-1996251600
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] Lambda [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32210:
URL: https://github.com/apache/doris/pull/32210#issuecomment-1996251282
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] Lambda [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32210:
URL: https://github.com/apache/doris/pull/32210#issuecomment-1996323966
TeamCity be ut coverage result:
Function Coverage: 35.00% (8576/24506)
Line Coverage: 26.75% (69477/259711)
Region Coverage: 26.02% (36076/138665)
Branch Coverage: 22.97% (18419/80196)
Coverage Report: http://coverage.selectdb-in.cc/coverage/a746ddb93ad84eca6b4777979debcd5b97e85f42_a746ddb93ad84eca6b4777979debcd5b97e85f42/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [RuntimeFilter](exec) support min max runtime filter and do refactor [doris]
Posted by "HappenLee (via GitHub)" <gi...@apache.org>.
HappenLee commented on PR #32210:
URL: https://github.com/apache/doris/pull/32210#issuecomment-1997323324
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [RuntimeFilter](exec) support min max runtime filter and do refactor [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32210:
URL: https://github.com/apache/doris/pull/32210#issuecomment-1997504479
TeamCity be ut coverage result:
Function Coverage: 34.98% (8573/24505)
Line Coverage: 26.73% (69453/259861)
Region Coverage: 26.00% (36073/138729)
Branch Coverage: 22.95% (18416/80244)
Coverage Report: http://coverage.selectdb-in.cc/coverage/062462f44ac3de9eb91831db9b4acfdfe036e0dc_062462f44ac3de9eb91831db9b4acfdfe036e0dc/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] Lambda [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32210:
URL: https://github.com/apache/doris/pull/32210#issuecomment-1996753470
<details>
<summary>TPC-H: <b>Total hot run time: 38398 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit e1997e2f1819a052ff6f13666689b81215baa23f, data reload: false
------ Round 1 ----------------------------------
q1 17618 4752 4121 4121
q2 2021 150 145 145
q3 10600 1102 898 898
q4 7783 743 745 743
q5 7494 2753 2738 2738
q6 184 125 125 125
q7 1183 842 806 806
q8 9344 2026 1993 1993
q9 7187 6453 6445 6445
q10 8489 3509 3601 3509
q11 421 220 215 215
q12 722 308 289 289
q13 17998 2859 2833 2833
q14 273 248 251 248
q15 501 448 447 447
q16 498 389 392 389
q17 974 533 579 533
q18 7150 6545 6433 6433
q19 3709 1424 1476 1424
q20 557 292 278 278
q21 6103 3508 3494 3494
q22 375 292 326 292
Total cold run time: 111184 ms
Total hot run time: 38398 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4133 4086 4064 4064
q2 322 226 225 225
q3 2939 2850 2852 2850
q4 1866 1560 1536 1536
q5 5208 5218 5205 5205
q6 209 115 116 115
q7 2245 1872 1862 1862
q8 3132 3287 3271 3271
q9 8539 8508 8598 8508
q10 3721 3669 3661 3661
q11 545 442 435 435
q12 725 556 571 556
q13 16939 2855 2898 2855
q14 275 247 253 247
q15 473 439 451 439
q16 445 420 383 383
q17 1748 1483 1454 1454
q18 7437 7163 7086 7086
q19 1601 1503 1550 1503
q20 1899 1699 1700 1699
q21 4915 4835 4771 4771
q22 534 466 438 438
Total cold run time: 69850 ms
Total hot run time: 53163 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [RuntimeFilter](exec) support min max runtime filter and do refactor [doris]
Posted by "zhangstar333 (via GitHub)" <gi...@apache.org>.
zhangstar333 merged PR #32210:
URL: https://github.com/apache/doris/pull/32210
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [RuntimeFilter](exec) support min max runtime filter and do refactor [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on code in PR #32210:
URL: https://github.com/apache/doris/pull/32210#discussion_r1524750213
##########
be/src/exprs/runtime_filter.cpp:
##########
@@ -747,9 +749,15 @@
// used by shuffle runtime filter
// assign this filter by protobuf
- Status assign(const PMinMaxFilter* minmax_filter) {
+ Status assign(const PMinMaxFilter* minmax_filter, bool contain_null) {
Review Comment:
warning: function 'assign' exceeds recommended size/complexity thresholds [readability-function-size]
```cpp
Status assign(const PMinMaxFilter* minmax_filter, bool contain_null) {
^
```
<details>
<summary>Additional context</summary>
**be/src/exprs/runtime_filter.cpp:751:** 134 lines including whitespace and comments (threshold 80)
```cpp
Status assign(const PMinMaxFilter* minmax_filter, bool contain_null) {
^
```
</details>
##########
be/src/exprs/runtime_filter.cpp:
##########
@@ -747,9 +749,15 @@ class RuntimePredicateWrapper {
// used by shuffle runtime filter
// assign this filter by protobuf
- Status assign(const PMinMaxFilter* minmax_filter) {
+ Status assign(const PMinMaxFilter* minmax_filter, bool contain_null) {
Review Comment:
warning: method 'assign' can be made static [readability-convert-member-functions-to-static]
```suggestion
static Status assign(const PMinMaxFilter* minmax_filter, bool contain_null) {
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [RuntimeFilter](exec) support min max runtime filter and do refactor [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32210:
URL: https://github.com/apache/doris/pull/32210#issuecomment-1997463742
<details>
<summary>TPC-H: <b>Total hot run time: 38624 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 062462f44ac3de9eb91831db9b4acfdfe036e0dc, data reload: false
------ Round 1 ----------------------------------
q1 17649 4232 4109 4109
q2 2018 165 143 143
q3 10582 1101 889 889
q4 7786 775 737 737
q5 7475 2723 2704 2704
q6 183 119 123 119
q7 1197 826 808 808
q8 9385 2018 2031 2018
q9 7215 6489 6448 6448
q10 8478 3523 3684 3523
q11 421 229 216 216
q12 596 301 305 301
q13 17780 2870 2859 2859
q14 270 264 252 252
q15 515 450 447 447
q16 500 397 390 390
q17 968 628 567 567
q18 7208 6625 6519 6519
q19 2853 1385 1460 1385
q20 542 287 274 274
q21 6286 3596 3609 3596
q22 352 320 332 320
Total cold run time: 110259 ms
Total hot run time: 38624 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4123 4129 4094 4094
q2 325 229 228 228
q3 2982 2886 2832 2832
q4 1859 1540 1614 1540
q5 5231 5237 5248 5237
q6 191 115 117 115
q7 2248 1865 1845 1845
q8 3173 3323 3297 3297
q9 8604 8554 8623 8554
q10 3701 3640 3634 3634
q11 551 447 446 446
q12 732 554 537 537
q13 16907 2898 2872 2872
q14 289 243 246 243
q15 483 440 452 440
q16 451 418 399 399
q17 1757 1486 1462 1462
q18 7604 7262 7112 7112
q19 1620 1518 1594 1518
q20 1903 1722 1691 1691
q21 4715 4722 4810 4722
q22 533 444 447 444
Total cold run time: 69982 ms
Total hot run time: 53262 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [RuntimeFilter](exec) support min max runtime filter and do refactor [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #32210:
URL: https://github.com/apache/doris/pull/32210#issuecomment-1998838588
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] Lambda [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32210:
URL: https://github.com/apache/doris/pull/32210#issuecomment-1996313930
<details>
<summary>TPC-H: <b>Total hot run time: 38538 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit a746ddb93ad84eca6b4777979debcd5b97e85f42, data reload: false
------ Round 1 ----------------------------------
q1 17625 4332 4140 4140
q2 2025 151 145 145
q3 10595 1121 916 916
q4 7083 715 724 715
q5 7482 2745 2710 2710
q6 189 129 133 129
q7 1191 848 802 802
q8 9441 2062 1973 1973
q9 7185 6448 6406 6406
q10 8525 3484 3616 3484
q11 430 225 234 225
q12 711 312 300 300
q13 18042 2864 2878 2864
q14 280 246 255 246
q15 508 458 450 450
q16 520 395 398 395
q17 963 523 613 523
q18 7269 6571 6575 6571
q19 3971 1437 1494 1437
q20 545 293 278 278
q21 6349 3520 3634 3520
q22 374 319 309 309
Total cold run time: 111303 ms
Total hot run time: 38538 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4141 4127 4085 4085
q2 322 223 230 223
q3 2963 2869 2821 2821
q4 1877 1545 1546 1545
q5 5233 5281 5256 5256
q6 193 114 116 114
q7 2251 1852 1807 1807
q8 3142 3267 3305 3267
q9 8601 8633 8603 8603
q10 3743 3677 3701 3677
q11 574 435 446 435
q12 731 538 537 537
q13 16942 2883 2903 2883
q14 285 249 263 249
q15 489 451 448 448
q16 468 427 416 416
q17 1728 1505 1475 1475
q18 7489 7244 7147 7147
q19 1634 1574 1513 1513
q20 1925 1728 1704 1704
q21 4783 4784 4762 4762
q22 544 473 468 468
Total cold run time: 70058 ms
Total hot run time: 53435 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] Lambda [doris]
Posted by "englefly (via GitHub)" <gi...@apache.org>.
englefly commented on code in PR #32210:
URL: https://github.com/apache/doris/pull/32210#discussion_r1524295400
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/physical/AbstractPhysicalPlan.java:
##########
@@ -135,10 +135,7 @@ public boolean pushDownRuntimeFilter(CascadesContext context, IdGenerator<Runtim
} else {
// null safe equal runtime filter only support bloom filter
Review Comment:
remove this comment please
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/physical/AbstractPhysicalPlan.java:
##########
@@ -135,10 +135,7 @@ public boolean pushDownRuntimeFilter(CascadesContext context, IdGenerator<Runtim
} else {
// null safe equal runtime filter only support bloom filter
EqualPredicate eq = (EqualPredicate) builderNode.getHashJoinConjuncts().get(exprOrder);
- if (eq instanceof NullSafeEqual && type == TRuntimeFilterType.IN_OR_BLOOM) {
- type = TRuntimeFilterType.BLOOM;
- }
- if (eq instanceof NullSafeEqual && type != TRuntimeFilterType.BLOOM) {
+ if (eq instanceof NullSafeEqual || type == TRuntimeFilterType.BITMAP) {
Review Comment:
do not need this "if"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] Lambda [doris]
Posted by "HappenLee (via GitHub)" <gi...@apache.org>.
HappenLee commented on PR #32210:
URL: https://github.com/apache/doris/pull/32210#issuecomment-1996626053
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org