You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "zhiqiang-hhhh (via GitHub)" <gi...@apache.org> on 2025/04/15 09:13:06 UTC
[PR] [chore](profile) Minor refactor on runtime filter producer profile [doris]
zhiqiang-hhhh opened a new pull request, #50055:
URL: https://github.com/apache/doris/pull/50055
### What problem does this PR solve?
Do same thing from https://github.com/apache/doris/pull/49777 to RuntimeFilterProducer
In execution profile:
```text
- RuntimeFilterInfo:
- BuildTime: 392.464us
- PublishTime: 69.942us
- RF4 Info: Producer: ([id: 4, state: [READY], type: MINMAX_FILTER, column_type: INT], mode: LOCAL, state: PUBLISHED)
- RF5 Info: Producer: ([id: 5, state: [READY], type: IN_OR_BLOOM_FILTER(BLOOM_FILTER), column_type: INT, bf_size: 1048576, build_bf_by_runtime_size: true], mode: LOCAL, state: PUBLISHED)
- SkipProcess: False
```
In merged profile
```
- RuntimeFilterInfo: sum , avg , max , min
- BuildTime: avg 0ns, max 0ns, min 0ns
- PublishTime: avg 103.959us, max 103.959us, min 103.959us
```
### Release note
None
### Check List (For Author)
- Test <!-- At least one of them must be included. -->
- [ ] Regression test
- [ ] Unit Test
- [ ] Manual test (add detailed scripts or steps below)
- [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
- [ ] Previous test can cover this change.
- [ ] No code files have been changed.
- [ ] Other reason <!-- Add your reason? -->
- Behavior changed:
- [ ] No.
- [ ] Yes. <!-- Explain the behavior change -->
- Does this need documentation?
- [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg: https://github.com/apache/doris-website/pull/1214 -->
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR should merge into -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [chore](profile) Minor refactor on runtime filter producer profile [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #50055:
URL: https://github.com/apache/doris/pull/50055#issuecomment-2804804867
# BE UT Coverage Report
Increment line coverage `82.14% (46/56)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/786645412145bf221bf3de7540faacce7da0798e_786645412145bf221bf3de7540faacce7da0798e/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/786645412145bf221bf3de7540faacce7da0798e_786645412145bf221bf3de7540faacce7da0798e/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 52.44% (14111/26910) |
| Line Coverage | 41.36% (122381/295867) |
| Region Coverage | 40.08% (62264/155360) |
| Branch Coverage | 34.78% (31257/89868) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [chore](profile) Minor refactor on runtime filter producer profile [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #50055:
URL: https://github.com/apache/doris/pull/50055#issuecomment-2804378632
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR).
Please clearly describe your PR:
1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
3. What features were added. Why was this function added?
4. Which code was refactored and why was this part of the code refactored?
5. Which functions were optimized and what is the difference before and after the optimization?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [chore](profile) Minor refactor on runtime filter producer profile [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #50055:
URL: https://github.com/apache/doris/pull/50055#issuecomment-2804623283
<details>
<summary>ClickBench: <b>Total hot run time: 31.2 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 786645412145bf221bf3de7540faacce7da0798e, data reload: false
query1 0.04 0.04 0.03
query2 0.13 0.11 0.11
query3 0.25 0.19 0.19
query4 1.59 0.19 0.19
query5 0.59 0.59 0.58
query6 1.25 0.71 0.72
query7 0.02 0.01 0.02
query8 0.04 0.04 0.04
query9 0.57 0.53 0.51
query10 0.57 0.58 0.57
query11 0.16 0.10 0.11
query12 0.14 0.11 0.12
query13 0.62 0.60 0.59
query14 2.70 2.71 2.74
query15 0.95 0.87 0.86
query16 0.37 0.38 0.39
query17 1.07 1.04 1.03
query18 0.21 0.19 0.19
query19 1.97 1.98 1.78
query20 0.01 0.01 0.01
query21 15.36 0.86 0.53
query22 0.75 1.23 0.75
query23 14.80 1.40 0.59
query24 7.13 1.76 0.85
query25 0.47 0.16 0.15
query26 0.68 0.17 0.14
query27 0.06 0.05 0.05
query28 9.58 0.85 0.43
query29 12.59 3.98 3.28
query30 0.25 0.09 0.07
query31 2.81 0.58 0.38
query32 3.22 0.54 0.47
query33 2.96 3.05 3.01
query34 15.72 5.16 4.48
query35 4.57 4.52 4.51
query36 0.66 0.49 0.48
query37 0.09 0.06 0.07
query38 0.05 0.04 0.04
query39 0.04 0.02 0.02
query40 0.17 0.14 0.13
query41 0.09 0.03 0.03
query42 0.04 0.02 0.02
query43 0.04 0.04 0.03
Total cold run time: 105.38 s
Total hot run time: 31.2 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [chore](profile) Minor refactor on runtime filter producer profile [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #50055:
URL: https://github.com/apache/doris/pull/50055#issuecomment-2804900160
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [chore](profile) Minor refactor on runtime filter producer profile [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #50055:
URL: https://github.com/apache/doris/pull/50055#issuecomment-2804579219
<details>
<summary>TPC-H: <b>Total hot run time: 34257 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 786645412145bf221bf3de7540faacce7da0798e, data reload: false
------ Round 1 ----------------------------------
q1 26130 5142 5097 5097
q2 2080 302 197 197
q3 10354 1270 720 720
q4 10221 1017 552 552
q5 7542 2465 2328 2328
q6 198 167 135 135
q7 931 740 605 605
q8 9303 1269 1110 1110
q9 6959 5124 5162 5124
q10 6877 2311 1887 1887
q11 504 283 283 283
q12 359 359 233 233
q13 17787 3707 3055 3055
q14 237 223 216 216
q15 545 493 484 484
q16 635 636 582 582
q17 610 875 358 358
q18 7547 7271 7060 7060
q19 1658 979 593 593
q20 324 334 214 214
q21 4354 3343 2451 2451
q22 1028 1020 973 973
Total cold run time: 116183 ms
Total hot run time: 34257 ms
----- Round 2, with runtime_filter_mode=off -----
q1 5279 5172 5162 5162
q2 241 330 237 237
q3 2144 2626 2284 2284
q4 1451 1868 1503 1503
q5 4498 4402 4379 4379
q6 236 169 131 131
q7 1998 1930 1755 1755
q8 2599 2566 2535 2535
q9 7282 7088 7263 7088
q10 3002 3169 2752 2752
q11 577 510 501 501
q12 698 791 631 631
q13 3549 3874 3371 3371
q14 277 307 273 273
q15 517 472 468 468
q16 665 714 650 650
q17 1222 1531 1415 1415
q18 7894 7531 7489 7489
q19 880 870 818 818
q20 2001 1986 1877 1877
q21 5436 4785 4811 4785
q22 1097 1036 993 993
Total cold run time: 53543 ms
Total hot run time: 51097 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [chore](profile) Minor refactor on runtime filter producer profile [doris]
Posted by "zhiqiang-hhhh (via GitHub)" <gi...@apache.org>.
zhiqiang-hhhh commented on PR #50055:
URL: https://github.com/apache/doris/pull/50055#issuecomment-2804454615
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [chore](profile) Minor refactor on runtime filter producer profile [doris]
Posted by "yiguolei (via GitHub)" <gi...@apache.org>.
yiguolei merged PR #50055:
URL: https://github.com/apache/doris/pull/50055
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [chore](profile) Minor refactor on runtime filter producer profile [doris]
Posted by "zhiqiang-hhhh (via GitHub)" <gi...@apache.org>.
zhiqiang-hhhh commented on PR #50055:
URL: https://github.com/apache/doris/pull/50055#issuecomment-2804392761
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [chore](profile) Minor refactor on runtime filter producer profile [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #50055:
URL: https://github.com/apache/doris/pull/50055#issuecomment-2804610082
<details>
<summary>TPC-DS: <b>Total hot run time: 194724 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 786645412145bf221bf3de7540faacce7da0798e, data reload: false
query1 1448 1084 1066 1066
query2 6082 2001 1952 1952
query3 11154 4830 4854 4830
query4 25425 23642 23646 23642
query5 5016 612 460 460
query6 304 200 194 194
query7 3983 493 277 277
query8 296 244 254 244
query9 8516 2617 2630 2617
query10 492 337 266 266
query11 15765 15423 14876 14876
query12 160 113 108 108
query13 1561 536 407 407
query14 9441 6668 6745 6668
query15 205 182 164 164
query16 7597 634 507 507
query17 1099 744 579 579
query18 2077 417 328 328
query19 191 195 159 159
query20 131 133 121 121
query21 211 124 107 107
query22 4396 4443 4221 4221
query23 34569 33854 33439 33439
query24 8524 2530 2491 2491
query25 535 459 407 407
query26 1302 312 155 155
query27 2840 518 341 341
query28 4660 2437 2478 2437
query29 696 573 433 433
query30 284 233 219 219
query31 914 881 820 820
query32 72 68 66 66
query33 559 374 321 321
query34 857 896 528 528
query35 824 864 789 789
query36 1001 1036 931 931
query37 124 104 81 81
query38 4197 4317 4121 4121
query39 1486 1459 1454 1454
query40 217 123 108 108
query41 55 53 56 53
query42 130 111 110 110
query43 528 525 497 497
query44 1329 836 831 831
query45 183 178 175 175
query46 863 1049 648 648
query47 1810 1819 1775 1775
query48 390 440 315 315
query49 805 538 451 451
query50 682 700 417 417
query51 4294 4361 4345 4345
query52 125 116 107 107
query53 233 268 191 191
query54 645 585 504 504
query55 81 85 82 82
query56 315 304 297 297
query57 1178 1208 1127 1127
query58 279 261 256 256
query59 2831 2784 2669 2669
query60 343 318 313 313
query61 131 125 128 125
query62 804 792 701 701
query63 236 194 190 190
query64 3991 1101 712 712
query65 4451 4359 4402 4359
query66 987 428 334 334
query67 15949 15242 15250 15242
query68 8269 887 532 532
query69 508 313 276 276
query70 1220 1149 1069 1069
query71 488 338 296 296
query72 5838 4682 4760 4682
query73 699 630 345 345
query74 8940 9227 8949 8949
query75 3789 3251 2718 2718
query76 3734 1252 890 890
query77 796 388 291 291
query78 10016 10353 9237 9237
query79 2068 836 567 567
query80 719 507 454 454
query81 518 271 231 231
query82 454 123 98 98
query83 283 254 235 235
query84 289 105 98 98
query85 770 410 312 312
query86 344 325 284 284
query87 4611 4555 4405 4405
query88 3521 2250 2213 2213
query89 408 321 300 300
query90 1960 221 222 221
query91 143 142 115 115
query92 74 64 57 57
query93 1693 945 583 583
query94 667 414 287 287
query95 378 300 288 288
query96 500 621 277 277
query97 3168 3241 3144 3144
query98 232 211 209 209
query99 1471 1475 1301 1301
Total cold run time: 281427 ms
Total hot run time: 194724 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [chore](profile) Minor refactor on runtime filter producer profile [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #50055:
URL: https://github.com/apache/doris/pull/50055#issuecomment-2804900393
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org