You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "bobhan1 (via GitHub)" <gi...@apache.org> on 2025/04/16 02:48:49 UTC
[PR] [Opt](metric) Add metrics for invalid delete bitmap key count [doris]
bobhan1 opened a new pull request, #50080:
URL: https://github.com/apache/doris/pull/50080
### What problem does this PR solve?
Issue Number: close #xxx
Related PR: #xxx
Problem Summary:
### Release note
None
### Check List (For Author)
- Test <!-- At least one of them must be included. -->
- [ ] Regression test
- [ ] Unit Test
- [ ] Manual test (add detailed scripts or steps below)
- [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
- [ ] Previous test can cover this change.
- [ ] No code files have been changed.
- [ ] Other reason <!-- Add your reason? -->
- Behavior changed:
- [ ] No.
- [ ] Yes. <!-- Explain the behavior change -->
- Does this need documentation?
- [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg: https://github.com/apache/doris-website/pull/1214 -->
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR should merge into -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Opt](metric) Add metrics for invalid delete bitmap key count [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #50080:
URL: https://github.com/apache/doris/pull/50080#issuecomment-2839170749
# BE UT Coverage Report
Increment line coverage `3.70% (2/54)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/3e3ace510173441f8b9972f636d2479c5785c6e8_3e3ace510173441f8b9972f636d2479c5785c6e8/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/3e3ace510173441f8b9972f636d2479c5785c6e8_3e3ace510173441f8b9972f636d2479c5785c6e8/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 54.79% (14774/26963) |
| Line Coverage | 43.91% (129632/295194) |
| Region Coverage | 42.62% (66168/155240) |
| Branch Coverage | 37.22% (33395/89716) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Opt](metric) Add metrics for invalid delete bitmap key count [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #50080:
URL: https://github.com/apache/doris/pull/50080#issuecomment-2839692791
# BE Regression P0 && UT Coverage Report
Increment line coverage `20.27% (15/74)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/50080_3e3ace510173441f8b9972f636d2479c5785c6e8_merge/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/50080_3e3ace510173441f8b9972f636d2479c5785c6e8_merge/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 78.13% (20680/26468) |
| Line Coverage | 71.70% (211328/294729) |
| Region Coverage | 69.88% (126939/181653) |
| Branch Coverage | 63.14% (64585/102282) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Opt](metric) Add metrics for invalid delete bitmap key count [doris]
Posted by "bobhan1 (via GitHub)" <gi...@apache.org>.
bobhan1 commented on PR #50080:
URL: https://github.com/apache/doris/pull/50080#issuecomment-2840636091
run cloud_p0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Opt](metric) Add metrics for invalid delete bitmap key count [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #50080:
URL: https://github.com/apache/doris/pull/50080#issuecomment-2838765011
<details>
<summary>TPC-DS: <b>Total hot run time: 185351 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 3e3ace510173441f8b9972f636d2479c5785c6e8, data reload: false
query1 1025 471 490 471
query2 6552 1924 1881 1881
query3 6740 224 219 219
query4 26241 23399 23555 23399
query5 4396 654 461 461
query6 306 216 206 206
query7 4638 482 281 281
query8 290 233 230 230
query9 8581 2599 2609 2599
query10 489 318 258 258
query11 15600 15064 14847 14847
query12 155 110 102 102
query13 1660 543 403 403
query14 8819 6168 5993 5993
query15 204 185 180 180
query16 7138 667 489 489
query17 1227 749 581 581
query18 1988 412 311 311
query19 197 192 157 157
query20 125 126 127 126
query21 214 128 111 111
query22 4061 4272 4007 4007
query23 34191 32776 32935 32776
query24 8271 2402 2409 2402
query25 608 445 389 389
query26 1233 267 152 152
query27 2745 484 333 333
query28 4320 2099 2105 2099
query29 777 549 428 428
query30 281 218 189 189
query31 928 859 768 768
query32 72 68 68 68
query33 559 403 313 313
query34 796 856 510 510
query35 765 813 740 740
query36 953 985 876 876
query37 111 104 76 76
query38 4148 4125 4081 4081
query39 1462 1429 1375 1375
query40 212 119 105 105
query41 58 57 56 56
query42 116 105 105 105
query43 507 526 488 488
query44 1306 788 787 787
query45 180 177 174 174
query46 863 1042 628 628
query47 1749 1755 1687 1687
query48 366 415 298 298
query49 780 501 399 399
query50 658 684 418 418
query51 4058 4169 4121 4121
query52 114 107 96 96
query53 231 267 185 185
query54 580 576 523 523
query55 83 84 81 81
query56 300 307 291 291
query57 1123 1203 1076 1076
query58 266 245 254 245
query59 2690 2776 2679 2679
query60 316 326 312 312
query61 130 126 130 126
query62 778 731 640 640
query63 223 185 185 185
query64 4360 1014 668 668
query65 4320 4229 4251 4229
query66 1165 414 327 327
query67 15959 15392 15226 15226
query68 8564 880 506 506
query69 469 313 273 273
query70 1178 1064 1081 1064
query71 470 370 299 299
query72 5264 4761 4808 4761
query73 724 648 339 339
query74 8869 9087 8602 8602
query75 3964 3185 2680 2680
query76 3637 1186 767 767
query77 793 392 287 287
query78 9838 10115 9347 9347
query79 1991 804 571 571
query80 638 503 438 438
query81 471 252 214 214
query82 242 134 96 96
query83 276 249 232 232
query84 298 107 77 77
query85 775 349 311 311
query86 339 281 307 281
query87 4423 4418 4215 4215
query88 2769 2202 2209 2202
query89 452 309 283 283
query90 2082 210 210 210
query91 148 148 110 110
query92 74 64 59 59
query93 1151 931 585 585
query94 672 416 310 310
query95 372 293 282 282
query96 490 568 281 281
query97 3185 3241 3154 3154
query98 253 198 200 198
query99 1364 1399 1275 1275
Total cold run time: 272592 ms
Total hot run time: 185351 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Opt](metric) Add metrics for invalid delete bitmap key count [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #50080:
URL: https://github.com/apache/doris/pull/50080#issuecomment-2808692897
<details>
<summary>TPC-H: <b>Total hot run time: 34704 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit fd593d49bd896cc66d0568ed55645450dbf45c64, data reload: false
------ Round 1 ----------------------------------
q1 26007 5094 5059 5059
q2 2060 282 189 189
q3 10386 1255 712 712
q4 10224 1038 549 549
q5 7549 2380 2336 2336
q6 185 169 134 134
q7 942 764 609 609
q8 9306 1282 1073 1073
q9 6824 5099 5126 5099
q10 6873 2326 1885 1885
q11 496 285 279 279
q12 352 362 216 216
q13 17783 3708 3105 3105
q14 239 222 211 211
q15 528 480 480 480
q16 454 445 405 405
q17 582 871 372 372
q18 7702 7254 7093 7093
q19 1813 991 561 561
q20 336 342 228 228
q21 3971 3399 3143 3143
q22 1067 980 966 966
Total cold run time: 115679 ms
Total hot run time: 34704 ms
----- Round 2, with runtime_filter_mode=off -----
q1 5233 5142 5101 5101
q2 240 329 233 233
q3 2161 2678 2270 2270
q4 1476 1838 1504 1504
q5 4470 4435 4432 4432
q6 225 175 128 128
q7 1979 1926 1765 1765
q8 2651 2612 2652 2612
q9 7327 7163 7163 7163
q10 2982 3220 2737 2737
q11 577 503 515 503
q12 660 763 584 584
q13 3608 3928 3345 3345
q14 304 307 294 294
q15 525 480 478 478
q16 468 533 467 467
q17 1192 1655 1381 1381
q18 7850 7628 7435 7435
q19 846 813 1018 813
q20 2053 2014 1859 1859
q21 5380 4854 4845 4845
q22 1070 1078 1046 1046
Total cold run time: 53277 ms
Total hot run time: 50995 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Opt](metric) Add metrics for invalid delete bitmap key count [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #50080:
URL: https://github.com/apache/doris/pull/50080#issuecomment-2808735173
<details>
<summary>ClickBench: <b>Total hot run time: 30.18 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit fd593d49bd896cc66d0568ed55645450dbf45c64, data reload: false
query1 0.04 0.04 0.04
query2 0.12 0.11 0.12
query3 0.26 0.19 0.19
query4 1.60 0.20 0.19
query5 0.60 0.60 0.58
query6 1.18 0.71 0.72
query7 0.02 0.02 0.02
query8 0.05 0.04 0.04
query9 0.57 0.54 0.51
query10 0.58 0.58 0.55
query11 0.17 0.10 0.11
query12 0.15 0.12 0.11
query13 0.61 0.59 0.61
query14 1.16 1.19 1.17
query15 0.89 0.85 0.86
query16 0.38 0.38 0.39
query17 1.02 1.02 1.03
query18 0.21 0.20 0.19
query19 1.91 1.82 1.78
query20 0.01 0.01 0.01
query21 15.41 0.91 0.53
query22 0.75 1.18 0.69
query23 14.90 1.40 0.63
query24 6.68 1.52 1.40
query25 0.49 0.26 0.09
query26 0.59 0.15 0.12
query27 0.05 0.05 0.05
query28 10.05 0.89 0.45
query29 12.71 4.02 3.31
query30 0.25 0.10 0.06
query31 2.82 0.59 0.38
query32 3.22 0.54 0.46
query33 3.03 3.10 3.11
query34 15.79 5.07 4.51
query35 4.58 4.56 4.45
query36 0.67 0.50 0.48
query37 0.09 0.06 0.06
query38 0.05 0.04 0.04
query39 0.03 0.03 0.02
query40 0.16 0.14 0.13
query41 0.09 0.03 0.03
query42 0.03 0.02 0.03
query43 0.03 0.03 0.03
Total cold run time: 104 s
Total hot run time: 30.18 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Opt](metric) Add metrics for invalid delete bitmap key count [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #50080:
URL: https://github.com/apache/doris/pull/50080#issuecomment-2840909784
# BE Regression && UT Coverage Report
Increment line coverage `20.27% (15/74)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/50080_3e3ace510173441f8b9972f636d2479c5785c6e8_merge/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/50080_3e3ace510173441f8b9972f636d2479c5785c6e8_merge/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 80.31% (21257/26468) |
| Line Coverage | 74.40% (219282/294729) |
| Region Coverage | 72.71% (132071/181653) |
| Branch Coverage | 66.02% (67528/102282) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Opt](metric) Add metrics for invalid delete bitmap key count [doris]
Posted by "bobhan1 (via GitHub)" <gi...@apache.org>.
bobhan1 commented on PR #50080:
URL: https://github.com/apache/doris/pull/50080#issuecomment-2814502582
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Opt](metric) Add metrics for invalid delete bitmap key count [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #50080:
URL: https://github.com/apache/doris/pull/50080#issuecomment-2808954286
# BE UT Coverage Report
Increment line coverage `4.35% (2/46)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/fd593d49bd896cc66d0568ed55645450dbf45c64_fd593d49bd896cc66d0568ed55645450dbf45c64/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/fd593d49bd896cc66d0568ed55645450dbf45c64_fd593d49bd896cc66d0568ed55645450dbf45c64/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 52.81% (14221/26930) |
| Line Coverage | 41.71% (123442/295953) |
| Region Coverage | 40.45% (62872/155436) |
| Branch Coverage | 35.07% (31533/89902) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Opt](metric) Add metrics for invalid delete bitmap key count [doris]
Posted by "bobhan1 (via GitHub)" <gi...@apache.org>.
bobhan1 commented on PR #50080:
URL: https://github.com/apache/doris/pull/50080#issuecomment-2840635970
run p0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Opt](metric) Add metrics for invalid delete bitmap key count [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #50080:
URL: https://github.com/apache/doris/pull/50080#issuecomment-2814615079
<details>
<summary>TPC-DS: <b>Total hot run time: 185923 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 04f38c48de0b861180fd9a3e189b91326dbaad9e, data reload: false
query1 1000 479 508 479
query2 6597 1835 1811 1811
query3 6748 218 219 218
query4 26906 23770 23364 23364
query5 4332 600 447 447
query6 311 218 203 203
query7 4615 480 287 287
query8 280 242 215 215
query9 8628 2525 2535 2525
query10 444 300 250 250
query11 15397 15508 14810 14810
query12 154 116 108 108
query13 1659 520 393 393
query14 9292 6054 6074 6054
query15 207 195 165 165
query16 7198 646 472 472
query17 1192 722 540 540
query18 1957 401 288 288
query19 189 184 150 150
query20 124 115 111 111
query21 212 125 102 102
query22 4312 4364 4295 4295
query23 33785 32952 33146 32952
query24 8395 2371 2356 2356
query25 517 435 389 389
query26 1226 264 176 176
query27 2743 488 329 329
query28 4316 2104 2069 2069
query29 759 542 415 415
query30 283 220 187 187
query31 953 831 737 737
query32 76 65 61 61
query33 546 364 330 330
query34 787 851 491 491
query35 789 817 738 738
query36 945 991 886 886
query37 105 100 76 76
query38 4169 4158 4218 4158
query39 1452 1378 1375 1375
query40 210 114 104 104
query41 54 54 53 53
query42 122 104 104 104
query43 490 484 475 475
query44 1281 790 794 790
query45 178 170 173 170
query46 823 1027 615 615
query47 1752 1795 1714 1714
query48 362 404 290 290
query49 779 510 417 417
query50 630 688 402 402
query51 4185 4139 4135 4135
query52 108 109 96 96
query53 221 243 185 185
query54 565 573 497 497
query55 76 77 80 77
query56 296 290 298 290
query57 1111 1154 1086 1086
query58 261 264 245 245
query59 2604 2750 2637 2637
query60 338 308 302 302
query61 148 152 152 152
query62 792 730 689 689
query63 228 194 179 179
query64 4425 1007 673 673
query65 4248 4253 4292 4253
query66 1132 415 307 307
query67 15674 15517 15265 15265
query68 7676 864 502 502
query69 463 298 265 265
query70 1211 1085 1099 1085
query71 419 317 287 287
query72 5336 4676 4684 4676
query73 632 578 333 333
query74 9295 9127 8942 8942
query75 3722 3168 2779 2779
query76 3446 1175 736 736
query77 771 351 288 288
query78 9948 9991 9356 9356
query79 2103 815 560 560
query80 565 507 431 431
query81 506 259 255 255
query82 477 121 94 94
query83 252 250 238 238
query84 251 102 81 81
query85 833 349 304 304
query86 379 307 283 283
query87 4308 4413 4356 4356
query88 3834 2231 2241 2231
query89 374 312 275 275
query90 1967 205 210 205
query91 133 133 110 110
query92 84 58 55 55
query93 1831 952 573 573
query94 658 405 283 283
query95 367 293 284 284
query96 483 561 273 273
query97 3165 3245 3107 3107
query98 241 213 203 203
query99 1322 1410 1259 1259
Total cold run time: 273745 ms
Total hot run time: 185923 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Opt](metric) Add metrics for invalid delete bitmap key count [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #50080:
URL: https://github.com/apache/doris/pull/50080#issuecomment-2814623464
<details>
<summary>ClickBench: <b>Total hot run time: 29.37 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 04f38c48de0b861180fd9a3e189b91326dbaad9e, data reload: false
query1 0.04 0.03 0.03
query2 0.12 0.10 0.11
query3 0.25 0.20 0.19
query4 1.60 0.19 0.19
query5 0.59 0.57 0.58
query6 1.19 0.71 0.71
query7 0.03 0.01 0.02
query8 0.04 0.04 0.04
query9 0.59 0.53 0.51
query10 0.57 0.56 0.57
query11 0.15 0.11 0.10
query12 0.14 0.11 0.11
query13 0.60 0.61 0.59
query14 1.14 1.21 1.19
query15 0.88 0.86 0.84
query16 0.37 0.37 0.40
query17 1.05 1.02 1.02
query18 0.21 0.20 0.20
query19 1.90 1.78 1.74
query20 0.01 0.01 0.02
query21 15.40 0.90 0.54
query22 0.75 1.16 0.81
query23 14.80 1.40 0.58
query24 6.97 1.22 0.63
query25 0.46 0.35 0.09
query26 0.46 0.15 0.15
query27 0.05 0.05 0.06
query28 9.38 0.90 0.44
query29 12.56 4.07 3.34
query30 0.25 0.09 0.06
query31 2.82 0.58 0.37
query32 3.23 0.55 0.46
query33 2.97 2.98 3.06
query34 15.75 5.05 4.53
query35 4.47 4.55 4.48
query36 0.67 0.49 0.48
query37 0.09 0.06 0.06
query38 0.06 0.03 0.04
query39 0.03 0.03 0.02
query40 0.18 0.14 0.13
query41 0.08 0.02 0.02
query42 0.04 0.03 0.02
query43 0.04 0.03 0.02
Total cold run time: 102.98 s
Total hot run time: 29.37 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Opt](metric) Add metrics for invalid delete bitmap key count [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #50080:
URL: https://github.com/apache/doris/pull/50080#issuecomment-2808722935
<details>
<summary>TPC-DS: <b>Total hot run time: 192159 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit fd593d49bd896cc66d0568ed55645450dbf45c64, data reload: false
query1 1431 1090 1063 1063
query2 6227 1827 1753 1753
query3 11010 4539 4484 4484
query4 54422 25749 23268 23268
query5 5269 495 477 477
query6 389 208 204 204
query7 5113 487 303 303
query8 331 263 246 246
query9 6586 2589 2634 2589
query10 443 320 270 270
query11 15055 14978 14832 14832
query12 163 110 109 109
query13 1172 508 414 414
query14 10265 6543 6307 6307
query15 198 200 182 182
query16 7107 674 502 502
query17 1068 741 577 577
query18 1529 396 309 309
query19 212 198 164 164
query20 130 121 118 118
query21 216 129 121 121
query22 4361 4380 4417 4380
query23 34028 33191 33469 33191
query24 6693 2444 2481 2444
query25 478 470 410 410
query26 678 271 158 158
query27 2184 519 344 344
query28 3100 2149 2129 2129
query29 593 548 424 424
query30 275 220 185 185
query31 866 858 784 784
query32 74 64 73 64
query33 484 385 314 314
query34 774 858 531 531
query35 804 844 771 771
query36 959 977 886 886
query37 121 107 87 87
query38 4148 4217 4274 4217
query39 1520 1410 1589 1410
query40 218 128 117 117
query41 58 57 60 57
query42 127 110 110 110
query43 500 500 467 467
query44 1337 818 822 818
query45 183 178 168 168
query46 862 1035 656 656
query47 1843 1897 1823 1823
query48 381 416 310 310
query49 690 521 438 438
query50 650 691 432 432
query51 4224 4376 4228 4228
query52 109 106 95 95
query53 228 261 186 186
query54 589 578 528 528
query55 81 78 92 78
query56 305 328 300 300
query57 1180 1176 1088 1088
query58 266 258 260 258
query59 2667 2720 2643 2643
query60 329 333 304 304
query61 133 131 137 131
query62 762 738 685 685
query63 236 198 188 188
query64 1527 1021 730 730
query65 4415 4230 4228 4228
query66 723 394 301 301
query67 15900 15627 15356 15356
query68 7695 874 509 509
query69 546 346 259 259
query70 1198 1131 1079 1079
query71 524 309 303 303
query72 5720 4638 4610 4610
query73 1213 563 350 350
query74 8984 9120 8703 8703
query75 3753 3217 2712 2712
query76 4311 1194 770 770
query77 640 385 290 290
query78 10044 10160 9256 9256
query79 2268 820 560 560
query80 702 511 440 440
query81 484 253 221 221
query82 442 123 96 96
query83 374 247 243 243
query84 288 108 87 87
query85 788 337 306 306
query86 436 294 295 294
query87 4570 4506 4426 4426
query88 3384 2199 2204 2199
query89 388 311 282 282
query90 1934 217 218 217
query91 143 144 112 112
query92 77 62 59 59
query93 1817 948 574 574
query94 686 417 308 308
query95 370 291 280 280
query96 487 572 275 275
query97 3158 3227 3134 3134
query98 237 210 192 192
query99 1430 1417 1269 1269
Total cold run time: 300090 ms
Total hot run time: 192159 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Opt](metric) Add metrics for invalid delete bitmap key count [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #50080:
URL: https://github.com/apache/doris/pull/50080#issuecomment-2838798773
<details>
<summary>ClickBench: <b>Total hot run time: 29.24 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 3e3ace510173441f8b9972f636d2479c5785c6e8, data reload: false
query1 0.03 0.03 0.03
query2 0.12 0.10 0.10
query3 0.26 0.20 0.20
query4 1.59 0.20 0.11
query5 0.56 0.57 0.53
query6 1.20 0.72 0.72
query7 0.02 0.02 0.02
query8 0.04 0.03 0.03
query9 0.57 0.51 0.52
query10 0.59 0.57 0.57
query11 0.15 0.11 0.12
query12 0.15 0.12 0.12
query13 0.61 0.60 0.59
query14 0.80 0.79 0.80
query15 0.87 0.86 0.86
query16 0.37 0.38 0.37
query17 1.02 1.04 1.03
query18 0.20 0.20 0.20
query19 1.89 1.74 1.86
query20 0.01 0.02 0.01
query21 15.43 0.90 0.56
query22 0.76 1.22 0.71
query23 14.84 1.40 0.61
query24 7.00 1.32 0.80
query25 0.50 0.18 0.06
query26 0.58 0.17 0.15
query27 0.05 0.05 0.05
query28 9.07 0.85 0.45
query29 12.55 4.01 3.35
query30 0.25 0.10 0.06
query31 2.82 0.60 0.38
query32 3.23 0.56 0.49
query33 2.97 3.03 3.08
query34 15.88 5.13 4.55
query35 4.53 4.59 4.55
query36 0.66 0.49 0.48
query37 0.09 0.07 0.06
query38 0.05 0.04 0.04
query39 0.03 0.02 0.02
query40 0.18 0.14 0.13
query41 0.08 0.03 0.02
query42 0.03 0.03 0.02
query43 0.04 0.03 0.03
Total cold run time: 102.67 s
Total hot run time: 29.24 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Opt](metric) Add metrics for invalid delete bitmap key count [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #50080:
URL: https://github.com/apache/doris/pull/50080#issuecomment-2808083682
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR).
Please clearly describe your PR:
1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
3. What features were added. Why was this function added?
4. Which code was refactored and why was this part of the code refactored?
5. Which functions were optimized and what is the difference before and after the optimization?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Opt](metric) Add metrics for invalid delete bitmap key count [doris]
Posted by "bobhan1 (via GitHub)" <gi...@apache.org>.
bobhan1 closed pull request #50080: [Opt](metric) Add metrics for invalid delete bitmap key count
URL: https://github.com/apache/doris/pull/50080
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Opt](metric) Add metrics for invalid delete bitmap key count [doris]
Posted by "bobhan1 (via GitHub)" <gi...@apache.org>.
bobhan1 commented on PR #50080:
URL: https://github.com/apache/doris/pull/50080#issuecomment-2838385984
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Opt](metric) Add metrics for invalid delete bitmap key count [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #50080:
URL: https://github.com/apache/doris/pull/50080#issuecomment-2838701601
<details>
<summary>TPC-H: <b>Total hot run time: 34103 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 3e3ace510173441f8b9972f636d2479c5785c6e8, data reload: false
------ Round 1 ----------------------------------
q1 26118 5123 5041 5041
q2 2076 282 181 181
q3 10510 1261 718 718
q4 10226 998 523 523
q5 7704 2478 2294 2294
q6 186 164 130 130
q7 944 762 621 621
q8 9331 1357 1172 1172
q9 6882 5124 5217 5124
q10 6850 2317 1903 1903
q11 478 289 273 273
q12 353 354 211 211
q13 17778 3707 3082 3082
q14 219 231 217 217
q15 527 488 491 488
q16 430 435 369 369
q17 626 876 392 392
q18 7771 7119 7140 7119
q19 1515 982 567 567
q20 352 350 225 225
q21 4126 3388 2480 2480
q22 1047 1000 973 973
Total cold run time: 116049 ms
Total hot run time: 34103 ms
----- Round 2, with runtime_filter_mode=off -----
q1 5160 5054 5108 5054
q2 240 325 236 236
q3 2155 2647 2299 2299
q4 1343 1788 1471 1471
q5 4574 4458 4409 4409
q6 215 172 127 127
q7 1993 1952 1745 1745
q8 2636 2590 2604 2590
q9 7233 7273 7053 7053
q10 3007 3217 2728 2728
q11 569 516 487 487
q12 690 738 642 642
q13 3539 4002 3236 3236
q14 294 281 279 279
q15 525 497 461 461
q16 444 497 442 442
q17 1208 1503 1430 1430
q18 7708 7698 7439 7439
q19 815 857 922 857
q20 1975 1988 1812 1812
q21 4988 4717 4513 4513
q22 1077 1040 988 988
Total cold run time: 52388 ms
Total hot run time: 50298 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Opt](metric) Add metrics for invalid delete bitmap key count [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #50080:
URL: https://github.com/apache/doris/pull/50080#issuecomment-2814836295
# BE UT Coverage Report
Increment line coverage `4.35% (2/46)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/04f38c48de0b861180fd9a3e189b91326dbaad9e_04f38c48de0b861180fd9a3e189b91326dbaad9e/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/04f38c48de0b861180fd9a3e189b91326dbaad9e_04f38c48de0b861180fd9a3e189b91326dbaad9e/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 53.14% (14422/27141) |
| Line Coverage | 42.00% (124961/297531) |
| Region Coverage | 40.82% (63845/156401) |
| Branch Coverage | 35.46% (32106/90546) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Opt](metric) Add metrics for invalid delete bitmap key count [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #50080:
URL: https://github.com/apache/doris/pull/50080#issuecomment-2814602061
<details>
<summary>TPC-H: <b>Total hot run time: 34180 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 04f38c48de0b861180fd9a3e189b91326dbaad9e, data reload: false
------ Round 1 ----------------------------------
q1 26387 5061 5087 5061
q2 2079 282 196 196
q3 10507 1245 712 712
q4 10237 1010 532 532
q5 7752 2387 2387 2387
q6 184 177 137 137
q7 915 737 629 629
q8 9315 1337 1158 1158
q9 6844 5059 5077 5059
q10 6814 2288 1872 1872
q11 475 285 273 273
q12 338 348 217 217
q13 17776 3650 3065 3065
q14 220 221 214 214
q15 538 492 498 492
q16 437 452 396 396
q17 619 872 382 382
q18 7747 7361 7177 7177
q19 1785 942 549 549
q20 343 339 224 224
q21 4089 3353 2489 2489
q22 1043 1037 959 959
Total cold run time: 116444 ms
Total hot run time: 34180 ms
----- Round 2, with runtime_filter_mode=off -----
q1 5120 5068 5081 5068
q2 236 337 239 239
q3 2147 2646 2286 2286
q4 1426 1829 1524 1524
q5 4489 4465 4362 4362
q6 211 170 129 129
q7 1963 1904 1759 1759
q8 2598 2513 2583 2513
q9 7296 7278 7120 7120
q10 2992 3186 2738 2738
q11 578 496 480 480
q12 648 758 600 600
q13 3489 3840 3321 3321
q14 271 286 284 284
q15 515 479 469 469
q16 478 537 457 457
q17 1143 1519 1401 1401
q18 7713 7444 7353 7353
q19 805 792 781 781
q20 1963 2024 1839 1839
q21 5073 4673 4571 4571
q22 1029 1015 996 996
Total cold run time: 52183 ms
Total hot run time: 50290 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Opt](metric) Add metrics for invalid delete bitmap key count [doris]
Posted by "zhannngchen (via GitHub)" <gi...@apache.org>.
zhannngchen commented on code in PR #50080:
URL: https://github.com/apache/doris/pull/50080#discussion_r2052078374
##########
be/src/olap/tablet_meta.cpp:
##########
@@ -1390,6 +1391,19 @@ DeleteBitmap::Version DeleteBitmap::_get_rowset_cache_version(const BitmapKey& b
return 0;
}
+void DeleteBitmap::traverse_rowset_id_prefix(
+ const std::function<void(const RowsetId& rowsetId)>& func) const {
+ std::shared_lock rlock {lock};
+ auto it = delete_bitmap.cbegin();
+ while (it != delete_bitmap.cend()) {
+ RowsetId rowset_id = std::get<0>(it->first);
+ func(rowset_id);
+ // find next rowset id
+ it = delete_bitmap.upper_bound({rowset_id, std::numeric_limits<SegmentId>::max(),
Review Comment:
the key count is not accurate if you skip to next rowsetid
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Opt](metric) Add metrics for invalid delete bitmap key count [doris]
Posted by "bobhan1 (via GitHub)" <gi...@apache.org>.
bobhan1 commented on PR #50080:
URL: https://github.com/apache/doris/pull/50080#issuecomment-2840734099
run p0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Opt](metric) Add metrics for invalid delete bitmap key count [doris]
Posted by "bobhan1 (via GitHub)" <gi...@apache.org>.
bobhan1 commented on PR #50080:
URL: https://github.com/apache/doris/pull/50080#issuecomment-2811535140
run cloud_p0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Opt](metric) Add metrics for invalid delete bitmap key count [doris]
Posted by "bobhan1 (via GitHub)" <gi...@apache.org>.
bobhan1 commented on PR #50080:
URL: https://github.com/apache/doris/pull/50080#issuecomment-2808470074
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org