You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "Yukang-Lian (via GitHub)" <gi...@apache.org> on 2025/05/21 10:52:50 UTC
[PR] [Cherry-pick](branch-3.0) Pick "[Feature](Compaction)Improve Compaction Profiling and Logging #50950" [doris]
Yukang-Lian opened a new pull request, #51125:
URL: https://github.com/apache/doris/pull/51125
### What problem does this PR solve?
Issue Number: close #xxx
Related PR: #xxx
Pick #50950
Problem Summary:
### Release note
None
### Check List (For Author)
- Test <!-- At least one of them must be included. -->
- [ ] Regression test
- [ ] Unit Test
- [ ] Manual test (add detailed scripts or steps below)
- [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
- [ ] Previous test can cover this change.
- [ ] No code files have been changed.
- [ ] Other reason <!-- Add your reason? -->
- Behavior changed:
- [ ] No.
- [ ] Yes. <!-- Explain the behavior change -->
- Does this need documentation?
- [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg: https://github.com/apache/doris-website/pull/1214 -->
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR should merge into -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Cherry-pick](branch-3.0) Pick "[Feature](Compaction)Improve Compaction Profiling and Logging #50950" [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #51125:
URL: https://github.com/apache/doris/pull/51125#issuecomment-2897695750
# BE UT Coverage Report
Increment line coverage `2.68% (8/298)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/2bae7250d66a02286b7ce9140b822d411dbd663e_2bae7250d66a02286b7ce9140b822d411dbd663e/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/2bae7250d66a02286b7ce9140b822d411dbd663e_2bae7250d66a02286b7ce9140b822d411dbd663e/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 41.07% (10861/26443) |
| Line Coverage | 31.88% (92728/290863) |
| Region Coverage | 30.97% (47830/154453) |
| Branch Coverage | 27.43% (24486/89252) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Cherry-pick](branch-3.0) Pick "[Feature](Compaction)Improve Compaction Profiling and Logging #50950" [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #51125:
URL: https://github.com/apache/doris/pull/51125#issuecomment-2898252971
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Cherry-pick](branch-3.0) Pick "[Feature](Compaction)Improve Compaction Profiling and Logging #50950" [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #51125:
URL: https://github.com/apache/doris/pull/51125#issuecomment-2897714808
<details>
<summary>TPC-DS: <b>Total hot run time: 197600 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 2bae7250d66a02286b7ce9140b822d411dbd663e, data reload: false
query1 1273 915 879 879
query2 6390 1902 1902 1902
query3 10938 4554 4493 4493
query4 61562 28800 23792 23792
query5 5245 458 439 439
query6 385 177 166 166
query7 5460 325 306 306
query8 314 239 223 223
query9 8390 2597 2595 2595
query10 457 272 256 256
query11 17617 15623 16105 15623
query12 167 106 104 104
query13 1440 436 416 416
query14 9791 7752 7397 7397
query15 202 189 179 179
query16 7493 496 512 496
query17 1153 590 575 575
query18 1837 331 306 306
query19 217 158 153 153
query20 119 113 109 109
query21 205 100 105 100
query22 4620 4349 4663 4349
query23 34585 34048 34133 34048
query24 6136 2930 2905 2905
query25 545 425 417 417
query26 652 168 173 168
query27 1767 355 381 355
query28 3883 2162 2109 2109
query29 707 480 428 428
query30 227 157 153 153
query31 985 819 826 819
query32 69 53 56 53
query33 419 292 308 292
query34 909 506 510 506
query35 843 719 741 719
query36 1082 939 960 939
query37 114 70 72 70
query38 4041 3926 3998 3926
query39 1502 1456 1476 1456
query40 201 96 100 96
query41 52 47 48 47
query42 115 110 101 101
query43 526 481 495 481
query44 1151 815 808 808
query45 187 165 168 165
query46 1171 727 712 712
query47 1979 1875 1916 1875
query48 470 400 379 379
query49 704 413 399 399
query50 841 427 412 412
query51 7465 7172 7404 7172
query52 106 93 94 93
query53 257 187 181 181
query54 550 479 475 475
query55 79 79 76 76
query56 274 243 265 243
query57 1267 1194 1144 1144
query58 235 207 212 207
query59 3160 3116 3081 3081
query60 300 251 256 251
query61 114 112 105 105
query62 790 717 680 680
query63 224 187 187 187
query64 1352 646 619 619
query65 3263 3153 3212 3153
query66 711 327 296 296
query67 15893 15504 15395 15395
query68 4366 571 563 563
query69 423 261 264 261
query70 1160 1089 1089 1089
query71 323 262 247 247
query72 6357 3989 3928 3928
query73 744 342 355 342
query74 10412 9249 9345 9249
query75 3353 2644 2660 2644
query76 2115 1033 1156 1033
query77 477 270 260 260
query78 10571 9552 9596 9552
query79 2202 591 596 591
query80 1408 444 431 431
query81 507 220 222 220
query82 1246 87 87 87
query83 271 138 140 138
query84 279 82 79 79
query85 1035 317 292 292
query86 411 283 300 283
query87 4385 4214 4270 4214
query88 3866 2394 2380 2380
query89 416 293 289 289
query90 1959 186 188 186
query91 179 147 146 146
query92 61 46 49 46
query93 2966 560 541 541
query94 802 302 284 284
query95 364 251 251 251
query96 617 282 283 282
query97 3284 3134 3143 3134
query98 213 206 196 196
query99 1582 1308 1338 1308
Total cold run time: 315532 ms
Total hot run time: 197600 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Cherry-pick](branch-3.0) Pick "[Feature](Compaction)Improve Compaction Profiling and Logging #50950" [doris]
Posted by "dataroaring (via GitHub)" <gi...@apache.org>.
dataroaring merged PR #51125:
URL: https://github.com/apache/doris/pull/51125
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Cherry-pick](branch-3.0) Pick "[Feature](Compaction)Improve Compaction Profiling and Logging #50950" [doris]
Posted by "Yukang-Lian (via GitHub)" <gi...@apache.org>.
Yukang-Lian commented on PR #51125:
URL: https://github.com/apache/doris/pull/51125#issuecomment-2897504779
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Cherry-pick](branch-3.0) Pick "[Feature](Compaction)Improve Compaction Profiling and Logging #50950" [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #51125:
URL: https://github.com/apache/doris/pull/51125#issuecomment-2897670724
<details>
<summary>TPC-H: <b>Total hot run time: 39944 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 2bae7250d66a02286b7ce9140b822d411dbd663e, data reload: false
------ Round 1 ----------------------------------
q1 17605 6971 6661 6661
q2 2058 170 176 170
q3 10642 1084 1163 1084
q4 10546 761 776 761
q5 7779 2907 2873 2873
q6 215 139 134 134
q7 998 631 612 612
q8 9374 1966 2081 1966
q9 6628 6424 6386 6386
q10 7043 2290 2306 2290
q11 478 268 257 257
q12 394 213 217 213
q13 17768 2969 2986 2969
q14 229 213 215 213
q15 525 466 460 460
q16 491 393 381 381
q17 984 601 534 534
q18 7275 6641 6685 6641
q19 1393 1067 977 977
q20 488 203 208 203
q21 4000 3195 3183 3183
q22 1108 994 976 976
Total cold run time: 108021 ms
Total hot run time: 39944 ms
----- Round 2, with runtime_filter_mode=off -----
q1 6614 6577 6572 6572
q2 335 231 224 224
q3 2880 2714 2910 2714
q4 2037 1814 1830 1814
q5 5753 5773 5751 5751
q6 210 129 137 129
q7 2199 1797 1777 1777
q8 3346 3568 3527 3527
q9 8999 8776 8866 8776
q10 3613 3505 3511 3505
q11 595 503 484 484
q12 799 577 601 577
q13 8895 3128 3130 3128
q14 294 276 297 276
q15 527 478 459 459
q16 476 441 431 431
q17 1847 1651 1599 1599
q18 8307 7707 7808 7707
q19 1699 1582 1649 1582
q20 2109 1869 1840 1840
q21 5462 5212 5223 5212
q22 1121 1086 1048 1048
Total cold run time: 68117 ms
Total hot run time: 59132 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Cherry-pick](branch-3.0) Pick "[Feature](Compaction)Improve Compaction Profiling and Logging #50950" [doris]
Posted by "Thearas (via GitHub)" <gi...@apache.org>.
Thearas commented on PR #51125:
URL: https://github.com/apache/doris/pull/51125#issuecomment-2897504517
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR).
Please clearly describe your PR:
1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
3. What features were added. Why was this function added?
4. Which code was refactored and why was this part of the code refactored?
5. Which functions were optimized and what is the difference before and after the optimization?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Cherry-pick](branch-3.0) Pick "[Feature](Compaction)Improve Compaction Profiling and Logging #50950" [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #51125:
URL: https://github.com/apache/doris/pull/51125#issuecomment-2897730685
<details>
<summary>ClickBench: <b>Total hot run time: 29.35 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 2bae7250d66a02286b7ce9140b822d411dbd663e, data reload: false
query1 0.03 0.03 0.03
query2 0.06 0.03 0.03
query3 0.23 0.07 0.06
query4 1.62 0.10 0.10
query5 0.52 0.51 0.50
query6 1.14 0.73 0.73
query7 0.02 0.02 0.01
query8 0.04 0.03 0.04
query9 0.57 0.50 0.50
query10 0.55 0.54 0.54
query11 0.14 0.10 0.11
query12 0.14 0.11 0.10
query13 0.61 0.59 0.60
query14 0.77 0.78 0.82
query15 0.84 0.82 0.82
query16 0.37 0.36 0.38
query17 0.97 1.00 1.02
query18 0.23 0.22 0.23
query19 1.95 1.77 1.84
query20 0.01 0.01 0.01
query21 15.39 0.62 0.59
query22 2.76 2.27 1.45
query23 17.09 0.94 0.86
query24 3.10 1.23 0.46
query25 0.25 0.24 0.08
query26 0.28 0.14 0.13
query27 0.05 0.04 0.04
query28 10.88 0.47 0.46
query29 12.57 3.29 3.30
query30 0.24 0.06 0.07
query31 2.86 0.38 0.39
query32 3.25 0.46 0.46
query33 2.93 3.01 2.98
query34 17.08 4.50 4.50
query35 4.54 4.58 4.58
query36 0.66 0.48 0.47
query37 0.08 0.05 0.06
query38 0.05 0.04 0.03
query39 0.04 0.02 0.03
query40 0.16 0.13 0.12
query41 0.08 0.02 0.02
query42 0.03 0.02 0.02
query43 0.03 0.02 0.03
Total cold run time: 105.21 s
Total hot run time: 29.35 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Cherry-pick](branch-3.0) Pick "[Feature](Compaction)Improve Compaction Profiling and Logging #50950" [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #51125:
URL: https://github.com/apache/doris/pull/51125#issuecomment-2898253140
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org