You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "zhiqiang-hhhh (via GitHub)" <gi...@apache.org> on 2025/04/09 13:12:21 UTC
[PR] [fix](profile) Fix incorrect check of Profile.shouldStoreToStorage [doris]
zhiqiang-hhhh opened a new pull request, #49925:
URL: https://github.com/apache/doris/pull/49925
### What problem does this PR solve?
If profile collection is finished, we should write it to storage immediately.
Add unit test for class Profile.
Bug is introduced by https://github.com/apache/doris/pull/48653
Issue Number: close #xxx
Related PR: https://github.com/apache/doris/pull/48653
Problem Summary:
### Release note
None
### Check List (For Author)
- Test <!-- At least one of them must be included. -->
- [ ] Regression test
- [ ] Unit Test
- [ ] Manual test (add detailed scripts or steps below)
- [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
- [ ] Previous test can cover this change.
- [ ] No code files have been changed.
- [ ] Other reason <!-- Add your reason? -->
- Behavior changed:
- [ ] No.
- [ ] Yes. <!-- Explain the behavior change -->
- Does this need documentation?
- [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg: https://github.com/apache/doris-website/pull/1214 -->
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR should merge into -->
This pull request includes changes to the `Profile` class and its associated test suite to enhance the functionality for determining when a profile should be stored to storage. The most important changes include adding a new condition to check if execution profiles are completed before storing, and the addition of a comprehensive test suite for the `Profile` class.
Enhancements to `Profile` class:
* Added logic to `shouldStoreToStorage` method to check if all execution profiles are completed before deciding to store the profile. [[1]](diffhunk://#diff-c9f83a512334f9031efeea62a2121286ce58e16b7f14aaee354d42e094a594abR453-R465) [[2]](diffhunk://#diff-c9f83a512334f9031efeea62a2121286ce58e16b7f14aaee354d42e094a594abL468-R481)
Addition of test suite for `Profile` class:
* Created `ProfileTest` class with multiple test cases to verify the behavior of `Profile` methods, including `shouldStoreToStorage`, `writeToStorage`, `readFromStorage`, and `deleteFromStorage`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](profile) Fix incorrect check of Profile.shouldStoreToStorage [doris]
Posted by "zhiqiang-hhhh (via GitHub)" <gi...@apache.org>.
zhiqiang-hhhh commented on PR #49925:
URL: https://github.com/apache/doris/pull/49925#issuecomment-2789722499
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](profile) Fix incorrect check of Profile.shouldStoreToStorage [doris]
Posted by "zhiqiang-hhhh (via GitHub)" <gi...@apache.org>.
zhiqiang-hhhh commented on PR #49925:
URL: https://github.com/apache/doris/pull/49925#issuecomment-2789659464
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](profile) Fix incorrect check of Profile.shouldStoreToStorage [doris]
Posted by "zhiqiang-hhhh (via GitHub)" <gi...@apache.org>.
zhiqiang-hhhh commented on PR #49925:
URL: https://github.com/apache/doris/pull/49925#issuecomment-2791327442
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](profile) Fix incorrect check of Profile.shouldStoreToStorage [doris]
Posted by "BiteTheDDDDt (via GitHub)" <gi...@apache.org>.
BiteTheDDDDt merged PR #49925:
URL: https://github.com/apache/doris/pull/49925
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](profile) Fix incorrect check of Profile.shouldStoreToStorage [doris]
Posted by "Thearas (via GitHub)" <gi...@apache.org>.
Thearas commented on PR #49925:
URL: https://github.com/apache/doris/pull/49925#issuecomment-2789659223
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR).
Please clearly describe your PR:
1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
3. What features were added. Why was this function added?
4. Which code was refactored and why was this part of the code refactored?
5. Which functions were optimized and what is the difference before and after the optimization?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](profile) Fix incorrect check of Profile.shouldStoreToStorage [doris]
Posted by "zhiqiang-hhhh (via GitHub)" <gi...@apache.org>.
zhiqiang-hhhh commented on PR #49925:
URL: https://github.com/apache/doris/pull/49925#issuecomment-2791469945
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](profile) Fix incorrect check of Profile.shouldStoreToStorage [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #49925:
URL: https://github.com/apache/doris/pull/49925#issuecomment-2791519716
<details>
<summary>ClickBench: <b>Total hot run time: 30.99 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 6c93a4c55be88ce4529758ac27f32e88f82e7f1f, data reload: false
query1 0.04 0.04 0.02
query2 0.12 0.11 0.11
query3 0.24 0.19 0.19
query4 1.60 0.18 0.20
query5 0.59 0.58 0.58
query6 1.20 0.70 0.72
query7 0.03 0.02 0.01
query8 0.04 0.03 0.03
query9 0.58 0.52 0.51
query10 0.56 0.58 0.57
query11 0.15 0.11 0.11
query12 0.14 0.11 0.11
query13 0.61 0.60 0.59
query14 2.69 2.82 2.82
query15 0.93 0.85 0.85
query16 0.40 0.38 0.39
query17 1.02 1.04 1.06
query18 0.22 0.20 0.20
query19 1.93 1.98 1.79
query20 0.01 0.01 0.01
query21 15.36 0.90 0.54
query22 0.75 1.21 0.65
query23 14.89 1.38 0.60
query24 7.03 1.69 0.56
query25 0.50 0.18 0.20
query26 0.68 0.15 0.15
query27 0.06 0.05 0.05
query28 9.92 0.86 0.43
query29 12.54 4.07 3.36
query30 0.25 0.08 0.06
query31 2.82 0.61 0.38
query32 3.23 0.55 0.48
query33 3.07 3.00 3.07
query34 15.70 5.10 4.48
query35 4.52 4.52 4.47
query36 0.66 0.50 0.49
query37 0.09 0.06 0.06
query38 0.05 0.04 0.03
query39 0.03 0.02 0.02
query40 0.17 0.16 0.13
query41 0.08 0.02 0.03
query42 0.03 0.02 0.02
query43 0.03 0.04 0.03
Total cold run time: 105.56 s
Total hot run time: 30.99 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](profile) Fix incorrect check of Profile.shouldStoreToStorage [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #49925:
URL: https://github.com/apache/doris/pull/49925#issuecomment-2791501129
<details>
<summary>TPC-H: <b>Total hot run time: 34378 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 6c93a4c55be88ce4529758ac27f32e88f82e7f1f, data reload: false
------ Round 1 ----------------------------------
q1 25968 5081 5063 5063
q2 2067 277 187 187
q3 10392 1204 696 696
q4 10236 998 544 544
q5 7579 2292 2337 2292
q6 180 159 131 131
q7 885 729 625 625
q8 9310 1229 1017 1017
q9 6948 5095 4987 4987
q10 6819 2285 1889 1889
q11 489 295 275 275
q12 353 349 215 215
q13 17796 3668 3089 3089
q14 245 228 208 208
q15 536 485 499 485
q16 632 620 602 602
q17 613 849 380 380
q18 7970 7445 7459 7445
q19 1225 936 564 564
q20 347 341 232 232
q21 4517 3498 2482 2482
q22 1039 1067 970 970
Total cold run time: 116146 ms
Total hot run time: 34378 ms
----- Round 2, with runtime_filter_mode=off -----
q1 5140 5057 5095 5057
q2 231 324 234 234
q3 2116 2623 2245 2245
q4 1427 1811 1491 1491
q5 4461 4384 4326 4326
q6 218 168 130 130
q7 1952 1937 1752 1752
q8 2553 2427 2554 2427
q9 7283 7139 7019 7019
q10 2932 3163 2733 2733
q11 573 519 512 512
q12 670 747 608 608
q13 3451 3806 3307 3307
q14 288 286 265 265
q15 515 475 474 474
q16 640 670 649 649
q17 1142 1506 1395 1395
q18 7723 7491 7536 7491
q19 808 870 961 870
q20 1892 1954 1797 1797
q21 5303 4846 4713 4713
q22 1076 1036 1010 1010
Total cold run time: 52394 ms
Total hot run time: 50505 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](profile) Fix incorrect check of Profile.shouldStoreToStorage [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #49925:
URL: https://github.com/apache/doris/pull/49925#issuecomment-2796179805
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](profile) Fix incorrect check of Profile.shouldStoreToStorage [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #49925:
URL: https://github.com/apache/doris/pull/49925#issuecomment-2796179882
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](profile) Fix incorrect check of Profile.shouldStoreToStorage [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #49925:
URL: https://github.com/apache/doris/pull/49925#issuecomment-2791513702
<details>
<summary>TPC-DS: <b>Total hot run time: 193060 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 6c93a4c55be88ce4529758ac27f32e88f82e7f1f, data reload: false
query1 1408 1083 1059 1059
query2 6004 1949 1942 1942
query3 10997 4554 4526 4526
query4 25462 23678 23375 23375
query5 5266 622 448 448
query6 332 211 208 208
query7 3992 493 281 281
query8 315 251 245 245
query9 8526 2561 2567 2561
query10 517 333 264 264
query11 15257 14971 14784 14784
query12 154 123 101 101
query13 1546 501 370 370
query14 9198 6205 6006 6006
query15 204 183 166 166
query16 7607 644 461 461
query17 1135 773 592 592
query18 2035 422 335 335
query19 206 194 173 173
query20 126 128 126 126
query21 210 129 116 116
query22 4543 4636 4274 4274
query23 34880 33490 33665 33490
query24 8667 2508 2448 2448
query25 528 489 424 424
query26 1209 284 151 151
query27 3118 510 344 344
query28 4688 2452 2446 2446
query29 710 562 448 448
query30 277 229 198 198
query31 909 856 830 830
query32 86 63 62 62
query33 553 371 313 313
query34 805 874 527 527
query35 855 866 756 756
query36 967 1004 900 900
query37 118 107 80 80
query38 4153 4204 4214 4204
query39 1529 1482 1511 1482
query40 216 125 110 110
query41 54 54 52 52
query42 125 122 111 111
query43 516 520 483 483
query44 1309 812 829 812
query45 196 180 175 175
query46 849 1004 644 644
query47 1912 1873 1795 1795
query48 390 415 327 327
query49 784 537 426 426
query50 687 709 420 420
query51 4290 4251 4172 4172
query52 108 105 102 102
query53 238 278 199 199
query54 587 595 523 523
query55 83 82 81 81
query56 339 295 296 295
query57 1181 1200 1109 1109
query58 284 278 264 264
query59 2868 2964 2775 2775
query60 352 331 311 311
query61 140 130 127 127
query62 771 749 703 703
query63 233 190 185 185
query64 4126 1062 726 726
query65 4430 4372 4454 4372
query66 1101 435 311 311
query67 15945 15792 15203 15203
query68 9046 879 515 515
query69 504 312 262 262
query70 1238 1090 1101 1090
query71 478 315 297 297
query72 5554 4729 4745 4729
query73 708 591 350 350
query74 9005 9039 8975 8975
query75 4332 3222 2666 2666
query76 3788 1172 748 748
query77 794 382 274 274
query78 10085 9955 9348 9348
query79 2670 798 566 566
query80 663 502 457 457
query81 502 267 225 225
query82 534 126 98 98
query83 279 255 242 242
query84 301 112 86 86
query85 832 356 315 315
query86 368 293 303 293
query87 4448 4494 4274 4274
query88 3523 2212 2232 2212
query89 393 309 287 287
query90 1925 212 213 212
query91 144 146 111 111
query92 79 57 59 57
query93 1893 950 583 583
query94 665 420 278 278
query95 441 298 286 286
query96 484 563 269 269
query97 3154 3234 3122 3122
query98 244 212 197 197
query99 1465 1407 1307 1307
Total cold run time: 283745 ms
Total hot run time: 193060 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org