You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "mrhhsg (via GitHub)" <gi...@apache.org> on 2026/04/03 02:11:14 UTC
[PR] [fix](be) Restore workload group limit refresh after memory recovery [doris]
mrhhsg opened a new pull request, #62070:
URL: https://github.com/apache/doris/pull/62070
### What problem does this PR solve?
Issue Number: close #xxx
Related PR: #xxx
Problem Summary:
### Release note
None
### Check List (For Author)
- Test <!-- At least one of them must be included. -->
- [ ] Regression test
- [ ] Unit Test
- [ ] Manual test (add detailed scripts or steps below)
- [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
- [ ] Previous test can cover this change.
- [ ] No code files have been changed.
- [ ] Other reason <!-- Add your reason? -->
- Behavior changed:
- [ ] No.
- [ ] Yes. <!-- Explain the behavior change -->
- Does this need documentation?
- [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg: https://github.com/apache/doris-website/pull/1214 -->
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR should merge into -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](be) Restore workload group limit refresh after memory recovery [doris]
Posted by "mrhhsg (via GitHub)" <gi...@apache.org>.
mrhhsg commented on PR #62070:
URL: https://github.com/apache/doris/pull/62070#issuecomment-4181583401
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](be) Restore workload group limit refresh after memory recovery [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #62070:
URL: https://github.com/apache/doris/pull/62070#issuecomment-4181833213
# BE UT Coverage Report
Increment line coverage ` ` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/bba902b69dda463b4b6358fc58df47884a91dca3_bba902b69dda463b4b6358fc58df47884a91dca3/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/bba902b69dda463b4b6358fc58df47884a91dca3_bba902b69dda463b4b6358fc58df47884a91dca3/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 52.95% (20061/37885) |
| Line Coverage | 36.58% (188563/515539) |
| Region Coverage | 32.83% (146410/445901) |
| Branch Coverage | 33.99% (64119/188625) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](be) Restore workload group limit refresh after memory recovery [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #62070:
URL: https://github.com/apache/doris/pull/62070#issuecomment-4181715798
<details>
<summary>TPC-H: <b>Total hot run time: 29204 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit bba902b69dda463b4b6358fc58df47884a91dca3, data reload: false
------ Round 1 ----------------------------------
orders Doris NULL NULL 0 0 0 NULL 0 NULL NULL 2023-12-26 18:27:23 2023-12-26 18:42:55 NULL utf-8 NULL NULL
============================================
q1 17910 3787 3759 3759
q2 q3 10680 924 611 611
q4 4688 466 355 355
q5 7466 1358 1164 1164
q6 183 162 136 136
q7 923 946 777 777
q8 9310 1386 1373 1373
q9 5458 5330 5343 5330
q10 6246 2033 1806 1806
q11 462 272 277 272
q12 647 411 288 288
q13 18119 2796 2151 2151
q14 286 293 258 258
q15 q16 862 858 788 788
q17 1023 1046 809 809
q18 6447 5691 5551 5551
q19 1181 1273 1031 1031
q20 552 421 301 301
q21 4892 2544 2085 2085
q22 506 408 359 359
Total cold run time: 97841 ms
Total hot run time: 29204 ms
----- Round 2, with runtime_filter_mode=off -----
orders Doris NULL NULL 150000000 42 6422171781 NULL 22778155 NULL NULL 2023-12-26 18:27:23 2023-12-26 18:42:55 NULL utf-8 NULL NULL
============================================
q1 4559 4498 4681 4498
q2 q3 4630 4714 4246 4246
q4 2022 2075 1348 1348
q5 4899 5053 5189 5053
q6 194 163 126 126
q7 2014 1732 1636 1636
q8 3439 3148 3206 3148
q9 8283 8351 8345 8345
q10 4529 4592 4245 4245
q11 608 435 393 393
q12 668 714 495 495
q13 2655 3034 2304 2304
q14 291 298 277 277
q15 q16 771 787 694 694
q17 1305 1291 1236 1236
q18 7898 7315 6991 6991
q19 1142 1141 1120 1120
q20 2277 2282 1949 1949
q21 6032 5301 4766 4766
q22 536 508 420 420
Total cold run time: 58752 ms
Total hot run time: 53290 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](be) Restore workload group limit refresh after memory recovery [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #62070:
URL: https://github.com/apache/doris/pull/62070#issuecomment-4181517486
No issues found in this review.
Checkpoint conclusions:
- Goal/task: The patch addresses the reported early-return limit-refresh bug and the paused-query cleanup/recovery branches. The added unit tests cover the zero-WG-memory refresh path, Phase 2/3/4 paused-query handling, and cross-WG blocking scenarios.
- Scope/minimality: The change stays focused to `be/src/runtime/workload_group/workload_group_manager.cpp` plus targeted BE unit tests.
- Concurrency: The touched shared state remains guarded by the existing `_paused_queries_lock`, and the changed `WorkloadGroup` accesses continue to use the existing WG locking model. I did not find a new lock-order or deadlock regression in the modified paths.
- Lifecycle/static init/config/compatibility: No new lifecycle hazards, static initialization issues, config items, or compatibility concerns were introduced.
- Parallel paths/conditions: The fix is applied consistently across the affected paused-query branches, and the new same-WG/global recent-cancel conditions match the surrounding memory semantics.
- Test coverage: Coverage is materially improved for the fixed branches. I could not rerun `./run-be-ut.sh --run --filter="WorkloadGroupManagerTest.*"` in this runner because `thirdparty/installed/bin/protoc` is missing, so execution here could not validate the tests.
- Observability: Existing INFO logging remains sufficient for these branches.
- Transaction/persistence/data-write/FE-BE protocol: Not applicable to this change.
- Performance/other issues: No additional correctness or performance issues were identified in the touched logic.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](be) Restore workload group limit refresh after memory recovery [doris]
Posted by "Thearas (via GitHub)" <gi...@apache.org>.
Thearas commented on PR #62070:
URL: https://github.com/apache/doris/pull/62070#issuecomment-4181412252
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR).
Please clearly describe your PR:
1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
3. What features were added. Why was this function added?
4. Which code was refactored and why was this part of the code refactored?
5. Which functions were optimized and what is the difference before and after the optimization?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](be) Restore workload group limit refresh after memory recovery [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #62070:
URL: https://github.com/apache/doris/pull/62070#issuecomment-4181516852
No issues found in this review.
Checkpoint conclusions:
- Goal/task: The patch addresses the reported early-return limit-refresh bug and the paused-query cleanup/recovery branches. The added unit tests cover the zero-WG-memory refresh path, Phase 2/3/4 paused-query handling, and cross-WG blocking scenarios.
- Scope/minimality: The change stays focused to plus targeted BE unit tests.
- Concurrency: The touched shared state remains guarded by the existing , and the changed accesses continue to use the existing WG locking model. I did not find a new lock-order or deadlock regression in the modified paths.
- Lifecycle/static init/config/compatibility: No new lifecycle hazards, static initialization issues, config items, or compatibility concerns were introduced.
- Parallel paths/conditions: The fix is applied consistently across the affected paused-query branches, and the new same-WG/global recent-cancel conditions match the surrounding memory semantics.
- Test coverage: Coverage is materially improved for the fixed branches. I could not rerun Target system: Linux; Target arch: x86_64
Python 3.12.3
Check JAVA_HOME version
Apache Maven 3.9.14 (996c630dbc656c76214ce58821dcc58be960875b)
Maven home: /usr/share/apache-maven-3.9.14
Java version: 17.0.18, vendor: Eclipse Adoptium, runtime: /usr/lib/jvm/temurin-17-jdk-amd64
Default locale: en_US, platform encoding: UTF-8
OS name: "linux", version: "6.17.0-1008-azure", arch: "amd64", family: "unix"
cmake version 3.31.6
CMake suite maintained and supported by Kitware (kitware.com/cmake).
ninja 1.13.2
Get params:
PARALLEL -- 1
CLEAN -- 0
ENABLE_PCH -- ON
WITH_TDE_DIR --
Build Backend UT
Update apache-orc submodule ...
Update clucene submodule ...
-- Make program: /usr/local/bin/ninja
-- Use ccache: and
-- Extra cxx flags:
-- GLIBC_COMPATIBILITY is ON
-- USE_LIBCPP is OFF
-- USE_JEMALLOC is OFF
-- USE_UNWIND is ON
-- ENABLE_PCH is ON
-- USE_AVX2 is ON
-- Build type is ASAN_UT
-- Build target arch is x86_64
-- DORIS_HOME is /home/runner/work/doris/doris
-- THIRDPARTY_DIR is /home/runner/work/doris/doris/thirdparty/installed
-- make test: ON
-- make benchmark: OFF
-- build fs benchmark tool: OFF
-- build task executor simulator: OFF
-- build file cache lru tool: OFF
-- build gensrc if necessary
make: Entering directory '/home/runner/work/doris/doris/gensrc'
make -C script
make[1]: Entering directory '/home/runner/work/doris/doris/gensrc/script'
/home/runner/work/doris/doris/gensrc/script/gen_build_version.sh
make[1]: Leaving directory '/home/runner/work/doris/doris/gensrc/script'
make -C proto
make[1]: Entering directory '/home/runner/work/doris/doris/gensrc/proto'
/home/runner/work/doris/doris/thirdparty/installed/bin/protoc --proto_path=/home/runner/work/doris/doris/gensrc/proto --cpp_out=/home/runner/work/doris/doris/gensrc/proto/../build//gen_cpp /home/runner/work/doris/doris/gensrc/proto/descriptors.proto
make[1]: Leaving directory '/home/runner/work/doris/doris/gensrc/proto'
make: Leaving directory '/home/runner/work/doris/doris/gensrc'
-- Configuring incomplete, errors occurred! in this runner because is missing, so execution here could not validate the tests.
- Observability: Existing INFO logging remains sufficient for these branches.
- Transaction/persistence/data-write/FE-BE protocol: Not applicable to this change.
- Performance/other issues: No additional correctness or performance issues were identified in the touched logic.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](be) Restore workload group limit refresh after memory recovery [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #62070:
URL: https://github.com/apache/doris/pull/62070#issuecomment-4213255501
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](be) Restore workload group limit refresh after memory recovery [doris]
Posted by "mrhhsg (via GitHub)" <gi...@apache.org>.
mrhhsg commented on PR #62070:
URL: https://github.com/apache/doris/pull/62070#issuecomment-4181501752
/review
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](be) Restore workload group limit refresh after memory recovery [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #62070:
URL: https://github.com/apache/doris/pull/62070#issuecomment-4181768771
<details>
<summary>TPC-DS: <b>Total hot run time: 178838 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit bba902b69dda463b4b6358fc58df47884a91dca3, data reload: false
query5 4363 706 528 528
query6 356 233 216 216
query7 4258 622 336 336
query8 353 257 236 236
query9 8734 3898 3896 3896
query10 473 364 329 329
query11 6652 5484 5135 5135
query12 188 135 127 127
query13 1298 605 448 448
query14 5665 5149 4767 4767
query14_1 4143 4152 4086 4086
query15 209 205 185 185
query16 994 464 445 445
query17 1074 787 664 664
query18 2448 503 379 379
query19 248 213 176 176
query20 137 134 129 129
query21 222 141 119 119
query22 13919 14938 14765 14765
query23 18098 17173 16710 16710
query23_1 16769 16837 17049 16837
query24 7510 1741 1350 1350
query24_1 1352 1373 1411 1373
query25 605 485 426 426
query26 1268 325 170 170
query27 2693 612 380 380
query28 4442 1893 1868 1868
query29 971 660 561 561
query30 295 229 193 193
query31 1088 1052 940 940
query32 87 74 70 70
query33 539 364 279 279
query34 1209 1153 670 670
query35 728 789 666 666
query36 1256 1224 1063 1063
query37 151 99 88 88
query38 3093 2974 2965 2965
query39 909 890 848 848
query39_1 839 852 822 822
query40 242 148 129 129
query41 62 63 58 58
query42 112 105 108 105
query43 310 318 276 276
query44
query45 223 195 187 187
query46 1136 1312 773 773
query47 2338 2324 2233 2233
query48 406 416 302 302
query49 631 514 425 425
query50 767 285 218 218
query51 4346 4334 4451 4334
query52 108 110 98 98
query53 241 270 203 203
query54 316 280 255 255
query55 104 97 90 90
query56 318 316 304 304
query57 1760 1722 1649 1649
query58 302 279 273 273
query59 2916 2961 2711 2711
query60 326 331 351 331
query61 161 146 155 146
query62 698 623 570 570
query63 236 199 190 190
query64 5313 1315 955 955
query65
query66 1489 473 373 373
query67 24441 24275 24171 24171
query68
query69 459 339 308 308
query70 1009 982 998 982
query71 309 281 275 275
query72 2986 2711 2420 2420
query73 855 831 448 448
query74 9871 9712 9583 9583
query75 2739 2631 2310 2310
query76 2315 1113 755 755
query77 390 400 322 322
query78 11430 11388 10780 10780
query79 1527 1105 829 829
query80 688 567 508 508
query81 446 277 237 237
query82 1318 169 125 125
query83 366 301 277 277
query84 281 155 121 121
query85 854 545 438 438
query86 379 327 312 312
query87 3296 3208 3122 3122
query88 3536 2686 2690 2686
query89 444 384 359 359
query90 1987 182 173 173
query91 175 168 149 149
query92 76 75 73 73
query93 975 960 556 556
query94 531 343 301 301
query95 639 457 343 343
query96 1069 774 336 336
query97 2689 2683 2573 2573
query98 242 226 224 224
query99 1064 1074 981 981
Total cold run time: 257101 ms
Total hot run time: 178838 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](be) Restore workload group limit refresh after memory recovery [doris]
Posted by "mrhhsg (via GitHub)" <gi...@apache.org>.
mrhhsg merged PR #62070:
URL: https://github.com/apache/doris/pull/62070
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](be) Restore workload group limit refresh after memory recovery [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #62070:
URL: https://github.com/apache/doris/pull/62070#issuecomment-4182007519
# BE Regression && UT Coverage Report
Increment line coverage `92.44% (428/463)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/62070_bba902b69dda463b4b6358fc58df47884a91dca3_merge/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/62070_bba902b69dda463b4b6358fc58df47884a91dca3_merge/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 73.66% (27330/37105) |
| Line Coverage | 57.30% (294481/513969) |
| Region Coverage | 54.36% (244635/450040) |
| Branch Coverage | 56.17% (106273/189207) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](be) Restore workload group limit refresh after memory recovery [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #62070:
URL: https://github.com/apache/doris/pull/62070#issuecomment-4213255283
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org