You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "Yukang-Lian (via GitHub)" <gi...@apache.org> on 2026/03/11 08:14:37 UTC
[PR] [refactor](compaction) submit manual full compaction task to thread pool instead of detached thread [doris]
Yukang-Lian opened a new pull request, #61222:
URL: https://github.com/apache/doris/pull/61222
### What problem does this PR solve?
Issue Number: close #xxx
Related PR: #xxx
Problem Summary:
Previously, manually triggering full compaction on a single tablet via HTTP API would create a detached thread for each request, which lacks concurrency control, deduplication, and metrics tracking.
This PR changes the single-tablet full compaction path to use submit_compaction_task(), submitting to the base compaction thread pool — consistent with the table-level path and the cloud engine. Base and cumulative compaction remain unchanged.
### Release note
None
### Check List (For Author)
- Test <!-- At least one of them must be included. -->
- [ ] Regression test
- [ ] Unit Test
- [ ] Manual test (add detailed scripts or steps below)
- [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
- [ ] Previous test can cover this change.
- [ ] No code files have been changed.
- [ ] Other reason <!-- Add your reason? -->
- Behavior changed:
- [ ] No.
- [ ] Yes. <!-- Explain the behavior change -->
- Does this need documentation?
- [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg: https://github.com/apache/doris-website/pull/1214 -->
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR should merge into -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](compaction) submit manual full compaction task to thread pool instead of detached thread [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #61222:
URL: https://github.com/apache/doris/pull/61222#issuecomment-4074004838
<details>
<summary>TPC-H: <b>Total hot run time: 26743 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 98b10a25524479f1293d8949e4bb6ac412c95111, data reload: false
------ Round 1 ----------------------------------
orders Doris NULL NULL 0 0 0 NULL 0 NULL NULL 2023-12-26 18:27:23 2023-12-26 18:42:55 NULL utf-8 NULL NULL
============================================
q1 17656 4399 4303 4303
q2 q3 10650 781 518 518
q4 4683 357 250 250
q5 7539 1221 1030 1030
q6 179 178 148 148
q7 803 835 697 697
q8 9315 1459 1327 1327
q9 4845 4733 4679 4679
q10 6287 1913 1627 1627
q11 489 254 235 235
q12 791 586 475 475
q13 18069 2955 2160 2160
q14 227 252 217 217
q15 q16 735 749 672 672
q17 735 819 473 473
q18 6020 5494 5253 5253
q19 1220 997 623 623
q20 531 494 377 377
q21 4508 1839 1400 1400
q22 368 504 279 279
Total cold run time: 95650 ms
Total hot run time: 26743 ms
----- Round 2, with runtime_filter_mode=off -----
orders Doris NULL NULL 150000000 42 6422171781 NULL 22778155 NULL NULL 2023-12-26 18:27:23 2023-12-26 18:42:55 NULL utf-8 NULL NULL
============================================
q1 4912 4758 4610 4610
q2 q3 3847 4368 3863 3863
q4 888 1224 780 780
q5 4081 4434 4328 4328
q6 180 168 135 135
q7 1779 1672 1514 1514
q8 2477 2804 2541 2541
q9 7593 7507 7351 7351
q10 3731 3955 3678 3678
q11 533 490 408 408
q12 508 593 442 442
q13 2869 3182 2358 2358
q14 290 304 288 288
q15 q16 714 783 735 735
q17 1143 1490 1340 1340
q18 7286 6932 6658 6658
q19 897 882 891 882
q20 2039 2177 2041 2041
q21 3988 3479 3298 3298
q22 490 541 487 487
Total cold run time: 50245 ms
Total hot run time: 47737 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](compaction) submit manual full compaction task to thread pool instead of detached thread [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #61222:
URL: https://github.com/apache/doris/pull/61222#issuecomment-4074694350
# BE Regression && UT Coverage Report
Increment line coverage `100% (0/0)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/61222_98b10a25524479f1293d8949e4bb6ac412c95111_merge/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/61222_98b10a25524479f1293d8949e4bb6ac412c95111_merge/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 71.20% (26158/36738) |
| Line Coverage | 54.02% (274312/507778) |
| Region Coverage | 51.13% (226799/443595) |
| Branch Coverage | 52.68% (98022/186086) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](compaction) submit manual full compaction task to thread pool instead of detached thread [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen closed pull request #61222: [refactor](compaction) submit manual full compaction task to thread pool instead of detached thread
URL: https://github.com/apache/doris/pull/61222
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](compaction) submit manual full compaction task to thread pool instead of detached thread [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #61222:
URL: https://github.com/apache/doris/pull/61222#issuecomment-4196784878
# BE UT Coverage Report
Increment line coverage `12.90% (4/31)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/11617583eee95d30b13e1425acb9cec01e156d28_11617583eee95d30b13e1425acb9cec01e156d28/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/11617583eee95d30b13e1425acb9cec01e156d28_11617583eee95d30b13e1425acb9cec01e156d28/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 53.00% (20108/37943) |
| Line Coverage | 36.55% (188984/517095) |
| Region Coverage | 32.83% (146720/446939) |
| Branch Coverage | 33.94% (64212/189195) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](compaction) submit manual full compaction task to thread pool instead of detached thread [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #61222:
URL: https://github.com/apache/doris/pull/61222#issuecomment-4213045977
# BE UT Coverage Report
Increment line coverage `12.90% (4/31)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/4de9c7cf86ca60835f9b9c5d1d9e7000d9a29da2_4de9c7cf86ca60835f9b9c5d1d9e7000d9a29da2/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/4de9c7cf86ca60835f9b9c5d1d9e7000d9a29da2_4de9c7cf86ca60835f9b9c5d1d9e7000d9a29da2/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 53.06% (20146/37971) |
| Line Coverage | 36.60% (189438/517660) |
| Region Coverage | 32.86% (147097/447629) |
| Branch Coverage | 33.96% (64342/189474) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](compaction) submit manual full compaction task to thread pool instead of detached thread [doris]
Posted by "Yukang-Lian (via GitHub)" <gi...@apache.org>.
Yukang-Lian commented on PR #61222:
URL: https://github.com/apache/doris/pull/61222#issuecomment-4037432774
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](compaction) submit manual full compaction task to thread pool instead of detached thread [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #61222:
URL: https://github.com/apache/doris/pull/61222#issuecomment-4037587950
<details>
<summary>TPC-H: <b>Total hot run time: 27717 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 4420035c2c3afc07f46a1e0473141101f9fffae5, data reload: false
------ Round 1 ----------------------------------
============================================
q1 17641 4571 4315 4315
q2 q3 10643 807 513 513
q4 4679 368 260 260
q5 7553 1198 1019 1019
q6 180 179 148 148
q7 802 845 681 681
q8 9282 1477 1380 1380
q9 4945 4736 4642 4642
q10 6248 1904 1652 1652
q11 447 263 241 241
q12 674 564 468 468
q13 18070 2928 2169 2169
q14 233 231 215 215
q15 886 793 793 793
q16 734 722 667 667
q17 712 865 438 438
q18 6020 5373 5293 5293
q19 1112 1011 615 615
q20 503 507 389 389
q21 4342 2082 1550 1550
q22 426 309 269 269
Total cold run time: 96132 ms
Total hot run time: 27717 ms
----- Round 2, with runtime_filter_mode=off -----
============================================
q1 4804 4549 4527 4527
q2 q3 3863 4331 3819 3819
q4 882 1219 790 790
q5 4055 4439 4340 4340
q6 186 181 145 145
q7 1757 1699 1499 1499
q8 2482 2713 2734 2713
q9 7445 7154 7427 7154
q10 3782 3945 3640 3640
q11 509 479 486 479
q12 539 603 456 456
q13 2675 3143 2365 2365
q14 289 301 294 294
q15 890 827 796 796
q16 722 754 751 751
q17 1141 1425 1353 1353
q18 7051 6721 6578 6578
q19 958 878 893 878
q20 2064 2198 2034 2034
q21 4029 3498 3413 3413
q22 528 462 393 393
Total cold run time: 50651 ms
Total hot run time: 48417 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](compaction) submit manual full compaction task to thread pool instead of detached thread [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #61222:
URL: https://github.com/apache/doris/pull/61222#issuecomment-4047092188
<details>
<summary>TPC-H: <b>Total hot run time: 27766 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 7ff50ae2caf139d0337bed64698ba211c5ec102d, data reload: false
------ Round 1 ----------------------------------
============================================
q1 17629 4539 4326 4326
q2 q3 10665 800 519 519
q4 4735 350 257 257
q5 8002 1217 1012 1012
q6 212 174 147 147
q7 814 831 667 667
q8 10345 1540 1376 1376
q9 6629 4752 4757 4752
q10 6437 1922 1648 1648
q11 492 266 262 262
q12 749 566 473 473
q13 18098 2934 2187 2187
q14 239 231 215 215
q15 928 822 797 797
q16 789 731 686 686
q17 723 883 419 419
q18 6032 5495 5215 5215
q19 1149 1004 616 616
q20 505 500 391 391
q21 4924 1979 1531 1531
q22 393 311 270 270
Total cold run time: 100489 ms
Total hot run time: 27766 ms
----- Round 2, with runtime_filter_mode=off -----
============================================
q1 4737 4526 4526 4526
q2 q3 3865 4328 3782 3782
q4 909 1189 803 803
q5 4083 4372 4338 4338
q6 207 187 154 154
q7 1833 1698 1551 1551
q8 2501 2808 2598 2598
q9 7436 7334 7371 7334
q10 3775 4006 3685 3685
q11 503 457 410 410
q12 492 614 433 433
q13 2727 3120 2387 2387
q14 292 310 287 287
q15 828 785 787 785
q16 720 755 722 722
q17 1152 1500 1462 1462
q18 7234 6979 6583 6583
q19 911 927 944 927
q20 2084 2244 2029 2029
q21 4042 3482 3340 3340
q22 469 451 395 395
Total cold run time: 50800 ms
Total hot run time: 48531 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](compaction) submit manual full compaction task to thread pool instead of detached thread [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #61222:
URL: https://github.com/apache/doris/pull/61222#issuecomment-4116620139
<details>
<summary>TPC-H: <b>Total hot run time: 26663 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit ab484dc10b44aeee407b98ac3a0922e35e1001b5, data reload: false
------ Round 1 ----------------------------------
orders Doris NULL NULL 0 0 0 NULL 0 NULL NULL 2023-12-26 18:27:23 2023-12-26 18:42:55 NULL utf-8 NULL NULL
============================================
q1 17636 4394 4291 4291
q2 q3 10646 762 515 515
q4 4663 354 244 244
q5 7571 1200 1007 1007
q6 178 181 143 143
q7 761 853 667 667
q8 9303 1476 1336 1336
q9 4936 4823 4774 4774
q10 6481 1986 1682 1682
q11 470 260 249 249
q12 766 592 467 467
q13 18669 2657 1969 1969
q14 230 236 216 216
q15 q16 737 737 673 673
q17 746 837 454 454
q18 5872 5485 5269 5269
q19 1126 976 608 608
q20 537 484 375 375
q21 4456 1821 1433 1433
q22 505 344 291 291
Total cold run time: 96289 ms
Total hot run time: 26663 ms
----- Round 2, with runtime_filter_mode=off -----
orders Doris NULL NULL 150000000 42 6422171781 NULL 22778155 NULL NULL 2023-12-26 18:27:23 2023-12-26 18:42:55 NULL utf-8 NULL NULL
============================================
q1 4771 4642 4579 4579
q2 q3 3860 4364 3830 3830
q4 876 1213 795 795
q5 4047 4464 4321 4321
q6 183 178 139 139
q7 1768 1659 1556 1556
q8 2486 2777 2570 2570
q9 7539 7410 7451 7410
q10 3754 3959 3596 3596
q11 516 427 425 425
q12 514 595 436 436
q13 2453 3095 2113 2113
q14 309 316 282 282
q15 q16 711 793 733 733
q17 1194 1358 1374 1358
q18 7218 6885 6656 6656
q19 885 865 901 865
q20 2120 2157 1991 1991
q21 3896 3480 3293 3293
q22 479 435 392 392
Total cold run time: 49579 ms
Total hot run time: 47340 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](compaction) submit manual full compaction task to thread pool instead of detached thread [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #61222:
URL: https://github.com/apache/doris/pull/61222#issuecomment-4179123217
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](compaction) submit manual full compaction task to thread pool instead of detached thread [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #61222:
URL: https://github.com/apache/doris/pull/61222#issuecomment-4124164640
# BE UT Coverage Report
Increment line coverage ` ` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/ab484dc10b44aeee407b98ac3a0922e35e1001b5_ab484dc10b44aeee407b98ac3a0922e35e1001b5/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/ab484dc10b44aeee407b98ac3a0922e35e1001b5_ab484dc10b44aeee407b98ac3a0922e35e1001b5/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 52.82% (19870/37619) |
| Line Coverage | 36.34% (185709/511097) |
| Region Coverage | 32.58% (143804/441454) |
| Branch Coverage | 33.79% (62968/186369) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](compaction) submit manual full compaction task to thread pool instead of detached thread [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #61222:
URL: https://github.com/apache/doris/pull/61222#issuecomment-4074070646
<details>
<summary>TPC-DS: <b>Total hot run time: 168811 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 98b10a25524479f1293d8949e4bb6ac412c95111, data reload: false
query5 4354 644 501 501
query6 336 224 206 206
query7 4221 490 284 284
query8 343 238 217 217
query9 8729 2770 2725 2725
query10 528 383 346 346
query11 6951 5099 4899 4899
query12 183 132 125 125
query13 1278 461 344 344
query14 5770 3718 3471 3471
query14_1 2831 2825 2847 2825
query15 205 195 174 174
query16 990 462 445 445
query17 1084 699 613 613
query18 2441 447 332 332
query19 213 203 195 195
query20 136 126 123 123
query21 212 131 105 105
query22 13186 14172 14718 14172
query23 16293 15898 15578 15578
query23_1 15714 15691 15525 15525
query24 7142 1622 1199 1199
query24_1 1213 1214 1210 1210
query25 551 485 440 440
query26 1231 262 151 151
query27 2772 491 292 292
query28 4518 1870 1880 1870
query29 825 563 490 490
query30 298 231 192 192
query31 1021 938 879 879
query32 80 73 68 68
query33 512 338 291 291
query34 910 887 535 535
query35 633 682 592 592
query36 1109 1133 1026 1026
query37 138 101 84 84
query38 2908 2935 2953 2935
query39 864 826 817 817
query39_1 788 789 790 789
query40 233 154 140 140
query41 66 59 60 59
query42 261 262 256 256
query43 248 251 222 222
query44
query45 197 192 185 185
query46 881 972 627 627
query47 2159 2160 2082 2082
query48 316 312 228 228
query49 631 455 389 389
query50 683 275 218 218
query51 4159 4115 4080 4080
query52 274 274 259 259
query53 296 340 290 290
query54 320 276 284 276
query55 97 93 93 93
query56 353 332 318 318
query57 1938 1923 1700 1700
query58 303 297 283 283
query59 2804 2949 2724 2724
query60 357 354 339 339
query61 183 183 178 178
query62 626 587 540 540
query63 316 291 284 284
query64 5160 1395 1119 1119
query65
query66 1486 478 379 379
query67 24272 24306 24286 24286
query68
query69 442 315 305 305
query70 989 1005 980 980
query71 341 322 305 305
query72 3006 2728 2430 2430
query73 537 555 325 325
query74 9631 9574 9399 9399
query75 2854 2772 2449 2449
query76 2272 1046 680 680
query77 364 378 312 312
query78 10923 11137 10428 10428
query79 1105 822 580 580
query80 1353 629 549 549
query81 558 265 230 230
query82 1051 151 124 124
query83 335 269 245 245
query84 253 118 96 96
query85 915 512 448 448
query86 426 324 295 295
query87 3142 3077 3028 3028
query88 3543 2672 2658 2658
query89 431 377 350 350
query90 2038 175 180 175
query91 176 174 134 134
query92 79 75 74 74
query93 925 880 497 497
query94 647 323 285 285
query95 584 343 382 343
query96 656 517 223 223
query97 2428 2463 2391 2391
query98 241 222 221 221
query99 1029 1009 871 871
Total cold run time: 249533 ms
Total hot run time: 168811 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](compaction) submit manual full compaction task to thread pool instead of detached thread [doris]
Posted by "Yukang-Lian (via GitHub)" <gi...@apache.org>.
Yukang-Lian commented on PR #61222:
URL: https://github.com/apache/doris/pull/61222#issuecomment-4197973709
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](compaction) submit manual full compaction task to thread pool instead of detached thread [doris]
Posted by "Thearas (via GitHub)" <gi...@apache.org>.
Thearas commented on PR #61222:
URL: https://github.com/apache/doris/pull/61222#issuecomment-4037324287
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR).
Please clearly describe your PR:
1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
3. What features were added. Why was this function added?
4. Which code was refactored and why was this part of the code refactored?
5. Which functions were optimized and what is the difference before and after the optimization?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](compaction) submit manual full compaction task to thread pool instead of detached thread [doris]
Posted by "Yukang-Lian (via GitHub)" <gi...@apache.org>.
Yukang-Lian commented on PR #61222:
URL: https://github.com/apache/doris/pull/61222#issuecomment-4053223780
run p0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](compaction) submit manual full compaction task to thread pool instead of detached thread [doris]
Posted by "Yukang-Lian (via GitHub)" <gi...@apache.org>.
Yukang-Lian commented on PR #61222:
URL: https://github.com/apache/doris/pull/61222#issuecomment-4046452928
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](compaction) submit manual full compaction task to thread pool instead of detached thread [doris]
Posted by "Yukang-Lian (via GitHub)" <gi...@apache.org>.
Yukang-Lian commented on PR #61222:
URL: https://github.com/apache/doris/pull/61222#issuecomment-4124609629
run nonConcurrent
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](compaction) submit manual full compaction task to thread pool instead of detached thread [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #61222:
URL: https://github.com/apache/doris/pull/61222#issuecomment-4037932540
# BE UT Coverage Report
Increment line coverage `0.00% (0/23)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/4420035c2c3afc07f46a1e0473141101f9fffae5_4420035c2c3afc07f46a1e0473141101f9fffae5/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/4420035c2c3afc07f46a1e0473141101f9fffae5_4420035c2c3afc07f46a1e0473141101f9fffae5/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 52.57% (19653/37384) |
| Line Coverage | 36.21% (183434/506642) |
| Region Coverage | 32.33% (141631/438061) |
| Branch Coverage | 33.48% (61774/184503) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](compaction) submit manual full compaction task to thread pool instead of detached thread [doris]
Posted by "Yukang-Lian (via GitHub)" <gi...@apache.org>.
Yukang-Lian commented on PR #61222:
URL: https://github.com/apache/doris/pull/61222#issuecomment-4197874580
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](compaction) submit manual full compaction task to thread pool instead of detached thread [doris]
Posted by "Yukang-Lian (via GitHub)" <gi...@apache.org>.
Yukang-Lian commented on PR #61222:
URL: https://github.com/apache/doris/pull/61222#issuecomment-4122706758
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](compaction) submit manual full compaction task to thread pool instead of detached thread [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #61222:
URL: https://github.com/apache/doris/pull/61222#issuecomment-4108993100
# BE Regression && UT Coverage Report
Increment line coverage `100% (0/0)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/61222_98b10a25524479f1293d8949e4bb6ac412c95111_merge/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/61222_98b10a25524479f1293d8949e4bb6ac412c95111_merge/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 71.32% (26201/36738) |
| Line Coverage | 54.16% (274998/507778) |
| Region Coverage | 51.27% (227421/443595) |
| Branch Coverage | 52.80% (98245/186086) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](compaction) submit manual full compaction task to thread pool instead of detached thread [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #61222:
URL: https://github.com/apache/doris/pull/61222#issuecomment-4074292655
# BE UT Coverage Report
Increment line coverage ` ` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/98b10a25524479f1293d8949e4bb6ac412c95111_98b10a25524479f1293d8949e4bb6ac412c95111/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/98b10a25524479f1293d8949e4bb6ac412c95111_98b10a25524479f1293d8949e4bb6ac412c95111/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 52.69% (19776/37533) |
| Line Coverage | 36.27% (184795/509493) |
| Region Coverage | 32.42% (142516/439547) |
| Branch Coverage | 33.56% (62293/185604) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](compaction) submit manual full compaction task to thread pool instead of detached thread [doris]
Posted by "luwei16 (via GitHub)" <gi...@apache.org>.
luwei16 merged PR #61222:
URL: https://github.com/apache/doris/pull/61222
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](compaction) submit manual full compaction task to thread pool instead of detached thread [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #61222:
URL: https://github.com/apache/doris/pull/61222#issuecomment-4125270089
# BE Regression && UT Coverage Report
Increment line coverage `100% (0/0)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/61222_ab484dc10b44aeee407b98ac3a0922e35e1001b5_merge/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/61222_ab484dc10b44aeee407b98ac3a0922e35e1001b5_merge/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 57.35% (21124/36836) |
| Line Coverage | 40.41% (205928/509534) |
| Region Coverage | 37.15% (165531/445580) |
| Branch Coverage | 37.95% (70944/186933) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](compaction) submit manual full compaction task to thread pool instead of detached thread [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #61222:
URL: https://github.com/apache/doris/pull/61222#issuecomment-4037642038
<details>
<summary>TPC-DS: <b>Total hot run time: 153336 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 4420035c2c3afc07f46a1e0473141101f9fffae5, data reload: false
query5 4345 629 508 508
query6 330 226 222 222
query7 4219 474 263 263
query8 342 250 259 250
query9 8719 2766 2745 2745
query10 557 389 366 366
query11 7391 5811 5575 5575
query12 180 135 127 127
query13 1284 456 351 351
query14 6362 3808 3527 3527
query14_1 2831 2849 2833 2833
query15 208 194 180 180
query16 1001 490 477 477
query17 1111 721 621 621
query18 2654 448 353 353
query19 210 209 180 180
query20 137 136 127 127
query21 227 151 125 125
query22 4917 5003 4828 4828
query23 16556 16232 15738 15738
query23_1 15772 16003 15953 15953
query24 7991 1685 1254 1254
query24_1 1260 1212 1231 1212
query25 572 505 490 490
query26 1243 261 148 148
query27 2782 468 286 286
query28 4478 1849 1849 1849
query29 888 561 469 469
query30 305 246 209 209
query31 1355 1287 1219 1219
query32 100 75 72 72
query33 504 329 274 274
query34 958 914 579 579
query35 635 677 609 609
query36 1040 1172 920 920
query37 131 96 81 81
query38 2969 2897 2894 2894
query39 886 877 851 851
query39_1 820 840 823 823
query40 230 153 134 134
query41 65 62 61 61
query42 342 293 300 293
query43 245 258 226 226
query44
query45 201 190 186 186
query46 856 983 604 604
query47 2123 2136 2067 2067
query48 318 306 234 234
query49 643 478 379 379
query50 674 275 215 215
query51 4102 4107 4065 4065
query52 284 286 280 280
query53 293 339 284 284
query54 293 275 258 258
query55 94 89 86 86
query56 307 320 302 302
query57 1367 1341 1259 1259
query58 285 279 275 275
query59 1302 1468 1325 1325
query60 336 321 318 318
query61 143 143 144 143
query62 611 584 542 542
query63 313 278 269 269
query64 5135 1288 1011 1011
query65
query66 1399 462 351 351
query67 16353 16431 16379 16379
query68
query69 383 297 280 280
query70 1024 989 982 982
query71 340 311 303 303
query72 2921 2426 2492 2426
query73 539 551 317 317
query74 9984 9933 9746 9746
query75 2864 2765 2460 2460
query76 2301 1031 671 671
query77 360 400 311 311
query78 11149 11537 10678 10678
query79 1134 800 600 600
query80 1512 625 520 520
query81 576 279 243 243
query82 1015 155 118 118
query83 347 265 241 241
query84 241 115 104 104
query85 1072 489 431 431
query86 419 313 301 301
query87 3133 3122 3007 3007
query88 3559 2663 2670 2663
query89 426 367 349 349
query90 1958 177 173 173
query91 168 157 135 135
query92 80 86 71 71
query93 961 818 505 505
query94 640 328 299 299
query95 590 410 332 332
query96 652 512 231 231
query97 2441 2516 2481 2481
query98 233 221 221 221
query99 943 996 925 925
Total cold run time: 235540 ms
Total hot run time: 153336 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](compaction) submit manual full compaction task to thread pool instead of detached thread [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #61222:
URL: https://github.com/apache/doris/pull/61222#issuecomment-4054254005
# BE Regression && UT Coverage Report
Increment line coverage `100% (0/0)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/61222_7ff50ae2caf139d0337bed64698ba211c5ec102d_merge/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/61222_7ff50ae2caf139d0337bed64698ba211c5ec102d_merge/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 73.35% (26868/36631) |
| Line Coverage | 56.69% (287080/506373) |
| Region Coverage | 54.09% (239759/443225) |
| Branch Coverage | 55.71% (103393/185597) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](compaction) submit manual full compaction task to thread pool instead of detached thread [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #61222:
URL: https://github.com/apache/doris/pull/61222#issuecomment-4109090939
# BE Regression && UT Coverage Report
Increment line coverage `100% (0/0)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/61222_98b10a25524479f1293d8949e4bb6ac412c95111_merge/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/61222_98b10a25524479f1293d8949e4bb6ac412c95111_merge/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 71.57% (26295/36738) |
| Line Coverage | 54.43% (276386/507778) |
| Region Coverage | 51.60% (228917/443595) |
| Branch Coverage | 53.09% (98785/186086) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](compaction) submit manual full compaction task to thread pool instead of detached thread [doris]
Posted by "Yukang-Lian (via GitHub)" <gi...@apache.org>.
Yukang-Lian commented on PR #61222:
URL: https://github.com/apache/doris/pull/61222#issuecomment-4115882894
run nonConcurrent
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](compaction) submit manual full compaction task to thread pool instead of detached thread [doris]
Posted by "Yukang-Lian (via GitHub)" <gi...@apache.org>.
Yukang-Lian commented on PR #61222:
URL: https://github.com/apache/doris/pull/61222#issuecomment-4116357359
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](compaction) submit manual full compaction task to thread pool instead of detached thread [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #61222:
URL: https://github.com/apache/doris/pull/61222#issuecomment-4047165404
<details>
<summary>TPC-DS: <b>Total hot run time: 153332 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 7ff50ae2caf139d0337bed64698ba211c5ec102d, data reload: false
query5 4327 638 527 527
query6 336 227 209 209
query7 4206 471 265 265
query8 339 240 231 231
query9 8718 2734 2706 2706
query10 508 391 333 333
query11 7312 5772 5644 5644
query12 182 127 127 127
query13 1252 459 337 337
query14 5664 3825 3573 3573
query14_1 2773 2794 2785 2785
query15 197 189 176 176
query16 975 475 457 457
query17 1083 692 594 594
query18 2421 436 340 340
query19 242 205 179 179
query20 133 126 133 126
query21 222 144 122 122
query22 4894 4935 4861 4861
query23 15976 15463 15339 15339
query23_1 15441 16217 15919 15919
query24 7271 1696 1295 1295
query24_1 1322 1284 1250 1250
query25 647 516 463 463
query26 1448 315 169 169
query27 2951 554 306 306
query28 4735 1934 1921 1921
query29 942 676 478 478
query30 310 254 218 218
query31 1347 1293 1252 1252
query32 81 69 72 69
query33 505 332 267 267
query34 932 934 568 568
query35 647 693 599 599
query36 1076 1130 998 998
query37 132 90 82 82
query38 2917 2928 2881 2881
query39 993 886 848 848
query39_1 838 809 830 809
query40 233 151 138 138
query41 62 60 61 60
query42 291 293 303 293
query43 236 263 214 214
query44
query45 202 191 191 191
query46 880 982 626 626
query47 2102 2130 2043 2043
query48 311 312 225 225
query49 633 454 380 380
query50 674 275 214 214
query51 4134 4143 4051 4051
query52 286 293 275 275
query53 283 335 280 280
query54 307 269 260 260
query55 88 94 84 84
query56 321 319 317 317
query57 1375 1365 1297 1297
query58 301 277 316 277
query59 1287 1447 1319 1319
query60 343 338 322 322
query61 157 151 149 149
query62 612 599 543 543
query63 308 285 273 273
query64 5141 1260 1043 1043
query65
query66 1460 472 351 351
query67 16368 16584 16177 16177
query68
query69 391 308 292 292
query70 1006 987 940 940
query71 337 313 301 301
query72 2888 2882 2727 2727
query73 534 548 328 328
query74 9985 9916 9740 9740
query75 2873 2766 2477 2477
query76 2302 1033 685 685
query77 363 381 318 318
query78 11312 11464 10710 10710
query79 1155 781 604 604
query80 1459 635 580 580
query81 580 282 250 250
query82 1004 158 122 122
query83 378 271 250 250
query84 252 125 99 99
query85 1285 565 442 442
query86 418 311 295 295
query87 3210 3071 2998 2998
query88 3601 2691 2670 2670
query89 432 369 357 357
query90 1885 176 177 176
query91 178 161 136 136
query92 81 73 74 73
query93 1135 838 515 515
query94 641 327 311 311
query95 595 399 319 319
query96 632 530 229 229
query97 2513 2518 2413 2413
query98 240 221 220 220
query99 1015 1002 917 917
Total cold run time: 233711 ms
Total hot run time: 153332 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](compaction) submit manual full compaction task to thread pool instead of detached thread [doris]
Posted by "Yukang-Lian (via GitHub)" <gi...@apache.org>.
Yukang-Lian commented on PR #61222:
URL: https://github.com/apache/doris/pull/61222#issuecomment-4065610311
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](compaction) submit manual full compaction task to thread pool instead of detached thread [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #61222:
URL: https://github.com/apache/doris/pull/61222#issuecomment-4066932936
# BE Regression && UT Coverage Report
Increment line coverage `100% (0/0)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/61222_827471cc17d36d9670cc0d37b3131658451c19c8_merge/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/61222_827471cc17d36d9670cc0d37b3131658451c19c8_merge/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 71.59% (26226/36634) |
| Line Coverage | 54.41% (275698/506724) |
| Region Coverage | 51.65% (228652/442706) |
| Branch Coverage | 53.06% (98472/185572) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](compaction) submit manual full compaction task to thread pool instead of detached thread [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #61222:
URL: https://github.com/apache/doris/pull/61222#issuecomment-4116686469
<details>
<summary>TPC-DS: <b>Total hot run time: 168514 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit ab484dc10b44aeee407b98ac3a0922e35e1001b5, data reload: false
query5 4321 651 528 528
query6 341 251 211 211
query7 4223 474 277 277
query8 346 251 234 234
query9 8721 2709 2706 2706
query10 553 368 352 352
query11 6996 5077 4908 4908
query12 192 128 127 127
query13 1302 453 354 354
query14 5641 3680 3495 3495
query14_1 2848 2819 2819 2819
query15 207 195 175 175
query16 990 469 445 445
query17 990 727 632 632
query18 2447 465 356 356
query19 224 220 184 184
query20 139 130 131 130
query21 213 131 107 107
query22 13122 13776 14782 13776
query23 16737 16334 15905 15905
query23_1 16142 16040 15707 15707
query24 7172 1622 1206 1206
query24_1 1238 1231 1236 1231
query25 621 473 407 407
query26 1227 264 155 155
query27 2786 499 296 296
query28 4981 1840 1811 1811
query29 858 583 492 492
query30 323 236 194 194
query31 1031 940 878 878
query32 83 68 75 68
query33 520 346 284 284
query34 898 891 527 527
query35 655 686 615 615
query36 1154 1144 994 994
query37 135 97 83 83
query38 3012 2908 2875 2875
query39 856 828 807 807
query39_1 794 794 806 794
query40 269 157 136 136
query41 68 58 59 58
query42 259 252 257 252
query43 231 247 213 213
query44
query45 194 185 181 181
query46 887 977 606 606
query47 2151 2168 2104 2104
query48 304 311 225 225
query49 629 459 381 381
query50 682 279 210 210
query51 4039 4067 4089 4067
query52 262 264 256 256
query53 290 336 286 286
query54 297 275 288 275
query55 91 89 82 82
query56 310 322 317 317
query57 1928 1767 1635 1635
query58 293 276 269 269
query59 2777 2931 2743 2743
query60 344 344 329 329
query61 158 157 156 156
query62 634 594 545 545
query63 310 271 278 271
query64 5088 1285 1026 1026
query65
query66 1460 457 367 367
query67 24193 24284 24127 24127
query68
query69 403 317 279 279
query70 976 953 950 950
query71 340 299 301 299
query72 2836 2713 2440 2440
query73 537 534 316 316
query74 9667 9561 9398 9398
query75 2866 2768 2582 2582
query76 2308 1032 692 692
query77 355 372 307 307
query78 10916 11023 10477 10477
query79 3261 755 561 561
query80 1782 627 556 556
query81 573 267 221 221
query82 990 153 131 131
query83 336 262 244 244
query84 292 121 102 102
query85 934 519 458 458
query86 496 301 328 301
query87 3146 3131 2963 2963
query88 3966 2651 2635 2635
query89 423 365 338 338
query90 2123 178 177 177
query91 165 159 137 137
query92 91 77 71 71
query93 2827 800 488 488
query94 645 321 279 279
query95 581 398 316 316
query96 637 506 224 224
query97 2493 2482 2402 2402
query98 246 218 220 218
query99 1018 1012 885 885
Total cold run time: 255742 ms
Total hot run time: 168514 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](compaction) submit manual full compaction task to thread pool instead of detached thread [doris]
Posted by "Yukang-Lian (via GitHub)" <gi...@apache.org>.
Yukang-Lian commented on PR #61222:
URL: https://github.com/apache/doris/pull/61222#issuecomment-4196388846
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](compaction) submit manual full compaction task to thread pool instead of detached thread [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #61222:
URL: https://github.com/apache/doris/pull/61222#issuecomment-4264871751
run nonConcurrent
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](compaction) submit manual full compaction task to thread pool instead of detached thread [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #61222:
URL: https://github.com/apache/doris/pull/61222#issuecomment-4179123001
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](compaction) submit manual full compaction task to thread pool instead of detached thread [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #61222:
URL: https://github.com/apache/doris/pull/61222#issuecomment-4243082049
# BE Regression && UT Coverage Report
Increment line coverage `22.58% (7/31)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/61222_522bad517498a8a98ac14431ca93d0ec9446e9e2_merge/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/61222_522bad517498a8a98ac14431ca93d0ec9446e9e2_merge/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 73.61% (27369/37183) |
| Line Coverage | 57.28% (295638/516095) |
| Region Coverage | 54.30% (245322/451803) |
| Branch Coverage | 56.08% (106557/190002) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](compaction) submit manual full compaction task to thread pool instead of detached thread [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #61222:
URL: https://github.com/apache/doris/pull/61222#issuecomment-4048479930
# BE Regression && UT Coverage Report
Increment line coverage `100% (0/0)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/61222_7ff50ae2caf139d0337bed64698ba211c5ec102d_merge/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/61222_7ff50ae2caf139d0337bed64698ba211c5ec102d_merge/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 73.35% (26869/36631) |
| Line Coverage | 56.69% (287063/506373) |
| Region Coverage | 54.08% (239708/443225) |
| Branch Coverage | 55.71% (103395/185597) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](compaction) submit manual full compaction task to thread pool instead of detached thread [doris]
Posted by "Yukang-Lian (via GitHub)" <gi...@apache.org>.
Yukang-Lian commented on PR #61222:
URL: https://github.com/apache/doris/pull/61222#issuecomment-4053627591
run nonConcurrent
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](compaction) submit manual full compaction task to thread pool instead of detached thread [doris]
Posted by "Yukang-Lian (via GitHub)" <gi...@apache.org>.
Yukang-Lian commented on PR #61222:
URL: https://github.com/apache/doris/pull/61222#issuecomment-4046343752
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](compaction) submit manual full compaction task to thread pool instead of detached thread [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #61222:
URL: https://github.com/apache/doris/pull/61222#issuecomment-4066161313
<details>
<summary>TPC-DS: <b>Total hot run time: 168774 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 827471cc17d36d9670cc0d37b3131658451c19c8, data reload: false
query5 4321 635 511 511
query6 326 224 200 200
query7 4216 464 268 268
query8 372 249 239 239
query9 8696 2731 2760 2731
query10 537 387 359 359
query11 6946 5079 4895 4895
query12 185 129 124 124
query13 1271 454 348 348
query14 5731 3717 3453 3453
query14_1 2873 2799 2803 2799
query15 208 199 177 177
query16 986 463 446 446
query17 912 728 617 617
query18 2439 455 352 352
query19 218 211 190 190
query20 135 132 134 132
query21 216 136 114 114
query22 13220 13882 14926 13882
query23 16214 15968 15599 15599
query23_1 15762 15997 15877 15877
query24 7226 1615 1222 1222
query24_1 1225 1261 1196 1196
query25 554 447 397 397
query26 1240 254 149 149
query27 2800 469 291 291
query28 4459 1871 1926 1871
query29 830 556 469 469
query30 296 225 189 189
query31 1015 944 863 863
query32 77 67 65 65
query33 505 324 278 278
query34 884 879 531 531
query35 631 693 603 603
query36 1083 1141 975 975
query37 130 93 80 80
query38 2945 2982 2898 2898
query39 850 834 831 831
query39_1 804 811 802 802
query40 232 157 134 134
query41 61 61 62 61
query42 266 260 254 254
query43 251 240 230 230
query44
query45 198 186 181 181
query46 871 971 594 594
query47 2094 2536 2522 2522
query48 323 316 231 231
query49 629 455 386 386
query50 681 272 210 210
query51 4027 4085 4030 4030
query52 264 265 256 256
query53 295 332 277 277
query54 307 274 256 256
query55 93 89 89 89
query56 319 335 299 299
query57 1915 1893 1637 1637
query58 282 278 269 269
query59 2776 2947 2749 2749
query60 348 344 323 323
query61 152 140 147 140
query62 617 585 540 540
query63 315 284 273 273
query64 5038 1272 987 987
query65
query66 1471 451 349 349
query67 24232 24288 24248 24248
query68
query69 422 318 294 294
query70 965 957 936 936
query71 334 334 298 298
query72 2810 2833 2395 2395
query73 559 546 320 320
query74 9632 9564 9419 9419
query75 2833 2754 2456 2456
query76 2289 1018 678 678
query77 356 371 307 307
query78 10965 11139 10478 10478
query79 1112 847 571 571
query80 1308 617 569 569
query81 561 265 227 227
query82 1023 149 119 119
query83 349 277 260 260
query84 296 122 111 111
query85 886 490 436 436
query86 423 322 296 296
query87 3170 3115 3038 3038
query88 3534 2667 2665 2665
query89 422 377 343 343
query90 2022 176 175 175
query91 168 159 136 136
query92 81 68 67 67
query93 924 869 494 494
query94 647 322 278 278
query95 588 421 316 316
query96 648 506 236 236
query97 2459 2457 2391 2391
query98 231 231 224 224
query99 1017 980 911 911
Total cold run time: 248886 ms
Total hot run time: 168774 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](compaction) submit manual full compaction task to thread pool instead of detached thread [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #61222:
URL: https://github.com/apache/doris/pull/61222#issuecomment-4066107148
<details>
<summary>TPC-H: <b>Total hot run time: 27084 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 827471cc17d36d9670cc0d37b3131658451c19c8, data reload: false
------ Round 1 ----------------------------------
orders Doris NULL NULL 0 0 0 NULL 0 NULL NULL 2023-12-26 18:27:23 2023-12-26 18:42:55 NULL utf-8 NULL NULL
============================================
q1 17614 4486 4350 4350
q2 q3 10648 846 529 529
q4 4672 353 252 252
q5 7648 1228 1031 1031
q6 174 172 145 145
q7 820 850 672 672
q8 9804 1475 1345 1345
q9 5566 4788 4728 4728
q10 6328 1966 1686 1686
q11 470 247 236 236
q12 751 590 464 464
q13 18050 2936 2193 2193
q14 227 233 216 216
q15 q16 741 750 667 667
q17 728 877 401 401
q18 6039 5335 5255 5255
q19 1286 996 610 610
q20 537 507 373 373
q21 4569 1890 1642 1642
q22 398 357 289 289
Total cold run time: 97070 ms
Total hot run time: 27084 ms
----- Round 2, with runtime_filter_mode=off -----
orders Doris NULL NULL 150000000 42 6422171781 NULL 22778155 NULL NULL 2023-12-26 18:27:23 2023-12-26 18:42:55 NULL utf-8 NULL NULL
============================================
q1 4721 4540 4646 4540
q2 q3 3909 4338 3859 3859
q4 880 1206 803 803
q5 4074 4467 4343 4343
q6 183 174 142 142
q7 1746 1739 1585 1585
q8 2612 2694 2581 2581
q9 7533 7537 7199 7199
q10 3760 3959 3613 3613
q11 532 443 424 424
q12 510 600 452 452
q13 2665 3665 2379 2379
q14 288 309 283 283
q15 q16 744 878 812 812
q17 1189 1399 1347 1347
q18 7340 7137 6695 6695
q19 898 882 865 865
q20 2026 2141 1992 1992
q21 3988 3476 3291 3291
q22 461 421 410 410
Total cold run time: 50059 ms
Total hot run time: 47615 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](compaction) submit manual full compaction task to thread pool instead of detached thread [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #61222:
URL: https://github.com/apache/doris/pull/61222#issuecomment-4038526227
# BE Regression && UT Coverage Report
Increment line coverage `0.00% (0/23)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/61222_4420035c2c3afc07f46a1e0473141101f9fffae5_merge/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/61222_4420035c2c3afc07f46a1e0473141101f9fffae5_merge/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 71.62% (26219/36606) |
| Line Coverage | 54.40% (274792/505101) |
| Region Coverage | 51.64% (228349/442209) |
| Branch Coverage | 53.11% (98292/185079) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](compaction) submit manual full compaction task to thread pool instead of detached thread [doris]
Posted by "Yukang-Lian (via GitHub)" <gi...@apache.org>.
Yukang-Lian commented on PR #61222:
URL: https://github.com/apache/doris/pull/61222#issuecomment-4257206135
/review
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](compaction) submit manual full compaction task to thread pool instead of detached thread [doris]
Posted by "Yukang-Lian (via GitHub)" <gi...@apache.org>.
Yukang-Lian commented on PR #61222:
URL: https://github.com/apache/doris/pull/61222#issuecomment-4264836974
run nonconcurrent
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](compaction) submit manual full compaction task to thread pool instead of detached thread [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #61222:
URL: https://github.com/apache/doris/pull/61222#issuecomment-4265312046
# BE Regression && UT Coverage Report
Increment line coverage `22.58% (7/31)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/61222_522bad517498a8a98ac14431ca93d0ec9446e9e2_merge/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/61222_522bad517498a8a98ac14431ca93d0ec9446e9e2_merge/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 73.60% (27365/37183) |
| Line Coverage | 57.27% (295552/516095) |
| Region Coverage | 54.28% (245235/451803) |
| Branch Coverage | 56.06% (106517/190002) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](compaction) submit manual full compaction task to thread pool instead of detached thread [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on code in PR #61222:
URL: https://github.com/apache/doris/pull/61222#discussion_r3031337906
##########
be/src/service/http/action/compaction_action.cpp:
##########
@@ -177,33 +185,36 @@ Status CompactionAction::_handle_run_compaction(HttpRequest* req, std::string* j
return Status::OK();
})
- // 3. execute compaction task
- std::packaged_task<Status()> task([this, tablet, compaction_type, fetch_from_remote]() {
- return _execute_compaction_callback(tablet, compaction_type, fetch_from_remote);
- });
- std::future<Status> future_obj = task.get_future();
- std::thread(std::move(task)).detach();
-
- // 更新schedule_time
- if (compaction_type == PARAM_COMPACTION_BASE) {
- tablet->set_last_base_compaction_schedule_time(UnixMillis());
- } else if (compaction_type == PARAM_COMPACTION_CUMULATIVE) {
- tablet->set_last_cumu_compaction_schedule_time(UnixMillis());
- } else if (compaction_type == PARAM_COMPACTION_FULL) {
+ if (compaction_type == PARAM_COMPACTION_FULL) {
+ // 3. submit full compaction task to thread pool
tablet->set_last_full_compaction_schedule_time(UnixMillis());
- }
+ RETURN_IF_ERROR(
+ _engine.submit_compaction_task(tablet, CompactionType::FULL_COMPACTION, force));
+ } else {
+ // 3. execute base/cumulative compaction task in a detached thread
+ std::packaged_task<Status()> task([this, tablet, compaction_type, fetch_from_remote]() {
+ return _execute_compaction_callback(tablet, compaction_type, fetch_from_remote);
+ });
+ std::future<Status> future_obj = task.get_future();
Review Comment:
Switching single-tablet `full` compaction to `submit_compaction_task()` adds a queued state here, but `_handle_run_status_compaction()` still treats `full` as running only when `tablet->is_full_compaction_running()` is true. That means a request can return success from this line, sit in the base-compaction pool, and still report `run_status = false` to any caller polling `/api/compaction/run_status`. The new Groovy workaround masks that for one helper, but the HTTP API itself is now inconsistent. Can we expose pending/submitted full compactions in `run_status` before moving this path onto the queue?
##########
regression-test/plugins/plugin_compaction.groovy:
##########
@@ -137,8 +137,10 @@ Suite.metaClass.trigger_and_wait_compaction = { String table_name, String compac
assert exit_code == 0: "get tablet status failed, exit code: ${exit_code}, stdout: ${stdout}, stderr: ${stderr}"
def tabletStatus = parseJson(stdout.trim())
def oldStatus = be_tablet_compaction_status.get("${be_host}-${tablet.TabletId}")
- // last compaction success time isn't updated, indicates compaction is not started(so we treat it as running and wait)
- running = running || (oldStatus["last ${compaction_type} success time"] == tabletStatus["last ${compaction_type} success time"])
+ // last compaction success/failure time isn't updated, indicates compaction is not started(so we treat it as running and wait)
+ def success_time_unchanged = (oldStatus["last ${compaction_type} success time"] == tabletStatus["last ${compaction_type} success time"])
+ def failure_time_unchanged = (oldStatus["last ${compaction_type} failure time"] == tabletStatus["last ${compaction_type} failure time"])
+ running = running || (success_time_unchanged && failure_time_unchanged)
Review Comment:
This condition is too broad for `base`/`cumulative`. Those paths can change `last ... failure time` without ever completing the requested compaction. For example, cumulative tasks can be delayed/aborted in `StorageEngine::_handle_compaction()` after updating `last_cumu_compaction_failure_time`, with `run_status` already false. In that case `failure_time_unchanged` becomes false and this helper stops waiting even though no compaction actually ran. The timestamp workaround seems safe only for queued `full` compaction; for the other modes we still need the old behavior or an explicit pending-state signal from BE.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](compaction) submit manual full compaction task to thread pool instead of detached thread [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #61222:
URL: https://github.com/apache/doris/pull/61222#issuecomment-4123129934
<details>
<summary>TPC-DS: <b>Total hot run time: 169787 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit ab484dc10b44aeee407b98ac3a0922e35e1001b5, data reload: false
query5 4348 641 531 531
query6 355 258 216 216
query7 4215 464 273 273
query8 348 251 239 239
query9 8695 2765 2764 2764
query10 537 393 334 334
query11 6994 5123 4893 4893
query12 187 134 125 125
query13 1283 478 352 352
query14 5796 3718 3483 3483
query14_1 2891 2815 2862 2815
query15 214 192 174 174
query16 994 445 469 445
query17 1072 756 646 646
query18 2453 460 356 356
query19 224 218 189 189
query20 143 130 129 129
query21 213 139 116 116
query22 13211 14130 14973 14130
query23 16838 16490 16123 16123
query23_1 15907 15824 15780 15780
query24 7299 1619 1233 1233
query24_1 1268 1242 1252 1242
query25 631 477 424 424
query26 1235 272 146 146
query27 2771 479 293 293
query28 4470 1849 1847 1847
query29 856 566 475 475
query30 296 234 194 194
query31 1033 949 883 883
query32 87 73 69 69
query33 513 335 287 287
query34 896 894 523 523
query35 639 671 616 616
query36 1084 1111 1032 1032
query37 146 95 81 81
query38 2979 2939 2880 2880
query39 858 824 795 795
query39_1 788 800 795 795
query40 229 155 139 139
query41 65 109 74 74
query42 267 258 259 258
query43 254 254 219 219
query44
query45 202 197 184 184
query46 862 978 621 621
query47 2124 2139 2061 2061
query48 337 317 233 233
query49 638 466 381 381
query50 685 289 209 209
query51 4108 4123 4049 4049
query52 261 272 262 262
query53 287 333 284 284
query54 299 274 267 267
query55 90 87 90 87
query56 318 317 319 317
query57 1939 1643 1677 1643
query58 289 280 274 274
query59 2834 2917 2755 2755
query60 354 350 327 327
query61 161 161 161 161
query62 630 585 542 542
query63 308 278 271 271
query64 5063 1300 1013 1013
query65
query66 1473 449 380 380
query67 24324 24377 24268 24268
query68
query69 411 323 298 298
query70 975 962 936 936
query71 342 316 307 307
query72 2949 2776 2508 2508
query73 537 542 314 314
query74 9627 9613 9484 9484
query75 2954 2777 2455 2455
query76 2296 1058 671 671
query77 376 390 312 312
query78 10941 11098 10531 10531
query79 1100 760 579 579
query80 693 618 546 546
query81 479 259 220 220
query82 1332 153 122 122
query83 372 273 242 242
query84 268 121 105 105
query85 861 505 479 479
query86 375 303 301 301
query87 3160 3175 3014 3014
query88 3589 2686 2677 2677
query89 430 375 347 347
query90 1973 189 182 182
query91 178 170 144 144
query92 85 78 68 68
query93 909 869 494 494
query94 455 332 294 294
query95 611 347 325 325
query96 648 521 231 231
query97 2463 2541 2412 2412
query98 240 233 221 221
query99 1037 1011 921 921
Total cold run time: 250011 ms
Total hot run time: 169787 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](compaction) submit manual full compaction task to thread pool instead of detached thread [doris]
Posted by "Yukang-Lian (via GitHub)" <gi...@apache.org>.
Yukang-Lian commented on PR #61222:
URL: https://github.com/apache/doris/pull/61222#issuecomment-4242114118
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](compaction) submit manual full compaction task to thread pool instead of detached thread [doris]
Posted by "Yukang-Lian (via GitHub)" <gi...@apache.org>.
Yukang-Lian commented on PR #61222:
URL: https://github.com/apache/doris/pull/61222#issuecomment-4037323345
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](compaction) submit manual full compaction task to thread pool instead of detached thread [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #61222:
URL: https://github.com/apache/doris/pull/61222#issuecomment-4054289505
# BE Regression && UT Coverage Report
Increment line coverage `100% (0/0)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/61222_7ff50ae2caf139d0337bed64698ba211c5ec102d_merge/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/61222_7ff50ae2caf139d0337bed64698ba211c5ec102d_merge/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 73.35% (26870/36631) |
| Line Coverage | 56.70% (287103/506373) |
| Region Coverage | 54.12% (239861/443225) |
| Branch Coverage | 55.72% (103415/185597) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](compaction) submit manual full compaction task to thread pool instead of detached thread [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #61222:
URL: https://github.com/apache/doris/pull/61222#issuecomment-4066910604
# BE UT Coverage Report
Increment line coverage ` ` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/827471cc17d36d9670cc0d37b3131658451c19c8_827471cc17d36d9670cc0d37b3131658451c19c8/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/827471cc17d36d9670cc0d37b3131658451c19c8_827471cc17d36d9670cc0d37b3131658451c19c8/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 52.68% (19718/37428) |
| Line Coverage | 36.26% (184352/508437) |
| Region Coverage | 32.44% (142295/438657) |
| Branch Coverage | 33.59% (62180/185090) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](compaction) submit manual full compaction task to thread pool instead of detached thread [doris]
Posted by "Yukang-Lian (via GitHub)" <gi...@apache.org>.
Yukang-Lian commented on PR #61222:
URL: https://github.com/apache/doris/pull/61222#issuecomment-4108440695
run external
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](compaction) submit manual full compaction task to thread pool instead of detached thread [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #61222:
URL: https://github.com/apache/doris/pull/61222#issuecomment-4123023005
<details>
<summary>TPC-H: <b>Total hot run time: 27120 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit ab484dc10b44aeee407b98ac3a0922e35e1001b5, data reload: false
------ Round 1 ----------------------------------
orders Doris NULL NULL 0 0 0 NULL 0 NULL NULL 2023-12-26 18:27:23 2023-12-26 18:42:55 NULL utf-8 NULL NULL
============================================
q1 17616 4470 4342 4342
q2 q3 10632 833 550 550
q4 4681 368 262 262
q5 7577 1249 1040 1040
q6 181 178 149 149
q7 782 861 688 688
q8 9301 1496 1390 1390
q9 4881 4783 4751 4751
q10 6276 1940 1655 1655
q11 452 252 254 252
q12 732 585 465 465
q13 18047 2732 1958 1958
q14 234 230 213 213
q15 q16 751 732 676 676
q17 744 849 456 456
q18 6279 5547 5390 5390
q19 1114 1000 635 635
q20 561 514 386 386
q21 4357 1843 1541 1541
q22 496 381 321 321
Total cold run time: 95694 ms
Total hot run time: 27120 ms
----- Round 2, with runtime_filter_mode=off -----
orders Doris NULL NULL 150000000 42 6422171781 NULL 22778155 NULL NULL 2023-12-26 18:27:23 2023-12-26 18:42:55 NULL utf-8 NULL NULL
============================================
q1 4931 4683 4637 4637
q2 q3 3969 4368 3882 3882
q4 909 1198 772 772
q5 4139 4477 4385 4385
q6 187 182 143 143
q7 1819 1705 1563 1563
q8 2504 3133 2774 2774
q9 8264 7563 7548 7548
q10 3799 4001 3650 3650
q11 506 437 417 417
q12 488 593 451 451
q13 2555 2992 2065 2065
q14 287 293 280 280
q15 q16 722 762 709 709
q17 1183 1384 1436 1384
q18 7235 6911 6753 6753
q19 1033 1002 947 947
q20 2157 2203 2059 2059
q21 4013 3547 3356 3356
q22 478 459 483 459
Total cold run time: 51178 ms
Total hot run time: 48234 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](compaction) submit manual full compaction task to thread pool instead of detached thread [doris]
Posted by "Yukang-Lian (via GitHub)" <gi...@apache.org>.
Yukang-Lian commented on PR #61222:
URL: https://github.com/apache/doris/pull/61222#issuecomment-4212324380
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](compaction) submit manual full compaction task to thread pool instead of detached thread [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #61222:
URL: https://github.com/apache/doris/pull/61222#issuecomment-4213364319
# BE Regression && UT Coverage Report
Increment line coverage `12.90% (4/31)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/61222_4de9c7cf86ca60835f9b9c5d1d9e7000d9a29da2_merge/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/61222_4de9c7cf86ca60835f9b9c5d1d9e7000d9a29da2_merge/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 71.95% (26757/37186) |
| Line Coverage | 54.94% (283522/516078) |
| Region Coverage | 51.91% (234495/451762) |
| Branch Coverage | 53.38% (101451/190056) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](compaction) submit manual full compaction task to thread pool instead of detached thread [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #61222:
URL: https://github.com/apache/doris/pull/61222#issuecomment-4265977210
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](compaction) submit manual full compaction task to thread pool instead of detached thread [doris]
Posted by "Yukang-Lian (via GitHub)" <gi...@apache.org>.
Yukang-Lian commented on PR #61222:
URL: https://github.com/apache/doris/pull/61222#issuecomment-4037456991
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](compaction) submit manual full compaction task to thread pool instead of detached thread [doris]
Posted by "Yukang-Lian (via GitHub)" <gi...@apache.org>.
Yukang-Lian commented on PR #61222:
URL: https://github.com/apache/doris/pull/61222#issuecomment-4053231524
run nonConcurrent
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](compaction) submit manual full compaction task to thread pool instead of detached thread [doris]
Posted by "Yukang-Lian (via GitHub)" <gi...@apache.org>.
Yukang-Lian commented on PR #61222:
URL: https://github.com/apache/doris/pull/61222#issuecomment-4053627223
run p0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](compaction) submit manual full compaction task to thread pool instead of detached thread [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #61222:
URL: https://github.com/apache/doris/pull/61222#issuecomment-4048210380
# BE UT Coverage Report
Increment line coverage ` ` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/7ff50ae2caf139d0337bed64698ba211c5ec102d_7ff50ae2caf139d0337bed64698ba211c5ec102d/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/7ff50ae2caf139d0337bed64698ba211c5ec102d_7ff50ae2caf139d0337bed64698ba211c5ec102d/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 52.70% (19722/37426) |
| Line Coverage | 36.24% (184131/508095) |
| Region Coverage | 32.34% (142037/439177) |
| Branch Coverage | 33.50% (62021/185115) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](compaction) submit manual full compaction task to thread pool instead of detached thread [doris]
Posted by "Yukang-Lian (via GitHub)" <gi...@apache.org>.
Yukang-Lian commented on PR #61222:
URL: https://github.com/apache/doris/pull/61222#issuecomment-4108441072
run nonConcurrent
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](compaction) submit manual full compaction task to thread pool instead of detached thread [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #61222:
URL: https://github.com/apache/doris/pull/61222#issuecomment-4124186147
# BE Regression && UT Coverage Report
Increment line coverage `100% (0/0)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/61222_ab484dc10b44aeee407b98ac3a0922e35e1001b5_merge/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/61222_ab484dc10b44aeee407b98ac3a0922e35e1001b5_merge/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 57.29% (21104/36836) |
| Line Coverage | 40.37% (205717/509534) |
| Region Coverage | 37.16% (165578/445580) |
| Branch Coverage | 37.97% (70979/186933) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](compaction) submit manual full compaction task to thread pool instead of detached thread [doris]
Posted by "Yukang-Lian (via GitHub)" <gi...@apache.org>.
Yukang-Lian commented on PR #61222:
URL: https://github.com/apache/doris/pull/61222#issuecomment-4124310715
run nonConcurrent
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](compaction) submit manual full compaction task to thread pool instead of detached thread [doris]
Posted by "Yukang-Lian (via GitHub)" <gi...@apache.org>.
Yukang-Lian commented on PR #61222:
URL: https://github.com/apache/doris/pull/61222#issuecomment-4073836570
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](compaction) submit manual full compaction task to thread pool instead of detached thread [doris]
Posted by "Yukang-Lian (via GitHub)" <gi...@apache.org>.
Yukang-Lian commented on PR #61222:
URL: https://github.com/apache/doris/pull/61222#issuecomment-4257205350
run nonconcurrent
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](compaction) submit manual full compaction task to thread pool instead of detached thread [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on code in PR #61222:
URL: https://github.com/apache/doris/pull/61222#discussion_r3090684397
##########
be/src/service/http/action/compaction_action.cpp:
##########
@@ -178,33 +186,36 @@ Status CompactionAction::_handle_run_compaction(HttpRequest* req, std::string* j
return Status::OK();
})
- // 3. execute compaction task
- std::packaged_task<Status()> task([this, tablet, compaction_type, fetch_from_remote]() {
- return _execute_compaction_callback(tablet, compaction_type, fetch_from_remote);
- });
- std::future<Status> future_obj = task.get_future();
- std::thread(std::move(task)).detach();
-
- // 更新schedule_time
- if (compaction_type == PARAM_COMPACTION_BASE) {
- tablet->set_last_base_compaction_schedule_time(UnixMillis());
- } else if (compaction_type == PARAM_COMPACTION_CUMULATIVE) {
- tablet->set_last_cumu_compaction_schedule_time(UnixMillis());
- } else if (compaction_type == PARAM_COMPACTION_FULL) {
+ if (compaction_type == PARAM_COMPACTION_FULL) {
+ // 3. submit full compaction task to thread pool
Review Comment:
This changes the single-tablet `full` path from "spawn a detached worker and return after `wait_for(2s)`" to a synchronous `submit_compaction_task()` call. On the local engine that helper immediately runs `Tablet::prepare_compaction_and_calculate_permits()`, and `FullCompaction::prepare_compact()` takes the base/cumulative compaction locks with blocking `std::unique_lock`s. If a base or cumulative compaction is already running on this tablet, the POST handler now waits for that compaction to finish before replying, which is a user-visible regression for `/api/compaction/run` and can tie up an HTTP worker for minutes. Can we keep full-compaction `prepare_compact()` off the request thread (or fail fast when those locks are busy) so the API stays asynchronous like before?
##########
be/src/service/http/action/compaction_action.cpp:
##########
@@ -178,33 +186,36 @@ Status CompactionAction::_handle_run_compaction(HttpRequest* req, std::string* j
return Status::OK();
})
- // 3. execute compaction task
- std::packaged_task<Status()> task([this, tablet, compaction_type, fetch_from_remote]() {
- return _execute_compaction_callback(tablet, compaction_type, fetch_from_remote);
- });
- std::future<Status> future_obj = task.get_future();
- std::thread(std::move(task)).detach();
-
- // 更新schedule_time
- if (compaction_type == PARAM_COMPACTION_BASE) {
- tablet->set_last_base_compaction_schedule_time(UnixMillis());
- } else if (compaction_type == PARAM_COMPACTION_CUMULATIVE) {
- tablet->set_last_cumu_compaction_schedule_time(UnixMillis());
- } else if (compaction_type == PARAM_COMPACTION_FULL) {
+ if (compaction_type == PARAM_COMPACTION_FULL) {
+ // 3. submit full compaction task to thread pool
tablet->set_last_full_compaction_schedule_time(UnixMillis());
- }
+ RETURN_IF_ERROR(_engine.submit_compaction_task(tablet, CompactionType::FULL_COMPACTION,
Review Comment:
This new call site now inherits a latent bug in `StorageEngine::_submit_compaction_task()`: after `auto status = thread_pool->submit_func(...)`, the code still checks `if (!st.ok())`, where `st` is the earlier prepare status and is already `OK`. So if enqueueing fails (for example because the pool is shutting down, or no worker thread can be created), the error is dropped and this request still returns success while leaving the registry / permit / task-tracker state behind. Single-tablet `full` compaction did not depend on that helper before this PR, so this becomes user-visible here. Please fix the submit-status check before routing this path through the thread pool.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](compaction) submit manual full compaction task to thread pool instead of detached thread [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #61222:
URL: https://github.com/apache/doris/pull/61222#issuecomment-4197018340
# BE Regression && UT Coverage Report
Increment line coverage `12.90% (4/31)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/61222_11617583eee95d30b13e1425acb9cec01e156d28_merge/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/61222_11617583eee95d30b13e1425acb9cec01e156d28_merge/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 73.52% (27320/37158) |
| Line Coverage | 57.19% (294836/515513) |
| Region Coverage | 54.51% (245884/451072) |
| Branch Coverage | 56.15% (106559/189777) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](compaction) submit manual full compaction task to thread pool instead of detached thread [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #61222:
URL: https://github.com/apache/doris/pull/61222#issuecomment-4242594203
# BE UT Coverage Report
Increment line coverage `22.58% (7/31)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/522bad517498a8a98ac14431ca93d0ec9446e9e2_522bad517498a8a98ac14431ca93d0ec9446e9e2/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/522bad517498a8a98ac14431ca93d0ec9446e9e2_522bad517498a8a98ac14431ca93d0ec9446e9e2/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 53.06% (20146/37969) |
| Line Coverage | 36.61% (189545/517679) |
| Region Coverage | 32.85% (147062/447671) |
| Branch Coverage | 34.00% (64404/189420) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](compaction) submit manual full compaction task to thread pool instead of detached thread [doris]
Posted by "Hastyshell (via GitHub)" <gi...@apache.org>.
Hastyshell commented on PR #61222:
URL: https://github.com/apache/doris/pull/61222#issuecomment-4181708182
/review
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org