You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "gavinchou (via GitHub)" <gi...@apache.org> on 2025/05/23 09:35:00 UTC
[PR] [fix](cloud) Cloud tablet cache should not be pruned [doris]
gavinchou opened a new pull request, #51193:
URL: https://github.com/apache/doris/pull/51193
Otherwise there may be severe performance issue for query associated to many tablets
Related PR: #28940
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](cloud) Cloud tablet cache should not be pruned [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #51193:
URL: https://github.com/apache/doris/pull/51193#issuecomment-2904206955
<details>
<summary>TPC-DS: <b>Total hot run time: 192994 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 6cd2b50570a936302831f4687f6d0dbe13cae3f1, data reload: false
query1 1405 1103 1058 1058
query2 6219 1841 1822 1822
query3 11032 4441 4420 4420
query4 54152 24178 23114 23114
query5 5206 622 455 455
query6 356 206 198 198
query7 4882 511 296 296
query8 322 260 246 246
query9 5492 2665 2663 2663
query10 442 344 289 289
query11 15126 15006 14814 14814
query12 175 108 114 108
query13 1046 545 419 419
query14 10201 6521 6568 6521
query15 218 202 192 192
query16 7049 665 490 490
query17 1083 733 619 619
query18 1602 403 308 308
query19 196 203 170 170
query20 135 133 122 122
query21 206 129 109 109
query22 4411 4504 4389 4389
query23 34328 33570 33479 33479
query24 6592 2402 2439 2402
query25 466 467 423 423
query26 698 273 159 159
query27 2333 523 355 355
query28 3071 2204 2173 2173
query29 588 566 458 458
query30 297 216 189 189
query31 861 841 779 779
query32 74 65 64 64
query33 463 365 316 316
query34 780 872 551 551
query35 815 845 766 766
query36 960 1004 892 892
query37 134 96 83 83
query38 4208 4330 4358 4330
query39 1526 1428 1486 1428
query40 218 119 106 106
query41 60 56 52 52
query42 128 112 117 112
query43 511 532 488 488
query44 1373 873 864 864
query45 194 176 172 172
query46 862 1043 650 650
query47 1826 1851 1821 1821
query48 398 443 336 336
query49 705 538 446 446
query50 687 690 427 427
query51 4280 4253 4266 4253
query52 109 108 101 101
query53 230 257 191 191
query54 609 579 533 533
query55 91 92 81 81
query56 322 313 298 298
query57 1173 1220 1134 1134
query58 278 265 274 265
query59 2646 2904 2657 2657
query60 329 326 324 324
query61 131 148 182 148
query62 740 792 664 664
query63 227 200 193 193
query64 1497 1060 693 693
query65 4320 4290 4228 4228
query66 731 398 300 300
query67 16463 15930 15486 15486
query68 6818 897 529 529
query69 557 313 275 275
query70 1199 1095 1113 1095
query71 502 332 300 300
query72 6054 4789 4604 4604
query73 1483 640 359 359
query74 8993 9223 8914 8914
query75 4071 3217 2704 2704
query76 4217 1187 767 767
query77 754 366 281 281
query78 10041 10033 9330 9330
query79 2407 826 577 577
query80 642 519 435 435
query81 472 254 220 220
query82 416 126 95 95
query83 307 245 242 242
query84 293 107 85 85
query85 802 358 315 315
query86 353 308 283 283
query87 4367 4544 4316 4316
query88 3404 2295 2288 2288
query89 403 319 292 292
query90 1959 204 204 204
query91 144 148 115 115
query92 77 61 55 55
query93 1157 931 587 587
query94 676 390 311 311
query95 362 288 288 288
query96 506 574 278 278
query97 2715 2768 2688 2688
query98 238 216 201 201
query99 1424 1383 1269 1269
Total cold run time: 298101 ms
Total hot run time: 192994 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](cloud) Cloud tablet cache should not be pruned [doris]
Posted by "dataroaring (via GitHub)" <gi...@apache.org>.
dataroaring merged PR #51193:
URL: https://github.com/apache/doris/pull/51193
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](cloud) Cloud tablet cache should not be pruned [doris]
Posted by "Thearas (via GitHub)" <gi...@apache.org>.
Thearas commented on PR #51193:
URL: https://github.com/apache/doris/pull/51193#issuecomment-2903857974
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR).
Please clearly describe your PR:
1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
3. What features were added. Why was this function added?
4. Which code was refactored and why was this part of the code refactored?
5. Which functions were optimized and what is the difference before and after the optimization?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](cloud) Cloud tablet cache should not be pruned [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #51193:
URL: https://github.com/apache/doris/pull/51193#issuecomment-2906323125
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](cloud) Cloud tablet cache should not be pruned [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #51193:
URL: https://github.com/apache/doris/pull/51193#issuecomment-2906323152
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](cloud) Cloud tablet cache should not be pruned [doris]
Posted by "gavinchou (via GitHub)" <gi...@apache.org>.
gavinchou commented on PR #51193:
URL: https://github.com/apache/doris/pull/51193#issuecomment-2903858757
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](cloud) Cloud tablet cache should not be pruned [doris]
Posted by "gavinchou (via GitHub)" <gi...@apache.org>.
gavinchou commented on PR #51193:
URL: https://github.com/apache/doris/pull/51193#issuecomment-2903906376
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](cloud) Cloud tablet cache should not be pruned [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #51193:
URL: https://github.com/apache/doris/pull/51193#issuecomment-2904954154
# BE Regression && UT Coverage Report
Increment line coverage `100.00% (1/1)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/51193_6cd2b50570a936302831f4687f6d0dbe13cae3f1_merge/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/51193_6cd2b50570a936302831f4687f6d0dbe13cae3f1_merge/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 79.43% (20832/26226) |
| Line Coverage | 72.62% (215234/296368) |
| Region Coverage | 70.82% (126497/178618) |
| Branch Coverage | 64.56% (65567/101566) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](cloud) Cloud tablet cache should not be pruned [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #51193:
URL: https://github.com/apache/doris/pull/51193#issuecomment-2904167938
# BE UT Coverage Report
Increment line coverage `100.00% (1/1)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/6cd2b50570a936302831f4687f6d0dbe13cae3f1_6cd2b50570a936302831f4687f6d0dbe13cae3f1/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/6cd2b50570a936302831f4687f6d0dbe13cae3f1_6cd2b50570a936302831f4687f6d0dbe13cae3f1/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 56.00% (14920/26643) |
| Line Coverage | 44.82% (132839/296381) |
| Region Coverage | 43.90% (66771/152112) |
| Branch Coverage | 38.52% (34263/88942) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](cloud) Cloud tablet cache should not be pruned [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #51193:
URL: https://github.com/apache/doris/pull/51193#issuecomment-2904178528
<details>
<summary>TPC-H: <b>Total hot run time: 33918 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 6cd2b50570a936302831f4687f6d0dbe13cae3f1, data reload: false
------ Round 1 ----------------------------------
q1 26430 5106 5061 5061
q2 2078 286 194 194
q3 10497 1277 717 717
q4 10231 999 540 540
q5 7660 2349 2310 2310
q6 189 164 132 132
q7 921 728 602 602
q8 9298 1303 1100 1100
q9 6796 5033 5117 5033
q10 6876 2308 1893 1893
q11 503 298 280 280
q12 344 350 209 209
q13 17810 3727 3096 3096
q14 232 228 222 222
q15 526 490 474 474
q16 432 435 381 381
q17 640 866 387 387
q18 7629 7211 7179 7179
q19 1977 984 558 558
q20 333 346 224 224
q21 3959 3267 2365 2365
q22 1028 987 961 961
Total cold run time: 116389 ms
Total hot run time: 33918 ms
----- Round 2, with runtime_filter_mode=off -----
q1 5238 5124 5164 5124
q2 242 329 237 237
q3 2214 2642 2290 2290
q4 1419 1779 1417 1417
q5 4450 4399 4418 4399
q6 226 170 126 126
q7 2056 1964 1744 1744
q8 2654 2658 2565 2565
q9 7209 7225 6911 6911
q10 3056 3195 2813 2813
q11 621 519 490 490
q12 721 779 604 604
q13 3526 3882 3282 3282
q14 277 311 272 272
q15 523 486 493 486
q16 449 482 459 459
q17 1161 1591 1376 1376
q18 7707 7598 7429 7429
q19 837 863 883 863
q20 1969 2107 1871 1871
q21 4900 4504 4493 4493
q22 1083 1064 1016 1016
Total cold run time: 52538 ms
Total hot run time: 50267 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](cloud) Cloud tablet cache should not be pruned [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #51193:
URL: https://github.com/apache/doris/pull/51193#issuecomment-2904219341
<details>
<summary>ClickBench: <b>Total hot run time: 29.11 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 6cd2b50570a936302831f4687f6d0dbe13cae3f1, data reload: false
query1 0.04 0.04 0.03
query2 0.13 0.11 0.12
query3 0.27 0.19 0.20
query4 1.59 0.20 0.21
query5 0.45 0.44 0.44
query6 1.16 0.68 0.67
query7 0.03 0.02 0.01
query8 0.05 0.04 0.04
query9 0.63 0.54 0.53
query10 0.59 0.60 0.59
query11 0.17 0.11 0.12
query12 0.15 0.12 0.12
query13 0.61 0.59 0.60
query14 0.78 0.81 0.80
query15 0.89 0.85 0.88
query16 0.39 0.39 0.38
query17 1.04 1.04 1.03
query18 0.22 0.21 0.21
query19 1.96 1.80 1.79
query20 0.01 0.01 0.01
query21 15.39 0.88 0.53
query22 0.75 1.15 0.72
query23 14.90 1.37 0.63
query24 7.56 1.40 0.70
query25 0.51 0.07 0.09
query26 0.68 0.16 0.14
query27 0.06 0.05 0.05
query28 9.43 0.89 0.44
query29 12.55 4.09 3.37
query30 0.25 0.09 0.06
query31 2.82 0.61 0.40
query32 3.23 0.55 0.48
query33 3.10 3.09 3.13
query34 15.75 5.06 4.48
query35 4.52 4.53 4.44
query36 0.68 0.50 0.49
query37 0.09 0.06 0.06
query38 0.05 0.05 0.04
query39 0.03 0.02 0.03
query40 0.16 0.13 0.13
query41 0.08 0.03 0.02
query42 0.04 0.02 0.02
query43 0.03 0.04 0.03
Total cold run time: 103.82 s
Total hot run time: 29.11 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org