You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "shuke987 (via GitHub)" <gi...@apache.org> on 2026/03/03 12:22:51 UTC
[PR] [fix](cache) Invalidate ExternalRowCountCache entries when catalog is… [doris]
shuke987 opened a new pull request, #60999:
URL: https://github.com/apache/doris/pull/60999
… dropped
When an external catalog is dropped, ExternalRowCountCache entries for tables in that catalog were not invalidated. This caused Caffeine's refreshAfterWrite to endlessly retry loading row counts for non-existent tables, producing continuous WARN logs and wasting refresh thread pool resources.
Add invalidateCatalog() to ExternalRowCountCache and call it from ExternalMetaCacheMgr.removeCache() and invalidateCatalogCache().
### What problem does this PR solve?
Issue Number: close #xxx
Related PR: #xxx
Problem Summary:
### Release note
None
### Check List (For Author)
- Test <!-- At least one of them must be included. -->
- [ ] Regression test
- [ ] Unit Test
- [ ] Manual test (add detailed scripts or steps below)
- [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
- [ ] Previous test can cover this change.
- [ ] No code files have been changed.
- [ ] Other reason <!-- Add your reason? -->
- Behavior changed:
- [ ] No.
- [ ] Yes. <!-- Explain the behavior change -->
- Does this need documentation?
- [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg: https://github.com/apache/doris-website/pull/1214 -->
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR should merge into -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](cache) Invalidate ExternalRowCountCache entries when catalog is… [doris]
Posted by "Thearas (via GitHub)" <gi...@apache.org>.
Thearas commented on PR #60999:
URL: https://github.com/apache/doris/pull/60999#issuecomment-3990664835
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR).
Please clearly describe your PR:
1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
3. What features were added. Why was this function added?
4. Which code was refactored and why was this part of the code refactored?
5. Which functions were optimized and what is the difference before and after the optimization?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](cache) Invalidate ExternalRowCountCache entries when catalog is… [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #60999:
URL: https://github.com/apache/doris/pull/60999#issuecomment-3990861954
<details>
<summary>TPC-H: <b>Total hot run time: 28606 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 80054772d5849b2aeede94673f9bad440ea49171, data reload: false
------ Round 1 ----------------------------------
============================================
q1 17702 4459 4287 4287
q2 q3 10652 769 518 518
q4 4677 351 253 253
q5 7566 1199 1018 1018
q6 176 172 145 145
q7 764 854 648 648
q8 9304 1423 1317 1317
q9 4882 4635 4644 4635
q10 6794 1852 1643 1643
q11 471 253 250 250
q12 747 567 464 464
q13 17772 4195 3408 3408
q14 233 233 215 215
q15 945 806 794 794
q16 755 712 670 670
q17 723 869 427 427
q18 6270 5348 5130 5130
q19 1112 960 610 610
q20 502 485 393 393
q21 4648 1925 1518 1518
q22 382 350 263 263
Total cold run time: 97077 ms
Total hot run time: 28606 ms
----- Round 2, with runtime_filter_mode=off -----
============================================
q1 4538 4595 4742 4595
q2 q3 1778 2210 1748 1748
q4 856 1175 782 782
q5 4083 4487 4274 4274
q6 185 176 139 139
q7 1778 1649 1545 1545
q8 2496 2766 2518 2518
q9 7471 7356 7451 7356
q10 2599 2770 2377 2377
q11 498 425 427 425
q12 505 586 443 443
q13 4128 4669 3554 3554
q14 283 294 271 271
q15 849 828 836 828
q16 740 800 728 728
q17 1174 1502 1290 1290
q18 7190 6815 6689 6689
q19 908 832 863 832
q20 2065 2149 1993 1993
q21 3916 3532 3345 3345
q22 442 431 401 401
Total cold run time: 48482 ms
Total hot run time: 46133 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](cache) Invalidate ExternalRowCountCache entries when catalog is… [doris]
Posted by "shuke987 (via GitHub)" <gi...@apache.org>.
shuke987 commented on PR #60999:
URL: https://github.com/apache/doris/pull/60999#issuecomment-3990667187
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](cache) Invalidate ExternalRowCountCache entries when catalog is… [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #60999:
URL: https://github.com/apache/doris/pull/60999#issuecomment-5472207310
We're closing this PR because it hasn't been updated in a while.
This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue manageable.
If you'd like to revive this PR, please reopen it and feel free a maintainer to remove the Stale tag!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](cache) Invalidate ExternalRowCountCache entries when catalog is… [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #60999:
URL: https://github.com/apache/doris/pull/60999#issuecomment-3991801200
# FE Regression Coverage Report
Increment line coverage ` 100.00% (5/5)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/60999_80054772d5849b2aeede94673f9bad440ea49171_merge_fe/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/60999_80054772d5849b2aeede94673f9bad440ea49171_merge_fe/report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](cache) Invalidate ExternalRowCountCache entries when catalog is… [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #60999:
URL: https://github.com/apache/doris/pull/60999#issuecomment-3990964424
<details>
<summary>TPC-DS: <b>Total hot run time: 183614 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 80054772d5849b2aeede94673f9bad440ea49171, data reload: false
query5 4370 692 522 522
query6 330 221 207 207
query7 4208 472 274 274
query8 341 249 245 245
query9 8754 2724 2739 2724
query10 510 373 335 335
query11 16934 18026 17245 17245
query12 212 142 148 142
query13 1699 492 372 372
query14 7085 3459 3142 3142
query14_1 2972 2962 2950 2950
query15 211 194 180 180
query16 1002 471 490 471
query17 1119 838 650 650
query18 2937 525 373 373
query19 228 213 194 194
query20 137 138 128 128
query21 217 135 124 124
query22 5412 4864 4538 4538
query23 17083 16731 16609 16609
query23_1 16567 16562 16660 16562
query24 7167 1638 1215 1215
query24_1 1231 1240 1254 1240
query25 576 496 437 437
query26 820 261 211 211
query27 2679 470 294 294
query28 4403 1890 1881 1881
query29 779 565 475 475
query30 306 242 207 207
query31 847 724 658 658
query32 89 73 74 73
query33 507 346 280 280
query34 889 915 575 575
query35 646 664 612 612
query36 1096 1091 953 953
query37 137 92 82 82
query38 2986 2891 2936 2891
query39 890 885 864 864
query39_1 808 839 828 828
query40 232 153 136 136
query41 63 59 59 59
query42 107 102 102 102
query43 368 376 344 344
query44
query45 207 194 179 179
query46 872 987 622 622
query47 2092 2138 2035 2035
query48 302 310 241 241
query49 621 468 396 396
query50 670 282 219 219
query51 4053 4058 4155 4058
query52 106 106 96 96
query53 294 332 292 292
query54 299 269 266 266
query55 95 85 80 80
query56 316 320 321 320
query57 1336 1322 1249 1249
query58 282 281 277 277
query59 2558 2704 2535 2535
query60 338 321 338 321
query61 151 146 147 146
query62 604 597 518 518
query63 314 283 275 275
query64 4155 1250 972 972
query65
query66 1341 454 357 357
query67 16469 16378 16200 16200
query68
query69 405 302 292 292
query70 1009 1004 969 969
query71 332 303 308 303
query72 2771 2584 2383 2383
query73 539 547 328 328
query74 9953 9871 9804 9804
query75 2895 2735 2484 2484
query76 2309 1050 668 668
query77 366 389 311 311
query78 11250 11254 10676 10676
query79 3089 808 602 602
query80 1785 662 540 540
query81 588 288 265 265
query82 1008 149 118 118
query83 333 263 255 255
query84 256 112 96 96
query85 879 493 441 441
query86 565 314 298 298
query87 3133 3101 2974 2974
query88 3552 2664 2652 2652
query89 413 371 352 352
query90 2202 173 172 172
query91 165 159 136 136
query92 96 71 72 71
query93 2183 847 507 507
query94 667 305 289 289
query95 598 407 317 317
query96 636 517 222 222
query97 2459 2500 2382 2382
query98 231 234 219 219
query99 1024 993 918 918
Total cold run time: 256526 ms
Total hot run time: 183614 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](cache) Invalidate ExternalRowCountCache entries when catalog is… [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed pull request #60999: [fix](cache) Invalidate ExternalRowCountCache entries when catalog is…
URL: https://github.com/apache/doris/pull/60999
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org