You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "shuke987 (via GitHub)" <gi...@apache.org> on 2026/03/03 12:22:51 UTC

[PR] [fix](cache) Invalidate ExternalRowCountCache entries when catalog is… [doris]

shuke987 opened a new pull request, #60999:
URL: https://github.com/apache/doris/pull/60999

   … dropped
   
   When an external catalog is dropped, ExternalRowCountCache entries for tables in that catalog were not invalidated. This caused Caffeine's refreshAfterWrite to endlessly retry loading row counts for non-existent tables, producing continuous WARN logs and wasting refresh thread pool resources.
   
   Add invalidateCatalog() to ExternalRowCountCache and call it from ExternalMetaCacheMgr.removeCache() and invalidateCatalogCache().
   
   ### What problem does this PR solve?
   
   Issue Number: close #xxx
   
   Related PR: #xxx
   
   Problem Summary:
   
   ### Release note
   
   None
   
   ### Check List (For Author)
   
   - Test <!-- At least one of them must be included. -->
       - [ ] Regression test
       - [ ] Unit Test
       - [ ] Manual test (add detailed scripts or steps below)
       - [ ] No need to test or manual test. Explain why:
           - [ ] This is a refactor/code format and no logic has been changed.
           - [ ] Previous test can cover this change.
           - [ ] No code files have been changed.
           - [ ] Other reason <!-- Add your reason?  -->
   
   - Behavior changed:
       - [ ] No.
       - [ ] Yes. <!-- Explain the behavior change -->
   
   - Does this need documentation?
       - [ ] No.
       - [ ] Yes. <!-- Add document PR link here. eg: https://github.com/apache/doris-website/pull/1214 -->
   
   ### Check List (For Reviewer who merge this PR)
   
   - [ ] Confirm the release note
   - [ ] Confirm test cases
   - [ ] Confirm document
   - [ ] Add branch pick label <!-- Add branch pick label that this PR should merge into -->
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](cache) Invalidate ExternalRowCountCache entries when catalog is… [doris]

Posted by "Thearas (via GitHub)" <gi...@apache.org>.
Thearas commented on PR #60999:
URL: https://github.com/apache/doris/pull/60999#issuecomment-3990664835

   
   Thank you for your contribution to Apache Doris.
   Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR).
   
   Please clearly describe your PR:
   1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
   2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
   3. What features were added. Why was this function added?
   4. Which code was refactored and why was this part of the code refactored?
   5. Which functions were optimized and what is the difference before and after the optimization?
       


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](cache) Invalidate ExternalRowCountCache entries when catalog is… [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #60999:
URL: https://github.com/apache/doris/pull/60999#issuecomment-3990861954

   
   <details>
   <summary>TPC-H: <b>Total hot run time: 28606 ms</b></summary>
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
   Tpch sf100 test result on commit 80054772d5849b2aeede94673f9bad440ea49171, data reload: false
   
   ------ Round 1 ----------------------------------
   ============================================
   q1	17702	4459	4287	4287
   q2	q3	10652	769	518	518
   q4	4677	351	253	253
   q5	7566	1199	1018	1018
   q6	176	172	145	145
   q7	764	854	648	648
   q8	9304	1423	1317	1317
   q9	4882	4635	4644	4635
   q10	6794	1852	1643	1643
   q11	471	253	250	250
   q12	747	567	464	464
   q13	17772	4195	3408	3408
   q14	233	233	215	215
   q15	945	806	794	794
   q16	755	712	670	670
   q17	723	869	427	427
   q18	6270	5348	5130	5130
   q19	1112	960	610	610
   q20	502	485	393	393
   q21	4648	1925	1518	1518
   q22	382	350	263	263
   Total cold run time: 97077 ms
   Total hot run time: 28606 ms
   
   ----- Round 2, with runtime_filter_mode=off -----
   ============================================
   q1	4538	4595	4742	4595
   q2	q3	1778	2210	1748	1748
   q4	856	1175	782	782
   q5	4083	4487	4274	4274
   q6	185	176	139	139
   q7	1778	1649	1545	1545
   q8	2496	2766	2518	2518
   q9	7471	7356	7451	7356
   q10	2599	2770	2377	2377
   q11	498	425	427	425
   q12	505	586	443	443
   q13	4128	4669	3554	3554
   q14	283	294	271	271
   q15	849	828	836	828
   q16	740	800	728	728
   q17	1174	1502	1290	1290
   q18	7190	6815	6689	6689
   q19	908	832	863	832
   q20	2065	2149	1993	1993
   q21	3916	3532	3345	3345
   q22	442	431	401	401
   Total cold run time: 48482 ms
   Total hot run time: 46133 ms
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](cache) Invalidate ExternalRowCountCache entries when catalog is… [doris]

Posted by "shuke987 (via GitHub)" <gi...@apache.org>.
shuke987 commented on PR #60999:
URL: https://github.com/apache/doris/pull/60999#issuecomment-3990667187

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](cache) Invalidate ExternalRowCountCache entries when catalog is… [doris]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #60999:
URL: https://github.com/apache/doris/pull/60999#issuecomment-5472207310

   We're closing this PR because it hasn't been updated in a while.
   This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue manageable.
   If you'd like to revive this PR, please reopen it and feel free a maintainer to remove the Stale tag!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](cache) Invalidate ExternalRowCountCache entries when catalog is… [doris]

Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #60999:
URL: https://github.com/apache/doris/pull/60999#issuecomment-3991801200

   # FE Regression Coverage Report
   Increment line coverage `  100.00% (5/5)` :tada:
   [Increment coverage report](http://coverage.selectdb-in.cc/coverage/60999_80054772d5849b2aeede94673f9bad440ea49171_merge_fe/increment_report/index.html)
   [Complete coverage report](http://coverage.selectdb-in.cc/coverage/60999_80054772d5849b2aeede94673f9bad440ea49171_merge_fe/report/index.html)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](cache) Invalidate ExternalRowCountCache entries when catalog is… [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #60999:
URL: https://github.com/apache/doris/pull/60999#issuecomment-3990964424

   
   <details>
   <summary>TPC-DS: <b>Total hot run time: 183614 ms</b></summary>
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
   TPC-DS sf100 test result on commit 80054772d5849b2aeede94673f9bad440ea49171, data reload: false
   
   query5	4370	692	522	522
   query6	330	221	207	207
   query7	4208	472	274	274
   query8	341	249	245	245
   query9	8754	2724	2739	2724
   query10	510	373	335	335
   query11	16934	18026	17245	17245
   query12	212	142	148	142
   query13	1699	492	372	372
   query14	7085	3459	3142	3142
   query14_1	2972	2962	2950	2950
   query15	211	194	180	180
   query16	1002	471	490	471
   query17	1119	838	650	650
   query18	2937	525	373	373
   query19	228	213	194	194
   query20	137	138	128	128
   query21	217	135	124	124
   query22	5412	4864	4538	4538
   query23	17083	16731	16609	16609
   query23_1	16567	16562	16660	16562
   query24	7167	1638	1215	1215
   query24_1	1231	1240	1254	1240
   query25	576	496	437	437
   query26	820	261	211	211
   query27	2679	470	294	294
   query28	4403	1890	1881	1881
   query29	779	565	475	475
   query30	306	242	207	207
   query31	847	724	658	658
   query32	89	73	74	73
   query33	507	346	280	280
   query34	889	915	575	575
   query35	646	664	612	612
   query36	1096	1091	953	953
   query37	137	92	82	82
   query38	2986	2891	2936	2891
   query39	890	885	864	864
   query39_1	808	839	828	828
   query40	232	153	136	136
   query41	63	59	59	59
   query42	107	102	102	102
   query43	368	376	344	344
   query44	
   query45	207	194	179	179
   query46	872	987	622	622
   query47	2092	2138	2035	2035
   query48	302	310	241	241
   query49	621	468	396	396
   query50	670	282	219	219
   query51	4053	4058	4155	4058
   query52	106	106	96	96
   query53	294	332	292	292
   query54	299	269	266	266
   query55	95	85	80	80
   query56	316	320	321	320
   query57	1336	1322	1249	1249
   query58	282	281	277	277
   query59	2558	2704	2535	2535
   query60	338	321	338	321
   query61	151	146	147	146
   query62	604	597	518	518
   query63	314	283	275	275
   query64	4155	1250	972	972
   query65	
   query66	1341	454	357	357
   query67	16469	16378	16200	16200
   query68	
   query69	405	302	292	292
   query70	1009	1004	969	969
   query71	332	303	308	303
   query72	2771	2584	2383	2383
   query73	539	547	328	328
   query74	9953	9871	9804	9804
   query75	2895	2735	2484	2484
   query76	2309	1050	668	668
   query77	366	389	311	311
   query78	11250	11254	10676	10676
   query79	3089	808	602	602
   query80	1785	662	540	540
   query81	588	288	265	265
   query82	1008	149	118	118
   query83	333	263	255	255
   query84	256	112	96	96
   query85	879	493	441	441
   query86	565	314	298	298
   query87	3133	3101	2974	2974
   query88	3552	2664	2652	2652
   query89	413	371	352	352
   query90	2202	173	172	172
   query91	165	159	136	136
   query92	96	71	72	71
   query93	2183	847	507	507
   query94	667	305	289	289
   query95	598	407	317	317
   query96	636	517	222	222
   query97	2459	2500	2382	2382
   query98	231	234	219	219
   query99	1024	993	918	918
   Total cold run time: 256526 ms
   Total hot run time: 183614 ms
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](cache) Invalidate ExternalRowCountCache entries when catalog is… [doris]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed pull request #60999: [fix](cache) Invalidate ExternalRowCountCache entries when catalog is…
URL: https://github.com/apache/doris/pull/60999


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org