You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "zddr (via GitHub)" <gi...@apache.org> on 2025/05/07 12:18:47 UTC

[PR] [enhance](catalog)add remove listener for catalog cache [doris]

zddr opened a new pull request, #50673:
URL: https://github.com/apache/doris/pull/50673

   ### What problem does this PR solve?
   
   Print logs when cache is deleted
   ```
   2025-05-07 20:15:33,071 WARN (NotCheckpointCommonRefreshExecutor-19|203) [CacheLogRemovalListener.onRemoval():38] [ExternalSchemaCache schemaCache]Key: SchemaCacheKey{dbName='tpcds10000_oss', tblName='call_center'} was removed. Cause: SIZE
   ```
   
   Issue Number: close #xxx
   
   Related PR: #xxx
   
   Problem Summary:
   add remove listener for catalog cache
   ### Release note
   add remove listener for catalog cache
   
   ### Check List (For Author)
   
   - Test <!-- At least one of them must be included. -->
       - [ ] Regression test
       - [ ] Unit Test
       - [ ] Manual test (add detailed scripts or steps below)
       - [ ] No need to test or manual test. Explain why:
           - [ ] This is a refactor/code format and no logic has been changed.
           - [ ] Previous test can cover this change.
           - [ ] No code files have been changed.
           - [ ] Other reason <!-- Add your reason?  -->
   
   - Behavior changed:
       - [ ] No.
       - [ ] Yes. <!-- Explain the behavior change -->
   
   - Does this need documentation?
       - [ ] No.
       - [ ] Yes. <!-- Add document PR link here. eg: https://github.com/apache/doris-website/pull/1214 -->
   
   ### Check List (For Reviewer who merge this PR)
   
   - [ ] Confirm the release note
   - [ ] Confirm test cases
   - [ ] Confirm document
   - [ ] Add branch pick label <!-- Add branch pick label that this PR should merge into -->
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [enhance](catalog)add remove listener for catalog cache [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #50673:
URL: https://github.com/apache/doris/pull/50673#issuecomment-2858530709

   
   <details>
   <summary>ClickBench: <b>Total hot run time: 29.32 s</b></summary>
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
   ClickBench test result on commit 8560f77b30838cf68e3a7698a66bd8af03576555, data reload: false
   
   query1	0.03	0.04	0.03
   query2	0.14	0.10	0.11
   query3	0.26	0.18	0.19
   query4	1.59	0.19	0.19
   query5	0.59	0.59	0.61
   query6	1.18	0.72	0.73
   query7	0.02	0.01	0.02
   query8	0.05	0.04	0.04
   query9	0.57	0.52	0.51
   query10	0.56	0.59	0.58
   query11	0.15	0.11	0.12
   query12	0.14	0.12	0.11
   query13	0.61	0.60	0.60
   query14	0.80	0.78	0.81
   query15	0.89	0.84	0.85
   query16	0.40	0.38	0.38
   query17	1.05	1.05	1.02
   query18	0.20	0.19	0.19
   query19	1.88	1.75	1.80
   query20	0.01	0.01	0.02
   query21	15.39	0.92	0.54
   query22	0.75	1.12	0.64
   query23	15.03	1.38	0.62
   query24	8.06	1.06	0.99
   query25	0.49	0.10	0.14
   query26	0.58	0.17	0.15
   query27	0.05	0.05	0.05
   query28	9.51	0.85	0.44
   query29	12.52	3.88	3.24
   query30	0.26	0.09	0.06
   query31	2.81	0.60	0.39
   query32	3.22	0.55	0.47
   query33	3.07	3.17	3.07
   query34	15.72	5.10	4.50
   query35	4.52	4.53	4.52
   query36	0.66	0.50	0.48
   query37	0.09	0.06	0.06
   query38	0.06	0.04	0.04
   query39	0.03	0.02	0.03
   query40	0.17	0.14	0.14
   query41	0.08	0.02	0.02
   query42	0.03	0.02	0.02
   query43	0.04	0.03	0.02
   Total cold run time: 104.26 s
   Total hot run time: 29.32 s
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [enhance](catalog)add remove listener for catalog cache [doris]

Posted by "zddr (via GitHub)" <gi...@apache.org>.
zddr closed pull request #50673: [enhance](catalog)add remove listener for catalog cache
URL: https://github.com/apache/doris/pull/50673


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [enhance](catalog)add remove listener for catalog cache [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #50673:
URL: https://github.com/apache/doris/pull/50673#issuecomment-2858467189

   
   <details>
   <summary>TPC-H: <b>Total hot run time: 33705 ms</b></summary>
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
   Tpch sf100 test result on commit 8560f77b30838cf68e3a7698a66bd8af03576555, data reload: false
   
   ------ Round 1 ----------------------------------
   q1	26437	5062	4988	4988
   q2	2082	280	198	198
   q3	10441	1256	717	717
   q4	10259	1018	534	534
   q5	8114	2411	2350	2350
   q6	184	161	129	129
   q7	918	731	612	612
   q8	9313	1281	1104	1104
   q9	6825	5033	5089	5033
   q10	6864	2316	1860	1860
   q11	492	281	269	269
   q12	333	350	209	209
   q13	17769	3673	3037	3037
   q14	228	221	211	211
   q15	534	484	480	480
   q16	417	425	370	370
   q17	601	864	357	357
   q18	7613	7284	7086	7086
   q19	1497	956	547	547
   q20	328	331	220	220
   q21	3947	3395	2425	2425
   q22	1049	1001	969	969
   Total cold run time: 116245 ms
   Total hot run time: 33705 ms
   
   ----- Round 2, with runtime_filter_mode=off -----
   q1	5105	5083	5062	5062
   q2	233	323	230	230
   q3	2197	2666	2275	2275
   q4	1423	1800	1425	1425
   q5	4502	4445	4437	4437
   q6	206	171	126	126
   q7	1979	1902	1725	1725
   q8	2565	2485	2490	2485
   q9	7180	7201	7070	7070
   q10	3017	3201	2748	2748
   q11	566	496	488	488
   q12	653	757	631	631
   q13	3463	3945	3318	3318
   q14	287	308	267	267
   q15	524	470	473	470
   q16	422	481	445	445
   q17	1179	1488	1402	1402
   q18	7730	7550	7467	7467
   q19	822	839	833	833
   q20	2005	2016	1910	1910
   q21	5100	4659	4477	4477
   q22	1041	1025	978	978
   Total cold run time: 52199 ms
   Total hot run time: 50269 ms
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [enhance](catalog)add remove listener for catalog cache [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #50673:
URL: https://github.com/apache/doris/pull/50673#issuecomment-2858507011

   
   <details>
   <summary>TPC-DS: <b>Total hot run time: 185690 ms</b></summary>
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
   TPC-DS sf100 test result on commit 8560f77b30838cf68e3a7698a66bd8af03576555, data reload: false
   
   query1	974	498	498	498
   query2	6597	1845	1807	1807
   query3	6753	219	219	219
   query4	26123	23220	23008	23008
   query5	4296	612	447	447
   query6	293	193	202	193
   query7	4606	490	289	289
   query8	288	233	246	233
   query9	8624	2537	2541	2537
   query10	451	316	253	253
   query11	15895	15148	14824	14824
   query12	158	106	102	102
   query13	1635	503	390	390
   query14	9044	6216	6197	6197
   query15	205	188	172	172
   query16	7188	636	485	485
   query17	1211	727	569	569
   query18	1987	407	320	320
   query19	197	192	164	164
   query20	124	114	116	114
   query21	219	121	109	109
   query22	4022	4271	4196	4196
   query23	34472	33116	33005	33005
   query24	8380	2395	2376	2376
   query25	559	492	423	423
   query26	1251	270	172	172
   query27	2734	497	377	377
   query28	4290	2068	2049	2049
   query29	777	547	414	414
   query30	285	216	183	183
   query31	947	854	751	751
   query32	71	67	69	67
   query33	541	376	295	295
   query34	775	835	504	504
   query35	785	796	763	763
   query36	945	1024	903	903
   query37	112	101	85	85
   query38	4113	4173	4122	4122
   query39	1446	1394	1391	1391
   query40	209	121	103	103
   query41	56	56	56	56
   query42	124	103	103	103
   query43	498	506	472	472
   query44	1278	797	799	797
   query45	174	169	169	169
   query46	831	1025	625	625
   query47	1757	1758	1755	1755
   query48	379	408	299	299
   query49	779	519	428	428
   query50	628	672	393	393
   query51	4252	4108	4049	4049
   query52	98	110	94	94
   query53	219	244	186	186
   query54	588	568	504	504
   query55	83	79	85	79
   query56	308	320	295	295
   query57	1132	1110	1091	1091
   query58	268	250	262	250
   query59	2607	2618	2513	2513
   query60	311	314	294	294
   query61	135	129	129	129
   query62	814	726	652	652
   query63	224	181	182	181
   query64	4357	1051	654	654
   query65	4331	4252	4264	4252
   query66	1189	403	308	308
   query67	15679	15435	15275	15275
   query68	8417	876	503	503
   query69	474	300	261	261
   query70	1228	1113	1132	1113
   query71	458	311	286	286
   query72	5245	4641	4615	4615
   query73	707	566	346	346
   query74	9004	9117	8944	8944
   query75	3868	3189	2721	2721
   query76	3608	1228	760	760
   query77	800	371	292	292
   query78	10085	10144	9336	9336
   query79	2239	816	558	558
   query80	621	514	422	422
   query81	485	264	220	220
   query82	471	127	94	94
   query83	283	244	226	226
   query84	311	109	86	86
   query85	808	357	313	313
   query86	394	319	308	308
   query87	4438	4500	4344	4344
   query88	3590	2232	2215	2215
   query89	388	314	285	285
   query90	1828	205	210	205
   query91	145	141	119	119
   query92	71	59	57	57
   query93	1668	941	574	574
   query94	673	401	309	309
   query95	381	293	288	288
   query96	473	557	274	274
   query97	3155	3264	3110	3110
   query98	227	210	206	206
   query99	1457	1424	1288	1288
   Total cold run time: 274681 ms
   Total hot run time: 185690 ms
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [enhance](catalog)add remove listener for catalog cache [doris]

Posted by "zddr (via GitHub)" <gi...@apache.org>.
zddr commented on PR #50673:
URL: https://github.com/apache/doris/pull/50673#issuecomment-2858379365

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [enhance](catalog)add remove listener for catalog cache [doris]

Posted by "zddr (via GitHub)" <gi...@apache.org>.
zddr commented on PR #50673:
URL: https://github.com/apache/doris/pull/50673#issuecomment-2861292182

   run p0


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [enhance](catalog)add remove listener for catalog cache [doris]

Posted by "zddr (via GitHub)" <gi...@apache.org>.
zddr commented on PR #50673:
URL: https://github.com/apache/doris/pull/50673#issuecomment-2861292423

   run external


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [enhance](catalog)add remove listener for catalog cache [doris]

Posted by "Thearas (via GitHub)" <gi...@apache.org>.
Thearas commented on PR #50673:
URL: https://github.com/apache/doris/pull/50673#issuecomment-2858370705

   
   Thank you for your contribution to Apache Doris.
   Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR).
   
   Please clearly describe your PR:
   1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
   2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
   3. What features were added. Why was this function added?
   4. Which code was refactored and why was this part of the code refactored?
   5. Which functions were optimized and what is the difference before and after the optimization?
       


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org