You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "gavinchou (via GitHub)" <gi...@apache.org> on 2025/05/23 09:35:00 UTC

[PR] [fix](cloud) Cloud tablet cache should not be pruned [doris]

gavinchou opened a new pull request, #51193:
URL: https://github.com/apache/doris/pull/51193

   Otherwise there may be severe performance issue for query associated to many tablets
   
   Related PR:  #28940
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](cloud) Cloud tablet cache should not be pruned [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #51193:
URL: https://github.com/apache/doris/pull/51193#issuecomment-2904206955

   
   <details>
   <summary>TPC-DS: <b>Total hot run time: 192994 ms</b></summary>
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
   TPC-DS sf100 test result on commit 6cd2b50570a936302831f4687f6d0dbe13cae3f1, data reload: false
   
   query1	1405	1103	1058	1058
   query2	6219	1841	1822	1822
   query3	11032	4441	4420	4420
   query4	54152	24178	23114	23114
   query5	5206	622	455	455
   query6	356	206	198	198
   query7	4882	511	296	296
   query8	322	260	246	246
   query9	5492	2665	2663	2663
   query10	442	344	289	289
   query11	15126	15006	14814	14814
   query12	175	108	114	108
   query13	1046	545	419	419
   query14	10201	6521	6568	6521
   query15	218	202	192	192
   query16	7049	665	490	490
   query17	1083	733	619	619
   query18	1602	403	308	308
   query19	196	203	170	170
   query20	135	133	122	122
   query21	206	129	109	109
   query22	4411	4504	4389	4389
   query23	34328	33570	33479	33479
   query24	6592	2402	2439	2402
   query25	466	467	423	423
   query26	698	273	159	159
   query27	2333	523	355	355
   query28	3071	2204	2173	2173
   query29	588	566	458	458
   query30	297	216	189	189
   query31	861	841	779	779
   query32	74	65	64	64
   query33	463	365	316	316
   query34	780	872	551	551
   query35	815	845	766	766
   query36	960	1004	892	892
   query37	134	96	83	83
   query38	4208	4330	4358	4330
   query39	1526	1428	1486	1428
   query40	218	119	106	106
   query41	60	56	52	52
   query42	128	112	117	112
   query43	511	532	488	488
   query44	1373	873	864	864
   query45	194	176	172	172
   query46	862	1043	650	650
   query47	1826	1851	1821	1821
   query48	398	443	336	336
   query49	705	538	446	446
   query50	687	690	427	427
   query51	4280	4253	4266	4253
   query52	109	108	101	101
   query53	230	257	191	191
   query54	609	579	533	533
   query55	91	92	81	81
   query56	322	313	298	298
   query57	1173	1220	1134	1134
   query58	278	265	274	265
   query59	2646	2904	2657	2657
   query60	329	326	324	324
   query61	131	148	182	148
   query62	740	792	664	664
   query63	227	200	193	193
   query64	1497	1060	693	693
   query65	4320	4290	4228	4228
   query66	731	398	300	300
   query67	16463	15930	15486	15486
   query68	6818	897	529	529
   query69	557	313	275	275
   query70	1199	1095	1113	1095
   query71	502	332	300	300
   query72	6054	4789	4604	4604
   query73	1483	640	359	359
   query74	8993	9223	8914	8914
   query75	4071	3217	2704	2704
   query76	4217	1187	767	767
   query77	754	366	281	281
   query78	10041	10033	9330	9330
   query79	2407	826	577	577
   query80	642	519	435	435
   query81	472	254	220	220
   query82	416	126	95	95
   query83	307	245	242	242
   query84	293	107	85	85
   query85	802	358	315	315
   query86	353	308	283	283
   query87	4367	4544	4316	4316
   query88	3404	2295	2288	2288
   query89	403	319	292	292
   query90	1959	204	204	204
   query91	144	148	115	115
   query92	77	61	55	55
   query93	1157	931	587	587
   query94	676	390	311	311
   query95	362	288	288	288
   query96	506	574	278	278
   query97	2715	2768	2688	2688
   query98	238	216	201	201
   query99	1424	1383	1269	1269
   Total cold run time: 298101 ms
   Total hot run time: 192994 ms
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](cloud) Cloud tablet cache should not be pruned [doris]

Posted by "dataroaring (via GitHub)" <gi...@apache.org>.
dataroaring merged PR #51193:
URL: https://github.com/apache/doris/pull/51193


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](cloud) Cloud tablet cache should not be pruned [doris]

Posted by "Thearas (via GitHub)" <gi...@apache.org>.
Thearas commented on PR #51193:
URL: https://github.com/apache/doris/pull/51193#issuecomment-2903857974

   
   Thank you for your contribution to Apache Doris.
   Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR).
   
   Please clearly describe your PR:
   1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
   2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
   3. What features were added. Why was this function added?
   4. Which code was refactored and why was this part of the code refactored?
   5. Which functions were optimized and what is the difference before and after the optimization?
       


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](cloud) Cloud tablet cache should not be pruned [doris]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #51193:
URL: https://github.com/apache/doris/pull/51193#issuecomment-2906323125

   PR approved by at least one committer and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](cloud) Cloud tablet cache should not be pruned [doris]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #51193:
URL: https://github.com/apache/doris/pull/51193#issuecomment-2906323152

   PR approved by anyone and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](cloud) Cloud tablet cache should not be pruned [doris]

Posted by "gavinchou (via GitHub)" <gi...@apache.org>.
gavinchou commented on PR #51193:
URL: https://github.com/apache/doris/pull/51193#issuecomment-2903858757

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](cloud) Cloud tablet cache should not be pruned [doris]

Posted by "gavinchou (via GitHub)" <gi...@apache.org>.
gavinchou commented on PR #51193:
URL: https://github.com/apache/doris/pull/51193#issuecomment-2903906376

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](cloud) Cloud tablet cache should not be pruned [doris]

Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #51193:
URL: https://github.com/apache/doris/pull/51193#issuecomment-2904954154

   # BE Regression && UT Coverage Report
   Increment line coverage `100.00% (1/1)` :tada:
   
   [Increment coverage report](http://coverage.selectdb-in.cc/coverage/51193_6cd2b50570a936302831f4687f6d0dbe13cae3f1_merge/increment_report/index.html)
   [Complete coverage report](http://coverage.selectdb-in.cc/coverage/51193_6cd2b50570a936302831f4687f6d0dbe13cae3f1_merge/report/index.html)
   | Category          | Coverage           |
   |-------------------|--------------------|
   | Function Coverage | 79.43% (20832/26226) |
   | Line Coverage     | 72.62% (215234/296368) |
   | Region Coverage   | 70.82% (126497/178618) |
   | Branch Coverage   | 64.56% (65567/101566) |


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](cloud) Cloud tablet cache should not be pruned [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #51193:
URL: https://github.com/apache/doris/pull/51193#issuecomment-2904167938

   # BE UT Coverage Report
   Increment line coverage `100.00% (1/1)` :tada:
   
   [Increment coverage report](http://coverage.selectdb-in.cc/coverage/6cd2b50570a936302831f4687f6d0dbe13cae3f1_6cd2b50570a936302831f4687f6d0dbe13cae3f1/increment_report/index.html)
   [Complete coverage report](http://coverage.selectdb-in.cc/coverage/6cd2b50570a936302831f4687f6d0dbe13cae3f1_6cd2b50570a936302831f4687f6d0dbe13cae3f1/report/index.html)
   | Category          | Coverage           |
   |-------------------|--------------------|
   | Function Coverage | 56.00% (14920/26643) |
   | Line Coverage     | 44.82% (132839/296381) |
   | Region Coverage   | 43.90% (66771/152112) |
   | Branch Coverage   | 38.52% (34263/88942) |


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](cloud) Cloud tablet cache should not be pruned [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #51193:
URL: https://github.com/apache/doris/pull/51193#issuecomment-2904178528

   
   <details>
   <summary>TPC-H: <b>Total hot run time: 33918 ms</b></summary>
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
   Tpch sf100 test result on commit 6cd2b50570a936302831f4687f6d0dbe13cae3f1, data reload: false
   
   ------ Round 1 ----------------------------------
   q1	26430	5106	5061	5061
   q2	2078	286	194	194
   q3	10497	1277	717	717
   q4	10231	999	540	540
   q5	7660	2349	2310	2310
   q6	189	164	132	132
   q7	921	728	602	602
   q8	9298	1303	1100	1100
   q9	6796	5033	5117	5033
   q10	6876	2308	1893	1893
   q11	503	298	280	280
   q12	344	350	209	209
   q13	17810	3727	3096	3096
   q14	232	228	222	222
   q15	526	490	474	474
   q16	432	435	381	381
   q17	640	866	387	387
   q18	7629	7211	7179	7179
   q19	1977	984	558	558
   q20	333	346	224	224
   q21	3959	3267	2365	2365
   q22	1028	987	961	961
   Total cold run time: 116389 ms
   Total hot run time: 33918 ms
   
   ----- Round 2, with runtime_filter_mode=off -----
   q1	5238	5124	5164	5124
   q2	242	329	237	237
   q3	2214	2642	2290	2290
   q4	1419	1779	1417	1417
   q5	4450	4399	4418	4399
   q6	226	170	126	126
   q7	2056	1964	1744	1744
   q8	2654	2658	2565	2565
   q9	7209	7225	6911	6911
   q10	3056	3195	2813	2813
   q11	621	519	490	490
   q12	721	779	604	604
   q13	3526	3882	3282	3282
   q14	277	311	272	272
   q15	523	486	493	486
   q16	449	482	459	459
   q17	1161	1591	1376	1376
   q18	7707	7598	7429	7429
   q19	837	863	883	863
   q20	1969	2107	1871	1871
   q21	4900	4504	4493	4493
   q22	1083	1064	1016	1016
   Total cold run time: 52538 ms
   Total hot run time: 50267 ms
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](cloud) Cloud tablet cache should not be pruned [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #51193:
URL: https://github.com/apache/doris/pull/51193#issuecomment-2904219341

   
   <details>
   <summary>ClickBench: <b>Total hot run time: 29.11 s</b></summary>
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
   ClickBench test result on commit 6cd2b50570a936302831f4687f6d0dbe13cae3f1, data reload: false
   
   query1	0.04	0.04	0.03
   query2	0.13	0.11	0.12
   query3	0.27	0.19	0.20
   query4	1.59	0.20	0.21
   query5	0.45	0.44	0.44
   query6	1.16	0.68	0.67
   query7	0.03	0.02	0.01
   query8	0.05	0.04	0.04
   query9	0.63	0.54	0.53
   query10	0.59	0.60	0.59
   query11	0.17	0.11	0.12
   query12	0.15	0.12	0.12
   query13	0.61	0.59	0.60
   query14	0.78	0.81	0.80
   query15	0.89	0.85	0.88
   query16	0.39	0.39	0.38
   query17	1.04	1.04	1.03
   query18	0.22	0.21	0.21
   query19	1.96	1.80	1.79
   query20	0.01	0.01	0.01
   query21	15.39	0.88	0.53
   query22	0.75	1.15	0.72
   query23	14.90	1.37	0.63
   query24	7.56	1.40	0.70
   query25	0.51	0.07	0.09
   query26	0.68	0.16	0.14
   query27	0.06	0.05	0.05
   query28	9.43	0.89	0.44
   query29	12.55	4.09	3.37
   query30	0.25	0.09	0.06
   query31	2.82	0.61	0.40
   query32	3.23	0.55	0.48
   query33	3.10	3.09	3.13
   query34	15.75	5.06	4.48
   query35	4.52	4.53	4.44
   query36	0.68	0.50	0.49
   query37	0.09	0.06	0.06
   query38	0.05	0.05	0.04
   query39	0.03	0.02	0.03
   query40	0.16	0.13	0.13
   query41	0.08	0.03	0.02
   query42	0.04	0.02	0.02
   query43	0.03	0.04	0.03
   Total cold run time: 103.82 s
   Total hot run time: 29.11 s
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org