You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "mymeiyi (via GitHub)" <gi...@apache.org> on 2025/12/26 06:38:55 UTC

[PR] [fix](storage-engine) _clean_unused_rowset_metas should skip rowsets in _unused_rowsets map [doris]

mymeiyi opened a new pull request, #59390:
URL: https://github.com/apache/doris/pull/59390

   ### What problem does this PR solve?
   
   Issue Number: close #xxx
   
   Related PR: #xxx
   
   Problem Summary:
   
   ### Release note
   
   None
   
   ### Check List (For Author)
   
   - Test <!-- At least one of them must be included. -->
       - [ ] Regression test
       - [ ] Unit Test
       - [ ] Manual test (add detailed scripts or steps below)
       - [ ] No need to test or manual test. Explain why:
           - [ ] This is a refactor/code format and no logic has been changed.
           - [ ] Previous test can cover this change.
           - [ ] No code files have been changed.
           - [ ] Other reason <!-- Add your reason?  -->
   
   - Behavior changed:
       - [ ] No.
       - [ ] Yes. <!-- Explain the behavior change -->
   
   - Does this need documentation?
       - [ ] No.
       - [ ] Yes. <!-- Add document PR link here. eg: https://github.com/apache/doris-website/pull/1214 -->
   
   ### Check List (For Reviewer who merge this PR)
   
   - [ ] Confirm the release note
   - [ ] Confirm test cases
   - [ ] Confirm document
   - [ ] Add branch pick label <!-- Add branch pick label that this PR should merge into -->
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](storage-engine) _clean_unused_rowset_metas should skip rowsets in _unused_rowsets map [doris]

Posted by "mymeiyi (via GitHub)" <gi...@apache.org>.
mymeiyi commented on PR #59390:
URL: https://github.com/apache/doris/pull/59390#issuecomment-3692358240

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](storage-engine) _clean_unused_rowset_metas should skip rowsets in _unused_rowsets map [doris]

Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #59390:
URL: https://github.com/apache/doris/pull/59390#issuecomment-3692262647

   
   Thank you for your contribution to Apache Doris.
   Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR).
   
   Please clearly describe your PR:
   1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
   2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
   3. What features were added. Why was this function added?
   4. Which code was refactored and why was this part of the code refactored?
   5. Which functions were optimized and what is the difference before and after the optimization?
       


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](storage-engine) _clean_unused_rowset_metas should skip rowsets in _unused_rowsets map [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59390:
URL: https://github.com/apache/doris/pull/59390#issuecomment-3692448755

   
   <details>
   <summary>ClickBench: <b>Total hot run time: 27.35 s</b></summary>
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
   ClickBench test result on commit ca41413bb2b59259110f538e21d666ac11541da0, data reload: false
   
   query1	0.05	0.04	0.04
   query2	0.10	0.06	0.05
   query3	0.26	0.09	0.10
   query4	1.61	0.11	0.12
   query5	0.27	0.25	0.25
   query6	1.20	0.64	0.64
   query7	0.03	0.02	0.02
   query8	0.06	0.04	0.04
   query9	0.56	0.50	0.49
   query10	0.56	0.55	0.55
   query11	0.15	0.11	0.11
   query12	0.15	0.11	0.11
   query13	0.62	0.61	0.59
   query14	0.98	1.00	0.99
   query15	0.80	0.79	0.80
   query16	0.40	0.42	0.42
   query17	1.00	1.04	1.04
   query18	0.23	0.22	0.22
   query19	1.86	1.86	1.86
   query20	0.02	0.01	0.01
   query21	15.44	0.27	0.14
   query22	5.05	0.05	0.05
   query23	16.12	0.27	0.10
   query24	1.12	0.88	0.26
   query25	0.11	0.09	0.06
   query26	0.15	0.14	0.13
   query27	0.05	0.06	0.08
   query28	5.14	1.24	1.03
   query29	12.63	4.00	3.20
   query30	0.29	0.14	0.12
   query31	2.81	0.64	0.40
   query32	3.23	0.53	0.47
   query33	2.96	3.01	2.99
   query34	17.06	5.19	4.53
   query35	4.56	4.60	4.56
   query36	0.65	0.50	0.48
   query37	0.11	0.07	0.06
   query38	0.07	0.04	0.04
   query39	0.05	0.03	0.03
   query40	0.16	0.16	0.13
   query41	0.09	0.03	0.03
   query42	0.04	0.03	0.03
   query43	0.04	0.03	0.03
   Total cold run time: 98.84 s
   Total hot run time: 27.35 s
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](storage-engine) _clean_unused_rowset_metas should skip rowsets in _unused_rowsets map [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59390:
URL: https://github.com/apache/doris/pull/59390#issuecomment-3692411584

   
   <details>
   <summary>TPC-H: <b>Total hot run time: 35391 ms</b></summary>
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
   Tpch sf100 test result on commit ca41413bb2b59259110f538e21d666ac11541da0, data reload: false
   
   ------ Round 1 ----------------------------------
   q1	17626	4320	4112	4112
   q2	2025	372	237	237
   q3	10143	1340	775	775
   q4	10241	933	327	327
   q5	7535	2135	1918	1918
   q6	190	170	138	138
   q7	1018	875	720	720
   q8	9357	1422	1167	1167
   q9	7058	5284	5438	5284
   q10	6793	2414	1996	1996
   q11	536	343	318	318
   q12	670	743	568	568
   q13	17778	3688	3099	3099
   q14	296	299	272	272
   q15	595	520	522	520
   q16	702	698	636	636
   q17	727	827	510	510
   q18	7455	7211	7187	7187
   q19	928	981	612	612
   q20	408	366	248	248
   q21	4228	3898	3821	3821
   q22	1050	1004	926	926
   Total cold run time: 107359 ms
   Total hot run time: 35391 ms
   
   ----- Round 2, with runtime_filter_mode=off -----
   q1	4171	4058	4102	4058
   q2	334	396	313	313
   q3	2156	2674	2325	2325
   q4	1328	1739	1297	1297
   q5	4230	4678	4710	4678
   q6	224	182	130	130
   q7	2061	1955	1806	1806
   q8	2707	2528	2575	2528
   q9	7718	7405	7462	7405
   q10	3105	3286	2848	2848
   q11	614	526	534	526
   q12	726	771	631	631
   q13	3667	3983	3229	3229
   q14	300	322	290	290
   q15	556	518	486	486
   q16	676	722	631	631
   q17	1153	1448	1362	1362
   q18	8118	7727	7470	7470
   q19	878	870	917	870
   q20	2026	2062	2002	2002
   q21	4979	4511	4144	4144
   q22	1102	1024	997	997
   Total cold run time: 52829 ms
   Total hot run time: 50026 ms
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](storage-engine) _clean_unused_rowset_metas should skip rowsets in _unused_rowsets map [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59390:
URL: https://github.com/apache/doris/pull/59390#issuecomment-3692503282

   # BE UT Coverage Report
   Increment line coverage `66.67% (4/6)` :tada:
   
   [Increment coverage report](http://coverage.selectdb-in.cc/coverage/ca41413bb2b59259110f538e21d666ac11541da0_ca41413bb2b59259110f538e21d666ac11541da0/increment_report/index.html)
   [Complete coverage report](http://coverage.selectdb-in.cc/coverage/ca41413bb2b59259110f538e21d666ac11541da0_ca41413bb2b59259110f538e21d666ac11541da0/report/index.html)
   | Category          | Coverage           |
   |-------------------|--------------------|
   | Function Coverage | 53.40% (18952/35489) |
   | Line Coverage     | 39.27% (175803/447678) |
   | Region Coverage   | 33.87% (136157/402009) |
   | Branch Coverage   | 34.78% (58761/168940) |


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](storage-engine) _clean_unused_rowset_metas should skip rowsets in _unused_rowsets map [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59390:
URL: https://github.com/apache/doris/pull/59390#issuecomment-3692439224

   
   <details>
   <summary>TPC-DS: <b>Total hot run time: 180589 ms</b></summary>
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
   TPC-DS sf100 test result on commit ca41413bb2b59259110f538e21d666ac11541da0, data reload: false
   
   query5	4427	593	453	453
   query6	329	237	215	215
   query7	4219	463	280	280
   query8	315	274	275	274
   query9	8763	2571	2548	2548
   query10	496	390	358	358
   query11	15336	15365	15004	15004
   query12	190	120	121	120
   query13	1276	502	392	392
   query14	5511	3068	2750	2750
   query14_1	2665	2646	2639	2639
   query15	210	199	177	177
   query16	922	484	458	458
   query17	1156	737	621	621
   query18	2451	452	359	359
   query19	233	240	220	220
   query20	120	124	119	119
   query21	227	139	120	120
   query22	3919	4066	4047	4047
   query23	16619	16224	16173	16173
   query23_1	16043	16356	16183	16183
   query24	7361	1663	1238	1238
   query24_1	1227	1250	1258	1250
   query25	592	510	461	461
   query26	1247	277	175	175
   query27	2745	475	313	313
   query28	4482	2151	2133	2133
   query29	857	593	475	475
   query30	320	248	222	222
   query31	853	727	631	631
   query32	84	77	72	72
   query33	559	345	309	309
   query34	893	899	548	548
   query35	781	804	741	741
   query36	860	963	810	810
   query37	126	90	82	82
   query38	2960	2950	2924	2924
   query39	759	744	717	717
   query39_1	691	697	709	697
   query40	226	150	128	128
   query41	68	64	63	63
   query42	105	100	101	100
   query43	418	440	410	410
   query44	1326	727	743	727
   query45	191	192	186	186
   query46	865	1025	614	614
   query47	1692	1706	1697	1697
   query48	309	325	247	247
   query49	657	436	347	347
   query50	663	298	222	222
   query51	3826	3889	3829	3829
   query52	115	115	105	105
   query53	324	355	290	290
   query54	287	256	251	251
   query55	76	81	73	73
   query56	297	307	297	297
   query57	1177	1175	1118	1118
   query58	276	257	257	257
   query59	2428	2459	2520	2459
   query60	327	307	299	299
   query61	204	167	164	164
   query62	706	720	665	665
   query63	328	296	297	296
   query64	5089	1291	998	998
   query65	4009	3958	3946	3946
   query66	1454	450	316	316
   query67	15501	15094	15011	15011
   query68	6981	983	716	716
   query69	496	347	312	312
   query70	1051	962	951	951
   query71	368	295	284	284
   query72	6151	5022	5058	5022
   query73	678	603	310	310
   query74	9029	8985	8751	8751
   query75	3183	3182	2804	2804
   query76	3843	1141	747	747
   query77	533	405	298	298
   query78	9614	9472	8759	8759
   query79	1539	910	612	612
   query80	741	703	561	561
   query81	538	269	237	237
   query82	408	137	101	101
   query83	266	262	243	243
   query84	258	117	119	117
   query85	936	515	473	473
   query86	406	280	281	280
   query87	3134	3182	3081	3081
   query88	3286	2290	2295	2290
   query89	465	417	392	392
   query90	2186	160	156	156
   query91	179	172	146	146
   query92	87	68	66	66
   query93	1762	901	539	539
   query94	477	312	276	276
   query95	584	384	313	313
   query96	593	458	209	209
   query97	2260	2303	2206	2206
   query98	207	196	194	194
   query99	1313	1325	1307	1307
   Total cold run time: 259255 ms
   Total hot run time: 180589 ms
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](storage-engine) _clean_unused_rowset_metas should skip rowsets in _unused_rowsets map [doris]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #59390:
URL: https://github.com/apache/doris/pull/59390#issuecomment-3692359036

   PR approved by anyone and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](storage-engine) _clean_unused_rowset_metas should skip rowsets in _unused_rowsets map [doris]

Posted by "dataroaring (via GitHub)" <gi...@apache.org>.
dataroaring merged PR #59390:
URL: https://github.com/apache/doris/pull/59390


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](storage-engine) _clean_unused_rowset_metas should skip rowsets in _unused_rowsets map [doris]

Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #59390:
URL: https://github.com/apache/doris/pull/59390#issuecomment-3692742485

   # BE Regression && UT Coverage Report
   Increment line coverage `100.00% (6/6)` :tada:
   
   [Increment coverage report](http://coverage.selectdb-in.cc/coverage/59390_ca41413bb2b59259110f538e21d666ac11541da0_merge/increment_report/index.html)
   [Complete coverage report](http://coverage.selectdb-in.cc/coverage/59390_ca41413bb2b59259110f538e21d666ac11541da0_merge/report/index.html)
   | Category          | Coverage           |
   |-------------------|--------------------|
   | Function Coverage | 72.21% (25051/34694) |
   | Line Coverage     | 58.92% (263048/446486) |
   | Region Coverage   | 53.88% (218847/406174) |
   | Branch Coverage   | 55.34% (93810/169504) |


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](storage-engine) _clean_unused_rowset_metas should skip rowsets in _unused_rowsets map [doris]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #59390:
URL: https://github.com/apache/doris/pull/59390#issuecomment-3693173604

   PR approved by at least one committer and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org