You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "sollhui (via GitHub)" <gi...@apache.org> on 2025/04/15 13:52:24 UTC

[PR] [fix](transaction) remove visible rowset from memory during deletion transaction [doris]

sollhui opened a new pull request, #50066:
URL: https://github.com/apache/doris/pull/50066

   ### What problem does this PR solve?
   
   Repeatedly reporting errors in the log:
   ```
   W20250415 17:54:01.151019 218169 storage_engine.cpp:785] failed to clear transaction. txn_id=1002, partition_id=1744709709398, tablet_id=1744709709485, status=[E-228]could not delete transaction from engine, just remove it from memory not delete from disk, because related rowset already published. partition_id: 1744709709398, transaction_id: 1002, tablet: 1744709709485.fb491f2a6f29dad0-28fe3fdfba3272b5, rowset id: 020000000000001fc6430de53121366b7d7bc36d82a1ae92, version: [2-2], state: VISIBLE
   W20250415 17:54:01.152154 218169 storage_engine.cpp:785] failed to clear transaction. txn_id=1002, partition_id=1744709709398, tablet_id=1744709709493, status=[E-228]could not delete transaction from engine, just remove it from memory not delete from disk, because related rowset already published. partition_id: 1744709709398, transaction_id: 1002, tablet: 1744709709493.7a47a3bf7dcc70f0-353230158f6c2390, rowset id: 0200000000000019c6430de53121366b7d7bc36d82a1ae92, version: [2-2], state: VISIBLE
   W20250415 17:54:01.152177 218169 storage_engine.cpp:785] failed to clear transaction. txn_id=1002, partition_id=1744709709398, tablet_id=1744709709509, status=[E-228]could not delete transaction from engine, just remove it from memory not delete from disk, because related rowset already published. partition_id: 1744709709398, transaction_id: 1002, tablet: 1744709709509.7f49efdb3b6c2c1d-ed627da9d22c9884, rowset id: 020000000000001bc6430de53121366b7d7bc36d82a1ae92, version: [2-2], state: VISIBLE
   ```
   This bug can occur in the following scenarios:
   1. After load the three replicas of the table, the publish task failed after making the rowset as visible in one of the replica.
   2. The transactions became visible. Then FE clear visible transaction.
   3. The BE node that fails to publish will not delete the memory transaction and report it to FE. FE cannot find the transaction and issues a clear command. However, the deletion fails due to the rowset already published.
   
   ### Release note
   
   None
   
   ### Check List (For Author)
   
   - Test <!-- At least one of them must be included. -->
       - [ ] Regression test
       - [ ] Unit Test
       - [ ] Manual test (add detailed scripts or steps below)
       - [ ] No need to test or manual test. Explain why:
           - [ ] This is a refactor/code format and no logic has been changed.
           - [ ] Previous test can cover this change.
           - [ ] No code files have been changed.
           - [ ] Other reason <!-- Add your reason?  -->
   
   - Behavior changed:
       - [ ] No.
       - [ ] Yes. <!-- Explain the behavior change -->
   
   - Does this need documentation?
       - [ ] No.
       - [ ] Yes. <!-- Add document PR link here. eg: https://github.com/apache/doris-website/pull/1214 -->
   
   ### Check List (For Reviewer who merge this PR)
   
   - [ ] Confirm the release note
   - [ ] Confirm test cases
   - [ ] Confirm document
   - [ ] Add branch pick label <!-- Add branch pick label that this PR should merge into -->
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](transaction) remove visible rowset from memory during deletion transaction [doris]

Posted by "sollhui (via GitHub)" <gi...@apache.org>.
sollhui commented on PR #50066:
URL: https://github.com/apache/doris/pull/50066#issuecomment-2805451088

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](transaction) remove visible rowset from memory during deletion transaction [doris]

Posted by "sollhui (via GitHub)" <gi...@apache.org>.
sollhui commented on PR #50066:
URL: https://github.com/apache/doris/pull/50066#issuecomment-2805455736

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](transaction) remove visible rowset from memory during deletion transaction [doris]

Posted by "sollhui (via GitHub)" <gi...@apache.org>.
sollhui commented on PR #50066:
URL: https://github.com/apache/doris/pull/50066#issuecomment-2805135023

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](transaction) remove visible rowset from memory during deletion transaction [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #50066:
URL: https://github.com/apache/doris/pull/50066#issuecomment-2808239253

   
   <details>
   <summary>TPC-H: <b>Total hot run time: 33771 ms</b></summary>
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
   Tpch sf100 test result on commit d6648d75ef7b15da8e0037f725735fed22ec1350, data reload: false
   
   ------ Round 1 ----------------------------------
   q1	25847	5070	4990	4990
   q2	2065	283	186	186
   q3	10379	1284	719	719
   q4	10222	998	541	541
   q5	7508	2443	2290	2290
   q6	185	167	132	132
   q7	943	732	623	623
   q8	9316	1300	1071	1071
   q9	6795	5089	5122	5089
   q10	6818	2299	1893	1893
   q11	481	282	271	271
   q12	353	354	217	217
   q13	17781	3649	3081	3081
   q14	246	222	202	202
   q15	536	495	500	495
   q16	444	445	394	394
   q17	615	867	375	375
   q18	7590	7217	7040	7040
   q19	1230	942	559	559
   q20	345	331	223	223
   q21	4258	3422	2424	2424
   q22	1017	1011	956	956
   Total cold run time: 114974 ms
   Total hot run time: 33771 ms
   
   ----- Round 2, with runtime_filter_mode=off -----
   q1	5075	5084	5074	5074
   q2	245	332	226	226
   q3	2155	2691	2318	2318
   q4	1430	1798	1397	1397
   q5	4441	4385	4404	4385
   q6	218	172	130	130
   q7	2012	1907	1781	1781
   q8	2629	2585	2483	2483
   q9	7304	7242	7057	7057
   q10	3025	3188	2697	2697
   q11	589	529	503	503
   q12	689	783	614	614
   q13	3519	3935	3270	3270
   q14	287	313	294	294
   q15	532	496	467	467
   q16	465	502	464	464
   q17	1152	1562	1379	1379
   q18	7828	7620	7616	7616
   q19	840	820	889	820
   q20	2022	1990	1817	1817
   q21	5205	4894	4817	4817
   q22	1118	1040	996	996
   Total cold run time: 52780 ms
   Total hot run time: 50605 ms
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](transaction) remove visible rowset from memory during deletion transaction [doris]

Posted by "dataroaring (via GitHub)" <gi...@apache.org>.
dataroaring merged PR #50066:
URL: https://github.com/apache/doris/pull/50066


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](transaction) remove visible rowset from memory during deletion transaction [doris]

Posted by "sollhui (via GitHub)" <gi...@apache.org>.
sollhui commented on PR #50066:
URL: https://github.com/apache/doris/pull/50066#issuecomment-2808144130

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](transaction) remove visible rowset from memory during deletion transaction [doris]

Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #50066:
URL: https://github.com/apache/doris/pull/50066#issuecomment-2805134721

   
   Thank you for your contribution to Apache Doris.
   Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR).
   
   Please clearly describe your PR:
   1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
   2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
   3. What features were added. Why was this function added?
   4. Which code was refactored and why was this part of the code refactored?
   5. Which functions were optimized and what is the difference before and after the optimization?
       


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](transaction) remove visible rowset from memory during deletion transaction [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #50066:
URL: https://github.com/apache/doris/pull/50066#issuecomment-2806625935

   
   <details>
   <summary>ClickBench: <b>Total hot run time: 31.24 s</b></summary>
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
   ClickBench test result on commit 5cbeb96c2c72a3af21e380789445b82184ac231a, data reload: false
   
   query1	0.04	0.03	0.03
   query2	0.12	0.10	0.11
   query3	0.24	0.20	0.19
   query4	1.60	0.19	0.19
   query5	0.58	0.60	0.58
   query6	1.20	0.71	0.71
   query7	0.03	0.02	0.01
   query8	0.05	0.03	0.03
   query9	0.58	0.54	0.52
   query10	0.58	0.57	0.56
   query11	0.15	0.11	0.11
   query12	0.15	0.12	0.11
   query13	0.62	0.62	0.59
   query14	2.68	2.84	2.80
   query15	0.95	0.86	0.86
   query16	0.38	0.37	0.38
   query17	1.04	1.05	1.02
   query18	0.21	0.20	0.20
   query19	1.86	1.97	1.86
   query20	0.01	0.02	0.01
   query21	15.91	0.87	0.56
   query22	0.90	1.14	0.65
   query23	15.12	1.40	0.60
   query24	7.04	1.81	0.83
   query25	0.56	0.20	0.19
   query26	0.63	0.15	0.13
   query27	0.05	0.04	0.05
   query28	9.38	0.90	0.41
   query29	12.56	3.98	3.30
   query30	0.25	0.09	0.06
   query31	2.84	0.59	0.39
   query32	3.22	0.58	0.48
   query33	3.00	2.99	3.06
   query34	15.85	5.07	4.46
   query35	4.47	4.51	4.52
   query36	0.65	0.49	0.49
   query37	0.09	0.06	0.06
   query38	0.05	0.04	0.03
   query39	0.04	0.02	0.03
   query40	0.15	0.14	0.12
   query41	0.08	0.03	0.02
   query42	0.04	0.03	0.02
   query43	0.03	0.03	0.03
   Total cold run time: 105.98 s
   Total hot run time: 31.24 s
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](transaction) remove visible rowset from memory during deletion transaction [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #50066:
URL: https://github.com/apache/doris/pull/50066#issuecomment-2808283433

   
   <details>
   <summary>TPC-DS: <b>Total hot run time: 192575 ms</b></summary>
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
   TPC-DS sf100 test result on commit d6648d75ef7b15da8e0037f725735fed22ec1350, data reload: false
   
   query1	1427	1080	1050	1050
   query2	6453	1833	1811	1811
   query3	10982	4507	4405	4405
   query4	54905	24849	23801	23801
   query5	4986	555	482	482
   query6	356	206	191	191
   query7	4935	522	292	292
   query8	346	248	254	248
   query9	5373	2568	2572	2568
   query10	445	305	249	249
   query11	15315	14986	14768	14768
   query12	158	112	110	110
   query13	1030	550	394	394
   query14	10250	6305	6303	6303
   query15	211	191	180	180
   query16	7105	664	463	463
   query17	1086	701	577	577
   query18	1588	430	295	295
   query19	195	186	156	156
   query20	128	130	115	115
   query21	205	127	127	127
   query22	4375	4475	4236	4236
   query23	34197	33502	33325	33325
   query24	6763	2369	2378	2369
   query25	449	464	405	405
   query26	679	276	161	161
   query27	2389	509	334	334
   query28	3032	2172	2167	2167
   query29	578	561	424	424
   query30	268	224	185	185
   query31	860	883	771	771
   query32	76	65	64	64
   query33	454	367	320	320
   query34	769	874	519	519
   query35	815	841	775	775
   query36	941	1001	895	895
   query37	111	103	81	81
   query38	4192	4230	4185	4185
   query39	1489	1420	1435	1420
   query40	226	126	106	106
   query41	53	53	53	53
   query42	123	110	108	108
   query43	495	510	487	487
   query44	1303	816	812	812
   query45	178	177	169	169
   query46	833	1031	648	648
   query47	1876	1857	1760	1760
   query48	386	413	318	318
   query49	690	528	425	425
   query50	658	704	417	417
   query51	4248	4195	4277	4195
   query52	109	109	99	99
   query53	231	261	192	192
   query54	622	578	516	516
   query55	83	81	81	81
   query56	319	296	299	296
   query57	1151	1177	1153	1153
   query58	273	292	258	258
   query59	2780	2817	2723	2723
   query60	319	311	319	311
   query61	130	131	128	128
   query62	746	757	699	699
   query63	235	191	187	187
   query64	1524	1044	730	730
   query65	4431	4358	4219	4219
   query66	724	415	360	360
   query67	15954	15382	15344	15344
   query68	6893	876	521	521
   query69	532	289	261	261
   query70	1197	1131	1124	1124
   query71	470	318	280	280
   query72	5892	4814	4843	4814
   query73	1288	646	339	339
   query74	8929	9141	8753	8753
   query75	3659	3209	2693	2693
   query76	4072	1189	747	747
   query77	626	368	272	272
   query78	9993	10039	9284	9284
   query79	2666	801	554	554
   query80	665	503	447	447
   query81	475	256	216	216
   query82	601	125	94	94
   query83	247	244	227	227
   query84	287	102	89	89
   query85	780	358	315	315
   query86	367	316	275	275
   query87	4405	4472	4370	4370
   query88	3219	2233	2193	2193
   query89	387	322	276	276
   query90	1761	213	215	213
   query91	146	143	114	114
   query92	78	61	55	55
   query93	2425	960	592	592
   query94	689	420	313	313
   query95	366	294	283	283
   query96	484	568	266	266
   query97	3163	3189	3098	3098
   query98	229	211	206	206
   query99	1440	1425	1276	1276
   Total cold run time: 299047 ms
   Total hot run time: 192575 ms
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](transaction) remove visible rowset from memory during deletion transaction [doris]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #50066:
URL: https://github.com/apache/doris/pull/50066#issuecomment-2805420442

   PR approved by at least one committer and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](transaction) remove visible rowset from memory during deletion transaction [doris]

Posted by "sollhui (via GitHub)" <gi...@apache.org>.
sollhui commented on PR #50066:
URL: https://github.com/apache/doris/pull/50066#issuecomment-2805563700

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](transaction) remove visible rowset from memory during deletion transaction [doris]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #50066:
URL: https://github.com/apache/doris/pull/50066#issuecomment-2808154379

   PR approved by at least one committer and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](transaction) remove visible rowset from memory during deletion transaction [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #50066:
URL: https://github.com/apache/doris/pull/50066#issuecomment-2808289833

   
   <details>
   <summary>ClickBench: <b>Total hot run time: 29.5 s</b></summary>
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
   ClickBench test result on commit d6648d75ef7b15da8e0037f725735fed22ec1350, data reload: false
   
   query1	0.03	0.03	0.03
   query2	0.12	0.10	0.10
   query3	0.24	0.19	0.19
   query4	1.59	0.19	0.19
   query5	0.59	0.58	0.58
   query6	1.16	0.72	0.71
   query7	0.02	0.01	0.01
   query8	0.04	0.04	0.03
   query9	0.58	0.52	0.51
   query10	0.55	0.56	0.58
   query11	0.16	0.11	0.11
   query12	0.14	0.11	0.12
   query13	0.62	0.59	0.60
   query14	1.14	1.18	1.18
   query15	0.86	0.83	0.85
   query16	0.36	0.38	0.38
   query17	1.01	1.03	1.03
   query18	0.21	0.21	0.20
   query19	1.86	1.83	1.77
   query20	0.01	0.00	0.01
   query21	15.42	0.91	0.55
   query22	0.74	1.09	0.65
   query23	15.10	1.37	0.63
   query24	6.92	1.98	0.81
   query25	0.45	0.23	0.15
   query26	0.62	0.17	0.13
   query27	0.06	0.05	0.05
   query28	9.31	0.81	0.42
   query29	12.54	4.04	3.39
   query30	0.26	0.10	0.06
   query31	2.84	0.58	0.38
   query32	3.22	0.54	0.47
   query33	3.06	3.02	3.08
   query34	15.74	5.09	4.42
   query35	4.52	4.52	4.47
   query36	0.66	0.49	0.48
   query37	0.09	0.06	0.06
   query38	0.04	0.03	0.03
   query39	0.03	0.03	0.02
   query40	0.17	0.13	0.13
   query41	0.07	0.02	0.02
   query42	0.03	0.03	0.02
   query43	0.04	0.03	0.03
   Total cold run time: 103.22 s
   Total hot run time: 29.5 s
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](transaction) remove visible rowset from memory during deletion transaction [doris]

Posted by "sollhui (via GitHub)" <gi...@apache.org>.
sollhui commented on PR #50066:
URL: https://github.com/apache/doris/pull/50066#issuecomment-2808142210

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](transaction) remove visible rowset from memory during deletion transaction [doris]

Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #50066:
URL: https://github.com/apache/doris/pull/50066#issuecomment-2808430868

   # BE UT Coverage Report
   Increment line coverage `100.00% (3/3)` :tada:
   
   [Increment coverage report](http://coverage.selectdb-in.cc/coverage/d6648d75ef7b15da8e0037f725735fed22ec1350_d6648d75ef7b15da8e0037f725735fed22ec1350/increment_report/index.html)
   [Complete coverage report](http://coverage.selectdb-in.cc/coverage/d6648d75ef7b15da8e0037f725735fed22ec1350_d6648d75ef7b15da8e0037f725735fed22ec1350/report/index.html)
   | Category          | Coverage           |
   |-------------------|--------------------|
   | Function Coverage | 52.83% (14222/26921) |
   | Line Coverage     | 41.72% (123433/295891) |
   | Region Coverage   | 40.44% (62856/155413) |
   | Branch Coverage   | 35.07% (31524/89892) |


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](transaction) remove visible rowset from memory during deletion transaction [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #50066:
URL: https://github.com/apache/doris/pull/50066#issuecomment-2806451093

   
   <details>
   <summary>TPC-H: <b>Total hot run time: 34447 ms</b></summary>
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
   Tpch sf100 test result on commit 5cbeb96c2c72a3af21e380789445b82184ac231a, data reload: false
   
   ------ Round 1 ----------------------------------
   q1	26614	5028	5058	5028
   q2	2108	291	189	189
   q3	10366	1245	696	696
   q4	10217	1060	541	541
   q5	7547	3085	2416	2416
   q6	197	165	132	132
   q7	908	771	616	616
   q8	9316	1393	1163	1163
   q9	6899	5097	5185	5097
   q10	6800	2321	1893	1893
   q11	485	300	275	275
   q12	351	356	219	219
   q13	17763	3679	3136	3136
   q14	226	222	213	213
   q15	523	492	479	479
   q16	631	613	576	576
   q17	619	887	406	406
   q18	7609	7370	7100	7100
   q19	1229	953	580	580
   q20	344	335	226	226
   q21	4398	3382	2490	2490
   q22	1034	976	979	976
   Total cold run time: 116184 ms
   Total hot run time: 34447 ms
   
   ----- Round 2, with runtime_filter_mode=off -----
   q1	5101	5089	5075	5075
   q2	242	330	224	224
   q3	2189	2656	2317	2317
   q4	1458	1947	1508	1508
   q5	4555	4434	4439	4434
   q6	216	165	124	124
   q7	1984	1910	1775	1775
   q8	2578	2447	2574	2447
   q9	7296	7272	7184	7184
   q10	2968	3171	2703	2703
   q11	575	510	485	485
   q12	656	766	579	579
   q13	3588	3903	3274	3274
   q14	297	303	273	273
   q15	517	485	475	475
   q16	644	684	668	668
   q17	1161	1598	1364	1364
   q18	7724	7444	7499	7444
   q19	835	841	972	841
   q20	1988	2062	1842	1842
   q21	5214	4638	4566	4566
   q22	1066	1047	982	982
   Total cold run time: 52852 ms
   Total hot run time: 50584 ms
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](transaction) remove visible rowset from memory during deletion transaction [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #50066:
URL: https://github.com/apache/doris/pull/50066#issuecomment-2806582083

   
   <details>
   <summary>TPC-DS: <b>Total hot run time: 186345 ms</b></summary>
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
   TPC-DS sf100 test result on commit 5cbeb96c2c72a3af21e380789445b82184ac231a, data reload: false
   
   query1	1016	479	505	479
   query2	6547	1946	1931	1931
   query3	6761	220	218	218
   query4	26197	23399	23415	23399
   query5	4376	628	481	481
   query6	305	193	198	193
   query7	4639	488	277	277
   query8	280	232	237	232
   query9	8624	2566	2585	2566
   query10	483	303	260	260
   query11	15605	15120	14813	14813
   query12	156	108	107	107
   query13	1646	510	376	376
   query14	9193	6039	5995	5995
   query15	205	182	182	182
   query16	7177	635	487	487
   query17	1193	712	573	573
   query18	1958	402	313	313
   query19	191	181	157	157
   query20	120	117	156	117
   query21	211	122	102	102
   query22	3971	4136	3973	3973
   query23	33924	33099	33045	33045
   query24	8461	2381	2394	2381
   query25	524	444	410	410
   query26	1236	267	155	155
   query27	2748	513	335	335
   query28	4354	2446	2399	2399
   query29	807	573	456	456
   query30	287	225	194	194
   query31	918	854	756	756
   query32	75	71	64	64
   query33	564	388	336	336
   query34	795	874	511	511
   query35	828	836	724	724
   query36	946	988	871	871
   query37	124	102	78	78
   query38	4108	4165	4159	4159
   query39	1477	1404	1419	1404
   query40	219	125	117	117
   query41	62	62	62	62
   query42	122	106	109	106
   query43	517	515	479	479
   query44	1313	812	791	791
   query45	181	173	171	171
   query46	846	1012	616	616
   query47	1745	1804	1773	1773
   query48	381	409	304	304
   query49	810	553	447	447
   query50	668	687	406	406
   query51	4180	4167	4057	4057
   query52	107	100	97	97
   query53	230	248	188	188
   query54	571	569	494	494
   query55	80	79	79	79
   query56	290	309	277	277
   query57	1135	1156	1067	1067
   query58	264	264	243	243
   query59	2620	2815	2799	2799
   query60	326	317	296	296
   query61	133	131	132	131
   query62	803	741	649	649
   query63	231	190	187	187
   query64	4469	1028	707	707
   query65	4367	4290	4239	4239
   query66	1140	431	311	311
   query67	15841	15423	15199	15199
   query68	8349	901	515	515
   query69	473	309	261	261
   query70	1231	1090	1112	1090
   query71	444	337	290	290
   query72	5296	4803	4726	4726
   query73	712	598	334	334
   query74	9224	9091	8740	8740
   query75	3832	3200	2689	2689
   query76	3663	1200	769	769
   query77	786	376	289	289
   query78	10061	10128	9270	9270
   query79	2295	807	576	576
   query80	606	497	434	434
   query81	537	252	225	225
   query82	443	123	96	96
   query83	287	265	238	238
   query84	290	98	84	84
   query85	793	354	313	313
   query86	338	294	273	273
   query87	4616	4483	4362	4362
   query88	3330	2187	2237	2187
   query89	389	320	285	285
   query90	1931	215	208	208
   query91	146	142	110	110
   query92	77	61	63	61
   query93	1300	952	595	595
   query94	677	403	293	293
   query95	411	299	287	287
   query96	489	561	279	279
   query97	3219	3234	3166	3166
   query98	232	202	207	202
   query99	1499	1427	1297	1297
   Total cold run time: 274657 ms
   Total hot run time: 186345 ms
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](transaction) remove visible rowset from memory during deletion transaction [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #50066:
URL: https://github.com/apache/doris/pull/50066#issuecomment-2806987756

   # BE UT Coverage Report
   Increment line coverage `66.67% (2/3)` :tada:
   
   [Increment coverage report](http://coverage.selectdb-in.cc/coverage/5cbeb96c2c72a3af21e380789445b82184ac231a_5cbeb96c2c72a3af21e380789445b82184ac231a/increment_report/index.html)
   [Complete coverage report](http://coverage.selectdb-in.cc/coverage/5cbeb96c2c72a3af21e380789445b82184ac231a_5cbeb96c2c72a3af21e380789445b82184ac231a/report/index.html)
   | Category          | Coverage           |
   |-------------------|--------------------|
   | Function Coverage | 52.43% (14109/26908) |
   | Line Coverage     | 41.35% (122333/295846) |
   | Region Coverage   | 40.07% (62252/155346) |
   | Branch Coverage   | 34.78% (31253/89858) |


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](transaction) remove visible rowset from memory during deletion transaction [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #50066:
URL: https://github.com/apache/doris/pull/50066#issuecomment-2805447380

   
   <details>
   <summary>TPC-H: <b>Total hot run time: 35119 ms</b></summary>
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
   Tpch sf100 test result on commit 8ac61e231b42553617ea09143f9c784871b368c3, data reload: false
   
   ------ Round 1 ----------------------------------
   q1	26157	5132	5094	5094
   q2	2055	298	179	179
   q3	10378	1225	701	701
   q4	10216	1001	553	553
   q5	7521	2419	2287	2287
   q6	185	170	133	133
   q7	916	755	625	625
   q8	9325	1263	1105	1105
   q9	6817	5047	5057	5047
   q10	6863	2277	1875	1875
   q11	480	296	280	280
   q12	369	362	227	227
   q13	17777	3695	3098	3098
   q14	223	226	233	226
   q15	531	489	485	485
   q16	626	604	569	569
   q17	619	852	365	365
   q18	7761	7445	7280	7280
   q19	1396	953	563	563
   q20	337	356	228	228
   q21	4460	3609	3207	3207
   q22	1062	1019	992	992
   Total cold run time: 116074 ms
   Total hot run time: 35119 ms
   
   ----- Round 2, with runtime_filter_mode=off -----
   q1	5309	5109	5099	5099
   q2	242	329	229	229
   q3	2154	2651	2226	2226
   q4	1419	1924	1466	1466
   q5	4458	4441	4370	4370
   q6	215	171	133	133
   q7	1985	1896	1792	1792
   q8	2556	2550	2593	2550
   q9	7207	6988	7230	6988
   q10	2976	3147	2768	2768
   q11	583	516	498	498
   q12	683	803	601	601
   q13	3496	3859	3402	3402
   q14	273	298	279	279
   q15	513	475	482	475
   q16	659	683	672	672
   q17	1157	1557	1395	1395
   q18	7732	7604	7413	7413
   q19	847	842	868	842
   q20	1935	1988	1856	1856
   q21	5299	4879	4728	4728
   q22	1063	1039	1026	1026
   Total cold run time: 52761 ms
   Total hot run time: 50808 ms
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](transaction) remove visible rowset from memory during deletion transaction [doris]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #50066:
URL: https://github.com/apache/doris/pull/50066#issuecomment-2805420906

   PR approved by anyone and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](transaction) remove visible rowset from memory during deletion transaction [doris]

Posted by "sollhui (via GitHub)" <gi...@apache.org>.
sollhui closed pull request #50066: [fix](transaction) remove visible rowset from memory during deletion transaction
URL: https://github.com/apache/doris/pull/50066


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org