You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "Mryange (via GitHub)" <gi...@apache.org> on 2025/12/26 03:06:52 UTC

[PR] [fix](profile) Fix inaccurate accounting of memory_used_counter in ExchangeSink [doris]

Mryange opened a new pull request, #59374:
URL: https://github.com/apache/doris/pull/59374

   ### What problem does this PR solve?
   
   Previously, when increasing memory usage the code did not account for the channels' memory, but it did include it when freeing memory.
   
   
   ### Release note
   
   None
   
   ### Check List (For Author)
   
   - Test <!-- At least one of them must be included. -->
       - [ ] Regression test
       - [ ] Unit Test
       - [ ] Manual test (add detailed scripts or steps below)
       - [ ] No need to test or manual test. Explain why:
           - [ ] This is a refactor/code format and no logic has been changed.
           - [ ] Previous test can cover this change.
           - [ ] No code files have been changed.
           - [ ] Other reason <!-- Add your reason?  -->
   
   - Behavior changed:
       - [ ] No.
       - [ ] Yes. <!-- Explain the behavior change -->
   
   - Does this need documentation?
       - [ ] No.
       - [ ] Yes. <!-- Add document PR link here. eg: https://github.com/apache/doris-website/pull/1214 -->
   
   ### Check List (For Reviewer who merge this PR)
   
   - [ ] Confirm the release note
   - [ ] Confirm test cases
   - [ ] Confirm document
   - [ ] Add branch pick label <!-- Add branch pick label that this PR should merge into -->
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](profile) Fix inaccurate accounting of memory_used_counter in ExchangeSink [doris]

Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #59374:
URL: https://github.com/apache/doris/pull/59374#issuecomment-3692052580

   # BE UT Coverage Report
   Increment line coverage `100.00% (12/12)` :tada:
   
   [Increment coverage report](http://coverage.selectdb-in.cc/coverage/d993a41e4d5a2d3d829b99339131d1f14f675092_d993a41e4d5a2d3d829b99339131d1f14f675092/increment_report/index.html)
   [Complete coverage report](http://coverage.selectdb-in.cc/coverage/d993a41e4d5a2d3d829b99339131d1f14f675092_d993a41e4d5a2d3d829b99339131d1f14f675092/report/index.html)
   | Category          | Coverage           |
   |-------------------|--------------------|
   | Function Coverage | 53.40% (18952/35489) |
   | Line Coverage     | 39.27% (175803/447680) |
   | Region Coverage   | 33.85% (136069/402007) |
   | Branch Coverage   | 34.76% (58731/168940) |


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](profile) Fix inaccurate accounting of memory_used_counter in ExchangeSink [doris]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #59374:
URL: https://github.com/apache/doris/pull/59374#issuecomment-3695261962

   PR approved by at least one committer and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](profile) Fix inaccurate accounting of memory_used_counter in ExchangeSink [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59374:
URL: https://github.com/apache/doris/pull/59374#issuecomment-3692334369

   
   <details>
   <summary>TPC-DS: <b>Total hot run time: 180172 ms</b></summary>
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
   TPC-DS sf100 test result on commit d993a41e4d5a2d3d829b99339131d1f14f675092, data reload: false
   
   query5	5011	598	454	454
   query6	338	253	233	233
   query7	4235	470	285	285
   query8	309	257	260	257
   query9	8799	2525	2521	2521
   query10	546	381	335	335
   query11	15743	15315	14844	14844
   query12	189	122	116	116
   query13	1264	500	393	393
   query14	6421	3000	2788	2788
   query14_1	2692	2655	2679	2655
   query15	210	194	178	178
   query16	912	484	441	441
   query17	1134	712	617	617
   query18	2687	443	354	354
   query19	236	235	211	211
   query20	125	117	114	114
   query21	224	139	116	116
   query22	4132	4179	4074	4074
   query23	16664	16257	16002	16002
   query23_1	16121	16099	16155	16099
   query24	7342	1669	1249	1249
   query24_1	1249	1213	1264	1213
   query25	585	510	445	445
   query26	1254	277	171	171
   query27	2732	459	311	311
   query28	4472	2113	2098	2098
   query29	848	591	504	504
   query30	316	233	220	220
   query31	814	721	618	618
   query32	87	69	78	69
   query33	546	349	324	324
   query34	903	915	537	537
   query35	753	806	705	705
   query36	880	922	827	827
   query37	126	93	77	77
   query38	2981	3081	2986	2986
   query39	750	769	716	716
   query39_1	697	716	699	699
   query40	230	135	118	118
   query41	65	63	62	62
   query42	105	104	106	104
   query43	429	445	406	406
   query44	1341	746	740	740
   query45	187	188	180	180
   query46	873	961	616	616
   query47	1694	1719	1619	1619
   query48	312	331	243	243
   query49	638	428	393	393
   query50	668	299	220	220
   query51	3861	3795	3839	3795
   query52	105	114	103	103
   query53	316	344	286	286
   query54	283	301	255	255
   query55	79	76	70	70
   query56	305	304	285	285
   query57	1194	1171	1082	1082
   query58	279	256	254	254
   query59	2388	2604	2354	2354
   query60	309	308	285	285
   query61	167	156	159	156
   query62	761	686	663	663
   query63	329	293	299	293
   query64	4979	1296	988	988
   query65	3998	3952	3964	3952
   query66	1400	437	314	314
   query67	15284	15164	14955	14955
   query68	6916	1001	719	719
   query69	507	345	310	310
   query70	1082	991	976	976
   query71	365	303	286	286
   query72	6065	5062	5113	5062
   query73	715	649	307	307
   query74	8968	8904	8678	8678
   query75	3206	3154	2853	2853
   query76	3898	1146	750	750
   query77	535	387	295	295
   query78	9518	9712	8863	8863
   query79	1261	867	621	621
   query80	1439	678	573	573
   query81	545	266	233	233
   query82	400	134	104	104
   query83	273	259	241	241
   query84	259	124	99	99
   query85	911	497	457	457
   query86	333	285	276	276
   query87	3222	3182	3207	3182
   query88	3272	2310	2307	2307
   query89	484	434	401	401
   query90	1934	169	156	156
   query91	175	172	145	145
   query92	71	69	74	69
   query93	1109	899	553	553
   query94	557	306	266	266
   query95	567	389	302	302
   query96	600	459	211	211
   query97	2298	2279	2253	2253
   query98	212	193	193	193
   query99	1321	1338	1315	1315
   Total cold run time: 260866 ms
   Total hot run time: 180172 ms
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](profile) Fix inaccurate accounting of memory_used_counter in ExchangeSink [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59374:
URL: https://github.com/apache/doris/pull/59374#issuecomment-3692341251

   
   <details>
   <summary>ClickBench: <b>Total hot run time: 27.23 s</b></summary>
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
   ClickBench test result on commit d993a41e4d5a2d3d829b99339131d1f14f675092, data reload: false
   
   query1	0.08	0.05	0.05
   query2	0.10	0.05	0.05
   query3	0.26	0.09	0.08
   query4	1.60	0.11	0.11
   query5	0.27	0.25	0.26
   query6	1.19	0.64	0.64
   query7	0.03	0.02	0.02
   query8	0.05	0.04	0.04
   query9	0.56	0.50	0.50
   query10	0.55	0.55	0.53
   query11	0.15	0.11	0.11
   query12	0.16	0.11	0.11
   query13	0.61	0.60	0.60
   query14	0.99	0.99	0.99
   query15	0.82	0.79	0.81
   query16	0.39	0.39	0.40
   query17	0.98	1.08	1.02
   query18	0.22	0.21	0.22
   query19	1.93	1.85	1.76
   query20	0.02	0.02	0.02
   query21	15.42	0.27	0.15
   query22	4.79	0.05	0.04
   query23	16.06	0.30	0.10
   query24	0.96	0.28	0.48
   query25	0.08	0.08	0.06
   query26	0.15	0.14	0.13
   query27	0.07	0.07	0.04
   query28	3.79	1.20	1.03
   query29	12.63	4.09	3.19
   query30	0.28	0.14	0.13
   query31	2.82	0.62	0.39
   query32	3.24	0.55	0.45
   query33	3.03	3.03	3.10
   query34	16.93	5.20	4.54
   query35	4.57	4.56	4.59
   query36	0.65	0.49	0.51
   query37	0.11	0.07	0.06
   query38	0.07	0.04	0.03
   query39	0.05	0.03	0.03
   query40	0.18	0.14	0.14
   query41	0.09	0.04	0.03
   query42	0.04	0.03	0.04
   query43	0.05	0.03	0.04
   Total cold run time: 97.02 s
   Total hot run time: 27.23 s
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](profile) Fix inaccurate accounting of memory_used_counter in ExchangeSink [doris]

Posted by "zclllyybb (via GitHub)" <gi...@apache.org>.
zclllyybb merged PR #59374:
URL: https://github.com/apache/doris/pull/59374


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](profile) Fix inaccurate accounting of memory_used_counter in ExchangeSink [doris]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #59374:
URL: https://github.com/apache/doris/pull/59374#issuecomment-3695262000

   PR approved by anyone and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](profile) Fix inaccurate accounting of memory_used_counter in ExchangeSink [doris]

Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #59374:
URL: https://github.com/apache/doris/pull/59374#issuecomment-3692341881

   # BE Regression && UT Coverage Report
   Increment line coverage `100.00% (12/12)` :tada:
   
   [Increment coverage report](http://coverage.selectdb-in.cc/coverage/59374_d993a41e4d5a2d3d829b99339131d1f14f675092_merge/increment_report/index.html)
   [Complete coverage report](http://coverage.selectdb-in.cc/coverage/59374_d993a41e4d5a2d3d829b99339131d1f14f675092_merge/report/index.html)
   | Category          | Coverage           |
   |-------------------|--------------------|
   | Function Coverage | 72.25% (25066/34694) |
   | Line Coverage     | 58.97% (263272/446488) |
   | Region Coverage   | 53.77% (218414/406172) |
   | Branch Coverage   | 55.35% (93816/169504) |


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](profile) Fix inaccurate accounting of memory_used_counter in ExchangeSink [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59374:
URL: https://github.com/apache/doris/pull/59374#issuecomment-3692317811

   
   <details>
   <summary>TPC-H: <b>Total hot run time: 36405 ms</b></summary>
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
   Tpch sf100 test result on commit d993a41e4d5a2d3d829b99339131d1f14f675092, data reload: false
   
   ------ Round 1 ----------------------------------
   q1	17606	4307	4091	4091
   q2	2015	336	232	232
   q3	10197	1355	782	782
   q4	10221	874	318	318
   q5	7505	2102	2000	2000
   q6	194	170	135	135
   q7	997	859	741	741
   q8	9362	1432	1208	1208
   q9	6930	5309	5357	5309
   q10	6882	2395	1984	1984
   q11	565	325	299	299
   q12	650	740	562	562
   q13	17770	3686	3057	3057
   q14	291	285	271	271
   q15	591	509	506	506
   q16	703	681	628	628
   q17	687	839	507	507
   q18	7514	7610	7882	7610
   q19	1235	1028	646	646
   q20	423	377	254	254
   q21	4632	4228	4364	4228
   q22	1146	1111	1037	1037
   Total cold run time: 108116 ms
   Total hot run time: 36405 ms
   
   ----- Round 2, with runtime_filter_mode=off -----
   q1	4404	4342	4551	4342
   q2	354	424	308	308
   q3	2443	2855	2498	2498
   q4	1486	1907	1474	1474
   q5	4645	4540	4433	4433
   q6	211	161	128	128
   q7	1982	1941	1811	1811
   q8	2721	2479	2574	2479
   q9	7614	7051	6895	6895
   q10	2906	3088	2677	2677
   q11	602	513	485	485
   q12	644	728	565	565
   q13	3291	3610	3002	3002
   q14	267	274	251	251
   q15	534	504	485	485
   q16	613	664	605	605
   q17	1130	1449	1413	1413
   q18	7237	7207	7025	7025
   q19	889	839	868	839
   q20	1881	1952	1787	1787
   q21	4570	4350	4156	4156
   q22	1104	994	991	991
   Total cold run time: 51528 ms
   Total hot run time: 48649 ms
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](profile) Fix inaccurate accounting of memory_used_counter in ExchangeSink [doris]

Posted by "Thearas (via GitHub)" <gi...@apache.org>.
Thearas commented on PR #59374:
URL: https://github.com/apache/doris/pull/59374#issuecomment-3691982280

   
   Thank you for your contribution to Apache Doris.
   Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR).
   
   Please clearly describe your PR:
   1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
   2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
   3. What features were added. Why was this function added?
   4. Which code was refactored and why was this part of the code refactored?
   5. Which functions were optimized and what is the difference before and after the optimization?
       


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](profile) Fix inaccurate accounting of memory_used_counter in ExchangeSink [doris]

Posted by "Mryange (via GitHub)" <gi...@apache.org>.
Mryange commented on PR #59374:
URL: https://github.com/apache/doris/pull/59374#issuecomment-3691983133

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](profile) Fix inaccurate accounting of memory_used_counter in ExchangeSink [doris]

Posted by "Mryange (via GitHub)" <gi...@apache.org>.
Mryange commented on PR #59374:
URL: https://github.com/apache/doris/pull/59374#issuecomment-3692258684

   run performance


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org