You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "zhiqiang-hhhh (via GitHub)" <gi...@apache.org> on 2025/04/09 13:12:21 UTC

[PR] [fix](profile) Fix incorrect check of Profile.shouldStoreToStorage [doris]

zhiqiang-hhhh opened a new pull request, #49925:
URL: https://github.com/apache/doris/pull/49925

   ### What problem does this PR solve?
   
   If profile collection is finished, we should write it to storage immediately.
   
   Add unit test for class Profile.
   
   Bug is introduced by https://github.com/apache/doris/pull/48653
   
   Issue Number: close #xxx
   
   Related PR: https://github.com/apache/doris/pull/48653
   
   Problem Summary:
   
   ### Release note
   
   None
   
   ### Check List (For Author)
   
   - Test <!-- At least one of them must be included. -->
       - [ ] Regression test
       - [ ] Unit Test
       - [ ] Manual test (add detailed scripts or steps below)
       - [ ] No need to test or manual test. Explain why:
           - [ ] This is a refactor/code format and no logic has been changed.
           - [ ] Previous test can cover this change.
           - [ ] No code files have been changed.
           - [ ] Other reason <!-- Add your reason?  -->
   
   - Behavior changed:
       - [ ] No.
       - [ ] Yes. <!-- Explain the behavior change -->
   
   - Does this need documentation?
       - [ ] No.
       - [ ] Yes. <!-- Add document PR link here. eg: https://github.com/apache/doris-website/pull/1214 -->
   
   ### Check List (For Reviewer who merge this PR)
   
   - [ ] Confirm the release note
   - [ ] Confirm test cases
   - [ ] Confirm document
   - [ ] Add branch pick label <!-- Add branch pick label that this PR should merge into -->
   
   This pull request includes changes to the `Profile` class and its associated test suite to enhance the functionality for determining when a profile should be stored to storage. The most important changes include adding a new condition to check if execution profiles are completed before storing, and the addition of a comprehensive test suite for the `Profile` class.
   
   Enhancements to `Profile` class:
   
   * Added logic to `shouldStoreToStorage` method to check if all execution profiles are completed before deciding to store the profile. [[1]](diffhunk://#diff-c9f83a512334f9031efeea62a2121286ce58e16b7f14aaee354d42e094a594abR453-R465) [[2]](diffhunk://#diff-c9f83a512334f9031efeea62a2121286ce58e16b7f14aaee354d42e094a594abL468-R481)
   
   Addition of test suite for `Profile` class:
   
   * Created `ProfileTest` class with multiple test cases to verify the behavior of `Profile` methods, including `shouldStoreToStorage`, `writeToStorage`, `readFromStorage`, and `deleteFromStorage`.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](profile) Fix incorrect check of Profile.shouldStoreToStorage [doris]

Posted by "zhiqiang-hhhh (via GitHub)" <gi...@apache.org>.
zhiqiang-hhhh commented on PR #49925:
URL: https://github.com/apache/doris/pull/49925#issuecomment-2789722499

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](profile) Fix incorrect check of Profile.shouldStoreToStorage [doris]

Posted by "zhiqiang-hhhh (via GitHub)" <gi...@apache.org>.
zhiqiang-hhhh commented on PR #49925:
URL: https://github.com/apache/doris/pull/49925#issuecomment-2789659464

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](profile) Fix incorrect check of Profile.shouldStoreToStorage [doris]

Posted by "zhiqiang-hhhh (via GitHub)" <gi...@apache.org>.
zhiqiang-hhhh commented on PR #49925:
URL: https://github.com/apache/doris/pull/49925#issuecomment-2791327442

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](profile) Fix incorrect check of Profile.shouldStoreToStorage [doris]

Posted by "BiteTheDDDDt (via GitHub)" <gi...@apache.org>.
BiteTheDDDDt merged PR #49925:
URL: https://github.com/apache/doris/pull/49925


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](profile) Fix incorrect check of Profile.shouldStoreToStorage [doris]

Posted by "Thearas (via GitHub)" <gi...@apache.org>.
Thearas commented on PR #49925:
URL: https://github.com/apache/doris/pull/49925#issuecomment-2789659223

   
   Thank you for your contribution to Apache Doris.
   Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR).
   
   Please clearly describe your PR:
   1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
   2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
   3. What features were added. Why was this function added?
   4. Which code was refactored and why was this part of the code refactored?
   5. Which functions were optimized and what is the difference before and after the optimization?
       


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](profile) Fix incorrect check of Profile.shouldStoreToStorage [doris]

Posted by "zhiqiang-hhhh (via GitHub)" <gi...@apache.org>.
zhiqiang-hhhh commented on PR #49925:
URL: https://github.com/apache/doris/pull/49925#issuecomment-2791469945

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](profile) Fix incorrect check of Profile.shouldStoreToStorage [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #49925:
URL: https://github.com/apache/doris/pull/49925#issuecomment-2791519716

   
   <details>
   <summary>ClickBench: <b>Total hot run time: 30.99 s</b></summary>
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
   ClickBench test result on commit 6c93a4c55be88ce4529758ac27f32e88f82e7f1f, data reload: false
   
   query1	0.04	0.04	0.02
   query2	0.12	0.11	0.11
   query3	0.24	0.19	0.19
   query4	1.60	0.18	0.20
   query5	0.59	0.58	0.58
   query6	1.20	0.70	0.72
   query7	0.03	0.02	0.01
   query8	0.04	0.03	0.03
   query9	0.58	0.52	0.51
   query10	0.56	0.58	0.57
   query11	0.15	0.11	0.11
   query12	0.14	0.11	0.11
   query13	0.61	0.60	0.59
   query14	2.69	2.82	2.82
   query15	0.93	0.85	0.85
   query16	0.40	0.38	0.39
   query17	1.02	1.04	1.06
   query18	0.22	0.20	0.20
   query19	1.93	1.98	1.79
   query20	0.01	0.01	0.01
   query21	15.36	0.90	0.54
   query22	0.75	1.21	0.65
   query23	14.89	1.38	0.60
   query24	7.03	1.69	0.56
   query25	0.50	0.18	0.20
   query26	0.68	0.15	0.15
   query27	0.06	0.05	0.05
   query28	9.92	0.86	0.43
   query29	12.54	4.07	3.36
   query30	0.25	0.08	0.06
   query31	2.82	0.61	0.38
   query32	3.23	0.55	0.48
   query33	3.07	3.00	3.07
   query34	15.70	5.10	4.48
   query35	4.52	4.52	4.47
   query36	0.66	0.50	0.49
   query37	0.09	0.06	0.06
   query38	0.05	0.04	0.03
   query39	0.03	0.02	0.02
   query40	0.17	0.16	0.13
   query41	0.08	0.02	0.03
   query42	0.03	0.02	0.02
   query43	0.03	0.04	0.03
   Total cold run time: 105.56 s
   Total hot run time: 30.99 s
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](profile) Fix incorrect check of Profile.shouldStoreToStorage [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #49925:
URL: https://github.com/apache/doris/pull/49925#issuecomment-2791501129

   
   <details>
   <summary>TPC-H: <b>Total hot run time: 34378 ms</b></summary>
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
   Tpch sf100 test result on commit 6c93a4c55be88ce4529758ac27f32e88f82e7f1f, data reload: false
   
   ------ Round 1 ----------------------------------
   q1	25968	5081	5063	5063
   q2	2067	277	187	187
   q3	10392	1204	696	696
   q4	10236	998	544	544
   q5	7579	2292	2337	2292
   q6	180	159	131	131
   q7	885	729	625	625
   q8	9310	1229	1017	1017
   q9	6948	5095	4987	4987
   q10	6819	2285	1889	1889
   q11	489	295	275	275
   q12	353	349	215	215
   q13	17796	3668	3089	3089
   q14	245	228	208	208
   q15	536	485	499	485
   q16	632	620	602	602
   q17	613	849	380	380
   q18	7970	7445	7459	7445
   q19	1225	936	564	564
   q20	347	341	232	232
   q21	4517	3498	2482	2482
   q22	1039	1067	970	970
   Total cold run time: 116146 ms
   Total hot run time: 34378 ms
   
   ----- Round 2, with runtime_filter_mode=off -----
   q1	5140	5057	5095	5057
   q2	231	324	234	234
   q3	2116	2623	2245	2245
   q4	1427	1811	1491	1491
   q5	4461	4384	4326	4326
   q6	218	168	130	130
   q7	1952	1937	1752	1752
   q8	2553	2427	2554	2427
   q9	7283	7139	7019	7019
   q10	2932	3163	2733	2733
   q11	573	519	512	512
   q12	670	747	608	608
   q13	3451	3806	3307	3307
   q14	288	286	265	265
   q15	515	475	474	474
   q16	640	670	649	649
   q17	1142	1506	1395	1395
   q18	7723	7491	7536	7491
   q19	808	870	961	870
   q20	1892	1954	1797	1797
   q21	5303	4846	4713	4713
   q22	1076	1036	1010	1010
   Total cold run time: 52394 ms
   Total hot run time: 50505 ms
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](profile) Fix incorrect check of Profile.shouldStoreToStorage [doris]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #49925:
URL: https://github.com/apache/doris/pull/49925#issuecomment-2796179805

   PR approved by at least one committer and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](profile) Fix incorrect check of Profile.shouldStoreToStorage [doris]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #49925:
URL: https://github.com/apache/doris/pull/49925#issuecomment-2796179882

   PR approved by anyone and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](profile) Fix incorrect check of Profile.shouldStoreToStorage [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #49925:
URL: https://github.com/apache/doris/pull/49925#issuecomment-2791513702

   
   <details>
   <summary>TPC-DS: <b>Total hot run time: 193060 ms</b></summary>
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
   TPC-DS sf100 test result on commit 6c93a4c55be88ce4529758ac27f32e88f82e7f1f, data reload: false
   
   query1	1408	1083	1059	1059
   query2	6004	1949	1942	1942
   query3	10997	4554	4526	4526
   query4	25462	23678	23375	23375
   query5	5266	622	448	448
   query6	332	211	208	208
   query7	3992	493	281	281
   query8	315	251	245	245
   query9	8526	2561	2567	2561
   query10	517	333	264	264
   query11	15257	14971	14784	14784
   query12	154	123	101	101
   query13	1546	501	370	370
   query14	9198	6205	6006	6006
   query15	204	183	166	166
   query16	7607	644	461	461
   query17	1135	773	592	592
   query18	2035	422	335	335
   query19	206	194	173	173
   query20	126	128	126	126
   query21	210	129	116	116
   query22	4543	4636	4274	4274
   query23	34880	33490	33665	33490
   query24	8667	2508	2448	2448
   query25	528	489	424	424
   query26	1209	284	151	151
   query27	3118	510	344	344
   query28	4688	2452	2446	2446
   query29	710	562	448	448
   query30	277	229	198	198
   query31	909	856	830	830
   query32	86	63	62	62
   query33	553	371	313	313
   query34	805	874	527	527
   query35	855	866	756	756
   query36	967	1004	900	900
   query37	118	107	80	80
   query38	4153	4204	4214	4204
   query39	1529	1482	1511	1482
   query40	216	125	110	110
   query41	54	54	52	52
   query42	125	122	111	111
   query43	516	520	483	483
   query44	1309	812	829	812
   query45	196	180	175	175
   query46	849	1004	644	644
   query47	1912	1873	1795	1795
   query48	390	415	327	327
   query49	784	537	426	426
   query50	687	709	420	420
   query51	4290	4251	4172	4172
   query52	108	105	102	102
   query53	238	278	199	199
   query54	587	595	523	523
   query55	83	82	81	81
   query56	339	295	296	295
   query57	1181	1200	1109	1109
   query58	284	278	264	264
   query59	2868	2964	2775	2775
   query60	352	331	311	311
   query61	140	130	127	127
   query62	771	749	703	703
   query63	233	190	185	185
   query64	4126	1062	726	726
   query65	4430	4372	4454	4372
   query66	1101	435	311	311
   query67	15945	15792	15203	15203
   query68	9046	879	515	515
   query69	504	312	262	262
   query70	1238	1090	1101	1090
   query71	478	315	297	297
   query72	5554	4729	4745	4729
   query73	708	591	350	350
   query74	9005	9039	8975	8975
   query75	4332	3222	2666	2666
   query76	3788	1172	748	748
   query77	794	382	274	274
   query78	10085	9955	9348	9348
   query79	2670	798	566	566
   query80	663	502	457	457
   query81	502	267	225	225
   query82	534	126	98	98
   query83	279	255	242	242
   query84	301	112	86	86
   query85	832	356	315	315
   query86	368	293	303	293
   query87	4448	4494	4274	4274
   query88	3523	2212	2232	2212
   query89	393	309	287	287
   query90	1925	212	213	212
   query91	144	146	111	111
   query92	79	57	59	57
   query93	1893	950	583	583
   query94	665	420	278	278
   query95	441	298	286	286
   query96	484	563	269	269
   query97	3154	3234	3122	3122
   query98	244	212	197	197
   query99	1465	1407	1307	1307
   Total cold run time: 283745 ms
   Total hot run time: 193060 ms
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org