You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "Gabriel39 (via GitHub)" <gi...@apache.org> on 2024/07/22 06:30:37 UTC

[PR] [conf](parallel) Reduce parallel tasks for large cluster [doris]

Gabriel39 opened a new pull request, #38196:
URL: https://github.com/apache/doris/pull/38196

   ## Proposed changes
   
   Issue Number: close #xxx
   
   <!--Describe your changes.-->
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [conf](parallel) Reduce parallel tasks for large cluster [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #38196:
URL: https://github.com/apache/doris/pull/38196#issuecomment-2242293199

   
   <details>
   <summary>TPC-DS: <b>Total hot run time: 173913 ms</b></summary>
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
   TPC-DS sf100 test result on commit 03c2d2d18544b8eb5cc45a564dd67829278c2c74, data reload: false
   
   query1	926	373	369	369
   query2	6432	1966	1861	1861
   query3	6638	210	224	210
   query4	21173	17619	17568	17568
   query5	3770	479	496	479
   query6	256	188	180	180
   query7	4594	292	286	286
   query8	250	194	195	194
   query9	8870	2454	2435	2435
   query10	435	304	277	277
   query11	11019	10243	10140	10140
   query12	114	88	81	81
   query13	1648	372	369	369
   query14	10389	7512	7773	7512
   query15	213	163	165	163
   query16	7655	494	458	458
   query17	1568	558	529	529
   query18	1847	277	283	277
   query19	202	150	161	150
   query20	91	84	82	82
   query21	197	127	126	126
   query22	4405	4203	4219	4203
   query23	34280	33454	33727	33454
   query24	11068	2977	2935	2935
   query25	656	410	399	399
   query26	1116	157	161	157
   query27	2496	273	280	273
   query28	7092	2054	2048	2048
   query29	910	631	657	631
   query30	256	155	149	149
   query31	955	771	746	746
   query32	102	52	58	52
   query33	733	334	335	334
   query34	956	491	502	491
   query35	885	761	754	754
   query36	1148	992	956	956
   query37	158	94	80	80
   query38	2907	2860	2830	2830
   query39	932	897	847	847
   query40	195	118	119	118
   query41	47	44	43	43
   query42	113	98	100	98
   query43	494	465	475	465
   query44	1156	713	721	713
   query45	190	164	165	164
   query46	1099	755	732	732
   query47	1884	1781	1821	1781
   query48	379	295	285	285
   query49	836	401	410	401
   query50	775	387	390	387
   query51	6823	6746	6709	6709
   query52	99	93	88	88
   query53	363	288	281	281
   query54	895	445	439	439
   query55	74	73	72	72
   query56	293	264	269	264
   query57	1165	1068	1025	1025
   query58	256	246	258	246
   query59	3050	2604	2689	2604
   query60	293	280	267	267
   query61	128	94	97	94
   query62	809	648	648	648
   query63	318	282	281	281
   query64	9445	2248	1684	1684
   query65	3162	3089	3144	3089
   query66	704	323	324	323
   query67	15516	14961	15137	14961
   query68	4444	527	522	522
   query69	464	332	342	332
   query70	1213	1086	1064	1064
   query71	407	280	265	265
   query72	7062	5094	5794	5094
   query73	752	322	320	320
   query74	6166	5668	5649	5649
   query75	3366	2698	2727	2698
   query76	2725	967	959	959
   query77	454	318	316	316
   query78	10656	8982	9418	8982
   query79	2587	514	509	509
   query80	2641	518	467	467
   query81	598	221	223	221
   query82	794	134	135	134
   query83	309	164	168	164
   query84	273	85	87	85
   query85	2192	322	296	296
   query86	495	328	319	319
   query87	3310	3139	3101	3101
   query88	4115	2449	2472	2449
   query89	484	383	389	383
   query90	1924	197	193	193
   query91	126	102	99	99
   query92	62	52	51	51
   query93	2447	497	486	486
   query94	1292	294	311	294
   query95	407	325	324	324
   query96	639	281	273	273
   query97	3175	3060	3060	3060
   query98	216	199	197	197
   query99	1547	1265	1260	1260
   Total cold run time: 277245 ms
   Total hot run time: 173913 ms
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [conf](parallel) Reduce parallel tasks for large cluster [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #38196:
URL: https://github.com/apache/doris/pull/38196#issuecomment-2242302554

   
   <details>
   <summary>ClickBench: <b>Total hot run time: 30.69 s</b></summary>
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
   ClickBench test result on commit 03c2d2d18544b8eb5cc45a564dd67829278c2c74, data reload: false
   
   query1	0.04	0.03	0.03
   query2	0.08	0.04	0.04
   query3	0.23	0.06	0.06
   query4	1.68	0.10	0.09
   query5	0.50	0.48	0.49
   query6	1.12	0.73	0.73
   query7	0.02	0.01	0.02
   query8	0.06	0.04	0.04
   query9	0.54	0.50	0.48
   query10	0.53	0.53	0.54
   query11	0.16	0.11	0.11
   query12	0.15	0.13	0.13
   query13	0.60	0.59	0.59
   query14	0.77	0.76	0.78
   query15	0.87	0.82	0.81
   query16	0.36	0.36	0.36
   query17	1.03	0.98	1.03
   query18	0.23	0.22	0.22
   query19	1.88	1.67	1.84
   query20	0.02	0.01	0.01
   query21	15.39	0.74	0.65
   query22	4.32	6.71	2.11
   query23	18.28	1.44	1.20
   query24	2.07	0.24	0.22
   query25	0.15	0.08	0.08
   query26	0.30	0.22	0.21
   query27	0.45	0.23	0.22
   query28	13.29	1.02	0.99
   query29	12.64	3.38	3.36
   query30	0.25	0.07	0.06
   query31	2.87	0.40	0.38
   query32	3.26	0.47	0.47
   query33	2.90	2.93	2.86
   query34	16.99	4.32	4.33
   query35	4.46	4.44	4.39
   query36	0.65	0.47	0.46
   query37	0.19	0.16	0.17
   query38	0.14	0.15	0.14
   query39	0.04	0.04	0.03
   query40	0.15	0.13	0.12
   query41	0.09	0.04	0.05
   query42	0.05	0.05	0.05
   query43	0.04	0.04	0.04
   Total cold run time: 109.84 s
   Total hot run time: 30.69 s
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [conf](parallel) Reduce parallel tasks for large cluster [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #38196:
URL: https://github.com/apache/doris/pull/38196#issuecomment-2242274408

   
   <details>
   <summary>TPC-H: <b>Total hot run time: 39940 ms</b></summary>
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
   Tpch sf100 test result on commit 03c2d2d18544b8eb5cc45a564dd67829278c2c74, data reload: false
   
   ------ Round 1 ----------------------------------
   q1	17623	4534	4259	4259
   q2	2005	189	186	186
   q3	10459	1250	1139	1139
   q4	10187	778	915	778
   q5	7617	2803	2628	2628
   q6	219	136	137	136
   q7	956	597	607	597
   q8	9213	2090	2115	2090
   q9	8756	6600	6589	6589
   q10	8883	3782	3720	3720
   q11	473	236	243	236
   q12	418	229	229	229
   q13	17780	2968	2986	2968
   q14	286	226	242	226
   q15	522	491	482	482
   q16	519	389	377	377
   q17	969	681	754	681
   q18	8096	7449	7408	7408
   q19	7802	1464	1446	1446
   q20	696	333	325	325
   q21	4847	3155	3172	3155
   q22	348	293	285	285
   Total cold run time: 118674 ms
   Total hot run time: 39940 ms
   
   ----- Round 2, with runtime_filter_mode=off -----
   q1	4362	4298	4277	4277
   q2	373	261	257	257
   q3	3035	2912	3002	2912
   q4	2009	1673	1726	1673
   q5	5578	5527	5480	5480
   q6	223	137	141	137
   q7	2307	1852	1873	1852
   q8	3262	3436	3447	3436
   q9	8788	8880	8871	8871
   q10	4197	3779	3890	3779
   q11	597	490	505	490
   q12	827	662	645	645
   q13	17004	3110	3204	3110
   q14	318	282	282	282
   q15	548	494	490	490
   q16	498	430	434	430
   q17	1837	1528	1493	1493
   q18	8107	8012	7879	7879
   q19	1816	1511	1523	1511
   q20	2130	1887	1885	1885
   q21	10067	4876	4905	4876
   q22	588	498	504	498
   Total cold run time: 78471 ms
   Total hot run time: 56263 ms
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [conf](parallel) Reduce parallel tasks for large cluster [doris]

Posted by "Gabriel39 (via GitHub)" <gi...@apache.org>.
Gabriel39 merged PR #38196:
URL: https://github.com/apache/doris/pull/38196


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [conf](parallel) Reduce parallel tasks for large cluster [doris]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #38196:
URL: https://github.com/apache/doris/pull/38196#issuecomment-2242214522

   PR approved by anyone and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [conf](parallel) Reduce parallel tasks for large cluster [doris]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #38196:
URL: https://github.com/apache/doris/pull/38196#issuecomment-2242214479

   PR approved by at least one committer and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [conf](parallel) Reduce parallel tasks for large cluster [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #38196:
URL: https://github.com/apache/doris/pull/38196#issuecomment-2242188627

   Thank you for your contribution to Apache Doris.
   Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR)
   
   Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website).
   See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [conf](parallel) Reduce parallel tasks for large cluster [doris]

Posted by "Gabriel39 (via GitHub)" <gi...@apache.org>.
Gabriel39 commented on PR #38196:
URL: https://github.com/apache/doris/pull/38196#issuecomment-2242188694

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org