You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "Jibing-Li (via GitHub)" <gi...@apache.org> on 2024/02/25 08:21:06 UTC

[PR] [improvement](statistics)Remove useless async loader code. [doris]

Jibing-Li opened a new pull request, #31380:
URL: https://github.com/apache/doris/pull/31380

   ## Proposed changes
   
   Issue Number: close #xxx
   
   <!--Describe your changes.-->
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [improvement](statistics)Remove useless async loader code. [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31380:
URL: https://github.com/apache/doris/pull/31380#issuecomment-1962950587

   
   <details>
   <summary>TPC-DS: <b>Total hot run time: 175868 ms</b></summary>
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
   TPC-DS sf100 test result on commit 2d133b8dfae9c8db669a0c030e3876c99dd64852, data reload: false
   
   query1	927	350	342	342
   query2	6524	1704	1717	1704
   query3	6697	205	199	199
   query4	23433	21045	21059	21045
   query5	4313	388	390	388
   query6	280	173	165	165
   query7	4613	300	294	294
   query8	239	201	212	201
   query9	8428	2785	2791	2785
   query10	413	237	227	227
   query11	14906	14630	14406	14406
   query12	132	83	84	83
   query13	1708	416	413	413
   query14	9068	7692	7617	7617
   query15	220	188	193	188
   query16	7366	255	251	251
   query17	1400	543	530	530
   query18	1958	265	265	265
   query19	192	149	158	149
   query20	89	81	82	81
   query21	181	119	123	119
   query22	4750	4693	4699	4693
   query23	32456	31422	31372	31372
   query24	12819	3383	3421	3383
   query25	658	374	361	361
   query26	1874	159	160	159
   query27	3046	311	318	311
   query28	6617	1854	1842	1842
   query29	1170	630	622	622
   query30	287	136	143	136
   query31	943	744	760	744
   query32	97	59	56	56
   query33	733	224	244	224
   query34	1080	496	501	496
   query35	947	869	829	829
   query36	938	890	924	890
   query37	167	65	63	63
   query38	3320	3168	3207	3168
   query39	1370	1346	1320	1320
   query40	292	111	107	107
   query41	39	37	38	37
   query42	107	100	100	100
   query43	475	448	448	448
   query44	1135	687	697	687
   query45	195	181	182	181
   query46	1046	780	764	764
   query47	1666	1539	1553	1539
   query48	424	354	344	344
   query49	1209	309	307	307
   query50	785	380	377	377
   query51	4437	4344	4281	4281
   query52	106	101	91	91
   query53	393	303	307	303
   query54	289	227	226	226
   query55	87	84	76	76
   query56	222	200	208	200
   query57	1082	976	981	976
   query58	205	198	200	198
   query59	2258	2088	2102	2088
   query60	255	216	220	216
   query61	87	104	82	82
   query62	586	352	375	352
   query63	337	294	286	286
   query64	6409	3052	3129	3052
   query65	3301	3237	3238	3237
   query66	1340	327	327	327
   query67	14406	14379	13947	13947
   query68	5137	554	562	554
   query69	527	347	340	340
   query70	1254	1192	1242	1192
   query71	383	250	255	250
   query72	6397	2774	2604	2604
   query73	692	318	310	310
   query74	6826	6494	6376	6376
   query75	3212	2578	2545	2545
   query76	3242	1108	1207	1108
   query77	359	245	234	234
   query78	9448	8750	8741	8741
   query79	979	528	518	518
   query80	530	374	350	350
   query81	436	208	211	208
   query82	160	87	92	87
   query83	145	129	124	124
   query84	223	78	80	78
   query85	1013	354	341	341
   query86	295	308	303	303
   query87	3467	3304	3225	3225
   query88	2707	2291	2291	2291
   query89	435	361	370	361
   query90	1915	169	164	164
   query91	158	126	131	126
   query92	55	52	49	49
   query93	1034	526	511	511
   query94	1090	184	173	173
   query95	438	343	337	337
   query96	592	272	268	268
   query97	4453	4278	4278	4278
   query98	214	207	200	200
   query99	1097	738	672	672
   Total cold run time: 268445 ms
   Total hot run time: 175868 ms
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [improvement](statistics)Remove useless async loader code. [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31380:
URL: https://github.com/apache/doris/pull/31380#issuecomment-1962947172

   
   <details>
   <summary>TPC-H: <b>Total hot run time: 40971 ms</b></summary>
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
   Tpch sf100 test result on commit 2d133b8dfae9c8db669a0c030e3876c99dd64852, data reload: false
   
   ------ Round 1 ----------------------------------
   q1	17644	4927	4848	4848
   q2	2033	136	132	132
   q3	10599	1012	1000	1000
   q4	4651	977	972	972
   q5	7627	3122	3186	3122
   q6	190	138	132	132
   q7	1253	779	763	763
   q8	9271	2051	2045	2045
   q9	7443	6535	6576	6535
   q10	8294	2648	2637	2637
   q11	414	201	221	201
   q12	782	339	336	336
   q13	17920	3579	3592	3579
   q14	286	264	255	255
   q15	570	540	528	528
   q16	464	417	407	407
   q17	911	876	888	876
   q18	7508	6667	6585	6585
   q19	1551	1499	1498	1498
   q20	556	275	264	264
   q21	6933	3942	3925	3925
   q22	857	337	331	331
   Total cold run time: 107757 ms
   Total hot run time: 40971 ms
   
   ----- Round 2, with runtime_filter_mode=off -----
   q1	4809	4799	4809	4799
   q2	292	183	185	183
   q3	3575	3565	3579	3565
   q4	2529	2486	2480	2480
   q5	5723	5723	5731	5723
   q6	212	128	122	122
   q7	2219	1631	1656	1631
   q8	2975	3046	3085	3046
   q9	8657	8655	8656	8655
   q10	6767	4225	4211	4211
   q11	498	370	370	370
   q12	764	536	531	531
   q13	4417	3384	3423	3384
   q14	267	236	226	226
   q15	588	498	490	490
   q16	481	439	429	429
   q17	1644	1597	1596	1596
   q18	8306	7857	7843	7843
   q19	1634	1649	1631	1631
   q20	2102	1834	1820	1820
   q21	6559	6153	6133	6133
   q22	557	494	509	494
   Total cold run time: 65575 ms
   Total hot run time: 59362 ms
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [improvement](statistics)Remove useless async loader code. [doris]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #31380:
URL: https://github.com/apache/doris/pull/31380#issuecomment-1962871295

   PR approved by at least one committer and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [improvement](statistics)Remove useless async loader code. [doris]

Posted by "Jibing-Li (via GitHub)" <gi...@apache.org>.
Jibing-Li commented on PR #31380:
URL: https://github.com/apache/doris/pull/31380#issuecomment-1962854400

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [improvement](statistics)Remove useless async loader code. [doris]

Posted by "Jibing-Li (via GitHub)" <gi...@apache.org>.
Jibing-Li commented on PR #31380:
URL: https://github.com/apache/doris/pull/31380#issuecomment-1962938163

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [improvement](statistics)Remove useless async loader code. [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31380:
URL: https://github.com/apache/doris/pull/31380#issuecomment-1962952122

   
   <details>
   <summary>ClickBench: <b>Total hot run time: 30.98 s</b></summary>
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
   ClickBench test result on commit 2d133b8dfae9c8db669a0c030e3876c99dd64852, data reload: false
   
   query1	0.03	0.02	0.02
   query2	0.07	0.03	0.03
   query3	0.23	0.07	0.07
   query4	1.66	0.09	0.09
   query5	0.49	0.48	0.48
   query6	1.36	0.62	0.62
   query7	0.02	0.01	0.01
   query8	0.03	0.02	0.03
   query9	0.53	0.46	0.46
   query10	0.50	0.49	0.50
   query11	0.13	0.10	0.10
   query12	0.12	0.10	0.10
   query13	0.59	0.59	0.58
   query14	0.78	0.80	0.78
   query15	0.82	0.79	0.78
   query16	0.34	0.35	0.33
   query17	0.90	0.92	0.91
   query18	0.19	0.19	0.17
   query19	1.75	1.72	1.64
   query20	0.01	0.01	0.02
   query21	15.43	0.66	0.59
   query22	2.74	3.67	2.38
   query23	17.23	1.06	1.11
   query24	2.19	0.32	0.32
   query25	0.66	0.07	0.04
   query26	0.15	0.12	0.14
   query27	0.05	0.06	0.05
   query28	12.17	0.83	0.83
   query29	12.54	3.32	3.24
   query30	0.51	0.49	0.48
   query31	2.76	0.36	0.36
   query32	3.36	0.47	0.48
   query33	3.15	3.17	3.14
   query34	15.37	4.52	4.48
   query35	4.50	4.50	4.48
   query36	1.07	0.95	0.98
   query37	0.07	0.06	0.06
   query38	0.04	0.03	0.03
   query39	0.02	0.02	0.02
   query40	0.17	0.15	0.15
   query41	0.07	0.02	0.01
   query42	0.02	0.02	0.01
   query43	0.02	0.02	0.02
   Total cold run time: 104.84 s
   Total hot run time: 30.98 s
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [improvement](statistics)Remove useless async loader code. [doris]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #31380:
URL: https://github.com/apache/doris/pull/31380#issuecomment-1962871307

   PR approved by anyone and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [improvement](statistics)Remove useless async loader code. [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31380:
URL: https://github.com/apache/doris/pull/31380#issuecomment-1962953052

   
   Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   ```
   Load test result on commit 2d133b8dfae9c8db669a0c030e3876c99dd64852 with default session variables
   Stream load json:         19 seconds loaded 2358488459 Bytes, about 118 MB/s
   Stream load orc:          60 seconds loaded 1101869774 Bytes, about 17 MB/s
   Stream load parquet:      32 seconds loaded 861443392 Bytes, about 25 MB/s
   Insert into select:       15.1 seconds inserted 10000000 Rows, about 662K ops/s
   ```
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [improvement](statistics)Remove useless async loader code. [doris]

Posted by "morrySnow (via GitHub)" <gi...@apache.org>.
morrySnow merged PR #31380:
URL: https://github.com/apache/doris/pull/31380


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [improvement](statistics)Remove useless async loader code. [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31380:
URL: https://github.com/apache/doris/pull/31380#issuecomment-1962853920

   Thank you for your contribution to Apache Doris.
   Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org