You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "morningman (via GitHub)" <gi...@apache.org> on 2024/02/19 16:12:50 UTC

[PR] [opt](deps) reduce the size of FE dependencies [doris]

morningman opened a new pull request, #31129:
URL: https://github.com/apache/doris/pull/31129

   ## Proposed changes
   
   the `ranger-plugins-common` depends `aws-java-sdk-bundle`, which is too large and unnecessary.
   exclude it and add aws-java-sdk-logs alone. 
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [opt](deps) reduce the size of FE dependencies [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31129:
URL: https://github.com/apache/doris/pull/31129#issuecomment-1952790663

   Thank you for your contribution to Apache Doris.
   Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [opt](deps) reduce the size of FE dependencies [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31129:
URL: https://github.com/apache/doris/pull/31129#issuecomment-1952911246

   
   Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   ```
   Load test result on commit fa88d09a26040c2a42ea3bba89e9f2c38db65d9b with default session variables
   Stream load json:         19 seconds loaded 2358488459 Bytes, about 118 MB/s
   Stream load orc:          59 seconds loaded 1101869774 Bytes, about 17 MB/s
   Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
   Insert into select:       13.5 seconds inserted 10000000 Rows, about 740K ops/s
   ```
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [opt](deps) reduce the size of FE dependencies [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31129:
URL: https://github.com/apache/doris/pull/31129#issuecomment-1952899761

   
   <details>
   <summary>TPC-DS: <b>Total hot run time: 178981 ms</b></summary>
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
   TPC-DS sf100 test result on commit fa88d09a26040c2a42ea3bba89e9f2c38db65d9b, data reload: false
   
   query1	915	355	349	349
   query2	6536	1761	1658	1658
   query3	6691	206	201	201
   query4	23116	21177	21325	21177
   query5	4231	391	387	387
   query6	254	171	169	169
   query7	4607	294	292	292
   query8	255	213	202	202
   query9	8821	2880	2863	2863
   query10	416	224	225	224
   query11	15153	14779	14647	14647
   query12	162	84	84	84
   query13	1703	435	426	426
   query14	10167	8462	8817	8462
   query15	790	212	217	212
   query16	9787	314	289	289
   query17	5266	1379	557	557
   query18	3645	287	291	287
   query19	981	154	168	154
   query20	359	85	89	85
   query21	908	129	140	129
   query22	4916	4705	4698	4698
   query23	32452	31607	31501	31501
   query24	11535	3405	3508	3405
   query25	627	372	382	372
   query26	1570	156	161	156
   query27	3036	311	317	311
   query28	6495	1867	1853	1853
   query29	1096	623	621	621
   query30	275	138	145	138
   query31	921	757	760	757
   query32	95	60	56	56
   query33	749	240	250	240
   query34	1045	491	502	491
   query35	948	855	849	849
   query36	979	881	876	876
   query37	129	60	65	60
   query38	3252	3208	3143	3143
   query39	1372	1342	1322	1322
   query40	278	105	106	105
   query41	37	35	35	35
   query42	105	100	100	100
   query43	468	441	446	441
   query44	1059	698	717	698
   query45	201	189	175	175
   query46	1057	773	751	751
   query47	1599	1551	1573	1551
   query48	416	338	344	338
   query49	1095	308	305	305
   query50	783	378	372	372
   query51	5295	5149	5219	5149
   query52	107	99	92	92
   query53	393	305	293	293
   query54	308	225	234	225
   query55	84	86	77	77
   query56	227	204	198	198
   query57	1075	935	954	935
   query58	220	206	202	202
   query59	2279	2223	2198	2198
   query60	252	225	230	225
   query61	92	83	82	82
   query62	582	365	370	365
   query63	337	298	287	287
   query64	6306	3067	3149	3067
   query65	3303	3245	3258	3245
   query66	1351	330	321	321
   query67	14680	14346	14240	14240
   query68	5136	561	567	561
   query69	529	370	360	360
   query70	1226	1290	1255	1255
   query71	443	285	257	257
   query72	6343	2804	2640	2640
   query73	707	313	310	310
   query74	6909	6429	6358	6358
   query75	3227	2580	2567	2567
   query76	3290	1102	1215	1102
   query77	370	239	249	239
   query78	9452	8867	8740	8740
   query79	966	515	523	515
   query80	507	353	347	347
   query81	439	217	209	209
   query82	165	92	90	90
   query83	143	128	130	128
   query84	231	83	79	79
   query85	1033	354	343	343
   query86	304	308	312	308
   query87	3439	3322	3325	3322
   query88	2733	2274	2268	2268
   query89	452	369	368	368
   query90	2000	171	169	169
   query91	159	130	132	130
   query92	54	53	48	48
   query93	1021	500	513	500
   query94	1307	186	178	178
   query95	8639	375	8486	375
   query96	588	269	264	264
   query97	4433	4259	4268	4259
   query98	221	211	200	200
   query99	1068	743	717	717
   Total cold run time: 287778 ms
   Total hot run time: 178981 ms
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [opt](deps) reduce the size of FE dependencies [doris]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #31129:
URL: https://github.com/apache/doris/pull/31129#issuecomment-1953334290

   PR approved by at least one committer and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [opt](deps) reduce the size of FE dependencies [doris]

Posted by "morningman (via GitHub)" <gi...@apache.org>.
morningman commented on PR #31129:
URL: https://github.com/apache/doris/pull/31129#issuecomment-1952820679

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [opt](deps) reduce the size of FE dependencies [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31129:
URL: https://github.com/apache/doris/pull/31129#issuecomment-1952907191

   
   <details>
   <summary>ClickBench: <b>Total hot run time: 30.94 s</b></summary>
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
   ClickBench test result on commit fa88d09a26040c2a42ea3bba89e9f2c38db65d9b, data reload: false
   
   query1	0.04	0.03	0.02
   query2	0.06	0.02	0.02
   query3	0.23	0.07	0.08
   query4	1.64	0.10	0.08
   query5	0.49	0.48	0.48
   query6	1.36	0.60	0.62
   query7	0.02	0.02	0.01
   query8	0.04	0.02	0.02
   query9	0.54	0.46	0.46
   query10	0.49	0.50	0.51
   query11	0.12	0.10	0.10
   query12	0.12	0.10	0.11
   query13	0.60	0.58	0.59
   query14	0.76	0.80	0.78
   query15	0.82	0.81	0.80
   query16	0.33	0.36	0.35
   query17	0.91	0.87	0.93
   query18	0.20	0.18	0.17
   query19	1.77	1.63	1.67
   query20	0.02	0.01	0.02
   query21	15.41	0.66	0.58
   query22	3.67	3.97	2.41
   query23	17.37	1.07	1.01
   query24	2.05	0.71	0.20
   query25	0.64	0.05	0.07
   query26	0.16	0.14	0.16
   query27	0.06	0.06	0.05
   query28	11.95	0.85	0.84
   query29	12.59	3.35	3.32
   query30	0.51	0.50	0.44
   query31	2.79	0.37	0.38
   query32	3.33	0.48	0.48
   query33	3.14	3.14	3.17
   query34	15.36	4.54	4.51
   query35	4.50	4.53	4.52
   query36	1.09	0.96	0.96
   query37	0.07	0.05	0.06
   query38	0.04	0.03	0.04
   query39	0.02	0.02	0.01
   query40	0.16	0.14	0.14
   query41	0.07	0.02	0.01
   query42	0.03	0.01	0.01
   query43	0.03	0.02	0.02
   Total cold run time: 105.6 s
   Total hot run time: 30.94 s
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [opt](deps) reduce the size of FE dependencies [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31129:
URL: https://github.com/apache/doris/pull/31129#issuecomment-1952881422

   
   <details>
   <summary>TPC-H: <b>Total hot run time: 41299 ms</b></summary>
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
   Tpch sf100 test result on commit fa88d09a26040c2a42ea3bba89e9f2c38db65d9b, data reload: false
   
   ------ Round 1 ----------------------------------
   q1	17701	4927	4861	4861
   q2	2041	143	131	131
   q3	10580	1034	994	994
   q4	4649	964	971	964
   q5	7693	3209	3260	3209
   q6	194	134	135	134
   q7	1257	788	761	761
   q8	9228	2063	2069	2063
   q9	7618	6686	6689	6686
   q10	8310	2652	2644	2644
   q11	405	220	224	220
   q12	729	336	338	336
   q13	18001	3680	3670	3670
   q14	288	259	255	255
   q15	590	518	509	509
   q16	491	408	408	408
   q17	931	876	850	850
   q18	7387	6756	6521	6521
   q19	1539	1486	1485	1485
   q20	659	324	344	324
   q21	6481	3931	3946	3931
   q22	888	358	343	343
   Total cold run time: 107660 ms
   Total hot run time: 41299 ms
   
   ----- Round 2, with runtime_filter_mode=off -----
   q1	4895	4921	4947	4921
   q2	314	198	197	197
   q3	3610	3614	3627	3614
   q4	2547	2565	2554	2554
   q5	5748	5745	5746	5745
   q6	210	128	130	128
   q7	2282	1664	1697	1664
   q8	3076	3136	3200	3136
   q9	8705	8699	8681	8681
   q10	6793	4253	4226	4226
   q11	573	424	432	424
   q12	798	559	559	559
   q13	4342	3408	3423	3408
   q14	264	248	254	248
   q15	607	510	529	510
   q16	509	451	442	442
   q17	1825	1767	1787	1767
   q18	8454	7908	7819	7819
   q19	1626	1625	1628	1625
   q20	2127	1843	1853	1843
   q21	6735	6405	6317	6317
   q22	609	530	537	530
   Total cold run time: 66649 ms
   Total hot run time: 60358 ms
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [opt](deps) reduce the size of FE dependencies [doris]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #31129:
URL: https://github.com/apache/doris/pull/31129#issuecomment-1953334306

   PR approved by anyone and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [opt](deps) reduce the size of FE dependencies [doris]

Posted by "yiguolei (via GitHub)" <gi...@apache.org>.
yiguolei merged PR #31129:
URL: https://github.com/apache/doris/pull/31129


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org