You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "ByteYue (via GitHub)" <gi...@apache.org> on 2024/03/21 10:57:34 UTC

[PR] [feature](Resource) Support to specify the root path for hdfs resource [doris]

ByteYue opened a new pull request, #32632:
URL: https://github.com/apache/doris/pull/32632

   ## Proposed changes
   
   Issue Number: close #xxx
   
   <!--Describe your changes.-->
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [feature](Resource) Support to specify the root path for hdfs resource [doris]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #32632:
URL: https://github.com/apache/doris/pull/32632#issuecomment-2011954173

   clang-tidy review says "All clean, LGTM! :+1:"


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [feature](Resource) Support to specify the root path for hdfs resource [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32632:
URL: https://github.com/apache/doris/pull/32632#issuecomment-2014853422

   
   Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   ```
   Load test result on commit c00225bef8cb45d35daa16e817ae589b4d099fc6 with default session variables
   Stream load json:         19 seconds loaded 2358488459 Bytes, about 118 MB/s
   Stream load orc:          59 seconds loaded 1101869774 Bytes, about 17 MB/s
   Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
   Insert into select:       19.4 seconds inserted 10000000 Rows, about 515K ops/s
   ```
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [feature](Resource) Support to specify the root path for hdfs resource [doris]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #32632:
URL: https://github.com/apache/doris/pull/32632#issuecomment-2014200213

   clang-tidy review says "All clean, LGTM! :+1:"


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [feature](Resource) Support to specify the root path for hdfs resource [doris]

Posted by "dataroaring (via GitHub)" <gi...@apache.org>.
dataroaring commented on PR #32632:
URL: https://github.com/apache/doris/pull/32632#issuecomment-2014746841

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [feature](Resource) Support to specify the root path for hdfs resource [doris]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #32632:
URL: https://github.com/apache/doris/pull/32632#issuecomment-2014198795

   clang-tidy review says "All clean, LGTM! :+1:"


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [feature](Resource) Support to specify the root path for hdfs resource [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32632:
URL: https://github.com/apache/doris/pull/32632#issuecomment-2011937471

   Thank you for your contribution to Apache Doris.
   Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR)
   
   Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website).
   See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [feature](Resource) Support to specify the root path for hdfs resource [doris]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #32632:
URL: https://github.com/apache/doris/pull/32632#issuecomment-2018243414

   PR approved by anyone and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [feature](Resource) Support to specify the root path for hdfs resource [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32632:
URL: https://github.com/apache/doris/pull/32632#issuecomment-2014848423

   
   <details>
   <summary>TPC-DS: <b>Total hot run time: 181503 ms</b></summary>
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
   TPC-DS sf100 test result on commit c00225bef8cb45d35daa16e817ae589b4d099fc6, data reload: false
   
   query1	930	364	348	348
   query2	7461	2059	1949	1949
   query3	6720	221	216	216
   query4	31284	20768	20772	20768
   query5	4419	426	406	406
   query6	276	184	169	169
   query7	4629	307	295	295
   query8	245	180	176	176
   query9	9256	2366	2402	2366
   query10	571	252	260	252
   query11	15497	14245	14230	14230
   query12	143	88	91	88
   query13	1623	416	434	416
   query14	12389	10366	11180	10366
   query15	357	199	197	197
   query16	8145	270	265	265
   query17	2093	601	552	552
   query18	2096	307	281	281
   query19	368	154	155	154
   query20	98	83	90	83
   query21	201	126	130	126
   query22	4602	4423	4389	4389
   query23	32032	31086	31203	31086
   query24	10697	2835	2852	2835
   query25	639	381	383	381
   query26	1381	161	161	161
   query27	2986	354	359	354
   query28	7543	1947	1924	1924
   query29	898	662	620	620
   query30	298	152	150	150
   query31	978	754	743	743
   query32	94	57	54	54
   query33	770	343	255	255
   query34	963	478	487	478
   query35	836	615	619	615
   query36	993	861	904	861
   query37	129	79	76	76
   query38	3531	3488	3458	3458
   query39	1421	1399	1359	1359
   query40	208	114	113	113
   query41	52	50	50	50
   query42	110	93	101	93
   query43	503	461	453	453
   query44	1234	717	730	717
   query45	283	264	264	264
   query46	1099	722	695	695
   query47	1639	1582	1594	1582
   query48	475	359	370	359
   query49	1117	357	346	346
   query50	785	387	378	378
   query51	6680	6668	6629	6629
   query52	105	97	91	91
   query53	348	275	286	275
   query54	310	254	253	253
   query55	84	80	79	79
   query56	249	240	238	238
   query57	1090	1006	1008	1006
   query58	241	214	218	214
   query59	2805	2644	2551	2551
   query60	281	259	267	259
   query61	118	116	110	110
   query62	596	400	394	394
   query63	301	279	280	279
   query64	5761	3902	3756	3756
   query65	3058	3016	3018	3016
   query66	1001	341	336	336
   query67	14782	14372	14334	14334
   query68	9199	539	547	539
   query69	676	377	382	377
   query70	1433	1147	1161	1147
   query71	550	290	266	266
   query72	6814	2639	2521	2521
   query73	1606	321	318	318
   query74	6991	6574	6511	6511
   query75	5227	2828	2811	2811
   query76	5832	934	804	804
   query77	869	276	263	263
   query78	10368	9739	9636	9636
   query79	10799	530	535	530
   query80	1119	400	395	395
   query81	522	211	213	211
   query82	350	201	202	201
   query83	206	148	160	148
   query84	290	78	79	78
   query85	1126	343	313	313
   query86	369	285	263	263
   query87	3781	3581	3622	3581
   query88	5206	2364	2375	2364
   query89	490	383	378	378
   query90	2139	180	177	177
   query91	172	136	141	136
   query92	59	51	48	48
   query93	6358	520	496	496
   query94	1399	184	179	179
   query95	431	341	344	341
   query96	594	275	270	270
   query97	3021	2899	2871	2871
   query98	226	220	213	213
   query99	990	745	749	745
   Total cold run time: 314119 ms
   Total hot run time: 181503 ms
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [feature](Resource) Support to specify the root path for hdfs resource [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32632:
URL: https://github.com/apache/doris/pull/32632#issuecomment-2014898715

   TeamCity be ut coverage result:
    Function Coverage: 35.26% (8733/24764) 
    Line Coverage: 27.06% (71513/264273)
    Region Coverage: 26.30% (37103/141090)
    Branch Coverage: 23.19% (18975/81814)
    Coverage Report: http://coverage.selectdb-in.cc/coverage/c00225bef8cb45d35daa16e817ae589b4d099fc6_c00225bef8cb45d35daa16e817ae589b4d099fc6/report/index.html


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [feature](Resource) Support to specify the root path for hdfs resource [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32632:
URL: https://github.com/apache/doris/pull/32632#issuecomment-2014830113

   
   <details>
   <summary>TPC-H: <b>Total hot run time: 37901 ms</b></summary>
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
   Tpch sf100 test result on commit c00225bef8cb45d35daa16e817ae589b4d099fc6, data reload: false
   
   ------ Round 1 ----------------------------------
   q1	17882	4410	4190	4190
   q2	2537	162	161	161
   q3	11484	1171	1226	1171
   q4	10651	781	803	781
   q5	8562	3032	2976	2976
   q6	202	125	128	125
   q7	1062	576	560	560
   q8	9359	2055	2010	2010
   q9	7027	6520	6514	6514
   q10	8417	3406	3501	3406
   q11	442	230	217	217
   q12	370	212	198	198
   q13	17798	2867	2884	2867
   q14	231	201	213	201
   q15	513	465	447	447
   q16	481	373	376	373
   q17	978	543	572	543
   q18	7406	6503	6415	6415
   q19	4835	1411	1445	1411
   q20	546	261	246	246
   q21	3536	2874	2809	2809
   q22	351	280	296	280
   Total cold run time: 114670 ms
   Total hot run time: 37901 ms
   
   ----- Round 2, with runtime_filter_mode=off -----
   q1	4146	4091	4092	4091
   q2	339	232	227	227
   q3	2984	2886	2835	2835
   q4	1881	1576	1555	1555
   q5	5230	5274	5244	5244
   q6	194	115	117	115
   q7	2240	1873	1874	1873
   q8	3174	3310	3310	3310
   q9	8635	8586	8597	8586
   q10	3670	3717	3700	3700
   q11	543	455	438	438
   q12	733	535	540	535
   q13	16920	2882	2845	2845
   q14	277	251	259	251
   q15	492	457	473	457
   q16	478	421	421	421
   q17	1747	1506	1478	1478
   q18	7443	7283	7070	7070
   q19	1624	1526	1525	1525
   q20	1940	1739	1716	1716
   q21	4811	4767	4654	4654
   q22	525	444	434	434
   Total cold run time: 70026 ms
   Total hot run time: 53360 ms
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [feature](Resource) Support to specify the root path for hdfs resource [doris]

Posted by "dataroaring (via GitHub)" <gi...@apache.org>.
dataroaring merged PR #32632:
URL: https://github.com/apache/doris/pull/32632


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [feature](Resource) Support to specify the root path for hdfs resource [doris]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #32632:
URL: https://github.com/apache/doris/pull/32632#issuecomment-2011948047

   clang-tidy review says "All clean, LGTM! :+1:"


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org