You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "suxiaogang223 (via GitHub)" <gi...@apache.org> on 2026/03/24 06:31:33 UTC

[PR] [refactor](fe) Extract Iceberg planning split producer [doris]

suxiaogang223 opened a new pull request, #61663:
URL: https://github.com/apache/doris/pull/61663

   ### What problem does this PR solve?
   
   Issue Number: N/A
   
   Related PR: N/A
   
   Problem Summary:
   
   - Extract Iceberg batch split planning into a dedicated FE-side producer abstraction.
   - Add PlanningSplitProducer and SplitSink so split production can evolve independently from SplitAssignment and SplitSource.
   - Move the existing Iceberg local async planning path into LocalParallelPlanningSplitProducer without changing the downstream split consumption flow.
   - Add FE unit coverage for producer behavior and SplitAssignment lifecycle delegation.
   
   ### Release note
   
   None
   
   ### Check List (For Author)
   
   - Test <!-- At least one of them must be included. -->
       - [ ] Regression test
       - [x] Unit Test
       - [ ] Manual test (add detailed scripts or steps below)
       - [ ] No need to test or manual test. Explain why:
           - [ ] This is a refactor/code format and no logic has been changed.
           - [ ] Previous test can cover this change.
           - [ ] No code files have been changed.
           - [ ] Other reason <!-- Add your reason?  -->
   
   - Behavior changed:
       - [x] No.
       - [ ] Yes. <!-- Explain the behavior change -->
   
   - Does this need documentation?
       - [x] No.
       - [ ] Yes. <!-- Add document PR link here. eg: https://github.com/apache/doris-website/pull/1214 -->
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [refactor](fe) Unify Iceberg split planning via producer [doris]

Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #61663:
URL: https://github.com/apache/doris/pull/61663#issuecomment-4132598646

   # FE UT Coverage Report
   Increment line coverage `  35.32% (178/504)` :tada:
   [Increment coverage report](http://coverage.selectdb-in.cc/coverage/61663_cde10de764019768092d0ab7aacfb5dc97bad890/fe_increment_report/index.html)
   [Complete coverage report](http://coverage.selectdb-in.cc/coverage/61663_cde10de764019768092d0ab7aacfb5dc97bad890/fe_report/index.html)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [refactor](fe) Unify Iceberg split planning via producer [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #61663:
URL: https://github.com/apache/doris/pull/61663#issuecomment-4128306148

   
   <details>
   <summary>TPC-H: <b>Total hot run time: 26628 ms</b></summary>
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
   Tpch sf100 test result on commit 717650219675b2da4548a5addf955827a5b7f44f, data reload: false
   
   ------ Round 1 ----------------------------------
   orders	Doris	NULL	NULL	0	0	0	NULL	0	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL	
   ============================================
   q1	17595	4440	4308	4308
   q2	q3	10651	790	529	529
   q4	4672	368	255	255
   q5	7558	1216	1023	1023
   q6	177	176	147	147
   q7	783	831	677	677
   q8	9310	1489	1304	1304
   q9	4846	4752	4661	4661
   q10	6266	1920	1632	1632
   q11	459	268	259	259
   q12	734	587	474	474
   q13	18022	2676	1942	1942
   q14	221	233	217	217
   q15	q16	733	748	670	670
   q17	729	859	443	443
   q18	5988	5407	5359	5359
   q19	1242	993	606	606
   q20	535	481	371	371
   q21	4934	1846	1435	1435
   q22	446	459	316	316
   Total cold run time: 95901 ms
   Total hot run time: 26628 ms
   
   ----- Round 2, with runtime_filter_mode=off -----
   orders	Doris	NULL	NULL	150000000	42	6422171781	NULL	22778155	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL	
   ============================================
   q1	4820	4687	4601	4601
   q2	q3	3857	4321	3823	3823
   q4	887	1195	768	768
   q5	4106	4356	4382	4356
   q6	187	180	140	140
   q7	1762	1704	1549	1549
   q8	2486	2679	2592	2592
   q9	7712	7411	7486	7411
   q10	3738	4015	3629	3629
   q11	524	455	419	419
   q12	495	577	458	458
   q13	2445	2875	2075	2075
   q14	275	293	285	285
   q15	q16	707	774	770	770
   q17	1213	1482	1363	1363
   q18	7097	6799	6698	6698
   q19	936	918	903	903
   q20	2103	2228	2009	2009
   q21	3952	3461	3411	3411
   q22	466	422	379	379
   Total cold run time: 49768 ms
   Total hot run time: 47639 ms
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [refactor](fe) Unify Iceberg split planning via producer [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #61663:
URL: https://github.com/apache/doris/pull/61663#issuecomment-4133611330

   
   <details>
   <summary>TPC-DS: <b>Total hot run time: 169458 ms</b></summary>
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
   TPC-DS sf100 test result on commit 4559afcc163219692c584be26eb7a3e9410d8e5e, data reload: false
   
   query5	4318	655	519	519
   query6	327	228	212	212
   query7	4230	460	264	264
   query8	334	249	238	238
   query9	8782	2770	2791	2770
   query10	521	369	341	341
   query11	6990	5097	4869	4869
   query12	183	132	124	124
   query13	1272	457	349	349
   query14	5818	3741	3440	3440
   query14_1	2884	2836	2821	2821
   query15	213	201	174	174
   query16	1004	472	371	371
   query17	1120	725	640	640
   query18	2450	455	358	358
   query19	230	211	183	183
   query20	141	126	127	126
   query21	218	136	113	113
   query22	13282	14418	14764	14418
   query23	16699	16309	16202	16202
   query23_1	15974	16480	15704	15704
   query24	7155	1618	1217	1217
   query24_1	1255	1243	1216	1216
   query25	547	468	403	403
   query26	1224	258	145	145
   query27	2786	483	295	295
   query28	4991	1905	1864	1864
   query29	913	588	501	501
   query30	308	226	191	191
   query31	1036	969	920	920
   query32	82	70	73	70
   query33	530	340	296	296
   query34	895	883	531	531
   query35	662	693	628	628
   query36	1124	1151	1021	1021
   query37	132	96	86	86
   query38	2983	2926	2857	2857
   query39	863	839	812	812
   query39_1	810	801	814	801
   query40	234	153	132	132
   query41	64	60	62	60
   query42	259	257	258	257
   query43	255	261	222	222
   query44	
   query45	200	190	181	181
   query46	877	1011	613	613
   query47	2081	2125	2047	2047
   query48	323	321	233	233
   query49	636	470	390	390
   query50	679	278	215	215
   query51	4103	4092	3982	3982
   query52	258	270	257	257
   query53	289	343	280	280
   query54	295	281	270	270
   query55	87	84	85	84
   query56	334	330	306	306
   query57	1906	1824	1610	1610
   query58	282	273	270	270
   query59	2791	2914	2741	2741
   query60	339	332	331	331
   query61	159	156	151	151
   query62	652	585	547	547
   query63	307	275	271	271
   query64	4989	1280	1055	1055
   query65	
   query66	1457	450	361	361
   query67	24202	24226	24200	24200
   query68	
   query69	415	318	284	284
   query70	980	915	978	915
   query71	332	304	285	285
   query72	2880	2709	2414	2414
   query73	544	549	319	319
   query74	9632	9611	9400	9400
   query75	2893	2750	2425	2425
   query76	2281	1032	677	677
   query77	368	396	299	299
   query78	10893	11103	10461	10461
   query79	1107	771	563	563
   query80	769	641	535	535
   query81	489	258	224	224
   query82	1342	157	123	123
   query83	344	266	245	245
   query84	252	120	97	97
   query85	928	498	464	464
   query86	385	337	301	301
   query87	3148	3106	3087	3087
   query88	3571	2685	2660	2660
   query89	423	363	347	347
   query90	1842	185	177	177
   query91	180	167	147	147
   query92	77	76	77	76
   query93	917	834	507	507
   query94	485	335	286	286
   query95	589	341	384	341
   query96	655	541	230	230
   query97	2452	2464	2429	2429
   query98	241	218	225	218
   query99	1025	1000	906	906
   Total cold run time: 250086 ms
   Total hot run time: 169458 ms
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [refactor](fe) Unify Iceberg split planning via producer [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #61663:
URL: https://github.com/apache/doris/pull/61663#issuecomment-4128376442

   
   <details>
   <summary>TPC-DS: <b>Total hot run time: 168903 ms</b></summary>
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
   TPC-DS sf100 test result on commit 717650219675b2da4548a5addf955827a5b7f44f, data reload: false
   
   query5	4341	625	498	498
   query6	325	224	203	203
   query7	4218	462	263	263
   query8	332	238	231	231
   query9	8662	2695	2703	2695
   query10	533	379	337	337
   query11	6981	5094	4855	4855
   query12	184	127	124	124
   query13	1270	471	350	350
   query14	5732	3644	3452	3452
   query14_1	2910	2840	2819	2819
   query15	202	196	177	177
   query16	996	470	461	461
   query17	1125	734	634	634
   query18	2462	454	355	355
   query19	218	210	185	185
   query20	139	127	134	127
   query21	217	138	110	110
   query22	13253	14022	14758	14022
   query23	16795	16093	16245	16093
   query23_1	15654	15647	15617	15617
   query24	7222	1592	1215	1215
   query24_1	1212	1207	1230	1207
   query25	550	474	419	419
   query26	1220	261	147	147
   query27	2787	477	289	289
   query28	4482	1858	1826	1826
   query29	825	556	472	472
   query30	298	225	188	188
   query31	1011	937	898	898
   query32	85	68	69	68
   query33	507	334	293	293
   query34	885	865	509	509
   query35	658	674	601	601
   query36	1081	1147	992	992
   query37	132	95	87	87
   query38	2963	2855	2806	2806
   query39	865	818	803	803
   query39_1	806	784	803	784
   query40	235	152	139	139
   query41	64	58	61	58
   query42	267	257	254	254
   query43	236	247	212	212
   query44	
   query45	198	186	180	180
   query46	867	991	607	607
   query47	2114	2150	2056	2056
   query48	312	312	266	266
   query49	626	465	376	376
   query50	686	283	219	219
   query51	4091	4124	4006	4006
   query52	259	266	253	253
   query53	288	338	278	278
   query54	293	265	270	265
   query55	89	86	79	79
   query56	305	321	303	303
   query57	1901	1720	1789	1720
   query58	283	276	269	269
   query59	2832	2940	2740	2740
   query60	347	339	329	329
   query61	159	149	156	149
   query62	619	595	544	544
   query63	310	275	283	275
   query64	5056	1279	1009	1009
   query65	
   query66	1466	444	355	355
   query67	24333	24350	24237	24237
   query68	
   query69	401	311	301	301
   query70	950	964	954	954
   query71	353	304	301	301
   query72	2866	2865	2599	2599
   query73	549	555	310	310
   query74	9648	9540	9399	9399
   query75	2886	2785	2463	2463
   query76	2299	1035	664	664
   query77	374	396	342	342
   query78	11017	11081	10543	10543
   query79	1122	801	567	567
   query80	680	619	539	539
   query81	488	258	228	228
   query82	1359	149	117	117
   query83	372	265	238	238
   query84	252	118	104	104
   query85	858	519	451	451
   query86	382	310	324	310
   query87	3161	3115	3012	3012
   query88	3502	2627	2608	2608
   query89	442	363	340	340
   query90	1953	177	173	173
   query91	180	160	145	145
   query92	79	76	72	72
   query93	901	840	489	489
   query94	461	302	280	280
   query95	585	337	392	337
   query96	645	529	223	223
   query97	2432	2505	2408	2408
   query98	241	232	215	215
   query99	1009	954	918	918
   Total cold run time: 249323 ms
   Total hot run time: 168903 ms
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [refactor](fe) Unify Iceberg split planning via producer [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #61663:
URL: https://github.com/apache/doris/pull/61663#issuecomment-4125159344

   
   <details>
   <summary>TPC-DS: <b>Total hot run time: 168573 ms</b></summary>
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
   TPC-DS sf100 test result on commit 533cba96c354a22d67f2853a028698a2f84494e5, data reload: false
   
   query5	4330	612	512	512
   query6	331	217	200	200
   query7	4223	459	280	280
   query8	352	239	220	220
   query9	8721	2685	2691	2685
   query10	614	399	327	327
   query11	7013	5082	4851	4851
   query12	179	126	123	123
   query13	1281	438	356	356
   query14	5737	3666	3408	3408
   query14_1	2826	2773	2758	2758
   query15	202	192	171	171
   query16	968	450	435	435
   query17	857	699	595	595
   query18	2433	432	341	341
   query19	243	202	175	175
   query20	138	128	128	128
   query21	214	130	107	107
   query22	13338	14285	14532	14285
   query23	16572	16496	16106	16106
   query23_1	16340	15951	15657	15657
   query24	7267	1589	1214	1214
   query24_1	1213	1202	1211	1202
   query25	544	457	390	390
   query26	1237	259	150	150
   query27	2786	481	291	291
   query28	4552	1828	1848	1828
   query29	839	565	474	474
   query30	304	212	187	187
   query31	1014	966	867	867
   query32	84	67	69	67
   query33	513	323	275	275
   query34	914	863	539	539
   query35	637	678	590	590
   query36	1063	1115	1036	1036
   query37	141	99	89	89
   query38	2900	2914	2855	2855
   query39	863	825	814	814
   query39_1	797	791	790	790
   query40	242	159	172	159
   query41	63	58	60	58
   query42	260	255	251	251
   query43	233	253	213	213
   query44	
   query45	195	188	183	183
   query46	887	978	592	592
   query47	2138	2131	2065	2065
   query48	308	316	228	228
   query49	630	465	377	377
   query50	683	276	205	205
   query51	4028	4017	4044	4017
   query52	273	268	255	255
   query53	287	342	289	289
   query54	301	262	257	257
   query55	96	86	81	81
   query56	319	323	302	302
   query57	1908	1763	1573	1573
   query58	280	280	283	280
   query59	2772	2923	2734	2734
   query60	341	339	317	317
   query61	151	150	149	149
   query62	634	594	521	521
   query63	309	278	278	278
   query64	5060	1271	981	981
   query65	
   query66	1467	450	352	352
   query67	24186	24282	24101	24101
   query68	
   query69	395	310	284	284
   query70	956	897	937	897
   query71	345	302	298	298
   query72	2752	2629	2511	2511
   query73	548	539	320	320
   query74	9626	9557	9390	9390
   query75	2900	2767	2468	2468
   query76	2291	1035	686	686
   query77	369	386	319	319
   query78	10927	11080	10452	10452
   query79	1106	760	558	558
   query80	727	653	569	569
   query81	472	262	224	224
   query82	1353	149	122	122
   query83	370	270	248	248
   query84	304	122	96	96
   query85	943	562	513	513
   query86	361	308	308	308
   query87	3168	3124	2999	2999
   query88	3551	2652	2646	2646
   query89	441	364	345	345
   query90	1937	181	170	170
   query91	183	177	181	177
   query92	73	74	70	70
   query93	908	835	495	495
   query94	454	333	305	305
   query95	601	332	308	308
   query96	652	531	232	232
   query97	2496	2485	2381	2381
   query98	236	222	219	219
   query99	903	981	918	918
   Total cold run time: 249182 ms
   Total hot run time: 168573 ms
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [refactor](fe) Unify Iceberg split planning via producer [doris]

Posted by "suxiaogang223 (via GitHub)" <gi...@apache.org>.
suxiaogang223 commented on PR #61663:
URL: https://github.com/apache/doris/pull/61663#issuecomment-4133341978

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [refactor](fe) Unify Iceberg split planning via producer [doris]

Posted by "suxiaogang223 (via GitHub)" <gi...@apache.org>.
suxiaogang223 commented on PR #61663:
URL: https://github.com/apache/doris/pull/61663#issuecomment-4128182987

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [refactor](fe) Extract Iceberg planning split producer [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #61663:
URL: https://github.com/apache/doris/pull/61663#issuecomment-4115947233

   
   <details>
   <summary>TPC-DS: <b>Total hot run time: 168724 ms</b></summary>
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
   TPC-DS sf100 test result on commit 7dd02767bed05a619a228a99ff8e9df1be84faf4, data reload: false
   
   query5	4344	632	494	494
   query6	350	225	207	207
   query7	4201	465	264	264
   query8	354	245	232	232
   query9	8726	2731	2705	2705
   query10	521	374	336	336
   query11	6954	5122	4897	4897
   query12	180	130	124	124
   query13	1278	471	345	345
   query14	5736	3685	3465	3465
   query14_1	2842	2841	2789	2789
   query15	208	186	175	175
   query16	967	465	432	432
   query17	887	694	604	604
   query18	2431	439	336	336
   query19	217	202	180	180
   query20	136	126	124	124
   query21	222	144	113	113
   query22	13317	13971	14402	13971
   query23	16870	16204	15975	15975
   query23_1	16313	15525	15687	15525
   query24	7184	1616	1228	1228
   query24_1	1208	1246	1218	1218
   query25	544	457	398	398
   query26	1233	263	149	149
   query27	2764	487	293	293
   query28	4517	1827	1853	1827
   query29	856	567	473	473
   query30	296	224	192	192
   query31	993	963	861	861
   query32	77	71	69	69
   query33	520	339	283	283
   query34	900	879	517	517
   query35	629	686	597	597
   query36	1064	1073	989	989
   query37	134	95	85	85
   query38	2972	2939	2875	2875
   query39	865	834	816	816
   query39_1	796	776	796	776
   query40	235	151	138	138
   query41	60	59	58	58
   query42	258	262	257	257
   query43	244	250	228	228
   query44	
   query45	204	199	184	184
   query46	892	1011	606	606
   query47	2114	2156	2453	2156
   query48	306	321	234	234
   query49	637	452	383	383
   query50	688	288	217	217
   query51	4050	4169	3963	3963
   query52	264	265	258	258
   query53	301	334	294	294
   query54	304	274	285	274
   query55	91	85	85	85
   query56	322	309	321	309
   query57	1899	1722	1731	1722
   query58	291	279	282	279
   query59	2764	2921	2736	2736
   query60	349	339	312	312
   query61	163	155	160	155
   query62	630	590	525	525
   query63	311	288	285	285
   query64	5134	1297	1019	1019
   query65	
   query66	1468	463	373	373
   query67	24249	24346	24219	24219
   query68	
   query69	419	316	319	316
   query70	1014	890	926	890
   query71	339	312	309	309
   query72	2950	2847	2646	2646
   query73	542	545	332	332
   query74	9654	9560	9362	9362
   query75	2854	2748	2472	2472
   query76	2276	1042	671	671
   query77	359	387	300	300
   query78	10904	10951	10496	10496
   query79	3186	749	572	572
   query80	1738	644	543	543
   query81	572	269	229	229
   query82	1011	155	119	119
   query83	338	255	245	245
   query84	303	122	97	97
   query85	904	499	450	450
   query86	493	310	327	310
   query87	3123	3127	3012	3012
   query88	3551	2652	2626	2626
   query89	421	369	352	352
   query90	2104	186	175	175
   query91	168	162	142	142
   query92	87	76	73	73
   query93	2276	827	501	501
   query94	646	312	290	290
   query95	579	346	380	346
   query96	646	515	227	227
   query97	2468	2452	2357	2357
   query98	241	223	220	220
   query99	1006	996	934	934
   Total cold run time: 253394 ms
   Total hot run time: 168724 ms
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [refactor](fe) Extract Iceberg planning split producer [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #61663:
URL: https://github.com/apache/doris/pull/61663#issuecomment-4115904014

   
   <details>
   <summary>TPC-H: <b>Total hot run time: 26655 ms</b></summary>
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
   Tpch sf100 test result on commit 7dd02767bed05a619a228a99ff8e9df1be84faf4, data reload: false
   
   ------ Round 1 ----------------------------------
   orders	Doris	NULL	NULL	0	0	0	NULL	0	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL	
   ============================================
   q1	17636	4435	4274	4274
   q2	q3	10646	791	524	524
   q4	4678	351	245	245
   q5	7559	1213	1021	1021
   q6	184	173	146	146
   q7	771	854	684	684
   q8	9346	1490	1365	1365
   q9	4880	4752	4714	4714
   q10	6331	1928	1716	1716
   q11	474	260	246	246
   q12	737	595	457	457
   q13	18046	2713	1916	1916
   q14	223	234	210	210
   q15	q16	745	731	670	670
   q17	749	864	433	433
   q18	5945	5311	5322	5311
   q19	1377	968	618	618
   q20	542	498	381	381
   q21	4695	1862	1400	1400
   q22	383	512	324	324
   Total cold run time: 95947 ms
   Total hot run time: 26655 ms
   
   ----- Round 2, with runtime_filter_mode=off -----
   orders	Doris	NULL	NULL	150000000	42	6422171781	NULL	22778155	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL	
   ============================================
   q1	4766	4604	4562	4562
   q2	q3	4007	4416	3850	3850
   q4	918	1222	779	779
   q5	4040	4411	4394	4394
   q6	193	180	147	147
   q7	1802	1635	1514	1514
   q8	2494	2703	2545	2545
   q9	7700	7530	7322	7322
   q10	3728	4035	3659	3659
   q11	515	432	434	432
   q12	483	599	452	452
   q13	2466	3066	2043	2043
   q14	279	292	275	275
   q15	q16	761	778	705	705
   q17	1179	1385	1359	1359
   q18	7082	6758	6925	6758
   q19	968	929	937	929
   q20	2086	2154	2026	2026
   q21	4052	3556	3447	3447
   q22	458	436	389	389
   Total cold run time: 49977 ms
   Total hot run time: 47587 ms
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [refactor](fe) Unify Iceberg split planning via producer [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #61663:
URL: https://github.com/apache/doris/pull/61663#issuecomment-4125093750

   
   <details>
   <summary>TPC-H: <b>Total hot run time: 26553 ms</b></summary>
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
   Tpch sf100 test result on commit 533cba96c354a22d67f2853a028698a2f84494e5, data reload: false
   
   ------ Round 1 ----------------------------------
   orders	Doris	NULL	NULL	0	0	0	NULL	0	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL	
   ============================================
   q1	17624	4515	4291	4291
   q2	q3	10644	791	520	520
   q4	4678	351	249	249
   q5	7558	1208	1003	1003
   q6	175	173	151	151
   q7	790	847	682	682
   q8	9295	1454	1330	1330
   q9	4914	4734	4668	4668
   q10	6294	1903	1664	1664
   q11	461	249	249	249
   q12	754	567	459	459
   q13	18023	2679	1918	1918
   q14	237	233	216	216
   q15	q16	722	715	659	659
   q17	723	810	465	465
   q18	5915	5367	5355	5355
   q19	1125	989	625	625
   q20	531	495	372	372
   q21	4482	1816	1402	1402
   q22	342	299	275	275
   Total cold run time: 95287 ms
   Total hot run time: 26553 ms
   
   ----- Round 2, with runtime_filter_mode=off -----
   orders	Doris	NULL	NULL	150000000	42	6422171781	NULL	22778155	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL	
   ============================================
   q1	4856	4664	4780	4664
   q2	q3	3855	4370	3822	3822
   q4	883	1179	773	773
   q5	4125	4406	4312	4312
   q6	184	182	144	144
   q7	1759	1680	1528	1528
   q8	2496	2728	2556	2556
   q9	7593	7479	7435	7435
   q10	3769	4023	3583	3583
   q11	503	428	419	419
   q12	502	583	436	436
   q13	2488	2963	2245	2245
   q14	435	420	293	293
   q15	q16	727	786	712	712
   q17	1204	1397	1371	1371
   q18	7212	6968	6595	6595
   q19	940	896	1037	896
   q20	2174	2191	1993	1993
   q21	3969	3495	3359	3359
   q22	442	433	395	395
   Total cold run time: 50116 ms
   Total hot run time: 47531 ms
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [refactor](fe) Unify Iceberg split planning via producer [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #61663:
URL: https://github.com/apache/doris/pull/61663#issuecomment-4133545502

   
   <details>
   <summary>TPC-H: <b>Total hot run time: 26595 ms</b></summary>
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
   Tpch sf100 test result on commit 4559afcc163219692c584be26eb7a3e9410d8e5e, data reload: false
   
   ------ Round 1 ----------------------------------
   orders	Doris	NULL	NULL	0	0	0	NULL	0	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL	
   ============================================
   q1	17650	4494	4314	4314
   q2	q3	10647	759	504	504
   q4	4682	352	252	252
   q5	7562	1219	1031	1031
   q6	179	178	151	151
   q7	804	851	677	677
   q8	9791	1482	1353	1353
   q9	5363	4745	4714	4714
   q10	6310	1904	1637	1637
   q11	477	255	242	242
   q12	756	579	461	461
   q13	18048	2745	1957	1957
   q14	225	228	208	208
   q15	q16	734	720	660	660
   q17	747	864	422	422
   q18	5926	5447	5296	5296
   q19	1449	975	634	634
   q20	546	480	378	378
   q21	4550	1869	1457	1457
   q22	334	295	247	247
   Total cold run time: 96780 ms
   Total hot run time: 26595 ms
   
   ----- Round 2, with runtime_filter_mode=off -----
   orders	Doris	NULL	NULL	150000000	42	6422171781	NULL	22778155	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL	
   ============================================
   q1	4435	4351	4346	4346
   q2	q3	3829	4280	3773	3773
   q4	849	1154	774	774
   q5	4010	4315	4315	4315
   q6	186	177	143	143
   q7	1730	1591	1504	1504
   q8	2439	2631	2526	2526
   q9	7829	7633	7599	7599
   q10	3783	3984	3595	3595
   q11	510	441	425	425
   q12	497	604	455	455
   q13	2531	2913	2074	2074
   q14	308	309	274	274
   q15	q16	770	749	752	749
   q17	1228	1350	1402	1350
   q18	7246	6873	6728	6728
   q19	967	948	959	948
   q20	2113	2161	1985	1985
   q21	4114	3495	3372	3372
   q22	451	455	399	399
   Total cold run time: 49825 ms
   Total hot run time: 47334 ms
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [refactor](fe) Extract Iceberg planning split producer [doris]

Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #61663:
URL: https://github.com/apache/doris/pull/61663#issuecomment-4115758888

   
   Thank you for your contribution to Apache Doris.
   Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR).
   
   Please clearly describe your PR:
   1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
   2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
   3. What features were added. Why was this function added?
   4. Which code was refactored and why was this part of the code refactored?
   5. Which functions were optimized and what is the difference before and after the optimization?
       


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [refactor](fe) Unify Iceberg split planning via producer [doris]

Posted by "suxiaogang223 (via GitHub)" <gi...@apache.org>.
suxiaogang223 commented on PR #61663:
URL: https://github.com/apache/doris/pull/61663#issuecomment-4124943261

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [refactor](fe) Unify Iceberg split planning via producer [doris]

Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #61663:
URL: https://github.com/apache/doris/pull/61663#issuecomment-4125459709

   # FE UT Coverage Report
   Increment line coverage `  20.61% (101/490)` :tada:
   [Increment coverage report](http://coverage.selectdb-in.cc/coverage/61663_533cba96c354a22d67f2853a028698a2f84494e5/fe_increment_report/index.html)
   [Complete coverage report](http://coverage.selectdb-in.cc/coverage/61663_533cba96c354a22d67f2853a028698a2f84494e5/fe_report/index.html)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [refactor](fe) Extract Iceberg planning split producer [doris]

Posted by "suxiaogang223 (via GitHub)" <gi...@apache.org>.
suxiaogang223 commented on PR #61663:
URL: https://github.com/apache/doris/pull/61663#issuecomment-4115765120

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [refactor](fe) Unify Iceberg split planning via producer [doris]

Posted by "suxiaogang223 (via GitHub)" <gi...@apache.org>.
suxiaogang223 closed pull request #61663: [refactor](fe) Unify Iceberg split planning via producer
URL: https://github.com/apache/doris/pull/61663


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [refactor](fe) Unify Iceberg split planning via producer [doris]

Posted by "suxiaogang223 (via GitHub)" <gi...@apache.org>.
suxiaogang223 commented on PR #61663:
URL: https://github.com/apache/doris/pull/61663#issuecomment-4139793953

   run external


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [refactor](fe) Unify Iceberg split planning via producer [doris]

Posted by "suxiaogang223 (via GitHub)" <gi...@apache.org>.
suxiaogang223 commented on PR #61663:
URL: https://github.com/apache/doris/pull/61663#issuecomment-4132195351

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [refactor](fe) Unify Iceberg split planning via producer [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #61663:
URL: https://github.com/apache/doris/pull/61663#issuecomment-4132614375

   
   <details>
   <summary>TPC-H: <b>Total hot run time: 26779 ms</b></summary>
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
   Tpch sf100 test result on commit cde10de764019768092d0ab7aacfb5dc97bad890, data reload: false
   
   ------ Round 1 ----------------------------------
   orders	Doris	NULL	NULL	0	0	0	NULL	0	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL	
   ============================================
   q1	17627	4459	4288	4288
   q2	q3	10640	775	519	519
   q4	4670	369	248	248
   q5	7554	1238	1044	1044
   q6	175	174	145	145
   q7	789	861	684	684
   q8	9942	1514	1400	1400
   q9	5360	4715	4716	4715
   q10	6348	1941	1623	1623
   q11	478	255	236	236
   q12	750	575	458	458
   q13	18028	2690	1929	1929
   q14	237	235	209	209
   q15	q16	771	728	673	673
   q17	731	832	475	475
   q18	5991	5370	5235	5235
   q19	1233	994	661	661
   q20	551	493	380	380
   q21	4578	1833	1575	1575
   q22	464	366	282	282
   Total cold run time: 96917 ms
   Total hot run time: 26779 ms
   
   ----- Round 2, with runtime_filter_mode=off -----
   orders	Doris	NULL	NULL	150000000	42	6422171781	NULL	22778155	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL	
   ============================================
   q1	4772	4565	4562	4562
   q2	q3	3900	4355	3996	3996
   q4	906	1302	791	791
   q5	4113	4422	4402	4402
   q6	189	175	148	148
   q7	1777	1657	1553	1553
   q8	2517	3115	2563	2563
   q9	7529	7559	7527	7527
   q10	3778	3984	3618	3618
   q11	496	438	418	418
   q12	481	599	456	456
   q13	3012	2914	2081	2081
   q14	295	313	293	293
   q15	q16	739	775	733	733
   q17	1166	1397	1376	1376
   q18	7305	6865	6772	6772
   q19	934	1019	926	926
   q20	2089	2166	2009	2009
   q21	3947	3472	3351	3351
   q22	474	432	375	375
   Total cold run time: 50419 ms
   Total hot run time: 47950 ms
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [refactor](fe) Unify Iceberg split planning via producer [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #61663:
URL: https://github.com/apache/doris/pull/61663#issuecomment-4132673908

   
   <details>
   <summary>TPC-DS: <b>Total hot run time: 169433 ms</b></summary>
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
   TPC-DS sf100 test result on commit cde10de764019768092d0ab7aacfb5dc97bad890, data reload: false
   
   query5	4336	659	507	507
   query6	321	229	212	212
   query7	4225	464	267	267
   query8	367	254	234	234
   query9	8686	2735	2748	2735
   query10	533	389	356	356
   query11	6996	5109	4906	4906
   query12	193	126	126	126
   query13	1274	480	348	348
   query14	5723	3714	3494	3494
   query14_1	2875	2919	2775	2775
   query15	200	187	174	174
   query16	959	464	443	443
   query17	862	701	605	605
   query18	2430	437	346	346
   query19	212	206	178	178
   query20	139	126	122	122
   query21	210	129	111	111
   query22	13169	14148	14618	14148
   query23	16635	16210	15979	15979
   query23_1	16278	15773	15622	15622
   query24	7205	1624	1219	1219
   query24_1	1217	1256	1235	1235
   query25	535	461	395	395
   query26	1245	261	152	152
   query27	2784	479	296	296
   query28	4512	1847	1836	1836
   query29	832	565	465	465
   query30	299	221	187	187
   query31	1024	934	873	873
   query32	83	72	68	68
   query33	509	330	280	280
   query34	885	871	527	527
   query35	651	707	606	606
   query36	1050	1117	965	965
   query37	128	103	80	80
   query38	2906	2880	2903	2880
   query39	864	827	804	804
   query39_1	784	800	794	794
   query40	229	155	137	137
   query41	64	90	59	59
   query42	269	257	257	257
   query43	239	247	218	218
   query44	
   query45	201	191	179	179
   query46	877	980	609	609
   query47	2121	2114	2606	2114
   query48	321	312	245	245
   query49	633	451	389	389
   query50	685	278	210	210
   query51	4170	4070	4057	4057
   query52	268	263	258	258
   query53	290	337	292	292
   query54	307	281	269	269
   query55	93	92	82	82
   query56	321	327	320	320
   query57	1938	1664	1685	1664
   query58	283	268	265	265
   query59	2773	2956	2751	2751
   query60	349	341	325	325
   query61	152	152	161	152
   query62	609	583	538	538
   query63	304	282	276	276
   query64	5065	1308	1066	1066
   query65	
   query66	1469	454	362	362
   query67	24342	24490	24173	24173
   query68	
   query69	412	311	279	279
   query70	974	964	926	926
   query71	339	324	297	297
   query72	2842	2726	2578	2578
   query73	552	533	323	323
   query74	9650	9578	9439	9439
   query75	2877	2780	2474	2474
   query76	2290	1059	691	691
   query77	363	392	324	324
   query78	10857	11063	10446	10446
   query79	1120	772	583	583
   query80	738	643	580	580
   query81	490	261	231	231
   query82	1346	159	126	126
   query83	334	291	249	249
   query84	308	123	109	109
   query85	944	595	557	557
   query86	396	312	298	298
   query87	3157	3110	3034	3034
   query88	3524	2690	2665	2665
   query89	435	372	355	355
   query90	1944	193	183	183
   query91	192	180	163	163
   query92	79	76	77	76
   query93	907	849	511	511
   query94	479	329	296	296
   query95	608	376	395	376
   query96	643	525	230	230
   query97	2512	2471	2419	2419
   query98	250	223	221	221
   query99	1003	993	917	917
   Total cold run time: 248742 ms
   Total hot run time: 169433 ms
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org