You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "w41ter (via GitHub)" <gi...@apache.org> on 2024/03/19 09:33:28 UTC

[PR] [fix](merge-cloud) Set visible version for OlapScanNode at the plan phase [doris]

w41ter opened a new pull request, #32473:
URL: https://github.com/apache/doris/pull/32473

   
   
   ## Proposed changes
   
   Issue Number: close #xxx
   
   This PR fixes the spark read with an incorrect partition version in the cloud mode
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](merge-cloud) Set visible version for OlapScanNode at the plan phase [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32473:
URL: https://github.com/apache/doris/pull/32473#issuecomment-2011718169

   
   <details>
   <summary>TPC-DS: <b>Total hot run time: 182117 ms</b></summary>
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
   TPC-DS sf100 test result on commit b22f5fe1127d57eadb74c1f5a587d7c22233f97e, data reload: false
   
   query1	919	366	347	347
   query2	7539	2069	1954	1954
   query3	6705	224	214	214
   query4	31049	21118	20725	20725
   query5	4311	417	413	413
   query6	267	195	171	171
   query7	4629	290	287	287
   query8	231	169	170	169
   query9	9870	2278	2243	2243
   query10	457	240	260	240
   query11	17231	14347	14270	14270
   query12	137	94	85	85
   query13	1644	415	401	401
   query14	11615	10580	10731	10580
   query15	273	208	199	199
   query16	8115	294	266	266
   query17	1910	588	542	542
   query18	2067	289	294	289
   query19	243	163	155	155
   query20	92	91	88	88
   query21	200	130	129	129
   query22	4682	4476	4386	4386
   query23	31842	31390	31371	31371
   query24	10706	2884	2801	2801
   query25	624	388	381	381
   query26	1183	160	159	159
   query27	2465	347	348	347
   query28	7615	1911	1905	1905
   query29	877	658	629	629
   query30	308	147	148	147
   query31	966	756	736	736
   query32	94	62	58	58
   query33	784	277	342	277
   query34	1011	477	482	477
   query35	833	606	613	606
   query36	976	856	902	856
   query37	129	81	76	76
   query38	3561	3480	3505	3480
   query39	1412	1394	1380	1380
   query40	221	120	112	112
   query41	49	48	50	48
   query42	105	99	96	96
   query43	511	445	457	445
   query44	1210	717	721	717
   query45	280	265	254	254
   query46	1084	675	683	675
   query47	1659	1597	1591	1591
   query48	431	359	359	359
   query49	1127	332	334	332
   query50	763	369	371	369
   query51	6664	6547	6651	6547
   query52	100	89	84	84
   query53	337	272	274	272
   query54	304	242	259	242
   query55	82	75	77	75
   query56	244	294	228	228
   query57	1074	1016	1025	1016
   query58	228	219	203	203
   query59	2967	2703	2572	2572
   query60	261	240	245	240
   query61	99	94	98	94
   query62	602	406	409	406
   query63	300	278	276	276
   query64	5624	3872	3936	3872
   query65	3084	3004	3006	3004
   query66	855	375	338	338
   query67	14702	14607	14485	14485
   query68	5328	505	506	505
   query69	571	377	377	377
   query70	1253	1146	1176	1146
   query71	420	277	272	272
   query72	6365	2794	2692	2692
   query73	726	318	319	318
   query74	6867	6533	6598	6533
   query75	3605	2868	2898	2868
   query76	3578	882	1001	882
   query77	625	275	272	272
   query78	10359	9574	9619	9574
   query79	7746	516	521	516
   query80	1490	420	404	404
   query81	531	214	222	214
   query82	1445	201	218	201
   query83	209	153	148	148
   query84	286	83	85	83
   query85	1582	388	351	351
   query86	469	307	307	307
   query87	3740	3590	3575	3575
   query88	5184	2287	2272	2272
   query89	474	367	367	367
   query90	2021	180	177	177
   query91	165	157	149	149
   query92	58	49	47	47
   query93	5821	498	474	474
   query94	1154	179	177	177
   query95	438	338	330	330
   query96	586	271	267	267
   query97	3066	2886	2862	2862
   query98	217	210	207	207
   query99	1066	786	759	759
   Total cold run time: 301984 ms
   Total hot run time: 182117 ms
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](merge-cloud) Set visible version for OlapScanNode at the plan phase [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32473:
URL: https://github.com/apache/doris/pull/32473#issuecomment-2006507504

   Thank you for your contribution to Apache Doris.
   Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR)
   
   Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website).
   See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](merge-cloud) Set visible version for OlapScanNode at the plan phase [doris]

Posted by "w41ter (via GitHub)" <gi...@apache.org>.
w41ter commented on PR #32473:
URL: https://github.com/apache/doris/pull/32473#issuecomment-2006507601

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](merge-cloud) Set visible version for OlapScanNode at the plan phase [doris]

Posted by "w41ter (via GitHub)" <gi...@apache.org>.
w41ter commented on PR #32473:
URL: https://github.com/apache/doris/pull/32473#issuecomment-2011403510

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](merge-cloud) Set visible version for OlapScanNode at the plan phase [doris]

Posted by "dataroaring (via GitHub)" <gi...@apache.org>.
dataroaring merged PR #32473:
URL: https://github.com/apache/doris/pull/32473


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](merge-cloud) Set visible version for OlapScanNode at the plan phase [doris]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #32473:
URL: https://github.com/apache/doris/pull/32473#issuecomment-2006572467

   PR approved by anyone and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](merge-cloud) Set visible version for OlapScanNode at the plan phase [doris]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #32473:
URL: https://github.com/apache/doris/pull/32473#issuecomment-2011664168

   PR approved by at least one committer and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](merge-cloud) Set visible version for OlapScanNode at the plan phase [doris]

Posted by "w41ter (via GitHub)" <gi...@apache.org>.
w41ter commented on PR #32473:
URL: https://github.com/apache/doris/pull/32473#issuecomment-2009380877

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](merge-cloud) Set visible version for OlapScanNode at the plan phase [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32473:
URL: https://github.com/apache/doris/pull/32473#issuecomment-2011694260

   
   <details>
   <summary>TPC-H: <b>Total hot run time: 38055 ms</b></summary>
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
   Tpch sf100 test result on commit b22f5fe1127d57eadb74c1f5a587d7c22233f97e, data reload: false
   
   ------ Round 1 ----------------------------------
   q1	17613	4335	4318	4318
   q2	2032	148	145	145
   q3	10606	1127	1186	1127
   q4	10233	840	849	840
   q5	7470	3015	2944	2944
   q6	204	125	124	124
   q7	1046	580	568	568
   q8	9327	1994	2010	1994
   q9	7113	6541	6491	6491
   q10	8380	3444	3534	3444
   q11	430	216	226	216
   q12	415	203	198	198
   q13	17807	2846	2850	2846
   q14	226	206	209	206
   q15	512	466	456	456
   q16	472	353	358	353
   q17	962	617	544	544
   q18	7214	6633	6407	6407
   q19	4479	1458	1489	1458
   q20	546	251	282	251
   q21	3620	2919	2834	2834
   q22	334	301	291	291
   Total cold run time: 111041 ms
   Total hot run time: 38055 ms
   
   ----- Round 2, with runtime_filter_mode=off -----
   q1	4191	4085	4099	4085
   q2	316	220	225	220
   q3	2985	2892	2827	2827
   q4	1858	1550	1596	1550
   q5	5240	5296	5293	5293
   q6	198	117	118	117
   q7	2252	1857	1824	1824
   q8	3140	3378	3277	3277
   q9	8636	8581	8577	8577
   q10	3735	3649	3718	3649
   q11	551	457	442	442
   q12	715	571	536	536
   q13	16934	2878	2865	2865
   q14	270	244	254	244
   q15	486	443	451	443
   q16	464	424	436	424
   q17	1736	1481	1490	1481
   q18	7371	7180	7214	7180
   q19	1628	1584	1509	1509
   q20	1894	1697	1687	1687
   q21	4898	4730	4779	4730
   q22	528	478	483	478
   Total cold run time: 70026 ms
   Total hot run time: 53438 ms
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](merge-cloud) Set visible version for OlapScanNode at the plan phase [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32473:
URL: https://github.com/apache/doris/pull/32473#issuecomment-2011787389

   
   <details>
   <summary>TPC-H: <b>Total hot run time: 38374 ms</b></summary>
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
   Tpch sf100 test result on commit b22f5fe1127d57eadb74c1f5a587d7c22233f97e, data reload: false
   
   ------ Round 1 ----------------------------------
   q1	17775	4521	4436	4436
   q2	2991	173	150	150
   q3	11925	1159	1223	1159
   q4	10934	776	755	755
   q5	7633	3033	3025	3025
   q6	208	128	126	126
   q7	1061	619	586	586
   q8	9624	2093	2049	2049
   q9	7180	6633	6604	6604
   q10	9442	3237	3524	3237
   q11	430	225	214	214
   q12	375	194	196	194
   q13	17851	2841	2831	2831
   q14	235	217	204	204
   q15	497	466	457	457
   q16	468	361	356	356
   q17	973	574	604	574
   q18	7262	6538	6512	6512
   q19	1560	1473	1509	1473
   q20	542	249	248	248
   q21	3636	2972	2879	2879
   q22	350	305	314	305
   Total cold run time: 112952 ms
   Total hot run time: 38374 ms
   
   ----- Round 2, with runtime_filter_mode=off -----
   q1	4144	4166	4219	4166
   q2	331	244	226	226
   q3	3046	2862	2857	2857
   q4	1839	1547	1519	1519
   q5	5239	5298	5277	5277
   q6	194	119	119	119
   q7	2260	1876	1920	1876
   q8	3183	3307	3307	3307
   q9	8619	8634	8621	8621
   q10	3741	3662	3723	3662
   q11	546	443	434	434
   q12	741	546	564	546
   q13	16874	2889	2842	2842
   q14	274	246	251	246
   q15	490	452	454	452
   q16	468	410	414	410
   q17	1741	1507	1476	1476
   q18	7514	7168	7208	7168
   q19	2963	1588	1524	1524
   q20	1907	1694	1697	1694
   q21	4837	4853	4698	4698
   q22	532	437	467	437
   Total cold run time: 71483 ms
   Total hot run time: 53557 ms
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](merge-cloud) Set visible version for OlapScanNode at the plan phase [doris]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #32473:
URL: https://github.com/apache/doris/pull/32473#issuecomment-2006572348

   PR approved by at least one committer and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](merge-cloud) Set visible version for OlapScanNode at the plan phase [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32473:
URL: https://github.com/apache/doris/pull/32473#issuecomment-2011813565

   
   <details>
   <summary>TPC-DS: <b>Total hot run time: 181315 ms</b></summary>
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
   TPC-DS sf100 test result on commit b22f5fe1127d57eadb74c1f5a587d7c22233f97e, data reload: false
   
   query1	915	351	338	338
   query2	7631	2040	1983	1983
   query3	6704	212	215	212
   query4	31255	20786	20757	20757
   query5	4365	487	410	410
   query6	267	179	190	179
   query7	4621	291	295	291
   query8	248	189	167	167
   query9	9899	2273	2278	2273
   query10	457	251	239	239
   query11	14599	14263	14261	14261
   query12	136	94	93	93
   query13	1645	416	412	412
   query14	11716	10636	11143	10636
   query15	254	203	190	190
   query16	8089	253	256	253
   query17	1954	556	512	512
   query18	2086	274	261	261
   query19	216	145	164	145
   query20	93	85	83	83
   query21	197	125	132	125
   query22	4669	4536	4432	4432
   query23	31839	31324	31195	31195
   query24	11622	2831	2812	2812
   query25	642	381	385	381
   query26	1788	149	163	149
   query27	3077	342	340	340
   query28	8032	1858	1906	1858
   query29	1016	625	610	610
   query30	300	145	146	145
   query31	946	733	704	704
   query32	94	57	52	52
   query33	754	250	244	244
   query34	1106	461	489	461
   query35	828	604	594	594
   query36	949	877	866	866
   query37	115	77	73	73
   query38	3560	3454	3454	3454
   query39	1415	1359	1379	1359
   query40	290	110	109	109
   query41	52	45	43	43
   query42	99	92	97	92
   query43	496	477	479	477
   query44	1210	730	717	717
   query45	270	252	255	252
   query46	1059	674	674	674
   query47	1693	1592	1588	1588
   query48	434	340	350	340
   query49	1188	326	344	326
   query50	768	363	368	363
   query51	6655	6560	6617	6560
   query52	104	98	88	88
   query53	336	273	277	273
   query54	317	235	254	235
   query55	85	83	73	73
   query56	261	241	237	237
   query57	1089	1002	992	992
   query58	237	207	206	206
   query59	2877	2626	2694	2626
   query60	289	250	256	250
   query61	115	115	112	112
   query62	603	395	393	393
   query63	304	277	281	277
   query64	6783	3890	3765	3765
   query65	3064	3025	2988	2988
   query66	1108	356	352	352
   query67	14821	14433	14417	14417
   query68	5392	510	526	510
   query69	553	386	374	374
   query70	1218	1176	1211	1176
   query71	418	283	289	283
   query72	6362	2789	2608	2608
   query73	713	316	313	313
   query74	6968	6531	6604	6531
   query75	3476	2858	2869	2858
   query76	3395	808	1043	808
   query77	379	253	259	253
   query78	10300	9664	9574	9574
   query79	9453	505	516	505
   query80	2267	372	375	372
   query81	542	208	208	208
   query82	1420	202	203	202
   query83	316	141	147	141
   query84	279	83	81	81
   query85	2160	328	306	306
   query86	503	266	309	266
   query87	3680	3504	3534	3504
   query88	5111	2282	2286	2282
   query89	552	367	381	367
   query90	2010	178	175	175
   query91	167	129	132	129
   query92	69	49	49	49
   query93	7122	496	483	483
   query94	1257	177	174	174
   query95	433	323	326	323
   query96	597	272	263	263
   query97	3054	2872	2872	2872
   query98	226	220	205	205
   query99	1114	764	746	746
   Total cold run time: 308212 ms
   Total hot run time: 181315 ms
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [fix](merge-cloud) Set visible version for OlapScanNode at the plan phase [doris]

Posted by "dataroaring (via GitHub)" <gi...@apache.org>.
dataroaring commented on PR #32473:
URL: https://github.com/apache/doris/pull/32473#issuecomment-2011037096

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org