You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "w41ter (via GitHub)" <gi...@apache.org> on 2024/03/20 03:20:03 UTC

[PR] [feature](merge-cloud) Support check compatibility mode for FE [doris]

w41ter opened a new pull request, #32509:
URL: https://github.com/apache/doris/pull/32509

   ## Proposed changes
   
   Issue Number: close #xxx
   
   <!--Describe your changes.-->
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [feature](merge-cloud) Support check compatibility mode for FE [doris]

Posted by "w41ter (via GitHub)" <gi...@apache.org>.
w41ter commented on PR #32509:
URL: https://github.com/apache/doris/pull/32509#issuecomment-2009377434

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [feature](merge-cloud) Support check compatibility mode for FE [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32509:
URL: https://github.com/apache/doris/pull/32509#issuecomment-2008604266

   Thank you for your contribution to Apache Doris.
   Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR)
   
   Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website).
   See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [feature](merge-cloud) Support check compatibility mode for FE [doris]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #32509:
URL: https://github.com/apache/doris/pull/32509#issuecomment-2018219335

   PR approved by at least one committer and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [feature](merge-cloud) Support check compatibility mode for FE [doris]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #32509:
URL: https://github.com/apache/doris/pull/32509#issuecomment-2017481033

   PR approved by anyone and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [feature](merge-cloud) Support check compatibility mode for FE [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32509:
URL: https://github.com/apache/doris/pull/32509#issuecomment-2014273546

   
   <details>
   <summary>TPC-H: <b>Total hot run time: 38035 ms</b></summary>
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
   Tpch sf100 test result on commit ba8f77a88c6d75b285c032d290a8bfd64e582479, data reload: false
   
   ------ Round 1 ----------------------------------
   q1	17686	5984	4249	4249
   q2	2737	156	149	149
   q3	11668	1133	1222	1133
   q4	10750	775	774	774
   q5	7693	3100	3005	3005
   q6	208	126	126	126
   q7	1073	587	575	575
   q8	9568	2030	2017	2017
   q9	7162	6640	6594	6594
   q10	9620	3248	3478	3248
   q11	427	219	212	212
   q12	366	200	195	195
   q13	17811	2847	2854	2847
   q14	239	200	211	200
   q15	496	464	458	458
   q16	486	364	363	363
   q17	977	630	578	578
   q18	7260	6524	6461	6461
   q19	2623	1377	1409	1377
   q20	538	261	254	254
   q21	3543	2930	3028	2930
   q22	338	302	290	290
   Total cold run time: 113269 ms
   Total hot run time: 38035 ms
   
   ----- Round 2, with runtime_filter_mode=off -----
   q1	4138	4148	4063	4063
   q2	324	232	225	225
   q3	2959	2879	2842	2842
   q4	1843	1529	1539	1529
   q5	5238	5433	5435	5433
   q6	202	121	121	121
   q7	2342	1898	1936	1898
   q8	3220	3306	3334	3306
   q9	8599	8624	8633	8624
   q10	3673	3708	3684	3684
   q11	548	462	442	442
   q12	730	532	531	531
   q13	16922	2882	2851	2851
   q14	273	256	261	256
   q15	504	464	458	458
   q16	448	424	422	422
   q17	1732	1495	1447	1447
   q18	7625	7113	7125	7113
   q19	1637	1546	1558	1546
   q20	1899	1700	1689	1689
   q21	4947	4689	4728	4689
   q22	510	453	458	453
   Total cold run time: 70313 ms
   Total hot run time: 53622 ms
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [feature](merge-cloud) Support check compatibility mode for FE [doris]

Posted by "dataroaring (via GitHub)" <gi...@apache.org>.
dataroaring merged PR #32509:
URL: https://github.com/apache/doris/pull/32509


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [feature](merge-cloud) Support check compatibility mode for FE [doris]

Posted by "w41ter (via GitHub)" <gi...@apache.org>.
w41ter commented on PR #32509:
URL: https://github.com/apache/doris/pull/32509#issuecomment-2014248564

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [feature](merge-cloud) Support check compatibility mode for FE [doris]

Posted by "w41ter (via GitHub)" <gi...@apache.org>.
w41ter commented on PR #32509:
URL: https://github.com/apache/doris/pull/32509#issuecomment-2008614309

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [feature](merge-cloud) Support check compatibility mode for FE [doris]

Posted by "w41ter (via GitHub)" <gi...@apache.org>.
w41ter commented on PR #32509:
URL: https://github.com/apache/doris/pull/32509#issuecomment-2008604291

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [feature](merge-cloud) Support check compatibility mode for FE [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32509:
URL: https://github.com/apache/doris/pull/32509#issuecomment-2014285404

   
   Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   ```
   Load test result on commit ba8f77a88c6d75b285c032d290a8bfd64e582479 with default session variables
   Stream load json:         19 seconds loaded 2358488459 Bytes, about 118 MB/s
   Stream load orc:          59 seconds loaded 1101869774 Bytes, about 17 MB/s
   Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
   Insert into select:       20.9 seconds inserted 10000000 Rows, about 478K ops/s
   ```
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [feature](merge-cloud) Support check compatibility mode for FE [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32509:
URL: https://github.com/apache/doris/pull/32509#issuecomment-2014283035

   
   <details>
   <summary>TPC-DS: <b>Total hot run time: 180373 ms</b></summary>
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
   TPC-DS sf100 test result on commit ba8f77a88c6d75b285c032d290a8bfd64e582479, data reload: false
   
   query1	930	365	376	365
   query2	7366	2031	1917	1917
   query3	6711	211	210	210
   query4	31201	20636	20541	20541
   query5	4272	421	407	407
   query6	265	186	170	170
   query7	4621	291	293	291
   query8	223	163	168	163
   query9	9352	2283	2292	2283
   query10	464	238	243	238
   query11	15523	13936	14314	13936
   query12	139	88	87	87
   query13	1631	409	418	409
   query14	12833	10719	10399	10399
   query15	278	207	195	195
   query16	8059	256	259	256
   query17	1976	580	527	527
   query18	1363	276	282	276
   query19	345	148	149	148
   query20	92	88	85	85
   query21	192	126	124	124
   query22	4771	4449	4414	4414
   query23	31813	31178	31179	31178
   query24	12164	2809	2784	2784
   query25	628	367	352	352
   query26	1742	149	151	149
   query27	3076	339	348	339
   query28	7409	1881	1884	1881
   query29	943	615	643	615
   query30	302	153	147	147
   query31	954	714	722	714
   query32	93	58	55	55
   query33	768	261	337	261
   query34	932	481	472	472
   query35	848	617	612	612
   query36	1011	833	863	833
   query37	269	76	78	76
   query38	3532	3440	3463	3440
   query39	1412	1379	1389	1379
   query40	290	111	111	111
   query41	50	47	46	46
   query42	106	94	97	94
   query43	504	445	453	445
   query44	1187	722	744	722
   query45	266	272	256	256
   query46	1084	705	701	701
   query47	1688	1613	1597	1597
   query48	436	363	356	356
   query49	1229	343	340	340
   query50	764	369	371	369
   query51	6725	6630	6610	6610
   query52	103	91	92	91
   query53	336	273	284	273
   query54	329	253	249	249
   query55	85	81	79	79
   query56	247	226	233	226
   query57	1093	1031	1004	1004
   query58	234	207	211	207
   query59	2745	2789	2495	2495
   query60	269	246	261	246
   query61	114	134	130	130
   query62	604	413	412	412
   query63	301	274	275	274
   query64	6778	3866	4002	3866
   query65	3087	3047	3023	3023
   query66	1319	363	345	345
   query67	15224	14435	14333	14333
   query68	8571	534	536	534
   query69	630	384	383	383
   query70	1183	1095	1139	1095
   query71	516	280	308	280
   query72	6386	2654	2471	2471
   query73	748	321	309	309
   query74	8295	6527	6555	6527
   query75	4134	2736	2803	2736
   query76	5232	922	874	874
   query77	644	264	246	246
   query78	10428	9578	9629	9578
   query79	12935	512	511	511
   query80	1979	379	376	376
   query81	512	209	212	209
   query82	780	199	200	199
   query83	219	144	137	137
   query84	289	76	74	74
   query85	1480	342	310	310
   query86	482	305	318	305
   query87	3737	3515	3520	3515
   query88	5651	2304	2294	2294
   query89	504	369	358	358
   query90	1985	171	172	171
   query91	169	133	135	133
   query92	63	46	47	46
   query93	7549	513	499	499
   query94	1196	187	175	175
   query95	435	328	327	327
   query96	610	267	271	267
   query97	3084	2864	2851	2851
   query98	221	203	207	203
   query99	1241	775	752	752
   Total cold run time: 319588 ms
   Total hot run time: 180373 ms
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [feature](merge-cloud) Support check compatibility mode for FE [doris]

Posted by "gavinchou (via GitHub)" <gi...@apache.org>.
gavinchou commented on code in PR #32509:
URL: https://github.com/apache/doris/pull/32509#discussion_r1532135109


##########
fe/fe-core/src/main/java/org/apache/doris/catalog/Env.java:
##########
@@ -6110,4 +6128,19 @@ public void checkReadyOrThrowTException() throws TException {
             throw new TException(e);
         }
     }
+
+    private void replayJournalsAndExit() {
+        replayJournal(-1);
+        LOG.info("check metadata compatibility successfully");
+        System.out.println("check metadata compatibility successfully");
+
+        if (Config.checkpoint_after_check_compatibility) {
+            String imagePath = dumpImage();

Review Comment:
   could we call a procedure like `Checkpoint.doCheckpoint()` here? 
   or it is not necessary to keep it the same as a checkpoint procedure.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org