You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "w41ter (via GitHub)" <gi...@apache.org> on 2024/03/20 03:20:03 UTC
[PR] [feature](merge-cloud) Support check compatibility mode for FE [doris]
w41ter opened a new pull request, #32509:
URL: https://github.com/apache/doris/pull/32509
## Proposed changes
Issue Number: close #xxx
<!--Describe your changes.-->
## Further comments
If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](merge-cloud) Support check compatibility mode for FE [doris]
Posted by "w41ter (via GitHub)" <gi...@apache.org>.
w41ter commented on PR #32509:
URL: https://github.com/apache/doris/pull/32509#issuecomment-2009377434
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](merge-cloud) Support check compatibility mode for FE [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32509:
URL: https://github.com/apache/doris/pull/32509#issuecomment-2008604266
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR)
Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website).
See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](merge-cloud) Support check compatibility mode for FE [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #32509:
URL: https://github.com/apache/doris/pull/32509#issuecomment-2018219335
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](merge-cloud) Support check compatibility mode for FE [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #32509:
URL: https://github.com/apache/doris/pull/32509#issuecomment-2017481033
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](merge-cloud) Support check compatibility mode for FE [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32509:
URL: https://github.com/apache/doris/pull/32509#issuecomment-2014273546
<details>
<summary>TPC-H: <b>Total hot run time: 38035 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit ba8f77a88c6d75b285c032d290a8bfd64e582479, data reload: false
------ Round 1 ----------------------------------
q1 17686 5984 4249 4249
q2 2737 156 149 149
q3 11668 1133 1222 1133
q4 10750 775 774 774
q5 7693 3100 3005 3005
q6 208 126 126 126
q7 1073 587 575 575
q8 9568 2030 2017 2017
q9 7162 6640 6594 6594
q10 9620 3248 3478 3248
q11 427 219 212 212
q12 366 200 195 195
q13 17811 2847 2854 2847
q14 239 200 211 200
q15 496 464 458 458
q16 486 364 363 363
q17 977 630 578 578
q18 7260 6524 6461 6461
q19 2623 1377 1409 1377
q20 538 261 254 254
q21 3543 2930 3028 2930
q22 338 302 290 290
Total cold run time: 113269 ms
Total hot run time: 38035 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4138 4148 4063 4063
q2 324 232 225 225
q3 2959 2879 2842 2842
q4 1843 1529 1539 1529
q5 5238 5433 5435 5433
q6 202 121 121 121
q7 2342 1898 1936 1898
q8 3220 3306 3334 3306
q9 8599 8624 8633 8624
q10 3673 3708 3684 3684
q11 548 462 442 442
q12 730 532 531 531
q13 16922 2882 2851 2851
q14 273 256 261 256
q15 504 464 458 458
q16 448 424 422 422
q17 1732 1495 1447 1447
q18 7625 7113 7125 7113
q19 1637 1546 1558 1546
q20 1899 1700 1689 1689
q21 4947 4689 4728 4689
q22 510 453 458 453
Total cold run time: 70313 ms
Total hot run time: 53622 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](merge-cloud) Support check compatibility mode for FE [doris]
Posted by "dataroaring (via GitHub)" <gi...@apache.org>.
dataroaring merged PR #32509:
URL: https://github.com/apache/doris/pull/32509
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](merge-cloud) Support check compatibility mode for FE [doris]
Posted by "w41ter (via GitHub)" <gi...@apache.org>.
w41ter commented on PR #32509:
URL: https://github.com/apache/doris/pull/32509#issuecomment-2014248564
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](merge-cloud) Support check compatibility mode for FE [doris]
Posted by "w41ter (via GitHub)" <gi...@apache.org>.
w41ter commented on PR #32509:
URL: https://github.com/apache/doris/pull/32509#issuecomment-2008614309
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](merge-cloud) Support check compatibility mode for FE [doris]
Posted by "w41ter (via GitHub)" <gi...@apache.org>.
w41ter commented on PR #32509:
URL: https://github.com/apache/doris/pull/32509#issuecomment-2008604291
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](merge-cloud) Support check compatibility mode for FE [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32509:
URL: https://github.com/apache/doris/pull/32509#issuecomment-2014285404
Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
```
Load test result on commit ba8f77a88c6d75b285c032d290a8bfd64e582479 with default session variables
Stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc: 59 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select: 20.9 seconds inserted 10000000 Rows, about 478K ops/s
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](merge-cloud) Support check compatibility mode for FE [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32509:
URL: https://github.com/apache/doris/pull/32509#issuecomment-2014283035
<details>
<summary>TPC-DS: <b>Total hot run time: 180373 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit ba8f77a88c6d75b285c032d290a8bfd64e582479, data reload: false
query1 930 365 376 365
query2 7366 2031 1917 1917
query3 6711 211 210 210
query4 31201 20636 20541 20541
query5 4272 421 407 407
query6 265 186 170 170
query7 4621 291 293 291
query8 223 163 168 163
query9 9352 2283 2292 2283
query10 464 238 243 238
query11 15523 13936 14314 13936
query12 139 88 87 87
query13 1631 409 418 409
query14 12833 10719 10399 10399
query15 278 207 195 195
query16 8059 256 259 256
query17 1976 580 527 527
query18 1363 276 282 276
query19 345 148 149 148
query20 92 88 85 85
query21 192 126 124 124
query22 4771 4449 4414 4414
query23 31813 31178 31179 31178
query24 12164 2809 2784 2784
query25 628 367 352 352
query26 1742 149 151 149
query27 3076 339 348 339
query28 7409 1881 1884 1881
query29 943 615 643 615
query30 302 153 147 147
query31 954 714 722 714
query32 93 58 55 55
query33 768 261 337 261
query34 932 481 472 472
query35 848 617 612 612
query36 1011 833 863 833
query37 269 76 78 76
query38 3532 3440 3463 3440
query39 1412 1379 1389 1379
query40 290 111 111 111
query41 50 47 46 46
query42 106 94 97 94
query43 504 445 453 445
query44 1187 722 744 722
query45 266 272 256 256
query46 1084 705 701 701
query47 1688 1613 1597 1597
query48 436 363 356 356
query49 1229 343 340 340
query50 764 369 371 369
query51 6725 6630 6610 6610
query52 103 91 92 91
query53 336 273 284 273
query54 329 253 249 249
query55 85 81 79 79
query56 247 226 233 226
query57 1093 1031 1004 1004
query58 234 207 211 207
query59 2745 2789 2495 2495
query60 269 246 261 246
query61 114 134 130 130
query62 604 413 412 412
query63 301 274 275 274
query64 6778 3866 4002 3866
query65 3087 3047 3023 3023
query66 1319 363 345 345
query67 15224 14435 14333 14333
query68 8571 534 536 534
query69 630 384 383 383
query70 1183 1095 1139 1095
query71 516 280 308 280
query72 6386 2654 2471 2471
query73 748 321 309 309
query74 8295 6527 6555 6527
query75 4134 2736 2803 2736
query76 5232 922 874 874
query77 644 264 246 246
query78 10428 9578 9629 9578
query79 12935 512 511 511
query80 1979 379 376 376
query81 512 209 212 209
query82 780 199 200 199
query83 219 144 137 137
query84 289 76 74 74
query85 1480 342 310 310
query86 482 305 318 305
query87 3737 3515 3520 3515
query88 5651 2304 2294 2294
query89 504 369 358 358
query90 1985 171 172 171
query91 169 133 135 133
query92 63 46 47 46
query93 7549 513 499 499
query94 1196 187 175 175
query95 435 328 327 327
query96 610 267 271 267
query97 3084 2864 2851 2851
query98 221 203 207 203
query99 1241 775 752 752
Total cold run time: 319588 ms
Total hot run time: 180373 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](merge-cloud) Support check compatibility mode for FE [doris]
Posted by "gavinchou (via GitHub)" <gi...@apache.org>.
gavinchou commented on code in PR #32509:
URL: https://github.com/apache/doris/pull/32509#discussion_r1532135109
##########
fe/fe-core/src/main/java/org/apache/doris/catalog/Env.java:
##########
@@ -6110,4 +6128,19 @@ public void checkReadyOrThrowTException() throws TException {
throw new TException(e);
}
}
+
+ private void replayJournalsAndExit() {
+ replayJournal(-1);
+ LOG.info("check metadata compatibility successfully");
+ System.out.println("check metadata compatibility successfully");
+
+ if (Config.checkpoint_after_check_compatibility) {
+ String imagePath = dumpImage();
Review Comment:
could we call a procedure like `Checkpoint.doCheckpoint()` here?
or it is not necessary to keep it the same as a checkpoint procedure.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org