You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "mymeiyi (via GitHub)" <gi...@apache.org> on 2026/02/12 06:23:42 UTC
[PR] [fix](cloud) checkpoint save cloud version and tablet stats to image [doris]
mymeiyi opened a new pull request, #60705:
URL: https://github.com/apache/doris/pull/60705
### What problem does this PR solve?
Issue Number: close #xxx
Related PR: #xxx
Problem Summary:
### Release note
None
### Check List (For Author)
- Test <!-- At least one of them must be included. -->
- [ ] Regression test
- [ ] Unit Test
- [ ] Manual test (add detailed scripts or steps below)
- [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
- [ ] Previous test can cover this change.
- [ ] No code files have been changed.
- [ ] Other reason <!-- Add your reason? -->
- Behavior changed:
- [ ] No.
- [ ] Yes. <!-- Explain the behavior change -->
- Does this need documentation?
- [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg: https://github.com/apache/doris-website/pull/1214 -->
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR should merge into -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](cloud) checkpoint save cloud version and tablet stats to image [doris]
Posted by "mymeiyi (via GitHub)" <gi...@apache.org>.
mymeiyi commented on PR #60705:
URL: https://github.com/apache/doris/pull/60705#issuecomment-3888947432
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](cloud) checkpoint save cloud version and tablet stats to image [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #60705:
URL: https://github.com/apache/doris/pull/60705#issuecomment-3889407399
<details>
<summary>ClickBench: <b>Total hot run time: 28.85 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit f32e8e3b943cde1ee8c2d00157c6b2b77c387ac0, data reload: false
query1 0.05 0.05 0.05
query2 0.09 0.04 0.04
query3 0.25 0.08 0.08
query4 1.61 0.11 0.10
query5 0.27 0.25 0.25
query6 1.15 0.68 0.67
query7 0.03 0.03 0.02
query8 0.05 0.04 0.03
query9 0.57 0.49 0.50
query10 0.56 0.54 0.55
query11 0.14 0.09 0.09
query12 0.13 0.11 0.10
query13 0.63 0.60 0.61
query14 1.06 1.05 1.05
query15 0.89 0.88 0.88
query16 0.41 0.39 0.42
query17 1.12 1.15 1.11
query18 0.22 0.21 0.21
query19 2.09 2.02 2.05
query20 0.02 0.02 0.01
query21 15.41 0.26 0.15
query22 5.13 0.06 0.05
query23 15.87 0.29 0.11
query24 1.47 0.82 0.90
query25 0.10 0.12 0.14
query26 0.14 0.14 0.13
query27 0.08 0.06 0.06
query28 4.89 1.13 0.96
query29 12.56 3.98 3.18
query30 0.27 0.14 0.12
query31 2.81 0.67 0.40
query32 3.23 0.58 0.50
query33 3.27 3.28 3.20
query34 16.35 5.39 4.68
query35 4.76 4.79 4.76
query36 0.65 0.50 0.51
query37 0.11 0.07 0.07
query38 0.07 0.04 0.04
query39 0.05 0.03 0.03
query40 0.19 0.16 0.16
query41 0.09 0.04 0.03
query42 0.04 0.02 0.02
query43 0.05 0.03 0.04
Total cold run time: 98.93 s
Total hot run time: 28.85 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](cloud) checkpoint save cloud tablet stats to image [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #60705:
URL: https://github.com/apache/doris/pull/60705#issuecomment-4045876969
# FE UT Coverage Report
Increment line coverage ` 6.90% (6/87)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/60705_d3a9bc8692f529d25e85cc464de3f8d230680c39/fe_increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/60705_d3a9bc8692f529d25e85cc464de3f8d230680c39/fe_report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](cloud) checkpoint save cloud tablet stats to image [doris]
Posted by "mymeiyi (via GitHub)" <gi...@apache.org>.
mymeiyi commented on PR #60705:
URL: https://github.com/apache/doris/pull/60705#issuecomment-4051793255
run p0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](cloud) checkpoint save cloud version and tablet stats to image [doris]
Posted by "w41ter (via GitHub)" <gi...@apache.org>.
w41ter commented on code in PR #60705:
URL: https://github.com/apache/doris/pull/60705#discussion_r2866996850
##########
fe/fe-core/src/main/java/org/apache/doris/master/Checkpoint.java:
##########
@@ -146,6 +166,7 @@ public synchronized void doCheckpoint() throws CheckpointException {
checkPointVersion, env.getReplayedJournalId()));
}
env.postProcessAfterMetadataReplayed(false);
+ postProcessCloudMetadata();
Review Comment:
这里：
olapTable.setCachedTableVersion(servingOlapTable.getCachedTableVersion());
直接用的最新的 table version，是不是会存在 image 中存储的 table version 大于 image version 的情况？
比如拿到 checkPointVersion 后，后续有更新，postProcessCloudMetadata 时直接记录了对 last journal id 可见的这些 partition/table version。
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](cloud) checkpoint save cloud version and tablet stats to image [doris]
Posted by "mymeiyi (via GitHub)" <gi...@apache.org>.
mymeiyi commented on PR #60705:
URL: https://github.com/apache/doris/pull/60705#issuecomment-3949418128
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](cloud) checkpoint save cloud version and tablet stats to image [doris]
Posted by "Copilot (via GitHub)" <gi...@apache.org>.
Copilot commented on code in PR #60705:
URL: https://github.com/apache/doris/pull/60705#discussion_r2797003852
##########
fe/fe-core/src/main/java/org/apache/doris/master/Checkpoint.java:
##########
@@ -146,6 +154,7 @@ public synchronized void doCheckpoint() throws CheckpointException {
checkPointVersion, env.getReplayedJournalId()));
}
env.postProcessAfterMetadataReplayed(false);
+ postProcessCloudMetadata();
latestImageFilePath = env.saveImage();
Review Comment:
This introduces new cloud-mode checkpoint behavior (`postProcessCloudMetadata`) that mutates the replayed catalog before `saveImage()`. There are existing tests that exercise checkpoint serialization/deserialization, but nothing here asserts that table cached versions / partition versions / replica stats are actually persisted and restored correctly in cloud mode. Please add a unit/integration test that creates an OlapTable + replicas, sets these fields, runs checkpoint image serialize/deserialize, and verifies the values survive.
##########
fe/fe-core/src/main/java/org/apache/doris/catalog/OlapTable.java:
##########
@@ -240,6 +240,7 @@ public enum OlapTableState {
// Cache for table version in cloud mode
// This value is set when get the table version from meta-service, 0 means version is not cached yet
private volatile long lastTableVersionCachedTimeMs = 0;
+ @SerializedName(value = "cv")
private volatile long cachedTableVersion = -1;
Review Comment:
`cachedTableVersion` is now serialized, but `lastTableVersionCachedTimeMs` is still not serialized (and will reset to 0 when loading from image). That makes the cache look immediately expired after restart, so `getVisibleVersion()` will still go to meta-service and the persisted `cachedTableVersion` likely won’t be used. Consider either persisting `lastTableVersionCachedTimeMs` as well, or (preferred) initializing `lastTableVersionCachedTimeMs` during gson post-process when a non-(-1) cached version is loaded so the cache is treated as fresh at image load time.
##########
fe/fe-core/src/main/java/org/apache/doris/master/Checkpoint.java:
##########
@@ -395,4 +404,99 @@ private long getMemoryUsedPercent() {
public ReentrantReadWriteLock getLock() {
return lock;
}
+
+ private void postProcessCloudMetadata() {
+ if (Config.isNotCloudMode()) {
+ return;
+ }
+ Env servingEnv = Env.getServingEnv();
+ if (servingEnv == null) {
+ LOG.warn("serving env is null, skip process cloud metadata for checkpoint");
+ return;
+ }
+ long start = System.currentTimeMillis();
+ for (Database db : env.getInternalCatalog().getDbs()) {
+ Database servingDb = servingEnv.getInternalCatalog().getDbNullable(db.getId());
+ if (servingDb == null) {
+ LOG.warn("serving db is null. dbId: {}, dbName: {}", db.getId(), db.getFullName());
+ continue;
+ }
Review Comment:
The checkpoint env is replayed only up to `checkPointVersion`, so it’s expected that some db/table/partition/index objects may not exist (or may differ) in `servingEnv` due to concurrent DDLs after the checkpoint snapshot point. Logging each mismatch at WARN can create noisy logs during normal operation; consider downgrading these to DEBUG or aggregating counts and logging a single WARN/INFO summary per checkpoint.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](cloud) checkpoint save cloud version and tablet stats to image [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #60705:
URL: https://github.com/apache/doris/pull/60705#issuecomment-3889375126
<details>
<summary>TPC-DS: <b>Total hot run time: 189007 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit f32e8e3b943cde1ee8c2d00157c6b2b77c387ac0, data reload: false
query5 4400 627 520 520
query6 315 224 204 204
query7 4220 465 259 259
query8 332 250 233 233
query9 8725 2715 2740 2715
query10 523 370 325 325
query11 17287 17070 16828 16828
query12 179 129 131 129
query13 1257 447 352 352
query14 6052 3233 2917 2917
query14_1 2836 2774 2750 2750
query15 213 189 177 177
query16 968 477 449 449
query17 1084 688 595 595
query18 2439 433 340 340
query19 211 209 183 183
query20 138 128 129 128
query21 229 146 127 127
query22 4774 4877 4982 4877
query23 17444 17016 16681 16681
query23_1 16957 16851 16774 16774
query24 7233 1613 1234 1234
query24_1 1238 1240 1236 1236
query25 552 464 443 443
query26 1241 280 159 159
query27 2747 457 295 295
query28 4543 1879 1869 1869
query29 815 561 476 476
query30 321 261 215 215
query31 882 740 645 645
query32 88 82 77 77
query33 541 344 303 303
query34 927 921 571 571
query35 646 679 595 595
query36 1113 1148 972 972
query37 140 102 87 87
query38 2990 2931 2856 2856
query39 891 884 836 836
query39_1 807 914 812 812
query40 216 131 131 131
query41 65 64 67 64
query42 107 103 103 103
query43 397 399 352 352
query44 1311 714 713 713
query45 200 189 185 185
query46 882 985 599 599
query47 2132 2140 2064 2064
query48 301 311 231 231
query49 608 433 343 343
query50 687 280 209 209
query51 4190 4098 4082 4082
query52 103 105 96 96
query53 296 339 288 288
query54 299 276 259 259
query55 88 89 79 79
query56 317 305 312 305
query57 1363 1329 1261 1261
query58 285 272 265 265
query59 2613 2698 2592 2592
query60 348 333 306 306
query61 150 146 147 146
query62 622 595 539 539
query63 314 278 275 275
query64 4900 1235 952 952
query65 4623 4531 4530 4530
query66 1478 461 351 351
query67 16551 16559 16405 16405
query68 2501 1078 718 718
query69 404 310 270 270
query70 1020 965 1001 965
query71 337 313 298 298
query72 2938 2778 2591 2591
query73 532 544 320 320
query74 9605 9544 9325 9325
query75 2789 2757 2418 2418
query76 2296 1061 655 655
query77 359 372 310 310
query78 10893 11062 10415 10415
query79 1068 945 618 618
query80 1297 568 509 509
query81 543 279 249 249
query82 1267 150 114 114
query83 347 265 242 242
query84 244 119 104 104
query85 931 471 419 419
query86 415 331 299 299
query87 3140 3098 2974 2974
query88 3550 2680 2648 2648
query89 435 374 349 349
query90 1936 173 169 169
query91 167 162 131 131
query92 74 69 70 69
query93 896 854 496 496
query94 645 325 305 305
query95 591 408 324 324
query96 637 503 228 228
query97 2465 2470 2402 2402
query98 226 215 217 215
query99 1015 985 927 927
Total cold run time: 261421 ms
Total hot run time: 189007 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](cloud) checkpoint save cloud version and tablet stats to image [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #60705:
URL: https://github.com/apache/doris/pull/60705#issuecomment-3949629855
<details>
<summary>TPC-H: <b>Total hot run time: 28693 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 66cffc1356fb02c956d56aa828528ee6eed4f439, data reload: false
------ Round 1 ----------------------------------
============================================
q1 17613 4491 4274 4274
q2 q3 10652 785 521 521
q4 4683 363 257 257
q5 7549 1188 1022 1022
q6 177 176 143 143
q7 778 826 667 667
q8 9287 1452 1326 1326
q9 4799 4724 4769 4724
q10 6811 1876 1641 1641
q11 461 255 244 244
q12 689 557 464 464
q13 17767 4210 3418 3418
q14 225 225 215 215
q15 980 804 792 792
q16 752 727 665 665
q17 706 907 440 440
q18 6183 5404 5246 5246
q19 1106 970 622 622
q20 498 482 393 393
q21 4350 1853 1374 1374
q22 341 292 245 245
Total cold run time: 96407 ms
Total hot run time: 28693 ms
----- Round 2, with runtime_filter_mode=off -----
============================================
q1 4393 4339 4372 4339
q2 q3 1769 2164 1740 1740
q4 836 1150 764 764
q5 4016 4334 4378 4334
q6 179 171 142 142
q7 1696 1583 1475 1475
q8 2424 2698 2577 2577
q9 7449 7573 7475 7475
q10 2709 2895 2451 2451
q11 529 490 421 421
q12 515 589 459 459
q13 4119 4469 3599 3599
q14 304 310 296 296
q15 877 816 802 802
q16 733 787 765 765
q17 1218 1543 1364 1364
q18 7048 6860 6736 6736
q19 929 910 920 910
q20 2140 2251 2040 2040
q21 4056 3726 3388 3388
q22 501 467 391 391
Total cold run time: 48440 ms
Total hot run time: 46468 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](cloud) checkpoint save cloud tablet stats to image [doris]
Posted by "mymeiyi (via GitHub)" <gi...@apache.org>.
mymeiyi commented on PR #60705:
URL: https://github.com/apache/doris/pull/60705#issuecomment-4087642949
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](cloud) checkpoint save cloud version and tablet stats to image [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #60705:
URL: https://github.com/apache/doris/pull/60705#issuecomment-3996623223
<details>
<summary>TPC-H: <b>Total hot run time: 28889 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 4a34f9b1925725b582e045bd9e544d8b091cb53e, data reload: false
------ Round 1 ----------------------------------
============================================
q1 17623 4448 4294 4294
q2 q3 10647 772 521 521
q4 4684 362 255 255
q5 7538 1201 1013 1013
q6 175 179 150 150
q7 807 825 663 663
q8 9299 1475 1319 1319
q9 4797 4786 4770 4770
q10 6794 1855 1652 1652
q11 454 267 246 246
q12 704 572 472 472
q13 17758 4297 3427 3427
q14 230 229 212 212
q15 915 819 789 789
q16 736 708 683 683
q17 721 834 453 453
q18 5977 5325 5192 5192
q19 1251 974 612 612
q20 499 486 398 398
q21 5040 1995 1505 1505
q22 356 308 263 263
Total cold run time: 97005 ms
Total hot run time: 28889 ms
----- Round 2, with runtime_filter_mode=off -----
============================================
q1 4649 4563 4530 4530
q2 q3 1804 2250 1763 1763
q4 897 1188 782 782
q5 4070 4358 4367 4358
q6 182 186 146 146
q7 1811 1651 1518 1518
q8 2602 2754 2569 2569
q9 7511 7346 7293 7293
q10 2655 2903 2396 2396
q11 505 430 406 406
q12 506 597 450 450
q13 4009 4477 3609 3609
q14 276 289 276 276
q15 861 793 802 793
q16 720 777 723 723
q17 1190 1520 1342 1342
q18 7142 6889 6592 6592
q19 889 889 912 889
q20 2092 2278 1969 1969
q21 3964 3600 3432 3432
q22 481 425 393 393
Total cold run time: 48816 ms
Total hot run time: 46229 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](cloud) checkpoint save cloud tablet stats to image [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #60705:
URL: https://github.com/apache/doris/pull/60705#issuecomment-4045685443
<details>
<summary>TPC-H: <b>Total hot run time: 27945 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit d3a9bc8692f529d25e85cc464de3f8d230680c39, data reload: false
------ Round 1 ----------------------------------
============================================
q1 17614 4550 4285 4285
q2 q3 10642 795 528 528
q4 4677 371 256 256
q5 7544 1193 1034 1034
q6 176 173 147 147
q7 778 854 656 656
q8 9297 1515 1343 1343
q9 4995 4817 4806 4806
q10 6313 1955 1670 1670
q11 488 280 253 253
q12 735 569 481 481
q13 18056 2975 2183 2183
q14 234 239 233 233
q15 946 805 812 805
q16 770 727 699 699
q17 723 874 408 408
q18 6040 5381 5315 5315
q19 1123 996 618 618
q20 512 502 409 409
q21 4454 2231 1528 1528
q22 428 354 288 288
Total cold run time: 96545 ms
Total hot run time: 27945 ms
----- Round 2, with runtime_filter_mode=off -----
============================================
q1 4734 4558 4518 4518
q2 q3 3869 4335 3857 3857
q4 891 1240 797 797
q5 4117 4425 4367 4367
q6 197 185 147 147
q7 1794 1621 1546 1546
q8 2493 2836 2524 2524
q9 7728 7445 7523 7445
q10 3809 3981 3595 3595
q11 513 454 434 434
q12 518 579 458 458
q13 2766 3185 2264 2264
q14 273 321 272 272
q15 838 792 788 788
q16 910 760 713 713
q17 1433 1361 1406 1361
q18 7298 6877 6628 6628
q19 951 949 925 925
q20 2121 2169 2023 2023
q21 3996 3459 3330 3330
q22 482 423 396 396
Total cold run time: 51731 ms
Total hot run time: 48388 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](cloud) checkpoint save cloud version and tablet stats to image [doris]
Posted by "mymeiyi (via GitHub)" <gi...@apache.org>.
mymeiyi commented on PR #60705:
URL: https://github.com/apache/doris/pull/60705#issuecomment-3996502632
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](cloud) checkpoint save cloud tablet stats to image [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #60705:
URL: https://github.com/apache/doris/pull/60705#issuecomment-4087788467
<details>
<summary>TPC-DS: <b>Total hot run time: 169627 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 007ddd6647290f37a1362f1c5c2432a3b283a45f, data reload: false
query5 4337 640 532 532
query6 357 239 216 216
query7 4217 485 283 283
query8 345 262 235 235
query9 8718 2727 2732 2727
query10 484 397 352 352
query11 6980 5087 4913 4913
query12 190 131 125 125
query13 1284 468 354 354
query14 5794 3796 3506 3506
query14_1 2885 2835 2829 2829
query15 213 197 177 177
query16 985 490 447 447
query17 894 747 643 643
query18 2457 460 359 359
query19 221 214 194 194
query20 142 135 130 130
query21 218 137 113 113
query22 13311 14537 14627 14537
query23 16391 15776 15598 15598
query23_1 15829 15756 16171 15756
query24 7249 1630 1236 1236
query24_1 1263 1238 1270 1238
query25 603 465 424 424
query26 1238 260 148 148
query27 2797 481 298 298
query28 4468 1845 1841 1841
query29 872 562 474 474
query30 299 236 190 190
query31 1002 947 870 870
query32 80 68 73 68
query33 517 341 283 283
query34 890 863 535 535
query35 654 688 614 614
query36 1078 1077 946 946
query37 138 99 81 81
query38 2928 2941 2893 2893
query39 867 853 840 840
query39_1 785 807 806 806
query40 235 156 138 138
query41 63 60 59 59
query42 268 257 260 257
query43 237 253 221 221
query44
query45 195 189 201 189
query46 870 996 606 606
query47 2139 2149 2065 2065
query48 314 329 228 228
query49 630 470 386 386
query50 678 276 210 210
query51 4028 4072 3988 3988
query52 269 267 260 260
query53 298 332 284 284
query54 310 272 269 269
query55 101 85 89 85
query56 313 334 321 321
query57 1928 1595 1730 1595
query58 289 274 266 266
query59 2815 2968 2779 2779
query60 339 350 330 330
query61 161 165 175 165
query62 633 596 537 537
query63 320 295 277 277
query64 5062 1283 1035 1035
query65
query66 1462 479 371 371
query67 24192 24327 24224 24224
query68
query69 416 321 296 296
query70 984 998 982 982
query71 337 310 297 297
query72 2957 2683 2510 2510
query73 538 560 333 333
query74 9607 9604 9475 9475
query75 2877 2752 2462 2462
query76 2293 1037 692 692
query77 369 375 314 314
query78 10967 11125 10515 10515
query79 2394 773 569 569
query80 1769 626 564 564
query81 537 263 222 222
query82 938 149 117 117
query83 340 270 251 251
query84 302 121 99 99
query85 923 511 462 462
query86 405 335 303 303
query87 3180 3175 3027 3027
query88 3554 2652 2666 2652
query89 433 373 357 357
query90 2025 183 174 174
query91 175 161 144 144
query92 81 75 74 74
query93 1045 848 508 508
query94 648 327 287 287
query95 597 396 324 324
query96 666 516 228 228
query97 2457 2494 2431 2431
query98 251 225 225 225
query99 1013 1020 928 928
Total cold run time: 251956 ms
Total hot run time: 169627 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](cloud) checkpoint save cloud tablet stats to image [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #60705:
URL: https://github.com/apache/doris/pull/60705#issuecomment-4087760866
<details>
<summary>TPC-H: <b>Total hot run time: 27044 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 007ddd6647290f37a1362f1c5c2432a3b283a45f, data reload: false
------ Round 1 ----------------------------------
orders Doris NULL NULL 0 0 0 NULL 0 NULL NULL 2023-12-26 18:27:23 2023-12-26 18:42:55 NULL utf-8 NULL NULL
============================================
q1 17607 4521 4317 4317
q2 q3 10647 773 515 515
q4 4672 357 255 255
q5 7554 1216 1020 1020
q6 175 176 145 145
q7 767 836 674 674
q8 9341 1469 1342 1342
q9 4827 4747 4757 4747
q10 6336 1905 1704 1704
q11 450 260 246 246
q12 751 568 470 470
q13 18077 2974 2173 2173
q14 230 233 211 211
q15 q16 756 734 681 681
q17 746 837 441 441
q18 5864 5397 5357 5357
q19 1132 981 610 610
q20 544 502 381 381
q21 4559 1846 1456 1456
q22 517 377 299 299
Total cold run time: 95552 ms
Total hot run time: 27044 ms
----- Round 2, with runtime_filter_mode=off -----
orders Doris NULL NULL 150000000 42 6422171781 NULL 22778155 NULL NULL 2023-12-26 18:27:23 2023-12-26 18:42:55 NULL utf-8 NULL NULL
============================================
q1 4813 4552 4612 4552
q2 q3 3934 4382 3859 3859
q4 890 1237 796 796
q5 4042 4353 4352 4352
q6 185 173 139 139
q7 1792 1646 1527 1527
q8 2511 2767 2603 2603
q9 7515 7324 7322 7322
q10 3818 4003 3594 3594
q11 502 432 420 420
q12 513 613 485 485
q13 2788 3408 2338 2338
q14 302 334 303 303
q15 q16 724 750 782 750
q17 1229 1370 1343 1343
q18 7310 6797 6595 6595
q19 933 937 922 922
q20 2037 2213 2024 2024
q21 3989 3540 3276 3276
q22 477 422 385 385
Total cold run time: 50304 ms
Total hot run time: 47585 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](cloud) checkpoint save cloud tablet stats to image [doris]
Posted by "gavinchou (via GitHub)" <gi...@apache.org>.
gavinchou merged PR #60705:
URL: https://github.com/apache/doris/pull/60705
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](cloud) checkpoint save cloud tablet stats to image [doris]
Posted by "mymeiyi (via GitHub)" <gi...@apache.org>.
mymeiyi commented on PR #60705:
URL: https://github.com/apache/doris/pull/60705#issuecomment-4046727647
run external
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](cloud) checkpoint save cloud tablet stats to image [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #60705:
URL: https://github.com/apache/doris/pull/60705#issuecomment-4091722825
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](cloud) checkpoint save cloud version and tablet stats to image [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #60705:
URL: https://github.com/apache/doris/pull/60705#issuecomment-3897897735
# FE Regression Coverage Report
Increment line coverage ` 61.62% (61/99)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/60705_f7bf8949a054496cb7fe79b96f06f53c61a4c1d3_merge_fe/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/60705_f7bf8949a054496cb7fe79b96f06f53c61a4c1d3_merge_fe/report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](cloud) checkpoint save cloud version and tablet stats to image [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #60705:
URL: https://github.com/apache/doris/pull/60705#issuecomment-3996889299
# FE UT Coverage Report
Increment line coverage ` 7.07% (7/99)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/60705_4a34f9b1925725b582e045bd9e544d8b091cb53e/fe_increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/60705_4a34f9b1925725b582e045bd9e544d8b091cb53e/fe_report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](cloud) checkpoint save cloud version and tablet stats to image [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #60705:
URL: https://github.com/apache/doris/pull/60705#issuecomment-3996685415
<details>
<summary>TPC-DS: <b>Total hot run time: 184247 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 4a34f9b1925725b582e045bd9e544d8b091cb53e, data reload: false
query5 5008 634 530 530
query6 334 221 202 202
query7 4214 474 283 283
query8 330 251 241 241
query9 8745 2785 2802 2785
query10 544 368 337 337
query11 17008 17539 17077 17077
query12 219 137 129 129
query13 1302 499 379 379
query14 6944 3342 3288 3288
query14_1 2985 2955 2968 2955
query15 212 199 189 189
query16 1001 468 465 465
query17 1135 768 729 729
query18 2774 451 361 361
query19 222 213 187 187
query20 140 130 126 126
query21 225 138 117 117
query22 4812 4878 4677 4677
query23 17227 16753 16556 16556
query23_1 16594 16648 16735 16648
query24 7072 1607 1282 1282
query24_1 1288 1267 1274 1267
query25 566 487 480 480
query26 1223 261 153 153
query27 2763 466 287 287
query28 4481 1879 1897 1879
query29 791 572 480 480
query30 311 250 210 210
query31 871 726 664 664
query32 78 73 70 70
query33 515 333 289 289
query34 920 915 570 570
query35 643 663 599 599
query36 1025 1151 968 968
query37 134 94 88 88
query38 3028 2942 2912 2912
query39 893 870 866 866
query39_1 828 838 836 836
query40 231 153 139 139
query41 62 63 58 58
query42 107 106 110 106
query43 382 412 361 361
query44
query45 202 191 186 186
query46 915 1002 626 626
query47 2058 2148 2004 2004
query48 308 323 234 234
query49 638 466 384 384
query50 691 289 217 217
query51 4091 4118 4014 4014
query52 111 108 100 100
query53 300 349 290 290
query54 297 271 269 269
query55 92 90 86 86
query56 315 310 315 310
query57 1332 1356 1251 1251
query58 292 281 272 272
query59 2667 2679 2580 2580
query60 332 343 336 336
query61 148 150 145 145
query62 622 600 540 540
query63 310 277 287 277
query64 4818 1282 1012 1012
query65
query66 1384 445 368 368
query67 16413 16489 16289 16289
query68
query69 406 311 286 286
query70 939 967 870 870
query71 347 313 298 298
query72 2803 2664 2415 2415
query73 550 556 331 331
query74 10035 9916 9728 9728
query75 2831 2778 2477 2477
query76 2303 1038 718 718
query77 370 425 313 313
query78 11168 11346 10636 10636
query79 1503 809 627 627
query80 1388 627 570 570
query81 565 295 254 254
query82 1002 155 117 117
query83 380 274 247 247
query84 252 135 97 97
query85 938 524 442 442
query86 420 300 302 300
query87 3121 3108 3055 3055
query88 3608 2692 2690 2690
query89 446 400 352 352
query90 1990 180 173 173
query91 168 156 137 137
query92 82 76 69 69
query93 1017 859 513 513
query94 661 326 297 297
query95 598 339 306 306
query96 642 517 232 232
query97 2496 2476 2369 2369
query98 227 225 223 223
query99 1002 1009 895 895
Total cold run time: 254140 ms
Total hot run time: 184247 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](cloud) checkpoint save cloud tablet stats to image [doris]
Posted by "mymeiyi (via GitHub)" <gi...@apache.org>.
mymeiyi commented on code in PR #60705:
URL: https://github.com/apache/doris/pull/60705#discussion_r2963603395
##########
fe/fe-common/src/main/java/org/apache/doris/common/Config.java:
##########
@@ -653,6 +653,17 @@ public class Config extends ConfigBase {
"Randomly use V3 storage_format (ext_meta) for some tables in fuzzy tests to increase coverage"})
public static boolean random_use_v3_storage_format = true;
+ @ConfField(mutable = true, masterOnly = true, description = {
Review Comment:
非cloud目前不用，cloud是存储了tablet stats信息
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](cloud) checkpoint save cloud version and tablet stats to image [doris]
Posted by "deardeng (via GitHub)" <gi...@apache.org>.
deardeng commented on code in PR #60705:
URL: https://github.com/apache/doris/pull/60705#discussion_r2867039581
##########
fe/fe-core/src/main/java/org/apache/doris/master/Checkpoint.java:
##########
@@ -146,6 +166,7 @@ public synchronized void doCheckpoint() throws CheckpointException {
checkPointVersion, env.getReplayedJournalId()));
}
env.postProcessAfterMetadataReplayed(false);
+ postProcessCloudMetadata();
Review Comment:
@w41ter 这里是说，生成image后到post之间，table、partition version 被改了么？
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](cloud) checkpoint save cloud version and tablet stats to image [doris]
Posted by "mymeiyi (via GitHub)" <gi...@apache.org>.
mymeiyi commented on PR #60705:
URL: https://github.com/apache/doris/pull/60705#issuecomment-3896450085
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](cloud) checkpoint save cloud version and tablet stats to image [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #60705:
URL: https://github.com/apache/doris/pull/60705#issuecomment-3949797265
# FE UT Coverage Report
Increment line coverage ` 7.07% (7/99)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/60705_66cffc1356fb02c956d56aa828528ee6eed4f439/fe_increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/60705_66cffc1356fb02c956d56aa828528ee6eed4f439/fe_report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](cloud) checkpoint save cloud tablet stats to image [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #60705:
URL: https://github.com/apache/doris/pull/60705#issuecomment-4045746101
<details>
<summary>TPC-DS: <b>Total hot run time: 153290 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit d3a9bc8692f529d25e85cc464de3f8d230680c39, data reload: false
query5 4345 637 501 501
query6 330 231 233 231
query7 4213 468 267 267
query8 364 264 239 239
query9 8709 2688 2712 2688
query10 522 431 339 339
query11 7276 5890 5595 5595
query12 179 131 127 127
query13 1263 464 359 359
query14 5788 3828 3575 3575
query14_1 2806 2838 2839 2838
query15 204 193 176 176
query16 989 503 481 481
query17 1001 725 629 629
query18 2440 456 360 360
query19 217 227 185 185
query20 141 134 132 132
query21 226 151 129 129
query22 4813 4951 4925 4925
query23 16692 16006 15642 15642
query23_1 15862 15846 15750 15750
query24 7972 1686 1286 1286
query24_1 1316 1322 1336 1322
query25 599 511 483 483
query26 1228 280 151 151
query27 2777 475 280 280
query28 4517 1853 1839 1839
query29 837 559 478 478
query30 312 249 216 216
query31 1327 1303 1237 1237
query32 78 71 77 71
query33 505 327 279 279
query34 941 933 572 572
query35 640 677 592 592
query36 1081 1174 1000 1000
query37 132 90 82 82
query38 2966 2926 2860 2860
query39 880 893 838 838
query39_1 828 820 822 820
query40 230 152 135 135
query41 64 62 56 56
query42 307 301 308 301
query43 238 255 218 218
query44
query45 200 235 183 183
query46 899 986 603 603
query47 2124 2188 2062 2062
query48 309 321 235 235
query49 636 454 382 382
query50 694 272 207 207
query51 4122 4083 4046 4046
query52 281 293 285 285
query53 291 334 279 279
query54 301 270 288 270
query55 94 85 79 79
query56 321 333 317 317
query57 1348 1335 1289 1289
query58 291 282 288 282
query59 1313 1439 1255 1255
query60 333 330 322 322
query61 149 142 146 142
query62 631 590 537 537
query63 308 278 278 278
query64 5110 1251 986 986
query65
query66 1464 446 354 354
query67 16524 16514 16416 16416
query68
query69 390 316 285 285
query70 1005 961 989 961
query71 339 305 306 305
query72 2801 2685 2439 2439
query73 539 541 314 314
query74 9937 9864 9872 9864
query75 2881 2738 2491 2491
query76 2269 1027 666 666
query77 360 401 310 310
query78 11295 11446 10668 10668
query79 2338 804 621 621
query80 1614 635 541 541
query81 558 272 248 248
query82 996 149 117 117
query83 345 274 244 244
query84 284 119 96 96
query85 893 482 439 439
query86 424 313 292 292
query87 3225 3127 3023 3023
query88 3503 2632 2641 2632
query89 433 362 348 348
query90 2050 174 192 174
query91 163 161 137 137
query92 83 78 69 69
query93 999 829 499 499
query94 634 300 284 284
query95 598 337 391 337
query96 629 518 226 226
query97 2480 2538 2411 2411
query98 236 220 218 218
query99 1001 1021 892 892
Total cold run time: 235614 ms
Total hot run time: 153290 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](cloud) checkpoint save cloud tablet stats to image [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #60705:
URL: https://github.com/apache/doris/pull/60705#issuecomment-4087881758
# FE UT Coverage Report
Increment line coverage ` 6.59% (6/91)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/60705_007ddd6647290f37a1362f1c5c2432a3b283a45f/fe_increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/60705_007ddd6647290f37a1362f1c5c2432a3b283a45f/fe_report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](cloud) checkpoint save cloud version and tablet stats to image [doris]
Posted by "deardeng (via GitHub)" <gi...@apache.org>.
deardeng commented on code in PR #60705:
URL: https://github.com/apache/doris/pull/60705#discussion_r2867036193
##########
fe/fe-common/src/main/java/org/apache/doris/common/Config.java:
##########
@@ -653,6 +653,17 @@ public class Config extends ConfigBase {
"Randomly use V3 storage_format (ext_meta) for some tables in fuzzy tests to increase coverage"})
public static boolean random_use_v3_storage_format = true;
+ @ConfField(mutable = true, masterOnly = true, description = {
Review Comment:
非cloud是不是，也能这样做？
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](cloud) checkpoint save cloud version and tablet stats to image [doris]
Posted by "mymeiyi (via GitHub)" <gi...@apache.org>.
mymeiyi commented on code in PR #60705:
URL: https://github.com/apache/doris/pull/60705#discussion_r2882340730
##########
fe/fe-core/src/main/java/org/apache/doris/master/Checkpoint.java:
##########
@@ -146,6 +166,7 @@ public synchronized void doCheckpoint() throws CheckpointException {
checkPointVersion, env.getReplayedJournalId()));
}
env.postProcessAfterMetadataReplayed(false);
+ postProcessCloudMetadata();
Review Comment:
有这种可能，除了影响快照(快照这里可以在clone后save前改下)，还有什么其它的影响吗？
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](cloud) checkpoint save cloud version and tablet stats to image [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #60705:
URL: https://github.com/apache/doris/pull/60705#issuecomment-3949690133
<details>
<summary>TPC-DS: <b>Total hot run time: 183259 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 66cffc1356fb02c956d56aa828528ee6eed4f439, data reload: false
query5 5136 632 509 509
query6 326 222 203 203
query7 4218 462 268 268
query8 329 248 254 248
query9 8745 2723 2721 2721
query10 529 389 320 320
query11 16941 17461 17023 17023
query12 207 132 158 132
query13 2233 471 352 352
query14 6916 3507 3121 3121
query14_1 2978 2914 2871 2871
query15 208 199 186 186
query16 1023 481 496 481
query17 1136 763 633 633
query18 2738 451 361 361
query19 201 213 193 193
query20 136 134 138 134
query21 219 146 118 118
query22 5770 4952 4725 4725
query23 17489 16908 16585 16585
query23_1 16673 16654 16647 16647
query24 7075 1592 1204 1204
query24_1 1214 1225 1212 1212
query25 543 472 426 426
query26 1004 265 159 159
query27 2739 463 284 284
query28 4501 1860 1858 1858
query29 833 589 494 494
query30 308 253 213 213
query31 884 748 645 645
query32 83 71 73 71
query33 533 346 297 297
query34 916 910 561 561
query35 635 670 594 594
query36 1107 1126 970 970
query37 141 92 88 88
query38 2969 2907 2901 2901
query39 903 878 854 854
query39_1 825 834 819 819
query40 229 154 140 140
query41 68 65 64 64
query42 108 105 104 104
query43 374 374 352 352
query44
query45 202 193 188 188
query46 880 992 609 609
query47 2126 2152 2046 2046
query48 329 323 247 247
query49 658 482 391 391
query50 679 280 222 222
query51 4105 4067 4036 4036
query52 105 109 96 96
query53 296 336 287 287
query54 305 290 279 279
query55 83 88 90 88
query56 342 312 315 312
query57 1368 1319 1303 1303
query58 292 287 276 276
query59 2524 2664 2521 2521
query60 347 353 368 353
query61 137 141 143 141
query62 617 584 552 552
query63 312 279 276 276
query64 4570 1236 970 970
query65
query66 1345 445 353 353
query67 16293 16511 16304 16304
query68
query69 391 304 273 273
query70 986 978 913 913
query71 326 296 299 296
query72 2740 2673 2354 2354
query73 537 548 316 316
query74 10031 9969 9730 9730
query75 2825 2746 2470 2470
query76 2327 1022 676 676
query77 365 363 303 303
query78 11220 11376 10734 10734
query79 2583 791 608 608
query80 1821 606 543 543
query81 557 279 253 253
query82 973 148 114 114
query83 340 263 239 239
query84 257 117 100 100
query85 874 479 450 450
query86 409 339 304 304
query87 3090 3113 2963 2963
query88 3514 2667 2636 2636
query89 417 367 338 338
query90 1942 162 167 162
query91 164 155 129 129
query92 75 79 67 67
query93 1160 853 492 492
query94 645 327 289 289
query95 573 341 372 341
query96 632 516 227 227
query97 2449 2500 2415 2415
query98 225 211 213 211
query99 1018 975 912 912
Total cold run time: 256989 ms
Total hot run time: 183259 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](cloud) checkpoint save cloud version and tablet stats to image [doris]
Posted by "Thearas (via GitHub)" <gi...@apache.org>.
Thearas commented on PR #60705:
URL: https://github.com/apache/doris/pull/60705#issuecomment-3888912096
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR).
Please clearly describe your PR:
1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
3. What features were added. Why was this function added?
4. Which code was refactored and why was this part of the code refactored?
5. Which functions were optimized and what is the difference before and after the optimization?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](cloud) checkpoint save cloud version and tablet stats to image [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #60705:
URL: https://github.com/apache/doris/pull/60705#issuecomment-3889287617
<details>
<summary>TPC-H: <b>Total hot run time: 30543 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit f32e8e3b943cde1ee8c2d00157c6b2b77c387ac0, data reload: false
------ Round 1 ----------------------------------
q1 17604 4403 4301 4301
q2 2011 338 234 234
q3 10184 1324 760 760
q4 10192 790 309 309
q5 7503 2226 1975 1975
q6 195 178 144 144
q7 941 724 617 617
q8 9268 1426 1146 1146
q9 4852 4704 4649 4649
q10 6858 1955 1552 1552
q11 465 260 247 247
q12 402 377 225 225
q13 17778 4050 3249 3249
q14 231 241 215 215
q15 889 808 805 805
q16 698 668 645 645
q17 706 863 498 498
q18 6688 5712 6195 5712
q19 1148 1044 677 677
q20 560 523 411 411
q21 2811 1914 1936 1914
q22 342 287 258 258
Total cold run time: 102326 ms
Total hot run time: 30543 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4586 4567 4666 4567
q2 268 358 273 273
q3 2302 2859 2468 2468
q4 1526 1874 1380 1380
q5 4956 4721 4524 4524
q6 223 184 137 137
q7 2023 1907 1875 1875
q8 2570 2398 2457 2398
q9 7612 7501 7656 7501
q10 2847 3068 2708 2708
q11 540 488 423 423
q12 725 764 589 589
q13 4024 4258 3480 3480
q14 265 280 273 273
q15 820 795 770 770
q16 644 673 637 637
q17 1077 1285 1329 1285
q18 7576 7355 7382 7355
q19 837 770 806 770
q20 1966 2043 1874 1874
q21 4636 4307 4177 4177
q22 510 454 423 423
Total cold run time: 52533 ms
Total hot run time: 49887 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](cloud) checkpoint save cloud version and tablet stats to image [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #60705:
URL: https://github.com/apache/doris/pull/60705#issuecomment-3890336049
# FE Regression Coverage Report
Increment line coverage ` 62.82% (49/78)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/60705_f32e8e3b943cde1ee8c2d00157c6b2b77c387ac0_merge_fe/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/60705_f32e8e3b943cde1ee8c2d00157c6b2b77c387ac0_merge_fe/report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](cloud) checkpoint save cloud tablet stats to image [doris]
Posted by "mymeiyi (via GitHub)" <gi...@apache.org>.
mymeiyi commented on PR #60705:
URL: https://github.com/apache/doris/pull/60705#issuecomment-4095001085
run p0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](cloud) checkpoint save cloud tablet stats to image [doris]
Posted by "mymeiyi (via GitHub)" <gi...@apache.org>.
mymeiyi commented on PR #60705:
URL: https://github.com/apache/doris/pull/60705#issuecomment-4045378191
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](cloud) checkpoint save cloud version and tablet stats to image [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #60705:
URL: https://github.com/apache/doris/pull/60705#issuecomment-3950334145
# FE Regression Coverage Report
Increment line coverage ` 64.65% (64/99)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/60705_66cffc1356fb02c956d56aa828528ee6eed4f439_merge_fe/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/60705_66cffc1356fb02c956d56aa828528ee6eed4f439_merge_fe/report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](cloud) checkpoint save cloud version and tablet stats to image [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #60705:
URL: https://github.com/apache/doris/pull/60705#issuecomment-3976234412
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](cloud) checkpoint save cloud version and tablet stats to image [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #60705:
URL: https://github.com/apache/doris/pull/60705#issuecomment-3997566511
# FE Regression Coverage Report
Increment line coverage ` 55.56% (55/99)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/60705_4a34f9b1925725b582e045bd9e544d8b091cb53e_merge_fe/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/60705_4a34f9b1925725b582e045bd9e544d8b091cb53e_merge_fe/report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org