You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "suxiaogang223 (via GitHub)" <gi...@apache.org> on 2026/01/09 08:14:45 UTC
[PR] Refact meta cache [doris]
suxiaogang223 opened a new pull request, #59716:
URL: https://github.com/apache/doris/pull/59716
### What problem does this PR solve?
Issue Number: close #xxx
Related PR: #xxx
Problem Summary:
### Release note
None
### Check List (For Author)
- Test <!-- At least one of them must be included. -->
- [ ] Regression test
- [ ] Unit Test
- [ ] Manual test (add detailed scripts or steps below)
- [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
- [ ] Previous test can cover this change.
- [ ] No code files have been changed.
- [ ] Other reason <!-- Add your reason? -->
- Behavior changed:
- [ ] No.
- [ ] Yes. <!-- Explain the behavior change -->
- Does this need documentation?
- [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg: https://github.com/apache/doris-website/pull/1214 -->
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR should merge into -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](iceberg) Refactor Iceberg metadata cache structure and add table cache test cases [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59716:
URL: https://github.com/apache/doris/pull/59716#issuecomment-3737798164
<details>
<summary>TPC-H: <b>Total hot run time: 32189 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 1574f3a0840831ea973580b5275b27182d07b2bf, data reload: false
------ Round 1 ----------------------------------
q1 17636 4183 4086 4086
q2 2077 377 245 245
q3 10116 1275 724 724
q4 10208 855 309 309
q5 7538 2107 1841 1841
q6 187 168 135 135
q7 911 811 671 671
q8 9273 1414 1159 1159
q9 5100 4604 4559 4559
q10 6839 1852 1436 1436
q11 537 277 275 275
q12 735 756 590 590
q13 17802 3842 3078 3078
q14 297 298 274 274
q15 609 517 510 510
q16 667 684 647 647
q17 676 764 578 578
q18 6747 6583 6973 6583
q19 1355 996 669 669
q20 451 404 269 269
q21 3285 2654 2506 2506
q22 1170 1126 1045 1045
Total cold run time: 104216 ms
Total hot run time: 32189 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4395 4248 4257 4248
q2 324 411 333 333
q3 2299 2747 2449 2449
q4 1516 1815 1484 1484
q5 4601 4340 4422 4340
q6 224 177 129 129
q7 1958 1941 1816 1816
q8 2570 2413 2394 2394
q9 7135 7364 7264 7264
q10 2508 2691 2307 2307
q11 579 495 464 464
q12 701 723 561 561
q13 3349 3817 3040 3040
q14 268 289 266 266
q15 525 490 482 482
q16 616 674 616 616
q17 1099 1321 1362 1321
q18 7417 7489 7337 7337
q19 808 774 777 774
q20 1896 1974 1872 1872
q21 4485 4339 4196 4196
q22 1096 1020 995 995
Total cold run time: 50369 ms
Total hot run time: 48688 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](iceberg) Refactor Iceberg metadata cache structure and add table cache test cases [doris]
Posted by "suxiaogang223 (via GitHub)" <gi...@apache.org>.
suxiaogang223 commented on PR #59716:
URL: https://github.com/apache/doris/pull/59716#issuecomment-3809464214
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](iceberg) Refactor Iceberg metadata cache structure and add table cache test cases [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59716:
URL: https://github.com/apache/doris/pull/59716#issuecomment-3809692908
<details>
<summary>ClickBench: <b>Total hot run time: 28.46 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit b94d3ff7cf85ab07e06a96e72967af235065ef7f, data reload: false
query1 0.06 0.05 0.04
query2 0.09 0.04 0.05
query3 0.25 0.08 0.08
query4 1.61 0.11 0.11
query5 0.26 0.24 0.24
query6 1.16 0.68 0.69
query7 0.03 0.02 0.03
query8 0.06 0.04 0.05
query9 0.56 0.49 0.50
query10 0.55 0.54 0.54
query11 0.14 0.09 0.10
query12 0.15 0.11 0.11
query13 0.64 0.62 0.63
query14 1.07 1.08 1.07
query15 0.87 0.87 0.88
query16 0.40 0.40 0.38
query17 1.13 1.12 1.15
query18 0.24 0.22 0.22
query19 2.07 2.03 2.04
query20 0.02 0.02 0.01
query21 15.44 0.26 0.14
query22 5.25 0.06 0.05
query23 15.97 0.28 0.10
query24 2.51 0.34 0.58
query25 0.10 0.06 0.09
query26 0.13 0.13 0.13
query27 0.08 0.07 0.06
query28 4.30 1.17 0.97
query29 12.60 3.95 3.16
query30 0.27 0.15 0.11
query31 2.82 0.64 0.41
query32 3.24 0.60 0.50
query33 3.28 3.21 3.26
query34 16.43 5.37 4.77
query35 4.86 4.81 4.78
query36 0.65 0.50 0.49
query37 0.12 0.07 0.07
query38 0.07 0.04 0.03
query39 0.05 0.03 0.03
query40 0.19 0.17 0.15
query41 0.09 0.04 0.03
query42 0.05 0.03 0.03
query43 0.05 0.04 0.04
Total cold run time: 99.91 s
Total hot run time: 28.46 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](iceberg) Refactor Iceberg metadata cache structure and add table cache test cases [doris]
Posted by "suxiaogang223 (via GitHub)" <gi...@apache.org>.
suxiaogang223 commented on PR #59716:
URL: https://github.com/apache/doris/pull/59716#issuecomment-3803511874
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](iceberg) Refactor Iceberg metadata cache structure and add table cache test cases [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #59716:
URL: https://github.com/apache/doris/pull/59716#issuecomment-3804750318
# FE UT Coverage Report
Increment line coverage ` 0.00% (0/92)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/59716_991bc75b6530dc1f39b1b3d9dec7880b810253b7/fe_increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/59716_991bc75b6530dc1f39b1b3d9dec7880b810253b7/fe_report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](iceberg) Refactor Iceberg metadata cache structure and add table cache test cases [doris]
Posted by "suxiaogang223 (via GitHub)" <gi...@apache.org>.
suxiaogang223 commented on code in PR #59716:
URL: https://github.com/apache/doris/pull/59716#discussion_r2727344425
##########
fe/fe-core/src/main/java/org/apache/doris/datasource/iceberg/IcebergMetadataCache.java:
##########
@@ -55,9 +54,7 @@ public class IcebergMetadataCache {
private static final Logger LOG = LogManager.getLogger(IcebergMetadataCache.class);
private final ExecutorService executor;
private final IcebergExternalCatalog catalog;
- private LoadingCache<IcebergMetadataCacheKey, List<Snapshot>> snapshotListCache;
Review Comment:
This is not used by anything
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](iceberg) Refactor Iceberg metadata cache structure and add table cache test cases [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59716:
URL: https://github.com/apache/doris/pull/59716#issuecomment-3803620520
<details>
<summary>TPC-H: <b>Total hot run time: 32768 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 991bc75b6530dc1f39b1b3d9dec7880b810253b7, data reload: false
------ Round 1 ----------------------------------
q1 17622 5238 5057 5057
q2 2076 303 197 197
q3 10209 1359 799 799
q4 10252 808 330 330
q5 8239 2209 1958 1958
q6 230 186 154 154
q7 905 765 613 613
q8 9272 1543 1137 1137
q9 5400 4839 4828 4828
q10 6915 1974 1559 1559
q11 545 305 278 278
q12 410 378 234 234
q13 17774 4415 3263 3263
q14 250 237 221 221
q15 912 804 838 804
q16 705 678 627 627
q17 643 719 559 559
q18 6913 6469 6490 6469
q19 1658 982 651 651
q20 406 345 228 228
q21 2819 2121 1855 1855
q22 1040 1006 947 947
Total cold run time: 105195 ms
Total hot run time: 32768 ms
----- Round 2, with runtime_filter_mode=off -----
q1 5456 5288 5331 5288
q2 258 348 266 266
q3 2243 2732 2258 2258
q4 1403 1791 1335 1335
q5 4223 4199 5026 4199
q6 261 207 156 156
q7 2166 2039 1803 1803
q8 2560 2426 2455 2426
q9 7640 7742 7715 7715
q10 2866 3044 2569 2569
q11 557 480 449 449
q12 696 798 668 668
q13 3829 4501 3668 3668
q14 331 457 348 348
q15 862 851 830 830
q16 666 729 679 679
q17 1175 1354 1350 1350
q18 8076 7933 7786 7786
q19 983 951 914 914
q20 2137 2156 2046 2046
q21 5022 4443 4177 4177
q22 1107 1070 951 951
Total cold run time: 54517 ms
Total hot run time: 51881 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](iceberg) Refactor Iceberg metadata cache structure and add table cache test cases [doris]
Posted by "morningman (via GitHub)" <gi...@apache.org>.
morningman commented on code in PR #59716:
URL: https://github.com/apache/doris/pull/59716#discussion_r2697856228
##########
fe/fe-core/src/main/java/org/apache/doris/datasource/iceberg/IcebergMetadataCache.java:
##########
@@ -160,58 +134,51 @@ private Table loadTable(IcebergMetadataCacheKey key) {
if (LOG.isDebugEnabled()) {
LOG.debug("load iceberg table {}", nameMapping, new Exception());
}
- return ((ExternalCatalog) catalog).getExecutionAuthenticator().execute(()
- -> ops.loadTable(nameMapping.getRemoteDbName(), nameMapping.getRemoteTblName()));
+ Table table = ((ExternalCatalog) catalog).getExecutionAuthenticator()
+ .execute(()
+ -> ops.loadTable(nameMapping.getRemoteDbName(), nameMapping.getRemoteTblName()));
+ return new IcebergTableCacheValue(table);
} catch (Exception e) {
throw new RuntimeException(ExceptionUtils.getRootCauseMessage(e), e);
}
}
@NotNull
- private IcebergSnapshotCacheValue loadSnapshot(IcebergMetadataCacheKey key) throws AnalysisException {
- NameMapping nameMapping = key.nameMapping;
- TableIf dorisTable = Env.getCurrentEnv().getCatalogMgr().getCatalogOrAnalysisException(nameMapping.getCtlId())
- .getDbOrAnalysisException(nameMapping.getLocalDbName())
- .getTableOrAnalysisException(nameMapping.getLocalTblName());
-
+ private IcebergSnapshotCacheValue loadSnapshot(ExternalTable dorisTable) {
if (!(dorisTable instanceof MTMVRelatedTableIf)) {
- throw new AnalysisException(String.format("Table %s.%s is not a valid MTMV related table.",
- nameMapping.getLocalDbName(), nameMapping.getLocalTblName()));
+ throw new RuntimeException(String.format("Table %s.%s is not a valid MTMV related table.",
+ dorisTable.getDbName(), dorisTable.getName()));
}
- MTMVRelatedTableIf table = (MTMVRelatedTableIf) dorisTable;
- IcebergSnapshot lastedIcebergSnapshot = IcebergUtils.getLastedIcebergSnapshot((ExternalTable) table);
- IcebergPartitionInfo icebergPartitionInfo;
- if (!table.isValidRelatedTable()) {
- icebergPartitionInfo = IcebergPartitionInfo.empty();
- } else {
- icebergPartitionInfo = IcebergUtils.loadPartitionInfo((ExternalTable) table,
- lastedIcebergSnapshot.getSnapshotId());
+ try {
+ MTMVRelatedTableIf table = (MTMVRelatedTableIf) dorisTable;
+ IcebergSnapshot lastedIcebergSnapshot = IcebergUtils.getLastedIcebergSnapshot(dorisTable);
Review Comment:
The logic here is a bit convoluted. We reach this point via the `TableCache`, so we have already obtained an `IcebergTable` instance. However, `IcebergUtils.getLatestIcebergSnapshot` goes back to the `TableCache` again to fetch the `IcebergTable`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](iceberg) Refactor Iceberg metadata cache structure and add table cache test cases [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59716:
URL: https://github.com/apache/doris/pull/59716#issuecomment-3799501531
<details>
<summary>ClickBench: <b>Total hot run time: 28.32 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 35ab7872eaf6d9ed159f68593e694e5cd4ba731d, data reload: false
query1 0.06 0.05 0.05
query2 0.10 0.05 0.04
query3 0.25 0.08 0.08
query4 1.60 0.11 0.11
query5 0.28 0.26 0.25
query6 1.17 0.67 0.67
query7 0.03 0.03 0.03
query8 0.05 0.04 0.04
query9 0.56 0.51 0.50
query10 0.55 0.55 0.54
query11 0.15 0.10 0.10
query12 0.14 0.11 0.11
query13 0.63 0.62 0.61
query14 1.07 1.07 1.07
query15 0.86 0.87 0.88
query16 0.40 0.39 0.39
query17 1.16 1.16 1.14
query18 0.23 0.21 0.22
query19 2.09 2.04 1.96
query20 0.02 0.01 0.02
query21 15.40 0.27 0.14
query22 5.50 0.05 0.05
query23 15.82 0.28 0.10
query24 2.22 0.25 0.56
query25 0.09 0.07 0.11
query26 0.14 0.13 0.12
query27 0.09 0.08 0.07
query28 4.74 1.11 0.96
query29 12.57 3.97 3.19
query30 0.27 0.14 0.11
query31 2.82 0.63 0.39
query32 3.28 0.60 0.49
query33 3.27 3.22 3.20
query34 16.47 5.43 4.74
query35 4.77 4.84 4.80
query36 0.66 0.49 0.50
query37 0.11 0.08 0.07
query38 0.08 0.04 0.04
query39 0.04 0.02 0.02
query40 0.18 0.16 0.15
query41 0.09 0.04 0.03
query42 0.04 0.04 0.02
query43 0.05 0.04 0.04
Total cold run time: 100.1 s
Total hot run time: 28.32 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](iceberg) Refactor Iceberg metadata cache structure and add table cache test cases [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #59716:
URL: https://github.com/apache/doris/pull/59716#issuecomment-3815625649
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](iceberg) Refactor Iceberg metadata cache structure and add table cache test cases [doris]
Posted by "morningman (via GitHub)" <gi...@apache.org>.
morningman merged PR #59716:
URL: https://github.com/apache/doris/pull/59716
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](iceberg) Refactor Iceberg metadata cache structure and add table cache test cases [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #59716:
URL: https://github.com/apache/doris/pull/59716#issuecomment-3805601186
# FE Regression Coverage Report
Increment line coverage ` 46.74% (43/92)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/59716_991bc75b6530dc1f39b1b3d9dec7880b810253b7_merge_fe/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/59716_991bc75b6530dc1f39b1b3d9dec7880b810253b7_merge_fe/report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](iceberg) Refactor Iceberg metadata cache structure and add table cache test cases [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59716:
URL: https://github.com/apache/doris/pull/59716#issuecomment-3803681354
<details>
<summary>ClickBench: <b>Total hot run time: 28.25 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 991bc75b6530dc1f39b1b3d9dec7880b810253b7, data reload: false
query1 0.05 0.05 0.05
query2 0.10 0.04 0.04
query3 0.26 0.08 0.08
query4 1.64 0.11 0.11
query5 0.27 0.24 0.25
query6 1.17 0.68 0.66
query7 0.03 0.03 0.02
query8 0.06 0.04 0.04
query9 0.55 0.53 0.50
query10 0.55 0.54 0.56
query11 0.14 0.09 0.10
query12 0.14 0.10 0.10
query13 0.63 0.61 0.61
query14 1.07 1.06 1.05
query15 0.88 0.85 0.88
query16 0.40 0.40 0.39
query17 1.16 1.15 1.12
query18 0.22 0.21 0.21
query19 2.05 1.92 1.90
query20 0.02 0.02 0.01
query21 15.46 0.26 0.13
query22 4.97 0.05 0.05
query23 15.84 0.28 0.10
query24 1.95 0.88 0.25
query25 0.11 0.07 0.06
query26 0.15 0.13 0.14
query27 0.08 0.06 0.06
query28 4.50 1.16 0.97
query29 12.58 3.86 3.17
query30 0.28 0.13 0.12
query31 2.81 0.61 0.40
query32 3.24 0.59 0.50
query33 3.19 3.25 3.23
query34 16.03 5.36 4.80
query35 4.87 4.78 4.77
query36 0.64 0.50 0.50
query37 0.11 0.07 0.07
query38 0.08 0.04 0.04
query39 0.05 0.03 0.03
query40 0.18 0.16 0.16
query41 0.09 0.03 0.03
query42 0.05 0.03 0.03
query43 0.06 0.04 0.04
Total cold run time: 98.71 s
Total hot run time: 28.25 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](iceberg) Refactor Iceberg metadata cache structure and add table cache test cases [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #59716:
URL: https://github.com/apache/doris/pull/59716#issuecomment-3737786776
# FE UT Coverage Report
Increment line coverage ` 0.00% (0/36)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/59716_1574f3a0840831ea973580b5275b27182d07b2bf/fe_increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/59716_1574f3a0840831ea973580b5275b27182d07b2bf/fe_report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](iceberg) Refactor Iceberg metadata cache structure and add table cache test cases [doris]
Posted by "suxiaogang223 (via GitHub)" <gi...@apache.org>.
suxiaogang223 commented on PR #59716:
URL: https://github.com/apache/doris/pull/59716#issuecomment-3804422330
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](iceberg) Refactor Iceberg metadata cache structure and add table cache test cases [doris]
Posted by "morningman (via GitHub)" <gi...@apache.org>.
morningman commented on code in PR #59716:
URL: https://github.com/apache/doris/pull/59716#discussion_r2697875508
##########
fe/fe-core/src/main/java/org/apache/doris/datasource/iceberg/IcebergMetadataCache.java:
##########
@@ -55,9 +54,7 @@ public class IcebergMetadataCache {
private static final Logger LOG = LogManager.getLogger(IcebergMetadataCache.class);
private final ExecutorService executor;
private final IcebergExternalCatalog catalog;
- private LoadingCache<IcebergMetadataCacheKey, List<Snapshot>> snapshotListCache;
Review Comment:
This is not used before?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](iceberg) Refactor Iceberg metadata cache structure and add table cache test cases [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #59716:
URL: https://github.com/apache/doris/pull/59716#issuecomment-3810337891
# FE Regression Coverage Report
Increment line coverage ` 72.83% (67/92)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/59716_b94d3ff7cf85ab07e06a96e72967af235065ef7f_merge_fe/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/59716_b94d3ff7cf85ab07e06a96e72967af235065ef7f_merge_fe/report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](iceberg) Refactor Iceberg metadata cache structure and add table cache test cases [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59716:
URL: https://github.com/apache/doris/pull/59716#issuecomment-3804545937
<details>
<summary>TPC-H: <b>Total hot run time: 33103 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 991bc75b6530dc1f39b1b3d9dec7880b810253b7, data reload: false
------ Round 1 ----------------------------------
q1 17606 5320 5061 5061
q2 2074 344 200 200
q3 10138 1287 752 752
q4 10229 920 324 324
q5 7555 2187 1951 1951
q6 201 181 149 149
q7 883 725 605 605
q8 9276 1589 1141 1141
q9 5177 4759 4805 4759
q10 6873 1959 1534 1534
q11 547 295 275 275
q12 337 375 229 229
q13 17822 4088 3268 3268
q14 237 247 225 225
q15 881 829 809 809
q16 678 715 616 616
q17 654 786 478 478
q18 6827 6645 7692 6645
q19 1114 1042 667 667
q20 437 390 239 239
q21 2997 2359 2101 2101
q22 1144 1075 1185 1075
Total cold run time: 103687 ms
Total hot run time: 33103 ms
----- Round 2, with runtime_filter_mode=off -----
q1 5518 5526 5682 5526
q2 250 337 256 256
q3 2397 2860 2555 2555
q4 1527 1854 1394 1394
q5 4521 4515 4672 4515
q6 232 207 163 163
q7 2270 2023 1774 1774
q8 2565 2332 2335 2332
q9 7579 7465 7544 7465
q10 2923 3055 2695 2695
q11 554 464 462 462
q12 679 754 618 618
q13 3626 4047 3238 3238
q14 277 280 265 265
q15 842 794 782 782
q16 646 683 633 633
q17 1093 1238 1249 1238
q18 7604 7240 7255 7240
q19 845 790 811 790
q20 1977 2046 1891 1891
q21 4581 4259 4067 4067
q22 1104 1005 985 985
Total cold run time: 53610 ms
Total hot run time: 50884 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] Refact meta cache [doris]
Posted by "suxiaogang223 (via GitHub)" <gi...@apache.org>.
suxiaogang223 commented on PR #59716:
URL: https://github.com/apache/doris/pull/59716#issuecomment-3727796801
run external
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](iceberg) Refactor Iceberg metadata cache structure and add table cache test cases [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #59716:
URL: https://github.com/apache/doris/pull/59716#issuecomment-3800177695
# FE Regression Coverage Report
Increment line coverage ` 81.08% (30/37)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/59716_35ab7872eaf6d9ed159f68593e694e5cd4ba731d_merge_fe/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/59716_35ab7872eaf6d9ed159f68593e694e5cd4ba731d_merge_fe/report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](iceberg) Refactor Iceberg metadata cache structure and add table cache test cases [doris]
Posted by "suxiaogang223 (via GitHub)" <gi...@apache.org>.
suxiaogang223 commented on code in PR #59716:
URL: https://github.com/apache/doris/pull/59716#discussion_r2727373991
##########
fe/fe-core/src/main/java/org/apache/doris/datasource/iceberg/IcebergMetadataCache.java:
##########
@@ -160,58 +134,51 @@ private Table loadTable(IcebergMetadataCacheKey key) {
if (LOG.isDebugEnabled()) {
LOG.debug("load iceberg table {}", nameMapping, new Exception());
}
- return ((ExternalCatalog) catalog).getExecutionAuthenticator().execute(()
- -> ops.loadTable(nameMapping.getRemoteDbName(), nameMapping.getRemoteTblName()));
+ Table table = ((ExternalCatalog) catalog).getExecutionAuthenticator()
+ .execute(()
+ -> ops.loadTable(nameMapping.getRemoteDbName(), nameMapping.getRemoteTblName()));
+ return new IcebergTableCacheValue(table);
} catch (Exception e) {
throw new RuntimeException(ExceptionUtils.getRootCauseMessage(e), e);
}
}
@NotNull
- private IcebergSnapshotCacheValue loadSnapshot(IcebergMetadataCacheKey key) throws AnalysisException {
- NameMapping nameMapping = key.nameMapping;
- TableIf dorisTable = Env.getCurrentEnv().getCatalogMgr().getCatalogOrAnalysisException(nameMapping.getCtlId())
- .getDbOrAnalysisException(nameMapping.getLocalDbName())
- .getTableOrAnalysisException(nameMapping.getLocalTblName());
-
+ private IcebergSnapshotCacheValue loadSnapshot(ExternalTable dorisTable) {
if (!(dorisTable instanceof MTMVRelatedTableIf)) {
- throw new AnalysisException(String.format("Table %s.%s is not a valid MTMV related table.",
- nameMapping.getLocalDbName(), nameMapping.getLocalTblName()));
+ throw new RuntimeException(String.format("Table %s.%s is not a valid MTMV related table.",
+ dorisTable.getDbName(), dorisTable.getName()));
}
- MTMVRelatedTableIf table = (MTMVRelatedTableIf) dorisTable;
- IcebergSnapshot lastedIcebergSnapshot = IcebergUtils.getLastedIcebergSnapshot((ExternalTable) table);
- IcebergPartitionInfo icebergPartitionInfo;
- if (!table.isValidRelatedTable()) {
- icebergPartitionInfo = IcebergPartitionInfo.empty();
- } else {
- icebergPartitionInfo = IcebergUtils.loadPartitionInfo((ExternalTable) table,
- lastedIcebergSnapshot.getSnapshotId());
+ try {
+ MTMVRelatedTableIf table = (MTMVRelatedTableIf) dorisTable;
+ IcebergSnapshot lastedIcebergSnapshot = IcebergUtils.getLastedIcebergSnapshot(dorisTable);
Review Comment:
I will fix this
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](iceberg) Refactor Iceberg metadata cache structure and add table cache test cases [doris]
Posted by "suxiaogang223 (via GitHub)" <gi...@apache.org>.
suxiaogang223 commented on PR #59716:
URL: https://github.com/apache/doris/pull/59716#issuecomment-3799293601
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](iceberg) Refactor Iceberg metadata cache structure and add table cache test cases [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #59716:
URL: https://github.com/apache/doris/pull/59716#issuecomment-3815625757
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](iceberg) Refactor Iceberg metadata cache structure and add table cache test cases [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #59716:
URL: https://github.com/apache/doris/pull/59716#issuecomment-3809798574
# FE UT Coverage Report
Increment line coverage ` 0.00% (0/92)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/59716_b94d3ff7cf85ab07e06a96e72967af235065ef7f/fe_increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/59716_b94d3ff7cf85ab07e06a96e72967af235065ef7f/fe_report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](iceberg) Refactor Iceberg metadata cache structure and add table cache test cases [doris]
Posted by "suxiaogang223 (via GitHub)" <gi...@apache.org>.
suxiaogang223 commented on code in PR #59716:
URL: https://github.com/apache/doris/pull/59716#discussion_r2727343441
##########
fe/fe-core/src/main/java/org/apache/doris/datasource/iceberg/IcebergMetadataCache.java:
##########
@@ -55,9 +54,7 @@ public class IcebergMetadataCache {
private static final Logger LOG = LogManager.getLogger(IcebergMetadataCache.class);
private final ExecutorService executor;
private final IcebergExternalCatalog catalog;
- private LoadingCache<IcebergMetadataCacheKey, List<Snapshot>> snapshotListCache;
Review Comment:
yes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](iceberg) Refactor Iceberg metadata cache structure and add table cache test cases [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59716:
URL: https://github.com/apache/doris/pull/59716#issuecomment-3804615934
<details>
<summary>ClickBench: <b>Total hot run time: 28.19 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 991bc75b6530dc1f39b1b3d9dec7880b810253b7, data reload: false
query1 0.05 0.04 0.04
query2 0.10 0.04 0.04
query3 0.25 0.08 0.09
query4 1.60 0.11 0.11
query5 0.29 0.24 0.25
query6 1.17 0.67 0.68
query7 0.04 0.03 0.02
query8 0.06 0.04 0.03
query9 0.56 0.50 0.48
query10 0.56 0.55 0.55
query11 0.14 0.09 0.09
query12 0.14 0.10 0.11
query13 0.63 0.61 0.62
query14 1.04 1.05 1.07
query15 0.87 0.88 0.88
query16 0.39 0.39 0.38
query17 1.13 1.08 1.16
query18 0.23 0.21 0.20
query19 2.14 2.03 2.06
query20 0.02 0.01 0.02
query21 15.43 0.24 0.14
query22 5.28 0.05 0.04
query23 16.10 0.28 0.10
query24 1.04 0.79 0.26
query25 0.09 0.05 0.07
query26 0.14 0.13 0.12
query27 0.09 0.06 0.05
query28 4.13 1.15 0.96
query29 12.57 3.88 3.15
query30 0.28 0.13 0.12
query31 2.82 0.67 0.40
query32 3.23 0.60 0.49
query33 3.20 3.21 3.32
query34 16.40 5.39 4.75
query35 4.83 4.81 4.79
query36 0.66 0.49 0.48
query37 0.11 0.08 0.07
query38 0.07 0.04 0.04
query39 0.05 0.03 0.03
query40 0.18 0.17 0.15
query41 0.10 0.03 0.03
query42 0.04 0.03 0.03
query43 0.05 0.04 0.04
Total cold run time: 98.3 s
Total hot run time: 28.19 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](iceberg) Refactor Iceberg metadata cache structure and add table cache test cases [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59716:
URL: https://github.com/apache/doris/pull/59716#issuecomment-3809619155
<details>
<summary>TPC-H: <b>Total hot run time: 32384 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit b94d3ff7cf85ab07e06a96e72967af235065ef7f, data reload: false
------ Round 1 ----------------------------------
q1 17655 5260 5017 5017
q2 2018 309 198 198
q3 10239 1306 758 758
q4 10211 861 319 319
q5 7560 2134 1946 1946
q6 203 180 151 151
q7 905 734 599 599
q8 9268 1395 1089 1089
q9 5161 4863 4929 4863
q10 6811 1984 1544 1544
q11 527 290 285 285
q12 347 388 237 237
q13 17814 4117 3260 3260
q14 238 242 225 225
q15 889 836 822 822
q16 675 678 620 620
q17 644 793 554 554
q18 6760 6553 7347 6553
q19 1555 1030 701 701
q20 428 386 240 240
q21 2846 2357 2106 2106
q22 378 323 297 297
Total cold run time: 103132 ms
Total hot run time: 32384 ms
----- Round 2, with runtime_filter_mode=off -----
q1 5488 5440 5657 5440
q2 263 391 272 272
q3 2416 2839 2487 2487
q4 1467 1941 1403 1403
q5 4851 4487 4588 4487
q6 230 181 143 143
q7 2055 1939 1829 1829
q8 2680 2494 2581 2494
q9 7607 7489 7863 7489
q10 2974 3259 2582 2582
q11 540 470 447 447
q12 682 776 650 650
q13 3889 4128 3384 3384
q14 276 297 270 270
q15 858 827 791 791
q16 664 708 662 662
q17 1187 1331 1338 1331
q18 7575 7490 7425 7425
q19 838 857 875 857
q20 1962 2048 1888 1888
q21 4462 4299 4170 4170
q22 573 554 529 529
Total cold run time: 53537 ms
Total hot run time: 51030 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](iceberg) Refactor Iceberg metadata cache structure and add table cache test cases [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #59716:
URL: https://github.com/apache/doris/pull/59716#issuecomment-3738355931
# FE Regression Coverage Report
Increment line coverage ` 0.00% (0/36)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/59716_1574f3a0840831ea973580b5275b27182d07b2bf_merge_fe/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/59716_1574f3a0840831ea973580b5275b27182d07b2bf_merge_fe/report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](iceberg) Refactor Iceberg metadata cache structure and add table cache test cases [doris]
Posted by "suxiaogang223 (via GitHub)" <gi...@apache.org>.
suxiaogang223 commented on PR #59716:
URL: https://github.com/apache/doris/pull/59716#issuecomment-3737461425
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](iceberg) Refactor Iceberg metadata cache structure and add table cache test cases [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #59716:
URL: https://github.com/apache/doris/pull/59716#issuecomment-3799576116
# FE UT Coverage Report
Increment line coverage ` 0.00% (0/37)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/59716_35ab7872eaf6d9ed159f68593e694e5cd4ba731d/fe_increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/59716_35ab7872eaf6d9ed159f68593e694e5cd4ba731d/fe_report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](iceberg) Refactor Iceberg metadata cache structure and add table cache test cases [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59716:
URL: https://github.com/apache/doris/pull/59716#issuecomment-3799439300
<details>
<summary>TPC-H: <b>Total hot run time: 32499 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 35ab7872eaf6d9ed159f68593e694e5cd4ba731d, data reload: false
------ Round 1 ----------------------------------
q1 17650 5227 5099 5099
q2 2080 307 200 200
q3 10154 1297 735 735
q4 10219 885 315 315
q5 7555 2161 1894 1894
q6 196 178 147 147
q7 884 745 627 627
q8 9257 1375 1077 1077
q9 5452 4941 4818 4818
q10 6775 1952 1561 1561
q11 521 309 275 275
q12 337 387 230 230
q13 17782 4035 3236 3236
q14 230 244 217 217
q15 879 838 808 808
q16 685 680 613 613
q17 626 819 478 478
q18 6771 6449 6522 6449
q19 1240 983 594 594
q20 398 337 227 227
q21 2611 2008 1933 1933
q22 1046 992 966 966
Total cold run time: 103348 ms
Total hot run time: 32499 ms
----- Round 2, with runtime_filter_mode=off -----
q1 5324 5299 5284 5284
q2 261 334 262 262
q3 2207 2650 2266 2266
q4 1344 1737 1321 1321
q5 4285 4164 4322 4164
q6 302 205 163 163
q7 2351 1994 1836 1836
q8 2581 2397 2380 2380
q9 7611 7677 7487 7487
q10 2874 3173 2599 2599
q11 556 471 466 466
q12 702 781 680 680
q13 3922 4415 3537 3537
q14 318 328 289 289
q15 854 800 825 800
q16 680 766 720 720
q17 1175 1348 1383 1348
q18 8413 7967 7980 7967
q19 913 882 891 882
q20 2060 2176 1991 1991
q21 4828 4597 4186 4186
q22 1037 1012 971 971
Total cold run time: 54598 ms
Total hot run time: 51599 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](iceberg) Refactor Iceberg metadata cache structure and add table cache test cases [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #59716:
URL: https://github.com/apache/doris/pull/59716#issuecomment-3803808648
# FE UT Coverage Report
Increment line coverage ` 0.00% (0/92)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/59716_991bc75b6530dc1f39b1b3d9dec7880b810253b7/fe_increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/59716_991bc75b6530dc1f39b1b3d9dec7880b810253b7/fe_report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](iceberg) Refactor Iceberg metadata cache structure and add table cache test cases [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59716:
URL: https://github.com/apache/doris/pull/59716#issuecomment-3737840479
<details>
<summary>TPC-DS: <b>Total hot run time: 173580 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 1574f3a0840831ea973580b5275b27182d07b2bf, data reload: false
query5 4444 598 436 436
query6 334 231 222 222
query7 4218 466 269 269
query8 330 258 246 246
query9 8770 2715 2657 2657
query10 508 354 336 336
query11 15270 15196 14803 14803
query12 173 112 111 111
query13 1261 477 368 368
query14 6214 2967 2704 2704
query14_1 2632 2581 2635 2581
query15 206 194 175 175
query16 1012 462 468 462
query17 1112 690 588 588
query18 2507 450 353 353
query19 229 228 198 198
query20 126 117 115 115
query21 224 145 125 125
query22 3994 3948 3896 3896
query23 16088 15712 15492 15492
query23_1 15825 15645 15676 15645
query24 7694 1600 1175 1175
query24_1 1185 1186 1213 1186
query25 580 521 451 451
query26 1240 281 165 165
query27 2754 481 311 311
query28 4515 2151 2142 2142
query29 796 569 476 476
query30 315 247 211 211
query31 799 639 560 560
query32 82 77 76 76
query33 554 346 300 300
query34 909 886 521 521
query35 718 784 681 681
query36 921 925 824 824
query37 133 95 79 79
query38 2740 2667 2641 2641
query39 797 756 729 729
query39_1 721 726 747 726
query40 229 136 118 118
query41 74 68 69 68
query42 106 103 102 102
query43 446 433 412 412
query44 1324 729 717 717
query45 192 188 181 181
query46 854 950 598 598
query47 1370 1456 1308 1308
query48 319 341 248 248
query49 638 419 353 353
query50 647 280 216 216
query51 3752 3799 3749 3749
query52 106 116 111 111
query53 304 327 274 274
query54 304 268 266 266
query55 81 77 72 72
query56 343 299 278 278
query57 1036 983 954 954
query58 267 246 254 246
query59 2187 2247 2036 2036
query60 316 321 298 298
query61 165 157 153 153
query62 420 352 312 312
query63 294 261 266 261
query64 4860 1316 994 994
query65 3835 3776 3749 3749
query66 1428 420 311 311
query67 15460 16105 15120 15120
query68 6029 992 713 713
query69 517 353 312 312
query70 1076 958 927 927
query71 418 294 276 276
query72 6008 3420 3527 3420
query73 774 721 300 300
query74 8871 8891 8623 8623
query75 2817 2831 2450 2450
query76 3456 1050 633 633
query77 534 377 274 274
query78 9874 9836 9158 9158
query79 1584 944 598 598
query80 704 572 463 463
query81 513 257 226 226
query82 449 147 110 110
query83 272 258 239 239
query84 257 118 101 101
query85 902 512 465 465
query86 433 302 318 302
query87 2872 2850 2763 2763
query88 3224 2230 2231 2230
query89 399 351 345 345
query90 2148 156 158 156
query91 175 169 141 141
query92 80 64 68 64
query93 1495 904 530 530
query94 637 319 297 297
query95 582 381 296 296
query96 580 452 214 214
query97 2371 2391 2331 2331
query98 239 200 202 200
query99 578 594 490 490
Total cold run time: 254239 ms
Total hot run time: 173580 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] Refact meta cache [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #59716:
URL: https://github.com/apache/doris/pull/59716#issuecomment-3727678777
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR).
Please clearly describe your PR:
1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
3. What features were added. Why was this function added?
4. Which code was refactored and why was this part of the code refactored?
5. Which functions were optimized and what is the difference before and after the optimization?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org