You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "HonestManXin (via GitHub)" <gi...@apache.org> on 2025/11/11 06:47:03 UTC
[PR] [Feature] doris cross-cluster query [doris]
HonestManXin opened a new pull request, #57898:
URL: https://github.com/apache/doris/pull/57898
MySQL [demo]> show frontends;
+-----------------------------------------+--------------+-------------+----------+-----------+---------+--------------------+----------+----------+-----------+------+-------+-------------------+---------------------+---------------------+----------+--------+------------------------+------------------+---------------------+
| Name | Host | EditLogPort | HttpPort | QueryPort | RpcPort | ArrowFlightSqlPort | Role | IsMaster | ClusterId | Join | Alive | ReplayedJournalId | LastStartTime | LastHeartbeat | IsHelper | ErrMsg | Version | CurrentConnected | LiveSince |
+-----------------------------------------+--------------+-------------+----------+-----------+---------+--------------------+----------+----------+-----------+------+-------+-------------------+---------------------+---------------------+----------+--------+------------------------+------------------+---------------------+
| fe_a7c0b6d8_82c2_48f0_8220_fb65dd18be69 | 10.37.75.124 | 9010 | 8030 | 9030 | 9020 | 8070 | FOLLOWER | true | 742250121 | true | true | 2409 | 2025-11-11 14:42:16 | 2025-11-11 14:44:06 | true | | doris-0.0.0-009c3b552a | Yes | 2025-11-11 14:42:16 |
+-----------------------------------------+--------------+-------------+----------+-----------+---------+--------------------+----------+----------+-----------+------+-------+-------------------+---------------------+---------------------+----------+--------+------------------------+------------------+---------------------+
1 row in set (0.016 sec)
MySQL [demo]> show catalog edoris;
+-----------------+-------------------------------+
| Key | Value |
+-----------------+-------------------------------+
| create_time | 2025-11-11 11:25:33.488106853 |
| fe_arrow_hosts | 10.37.103.28:8070 |
| fe_http_hosts | 10.37.103.28:8030 |
| fe_thrift_hosts | 10.37.103.28:9020 |
| password | *XXX |
| type | doris |
| use_meta_cache | true |
| user | test |
+-----------------+-------------------------------+
8 rows in set (0.002 sec)
MySQL [demo]> select * from inner_table;
+----------+--------+
| log_type | reason |
+----------+--------+
| 2 | inner2 |
| 3 | inner3 |
| 4 | inner4 |
+----------+--------+
3 rows in set (0.032 sec)
MySQL [demo]> select * from edoris.external.example_tbl_duplicate;
+---------------------+----------+------------+-----------+-------+---------------------+
| log_time | log_type | error_code | error_msg | op_id | op_time |
+---------------------+----------+------------+-----------+-------+---------------------+
| 2024-11-01 00:00:00 | 2 | 2 | timeout | 12 | 2024-11-01 01:00:00 |
+---------------------+----------+------------+-----------+-------+---------------------+
1 row in set (0.059 sec)
MySQL [demo]> select * from inner_table a join edoris.external.example_tbl_duplicate b on (a.log_type = b.log_type);
+----------+--------+---------------------+----------+------------+-----------+-------+---------------------+
| log_type | reason | log_time | log_type | error_code | error_msg | op_id | op_time |
+----------+--------+---------------------+----------+------------+-----------+-------+---------------------+
| 2 | inner2 | 2024-11-01 00:00:00 | 2 | 2 | timeout | 12 | 2024-11-01 01:00:00 |
+----------+--------+---------------------+----------+------------+-----------+-------+---------------------+
1 row in set (0.050 sec)
MySQL [demo]> explain select * from inner_table a join edoris.external.example_tbl_duplicate b on (a.log_type = b.log_type) where error_code=2;
+-------------------------------------------------------------------------------------------------------------------------------------------+
| Explain String(Nereids Planner) |
+-------------------------------------------------------------------------------------------------------------------------------------------+
| PLAN FRAGMENT 0 |
| OUTPUT EXPRS: |
| log_type[#16] |
| reason[#17] |
| log_time[#18] |
| log_type[#19] |
| error_code[#20] |
| error_msg[#21] |
| op_id[#22] |
| op_time[#23] |
| PARTITION: HASH_PARTITIONED: log_type[#6] |
| |
| HAS_COLO_PLAN_NODE: false |
| |
| VRESULT SINK |
| MYSQL_PROTOCOL |
| |
| 3:VHASH JOIN(200) |
| | join op: INNER JOIN(BROADCAST)[] |
| | equal join conjunct: (log_type[#6] = log_type[#1]) |
| | cardinality=3 |
| | vec output tuple id: 3 |
| | output tuple id: 3 |
| | vIntermediate tuple ids: 2 |
| | hash output slot ids: 0 1 2 3 4 5 6 7 |
| | runtime filters: RF000[min_max] <- log_type[#1](1/1/1048576), RF001[in_or_bloom] <- log_type[#1](1/1/1048576) |
| | final projections: log_type[#8], reason[#9], log_time[#10], log_type[#11], error_code[#12], error_msg[#13], op_id[#14], op_time[#15] |
| | final project output tuple id: 3 |
| | distribute expr lists: log_type[#6] |
| | distribute expr lists: |
| | |
| |----1:VEXCHANGE |
| | offset: 0 |
| | distribute expr lists: log_type[#1] |
| | |
| 2:VOlapScanNode(187) |
| TABLE: demo.inner_table(inner_table), PREAGGREGATION: ON |
| partitions=1/1 (inner_table) |
| tablets=1/1, tabletList=1762832514491 |
| cardinality=3, avgRowSize=901.6666, numNodes=1 |
| pushAggOp=NONE |
| runtime filters: RF000[min_max] -> log_type[#6], RF001[in_or_bloom] -> log_type[#6] |
| |
| PLAN FRAGMENT 1 |
| |
| PARTITION: HASH_PARTITIONED: log_type[#1] |
| |
| HAS_COLO_PLAN_NODE: false |
| |
| STREAM DATA SINK |
| EXCHANGE ID: 01 |
| UNPARTITIONED |
| |
| 0:VOlapScanNode(188) |
| TABLE: external.example_tbl_duplicate(example_tbl_duplicate), PREAGGREGATION: ON |
| PREDICATES: (error_code[#2] = 2) |
| partitions=1/1 (example_tbl_duplicate) |
| tablets=1/1, tabletList=1762481736238 |
| cardinality=1, avgRowSize=7425.0, numNodes=1 |
| pushAggOp=NONE |
| |
| |
| |
| ========== STATISTICS ========== |
| planed with unknown column statistics |
+-------------------------------------------------------------------------------------------------------------------------------------------+
65 rows in set (0.040 sec)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature] doris cross-cluster query [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #57898:
URL: https://github.com/apache/doris/pull/57898#issuecomment-3572245897
# BE Regression && UT Coverage Report
Increment line coverage `100% (0/0)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/57898_397ab88835eda5e584314fdc61afe29e014f140a_merge/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/57898_397ab88835eda5e584314fdc61afe29e014f140a_merge/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 71.55% (24402/34104) |
| Line Coverage | 58.02% (253664/437170) |
| Region Coverage | 53.33% (211697/396939) |
| Branch Coverage | 54.70% (90325/165131) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature] doris cross-cluster query [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #57898:
URL: https://github.com/apache/doris/pull/57898#issuecomment-3574393335
<details>
<summary>TPC-H: <b>Total hot run time: 35001 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 113154a9292df06cdf5ac65f15d887c14e1a6f1d, data reload: false
------ Round 1 ----------------------------------
q1 17600 5070 4857 4857
q2 2029 331 209 209
q3 10176 1280 740 740
q4 10227 878 370 370
q5 7451 2401 2320 2320
q6 184 167 139 139
q7 922 773 642 642
q8 9349 1335 1119 1119
q9 7157 5347 5288 5288
q10 6907 2198 1837 1837
q11 496 309 285 285
q12 378 364 222 222
q13 17798 3637 2994 2994
q14 226 242 224 224
q15 614 512 528 512
q16 1037 1019 943 943
q17 592 898 375 375
q18 7810 7551 7749 7551
q19 1114 986 597 597
q20 377 353 232 232
q21 4313 3615 2506 2506
q22 1144 1109 1039 1039
Total cold run time: 107901 ms
Total hot run time: 35001 ms
----- Round 2, with runtime_filter_mode=off -----
q1 5249 5156 5125 5125
q2 342 416 326 326
q3 2401 2937 2467 2467
q4 1422 1965 1390 1390
q5 4637 4463 4567 4463
q6 207 169 130 130
q7 2024 1912 1842 1842
q8 2646 2559 2529 2529
q9 7598 7306 6893 6893
q10 2900 3093 2644 2644
q11 572 492 481 481
q12 625 727 574 574
q13 3254 3727 3048 3048
q14 288 279 269 269
q15 558 498 492 492
q16 1048 1060 1032 1032
q17 1114 1506 1321 1321
q18 7318 7062 6990 6990
q19 774 727 833 727
q20 1920 1972 1837 1837
q21 4701 4376 4325 4325
q22 1090 1059 1027 1027
Total cold run time: 52688 ms
Total hot run time: 49932 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature] doris cross-cluster query [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #57898:
URL: https://github.com/apache/doris/pull/57898#issuecomment-3574479474
<details>
<summary>ClickBench: <b>Total hot run time: 28.44 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 113154a9292df06cdf5ac65f15d887c14e1a6f1d, data reload: false
query1 0.06 0.05 0.05
query2 0.13 0.08 0.07
query3 0.31 0.08 0.07
query4 1.62 0.09 0.09
query5 0.26 0.27 0.24
query6 1.19 0.67 0.67
query7 0.03 0.02 0.02
query8 0.07 0.06 0.06
query9 0.64 0.53 0.56
query10 0.58 0.60 0.59
query11 0.26 0.13 0.13
query12 0.26 0.14 0.14
query13 0.64 0.62 0.62
query14 1.01 1.03 1.02
query15 0.93 0.87 0.85
query16 0.40 0.38 0.38
query17 1.06 1.06 1.05
query18 0.24 0.22 0.22
query19 1.95 1.78 1.82
query20 0.01 0.02 0.01
query21 15.40 0.29 0.24
query22 4.97 0.10 0.10
query23 15.38 0.39 0.23
query24 2.97 0.47 0.31
query25 0.10 0.09 0.09
query26 0.18 0.17 0.17
query27 0.10 0.09 0.09
query28 3.62 1.24 1.10
query29 12.55 4.06 3.31
query30 0.32 0.14 0.11
query31 2.83 0.64 0.45
query32 3.23 0.61 0.52
query33 3.13 3.06 3.10
query34 16.49 5.13 4.48
query35 4.47 4.56 4.62
query36 0.65 0.51 0.51
query37 0.23 0.09 0.08
query38 0.18 0.06 0.06
query39 0.06 0.04 0.05
query40 0.21 0.18 0.19
query41 0.10 0.06 0.06
query42 0.07 0.05 0.05
query43 0.06 0.06 0.05
Total cold run time: 98.95 s
Total hot run time: 28.44 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature] doris cross-cluster query [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #57898:
URL: https://github.com/apache/doris/pull/57898#issuecomment-3568136791
<details>
<summary>TPC-H: <b>Total hot run time: 34370 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 4b7d8a7a3ce6da25dd8a85bbb9a335b8df6d8b86, data reload: false
------ Round 1 ----------------------------------
q1 17626 5137 4905 4905
q2 2015 321 218 218
q3 10172 1291 731 731
q4 10242 912 359 359
q5 7505 2426 2387 2387
q6 186 174 141 141
q7 914 774 649 649
q8 9353 1372 1173 1173
q9 7289 5473 5299 5299
q10 6917 2264 1806 1806
q11 501 297 288 288
q12 368 367 224 224
q13 17823 3659 3046 3046
q14 234 241 224 224
q15 583 511 507 507
q16 1049 998 943 943
q17 591 882 364 364
q18 7504 7070 7132 7070
q19 1102 950 543 543
q20 348 350 227 227
q21 3732 3227 2299 2299
q22 1092 1044 967 967
Total cold run time: 107146 ms
Total hot run time: 34370 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4990 4952 4916 4916
q2 316 419 293 293
q3 2183 2679 2272 2272
q4 1390 1779 1297 1297
q5 4207 4498 4446 4446
q6 214 175 130 130
q7 2026 1910 1861 1861
q8 2740 2694 2562 2562
q9 7649 7663 7684 7663
q10 3098 3222 2781 2781
q11 601 518 510 510
q12 683 757 603 603
q13 3421 4165 3198 3198
q14 295 320 292 292
q15 555 491 498 491
q16 1098 1108 1051 1051
q17 1169 1618 1358 1358
q18 7932 7900 7465 7465
q19 839 804 845 804
q20 2083 2116 1813 1813
q21 4755 4431 4345 4345
q22 1056 1041 1019 1019
Total cold run time: 53300 ms
Total hot run time: 51170 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature] doris cross-cluster query [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #57898:
URL: https://github.com/apache/doris/pull/57898#issuecomment-3584994841
<details>
<summary>TPC-DS: <b>Total hot run time: 183635 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 2f096957f7645a9b79d50dcff93a47a5dc47aac8, data reload: false
query1 1021 404 430 404
query2 6587 1602 1585 1585
query3 6755 229 222 222
query4 25902 23000 22170 22170
query5 4427 641 468 468
query6 348 242 231 231
query7 4645 486 299 299
query8 314 255 240 240
query9 8741 2558 2568 2558
query10 523 363 314 314
query11 15167 14825 14530 14530
query12 178 121 113 113
query13 1691 578 448 448
query14 10123 8866 8866 8866
query15 215 201 187 187
query16 7383 702 554 554
query17 1269 761 642 642
query18 2009 434 343 343
query19 218 206 185 185
query20 134 131 122 122
query21 217 138 117 117
query22 3930 3995 3872 3872
query23 32998 32091 32097 32091
query24 8509 2441 2399 2399
query25 638 551 484 484
query26 1237 289 166 166
query27 2737 499 344 344
query28 4393 2150 2140 2140
query29 855 652 521 521
query30 323 241 215 215
query31 860 703 647 647
query32 82 73 74 73
query33 615 399 339 339
query34 782 880 534 534
query35 823 879 750 750
query36 923 962 881 881
query37 121 111 93 93
query38 3337 3357 3278 3278
query39 1455 1399 1411 1399
query40 225 129 119 119
query41 66 61 62 61
query42 123 112 111 111
query43 449 468 425 425
query44 1227 738 740 738
query45 198 191 192 191
query46 908 985 633 633
query47 1699 1730 1679 1679
query48 388 426 335 335
query49 778 497 416 416
query50 646 679 420 420
query51 3903 3897 3828 3828
query52 115 113 106 106
query53 240 275 197 197
query54 304 305 278 278
query55 96 88 88 88
query56 330 339 319 319
query57 1144 1167 1074 1074
query58 302 270 269 269
query59 2372 2528 2363 2363
query60 342 355 331 331
query61 161 159 158 158
query62 771 733 657 657
query63 227 190 190 190
query64 4548 1226 895 895
query65 4090 3952 3981 3952
query66 1170 433 332 332
query67 15244 14988 14781 14781
query68 8355 893 616 616
query69 510 340 298 298
query70 1311 1234 1136 1136
query71 478 327 300 300
query72 5719 4851 4841 4841
query73 684 574 340 340
query74 9010 8753 8544 8544
query75 3951 3349 2817 2817
query76 3733 1135 732 732
query77 830 417 310 310
query78 9498 9672 8961 8961
query79 2335 842 583 583
query80 691 572 503 503
query81 497 266 239 239
query82 438 159 129 129
query83 298 258 256 256
query84 310 127 98 98
query85 910 499 448 448
query86 389 312 283 283
query87 3536 3522 3472 3472
query88 3288 2250 2264 2250
query89 389 329 289 289
query90 1925 216 217 216
query91 173 179 138 138
query92 86 66 70 66
query93 1779 977 643 643
query94 717 468 349 349
query95 511 398 397 397
query96 520 580 281 281
query97 2940 3004 2847 2847
query98 233 219 203 203
query99 1425 1382 1280 1280
Total cold run time: 272020 ms
Total hot run time: 183635 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature] doris cross-cluster query [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #57898:
URL: https://github.com/apache/doris/pull/57898#issuecomment-3569398968
<details>
<summary>TPC-H: <b>Total hot run time: 34242 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 639c5c970ba78c1055b2c830bd4c20729ffa601d, data reload: false
------ Round 1 ----------------------------------
q1 17425 5106 4913 4913
q2 2041 320 207 207
q3 10146 1326 729 729
q4 10227 925 362 362
q5 7519 2394 2333 2333
q6 179 170 136 136
q7 909 773 625 625
q8 9344 1238 1145 1145
q9 7165 5442 5359 5359
q10 6869 2226 1809 1809
q11 497 302 276 276
q12 361 358 223 223
q13 17820 3683 3010 3010
q14 235 238 216 216
q15 567 517 505 505
q16 1015 996 940 940
q17 600 867 373 373
q18 7378 7081 7042 7042
q19 1180 940 533 533
q20 348 334 228 228
q21 3764 3192 2313 2313
q22 1014 1000 965 965
Total cold run time: 106603 ms
Total hot run time: 34242 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4949 4940 4919 4919
q2 322 390 310 310
q3 2182 2653 2318 2318
q4 1359 1776 1334 1334
q5 4186 4326 4611 4326
q6 222 173 137 137
q7 2036 1888 1824 1824
q8 2705 2550 2590 2550
q9 7682 7609 7686 7609
q10 3099 3241 2842 2842
q11 584 536 512 512
q12 679 800 886 800
q13 3458 3873 3352 3352
q14 290 310 288 288
q15 561 501 494 494
q16 1066 1098 1048 1048
q17 1219 1545 1417 1417
q18 7771 7739 7521 7521
q19 786 884 1007 884
q20 2074 1951 1844 1844
q21 4687 4300 4212 4212
q22 1083 1087 981 981
Total cold run time: 53000 ms
Total hot run time: 51522 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature] doris cross-cluster query [doris]
Posted by "HonestManXin (via GitHub)" <gi...@apache.org>.
HonestManXin commented on PR #57898:
URL: https://github.com/apache/doris/pull/57898#issuecomment-3585700028
run p0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature] doris cross-cluster query [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #57898:
URL: https://github.com/apache/doris/pull/57898#issuecomment-3568903193
<details>
<summary>ClickBench: <b>Total hot run time: 28.32 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 6ebf97d578357addf4ee3a69c8b6d2eeb8bfa59b, data reload: false
query1 0.05 0.05 0.04
query2 0.12 0.07 0.06
query3 0.32 0.08 0.07
query4 1.61 0.09 0.09
query5 0.26 0.26 0.25
query6 1.18 0.65 0.65
query7 0.03 0.02 0.02
query8 0.07 0.06 0.07
query9 0.63 0.54 0.53
query10 0.59 0.58 0.58
query11 0.26 0.13 0.14
query12 0.26 0.14 0.15
query13 0.64 0.62 0.62
query14 1.03 1.04 1.03
query15 0.95 0.86 0.86
query16 0.39 0.38 0.40
query17 1.05 1.05 1.05
query18 0.23 0.22 0.23
query19 2.00 1.76 1.85
query20 0.02 0.01 0.01
query21 15.40 0.29 0.25
query22 4.99 0.10 0.10
query23 15.34 0.38 0.23
query24 2.61 0.53 0.34
query25 0.09 0.09 0.10
query26 0.19 0.18 0.18
query27 0.10 0.10 0.09
query28 3.62 1.25 1.07
query29 12.55 3.99 3.32
query30 0.33 0.12 0.10
query31 2.82 0.62 0.45
query32 3.24 0.60 0.50
query33 3.11 3.07 3.20
query34 16.56 5.07 4.45
query35 4.52 4.55 4.51
query36 0.64 0.52 0.50
query37 0.23 0.09 0.09
query38 0.20 0.06 0.06
query39 0.05 0.05 0.05
query40 0.21 0.17 0.18
query41 0.10 0.07 0.06
query42 0.08 0.05 0.04
query43 0.06 0.06 0.05
Total cold run time: 98.73 s
Total hot run time: 28.32 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature] doris cross-cluster query [doris]
Posted by "HonestManXin (via GitHub)" <gi...@apache.org>.
HonestManXin commented on PR #57898:
URL: https://github.com/apache/doris/pull/57898#issuecomment-3573560246
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature] doris cross-cluster query [doris]
Posted by "HonestManXin (via GitHub)" <gi...@apache.org>.
HonestManXin commented on PR #57898:
URL: https://github.com/apache/doris/pull/57898#issuecomment-3588439438
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature] doris cross-cluster query [doris]
Posted by "HonestManXin (via GitHub)" <gi...@apache.org>.
HonestManXin commented on PR #57898:
URL: https://github.com/apache/doris/pull/57898#issuecomment-3568881828
run external
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature] doris cross-cluster query [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #57898:
URL: https://github.com/apache/doris/pull/57898#issuecomment-3576695635
# BE Regression && UT Coverage Report
Increment line coverage `100% (0/0)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/57898_608564073977a55e9151e8a1eed087104db5aea4_merge/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/57898_608564073977a55e9151e8a1eed087104db5aea4_merge/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 71.69% (24502/34180) |
| Line Coverage | 58.17% (255273/438825) |
| Region Coverage | 53.22% (212473/399249) |
| Branch Coverage | 54.84% (90985/165922) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature] doris cross-cluster query [doris]
Posted by "HonestManXin (via GitHub)" <gi...@apache.org>.
HonestManXin commented on PR #57898:
URL: https://github.com/apache/doris/pull/57898#issuecomment-3575484358
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature] doris cross-cluster query [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #57898:
URL: https://github.com/apache/doris/pull/57898#issuecomment-3592011520
# FE Regression Coverage Report
Increment line coverage ` 64.52% (320/496)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/57898_2019c7c87f06a8cb801b7ced0d007f2f7784b3be_merge_fe/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/57898_2019c7c87f06a8cb801b7ced0d007f2f7784b3be_merge_fe/report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature] doris cross-cluster query [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #57898:
URL: https://github.com/apache/doris/pull/57898#issuecomment-3584955944
<details>
<summary>TPC-H: <b>Total hot run time: 34659 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 2f096957f7645a9b79d50dcff93a47a5dc47aac8, data reload: false
------ Round 1 ----------------------------------
q1 17620 4995 4946 4946
q2 2032 332 209 209
q3 10186 1293 725 725
q4 10224 894 362 362
q5 7525 2412 2360 2360
q6 187 173 140 140
q7 933 794 669 669
q8 9353 1348 1173 1173
q9 7448 5355 5319 5319
q10 6889 2229 1819 1819
q11 516 306 287 287
q12 370 356 227 227
q13 17796 3657 3057 3057
q14 236 241 212 212
q15 589 510 509 509
q16 1050 1048 978 978
q17 589 881 349 349
q18 8108 7336 7105 7105
q19 1089 969 559 559
q20 359 361 249 249
q21 3992 3529 2392 2392
q22 1098 1042 1013 1013
Total cold run time: 108189 ms
Total hot run time: 34659 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4960 5019 4941 4941
q2 324 407 321 321
q3 2241 2794 2394 2394
q4 1387 1836 1348 1348
q5 4366 4575 4589 4575
q6 217 169 131 131
q7 2054 2031 1806 1806
q8 2664 2675 2562 2562
q9 7548 7556 7488 7488
q10 3051 3260 2823 2823
q11 592 522 520 520
q12 696 767 640 640
q13 3513 3930 3263 3263
q14 285 299 297 297
q15 550 502 500 500
q16 1065 1163 1086 1086
q17 1169 1651 1349 1349
q18 8069 7997 7718 7718
q19 805 827 958 827
q20 1957 1973 1807 1807
q21 4689 4361 4279 4279
q22 1106 1054 1050 1050
Total cold run time: 53308 ms
Total hot run time: 51725 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature] doris cross-cluster query [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #57898:
URL: https://github.com/apache/doris/pull/57898#issuecomment-3591000616
# FE Regression Coverage Report
Increment line coverage ` 64.83% (317/489)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/57898_6e75d38a8a85d1bc4670594b3bf7cdd32ba2ee3c_merge_fe/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/57898_6e75d38a8a85d1bc4670594b3bf7cdd32ba2ee3c_merge_fe/report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature] doris cross-cluster query [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #57898:
URL: https://github.com/apache/doris/pull/57898#issuecomment-3591849454
# BE UT Coverage Report
Increment line coverage ` ` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/2019c7c87f06a8cb801b7ced0d007f2f7784b3be_2019c7c87f06a8cb801b7ced0d007f2f7784b3be/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/2019c7c87f06a8cb801b7ced0d007f2f7784b3be_2019c7c87f06a8cb801b7ced0d007f2f7784b3be/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 52.90% (18512/34995) |
| Line Coverage | 38.45% (169631/441186) |
| Region Coverage | 33.23% (131625/396152) |
| Branch Coverage | 34.19% (56683/165811) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature] doris cross-cluster query [doris]
Posted by "HonestManXin (via GitHub)" <gi...@apache.org>.
HonestManXin commented on PR #57898:
URL: https://github.com/apache/doris/pull/57898#issuecomment-3588353572
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature] doris cross-cluster query [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #57898:
URL: https://github.com/apache/doris/pull/57898#issuecomment-3568091511
# FE UT Coverage Report
Increment line coverage ` 13.76% (63/458)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/57898_4b7d8a7a3ce6da25dd8a85bbb9a335b8df6d8b86/fe_increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/57898_4b7d8a7a3ce6da25dd8a85bbb9a335b8df6d8b86/fe_report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature] doris cross-cluster query [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #57898:
URL: https://github.com/apache/doris/pull/57898#issuecomment-3592345765
<details>
<summary>TPC-H: <b>Total hot run time: 34035 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 2019c7c87f06a8cb801b7ced0d007f2f7784b3be, data reload: false
------ Round 1 ----------------------------------
q1 16484 5082 4893 4893
q2 2031 319 205 205
q3 9891 1293 709 709
q4 9889 890 359 359
q5 7562 2499 2208 2208
q6 187 172 137 137
q7 950 787 636 636
q8 9233 1410 1034 1034
q9 6899 5342 5284 5284
q10 6809 2233 1812 1812
q11 512 312 278 278
q12 326 372 223 223
q13 17572 3669 2995 2995
q14 227 245 213 213
q15 573 528 533 528
q16 890 867 817 817
q17 576 763 493 493
q18 7401 7067 7075 7067
q19 974 946 592 592
q20 339 339 234 234
q21 3580 3442 2367 2367
q22 1011 972 951 951
Total cold run time: 103916 ms
Total hot run time: 34035 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4964 4943 4920 4920
q2 332 418 330 330
q3 2094 2656 2294 2294
q4 1319 1768 1271 1271
q5 4211 4143 4488 4143
q6 224 179 138 138
q7 2053 1962 1861 1861
q8 2681 2625 2456 2456
q9 7542 7530 7430 7430
q10 3065 3237 2820 2820
q11 612 543 509 509
q12 733 797 623 623
q13 3536 3932 3348 3348
q14 306 299 265 265
q15 550 524 519 519
q16 880 918 869 869
q17 1150 1319 1388 1319
q18 8129 7621 7717 7621
q19 822 782 757 757
q20 1991 2040 1901 1901
q21 4705 4188 4152 4152
q22 1094 1010 988 988
Total cold run time: 52993 ms
Total hot run time: 50534 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature] doris cross-cluster query [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #57898:
URL: https://github.com/apache/doris/pull/57898#issuecomment-3592309662
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature] doris cross-cluster query [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #57898:
URL: https://github.com/apache/doris/pull/57898#issuecomment-3589150426
# BE UT Coverage Report
Increment line coverage ` ` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/6e75d38a8a85d1bc4670594b3bf7cdd32ba2ee3c_6e75d38a8a85d1bc4670594b3bf7cdd32ba2ee3c/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/6e75d38a8a85d1bc4670594b3bf7cdd32ba2ee3c_6e75d38a8a85d1bc4670594b3bf7cdd32ba2ee3c/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 52.88% (18503/34989) |
| Line Coverage | 38.42% (169470/441144) |
| Region Coverage | 33.22% (131591/396097) |
| Branch Coverage | 34.17% (56652/165795) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature] doris cross-cluster query [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #57898:
URL: https://github.com/apache/doris/pull/57898#issuecomment-3588842027
<details>
<summary>ClickBench: <b>Total hot run time: 28.48 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 6e75d38a8a85d1bc4670594b3bf7cdd32ba2ee3c, data reload: false
query1 0.05 0.05 0.05
query2 0.14 0.07 0.07
query3 0.32 0.08 0.08
query4 1.61 0.10 0.11
query5 0.26 0.24 0.25
query6 1.17 0.69 0.64
query7 0.03 0.02 0.03
query8 0.07 0.06 0.05
query9 0.58 0.52 0.51
query10 0.56 0.56 0.56
query11 0.26 0.14 0.14
query12 0.26 0.14 0.14
query13 0.66 0.62 0.61
query14 1.02 1.00 1.00
query15 0.87 0.81 0.79
query16 0.40 0.40 0.38
query17 1.05 0.98 1.01
query18 0.24 0.21 0.22
query19 1.91 1.84 1.82
query20 0.02 0.01 0.02
query21 15.40 0.26 0.23
query22 4.98 0.09 0.10
query23 15.42 0.37 0.22
query24 2.50 0.43 0.31
query25 0.09 0.09 0.09
query26 0.18 0.18 0.17
query27 0.10 0.09 0.09
query28 3.93 1.36 1.16
query29 12.54 4.03 3.30
query30 0.32 0.12 0.11
query31 2.81 0.63 0.44
query32 3.23 0.57 0.49
query33 3.12 3.18 3.12
query34 16.58 5.15 4.61
query35 4.62 4.62 4.69
query36 0.61 0.49 0.49
query37 0.25 0.08 0.09
query38 0.21 0.06 0.06
query39 0.07 0.06 0.06
query40 0.20 0.18 0.17
query41 0.12 0.07 0.06
query42 0.08 0.05 0.05
query43 0.06 0.06 0.06
Total cold run time: 98.9 s
Total hot run time: 28.48 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature] doris cross-cluster query [doris]
Posted by "HonestManXin (via GitHub)" <gi...@apache.org>.
HonestManXin commented on PR #57898:
URL: https://github.com/apache/doris/pull/57898#issuecomment-3570497828
run external
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature] doris cross-cluster query [doris]
Posted by "HonestManXin (via GitHub)" <gi...@apache.org>.
HonestManXin commented on PR #57898:
URL: https://github.com/apache/doris/pull/57898#issuecomment-3568620206
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature] doris cross-cluster query [doris]
Posted by "HonestManXin (via GitHub)" <gi...@apache.org>.
HonestManXin commented on PR #57898:
URL: https://github.com/apache/doris/pull/57898#issuecomment-3568682996
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature] doris cross-cluster query [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #57898:
URL: https://github.com/apache/doris/pull/57898#issuecomment-3576769306
# FE Regression Coverage Report
Increment line coverage ` 65.31% (305/467)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/57898_608564073977a55e9151e8a1eed087104db5aea4_merge_fe/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/57898_608564073977a55e9151e8a1eed087104db5aea4_merge_fe/report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature] doris cross-cluster query [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #57898:
URL: https://github.com/apache/doris/pull/57898#issuecomment-3568878293
<details>
<summary>TPC-H: <b>Total hot run time: 34344 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 6ebf97d578357addf4ee3a69c8b6d2eeb8bfa59b, data reload: false
------ Round 1 ----------------------------------
q1 17439 5046 4926 4926
q2 2062 314 227 227
q3 10127 1288 711 711
q4 10232 976 371 371
q5 7488 2342 2401 2342
q6 182 174 134 134
q7 904 774 626 626
q8 9347 1505 1141 1141
q9 7279 5353 5394 5353
q10 6895 2220 1805 1805
q11 503 291 292 291
q12 354 381 213 213
q13 17799 3636 3010 3010
q14 231 251 218 218
q15 591 515 520 515
q16 1017 991 952 952
q17 599 863 366 366
q18 7913 7187 7173 7173
q19 1102 939 546 546
q20 362 334 229 229
q21 3701 2524 2252 2252
q22 1051 979 943 943
Total cold run time: 107178 ms
Total hot run time: 34344 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4957 4905 4926 4905
q2 319 391 319 319
q3 2148 2667 2286 2286
q4 1356 1767 1330 1330
q5 4210 4371 4384 4371
q6 224 185 131 131
q7 2097 1974 1740 1740
q8 2708 2633 2628 2628
q9 7647 7577 7583 7577
q10 3071 3231 2789 2789
q11 591 525 528 525
q12 681 1033 593 593
q13 3406 3935 3300 3300
q14 289 304 284 284
q15 570 502 487 487
q16 1083 1085 1067 1067
q17 1171 1533 1399 1399
q18 7815 7582 7651 7582
q19 800 811 864 811
q20 1999 2256 1877 1877
q21 4978 4387 4330 4330
q22 1093 1023 1011 1011
Total cold run time: 53213 ms
Total hot run time: 51342 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature] doris cross-cluster query [doris]
Posted by "HonestManXin (via GitHub)" <gi...@apache.org>.
HonestManXin commented on PR #57898:
URL: https://github.com/apache/doris/pull/57898#issuecomment-3568881997
run p0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature] doris cross-cluster query [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #57898:
URL: https://github.com/apache/doris/pull/57898#issuecomment-3569098793
# FE Regression Coverage Report
Increment line coverage ` 17.47% (80/458)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/57898_6ebf97d578357addf4ee3a69c8b6d2eeb8bfa59b_merge_fe/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/57898_6ebf97d578357addf4ee3a69c8b6d2eeb8bfa59b_merge_fe/report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature] doris cross-cluster query [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #57898:
URL: https://github.com/apache/doris/pull/57898#issuecomment-3569438115
<details>
<summary>TPC-DS: <b>Total hot run time: 187123 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 639c5c970ba78c1055b2c830bd4c20729ffa601d, data reload: false
query1 1061 422 398 398
query2 6611 1726 1721 1721
query3 6753 230 222 222
query4 25980 23350 22894 22894
query5 4425 651 504 504
query6 345 244 229 229
query7 4646 499 304 304
query8 315 263 248 248
query9 8725 2594 2611 2594
query10 499 366 307 307
query11 15664 14964 14925 14925
query12 179 124 116 116
query13 1718 585 474 474
query14 10747 9135 9139 9135
query15 190 192 177 177
query16 7394 662 485 485
query17 1238 747 615 615
query18 2004 446 319 319
query19 203 202 181 181
query20 133 125 124 124
query21 213 130 115 115
query22 3970 4096 3947 3947
query23 34290 33090 32954 32954
query24 8430 2428 2411 2411
query25 595 541 445 445
query26 1243 269 164 164
query27 2751 499 358 358
query28 4338 2200 2192 2192
query29 852 613 494 494
query30 297 226 191 191
query31 925 800 727 727
query32 84 79 83 79
query33 583 370 318 318
query34 800 851 507 507
query35 821 833 743 743
query36 925 997 889 889
query37 122 103 90 90
query38 3610 3543 3517 3517
query39 1490 1401 1407 1401
query40 220 124 114 114
query41 64 61 60 60
query42 125 112 111 111
query43 490 493 456 456
query44 1261 757 771 757
query45 192 179 174 174
query46 891 978 647 647
query47 1754 1823 1693 1693
query48 395 415 315 315
query49 773 482 395 395
query50 653 677 403 403
query51 3987 3835 3866 3835
query52 112 109 101 101
query53 258 269 192 192
query54 309 288 275 275
query55 92 85 85 85
query56 321 325 326 325
query57 1202 1189 1099 1099
query58 280 277 268 268
query59 2504 2549 2478 2478
query60 344 340 326 326
query61 166 162 152 152
query62 796 710 662 662
query63 232 191 192 191
query64 4447 1164 883 883
query65 4030 3935 3913 3913
query66 1137 440 331 331
query67 15386 15040 14813 14813
query68 5147 916 639 639
query69 494 329 289 289
query70 1277 1287 1244 1244
query71 435 328 313 313
query72 5855 5145 5300 5145
query73 661 588 362 362
query74 8890 8820 9015 8820
query75 3290 3320 2823 2823
query76 3286 1148 723 723
query77 517 404 337 337
query78 9651 9932 8869 8869
query79 1923 895 606 606
query80 1552 566 488 488
query81 569 264 224 224
query82 410 161 133 133
query83 268 263 254 254
query84 260 109 91 91
query85 947 474 436 436
query86 462 319 295 295
query87 3767 3625 3590 3590
query88 2866 2256 2223 2223
query89 377 325 287 287
query90 1884 223 215 215
query91 164 164 135 135
query92 78 66 62 62
query93 1663 1016 688 688
query94 743 460 346 346
query95 416 326 322 322
query96 491 573 281 281
query97 2876 2964 2859 2859
query98 242 217 208 208
query99 1322 1465 1248 1248
Total cold run time: 270373 ms
Total hot run time: 187123 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature] doris cross-cluster query [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #57898:
URL: https://github.com/apache/doris/pull/57898#issuecomment-3575584186
# Cloud UT Coverage Report
Increment line coverage ` ` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/608564073977a55e9151e8a1eed087104db5aea4_608564073977a55e9151e8a1eed087104db5aea4_cloud/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/608564073977a55e9151e8a1eed087104db5aea4_608564073977a55e9151e8a1eed087104db5aea4_cloud/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 80.86% (1707/2111) |
| Line Coverage | 66.86% (29890/44708) |
| Region Coverage | 67.39% (14903/22114) |
| Branch Coverage | 57.55% (7924/13768) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature] doris cross-cluster query [doris]
Posted by "Copilot (via GitHub)" <gi...@apache.org>.
Copilot commented on code in PR #57898:
URL: https://github.com/apache/doris/pull/57898#discussion_r2559243788
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/distribute/worker/BackendDistributedPlanWorkerManager.java:
##########
@@ -149,17 +166,17 @@ public long randomAvailableWorker(Map<TNetworkAddress, Long> addressToBackendID)
}
@Override
- public List<Backend> getAllBackends(boolean needAlive) {
+ public List<Backend> getAllBackends(long catalogId, boolean needAlive) {
List<Backend> backends = null;
if (needAlive) {
backends = Lists.newArrayList();
- for (Map.Entry<Long, Backend> entry : this.allClusterBackends.get().entrySet()) {
+ for (Map.Entry<Long, Backend> entry : this.allClusterBackends.get(catalogId).get().entrySet()) {
if (entry.getValue().isQueryAvailable()) {
backends.add(entry.getValue());
}
}
} else {
- backends = Lists.newArrayList(this.allClusterBackends.get().values());
+ backends = Lists.newArrayList(this.allClusterBackends.get(catalogId).get().values());
Review Comment:
Potential NullPointerException: Similar issue at line 179 where `allClusterBackends.get(catalogId)` could return null.
##########
fe/fe-core/src/main/java/org/apache/doris/qe/SessionVariable.java:
##########
@@ -4393,6 +4402,7 @@ public void setEnablePushDownMinMaxOnUnique(boolean enablePushDownMinMaxOnUnique
this.enablePushDownMinMaxOnUnique = enablePushDownMinMaxOnUnique;
}
+ /** canUseNereidsDistributePlanner */
Review Comment:
Incorrect JavaDoc comment: The comment `/** canUseNereidsDistributePlanner */` on line 4405 is placed above the wrong method (`isEnablePushDownStringMinMax`). This appears to be a copy-paste error. Remove or correct this misplaced comment.
```suggestion
```
##########
fe/fe-core/src/main/java/org/apache/doris/service/FrontendServiceImpl.java:
##########
@@ -4507,6 +4516,102 @@ public TGetTableTDEInfoResult getTableTDEInfo(TGetTableTDEInfoRequest request) t
return result;
}
+ /**
+ * only copy basic meta about table
+ * do not copy the partitions
+ * @param request
+ * @return
+ * @throws TException
+ */
+ @Override
+ public TGetOlapTableMetaResult getOlapTableMeta(TGetOlapTableMetaRequest request) throws TException {
+ String clientAddr = getClientAddrAsString();
+ if (LOG.isDebugEnabled()) {
+ LOG.debug("receive getOlapTableMeta request: {}, client: {}", request, clientAddr);
+ }
+ TGetOlapTableMetaResult result = new TGetOlapTableMetaResult();
+ TStatus status = new TStatus(TStatusCode.OK);
+ result.setStatus(status);
+ try {
+ checkSingleTablePasswordAndPrivs(request.getUser(), request.getPasswd(), request.getDb(),
+ request.getTable(), clientAddr, PrivPredicate.SELECT);
+ String dbName = request.getDb();
+ Database db = Env.getCurrentInternalCatalog().getDbNullable(dbName);
+ if (db == null) {
+ throw new UserException("unknown database, database=" + dbName);
+ }
+ OlapTable table = (OlapTable) db.getTableNullable(request.getTable());
+ if (table == null) {
+ throw new UserException("unknown table, table=" + request.getTable());
+ }
+ MetaContext metaContext = new MetaContext();
+ metaContext.setMetaVersion(FeConstants.meta_version);
+ metaContext.setThreadLocalInfo();
+ table.readLock();
+ try (ByteArrayOutputStream bOutputStream = new ByteArrayOutputStream(8192)) {
+ OlapTable copyTable = table.copyTableMeta();
+ try (DataOutputStream out = new DataOutputStream(bOutputStream)) {
+ copyTable.write(out);
+ out.flush();
+ result.setTableMeta(bOutputStream.toByteArray());
+ }
+ Set<Long> updatedPartitionIds = Sets.newHashSet(table.getPartitionIds());
+ List<TPartitionMeta> partitionMetas = request.getPartitionsSize() == 0 ? Lists.newArrayList()
+ : request.getPartitions();
+ for (TPartitionMeta partitionMeta : partitionMetas) {
+ if (request.getTableId() != table.getId()) {
+ result.addToRemovedPartitions(partitionMeta.getId());
+ continue;
+ }
+ Partition partition = table.getPartition(partitionMeta.getId());
+ if (partition == null) {
+ result.addToRemovedPartitions(partitionMeta.getId());
+ continue;
+ }
+ if (partition.getVisibleVersion() == partitionMeta.getVisibleVersion()
+ && partition.getVisibleVersionTime() == partitionMeta.getVisibleVersionTime()) {
+ updatedPartitionIds.remove(partitionMeta.getId());
+ }
Review Comment:
Logic error in partition check: The check `if (request.getTableId() != table.getId())` on line 4562 is inside the loop over partitions (line 4561), which means all partitions will be marked as removed if the table ID doesn't match. This check should be moved outside the loop, before iterating over partitions, to avoid unnecessary iteration and incorrect behavior.
```suggestion
// Move table ID check outside the loop for correct logic
if (request.getTableId() != table.getId()) {
for (TPartitionMeta partitionMeta : partitionMetas) {
result.addToRemovedPartitions(partitionMeta.getId());
}
} else {
for (TPartitionMeta partitionMeta : partitionMetas) {
Partition partition = table.getPartition(partitionMeta.getId());
if (partition == null) {
result.addToRemovedPartitions(partitionMeta.getId());
continue;
}
if (partition.getVisibleVersion() == partitionMeta.getVisibleVersion()
&& partition.getVisibleVersionTime() == partitionMeta.getVisibleVersionTime()) {
updatedPartitionIds.remove(partitionMeta.getId());
}
}
```
##########
fe/fe-core/src/main/java/org/apache/doris/datasource/doris/RemoteOlapTable.java:
##########
@@ -0,0 +1,129 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.datasource.doris;
+
+import org.apache.doris.catalog.Env;
+import org.apache.doris.catalog.OlapTable;
+import org.apache.doris.catalog.Partition;
+import org.apache.doris.catalog.Tablet;
+import org.apache.doris.common.AnalysisException;
+import org.apache.doris.system.Backend;
+
+import com.google.common.collect.ImmutableMap;
+import com.google.common.collect.Maps;
+import org.apache.logging.log4j.LogManager;
+import org.apache.logging.log4j.Logger;
+
+import java.lang.reflect.Field;
+import java.lang.reflect.Modifier;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.ConcurrentHashMap;
+
+public class RemoteOlapTable extends OlapTable {
+ private static final Logger LOG = LogManager.getLogger(RemoteOlapTable.class);
+
+ private RemoteDorisExternalCatalog catalog;
+ private RemoteDorisExternalDatabase database;
+
+ public RemoteDorisExternalCatalog getCatalog() {
+ return catalog;
+ }
+
+ public void setCatalog(RemoteDorisExternalCatalog catalog) {
+ this.catalog = catalog;
+ }
+
+ @Override
+ public RemoteDorisExternalDatabase getDatabase() {
+ return database;
+ }
+
+ public void setDatabase(RemoteDorisExternalDatabase database) {
+ this.database = database;
+ }
+
+ public static RemoteOlapTable fromOlapTable(OlapTable olapTable) {
+ try {
+ RemoteOlapTable externalOlapTable = new RemoteOlapTable();
+ Class<?> currentClass = olapTable.getClass();
+ while (currentClass != null) {
+ for (Field field : currentClass.getDeclaredFields()) {
+ if (Modifier.isStatic(field.getModifiers())) {
+ continue;
+ }
+ field.setAccessible(true);
+ field.set(externalOlapTable, field.get(olapTable));
+ }
+ currentClass = currentClass.getSuperclass();
+ }
+ return externalOlapTable;
+ } catch (Exception e) {
+ throw new RuntimeException("failed to initial external olap table", e);
+ }
Review Comment:
Security risk: Using reflection with `setAccessible(true)` on all fields bypasses Java's access control and could expose private fields. This approach is fragile and may break with future Java versions. Consider implementing a proper copy constructor or builder pattern instead of using reflection.
```suggestion
return new RemoteOlapTable(olapTable);
}
/**
* Copy constructor to create a RemoteOlapTable from an OlapTable.
* Only copies fields that are accessible and relevant.
*/
public RemoteOlapTable(OlapTable olapTable) {
super(olapTable.getId(), olapTable.getName(), olapTable.getBaseSchema(), olapTable.getKeysType(),
olapTable.getPartitionInfo(), olapTable.getDistributionInfo());
// Copy other relevant fields as needed
this.idToPartition = new ConcurrentHashMap<>(olapTable.getIdToPartition());
this.nameToPartition = Maps.newTreeMap();
for (Partition partition : this.idToPartition.values()) {
this.nameToPartition.put(partition.getName(), partition);
}
this.indexIdToMeta = Maps.newHashMap(olapTable.getIndexIdToMeta());
this.visibleIndexId = olapTable.getVisibleIndexId();
this.state = olapTable.getState();
this.createTime = olapTable.getCreateTime();
this.lastSchemaUpdateTime = olapTable.getLastSchemaUpdateTime();
this.lastUpdateTime = olapTable.getLastUpdateTime();
this.comment = olapTable.getComment();
this.baseIndexId = olapTable.getBaseIndexId();
this.colUniqueIdGenerator = olapTable.getColUniqueIdGenerator();
this.indexNameToId = Maps.newHashMap(olapTable.getIndexNameToId());
this.partitionColumnNames = olapTable.getPartitionColumnNames() != null
? List.copyOf(olapTable.getPartitionColumnNames()) : null;
this.defaultDistributionBucketNum = olapTable.getDefaultDistributionBucketNum();
this.storageType = olapTable.getStorageType();
this.storageFormat = olapTable.getStorageFormat();
this.enablePersistentIndex = olapTable.enablePersistentIndex();
this.enableUniqueKeyMergeOnWrite = olapTable.enableUniqueKeyMergeOnWrite();
this.enableSingleReplicaCompaction = olapTable.enableSingleReplicaCompaction();
this.enablePersistentIndex = olapTable.enablePersistentIndex();
this.enablePersistentIndex = olapTable.enablePersistentIndex();
// Add more fields as necessary, depending on what needs to be copied.
```
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/distribute/worker/BackendDistributedPlanWorkerManager.java:
##########
@@ -115,28 +122,38 @@ private ImmutableMap<Long, Backend> checkAndInitClusterBackends(
}
@Override
- public DistributedPlanWorker getWorker(long backendId) {
- ImmutableMap<Long, Backend> backends = this.allClusterBackends.get();
+ public void addBackends(long catalogId, ImmutableMap<Long, Backend> backends) {
+ if (!currentClusterBackends.containsKey(catalogId)) {
+ currentClusterBackends.put(catalogId, backends);
+ }
+ if (!allClusterBackends.containsKey(catalogId)) {
+ allClusterBackends.put(catalogId, Suppliers.ofInstance(backends));
+ }
+ }
+
+ @Override
+ public DistributedPlanWorker getWorker(long catalogId, long backendId) {
+ ImmutableMap<Long, Backend> backends = this.allClusterBackends.get(catalogId).get();
Backend backend = backends.get(backendId);
if (backend == null) {
throw new IllegalStateException("Backend " + backendId + " is not exist");
}
- return new BackendWorker(backend);
+ return new BackendWorker(catalogId, backend);
}
@Override
- public DistributedPlanWorker getWorker(Backend backend) {
- return new BackendWorker(backend);
+ public DistributedPlanWorker getWorker(long catalogId, Backend backend) {
+ return new BackendWorker(catalogId, backend);
}
@Override
- public DistributedPlanWorker randomAvailableWorker() {
+ public DistributedPlanWorker randomAvailableWorker(long catalogId) {
try {
Reference<Long> selectedBackendId = new Reference<>();
- ImmutableMap<Long, Backend> backends = this.currentClusterBackends;
+ ImmutableMap<Long, Backend> backends = this.currentClusterBackends.get(catalogId);
Review Comment:
Potential NullPointerException: If `catalogId` is not present in `currentClusterBackends`, calling `get(catalogId)` will return null, and then calling `SimpleScheduler.getHost()` with null will likely cause issues. Add a null check or ensure the catalog exists before calling this method.
```suggestion
ImmutableMap<Long, Backend> backends = this.currentClusterBackends.get(catalogId);
if (backends == null) {
throw new IllegalStateException("No backends found for catalogId: " + catalogId);
}
```
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/analysis/BindRelation.java:
##########
@@ -470,7 +471,21 @@ private LogicalPlan getLogicalPlan(TableIf table, UnboundRelation unboundRelatio
case MAX_COMPUTE_EXTERNAL_TABLE:
case TRINO_CONNECTOR_EXTERNAL_TABLE:
case LAKESOUl_EXTERNAL_TABLE:
+ return new LogicalFileScan(unboundRelation.getRelationId(), (ExternalTable) table,
+ qualifierWithoutTableName, ImmutableList.of(),
+ unboundRelation.getTableSample(),
+ unboundRelation.getTableSnapshot(),
+ Optional.ofNullable(unboundRelation.getScanParams()), Optional.empty());
case DORIS_EXTERNAL_TABLE:
+ ConnectContext ctx = cascadesContext.getConnectContext();
+ if (Config.isNotCloudMode() && ctx != null
+ && ctx.getSessionVariable().isQueryDorisExternalTableAsOlapTable()
+ // use isEnableNereidsDistributePlanner instead of canUseNereidsDistributePlanner
+ // bacause it cannot work in explain command
Review Comment:
Typo in comment: "bacause" should be "because".
```suggestion
// because it cannot work in explain command
```
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/distribute/worker/BackendDistributedPlanWorkerManager.java:
##########
@@ -149,17 +166,17 @@ public long randomAvailableWorker(Map<TNetworkAddress, Long> addressToBackendID)
}
@Override
- public List<Backend> getAllBackends(boolean needAlive) {
+ public List<Backend> getAllBackends(long catalogId, boolean needAlive) {
List<Backend> backends = null;
if (needAlive) {
backends = Lists.newArrayList();
- for (Map.Entry<Long, Backend> entry : this.allClusterBackends.get().entrySet()) {
+ for (Map.Entry<Long, Backend> entry : this.allClusterBackends.get(catalogId).get().entrySet()) {
Review Comment:
Potential NullPointerException: Similar to line 136, if `catalogId` is not present in `allClusterBackends`, this will throw a NullPointerException. Add proper null checking or validation.
##########
fe/fe-core/src/main/java/org/apache/doris/datasource/doris/RemoteOlapTable.java:
##########
@@ -0,0 +1,129 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.datasource.doris;
+
+import org.apache.doris.catalog.Env;
+import org.apache.doris.catalog.OlapTable;
+import org.apache.doris.catalog.Partition;
+import org.apache.doris.catalog.Tablet;
+import org.apache.doris.common.AnalysisException;
+import org.apache.doris.system.Backend;
+
+import com.google.common.collect.ImmutableMap;
+import com.google.common.collect.Maps;
+import org.apache.logging.log4j.LogManager;
+import org.apache.logging.log4j.Logger;
+
+import java.lang.reflect.Field;
+import java.lang.reflect.Modifier;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.ConcurrentHashMap;
+
+public class RemoteOlapTable extends OlapTable {
+ private static final Logger LOG = LogManager.getLogger(RemoteOlapTable.class);
+
+ private RemoteDorisExternalCatalog catalog;
+ private RemoteDorisExternalDatabase database;
+
+ public RemoteDorisExternalCatalog getCatalog() {
+ return catalog;
+ }
+
+ public void setCatalog(RemoteDorisExternalCatalog catalog) {
+ this.catalog = catalog;
+ }
+
+ @Override
+ public RemoteDorisExternalDatabase getDatabase() {
+ return database;
+ }
+
+ public void setDatabase(RemoteDorisExternalDatabase database) {
+ this.database = database;
+ }
+
+ public static RemoteOlapTable fromOlapTable(OlapTable olapTable) {
+ try {
+ RemoteOlapTable externalOlapTable = new RemoteOlapTable();
+ Class<?> currentClass = olapTable.getClass();
+ while (currentClass != null) {
+ for (Field field : currentClass.getDeclaredFields()) {
+ if (Modifier.isStatic(field.getModifiers())) {
+ continue;
+ }
+ field.setAccessible(true);
+ field.set(externalOlapTable, field.get(olapTable));
+ }
+ currentClass = currentClass.getSuperclass();
+ }
+ return externalOlapTable;
+ } catch (Exception e) {
+ throw new RuntimeException("failed to initial external olap table", e);
+ }
+ }
+
+ public void rebuildPartitions(List<Partition> oldPartitions, List<Partition> updatedPartitions,
+ List<Long> removedPartitions)
+ throws AnalysisException {
+ if (LOG.isDebugEnabled()) {
+ LOG.debug("rebuildPartitions oldPartitions: " + oldPartitions.size() + ", updatedPartitions: "
+ + updatedPartitions.size() + ", removedPartitions: " + removedPartitions.size());
+ }
+ ConcurrentHashMap<Long, Partition> newIdToPartition = new ConcurrentHashMap<>();
+ for (Partition oldPartition : oldPartitions) {
+ newIdToPartition.put(oldPartition.getId(), oldPartition);
+ }
+ for (Long removedPartition : removedPartitions) {
+ newIdToPartition.remove(removedPartition);
+ }
+ for (Partition updatedPartition : updatedPartitions) {
+ newIdToPartition.put(updatedPartition.getId(), updatedPartition);
+ }
+ Map<String, Partition> newNameToPartition = Maps.newTreeMap();
+ for (Partition partition : newIdToPartition.values()) {
+ newNameToPartition.put(partition.getName(), partition);
+ }
+ this.idToPartition = newIdToPartition;
+ this.nameToPartition = newNameToPartition;
+ }
+
+ public void invalidateBackendsIfNeed() {
+ ImmutableMap<Long, Backend> backends = getAllBackendsByAllCluster();
+ for (Partition partition : getPartitions()) {
+ for (Tablet tablet : partition.getBaseIndex().getTablets()) {
+ for (long backendId : tablet.getBackendIds()) {
+ if (!backends.containsKey(backendId)) {
+ Env.getCurrentEnv().getExtMetaCacheMgr().getDorisExternalMetaCacheMgr()
+ .invalidateBackendCache(catalog.getId());
+ return;
+ }
+ }
+ }
+ }
+ }
+
+ @Override
+ public long getCatalogId() {
+ return catalog.getId();
+ }
+
+ public ImmutableMap<Long, Backend> getAllBackendsByAllCluster() {
Review Comment:
Potential NullPointerException: If `catalog` is null, calling `catalog.getId()` at line 123 or 127 will throw a NullPointerException. Ensure catalog is set before these methods are called, or add null checks with appropriate error handling.
```suggestion
public long getCatalogId() {
if (catalog == null) {
throw new IllegalStateException("Catalog is not set in RemoteOlapTable");
}
return catalog.getId();
}
public ImmutableMap<Long, Backend> getAllBackendsByAllCluster() {
if (catalog == null) {
throw new IllegalStateException("Catalog is not set in RemoteOlapTable");
}
```
##########
fe/fe-core/src/main/java/org/apache/doris/catalog/OlapTable.java:
##########
@@ -3698,4 +3699,44 @@ public Index getInvertedIndex(Column column, List<String> subPath) {
.filter(Index::isAnalyzedInvertedIndex).findFirst().orElse(null);
}
}
+
+ /**
+ * caller should acquire the read lock and should not modify any field of the return obj
+ */
+ public OlapTable copyTableMeta() {
+ OlapTable table = new OlapTable();
+ // metaobj
+ table.signature = signature;
+ table.lastCheckTime = lastCheckTime;
+ // abstract table
+ table.id = id;
+ table.name = name;
+ table.qualifiedDbName = qualifiedDbName;
+ table.type = type;
+ table.createTime = createTime;
+ // olap table
+ // NOTE: currently do not need temp partitions, colocateGroup, autoIncrementGenerator
+ table.idToPartition = new ConcurrentHashMap<>();
+ table.tempPartitions = new TempPartitions();
+
+ table.state = state;
+ table.indexIdToMeta = indexIdToMeta;
+ table.indexNameToId = indexNameToId;
+ table.keysType = keysType;
+ table.partitionInfo = partitionInfo;
+ table.defaultDistributionInfo = defaultDistributionInfo;
+ table.bfColumns = bfColumns;
+ table.bfFpp = bfFpp;
+ table.indexes = indexes;
+ table.baseIndexId = baseIndexId;
+ return table;
Review Comment:
[nitpick] Incomplete shallow copy: The `copyTableMeta()` method performs a shallow copy of reference fields like `indexIdToMeta`, `indexNameToId`, `partitionInfo`, etc. (lines 3723-3731). This means modifications to these shared objects in the copy will affect the original table. Since the JavaDoc states "caller should not modify any field of the return obj", consider making this explicit with defensive copies or using immutable wrappers to prevent accidental modifications.
##########
fe/fe-core/src/main/java/org/apache/doris/datasource/doris/FeServiceClient.java:
##########
@@ -0,0 +1,278 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.datasource.doris;
+
+import org.apache.doris.catalog.OlapTable;
+import org.apache.doris.catalog.Partition;
+import org.apache.doris.common.ClientPool;
+import org.apache.doris.common.FeConstants;
+import org.apache.doris.common.UserException;
+import org.apache.doris.common.io.Text;
+import org.apache.doris.persist.gson.GsonUtils;
+import org.apache.doris.system.Backend;
+import org.apache.doris.thrift.FrontendService;
+import org.apache.doris.thrift.TGetBackendMetaRequest;
+import org.apache.doris.thrift.TGetBackendMetaResult;
+import org.apache.doris.thrift.TGetOlapTableMetaRequest;
+import org.apache.doris.thrift.TGetOlapTableMetaResult;
+import org.apache.doris.thrift.TNetworkAddress;
+import org.apache.doris.thrift.TPartitionMeta;
+import org.apache.doris.thrift.TStatusCode;
+
+import org.apache.logging.log4j.LogManager;
+import org.apache.logging.log4j.Logger;
+import org.apache.thrift.TException;
+
+import java.io.ByteArrayInputStream;
+import java.io.DataInputStream;
+import java.io.IOException;
+import java.nio.ByteBuffer;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Random;
+import java.util.stream.Collectors;
+
+public class FeServiceClient {
+ private static final Logger LOG = LogManager.getLogger(FeServiceClient.class);
+
+ private final Random random = new Random(System.currentTimeMillis());
+ private final String name;
+ private final List<TNetworkAddress> addresses;
+ private volatile TNetworkAddress master;
+ private final String user;
+ private final String password;
+ private final int retryCount;
+ private final int timeout;
+
+ public FeServiceClient(String name, List<TNetworkAddress> addresses, String user, String password,
+ int retryCount, int timeout) {
+ this.name = name;
+ this.addresses = addresses;
+ this.user = user;
+ this.password = password;
+ this.retryCount = retryCount;
+ this.timeout = timeout;
+ }
+
+ private List<TNetworkAddress> getAddresses() {
+ return addresses;
+ }
+
+ private FrontendService.Client getRemoteFeClient(TNetworkAddress address, int timeout) {
+ try {
+ return ClientPool.frontendPool.borrowObject(address, timeout);
+ } catch (Exception e) {
+ String msg = String.format("failed to get remote doris:%s fe connection", name);
+ throw new RuntimeException(msg, e);
+ }
+ }
+
+ private void returnClient(TNetworkAddress address, FrontendService.Client client, boolean returnObj) {
+ if (returnObj) {
+ ClientPool.frontendPool.returnObject(address, client);
+ } else {
+ ClientPool.frontendPool.invalidateObject(address, client);
+ }
+ }
+
+ private <T> T randomCallWithRetry(ThriftCall<T> call, String errorMsg, int timeout) {
+ List<TNetworkAddress> addresses = getAddresses();
+ int retries = 0;
+ Exception lastException = null;
+ while (retries < retryCount) {
+ int index = random.nextInt(addresses.size());
+ FrontendService.Client client = null;
+ for (int i = 0; i < addresses.size() && retries < retryCount; i++) {
+ TNetworkAddress address = addresses.get((index + i) % addresses.size());
+ client = getRemoteFeClient(address, timeout);
+ boolean returnObj = false;
+ try {
+ T result = call.call(client);
+ returnObj = true;
+ return result;
+ } catch (TException | IOException e) {
+ lastException = e;
+ retries++;
+ } catch (Exception e) {
+ throw new RuntimeException(errorMsg + ":" + e.getMessage(), e);
+ } finally {
+ returnClient(address, client, returnObj);
+ }
+ }
+ }
+ throw new RuntimeException(errorMsg + ":" + lastException.getMessage(), lastException);
+ }
+
+ private <T> T callFromMaster(ThriftCall<MasterResult<T>> call, String errorMsg, int timeout) {
+ TNetworkAddress address = master;
+ FrontendService.Client client = null;
+ Exception lastException = null;
+ if (address != null) {
+ client = getRemoteFeClient(address, timeout);
+ boolean returnObj = false;
+ try {
+ MasterResult<T> ret = call.call(client);
+ returnObj = true;
+ if (ret.isMaster) {
+ if (ret.hasError) {
+ throw new RuntimeException(ret.errorMsg);
+ }
+ return ret.result;
+ }
+ } catch (TException | IOException e) {
+ lastException = e;
+ } catch (Exception e) {
+ throw new RuntimeException(errorMsg + ":" + e.getMessage(), e);
+ } finally {
+ returnClient(address, client, returnObj);
+ }
+ }
+ master = null;
+ List<TNetworkAddress> addresses = getAddresses();
+ int retries = 0;
+ while (retries < retryCount) {
+ int index = random.nextInt(addresses.size());
+ for (int i = 0; i < addresses.size() && retries < retryCount; i++) {
+ address = addresses.get((index + i) % addresses.size());
+ client = getRemoteFeClient(address, timeout);
+ boolean returnObj = false;
+ try {
+ MasterResult<T> ret = call.call(client);
+ returnObj = true;
+ if (ret.isMaster) {
+ master = address;
+ if (ret.hasError) {
+ throw new RuntimeException(ret.errorMsg);
+ }
+ return ret.result;
+ }
+ } catch (TException | IOException e) {
+ lastException = e;
+ retries++;
+ } catch (Exception e) {
+ throw new RuntimeException(errorMsg + ":" + e.getMessage(), e);
+ } finally {
+ returnClient(address, client, returnObj);
+ }
+ }
+ }
+ throw new RuntimeException(errorMsg + ":" + lastException.getMessage(), lastException);
+ }
+
+ public List<Backend> listBackends() {
+ TGetBackendMetaRequest request = new TGetBackendMetaRequest();
+ request.setUser(user);
+ request.setPasswd(password);
+ String msg = String.format("failed to get backends from remote doris:%s", name);
+ return callFromMaster(client -> {
+ TGetBackendMetaResult result = client.getBackendMeta(request);
+ if (result.getStatus().getStatusCode() == TStatusCode.NOT_MASTER) {
+ return MasterResult.notMaster();
+ }
+ if (result.getStatus().getStatusCode() != TStatusCode.OK) {
+ return MasterResult.masterWithError(result.getStatus().toString());
+ }
+ List<Backend> backends = result.getBackends().stream()
+ .map(b -> Backend.fromThrift(b))
+ .collect(Collectors.toList());
+ return MasterResult.withResult(backends);
+ }, msg, timeout);
+ }
+
+ public RemoteOlapTable getOlapTable(String dbName, String table, long tableId, List<Partition> partitions) {
+ TGetOlapTableMetaRequest request = new TGetOlapTableMetaRequest();
+ request.setDb(dbName);
+ request.setTable(table);
+ request.setTableId(tableId);
+ request.setUser(user);
+ request.setPasswd(password);
+ request.setVersion(FeConstants.meta_version);
+ for (Partition partition : partitions) {
+ TPartitionMeta meta = new TPartitionMeta();
+ meta.setId(partition.getId());
+ meta.setVisibleVersion(partition.getVisibleVersion());
+ meta.setVisibleVersionTime(partition.getVisibleVersionTime());
+ request.addToPartitions(meta);
+ }
+ String msg = String.format("failed to get table meta from remote doris:%s", name);
+ return randomCallWithRetry(client -> {
+ TGetOlapTableMetaResult result = client.getOlapTableMeta(request);
+ if (result.getStatus().getStatusCode() != TStatusCode.OK) {
+ throw new UserException(result.getStatus().toString());
+ }
+ RemoteOlapTable remoteOlapTable = null;
+ try (DataInputStream in = new DataInputStream(new ByteArrayInputStream(result.getTableMeta()))) {
+ OlapTable olapTable = OlapTable.read(in);
+ remoteOlapTable = RemoteOlapTable.fromOlapTable(olapTable);
+ }
+ List<Partition> updatedPartitions = new ArrayList<>(result.getUpdatedPartitionsSize());
+ if (result.getUpdatedPartitionsSize() > 0) {
+ for (ByteBuffer buffer : result.getUpdatedPartitions()) {
+ try (ByteArrayInputStream in =
+ new ByteArrayInputStream(buffer.array(), buffer.position(), buffer.remaining());
+ DataInputStream dataInputStream = new DataInputStream(in)) {
+ String partitionStr = Text.readString(dataInputStream);
+ Partition partition = GsonUtils.GSON.fromJson(partitionStr, Partition.class);
+ updatedPartitions.add(partition);
+ }
+ }
+ }
+ List<Long> removedPartitions = result.getRemovedPartitions();
+ if (removedPartitions == null) {
+ removedPartitions = new ArrayList<>();
+ }
+ remoteOlapTable.rebuildPartitions(partitions, updatedPartitions, removedPartitions);
+ return remoteOlapTable;
+ }, msg, timeout);
+ }
+
+ private interface ThriftCall<T> {
+ public T call(FrontendService.Client client) throws Exception;
+ }
+
+ private static class MasterResult<T> {
+ boolean isMaster = true;
+ T result;
+ boolean hasError = false;
+ String errorMsg;
+
+ static <T> MasterResult<T> notMaster() {
+ MasterResult ret = new MasterResult();
+ ret.isMaster = false;
+ return ret;
+ }
+
+ static <T> MasterResult<T> withResult(T result) {
+ MasterResult ret = new MasterResult();
+ ret.isMaster = true;
+ ret.hasError = false;
+ ret.result = result;
+ return ret;
+ }
+
+ // is master but has error code
+ static <T> MasterResult<T> masterWithError(String errorMsg) {
+ MasterResult ret = new MasterResult();
+ ret.isMaster = true;
+ ret.hasError = true;
+ ret.errorMsg = errorMsg;
+ return ret;
+ }
Review Comment:
Missing generic type parameter: The `MasterResult` instantiation is missing the type parameter. This causes a raw type warning and potential type safety issues. Change `MasterResult ret = new MasterResult();` to `MasterResult<T> ret = new MasterResult<>();` for all three factory methods.
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/distribute/DistributePlanner.java:
##########
@@ -119,6 +122,17 @@ public FragmentIdMapping<DistributedPlan> plan() {
}
}
+ private void addExternalBackends(BackendDistributedPlanWorkerManager workerManager) throws AnalysisException {
+ for (PlanFragment planFragment : idToFragments.values()) {
+ List<OlapScanNode> scanNodes = planFragment.getPlanRoot()
+ .collectInCurrentFragment(OlapScanNode.class::isInstance);
+ for (OlapScanNode scanNode : scanNodes) {
+ workerManager.addBackends(scanNode.getCatalogId(),
+ scanNode.getOlapTable().getAllBackendsByAllCluster());
Review Comment:
[nitpick] Missing error handling: If `scanNode.getOlapTable().getAllBackendsByAllCluster()` throws an `AnalysisException` (as declared in the method signature at line 125), it will propagate up. However, there's no specific handling or logging for this exception within the method. Consider adding try-catch around line 130-131 to provide more specific error messages about which scan node or table caused the issue.
```suggestion
try {
workerManager.addBackends(scanNode.getCatalogId(),
scanNode.getOlapTable().getAllBackendsByAllCluster());
} catch (AnalysisException e) {
LOG.error("Failed to get all backends for scan node. "
+ "PlanFragmentId: {}, ScanNode: {}, Table: {}",
planFragment.getFragmentId(),
scanNode.getId(),
scanNode.getOlapTable() != null ? scanNode.getOlapTable().getName() : "null",
e);
throw e;
}
```
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/distribute/SelectedWorkers.java:
##########
@@ -48,19 +49,28 @@ public void onCreateAssignedJob(AssignedJob assignedJob) {
BackendWorker worker = (BackendWorker) assignedJob.getAssignedWorker();
if (usedWorkers.add(worker)) {
Backend backend = worker.getBackend();
- usedWorkersAddressToBackendID.put(
+ usedWorkersAddressToBackendID.computeIfAbsent(worker.getCatalogId(), k -> Maps.newLinkedHashMap());
+ usedWorkersAddressToBackendID.get(worker.getCatalogId()).put(
new TNetworkAddress(backend.getHost(), backend.getBePort()), backend.getId()
);
}
}
/** tryToSelectRandomUsedWorker */
public DistributedPlanWorker tryToSelectRandomUsedWorker() {
+ long catalogId = Env.getCurrentInternalCatalog().getId();
if (usedWorkers.isEmpty()) {
- return workerManager.randomAvailableWorker();
+ return workerManager.randomAvailableWorker(catalogId);
} else {
- long id = workerManager.randomAvailableWorker(usedWorkersAddressToBackendID);
- return workerManager.getWorker(id);
+ Map<TNetworkAddress, Long> backendIDs;
+ if (usedWorkersAddressToBackendID.containsKey(catalogId)) {
+ backendIDs = usedWorkersAddressToBackendID.get(catalogId);
+ } else {
+ catalogId = usedWorkers.iterator().next().getCatalogId();
+ backendIDs = usedWorkersAddressToBackendID.get(catalogId);
+ }
Review Comment:
Potential NullPointerException: On line 70, `usedWorkersAddressToBackendID.get(catalogId)` could return null if the catalog ID is not present in the map. This would cause a NullPointerException when passed to `workerManager.randomAvailableWorker()` on line 72. Add a null check or ensure the catalog exists in the map.
```suggestion
}
if (backendIDs == null || backendIDs.isEmpty()) {
// Fallback: select a random available worker for the catalogId
return workerManager.randomAvailableWorker(catalogId);
}
```
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/distribute/worker/BackendDistributedPlanWorkerManager.java:
##########
@@ -115,28 +122,38 @@ private ImmutableMap<Long, Backend> checkAndInitClusterBackends(
}
@Override
- public DistributedPlanWorker getWorker(long backendId) {
- ImmutableMap<Long, Backend> backends = this.allClusterBackends.get();
+ public void addBackends(long catalogId, ImmutableMap<Long, Backend> backends) {
+ if (!currentClusterBackends.containsKey(catalogId)) {
+ currentClusterBackends.put(catalogId, backends);
+ }
+ if (!allClusterBackends.containsKey(catalogId)) {
+ allClusterBackends.put(catalogId, Suppliers.ofInstance(backends));
+ }
+ }
+
+ @Override
+ public DistributedPlanWorker getWorker(long catalogId, long backendId) {
+ ImmutableMap<Long, Backend> backends = this.allClusterBackends.get(catalogId).get();
Review Comment:
Potential NullPointerException: If `catalogId` is not present in `allClusterBackends`, calling `get(catalogId)` will return null, and then calling `get()` on the null Supplier will cause a NullPointerException. Add a null check before accessing the backends.
```suggestion
Supplier<ImmutableMap<Long, Backend>> backendSupplier = this.allClusterBackends.get(catalogId);
if (backendSupplier == null) {
throw new IllegalStateException("Catalog " + catalogId + " is not present in allClusterBackends");
}
ImmutableMap<Long, Backend> backends = backendSupplier.get();
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature] doris cross-cluster query [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #57898:
URL: https://github.com/apache/doris/pull/57898#issuecomment-3588402293
# Cloud UT Coverage Report
Increment line coverage ` ` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/90747d876a8698c180f8bf9286fdb09c5aa33a8b_90747d876a8698c180f8bf9286fdb09c5aa33a8b_cloud/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/90747d876a8698c180f8bf9286fdb09c5aa33a8b_90747d876a8698c180f8bf9286fdb09c5aa33a8b_cloud/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 80.86% (1707/2111) |
| Line Coverage | 66.89% (29906/44708) |
| Region Coverage | 67.40% (14904/22114) |
| Branch Coverage | 57.63% (7934/13768) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature] doris cross-cluster query [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #57898:
URL: https://github.com/apache/doris/pull/57898#issuecomment-3590991610
# BE Regression && UT Coverage Report
Increment line coverage `100% (0/0)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/57898_6e75d38a8a85d1bc4670594b3bf7cdd32ba2ee3c_merge/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/57898_6e75d38a8a85d1bc4670594b3bf7cdd32ba2ee3c_merge/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 71.64% (24579/34307) |
| Line Coverage | 58.16% (256359/440751) |
| Region Coverage | 53.19% (213243/400909) |
| Branch Coverage | 54.74% (91246/166692) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature] doris cross-cluster query [doris]
Posted by "HonestManXin (via GitHub)" <gi...@apache.org>.
HonestManXin commented on PR #57898:
URL: https://github.com/apache/doris/pull/57898#issuecomment-3590900445
run nonConcurrent
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature] doris cross-cluster query [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #57898:
URL: https://github.com/apache/doris/pull/57898#issuecomment-3592382703
# BE Regression && UT Coverage Report
Increment line coverage `100% (0/0)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/57898_2019c7c87f06a8cb801b7ced0d007f2f7784b3be_merge/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/57898_2019c7c87f06a8cb801b7ced0d007f2f7784b3be_merge/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 71.67% (24593/34313) |
| Line Coverage | 58.21% (256590/440793) |
| Region Coverage | 53.28% (213633/400964) |
| Branch Coverage | 54.83% (91400/166708) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature] doris cross-cluster query [doris]
Posted by "HonestManXin (via GitHub)" <gi...@apache.org>.
HonestManXin commented on PR #57898:
URL: https://github.com/apache/doris/pull/57898#issuecomment-3573974203
run feut
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature] doris cross-cluster query [doris]
Posted by "HonestManXin (via GitHub)" <gi...@apache.org>.
HonestManXin commented on PR #57898:
URL: https://github.com/apache/doris/pull/57898#issuecomment-3584524167
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature] doris cross-cluster query [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #57898:
URL: https://github.com/apache/doris/pull/57898#issuecomment-3586669499
# BE Regression && UT Coverage Report
Increment line coverage `100% (0/0)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/57898_2f096957f7645a9b79d50dcff93a47a5dc47aac8_merge/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/57898_2f096957f7645a9b79d50dcff93a47a5dc47aac8_merge/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 71.64% (24565/34291) |
| Line Coverage | 58.16% (256133/440425) |
| Region Coverage | 53.23% (213248/400642) |
| Branch Coverage | 54.73% (91159/166572) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature] doris cross-cluster query [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #57898:
URL: https://github.com/apache/doris/pull/57898#issuecomment-3588784794
<details>
<summary>TPC-H: <b>Total hot run time: 34226 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 6e75d38a8a85d1bc4670594b3bf7cdd32ba2ee3c, data reload: false
------ Round 1 ----------------------------------
q1 17621 4949 4884 4884
q2 2011 315 205 205
q3 10218 1334 728 728
q4 10230 869 370 370
q5 7521 2500 2267 2267
q6 186 168 140 140
q7 960 785 637 637
q8 9342 1344 1073 1073
q9 7127 5269 5296 5269
q10 6898 2202 1845 1845
q11 526 304 294 294
q12 381 375 218 218
q13 17810 3647 3011 3011
q14 227 238 217 217
q15 600 527 507 507
q16 870 853 800 800
q17 583 745 515 515
q18 7712 7033 7087 7033
q19 1092 969 578 578
q20 338 341 227 227
q21 3585 3115 2460 2460
q22 1031 990 948 948
Total cold run time: 106869 ms
Total hot run time: 34226 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4935 4870 4885 4870
q2 339 416 315 315
q3 2109 2669 2300 2300
q4 1306 1718 1281 1281
q5 4276 4310 4566 4310
q6 211 171 126 126
q7 2150 2044 1787 1787
q8 2736 2393 2521 2393
q9 7631 7456 7525 7456
q10 3221 3342 2806 2806
q11 584 509 493 493
q12 705 747 649 649
q13 3575 3911 3269 3269
q14 297 301 306 301
q15 554 518 500 500
q16 889 912 880 880
q17 1135 1443 1416 1416
q18 7751 7707 7610 7610
q19 782 771 792 771
q20 1996 2012 1805 1805
q21 4708 4276 4132 4132
q22 1096 1019 971 971
Total cold run time: 52986 ms
Total hot run time: 50441 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature] doris cross-cluster query [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #57898:
URL: https://github.com/apache/doris/pull/57898#issuecomment-3568149142
<details>
<summary>ClickBench: <b>Total hot run time: 28.17 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 4b7d8a7a3ce6da25dd8a85bbb9a335b8df6d8b86, data reload: false
query1 0.06 0.05 0.05
query2 0.12 0.07 0.06
query3 0.29 0.08 0.07
query4 1.61 0.09 0.09
query5 0.27 0.25 0.24
query6 1.17 0.66 0.66
query7 0.03 0.02 0.03
query8 0.07 0.06 0.07
query9 0.63 0.53 0.52
query10 0.59 0.58 0.58
query11 0.25 0.13 0.13
query12 0.25 0.14 0.14
query13 0.65 0.64 0.62
query14 1.02 1.01 1.02
query15 0.93 0.88 0.84
query16 0.39 0.38 0.38
query17 1.06 1.02 1.07
query18 0.23 0.21 0.22
query19 1.95 1.75 1.82
query20 0.02 0.01 0.02
query21 15.39 0.31 0.25
query22 4.94 0.10 0.10
query23 15.35 0.37 0.23
query24 2.92 0.48 0.31
query25 0.09 0.09 0.10
query26 0.18 0.17 0.17
query27 0.09 0.09 0.09
query28 3.64 1.27 1.07
query29 12.55 4.02 3.31
query30 0.33 0.13 0.11
query31 2.80 0.64 0.45
query32 3.23 0.60 0.50
query33 3.07 3.01 3.13
query34 16.75 5.17 4.52
query35 4.59 4.53 4.50
query36 0.63 0.52 0.50
query37 0.22 0.08 0.08
query38 0.19 0.06 0.05
query39 0.05 0.05 0.05
query40 0.20 0.17 0.17
query41 0.11 0.06 0.06
query42 0.06 0.04 0.04
query43 0.05 0.06 0.05
Total cold run time: 99.02 s
Total hot run time: 28.17 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature] doris cross-cluster query [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #57898:
URL: https://github.com/apache/doris/pull/57898#issuecomment-3569384017
# FE UT Coverage Report
Increment line coverage ` 13.60% (62/456)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/57898_639c5c970ba78c1055b2c830bd4c20729ffa601d/fe_increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/57898_639c5c970ba78c1055b2c830bd4c20729ffa601d/fe_report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature] doris cross-cluster query [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #57898:
URL: https://github.com/apache/doris/pull/57898#issuecomment-3571665022
<details>
<summary>TPC-DS: <b>Total hot run time: 184979 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 397ab88835eda5e584314fdc61afe29e014f140a, data reload: false
query1 1057 429 392 392
query2 6594 1606 1611 1606
query3 6758 228 224 224
query4 25795 23012 22655 22655
query5 4411 626 475 475
query6 350 258 236 236
query7 4651 498 311 311
query8 314 273 271 271
query9 8736 2586 2614 2586
query10 527 364 315 315
query11 14921 14880 14593 14593
query12 176 121 115 115
query13 1705 584 466 466
query14 10445 8808 8795 8795
query15 208 202 184 184
query16 7780 690 543 543
query17 1251 764 660 660
query18 2054 451 342 342
query19 222 203 183 183
query20 132 131 126 126
query21 224 140 118 118
query22 4173 4082 4071 4071
query23 33047 32156 32215 32156
query24 8521 2411 2394 2394
query25 644 565 523 523
query26 1273 280 160 160
query27 2763 502 364 364
query28 4390 2172 2159 2159
query29 794 628 489 489
query30 312 245 211 211
query31 849 734 638 638
query32 82 79 75 75
query33 583 375 322 322
query34 774 858 523 523
query35 801 831 743 743
query36 893 954 867 867
query37 122 107 85 85
query38 3411 3343 3348 3343
query39 1494 1406 1402 1402
query40 225 133 120 120
query41 64 61 61 61
query42 129 119 113 113
query43 456 474 439 439
query44 1231 763 755 755
query45 198 185 181 181
query46 880 989 651 651
query47 1710 1729 1669 1669
query48 392 409 324 324
query49 779 502 440 440
query50 652 695 451 451
query51 3867 3920 3913 3913
query52 116 119 105 105
query53 244 266 197 197
query54 307 296 283 283
query55 86 88 85 85
query56 327 314 327 314
query57 1166 1146 1093 1093
query58 274 280 272 272
query59 2359 2458 2343 2343
query60 354 346 324 324
query61 163 159 163 159
query62 759 721 672 672
query63 227 200 194 194
query64 4464 1205 928 928
query65 4053 3985 3956 3956
query66 1099 456 330 330
query67 15458 14999 14964 14964
query68 8464 990 648 648
query69 516 339 308 308
query70 1273 1225 1190 1190
query71 508 339 308 308
query72 6069 4937 4969 4937
query73 697 610 378 378
query74 8563 8552 8529 8529
query75 4040 3350 2785 2785
query76 3832 1114 693 693
query77 836 403 313 313
query78 9608 9804 8858 8858
query79 2008 863 611 611
query80 607 588 509 509
query81 512 269 231 231
query82 425 157 133 133
query83 264 259 256 256
query84 261 117 97 97
query85 896 485 438 438
query86 342 304 280 280
query87 3589 3567 3390 3390
query88 3528 2230 2227 2227
query89 391 343 287 287
query90 1952 223 218 218
query91 172 169 141 141
query92 84 67 65 65
query93 1137 1041 688 688
query94 737 431 347 347
query95 510 402 395 395
query96 511 565 286 286
query97 2917 3014 2880 2880
query98 248 213 209 209
query99 1370 1408 1287 1287
Total cold run time: 272143 ms
Total hot run time: 184979 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature] doris cross-cluster query [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #57898:
URL: https://github.com/apache/doris/pull/57898#issuecomment-3574322661
# FE Regression Coverage Report
Increment line coverage ` 64.50% (298/462)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/57898_113154a9292df06cdf5ac65f15d887c14e1a6f1d_merge_fe/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/57898_113154a9292df06cdf5ac65f15d887c14e1a6f1d_merge_fe/report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature] doris cross-cluster query [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #57898:
URL: https://github.com/apache/doris/pull/57898#issuecomment-3574447384
<details>
<summary>TPC-DS: <b>Total hot run time: 183622 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 113154a9292df06cdf5ac65f15d887c14e1a6f1d, data reload: false
query1 1069 400 398 398
query2 6566 1586 1561 1561
query3 6761 224 217 217
query4 25177 22578 22309 22309
query5 4435 629 468 468
query6 333 246 223 223
query7 4636 493 288 288
query8 307 252 241 241
query9 8713 2574 2603 2574
query10 517 356 314 314
query11 15454 14799 14592 14592
query12 188 121 115 115
query13 1705 577 452 452
query14 11163 8905 8836 8836
query15 212 200 186 186
query16 7687 669 495 495
query17 1251 776 628 628
query18 2044 439 352 352
query19 215 209 181 181
query20 136 129 122 122
query21 224 141 119 119
query22 3883 4060 3835 3835
query23 32832 32358 32183 32183
query24 8363 2463 2428 2428
query25 610 548 482 482
query26 1255 277 172 172
query27 2743 512 367 367
query28 4350 2164 2143 2143
query29 838 612 479 479
query30 303 249 205 205
query31 886 716 645 645
query32 87 79 73 73
query33 595 377 331 331
query34 804 870 550 550
query35 801 843 731 731
query36 910 957 856 856
query37 123 110 93 93
query38 3333 3323 3321 3321
query39 1505 1409 1413 1409
query40 223 125 120 120
query41 65 67 65 65
query42 125 110 111 110
query43 485 477 430 430
query44 1240 760 743 743
query45 196 193 188 188
query46 912 1008 640 640
query47 1717 1708 1631 1631
query48 386 423 343 343
query49 779 495 411 411
query50 650 683 424 424
query51 3853 4065 3910 3910
query52 111 114 102 102
query53 232 267 200 200
query54 301 291 272 272
query55 90 84 84 84
query56 308 375 324 324
query57 1134 1154 1072 1072
query58 290 271 264 264
query59 2366 2520 2358 2358
query60 350 332 340 332
query61 159 154 154 154
query62 770 724 658 658
query63 230 192 203 192
query64 4488 1192 891 891
query65 4038 3969 3945 3945
query66 1107 436 337 337
query67 15054 14723 14693 14693
query68 8650 896 620 620
query69 515 338 293 293
query70 1327 1290 1180 1180
query71 514 340 327 327
query72 5761 4910 4777 4777
query73 643 583 372 372
query74 8582 8709 8474 8474
query75 4033 3359 2832 2832
query76 3690 1132 695 695
query77 818 403 320 320
query78 9494 9725 8909 8909
query79 2026 852 609 609
query80 641 591 493 493
query81 494 271 236 236
query82 447 159 133 133
query83 264 265 252 252
query84 256 110 98 98
query85 922 487 435 435
query86 335 308 308 308
query87 3476 3460 3408 3408
query88 3339 2249 2247 2247
query89 380 322 287 287
query90 1988 217 216 216
query91 172 174 138 138
query92 87 72 59 59
query93 1167 995 664 664
query94 695 440 349 349
query95 495 404 412 404
query96 497 578 285 285
query97 2929 2953 2854 2854
query98 246 212 201 201
query99 1314 1386 1280 1280
Total cold run time: 270870 ms
Total hot run time: 183622 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature] doris cross-cluster query [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #57898:
URL: https://github.com/apache/doris/pull/57898#issuecomment-3568895105
<details>
<summary>TPC-DS: <b>Total hot run time: 187199 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 6ebf97d578357addf4ee3a69c8b6d2eeb8bfa59b, data reload: false
query1 1054 441 392 392
query2 6562 1685 1655 1655
query3 6757 231 219 219
query4 25784 23593 23207 23207
query5 4508 636 481 481
query6 336 231 214 214
query7 4644 501 301 301
query8 294 259 236 236
query9 8693 2564 2582 2564
query10 503 332 291 291
query11 15824 15132 15018 15018
query12 179 118 112 112
query13 1704 586 456 456
query14 10341 9229 9208 9208
query15 212 194 176 176
query16 7259 689 492 492
query17 1273 829 720 720
query18 1996 431 335 335
query19 215 211 184 184
query20 133 125 123 123
query21 218 140 119 119
query22 4055 4024 3941 3941
query23 33928 32976 33103 32976
query24 8462 2539 2396 2396
query25 637 577 516 516
query26 1245 277 166 166
query27 2747 528 352 352
query28 4377 2194 2183 2183
query29 887 688 549 549
query30 310 222 200 200
query31 900 817 734 734
query32 90 80 78 78
query33 600 386 345 345
query34 796 857 522 522
query35 795 884 736 736
query36 939 998 897 897
query37 124 109 89 89
query38 3620 3581 3447 3447
query39 1487 1409 1410 1409
query40 231 135 117 117
query41 63 60 63 60
query42 125 113 111 111
query43 484 490 460 460
query44 1238 760 755 755
query45 188 177 169 169
query46 883 994 644 644
query47 1759 1767 1725 1725
query48 398 414 344 344
query49 797 514 429 429
query50 662 333 265 265
query51 3870 3938 3853 3853
query52 111 112 103 103
query53 233 257 193 193
query54 314 285 272 272
query55 90 87 80 80
query56 306 309 299 299
query57 1186 1187 1106 1106
query58 277 274 270 270
query59 2486 2579 2448 2448
query60 367 378 344 344
query61 161 158 156 156
query62 799 695 648 648
query63 228 188 193 188
query64 4510 1108 854 854
query65 3994 3920 3906 3906
query66 1190 435 327 327
query67 15345 15040 14878 14878
query68 8318 858 635 635
query69 489 321 290 290
query70 1361 1282 1285 1282
query71 475 337 312 312
query72 5594 4980 4842 4842
query73 664 568 364 364
query74 9010 9064 8708 8708
query75 3907 3330 2860 2860
query76 3707 1137 727 727
query77 818 414 315 315
query78 9742 9668 8954 8954
query79 2217 875 613 613
query80 672 596 498 498
query81 520 259 226 226
query82 479 157 133 133
query83 288 265 247 247
query84 267 113 92 92
query85 972 502 447 447
query86 396 318 295 295
query87 3703 3714 3572 3572
query88 3786 2264 2232 2232
query89 385 324 294 294
query90 1897 228 218 218
query91 168 169 134 134
query92 86 70 74 70
query93 2034 901 519 519
query94 701 424 339 339
query95 415 331 324 324
query96 479 563 272 272
query97 2910 2987 2899 2899
query98 243 214 214 214
query99 1320 1385 1229 1229
Total cold run time: 274806 ms
Total hot run time: 187199 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature] doris cross-cluster query [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #57898:
URL: https://github.com/apache/doris/pull/57898#issuecomment-3576148230
<details>
<summary>ClickBench: <b>Total hot run time: 28.36 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 608564073977a55e9151e8a1eed087104db5aea4, data reload: false
query1 0.06 0.05 0.05
query2 0.12 0.06 0.07
query3 0.31 0.08 0.07
query4 1.60 0.08 0.09
query5 0.27 0.24 0.26
query6 1.18 0.67 0.65
query7 0.04 0.03 0.03
query8 0.08 0.06 0.05
query9 0.64 0.54 0.53
query10 0.59 0.58 0.58
query11 0.26 0.13 0.14
query12 0.26 0.14 0.15
query13 0.65 0.63 0.62
query14 1.01 1.03 1.02
query15 0.93 0.85 0.85
query16 0.38 0.38 0.38
query17 1.05 1.02 1.04
query18 0.23 0.21 0.22
query19 1.98 1.80 1.77
query20 0.02 0.01 0.02
query21 15.39 0.29 0.24
query22 4.99 0.10 0.10
query23 15.37 0.40 0.23
query24 2.84 0.52 0.33
query25 0.10 0.08 0.08
query26 0.18 0.18 0.18
query27 0.10 0.09 0.09
query28 3.77 1.27 1.08
query29 12.63 3.93 3.28
query30 0.33 0.13 0.11
query31 2.81 0.65 0.46
query32 3.25 0.61 0.50
query33 3.09 3.17 3.19
query34 16.38 5.19 4.47
query35 4.57 4.53 4.57
query36 0.64 0.52 0.51
query37 0.23 0.09 0.09
query38 0.19 0.06 0.05
query39 0.06 0.05 0.05
query40 0.21 0.18 0.16
query41 0.11 0.06 0.07
query42 0.07 0.05 0.05
query43 0.06 0.06 0.05
Total cold run time: 99.03 s
Total hot run time: 28.36 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature] doris cross-cluster query [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #57898:
URL: https://github.com/apache/doris/pull/57898#issuecomment-3578796785
# FE Regression Coverage Report
Increment line coverage ` 64.88% (303/467)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/57898_608564073977a55e9151e8a1eed087104db5aea4_merge_fe/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/57898_608564073977a55e9151e8a1eed087104db5aea4_merge_fe/report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature] doris cross-cluster query [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #57898:
URL: https://github.com/apache/doris/pull/57898#issuecomment-3578738108
# BE Regression && UT Coverage Report
Increment line coverage `100% (0/0)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/57898_608564073977a55e9151e8a1eed087104db5aea4_merge/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/57898_608564073977a55e9151e8a1eed087104db5aea4_merge/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 71.69% (24502/34180) |
| Line Coverage | 58.17% (255273/438825) |
| Region Coverage | 53.22% (212473/399249) |
| Branch Coverage | 54.84% (90985/165922) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature] doris cross-cluster query [doris]
Posted by "morningman (via GitHub)" <gi...@apache.org>.
morningman commented on code in PR #57898:
URL: https://github.com/apache/doris/pull/57898#discussion_r2563922185
##########
fe/fe-core/src/main/java/org/apache/doris/datasource/doris/DorisExternalMetaCacheMgr.java:
##########
@@ -0,0 +1,83 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.datasource.doris;
+
+import org.apache.doris.catalog.Env;
+import org.apache.doris.common.CacheFactory;
+import org.apache.doris.common.Config;
+import org.apache.doris.system.Backend;
+
+import com.github.benmanes.caffeine.cache.LoadingCache;
+import com.google.common.collect.ImmutableMap;
+import com.google.common.collect.Maps;
+import org.apache.logging.log4j.LogManager;
+import org.apache.logging.log4j.Logger;
+
+import java.util.List;
+import java.util.Map;
+import java.util.OptionalLong;
+import java.util.concurrent.ExecutorService;
+import java.util.stream.Collectors;
+
+public class DorisExternalMetaCacheMgr {
+ private static final Logger LOG = LogManager.getLogger(DorisExternalMetaCacheMgr.class);
+ private final LoadingCache<Long, ImmutableMap<Long, Backend>> backendsCache;
+
+ public DorisExternalMetaCacheMgr(ExecutorService executor) {
+ CacheFactory cacheFactory = new CacheFactory(
+ OptionalLong.of(Config.external_cache_expire_time_seconds_after_access),
+ OptionalLong.of(Config.external_cache_refresh_time_minutes * 60),
+ Config.max_external_table_cache_num,
Review Comment:
It is strange to use `max_external_table_cache_num` for `backendsCache`. Actually the cache size if "num of catalogs". And there won't be too many catalogs there. So I suggest to use a fixed small number is enough, like 10 or 20? Or we can just add a new config for this.
##########
fe/fe-core/src/main/java/org/apache/doris/datasource/doris/RemoteDorisExternalTable.java:
##########
@@ -50,6 +55,27 @@ protected synchronized void makeSureInitialized() {
}
}
+ private RemoteOlapTable getDorisOlapTable() {
+ // TODO if multi thread call this method, only call once
Review Comment:
So for new, we will get olap table from remote cluster in every query?
##########
fe/fe-core/src/main/java/org/apache/doris/catalog/OlapTable.java:
##########
@@ -3698,4 +3699,44 @@ public Index getInvertedIndex(Column column, List<String> subPath) {
.filter(Index::isAnalyzedInvertedIndex).findFirst().orElse(null);
}
}
+
+ /**
+ * caller should acquire the read lock and should not modify any field of the return obj
+ */
+ public OlapTable copyTableMeta() {
Review Comment:
This is very error-prone. If someone add new field in this class, he may forget to modify here.
##########
fe/fe-core/src/main/java/org/apache/doris/datasource/doris/DorisExternalMetaCacheMgr.java:
##########
@@ -0,0 +1,83 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.datasource.doris;
+
+import org.apache.doris.catalog.Env;
+import org.apache.doris.common.CacheFactory;
+import org.apache.doris.common.Config;
+import org.apache.doris.system.Backend;
+
+import com.github.benmanes.caffeine.cache.LoadingCache;
+import com.google.common.collect.ImmutableMap;
+import com.google.common.collect.Maps;
+import org.apache.logging.log4j.LogManager;
+import org.apache.logging.log4j.Logger;
+
+import java.util.List;
+import java.util.Map;
+import java.util.OptionalLong;
+import java.util.concurrent.ExecutorService;
+import java.util.stream.Collectors;
+
+public class DorisExternalMetaCacheMgr {
+ private static final Logger LOG = LogManager.getLogger(DorisExternalMetaCacheMgr.class);
+ private final LoadingCache<Long, ImmutableMap<Long, Backend>> backendsCache;
+
+ public DorisExternalMetaCacheMgr(ExecutorService executor) {
+ CacheFactory cacheFactory = new CacheFactory(
+ OptionalLong.of(Config.external_cache_expire_time_seconds_after_access),
+ OptionalLong.of(Config.external_cache_refresh_time_minutes * 60),
+ Config.max_external_table_cache_num,
+ true,
+ null);
+ backendsCache = cacheFactory.buildCache(key -> loadBackends(key), executor);
+ }
+
+ private ImmutableMap<Long, Backend> loadBackends(Long catalogId) {
+ RemoteDorisExternalCatalog catalog = (RemoteDorisExternalCatalog) Env.getCurrentEnv().getCatalogMgr()
+ .getCatalog(catalogId);
+ List<Backend> backends = catalog.getFeServiceClient().listBackends();
+ if (LOG.isDebugEnabled()) {
+ List<String> names = backends.stream().map(b -> b.getAddress()).collect(Collectors.toList());
+ LOG.debug("load backends:{} from:{}", String.join(",", names), catalog.getName());
Review Comment:
if (LOG.isDebugEnable()) {
xxxx
}
##########
fe/fe-core/src/main/java/org/apache/doris/datasource/doris/RemoteOlapTable.java:
##########
@@ -0,0 +1,137 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.datasource.doris;
+
+import org.apache.doris.catalog.Env;
+import org.apache.doris.catalog.OlapTable;
+import org.apache.doris.catalog.Partition;
+import org.apache.doris.catalog.Tablet;
+import org.apache.doris.common.AnalysisException;
+import org.apache.doris.system.Backend;
+
+import com.google.common.collect.ImmutableMap;
+import com.google.common.collect.Maps;
+import org.apache.logging.log4j.LogManager;
+import org.apache.logging.log4j.Logger;
+
+import java.lang.reflect.Field;
+import java.lang.reflect.Modifier;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.ConcurrentHashMap;
+
+public class RemoteOlapTable extends OlapTable {
+ private static final Logger LOG = LogManager.getLogger(RemoteOlapTable.class);
+
+ private RemoteDorisExternalCatalog catalog;
+ private RemoteDorisExternalDatabase database;
+ private ImmutableMap<Long, Backend> backends = ImmutableMap.of();
Review Comment:
I think this info should be in Catalog level? not table level
##########
fe/fe-core/src/main/java/org/apache/doris/catalog/OlapTable.java:
##########
@@ -3698,4 +3699,44 @@ public Index getInvertedIndex(Column column, List<String> subPath) {
.filter(Index::isAnalyzedInvertedIndex).findFirst().orElse(null);
}
}
+
+ /**
+ * caller should acquire the read lock and should not modify any field of the return obj
+ */
+ public OlapTable copyTableMeta() {
Review Comment:
Can we use `selectiveCopy()` method to do this. This method is used for backup.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature] doris cross-cluster query [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #57898:
URL: https://github.com/apache/doris/pull/57898#issuecomment-3573684979
# Cloud UT Coverage Report
Increment line coverage ` ` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/d95c3d37e4d8819c0053ca5bb16dc122db99e3ef_d95c3d37e4d8819c0053ca5bb16dc122db99e3ef_cloud/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/d95c3d37e4d8819c0053ca5bb16dc122db99e3ef_d95c3d37e4d8819c0053ca5bb16dc122db99e3ef_cloud/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 80.86% (1707/2111) |
| Line Coverage | 66.86% (29890/44708) |
| Region Coverage | 67.39% (14902/22114) |
| Branch Coverage | 57.58% (7927/13768) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature] doris cross-cluster query [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #57898:
URL: https://github.com/apache/doris/pull/57898#issuecomment-3573764975
# Cloud UT Coverage Report
Increment line coverage ` ` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/113154a9292df06cdf5ac65f15d887c14e1a6f1d_113154a9292df06cdf5ac65f15d887c14e1a6f1d_cloud/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/113154a9292df06cdf5ac65f15d887c14e1a6f1d_113154a9292df06cdf5ac65f15d887c14e1a6f1d_cloud/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 80.86% (1707/2111) |
| Line Coverage | 66.90% (29911/44708) |
| Region Coverage | 67.41% (14907/22114) |
| Branch Coverage | 57.58% (7927/13768) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature] doris cross-cluster query [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #57898:
URL: https://github.com/apache/doris/pull/57898#issuecomment-3576471096
# BE UT Coverage Report
Increment line coverage ` ` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/608564073977a55e9151e8a1eed087104db5aea4_608564073977a55e9151e8a1eed087104db5aea4/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/608564073977a55e9151e8a1eed087104db5aea4_608564073977a55e9151e8a1eed087104db5aea4/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 52.83% (18396/34821) |
| Line Coverage | 38.31% (168081/438683) |
| Region Coverage | 33.15% (130607/393955) |
| Branch Coverage | 34.05% (56153/164929) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature] doris cross-cluster query [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #57898:
URL: https://github.com/apache/doris/pull/57898#issuecomment-3569169068
# Cloud UT Coverage Report
Increment line coverage ` ` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/639c5c970ba78c1055b2c830bd4c20729ffa601d_639c5c970ba78c1055b2c830bd4c20729ffa601d_cloud/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/639c5c970ba78c1055b2c830bd4c20729ffa601d_639c5c970ba78c1055b2c830bd4c20729ffa601d_cloud/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 80.86% (1707/2111) |
| Line Coverage | 66.87% (29896/44708) |
| Region Coverage | 67.41% (14906/22114) |
| Branch Coverage | 57.63% (7934/13768) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature] doris cross-cluster query [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #57898:
URL: https://github.com/apache/doris/pull/57898#issuecomment-3568194421
# BE UT Coverage Report
Increment line coverage ` ` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/4b7d8a7a3ce6da25dd8a85bbb9a335b8df6d8b86_4b7d8a7a3ce6da25dd8a85bbb9a335b8df6d8b86/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/4b7d8a7a3ce6da25dd8a85bbb9a335b8df6d8b86_4b7d8a7a3ce6da25dd8a85bbb9a335b8df6d8b86/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 52.63% (18278/34731) |
| Line Coverage | 38.07% (166300/436851) |
| Region Coverage | 32.99% (129198/391594) |
| Branch Coverage | 33.81% (55480/164091) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature] doris cross-cluster query [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #57898:
URL: https://github.com/apache/doris/pull/57898#issuecomment-3589479134
# FE Regression Coverage Report
Increment line coverage ` 64.83% (317/489)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/57898_6e75d38a8a85d1bc4670594b3bf7cdd32ba2ee3c_merge_fe/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/57898_6e75d38a8a85d1bc4670594b3bf7cdd32ba2ee3c_merge_fe/report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature] doris cross-cluster query [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #57898:
URL: https://github.com/apache/doris/pull/57898#issuecomment-3589424652
# BE Regression && UT Coverage Report
Increment line coverage `100% (0/0)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/57898_6e75d38a8a85d1bc4670594b3bf7cdd32ba2ee3c_merge/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/57898_6e75d38a8a85d1bc4670594b3bf7cdd32ba2ee3c_merge/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 71.64% (24577/34307) |
| Line Coverage | 58.15% (256315/440751) |
| Region Coverage | 53.19% (213228/400909) |
| Branch Coverage | 54.73% (91227/166692) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature] doris cross-cluster query [doris]
Posted by "HonestManXin (via GitHub)" <gi...@apache.org>.
HonestManXin commented on PR #57898:
URL: https://github.com/apache/doris/pull/57898#issuecomment-3585698616
run beut
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature] doris cross-cluster query [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #57898:
URL: https://github.com/apache/doris/pull/57898#issuecomment-3591752677
# Cloud UT Coverage Report
Increment line coverage ` ` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/2019c7c87f06a8cb801b7ced0d007f2f7784b3be_2019c7c87f06a8cb801b7ced0d007f2f7784b3be_cloud/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/2019c7c87f06a8cb801b7ced0d007f2f7784b3be_2019c7c87f06a8cb801b7ced0d007f2f7784b3be_cloud/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 80.86% (1707/2111) |
| Line Coverage | 66.82% (29884/44724) |
| Region Coverage | 67.35% (14897/22118) |
| Branch Coverage | 57.53% (7923/13772) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature] doris cross-cluster query [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #57898:
URL: https://github.com/apache/doris/pull/57898#issuecomment-3588520749
# Cloud UT Coverage Report
Increment line coverage ` ` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/6e75d38a8a85d1bc4670594b3bf7cdd32ba2ee3c_6e75d38a8a85d1bc4670594b3bf7cdd32ba2ee3c_cloud/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/6e75d38a8a85d1bc4670594b3bf7cdd32ba2ee3c_6e75d38a8a85d1bc4670594b3bf7cdd32ba2ee3c_cloud/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 80.86% (1707/2111) |
| Line Coverage | 66.87% (29896/44708) |
| Region Coverage | 67.41% (14908/22114) |
| Branch Coverage | 57.60% (7931/13768) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature] doris cross-cluster query [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #57898:
URL: https://github.com/apache/doris/pull/57898#issuecomment-3568377927
# FE Regression Coverage Report
Increment line coverage ` 65.07% (298/458)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/57898_4b7d8a7a3ce6da25dd8a85bbb9a335b8df6d8b86_merge_fe/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/57898_4b7d8a7a3ce6da25dd8a85bbb9a335b8df6d8b86_merge_fe/report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature] doris cross-cluster query [doris]
Posted by "HonestManXin (via GitHub)" <gi...@apache.org>.
HonestManXin commented on PR #57898:
URL: https://github.com/apache/doris/pull/57898#issuecomment-3568016602
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature] doris cross-cluster query [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #57898:
URL: https://github.com/apache/doris/pull/57898#issuecomment-3568043822
# Cloud UT Coverage Report
Increment line coverage ` ` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/4b7d8a7a3ce6da25dd8a85bbb9a335b8df6d8b86_4b7d8a7a3ce6da25dd8a85bbb9a335b8df6d8b86_cloud/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/4b7d8a7a3ce6da25dd8a85bbb9a335b8df6d8b86_4b7d8a7a3ce6da25dd8a85bbb9a335b8df6d8b86_cloud/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 80.86% (1707/2111) |
| Line Coverage | 66.89% (29904/44708) |
| Region Coverage | 67.43% (14911/22114) |
| Branch Coverage | 57.63% (7935/13768) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature] doris cross-cluster query [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #57898:
URL: https://github.com/apache/doris/pull/57898#issuecomment-3568145337
<details>
<summary>TPC-DS: <b>Total hot run time: 186990 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 4b7d8a7a3ce6da25dd8a85bbb9a335b8df6d8b86, data reload: false
query1 1067 418 393 393
query2 6585 1655 1683 1655
query3 6751 230 236 230
query4 26220 23303 23067 23067
query5 4508 659 467 467
query6 333 239 218 218
query7 4648 496 289 289
query8 294 249 248 248
query9 8689 2588 2572 2572
query10 485 343 286 286
query11 15607 15122 14885 14885
query12 179 117 112 112
query13 1683 571 476 476
query14 10412 9150 9005 9005
query15 194 186 164 164
query16 7178 653 515 515
query17 1235 830 685 685
query18 1969 419 322 322
query19 212 194 168 168
query20 126 138 131 131
query21 221 130 111 111
query22 4080 4204 4165 4165
query23 33910 33113 32998 32998
query24 8423 2471 2345 2345
query25 616 530 502 502
query26 1246 270 157 157
query27 2747 511 364 364
query28 4366 2202 2176 2176
query29 845 651 509 509
query30 316 223 193 193
query31 897 800 709 709
query32 82 71 69 69
query33 614 367 319 319
query34 775 849 516 516
query35 807 819 747 747
query36 950 1032 887 887
query37 125 119 86 86
query38 3528 3499 3435 3435
query39 1468 1416 1417 1416
query40 231 126 114 114
query41 65 63 65 63
query42 125 113 110 110
query43 497 483 462 462
query44 1230 781 761 761
query45 184 182 170 170
query46 876 974 634 634
query47 1736 1821 1698 1698
query48 390 417 332 332
query49 813 527 394 394
query50 668 337 262 262
query51 3860 3938 3832 3832
query52 110 109 110 109
query53 232 250 194 194
query54 307 288 272 272
query55 90 83 82 82
query56 307 320 312 312
query57 1172 1183 1134 1134
query58 291 276 265 265
query59 2481 2661 2483 2483
query60 353 348 342 342
query61 169 157 157 157
query62 769 696 637 637
query63 233 190 191 190
query64 4533 1269 990 990
query65 4021 3925 3947 3925
query66 1212 452 337 337
query67 15169 15116 14833 14833
query68 4760 896 643 643
query69 524 330 296 296
query70 1343 1297 1263 1263
query71 428 342 323 323
query72 6091 5162 4893 4893
query73 643 586 366 366
query74 9242 8728 8652 8652
query75 3319 3358 2813 2813
query76 3273 1123 725 725
query77 524 417 313 313
query78 9829 9884 9130 9130
query79 2041 900 619 619
query80 1659 598 528 528
query81 577 265 224 224
query82 401 160 135 135
query83 368 268 253 253
query84 266 111 94 94
query85 948 504 456 456
query86 439 328 307 307
query87 3746 3729 3561 3561
query88 2918 2258 2218 2218
query89 391 342 290 290
query90 1795 232 243 232
query91 174 165 134 134
query92 72 68 68 68
query93 1223 903 526 526
query94 718 437 346 346
query95 417 328 315 315
query96 501 568 274 274
query97 2928 2946 2835 2835
query98 235 222 209 209
query99 1301 1410 1263 1263
Total cold run time: 269609 ms
Total hot run time: 186990 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature] doris cross-cluster query [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #57898:
URL: https://github.com/apache/doris/pull/57898#issuecomment-3568784363
# BE UT Coverage Report
Increment line coverage ` ` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/6ebf97d578357addf4ee3a69c8b6d2eeb8bfa59b_6ebf97d578357addf4ee3a69c8b6d2eeb8bfa59b/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/6ebf97d578357addf4ee3a69c8b6d2eeb8bfa59b_6ebf97d578357addf4ee3a69c8b6d2eeb8bfa59b/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 52.63% (18278/34731) |
| Line Coverage | 38.07% (166303/436851) |
| Region Coverage | 33.02% (129322/391594) |
| Branch Coverage | 33.82% (55490/164091) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature] doris cross-cluster query [doris]
Posted by "HonestManXin (via GitHub)" <gi...@apache.org>.
HonestManXin commented on PR #57898:
URL: https://github.com/apache/doris/pull/57898#issuecomment-3569105525
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature] doris cross-cluster query [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #57898:
URL: https://github.com/apache/doris/pull/57898#issuecomment-3570223782
# FE Regression Coverage Report
Increment line coverage ` 66.45% (303/456)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/57898_639c5c970ba78c1055b2c830bd4c20729ffa601d_merge_fe/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/57898_639c5c970ba78c1055b2c830bd4c20729ffa601d_merge_fe/report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature] doris cross-cluster query [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #57898:
URL: https://github.com/apache/doris/pull/57898#issuecomment-3592354869
<details>
<summary>ClickBench: <b>Total hot run time: 28.5 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 2019c7c87f06a8cb801b7ced0d007f2f7784b3be, data reload: false
query1 0.05 0.04 0.04
query2 0.13 0.07 0.07
query3 0.30 0.08 0.07
query4 1.60 0.10 0.10
query5 0.26 0.26 0.25
query6 1.18 0.66 0.63
query7 0.03 0.03 0.02
query8 0.07 0.06 0.06
query9 0.59 0.52 0.51
query10 0.57 0.55 0.55
query11 0.26 0.14 0.13
query12 0.25 0.14 0.14
query13 0.65 0.63 0.61
query14 0.99 0.99 1.01
query15 0.88 0.81 0.83
query16 0.39 0.39 0.38
query17 0.96 1.02 1.01
query18 0.23 0.22 0.22
query19 1.95 1.80 1.79
query20 0.02 0.02 0.02
query21 15.41 0.27 0.24
query22 4.94 0.10 0.09
query23 15.38 0.38 0.22
query24 2.37 0.44 0.30
query25 0.09 0.10 0.10
query26 0.19 0.18 0.18
query27 0.11 0.09 0.09
query28 3.66 1.34 1.16
query29 12.65 4.05 3.33
query30 0.32 0.13 0.11
query31 2.80 0.63 0.44
query32 3.23 0.59 0.48
query33 3.09 3.14 3.06
query34 16.59 5.20 4.67
query35 4.67 4.66 4.63
query36 0.62 0.49 0.53
query37 0.25 0.09 0.08
query38 0.20 0.05 0.06
query39 0.07 0.06 0.05
query40 0.20 0.17 0.17
query41 0.12 0.06 0.06
query42 0.07 0.06 0.05
query43 0.06 0.06 0.05
Total cold run time: 98.45 s
Total hot run time: 28.5 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature] doris cross-cluster query [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #57898:
URL: https://github.com/apache/doris/pull/57898#issuecomment-3588822392
<details>
<summary>TPC-DS: <b>Total hot run time: 181390 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 6e75d38a8a85d1bc4670594b3bf7cdd32ba2ee3c, data reload: false
query1 1076 419 388 388
query2 6596 1196 1141 1141
query3 6756 221 226 221
query4 25441 23416 22787 22787
query5 4840 654 502 502
query6 358 237 220 220
query7 4658 506 304 304
query8 320 252 246 246
query9 8750 2585 2552 2552
query10 518 360 297 297
query11 15288 15159 14643 14643
query12 180 117 111 111
query13 1688 573 450 450
query14 9480 5918 5925 5918
query15 205 197 186 186
query16 7558 690 447 447
query17 1179 747 645 645
query18 2027 421 329 329
query19 207 202 173 173
query20 131 124 120 120
query21 217 138 112 112
query22 3805 3993 3974 3974
query23 33005 31921 31918 31918
query24 8529 2368 2380 2368
query25 633 522 455 455
query26 1244 271 161 161
query27 2773 490 339 339
query28 4374 2119 2108 2108
query29 800 628 496 496
query30 316 235 210 210
query31 847 729 617 617
query32 86 70 74 70
query33 596 389 341 341
query34 822 848 540 540
query35 813 828 734 734
query36 885 899 834 834
query37 125 107 85 85
query38 3891 3823 3728 3728
query39 1462 1407 1399 1399
query40 230 132 124 124
query41 74 64 66 64
query42 125 113 118 113
query43 420 461 419 419
query44 1286 752 747 747
query45 201 197 188 188
query46 888 1004 645 645
query47 1688 1721 1673 1673
query48 390 439 338 338
query49 805 538 465 465
query50 652 683 408 408
query51 3851 3860 3960 3860
query52 117 110 104 104
query53 240 267 197 197
query54 323 322 292 292
query55 96 95 94 94
query56 343 337 348 337
query57 1125 1151 1106 1106
query58 298 285 284 284
query59 2336 2406 2433 2406
query60 374 384 361 361
query61 193 190 189 189
query62 809 717 667 667
query63 232 194 195 194
query64 4669 1307 998 998
query65 4021 3960 4001 3960
query66 1117 447 329 329
query67 15148 14820 14952 14820
query68 8303 936 612 612
query69 524 338 308 308
query70 1094 1005 1003 1003
query71 448 345 303 303
query72 5848 4912 5013 4912
query73 678 590 343 343
query74 8941 8858 8629 8629
query75 3275 3022 2494 2494
query76 3465 1128 719 719
query77 719 388 312 312
query78 9613 9887 8862 8862
query79 1366 840 597 597
query80 700 594 499 499
query81 515 268 234 234
query82 218 139 116 116
query83 282 264 256 256
query84 258 107 102 102
query85 870 497 440 440
query86 322 307 290 290
query87 4068 4061 4043 4043
query88 2896 2282 2242 2242
query89 392 349 295 295
query90 1787 217 214 214
query91 173 177 139 139
query92 71 70 65 65
query93 1137 1002 649 649
query94 648 461 358 358
query95 501 412 408 408
query96 516 527 281 281
query97 2630 2731 2627 2627
query98 250 215 219 215
query99 1294 1359 1272 1272
Total cold run time: 268332 ms
Total hot run time: 181390 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature] doris cross-cluster query [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #57898:
URL: https://github.com/apache/doris/pull/57898#issuecomment-3571347443
# Cloud UT Coverage Report
Increment line coverage ` ` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/397ab88835eda5e584314fdc61afe29e014f140a_397ab88835eda5e584314fdc61afe29e014f140a_cloud/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/397ab88835eda5e584314fdc61afe29e014f140a_397ab88835eda5e584314fdc61afe29e014f140a_cloud/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 80.86% (1707/2111) |
| Line Coverage | 66.85% (29888/44708) |
| Region Coverage | 67.39% (14903/22114) |
| Branch Coverage | 57.54% (7922/13768) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature] doris cross-cluster query [doris]
Posted by "HonestManXin (via GitHub)" <gi...@apache.org>.
HonestManXin commented on code in PR #57898:
URL: https://github.com/apache/doris/pull/57898#discussion_r2564131298
##########
fe/fe-core/src/main/java/org/apache/doris/datasource/doris/RemoteOlapTable.java:
##########
@@ -0,0 +1,137 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.datasource.doris;
+
+import org.apache.doris.catalog.Env;
+import org.apache.doris.catalog.OlapTable;
+import org.apache.doris.catalog.Partition;
+import org.apache.doris.catalog.Tablet;
+import org.apache.doris.common.AnalysisException;
+import org.apache.doris.system.Backend;
+
+import com.google.common.collect.ImmutableMap;
+import com.google.common.collect.Maps;
+import org.apache.logging.log4j.LogManager;
+import org.apache.logging.log4j.Logger;
+
+import java.lang.reflect.Field;
+import java.lang.reflect.Modifier;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.ConcurrentHashMap;
+
+public class RemoteOlapTable extends OlapTable {
+ private static final Logger LOG = LogManager.getLogger(RemoteOlapTable.class);
+
+ private RemoteDorisExternalCatalog catalog;
+ private RemoteDorisExternalDatabase database;
+ private ImmutableMap<Long, Backend> backends = ImmutableMap.of();
Review Comment:
i want the backends not changed after table analyzed, or else backends may changes after catalog refresh
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature] doris cross-cluster query [doris]
Posted by "Thearas (via GitHub)" <gi...@apache.org>.
Thearas commented on PR #57898:
URL: https://github.com/apache/doris/pull/57898#issuecomment-3515228633
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR).
Please clearly describe your PR:
1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
3. What features were added. Why was this function added?
4. Which code was refactored and why was this part of the code refactored?
5. Which functions were optimized and what is the difference before and after the optimization?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature] doris cross-cluster query [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #57898:
URL: https://github.com/apache/doris/pull/57898#issuecomment-3589907097
# BE Regression && UT Coverage Report
Increment line coverage `100% (0/0)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/57898_6e75d38a8a85d1bc4670594b3bf7cdd32ba2ee3c_merge/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/57898_6e75d38a8a85d1bc4670594b3bf7cdd32ba2ee3c_merge/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 71.64% (24579/34307) |
| Line Coverage | 58.16% (256358/440751) |
| Region Coverage | 53.21% (213310/400909) |
| Branch Coverage | 54.74% (91250/166692) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature] doris cross-cluster query [doris]
Posted by "morningman (via GitHub)" <gi...@apache.org>.
morningman commented on PR #57898:
URL: https://github.com/apache/doris/pull/57898#issuecomment-3591727276
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature] doris cross-cluster query [doris]
Posted by "HonestManXin (via GitHub)" <gi...@apache.org>.
HonestManXin commented on PR #57898:
URL: https://github.com/apache/doris/pull/57898#issuecomment-3573974645
run performance
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature] doris cross-cluster query [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #57898:
URL: https://github.com/apache/doris/pull/57898#issuecomment-3568952962
# FE Regression Coverage Report
Increment line coverage ` 17.47% (80/458)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/57898_6ebf97d578357addf4ee3a69c8b6d2eeb8bfa59b_merge_fe/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/57898_6ebf97d578357addf4ee3a69c8b6d2eeb8bfa59b_merge_fe/report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature] doris cross-cluster query [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #57898:
URL: https://github.com/apache/doris/pull/57898#issuecomment-3569057745
# BE Regression && UT Coverage Report
Increment line coverage `100% (0/0)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/57898_6ebf97d578357addf4ee3a69c8b6d2eeb8bfa59b_merge/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/57898_6ebf97d578357addf4ee3a69c8b6d2eeb8bfa59b_merge/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 71.50% (24375/34092) |
| Line Coverage | 57.92% (253129/437017) |
| Region Coverage | 53.33% (211681/396902) |
| Branch Coverage | 54.61% (90163/165102) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature] doris cross-cluster query [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #57898:
URL: https://github.com/apache/doris/pull/57898#issuecomment-3585012758
<details>
<summary>ClickBench: <b>Total hot run time: 28.36 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 2f096957f7645a9b79d50dcff93a47a5dc47aac8, data reload: false
query1 0.05 0.05 0.05
query2 0.12 0.07 0.06
query3 0.30 0.07 0.06
query4 1.60 0.08 0.08
query5 0.25 0.25 0.26
query6 1.16 0.66 0.65
query7 0.03 0.02 0.03
query8 0.07 0.06 0.06
query9 0.64 0.52 0.54
query10 0.59 0.59 0.58
query11 0.25 0.13 0.13
query12 0.27 0.14 0.14
query13 0.66 0.63 0.63
query14 1.04 1.02 1.02
query15 0.94 0.85 0.85
query16 0.39 0.38 0.39
query17 1.04 1.06 1.01
query18 0.23 0.22 0.22
query19 1.95 1.90 1.87
query20 0.02 0.01 0.02
query21 15.38 0.27 0.24
query22 4.98 0.11 0.09
query23 15.37 0.37 0.23
query24 2.98 0.47 0.31
query25 0.10 0.09 0.09
query26 0.18 0.18 0.18
query27 0.09 0.09 0.09
query28 3.73 1.27 1.09
query29 12.54 3.96 3.31
query30 0.33 0.12 0.10
query31 2.81 0.63 0.44
query32 3.23 0.59 0.51
query33 3.12 3.14 3.09
query34 16.53 5.23 4.46
query35 4.54 4.52 4.54
query36 0.64 0.51 0.51
query37 0.22 0.09 0.08
query38 0.19 0.06 0.06
query39 0.06 0.05 0.05
query40 0.19 0.19 0.17
query41 0.12 0.06 0.06
query42 0.06 0.05 0.04
query43 0.07 0.06 0.05
Total cold run time: 99.06 s
Total hot run time: 28.36 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature] doris cross-cluster query [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #57898:
URL: https://github.com/apache/doris/pull/57898#issuecomment-3585711807
# FE Regression Coverage Report
Increment line coverage ` 16.26% (80/492)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/57898_2f096957f7645a9b79d50dcff93a47a5dc47aac8_merge_fe/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/57898_2f096957f7645a9b79d50dcff93a47a5dc47aac8_merge_fe/report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature] doris cross-cluster query [doris]
Posted by "HonestManXin (via GitHub)" <gi...@apache.org>.
HonestManXin commented on PR #57898:
URL: https://github.com/apache/doris/pull/57898#issuecomment-3585699216
run external
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature] doris cross-cluster query [doris]
Posted by "HonestManXin (via GitHub)" <gi...@apache.org>.
HonestManXin commented on PR #57898:
URL: https://github.com/apache/doris/pull/57898#issuecomment-3573705898
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature] doris cross-cluster query [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #57898:
URL: https://github.com/apache/doris/pull/57898#issuecomment-3592309648
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature] doris cross-cluster query [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #57898:
URL: https://github.com/apache/doris/pull/57898#issuecomment-3569458146
<details>
<summary>ClickBench: <b>Total hot run time: 28.21 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 639c5c970ba78c1055b2c830bd4c20729ffa601d, data reload: false
query1 0.06 0.05 0.04
query2 0.12 0.07 0.06
query3 0.32 0.07 0.07
query4 1.61 0.09 0.09
query5 0.27 0.25 0.24
query6 1.17 0.65 0.64
query7 0.04 0.02 0.03
query8 0.08 0.07 0.06
query9 0.64 0.53 0.53
query10 0.58 0.59 0.57
query11 0.25 0.13 0.14
query12 0.25 0.14 0.14
query13 0.66 0.63 0.62
query14 1.02 1.02 1.01
query15 0.93 0.86 0.84
query16 0.39 0.38 0.38
query17 1.01 1.02 1.02
query18 0.24 0.21 0.22
query19 2.00 1.80 1.82
query20 0.02 0.01 0.01
query21 15.39 0.28 0.24
query22 5.00 0.10 0.09
query23 15.37 0.39 0.22
query24 2.91 0.48 0.30
query25 0.10 0.09 0.10
query26 0.18 0.18 0.18
query27 0.09 0.09 0.08
query28 3.64 1.26 1.07
query29 12.56 4.00 3.34
query30 0.32 0.12 0.10
query31 2.82 0.64 0.45
query32 3.25 0.60 0.50
query33 3.14 3.04 3.05
query34 16.63 5.19 4.52
query35 4.50 4.62 4.53
query36 0.66 0.52 0.51
query37 0.22 0.09 0.08
query38 0.19 0.05 0.06
query39 0.06 0.05 0.05
query40 0.21 0.19 0.15
query41 0.11 0.06 0.05
query42 0.06 0.05 0.05
query43 0.06 0.05 0.04
Total cold run time: 99.13 s
Total hot run time: 28.21 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature] doris cross-cluster query [doris]
Posted by "HonestManXin (via GitHub)" <gi...@apache.org>.
HonestManXin commented on PR #57898:
URL: https://github.com/apache/doris/pull/57898#issuecomment-3578273101
run cloud_p0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature] doris cross-cluster query [doris]
Posted by "HonestManXin (via GitHub)" <gi...@apache.org>.
HonestManXin commented on PR #57898:
URL: https://github.com/apache/doris/pull/57898#issuecomment-3571132376
run external
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature] doris cross-cluster query [doris]
Posted by "HonestManXin (via GitHub)" <gi...@apache.org>.
HonestManXin commented on PR #57898:
URL: https://github.com/apache/doris/pull/57898#issuecomment-3571171452
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature] doris cross-cluster query [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #57898:
URL: https://github.com/apache/doris/pull/57898#issuecomment-3571697377
<details>
<summary>ClickBench: <b>Total hot run time: 27.64 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 397ab88835eda5e584314fdc61afe29e014f140a, data reload: false
query1 0.05 0.05 0.05
query2 0.09 0.06 0.05
query3 0.26 0.08 0.08
query4 1.61 0.11 0.11
query5 0.27 0.25 0.25
query6 1.17 0.64 0.65
query7 0.03 0.03 0.03
query8 0.05 0.04 0.04
query9 0.58 0.53 0.51
query10 0.58 0.58 0.57
query11 0.16 0.10 0.11
query12 0.15 0.12 0.12
query13 0.62 0.61 0.63
query14 1.00 1.00 1.00
query15 0.87 0.85 0.83
query16 0.40 0.40 0.38
query17 1.01 0.98 1.00
query18 0.22 0.20 0.20
query19 1.91 1.79 1.79
query20 0.02 0.02 0.01
query21 15.41 0.17 0.14
query22 5.10 0.07 0.05
query23 15.65 0.25 0.10
query24 2.65 0.56 0.96
query25 0.07 0.07 0.06
query26 0.15 0.14 0.12
query27 0.08 0.06 0.05
query28 4.67 1.13 0.95
query29 12.61 3.90 3.22
query30 0.28 0.14 0.11
query31 2.82 0.59 0.39
query32 3.23 0.55 0.46
query33 3.11 3.16 3.14
query34 15.88 5.17 4.55
query35 4.59 4.54 4.56
query36 0.66 0.50 0.48
query37 0.12 0.07 0.07
query38 0.06 0.04 0.04
query39 0.04 0.03 0.03
query40 0.17 0.15 0.13
query41 0.08 0.04 0.03
query42 0.04 0.04 0.04
query43 0.04 0.03 0.03
Total cold run time: 98.56 s
Total hot run time: 27.64 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature] doris cross-cluster query [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #57898:
URL: https://github.com/apache/doris/pull/57898#issuecomment-3574066636
# BE UT Coverage Report
Increment line coverage ` ` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/113154a9292df06cdf5ac65f15d887c14e1a6f1d_113154a9292df06cdf5ac65f15d887c14e1a6f1d/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/113154a9292df06cdf5ac65f15d887c14e1a6f1d_113154a9292df06cdf5ac65f15d887c14e1a6f1d/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 52.82% (18394/34827) |
| Line Coverage | 38.30% (168007/438701) |
| Region Coverage | 33.15% (130615/394022) |
| Branch Coverage | 34.04% (56147/164945) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature] doris cross-cluster query [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #57898:
URL: https://github.com/apache/doris/pull/57898#issuecomment-3571606155
<details>
<summary>TPC-H: <b>Total hot run time: 34159 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 397ab88835eda5e584314fdc61afe29e014f140a, data reload: false
------ Round 1 ----------------------------------
q1 17642 5082 4898 4898
q2 2062 311 207 207
q3 10180 1322 717 717
q4 10233 993 365 365
q5 7563 2325 2352 2325
q6 181 167 135 135
q7 928 779 630 630
q8 9345 1348 1068 1068
q9 7194 5402 5321 5321
q10 6875 2219 1817 1817
q11 497 308 287 287
q12 368 362 217 217
q13 17804 3689 3015 3015
q14 232 236 212 212
q15 586 513 500 500
q16 1053 1084 957 957
q17 588 866 362 362
q18 7874 7181 7092 7092
q19 1094 943 533 533
q20 350 362 234 234
q21 3589 2624 2277 2277
q22 1092 1036 990 990
Total cold run time: 107330 ms
Total hot run time: 34159 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4948 4935 4923 4923
q2 321 407 315 315
q3 2176 2676 2329 2329
q4 1349 1784 1320 1320
q5 4224 4405 4742 4405
q6 226 176 141 141
q7 2043 1977 1858 1858
q8 2662 2622 2549 2549
q9 7460 7567 7321 7321
q10 3051 3291 2831 2831
q11 591 509 493 493
q12 677 734 618 618
q13 3578 3972 3331 3331
q14 308 313 296 296
q15 550 515 520 515
q16 1085 1173 1098 1098
q17 1171 1606 1417 1417
q18 7943 7742 7610 7610
q19 831 808 803 803
q20 2017 2096 1935 1935
q21 4956 4452 4333 4333
q22 1091 1074 1030 1030
Total cold run time: 53258 ms
Total hot run time: 51471 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature] doris cross-cluster query [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #57898:
URL: https://github.com/apache/doris/pull/57898#issuecomment-3571925580
# BE UT Coverage Report
Increment line coverage ` ` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/397ab88835eda5e584314fdc61afe29e014f140a_397ab88835eda5e584314fdc61afe29e014f140a/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/397ab88835eda5e584314fdc61afe29e014f140a_397ab88835eda5e584314fdc61afe29e014f140a/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 52.64% (18292/34746) |
| Line Coverage | 38.07% (166381/437029) |
| Region Coverage | 33.02% (129301/391638) |
| Branch Coverage | 33.82% (55504/164134) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature] doris cross-cluster query [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #57898:
URL: https://github.com/apache/doris/pull/57898#issuecomment-3576057832
<details>
<summary>TPC-H: <b>Total hot run time: 34373 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 608564073977a55e9151e8a1eed087104db5aea4, data reload: false
------ Round 1 ----------------------------------
q1 17473 5087 4932 4932
q2 2039 329 205 205
q3 10139 1278 718 718
q4 10246 950 359 359
q5 7493 2361 2330 2330
q6 188 173 136 136
q7 917 799 654 654
q8 9373 1320 1085 1085
q9 7124 5354 5391 5354
q10 6878 2242 1827 1827
q11 505 310 283 283
q12 362 365 225 225
q13 17784 3676 3001 3001
q14 232 244 222 222
q15 565 507 509 507
q16 1033 1022 987 987
q17 601 865 374 374
q18 7946 7316 7085 7085
q19 1081 953 580 580
q20 353 350 246 246
q21 3790 2550 2297 2297
q22 1035 1005 966 966
Total cold run time: 107157 ms
Total hot run time: 34373 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4937 4984 4950 4950
q2 346 389 332 332
q3 2126 2692 2290 2290
q4 1399 1740 1337 1337
q5 4234 4343 4791 4343
q6 213 171 141 141
q7 2076 2026 1804 1804
q8 2667 2679 2523 2523
q9 7643 7549 7616 7549
q10 3072 3257 2784 2784
q11 586 523 491 491
q12 665 769 634 634
q13 3483 3968 3369 3369
q14 295 308 298 298
q15 554 504 516 504
q16 1052 1125 1098 1098
q17 1165 1625 1419 1419
q18 8019 7678 7397 7397
q19 804 798 849 798
q20 1911 1972 1804 1804
q21 4762 4379 4323 4323
q22 1074 1047 995 995
Total cold run time: 53083 ms
Total hot run time: 51183 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature] doris cross-cluster query [doris]
Posted by "HonestManXin (via GitHub)" <gi...@apache.org>.
HonestManXin commented on PR #57898:
URL: https://github.com/apache/doris/pull/57898#issuecomment-3589525798
run nonConcurrent
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature] doris cross-cluster query [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #57898:
URL: https://github.com/apache/doris/pull/57898#issuecomment-3592001236
# BE Regression && UT Coverage Report
Increment line coverage `100% (0/0)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/57898_2019c7c87f06a8cb801b7ced0d007f2f7784b3be_merge/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/57898_2019c7c87f06a8cb801b7ced0d007f2f7784b3be_merge/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 71.66% (24589/34313) |
| Line Coverage | 58.20% (256561/440793) |
| Region Coverage | 53.28% (213646/400964) |
| Branch Coverage | 54.82% (91391/166708) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature] doris cross-cluster query [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #57898:
URL: https://github.com/apache/doris/pull/57898#issuecomment-3568746953
# Cloud UT Coverage Report
Increment line coverage ` ` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/6ebf97d578357addf4ee3a69c8b6d2eeb8bfa59b_6ebf97d578357addf4ee3a69c8b6d2eeb8bfa59b_cloud/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/6ebf97d578357addf4ee3a69c8b6d2eeb8bfa59b_6ebf97d578357addf4ee3a69c8b6d2eeb8bfa59b_cloud/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 80.86% (1707/2111) |
| Line Coverage | 66.81% (29869/44708) |
| Region Coverage | 67.36% (14895/22114) |
| Branch Coverage | 57.55% (7924/13768) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature] doris cross-cluster query [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #57898:
URL: https://github.com/apache/doris/pull/57898#issuecomment-3568798148
# FE UT Coverage Report
Increment line coverage ` 13.76% (63/458)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/57898_6ebf97d578357addf4ee3a69c8b6d2eeb8bfa59b/fe_increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/57898_6ebf97d578357addf4ee3a69c8b6d2eeb8bfa59b/fe_report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature] doris cross-cluster query [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #57898:
URL: https://github.com/apache/doris/pull/57898#issuecomment-3568364167
# BE Regression && UT Coverage Report
Increment line coverage `100% (0/0)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/57898_4b7d8a7a3ce6da25dd8a85bbb9a335b8df6d8b86_merge/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/57898_4b7d8a7a3ce6da25dd8a85bbb9a335b8df6d8b86_merge/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 71.52% (24382/34092) |
| Line Coverage | 57.99% (253427/437017) |
| Region Coverage | 53.34% (211690/396902) |
| Branch Coverage | 54.67% (90256/165102) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature] doris cross-cluster query [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #57898:
URL: https://github.com/apache/doris/pull/57898#issuecomment-3569855449
# BE UT Coverage Report
Increment line coverage ` ` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/639c5c970ba78c1055b2c830bd4c20729ffa601d_639c5c970ba78c1055b2c830bd4c20729ffa601d/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/639c5c970ba78c1055b2c830bd4c20729ffa601d_639c5c970ba78c1055b2c830bd4c20729ffa601d/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 52.62% (18278/34739) |
| Line Coverage | 38.05% (166306/437051) |
| Region Coverage | 33.01% (129282/391644) |
| Branch Coverage | 33.81% (55485/164124) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature] doris cross-cluster query [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #57898:
URL: https://github.com/apache/doris/pull/57898#issuecomment-3570125941
# BE Regression && UT Coverage Report
Increment line coverage `100% (0/0)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/57898_639c5c970ba78c1055b2c830bd4c20729ffa601d_merge/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/57898_639c5c970ba78c1055b2c830bd4c20729ffa601d_merge/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 71.52% (24390/34102) |
| Line Coverage | 57.97% (253442/437228) |
| Region Coverage | 53.21% (211208/396965) |
| Branch Coverage | 54.59% (90140/165135) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature] doris cross-cluster query [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #57898:
URL: https://github.com/apache/doris/pull/57898#issuecomment-3568672307
# Cloud UT Coverage Report
Increment line coverage ` ` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/07d57c772b039674286ddf4a19f93189571b5972_07d57c772b039674286ddf4a19f93189571b5972_cloud/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/07d57c772b039674286ddf4a19f93189571b5972_07d57c772b039674286ddf4a19f93189571b5972_cloud/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 80.86% (1707/2111) |
| Line Coverage | 66.88% (29899/44708) |
| Region Coverage | 67.39% (14902/22114) |
| Branch Coverage | 57.62% (7933/13768) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature] doris cross-cluster query [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #57898:
URL: https://github.com/apache/doris/pull/57898#issuecomment-3568935192
# BE Regression && UT Coverage Report
Increment line coverage `100% (0/0)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/57898_6ebf97d578357addf4ee3a69c8b6d2eeb8bfa59b_merge/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/57898_6ebf97d578357addf4ee3a69c8b6d2eeb8bfa59b_merge/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 71.50% (24375/34092) |
| Line Coverage | 57.93% (253148/437017) |
| Region Coverage | 53.34% (211721/396902) |
| Branch Coverage | 54.62% (90183/165102) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature] doris cross-cluster query [doris]
Posted by "HonestManXin (via GitHub)" <gi...@apache.org>.
HonestManXin commented on code in PR #57898:
URL: https://github.com/apache/doris/pull/57898#discussion_r2564073603
##########
fe/fe-core/src/main/java/org/apache/doris/catalog/OlapTable.java:
##########
@@ -3698,4 +3699,44 @@ public Index getInvertedIndex(Column column, List<String> subPath) {
.filter(Index::isAnalyzedInvertedIndex).findFirst().orElse(null);
}
}
+
+ /**
+ * caller should acquire the read lock and should not modify any field of the return obj
+ */
+ public OlapTable copyTableMeta() {
Review Comment:
ok, i'll change the method name
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature] doris cross-cluster query [doris]
Posted by "HonestManXin (via GitHub)" <gi...@apache.org>.
HonestManXin commented on code in PR #57898:
URL: https://github.com/apache/doris/pull/57898#discussion_r2564065772
##########
fe/fe-core/src/main/java/org/apache/doris/datasource/doris/RemoteDorisExternalTable.java:
##########
@@ -50,6 +55,27 @@ protected synchronized void makeSureInitialized() {
}
}
+ private RemoteOlapTable getDorisOlapTable() {
+ // TODO if multi thread call this method, only call once
Review Comment:
yes, in 1.x cluster we get table in every query. i can optimize this
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature] doris cross-cluster query [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #57898:
URL: https://github.com/apache/doris/pull/57898#issuecomment-3576116252
<details>
<summary>TPC-DS: <b>Total hot run time: 184267 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 608564073977a55e9151e8a1eed087104db5aea4, data reload: false
query1 1059 403 405 403
query2 6551 1578 1612 1578
query3 6753 235 219 219
query4 25720 22856 22253 22253
query5 4326 612 477 477
query6 335 245 218 218
query7 4654 504 311 311
query8 315 261 258 258
query9 8670 2608 2615 2608
query10 525 362 313 313
query11 15350 14830 14536 14536
query12 181 119 112 112
query13 1698 596 459 459
query14 10920 8843 8936 8843
query15 210 208 193 193
query16 7650 723 517 517
query17 1280 775 645 645
query18 2048 444 344 344
query19 214 209 185 185
query20 138 124 134 124
query21 221 135 119 119
query22 3920 4193 4082 4082
query23 33380 32180 31991 31991
query24 8424 2471 2426 2426
query25 634 571 483 483
query26 1251 279 168 168
query27 2731 501 376 376
query28 4352 2160 2139 2139
query29 845 644 524 524
query30 324 249 223 223
query31 843 734 642 642
query32 81 82 73 73
query33 616 387 386 386
query34 784 883 533 533
query35 803 827 749 749
query36 938 949 886 886
query37 132 115 86 86
query38 3362 3339 3340 3339
query39 1462 1424 1432 1424
query40 219 132 115 115
query41 65 61 67 61
query42 122 111 111 111
query43 476 470 449 449
query44 1248 760 752 752
query45 201 194 179 179
query46 890 998 648 648
query47 1701 1754 1656 1656
query48 397 427 331 331
query49 754 494 434 434
query50 656 689 420 420
query51 3868 3926 4066 3926
query52 112 107 107 107
query53 238 261 194 194
query54 304 287 279 279
query55 87 85 85 85
query56 334 332 337 332
query57 1171 1193 1109 1109
query58 293 318 271 271
query59 2361 2523 2374 2374
query60 349 338 326 326
query61 159 162 159 159
query62 763 733 644 644
query63 234 191 190 190
query64 4538 1240 909 909
query65 4067 3957 3957 3957
query66 1107 438 331 331
query67 15354 14979 14787 14787
query68 8679 900 621 621
query69 506 342 298 298
query70 1319 1265 1160 1160
query71 506 328 301 301
query72 5679 4949 4875 4875
query73 711 588 371 371
query74 8614 8847 8426 8426
query75 3952 3401 2825 2825
query76 3803 1131 713 713
query77 817 398 311 311
query78 9492 9810 8953 8953
query79 1999 830 610 610
query80 626 586 502 502
query81 497 273 235 235
query82 407 165 148 148
query83 273 267 249 249
query84 256 117 95 95
query85 933 504 466 466
query86 340 300 289 289
query87 3485 3497 3467 3467
query88 3020 2271 2252 2252
query89 376 323 287 287
query90 1967 222 216 216
query91 172 172 142 142
query92 83 66 66 66
query93 1092 977 663 663
query94 710 462 354 354
query95 505 410 401 401
query96 497 573 277 277
query97 2915 2977 2887 2887
query98 243 211 218 211
query99 1478 1386 1215 1215
Total cold run time: 271720 ms
Total hot run time: 184267 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature] doris cross-cluster query [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #57898:
URL: https://github.com/apache/doris/pull/57898#issuecomment-3574264326
# BE Regression && UT Coverage Report
Increment line coverage `100% (0/0)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/57898_113154a9292df06cdf5ac65f15d887c14e1a6f1d_merge/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/57898_113154a9292df06cdf5ac65f15d887c14e1a6f1d_merge/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 71.66% (24499/34186) |
| Line Coverage | 58.16% (255224/438843) |
| Region Coverage | 53.37% (213125/399316) |
| Branch Coverage | 54.87% (91049/165938) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature] doris cross-cluster query [doris]
Posted by "HonestManXin (via GitHub)" <gi...@apache.org>.
HonestManXin commented on code in PR #57898:
URL: https://github.com/apache/doris/pull/57898#discussion_r2564099005
##########
fe/fe-core/src/main/java/org/apache/doris/datasource/doris/DorisExternalMetaCacheMgr.java:
##########
@@ -0,0 +1,83 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.datasource.doris;
+
+import org.apache.doris.catalog.Env;
+import org.apache.doris.common.CacheFactory;
+import org.apache.doris.common.Config;
+import org.apache.doris.system.Backend;
+
+import com.github.benmanes.caffeine.cache.LoadingCache;
+import com.google.common.collect.ImmutableMap;
+import com.google.common.collect.Maps;
+import org.apache.logging.log4j.LogManager;
+import org.apache.logging.log4j.Logger;
+
+import java.util.List;
+import java.util.Map;
+import java.util.OptionalLong;
+import java.util.concurrent.ExecutorService;
+import java.util.stream.Collectors;
+
+public class DorisExternalMetaCacheMgr {
+ private static final Logger LOG = LogManager.getLogger(DorisExternalMetaCacheMgr.class);
+ private final LoadingCache<Long, ImmutableMap<Long, Backend>> backendsCache;
+
+ public DorisExternalMetaCacheMgr(ExecutorService executor) {
+ CacheFactory cacheFactory = new CacheFactory(
+ OptionalLong.of(Config.external_cache_expire_time_seconds_after_access),
+ OptionalLong.of(Config.external_cache_refresh_time_minutes * 60),
+ Config.max_external_table_cache_num,
Review Comment:
ok
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature] doris cross-cluster query [doris]
Posted by "HonestManXin (via GitHub)" <gi...@apache.org>.
HonestManXin commented on PR #57898:
URL: https://github.com/apache/doris/pull/57898#issuecomment-3573613045
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature] doris cross-cluster query [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #57898:
URL: https://github.com/apache/doris/pull/57898#issuecomment-3584631895
# Cloud UT Coverage Report
Increment line coverage ` ` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/2f096957f7645a9b79d50dcff93a47a5dc47aac8_2f096957f7645a9b79d50dcff93a47a5dc47aac8_cloud/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/2f096957f7645a9b79d50dcff93a47a5dc47aac8_2f096957f7645a9b79d50dcff93a47a5dc47aac8_cloud/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 80.86% (1707/2111) |
| Line Coverage | 66.86% (29892/44708) |
| Region Coverage | 67.40% (14905/22114) |
| Branch Coverage | 57.58% (7928/13768) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature] doris cross-cluster query [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #57898:
URL: https://github.com/apache/doris/pull/57898#issuecomment-3586645459
# BE UT Coverage Report
Increment line coverage ` ` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/2f096957f7645a9b79d50dcff93a47a5dc47aac8_2f096957f7645a9b79d50dcff93a47a5dc47aac8/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/2f096957f7645a9b79d50dcff93a47a5dc47aac8_2f096957f7645a9b79d50dcff93a47a5dc47aac8/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 52.89% (18505/34985) |
| Line Coverage | 38.43% (169517/441050) |
| Region Coverage | 33.20% (131459/395952) |
| Branch Coverage | 34.19% (56671/165767) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature] doris cross-cluster query [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #57898:
URL: https://github.com/apache/doris/pull/57898#issuecomment-3589956324
# FE Regression Coverage Report
Increment line coverage ` 64.83% (317/489)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/57898_6e75d38a8a85d1bc4670594b3bf7cdd32ba2ee3c_merge_fe/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/57898_6e75d38a8a85d1bc4670594b3bf7cdd32ba2ee3c_merge_fe/report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature] doris cross-cluster query [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #57898:
URL: https://github.com/apache/doris/pull/57898#issuecomment-3592352151
<details>
<summary>TPC-DS: <b>Total hot run time: 181591 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 2019c7c87f06a8cb801b7ced0d007f2f7784b3be, data reload: false
query1 1051 409 393 393
query2 6567 1197 1136 1136
query3 6747 228 220 220
query4 25458 23142 22959 22959
query5 4862 691 504 504
query6 347 250 239 239
query7 4656 511 314 314
query8 313 274 243 243
query9 8740 2628 2629 2628
query10 545 358 326 326
query11 15133 14813 14865 14813
query12 180 116 118 116
query13 1726 572 441 441
query14 9307 6024 5958 5958
query15 206 200 194 194
query16 7715 698 518 518
query17 1208 769 646 646
query18 2065 440 348 348
query19 213 210 181 181
query20 131 125 121 121
query21 226 133 116 116
query22 3943 3883 3933 3883
query23 33014 32036 31863 31863
query24 8343 2411 2431 2411
query25 630 564 499 499
query26 1253 283 167 167
query27 2725 501 365 365
query28 4200 2150 2168 2150
query29 821 659 516 516
query30 311 242 205 205
query31 827 714 627 627
query32 86 75 75 75
query33 615 393 380 380
query34 809 864 537 537
query35 818 830 733 733
query36 900 915 849 849
query37 123 111 84 84
query38 3866 3798 3731 3731
query39 1612 1600 1388 1388
query40 223 128 121 121
query41 65 61 62 61
query42 130 112 111 111
query43 445 447 415 415
query44 1257 745 748 745
query45 207 196 184 184
query46 882 987 643 643
query47 1670 1723 1653 1653
query48 392 419 326 326
query49 808 500 415 415
query50 649 692 399 399
query51 3817 3892 3804 3804
query52 110 109 104 104
query53 235 261 192 192
query54 308 301 311 301
query55 94 94 91 91
query56 335 322 329 322
query57 1152 1156 1124 1124
query58 295 275 271 271
query59 2324 2424 2348 2348
query60 347 359 336 336
query61 154 162 158 158
query62 815 706 655 655
query63 233 194 192 192
query64 4289 1200 889 889
query65 4066 3975 3962 3962
query66 1097 433 334 334
query67 15295 14889 14800 14800
query68 8478 949 623 623
query69 517 337 316 316
query70 1085 1053 1036 1036
query71 485 337 310 310
query72 5778 4888 4971 4888
query73 749 601 363 363
query74 8575 8724 8572 8572
query75 3717 3021 2510 2510
query76 3817 1139 733 733
query77 807 395 312 312
query78 9430 9591 8863 8863
query79 2012 844 583 583
query80 644 577 487 487
query81 505 268 243 243
query82 432 139 111 111
query83 266 256 253 253
query84 260 112 98 98
query85 928 493 435 435
query86 384 320 304 304
query87 4047 4000 3949 3949
query88 3952 2295 2312 2295
query89 397 339 289 289
query90 1996 221 215 215
query91 167 177 140 140
query92 89 66 71 66
query93 1937 1007 666 666
query94 729 432 337 337
query95 509 415 400 400
query96 515 545 284 284
query97 2593 2665 2575 2575
query98 259 216 210 210
query99 1363 1375 1292 1292
Total cold run time: 271408 ms
Total hot run time: 181591 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature] doris cross-cluster query [doris]
Posted by "morningman (via GitHub)" <gi...@apache.org>.
morningman merged PR #57898:
URL: https://github.com/apache/doris/pull/57898
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org