You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "hubgeter (via GitHub)" <gi...@apache.org> on 2024/09/01 10:44:55 UTC
[PR] [Enhancement](MaxCompute)Refactoring maxCompute catalog using Storage API. [doris]
hubgeter opened a new pull request, #40225:
URL: https://github.com/apache/doris/pull/40225
## Proposed changes
Refactoring maxCompute catalog using Storage API.
Storage API : https://help.aliyun.com/zh/maxcompute/user-guide/open-storage-sample-java-sdk?spm=a2c4g.11186623.0.i0
```
Required parameters：
CREATE CATALOG mc PROPERTIES (
"type" = "max_compute",
"mc.default.project" = "jz_datalake",
"mc.access_key" = "xxxx",
"mc.secret_key" = "xxxx",
"mc.endpoint" = "http://service.cn-beijing-vpc.maxcompute.aliyun-inc.com/api",
"mc.quota" = "pay-as-you-go"
);
Optional parameters:
"split_strategy" = "byte_size" / "row_offset"
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](MaxCompute)Refactoring maxCompute catalog using Storage API. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40225:
URL: https://github.com/apache/doris/pull/40225#issuecomment-2323319579
<details>
<summary>TPC-H: <b>Total hot run time: 38016 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 9b818bedb956a247985af6cb0fbdfe19079b24d2, data reload: false
------ Round 1 ----------------------------------
q1 18036 4532 4332 4332
q2 2030 189 177 177
q3 11476 929 1065 929
q4 10429 791 777 777
q5 7771 2905 2851 2851
q6 235 147 143 143
q7 974 611 634 611
q8 9338 2098 2145 2098
q9 7242 6538 6554 6538
q10 7001 2212 2223 2212
q11 448 242 245 242
q12 394 220 226 220
q13 17766 3059 3070 3059
q14 281 228 231 228
q15 518 479 503 479
q16 591 514 526 514
q17 990 699 758 699
q18 7475 6760 6854 6760
q19 1412 1057 1109 1057
q20 694 337 344 337
q21 3980 3078 2749 2749
q22 1143 1019 1004 1004
Total cold run time: 110224 ms
Total hot run time: 38016 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4332 4491 4316 4316
q2 383 280 277 277
q3 2924 2688 2708 2688
q4 1946 1629 1658 1629
q5 5703 5738 5771 5738
q6 247 143 142 142
q7 2216 1858 1817 1817
q8 3325 3420 3477 3420
q9 8887 8947 8873 8873
q10 3614 3433 3391 3391
q11 621 504 513 504
q12 840 662 651 651
q13 14779 3134 3298 3134
q14 333 289 284 284
q15 533 499 487 487
q16 621 576 563 563
q17 1843 1524 1542 1524
q18 8165 7827 7772 7772
q19 1748 1724 1682 1682
q20 2217 1937 1874 1874
q21 5754 5565 5604 5565
q22 1120 1056 1050 1050
Total cold run time: 72151 ms
Total hot run time: 57381 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](MaxCompute)Refactoring maxCompute catalog using Storage API. [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #40225:
URL: https://github.com/apache/doris/pull/40225#issuecomment-2326960444
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](MaxCompute)Refactoring maxCompute catalog using Storage API. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40225:
URL: https://github.com/apache/doris/pull/40225#issuecomment-2329689317
TeamCity be ut coverage result:
Function Coverage: 36.84% (9394/25500)
Line Coverage: 28.28% (77462/273936)
Region Coverage: 27.66% (39961/144452)
Branch Coverage: 24.31% (20334/83652)
Coverage Report: http://coverage.selectdb-in.cc/coverage/8cb3173c7b8f8c032b956ab11d39828305edc1a1_8cb3173c7b8f8c032b956ab11d39828305edc1a1/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](MaxCompute)Refactoring maxCompute catalog using Storage API. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40225:
URL: https://github.com/apache/doris/pull/40225#issuecomment-2333251212
<details>
<summary>TPC-DS: <b>Total hot run time: 187278 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 04f750caf7f73efeb0902ec2a15ea7b9a6bc4294, data reload: false
query1 917 384 364 364
query2 6467 1966 1896 1896
query3 6649 214 216 214
query4 30999 23415 23063 23063
query5 4152 549 484 484
query6 262 170 156 156
query7 4584 300 290 290
query8 270 224 242 224
query9 8538 2465 2428 2428
query10 434 280 266 266
query11 17441 15092 14967 14967
query12 152 103 100 100
query13 1657 384 360 360
query14 9329 7206 7323 7206
query15 262 166 168 166
query16 7978 479 443 443
query17 1607 547 534 534
query18 2019 283 282 282
query19 356 144 143 143
query20 117 109 113 109
query21 217 103 100 100
query22 4379 4049 4189 4049
query23 33904 33519 33405 33405
query24 11134 2892 2862 2862
query25 599 391 385 385
query26 1170 151 152 151
query27 2729 274 274 274
query28 7383 2028 1986 1986
query29 827 395 399 395
query30 308 155 152 152
query31 956 743 759 743
query32 92 57 59 57
query33 754 285 288 285
query34 981 464 468 464
query35 851 713 724 713
query36 1074 940 945 940
query37 152 85 94 85
query38 3988 3874 3843 3843
query39 1443 1388 1401 1388
query40 200 115 115 115
query41 49 53 45 45
query42 114 95 94 94
query43 510 487 467 467
query44 1228 758 729 729
query45 195 164 197 164
query46 1095 754 764 754
query47 1869 1781 1813 1781
query48 363 285 288 285
query49 1076 444 442 442
query50 828 398 408 398
query51 7008 6939 6977 6939
query52 100 89 86 86
query53 251 187 175 175
query54 952 441 455 441
query55 79 75 77 75
query56 273 248 268 248
query57 1172 1061 1054 1054
query58 248 228 229 228
query59 3098 2977 2766 2766
query60 289 273 272 272
query61 108 152 98 98
query62 856 657 664 657
query63 214 183 187 183
query64 4218 671 639 639
query65 3223 3149 3155 3149
query66 1404 341 337 337
query67 15618 14962 15279 14962
query68 3587 578 574 574
query69 394 286 274 274
query70 1106 1106 1113 1106
query71 335 278 275 275
query72 6337 4002 3770 3770
query73 737 319 327 319
query74 9168 8808 8848 8808
query75 3388 2707 2735 2707
query76 1862 962 992 962
query77 480 334 309 309
query78 10219 9412 9852 9412
query79 1074 533 520 520
query80 688 520 506 506
query81 497 240 239 239
query82 248 144 145 144
query83 178 154 151 151
query84 223 79 78 78
query85 887 293 347 293
query86 311 312 308 308
query87 4545 4237 4241 4237
query88 2905 2274 2273 2273
query89 378 289 284 284
query90 1968 194 198 194
query91 126 99 109 99
query92 65 51 49 49
query93 1014 542 537 537
query94 691 289 300 289
query95 355 253 251 251
query96 578 268 264 264
query97 3171 3043 3102 3043
query98 223 195 196 195
query99 1456 1253 1282 1253
Total cold run time: 282963 ms
Total hot run time: 187278 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](MaxCompute)Refactoring maxCompute catalog using Storage API. [doris]
Posted by "hubgeter (via GitHub)" <gi...@apache.org>.
hubgeter commented on PR #40225:
URL: https://github.com/apache/doris/pull/40225#issuecomment-2342801173
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](MaxCompute)Refactoring maxCompute catalog using Storage API. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40225:
URL: https://github.com/apache/doris/pull/40225#issuecomment-2343054018
<details>
<summary>ClickBench: <b>Total hot run time: 31.45 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 9d37687020a10441967fc000292259bcbf7f75dd, data reload: false
query1 0.05 0.05 0.04
query2 0.08 0.04 0.04
query3 0.23 0.06 0.05
query4 1.66 0.07 0.08
query5 0.49 0.50 0.49
query6 1.12 0.73 0.73
query7 0.02 0.02 0.02
query8 0.06 0.05 0.05
query9 0.54 0.48 0.48
query10 0.56 0.55 0.54
query11 0.15 0.11 0.11
query12 0.16 0.12 0.12
query13 0.61 0.58 0.59
query14 1.38 1.45 1.40
query15 0.85 0.82 0.82
query16 0.38 0.37 0.39
query17 1.00 1.03 1.02
query18 0.23 0.20 0.20
query19 1.90 1.76 1.81
query20 0.01 0.01 0.01
query21 15.42 0.67 0.66
query22 4.58 7.31 1.72
query23 18.24 1.42 1.28
query24 2.12 0.21 0.22
query25 0.16 0.08 0.07
query26 0.27 0.17 0.17
query27 0.08 0.08 0.08
query28 13.25 1.03 1.01
query29 12.62 3.45 3.44
query30 0.25 0.06 0.07
query31 2.84 0.41 0.40
query32 3.24 0.48 0.48
query33 3.02 3.02 3.05
query34 17.11 4.40 4.42
query35 4.45 4.47 4.46
query36 0.67 0.49 0.49
query37 0.19 0.16 0.16
query38 0.16 0.15 0.14
query39 0.05 0.04 0.04
query40 0.16 0.13 0.13
query41 0.10 0.05 0.05
query42 0.05 0.04 0.04
query43 0.05 0.04 0.04
Total cold run time: 110.56 s
Total hot run time: 31.45 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](MaxCompute)Refactoring maxCompute catalog using Storage API. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40225:
URL: https://github.com/apache/doris/pull/40225#issuecomment-2323326304
<details>
<summary>ClickBench: <b>Total hot run time: 32.45 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 9b818bedb956a247985af6cb0fbdfe19079b24d2, data reload: false
query1 0.05 0.05 0.04
query2 0.07 0.04 0.05
query3 0.23 0.05 0.05
query4 1.68 0.08 0.08
query5 0.51 0.48 0.50
query6 1.14 0.73 0.74
query7 0.02 0.02 0.01
query8 0.05 0.05 0.04
query9 0.55 0.49 0.49
query10 0.54 0.54 0.54
query11 0.16 0.12 0.11
query12 0.15 0.12 0.12
query13 0.62 0.59 0.59
query14 2.05 2.12 2.07
query15 0.84 0.83 0.82
query16 0.35 0.36 0.37
query17 0.97 0.99 1.04
query18 0.21 0.20 0.19
query19 1.80 1.76 1.84
query20 0.01 0.01 0.01
query21 15.43 0.66 0.66
query22 4.19 6.66 2.28
query23 18.29 1.34 1.35
query24 2.15 0.21 0.23
query25 0.15 0.07 0.08
query26 0.27 0.19 0.18
query27 0.07 0.08 0.08
query28 13.21 1.01 1.00
query29 12.60 3.32 3.31
query30 0.25 0.06 0.06
query31 2.88 0.40 0.39
query32 3.25 0.49 0.47
query33 2.97 2.99 3.01
query34 17.06 4.43 4.42
query35 4.45 4.51 4.41
query36 0.65 0.47 0.47
query37 0.19 0.16 0.15
query38 0.15 0.16 0.15
query39 0.06 0.04 0.04
query40 0.15 0.12 0.12
query41 0.09 0.05 0.06
query42 0.06 0.05 0.04
query43 0.05 0.05 0.04
Total cold run time: 110.62 s
Total hot run time: 32.45 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](MaxCompute)Refactoring maxCompute catalog using Storage API. [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #40225:
URL: https://github.com/apache/doris/pull/40225#issuecomment-2333179001
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](MaxCompute)Refactoring maxCompute catalog using Storage API. [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #40225:
URL: https://github.com/apache/doris/pull/40225#issuecomment-2326935063
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](MaxCompute)Refactoring maxCompute catalog using Storage API. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40225:
URL: https://github.com/apache/doris/pull/40225#issuecomment-2333256054
<details>
<summary>ClickBench: <b>Total hot run time: 31.46 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 04f750caf7f73efeb0902ec2a15ea7b9a6bc4294, data reload: false
query1 0.04 0.05 0.04
query2 0.08 0.04 0.04
query3 0.23 0.06 0.06
query4 1.66 0.09 0.07
query5 0.55 0.49 0.49
query6 1.13 0.73 0.73
query7 0.02 0.01 0.02
query8 0.05 0.05 0.05
query9 0.53 0.49 0.49
query10 0.55 0.56 0.54
query11 0.16 0.12 0.11
query12 0.15 0.13 0.12
query13 0.61 0.59 0.58
query14 1.37 1.41 1.42
query15 0.84 0.84 0.81
query16 0.35 0.37 0.37
query17 1.06 1.04 1.04
query18 0.20 0.20 0.20
query19 1.92 1.80 1.88
query20 0.02 0.01 0.01
query21 15.39 0.66 0.64
query22 4.56 6.88 2.03
query23 18.24 1.37 1.21
query24 2.17 0.22 0.22
query25 0.15 0.08 0.08
query26 0.26 0.18 0.18
query27 0.07 0.08 0.08
query28 13.23 1.01 0.99
query29 12.56 3.31 3.25
query30 0.24 0.06 0.07
query31 2.87 0.40 0.39
query32 3.26 0.48 0.48
query33 2.93 3.04 2.96
query34 16.87 4.43 4.41
query35 4.43 4.42 4.45
query36 0.65 0.49 0.50
query37 0.18 0.16 0.16
query38 0.15 0.15 0.16
query39 0.05 0.03 0.04
query40 0.16 0.13 0.14
query41 0.10 0.05 0.05
query42 0.05 0.04 0.04
query43 0.04 0.04 0.04
Total cold run time: 110.13 s
Total hot run time: 31.46 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](MaxCompute)Refactoring maxCompute catalog using Storage API. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40225:
URL: https://github.com/apache/doris/pull/40225#issuecomment-2331753922
<details>
<summary>TPC-H: <b>Total hot run time: 38414 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 7e229aaf10594938d9bb8ff159deb52d1d673575, data reload: false
------ Round 1 ----------------------------------
q1 18229 4639 4431 4431
q2 2540 206 191 191
q3 11239 1136 1098 1098
q4 11287 794 764 764
q5 8599 2964 2802 2802
q6 230 140 135 135
q7 960 615 611 611
q8 9330 2091 2090 2090
q9 7236 6555 6559 6555
q10 7043 2265 2287 2265
q11 456 239 250 239
q12 414 226 217 217
q13 17778 3062 3043 3043
q14 279 237 241 237
q15 551 508 481 481
q16 590 537 526 526
q17 1009 692 699 692
q18 7353 6848 6804 6804
q19 1390 1025 1047 1025
q20 673 331 335 331
q21 3946 3159 2903 2903
q22 1117 1004 974 974
Total cold run time: 112249 ms
Total hot run time: 38414 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4378 4289 4290 4289
q2 393 275 277 275
q3 2936 2681 2690 2681
q4 1927 1656 1655 1655
q5 5451 5445 5392 5392
q6 223 131 133 131
q7 2112 1751 1771 1751
q8 3243 3384 3350 3350
q9 8473 8518 8424 8424
q10 3468 3230 3188 3188
q11 593 507 511 507
q12 771 599 603 599
q13 11837 3088 3027 3027
q14 298 268 284 268
q15 516 474 480 474
q16 619 551 564 551
q17 1817 1477 1476 1476
q18 7815 7639 7356 7356
q19 1669 1631 1571 1571
q20 2077 1835 1827 1827
q21 5546 5220 5206 5206
q22 1114 1031 1005 1005
Total cold run time: 67276 ms
Total hot run time: 55003 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](MaxCompute)Refactoring maxCompute catalog using Storage API. [doris]
Posted by "morningman (via GitHub)" <gi...@apache.org>.
morningman merged PR #40225:
URL: https://github.com/apache/doris/pull/40225
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](MaxCompute)Refactoring maxCompute catalog using Storage API. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40225:
URL: https://github.com/apache/doris/pull/40225#issuecomment-2331782313
<details>
<summary>TPC-DS: <b>Total hot run time: 187765 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 7e229aaf10594938d9bb8ff159deb52d1d673575, data reload: false
query1 917 384 386 384
query2 6495 2048 1938 1938
query3 6646 225 223 223
query4 31428 23128 23187 23128
query5 4181 515 502 502
query6 257 166 179 166
query7 4587 303 297 297
query8 273 228 231 228
query9 8657 2532 2525 2525
query10 440 300 282 282
query11 15862 15017 15246 15017
query12 150 101 101 101
query13 1635 393 371 371
query14 9302 6900 7168 6900
query15 284 169 184 169
query16 8084 465 455 455
query17 1603 568 547 547
query18 2069 287 284 284
query19 247 143 143 143
query20 120 109 112 109
query21 210 109 104 104
query22 4387 4276 4007 4007
query23 34141 33330 33294 33294
query24 11135 2819 2880 2819
query25 624 380 390 380
query26 1392 151 156 151
query27 2851 282 273 273
query28 7708 2077 2056 2056
query29 859 409 404 404
query30 311 158 158 158
query31 959 751 842 751
query32 103 56 57 56
query33 749 280 290 280
query34 983 496 478 478
query35 831 712 710 710
query36 1077 912 929 912
query37 166 94 102 94
query38 4019 3892 3826 3826
query39 1453 1439 1420 1420
query40 264 115 116 115
query41 49 47 48 47
query42 116 97 95 95
query43 520 475 474 474
query44 1173 757 756 756
query45 194 167 168 167
query46 1101 721 726 721
query47 1872 1795 1810 1795
query48 370 315 296 296
query49 1096 447 465 447
query50 802 411 424 411
query51 6951 6930 6872 6872
query52 101 89 87 87
query53 257 181 176 176
query54 854 457 464 457
query55 77 77 75 75
query56 275 258 259 258
query57 1207 1050 1072 1050
query58 254 242 249 242
query59 3098 2761 2835 2761
query60 313 266 268 266
query61 105 100 101 100
query62 877 686 666 666
query63 223 191 182 182
query64 5322 684 655 655
query65 3249 3164 3175 3164
query66 1415 347 339 339
query67 15403 15234 15214 15214
query68 3549 579 574 574
query69 411 286 294 286
query70 1136 1122 1080 1080
query71 339 281 288 281
query72 6506 4147 4125 4125
query73 751 331 330 330
query74 9146 8812 8902 8812
query75 3373 2718 2688 2688
query76 1917 986 1020 986
query77 477 317 320 317
query78 9720 9352 9012 9012
query79 1077 549 534 534
query80 901 536 543 536
query81 560 240 246 240
query82 1117 147 150 147
query83 244 160 160 160
query84 245 81 88 81
query85 1108 355 342 342
query86 306 270 296 270
query87 4446 4189 4285 4189
query88 3010 2390 2392 2390
query89 391 296 288 288
query90 1999 201 202 201
query91 146 113 118 113
query92 63 54 50 50
query93 1051 550 535 535
query94 828 301 310 301
query95 366 271 269 269
query96 595 278 271 271
query97 3169 3062 3052 3052
query98 353 250 201 201
query99 1713 1274 1317 1274
Total cold run time: 285746 ms
Total hot run time: 187765 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](MaxCompute)Refactoring maxCompute catalog using Storage API. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40225:
URL: https://github.com/apache/doris/pull/40225#issuecomment-2329689169
<details>
<summary>TPC-DS: <b>Total hot run time: 188053 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 8cb3173c7b8f8c032b956ab11d39828305edc1a1, data reload: false
query1 915 385 371 371
query2 6501 1994 1982 1982
query3 6646 218 216 216
query4 29224 23283 23175 23175
query5 4171 500 473 473
query6 252 166 166 166
query7 4577 294 293 293
query8 248 192 208 192
query9 8599 2518 2485 2485
query10 448 282 270 270
query11 17920 15065 15046 15046
query12 150 100 98 98
query13 1620 383 378 378
query14 8804 7268 7135 7135
query15 272 165 173 165
query16 7922 430 468 430
query17 1582 556 532 532
query18 2044 278 286 278
query19 315 141 142 141
query20 114 111 108 108
query21 211 103 101 101
query22 4522 4108 4079 4079
query23 34242 33442 33620 33442
query24 11167 2872 2846 2846
query25 646 373 389 373
query26 1175 156 155 155
query27 2483 283 277 277
query28 7516 2067 2060 2060
query29 819 415 406 406
query30 306 166 153 153
query31 990 769 781 769
query32 103 53 53 53
query33 756 307 292 292
query34 946 466 482 466
query35 851 730 725 725
query36 1111 924 924 924
query37 170 94 94 94
query38 3969 3867 4025 3867
query39 1442 1402 1383 1383
query40 203 117 112 112
query41 50 45 45 45
query42 118 95 97 95
query43 511 486 460 460
query44 1268 776 735 735
query45 217 169 171 169
query46 1100 742 771 742
query47 1875 1774 1807 1774
query48 395 305 302 302
query49 1104 435 430 430
query50 815 402 417 402
query51 7013 6865 6858 6858
query52 102 89 92 89
query53 260 187 182 182
query54 952 450 459 450
query55 77 75 75 75
query56 285 269 261 261
query57 1157 1074 1085 1074
query58 235 222 232 222
query59 2988 2804 2843 2804
query60 284 260 268 260
query61 104 95 105 95
query62 847 656 642 642
query63 224 184 180 180
query64 4316 692 643 643
query65 3204 3162 3122 3122
query66 1436 338 340 338
query67 15747 15236 15554 15236
query68 3266 598 576 576
query69 399 274 272 272
query70 1131 1145 1106 1106
query71 344 264 271 264
query72 6446 4049 4043 4043
query73 740 329 324 324
query74 9339 8860 8862 8860
query75 3418 2678 2720 2678
query76 1861 1016 965 965
query77 490 308 316 308
query78 9579 9143 9092 9092
query79 1041 537 526 526
query80 691 497 506 497
query81 475 236 234 234
query82 236 151 142 142
query83 167 157 156 156
query84 229 76 83 76
query85 683 291 278 278
query86 309 302 303 302
query87 4451 4251 4267 4251
query88 3034 2402 2438 2402
query89 380 292 281 281
query90 1792 186 184 184
query91 129 106 103 103
query92 61 49 50 49
query93 1029 552 539 539
query94 812 276 280 276
query95 348 253 249 249
query96 590 271 272 271
query97 3145 3099 3113 3099
query98 214 200 195 195
query99 1487 1259 1268 1259
Total cold run time: 280952 ms
Total hot run time: 188053 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](MaxCompute)Refactoring maxCompute catalog using Storage API. [doris]
Posted by "morrySnow (via GitHub)" <gi...@apache.org>.
morrySnow commented on code in PR #40225:
URL: https://github.com/apache/doris/pull/40225#discussion_r1751559686
##########
fe/fe-core/pom.xml:
##########
@@ -453,6 +454,11 @@ under the License.
</exclusion>
</exclusions>
</dependency>
+ <dependency>
+ <groupId>com.aliyun.odps</groupId>
+ <artifactId>odps-sdk-table-api</artifactId>
+ <version>0.48.8-public</version>
Review Comment:
better to use a property here to ensure odps deps always with same version
##########
fe/fe-core/src/main/java/org/apache/doris/datasource/maxcompute/source/MaxComputeScanNode.java:
##########
@@ -75,11 +99,290 @@ private void setScanParams(TFileRangeDesc rangeDesc, MaxComputeSplit maxComputeS
TTableFormatFileDesc tableFormatFileDesc = new TTableFormatFileDesc();
tableFormatFileDesc.setTableFormatType(TableFormatType.MAX_COMPUTE.value());
TMaxComputeFileDesc fileDesc = new TMaxComputeFileDesc();
- if (maxComputeSplit.getPartitionSpec().isPresent()) {
- fileDesc.setPartitionSpec(maxComputeSplit.getPartitionSpec().get());
- }
+ fileDesc.setPartitionSpec("deprecated");
+ fileDesc.setTableBatchReadSession(maxComputeSplit.scanSerialize);
+ fileDesc.setSessionId(maxComputeSplit.getSessionId());
tableFormatFileDesc.setMaxComputeParams(fileDesc);
rangeDesc.setTableFormatParams(tableFormatFileDesc);
+ rangeDesc.setPath("[ " + maxComputeSplit.getStart() + " , " + maxComputeSplit.getLength() + " ]");
+ rangeDesc.setStartOffset(maxComputeSplit.getStart());
+ rangeDesc.setSize(maxComputeSplit.getLength());
+ }
+
+ void createTableBatchReadSession() throws UserException {
+ Predicate filterPredicate = convertPredicate();
+
+
+ List<String> requiredPartitionColumns = new ArrayList<>();
+ List<String> orderedRequiredDataColumns = new ArrayList<>();
+
+ Set<String> requiredSlots =
+ desc.getSlots().stream().map(e -> e.getColumn().getName()).collect(Collectors.toSet());
+
+ Set<String> partitionColumns =
+ table.getPartitionColumns().stream().map(Column::getName).collect(Collectors.toSet());
+
+ for (Column column : table.getColumns()) {
+ String columnName = column.getName();
+ if (!requiredSlots.contains(columnName)) {
+ continue;
+ }
+ if (partitionColumns.contains(columnName)) {
+ requiredPartitionColumns.add(columnName);
+ } else {
+ orderedRequiredDataColumns.add(columnName);
+ }
+ }
+
+
+
+ MaxComputeExternalCatalog mcCatalog = (MaxComputeExternalCatalog) table.getCatalog();
+
+ try {
+ TableReadSessionBuilder scanBuilder = new TableReadSessionBuilder();
+ tableBatchReadSession =
+ scanBuilder.identifier(TableIdentifier.of(table.getDbName(), table.getName()))
+ .withSettings(mcCatalog.getSettings())
+ .withSplitOptions(mcCatalog.getSplitOption())
+ .requiredPartitionColumns(requiredPartitionColumns)
+ .requiredDataColumns(orderedRequiredDataColumns)
+ .withArrowOptions(
+ ArrowOptions.newBuilder()
+ .withDatetimeUnit(TimestampUnit.MILLI)
+ .withTimestampUnit(TimestampUnit.NANO)
+ .build()
+ )
+ .withFilterPredicate(filterPredicate)
+ .buildBatchReadSession();
+ } catch (java.io.IOException e) {
+ throw new RuntimeException(e);
+ }
+
+ }
+
+ protected Predicate convertPredicate() {
+ if (conjuncts.isEmpty()) {
+ return Predicate.NO_PREDICATE;
+ }
+
+ if (conjuncts.size() == 1) {
+ try {
+ return convertExprToOdpsPredicate(conjuncts.get(0));
+ } catch (AnalysisException e) {
+ Log.info("Failed to convert predicate " + conjuncts.get(0) + " to odps predicate");
+ Log.info("Reason: " + e.getMessage());
+ return Predicate.NO_PREDICATE;
+ }
+ }
+
+ com.aliyun.odps.table.optimizer.predicate.CompoundPredicate
+ filterPredicate = new com.aliyun.odps.table.optimizer.predicate.CompoundPredicate(
+ com.aliyun.odps.table.optimizer.predicate.CompoundPredicate.Operator.AND
+ );
+
+ for (Expr predicate : conjuncts) {
+ try {
+ filterPredicate.addPredicate(convertExprToOdpsPredicate(predicate));
+ } catch (AnalysisException e) {
+ Log.info("Failed to convert predicate " + predicate);
+ Log.info("Reason: " + e.getMessage());
+ return Predicate.NO_PREDICATE;
+ }
+ }
+ return filterPredicate;
+ }
+
+ private Predicate convertExprToOdpsPredicate(Expr expr) throws AnalysisException {
+ Predicate odpsPredicate = null;
+ if (expr instanceof CompoundPredicate) {
+ CompoundPredicate compoundPredicate = (CompoundPredicate) expr;
+
+ com.aliyun.odps.table.optimizer.predicate.CompoundPredicate.Operator odpsOp;
+ switch (compoundPredicate.getOp()) {
+ case AND:
+ odpsOp = com.aliyun.odps.table.optimizer.predicate.CompoundPredicate.Operator.AND;
+ break;
+ case OR:
+ odpsOp = com.aliyun.odps.table.optimizer.predicate.CompoundPredicate.Operator.OR;
+ break;
+ case NOT:
+ odpsOp = com.aliyun.odps.table.optimizer.predicate.CompoundPredicate.Operator.NOT;
+ break;
+ default:
+ throw new AnalysisException("Unknown operator: " + compoundPredicate.getOp());
+ }
+
+ List<Predicate> odpsPredicates = new ArrayList<>();
+
+ odpsPredicates.add(convertExprToOdpsPredicate(expr.getChild(0)));
+
+ if (compoundPredicate.getOp() != Operator.NOT) {
+ odpsPredicates.add(convertExprToOdpsPredicate(expr.getChild(1)));
+ }
+ odpsPredicate = new com.aliyun.odps.table.optimizer.predicate.CompoundPredicate(odpsOp, odpsPredicates);
+
+ } else if (expr instanceof InPredicate) {
+
+ InPredicate inPredicate = (InPredicate) expr;
+ if (inPredicate.getChildren().size() > 2) {
+ return Predicate.NO_PREDICATE;
+ }
+ com.aliyun.odps.table.optimizer.predicate.InPredicate.Operator odpsOp =
+ inPredicate.isNotIn()
+ ? com.aliyun.odps.table.optimizer.predicate.InPredicate.Operator.IN
+ : com.aliyun.odps.table.optimizer.predicate.InPredicate.Operator.NOT_IN;
+
+ String columnName = convertSlotRefToColumnName(expr.getChild(0));
+ com.aliyun.odps.OdpsType odpsType = table.getColumnNameToOdpsColumn().get(columnName).getType();
+
+ StringBuilder stringBuilder = new StringBuilder();
+
+
+ stringBuilder.append(columnName);
+ stringBuilder.append(" ");
+ stringBuilder.append(odpsOp.getDescription());
+ stringBuilder.append(" (");
+
+ for (int i = 1; i < inPredicate.getChildren().size(); i++) {
+ stringBuilder.append(convertLiteralToOdpsValues(odpsType, expr.getChild(i)));
+ if (i < inPredicate.getChildren().size() - 1) {
+ stringBuilder.append(", ");
+ }
+ }
+ stringBuilder.append(" )");
+
+ odpsPredicate = new com.aliyun.odps.table.optimizer.predicate.RawPredicate(stringBuilder.toString());
+
+ } else if (expr instanceof BinaryPredicate) {
+ BinaryPredicate binaryPredicate = (BinaryPredicate) expr;
+
+
+ com.aliyun.odps.table.optimizer.predicate.BinaryPredicate.Operator odpsOp;
+ switch (binaryPredicate.getOp()) {
+ case EQ: {
+ odpsOp = com.aliyun.odps.table.optimizer.predicate.BinaryPredicate.Operator.EQUALS;
+ break;
+ }
+ case NE: {
+ odpsOp = com.aliyun.odps.table.optimizer.predicate.BinaryPredicate.Operator.NOT_EQUALS;
+ break;
+ }
+ case GE: {
+ odpsOp = com.aliyun.odps.table.optimizer.predicate.BinaryPredicate.Operator.GREATER_THAN_OR_EQUAL;
+ break;
+ }
+ case LE: {
+ odpsOp = com.aliyun.odps.table.optimizer.predicate.BinaryPredicate.Operator.LESS_THAN_OR_EQUAL;
+ break;
+ }
+ case LT: {
+ odpsOp = com.aliyun.odps.table.optimizer.predicate.BinaryPredicate.Operator.LESS_THAN;
+ break;
+ }
+ case GT: {
+ odpsOp = com.aliyun.odps.table.optimizer.predicate.BinaryPredicate.Operator.GREATER_THAN;
+ break;
+ }
+ default: {
+ odpsOp = null;
+ break;
+ }
+ }
+
+ if (odpsOp != null) {
+ String columnName = convertSlotRefToColumnName(expr.getChild(0));
+ com.aliyun.odps.OdpsType odpsType = table.getColumnNameToOdpsColumn().get(columnName).getType();
+ StringBuilder stringBuilder = new StringBuilder();
+ stringBuilder.append(columnName);
+ stringBuilder.append(" ");
+ stringBuilder.append(odpsOp.getDescription());
+ stringBuilder.append(" ");
+ stringBuilder.append(convertLiteralToOdpsValues(odpsType, expr.getChild(1)));
+
+ odpsPredicate = new com.aliyun.odps.table.optimizer.predicate.RawPredicate(stringBuilder.toString());
+ }
+ } else if (expr instanceof IsNullPredicate) {
+ IsNullPredicate isNullPredicate = (IsNullPredicate) expr;
+ com.aliyun.odps.table.optimizer.predicate.UnaryPredicate.Operator odpsOp =
+ isNullPredicate.isNotNull()
+ ? com.aliyun.odps.table.optimizer.predicate.UnaryPredicate.Operator.NOT_NULL
+ : com.aliyun.odps.table.optimizer.predicate.UnaryPredicate.Operator.IS_NULL;
+
+ odpsPredicate = new com.aliyun.odps.table.optimizer.predicate.UnaryPredicate(odpsOp,
+ new com.aliyun.odps.table.optimizer.predicate.Attribute(
+ convertSlotRefToColumnName(expr.getChild(0))
+ )
+ );
+ }
+
+
+ if (odpsPredicate == null) {
+ throw new AnalysisException("Do not support convert ["
+ + expr.getExprName() + "] in convertExprToOdpsPredicate.");
+ }
+ return odpsPredicate;
+ }
+
+ private String convertSlotRefToColumnName(Expr expr) throws AnalysisException {
+ if (expr instanceof SlotRef) {
+ return ((SlotRef) expr).getColumnName();
+ } else if (expr instanceof CastExpr) {
+ if (expr.getChild(0) instanceof SlotRef) {
+ return ((SlotRef) expr.getChild(0)).getColumnName();
+ }
+ }
+
+ throw new AnalysisException("Do not support convert ["
+ + expr.getExprName() + "] in convertSlotRefToAttribute.");
+
+
+ }
+
+ private String convertLiteralToOdpsValues(OdpsType odpsType, Expr expr) throws AnalysisException {
+ if (!(expr instanceof LiteralExpr)) {
+ throw new AnalysisException("Do not support convert ["
+ + expr.getExprName() + "] in convertSlotRefToAttribute.");
+ }
+ LiteralExpr literalExpr = (LiteralExpr) expr;
+
+ literalExpr.toString();
Review Comment:
why call this here?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](MaxCompute)Refactoring maxCompute catalog using Storage API. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40225:
URL: https://github.com/apache/doris/pull/40225#issuecomment-2343032965
TeamCity be ut coverage result:
Function Coverage: 36.88% (9452/25628)
Line Coverage: 28.25% (77755/275202)
Region Coverage: 27.64% (40126/145150)
Branch Coverage: 24.26% (20391/84040)
Coverage Report: http://coverage.selectdb-in.cc/coverage/9d37687020a10441967fc000292259bcbf7f75dd_9d37687020a10441967fc000292259bcbf7f75dd/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](MaxCompute)Refactoring maxCompute catalog using Storage API. [doris]
Posted by "hubgeter (via GitHub)" <gi...@apache.org>.
hubgeter commented on PR #40225:
URL: https://github.com/apache/doris/pull/40225#issuecomment-2331433762
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](MaxCompute)Refactoring maxCompute catalog using Storage API. [doris]
Posted by "hubgeter (via GitHub)" <gi...@apache.org>.
hubgeter commented on PR #40225:
URL: https://github.com/apache/doris/pull/40225#issuecomment-2333175241
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](MaxCompute)Refactoring maxCompute catalog using Storage API. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40225:
URL: https://github.com/apache/doris/pull/40225#issuecomment-2333241015
<details>
<summary>TPC-H: <b>Total hot run time: 38029 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 04f750caf7f73efeb0902ec2a15ea7b9a6bc4294, data reload: false
------ Round 1 ----------------------------------
q1 17911 5323 4416 4416
q2 2561 194 188 188
q3 12126 1144 1134 1134
q4 10707 664 653 653
q5 7783 2942 2818 2818
q6 237 144 141 141
q7 970 643 602 602
q8 9299 2078 2094 2078
q9 7307 6540 6540 6540
q10 7022 2246 2195 2195
q11 465 240 242 240
q12 399 224 219 219
q13 17766 3077 3084 3077
q14 291 236 257 236
q15 555 477 479 477
q16 525 430 412 412
q17 989 737 698 698
q18 7405 6878 6971 6878
q19 1397 937 990 937
q20 679 348 338 338
q21 4777 2756 3050 2756
q22 1102 996 998 996
Total cold run time: 112273 ms
Total hot run time: 38029 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4358 4253 4297 4253
q2 380 279 274 274
q3 2911 2670 2651 2651
q4 1914 1640 1642 1640
q5 5422 5413 5442 5413
q6 227 133 135 133
q7 2139 1756 1755 1755
q8 3212 3374 3401 3374
q9 8447 8434 8411 8411
q10 3457 3197 3193 3193
q11 611 498 521 498
q12 778 578 616 578
q13 10517 3096 3060 3060
q14 300 273 288 273
q15 529 479 484 479
q16 527 496 463 463
q17 1810 1499 1470 1470
q18 7852 7617 7409 7409
q19 1681 1503 1497 1497
q20 2102 1870 1816 1816
q21 5404 5269 5212 5212
q22 1141 1035 1018 1018
Total cold run time: 65719 ms
Total hot run time: 54870 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](MaxCompute)Refactoring maxCompute catalog using Storage API. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40225:
URL: https://github.com/apache/doris/pull/40225#issuecomment-2323281437
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR)
Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website).
See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](MaxCompute)Refactoring maxCompute catalog using Storage API. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40225:
URL: https://github.com/apache/doris/pull/40225#issuecomment-2343041915
<details>
<summary>TPC-DS: <b>Total hot run time: 196793 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 9d37687020a10441967fc000292259bcbf7f75dd, data reload: false
query1 1276 902 920 902
query2 6313 1892 1860 1860
query3 10655 4065 4088 4065
query4 60440 25441 23393 23393
query5 4999 517 498 498
query6 411 165 166 165
query7 5625 303 293 293
query8 326 244 229 229
query9 7650 2504 2515 2504
query10 415 276 263 263
query11 17396 15156 15315 15156
query12 171 107 116 107
query13 1493 402 377 377
query14 10538 6610 7433 6610
query15 207 183 175 175
query16 6808 468 390 390
query17 1097 565 563 563
query18 1521 308 298 298
query19 197 146 156 146
query20 120 111 110 110
query21 208 107 108 107
query22 4708 4577 4461 4461
query23 34491 33361 33552 33361
query24 6003 2870 2904 2870
query25 509 401 418 401
query26 608 159 155 155
query27 1599 277 300 277
query28 3890 2057 2025 2025
query29 646 414 421 414
query30 243 154 155 154
query31 961 722 766 722
query32 68 57 51 51
query33 453 282 280 280
query34 873 471 472 471
query35 838 744 740 740
query36 1047 919 950 919
query37 143 84 81 81
query38 4003 3846 3914 3846
query39 1457 1396 1377 1377
query40 198 111 116 111
query41 45 49 46 46
query42 116 98 98 98
query43 499 468 460 460
query44 1110 749 732 732
query45 195 164 166 164
query46 1097 737 712 712
query47 1874 1773 1850 1773
query48 357 288 285 285
query49 758 440 453 440
query50 841 404 419 404
query51 7058 6939 6976 6939
query52 100 85 86 85
query53 256 178 178 178
query54 554 442 457 442
query55 75 71 73 71
query56 273 260 259 259
query57 1195 1090 1070 1070
query58 219 226 227 226
query59 2985 2837 2683 2683
query60 296 273 267 267
query61 104 99 105 99
query62 741 653 648 648
query63 219 185 183 183
query64 1371 727 660 660
query65 3273 3153 3167 3153
query66 652 344 342 342
query67 15741 15417 15316 15316
query68 1833 857 853 853
query69 417 323 315 315
query70 1114 1151 1083 1083
query71 342 340 332 332
query72 4613 3601 3468 3468
query73 589 568 585 568
query74 9105 8937 8942 8937
query75 3001 2943 2974 2943
query76 967 860 836 836
query77 402 424 409 409
query78 9321 9718 9233 9233
query79 907 881 862 862
query80 827 802 775 775
query81 456 262 267 262
query82 260 263 262 262
query83 247 193 185 185
query84 199 109 107 107
query85 595 393 389 389
query86 328 321 291 291
query87 4532 4410 4486 4410
query88 4136 4016 4016 4016
query89 372 363 361 361
query90 805 306 302 302
query91 124 130 124 124
query92 76 73 77 73
query93 903 904 903 903
query94 396 354 361 354
query95 417 404 408 404
query96 466 460 466 460
query97 3097 3098 3134 3098
query98 228 233 229 229
query99 1263 1288 1287 1287
Total cold run time: 294946 ms
Total hot run time: 196793 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](MaxCompute)Refactoring maxCompute catalog using Storage API. [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #40225:
URL: https://github.com/apache/doris/pull/40225#issuecomment-2343986457
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](MaxCompute)Refactoring maxCompute catalog using Storage API. [doris]
Posted by "morningman (via GitHub)" <gi...@apache.org>.
morningman commented on code in PR #40225:
URL: https://github.com/apache/doris/pull/40225#discussion_r1741557843
##########
fe/fe-core/src/main/java/org/apache/doris/datasource/maxcompute/MaxComputeExternalCatalog.java:
##########
@@ -45,87 +50,65 @@
public class MaxComputeExternalCatalog extends ExternalCatalog {
private Odps odps;
- private TableTunnel tunnel;
- @SerializedName(value = "region")
- private String region;
+
@SerializedName(value = "accessKey")
private String accessKey;
Review Comment:
I think these 2 fields no need to persist.
All property should be persisited in CatalogProperty
##########
fe/fe-core/src/main/java/org/apache/doris/datasource/maxcompute/MaxComputeExternalCatalog.java:
##########
@@ -45,87 +50,65 @@
public class MaxComputeExternalCatalog extends ExternalCatalog {
private Odps odps;
- private TableTunnel tunnel;
- @SerializedName(value = "region")
- private String region;
+
@SerializedName(value = "accessKey")
private String accessKey;
@SerializedName(value = "secretKey")
private String secretKey;
- @SerializedName(value = "publicAccess")
- private boolean enablePublicAccess;
- private static final String odpsUrlTemplate = "http://service.{}.maxcompute.aliyun-inc.com/api";
- private static final String tunnelUrlTemplate = "http://dt.{}.maxcompute.aliyun-inc.com";
- private static String odpsUrl;
- private static String tunnelUrl;
+ private String endpoint;
+ private String catalogOwner = "";
+ private String defaultProject;
+ String quota;
+
+ public EnvironmentSettings settings;
+
private static final List<String> REQUIRED_PROPERTIES = ImmutableList.of(
- MCProperties.REGION,
- MCProperties.PROJECT
+ MCProperties.PROJECT,
+ MCProperties.QUOTA,
+ MCProperties.ENDPOINT
);
public MaxComputeExternalCatalog(long catalogId, String name, String resource, Map<String, String> props,
String comment) {
super(catalogId, name, InitCatalogLog.Type.MAX_COMPUTE, comment);
catalogProperty = new CatalogProperty(resource, props);
- odpsUrl = props.getOrDefault(MCProperties.ODPS_ENDPOINT, "");
- tunnelUrl = props.getOrDefault(MCProperties.TUNNEL_SDK_ENDPOINT, "");
}
@Override
protected void initLocalObjectsImpl() {
Map<String, String> props = catalogProperty.getProperties();
- String region = props.get(MCProperties.REGION);
- String defaultProject = props.get(MCProperties.PROJECT);
- if (Strings.isNullOrEmpty(region)) {
- throw new IllegalArgumentException("Missing required property '" + MCProperties.REGION + "'.");
- }
+
+ defaultProject = props.get(MCProperties.PROJECT);
+ quota = props.get(MCProperties.QUOTA);
+ endpoint = props.getOrDefault(MCProperties.ENDPOINT, "");
+
if (Strings.isNullOrEmpty(defaultProject)) {
throw new IllegalArgumentException("Missing required property '" + MCProperties.PROJECT + "'.");
}
- if (region.startsWith("oss-")) {
- // may use oss-cn-beijing, ensure compatible
- region = region.replace("oss-", "");
+ if (Strings.isNullOrEmpty(quota)) {
Review Comment:
Is there a default value for QUOTA？
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](MaxCompute)Refactoring maxCompute catalog using Storage API. [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #40225:
URL: https://github.com/apache/doris/pull/40225#issuecomment-2331483928
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](MaxCompute)Refactoring maxCompute catalog using Storage API. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40225:
URL: https://github.com/apache/doris/pull/40225#issuecomment-2331795708
<details>
<summary>ClickBench: <b>Total hot run time: 32.4 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 7e229aaf10594938d9bb8ff159deb52d1d673575, data reload: false
query1 0.05 0.04 0.04
query2 0.08 0.04 0.04
query3 0.23 0.06 0.06
query4 1.66 0.08 0.08
query5 0.52 0.50 0.51
query6 1.12 0.72 0.72
query7 0.02 0.02 0.01
query8 0.05 0.04 0.04
query9 0.55 0.50 0.50
query10 0.54 0.57 0.55
query11 0.16 0.12 0.12
query12 0.15 0.12 0.12
query13 0.59 0.60 0.59
query14 2.09 2.04 2.04
query15 0.86 0.83 0.80
query16 0.38 0.37 0.38
query17 1.07 1.05 1.07
query18 0.22 0.20 0.21
query19 1.94 1.89 1.84
query20 0.02 0.01 0.01
query21 15.39 0.67 0.66
query22 4.09 6.46 2.17
query23 18.22 1.39 1.32
query24 2.11 0.23 0.23
query25 0.16 0.09 0.08
query26 0.26 0.17 0.17
query27 0.08 0.08 0.07
query28 13.25 1.03 1.00
query29 12.62 3.33 3.30
query30 0.24 0.05 0.06
query31 2.88 0.42 0.39
query32 3.25 0.49 0.47
query33 2.98 3.00 3.01
query34 16.90 4.33 4.37
query35 4.45 4.43 4.40
query36 0.66 0.47 0.48
query37 0.19 0.16 0.16
query38 0.16 0.15 0.15
query39 0.05 0.04 0.04
query40 0.16 0.13 0.12
query41 0.10 0.04 0.05
query42 0.06 0.05 0.05
query43 0.06 0.05 0.05
Total cold run time: 110.62 s
Total hot run time: 32.4 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](MaxCompute)Refactoring maxCompute catalog using Storage API. [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #40225:
URL: https://github.com/apache/doris/pull/40225#issuecomment-2329519356
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](MaxCompute)Refactoring maxCompute catalog using Storage API. [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #40225:
URL: https://github.com/apache/doris/pull/40225#issuecomment-2331323586
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](MaxCompute)Refactoring maxCompute catalog using Storage API. [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #40225:
URL: https://github.com/apache/doris/pull/40225#issuecomment-2343986373
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](MaxCompute)Refactoring maxCompute catalog using Storage API. [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #40225:
URL: https://github.com/apache/doris/pull/40225#issuecomment-2323283930
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](MaxCompute)Refactoring maxCompute catalog using Storage API. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40225:
URL: https://github.com/apache/doris/pull/40225#issuecomment-2323324118
<details>
<summary>TPC-DS: <b>Total hot run time: 193561 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 9b818bedb956a247985af6cb0fbdfe19079b24d2, data reload: false
query1 1283 882 874 874
query2 6264 1979 1989 1979
query3 10754 4151 4050 4050
query4 59959 25060 23137 23137
query5 5472 516 499 499
query6 414 166 169 166
query7 5790 292 287 287
query8 294 214 222 214
query9 8974 2518 2521 2518
query10 504 277 270 270
query11 17929 15018 15228 15018
query12 160 104 103 103
query13 1564 407 410 407
query14 11154 7399 7408 7399
query15 238 183 182 182
query16 7308 496 491 491
query17 1106 562 579 562
query18 1961 298 301 298
query19 302 151 150 150
query20 121 113 113 113
query21 215 106 101 101
query22 4920 4546 4482 4482
query23 34603 33662 33611 33611
query24 5939 2903 2848 2848
query25 547 426 376 376
query26 680 153 150 150
query27 1785 273 283 273
query28 3796 2129 2121 2121
query29 663 398 405 398
query30 239 149 155 149
query31 897 752 755 752
query32 80 54 59 54
query33 428 283 284 283
query34 859 473 479 473
query35 829 697 708 697
query36 1035 935 932 932
query37 148 98 94 94
query38 3918 3849 3946 3849
query39 1472 1379 1382 1379
query40 197 113 115 113
query41 46 46 45 45
query42 120 97 95 95
query43 508 476 489 476
query44 1103 747 742 742
query45 198 167 163 163
query46 1097 733 750 733
query47 1939 1809 1841 1809
query48 367 294 296 294
query49 753 426 455 426
query50 828 402 404 402
query51 7180 7113 7095 7095
query52 96 85 86 85
query53 247 180 180 180
query54 555 439 450 439
query55 76 76 76 76
query56 280 258 255 255
query57 1188 1068 1072 1068
query58 217 234 232 232
query59 3072 2775 2911 2775
query60 296 265 259 259
query61 99 99 104 99
query62 730 661 668 661
query63 221 184 185 184
query64 2839 682 677 677
query65 3220 3145 3166 3145
query66 674 359 345 345
query67 15564 15230 15275 15230
query68 3009 573 578 573
query69 407 277 273 273
query70 1133 1124 1140 1124
query71 365 278 275 275
query72 5374 4008 3968 3968
query73 763 329 332 329
query74 9064 8783 8884 8783
query75 3369 2698 2717 2698
query76 1557 1017 993 993
query77 579 385 314 314
query78 11226 9429 9096 9096
query79 1038 547 540 540
query80 682 495 511 495
query81 471 233 226 226
query82 249 147 146 146
query83 171 148 148 148
query84 264 77 74 74
query85 671 283 280 280
query86 310 296 304 296
query87 4423 4256 4315 4256
query88 2962 2308 2305 2305
query89 370 292 289 289
query90 1929 194 188 188
query91 123 100 103 100
query92 62 52 51 51
query93 1035 536 535 535
query94 699 294 295 294
query95 333 262 329 262
query96 601 265 267 265
query97 3203 3089 3113 3089
query98 212 210 196 196
query99 1520 1258 1258 1258
Total cold run time: 306488 ms
Total hot run time: 193561 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](MaxCompute)Refactoring maxCompute catalog using Storage API. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40225:
URL: https://github.com/apache/doris/pull/40225#issuecomment-2331808051
TeamCity be ut coverage result:
Function Coverage: 36.83% (9395/25509)
Line Coverage: 28.27% (77499/274107)
Region Coverage: 27.66% (39990/144559)
Branch Coverage: 24.30% (20347/83730)
Coverage Report: http://coverage.selectdb-in.cc/coverage/7e229aaf10594938d9bb8ff159deb52d1d673575_7e229aaf10594938d9bb8ff159deb52d1d673575/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](MaxCompute)Refactoring maxCompute catalog using Storage API. [doris]
Posted by "hubgeter (via GitHub)" <gi...@apache.org>.
hubgeter commented on PR #40225:
URL: https://github.com/apache/doris/pull/40225#issuecomment-2329503403
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](MaxCompute)Refactoring maxCompute catalog using Storage API. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40225:
URL: https://github.com/apache/doris/pull/40225#issuecomment-2329698397
<details>
<summary>ClickBench: <b>Total hot run time: 32.58 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 8cb3173c7b8f8c032b956ab11d39828305edc1a1, data reload: false
query1 0.05 0.04 0.04
query2 0.08 0.04 0.04
query3 0.23 0.05 0.06
query4 1.67 0.08 0.10
query5 0.51 0.48 0.50
query6 1.14 0.74 0.73
query7 0.02 0.01 0.01
query8 0.05 0.04 0.04
query9 0.55 0.48 0.48
query10 0.53 0.54 0.56
query11 0.15 0.11 0.11
query12 0.14 0.12 0.12
query13 0.60 0.58 0.58
query14 2.11 2.09 2.05
query15 0.89 0.83 0.81
query16 0.38 0.36 0.37
query17 1.00 0.99 1.03
query18 0.22 0.20 0.20
query19 1.87 1.71 1.79
query20 0.01 0.01 0.01
query21 15.42 0.67 0.68
query22 4.06 6.64 2.46
query23 18.26 1.37 1.33
query24 2.08 0.22 0.22
query25 0.15 0.09 0.08
query26 0.27 0.18 0.18
query27 0.07 0.08 0.08
query28 13.28 1.02 1.00
query29 12.63 3.36 3.33
query30 0.24 0.05 0.06
query31 2.87 0.40 0.39
query32 3.25 0.50 0.47
query33 2.98 3.00 3.11
query34 17.05 4.40 4.39
query35 4.44 4.48 4.42
query36 0.66 0.48 0.48
query37 0.20 0.15 0.16
query38 0.15 0.15 0.15
query39 0.05 0.04 0.04
query40 0.16 0.13 0.12
query41 0.09 0.05 0.05
query42 0.06 0.05 0.05
query43 0.04 0.04 0.04
Total cold run time: 110.66 s
Total hot run time: 32.58 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](MaxCompute)Refactoring maxCompute catalog using Storage API. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40225:
URL: https://github.com/apache/doris/pull/40225#issuecomment-2334134588
<details>
<summary>ClickBench: <b>Total hot run time: 31.26 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 0dd25050fa300530f5b1c949219d15a95dfe75cd, data reload: false
query1 0.04 0.04 0.04
query2 0.08 0.05 0.04
query3 0.23 0.05 0.06
query4 1.65 0.09 0.11
query5 0.52 0.48 0.49
query6 1.13 0.73 0.72
query7 0.02 0.01 0.01
query8 0.05 0.04 0.05
query9 0.54 0.49 0.50
query10 0.55 0.55 0.56
query11 0.15 0.12 0.12
query12 0.16 0.12 0.12
query13 0.59 0.58 0.59
query14 1.42 1.43 1.41
query15 0.84 0.81 0.81
query16 0.38 0.38 0.38
query17 1.06 1.06 1.06
query18 0.20 0.20 0.21
query19 1.86 1.73 1.72
query20 0.01 0.01 0.01
query21 15.40 0.69 0.68
query22 4.94 6.62 1.76
query23 18.32 1.36 1.33
query24 2.08 0.24 0.21
query25 0.14 0.08 0.08
query26 0.27 0.18 0.17
query27 0.09 0.08 0.08
query28 13.23 1.02 1.00
query29 12.64 3.35 3.34
query30 0.25 0.05 0.06
query31 2.86 0.39 0.40
query32 3.27 0.46 0.50
query33 2.94 2.96 3.05
query34 17.20 4.37 4.40
query35 4.39 4.47 4.41
query36 0.65 0.46 0.47
query37 0.19 0.15 0.15
query38 0.15 0.14 0.14
query39 0.05 0.04 0.04
query40 0.15 0.13 0.13
query41 0.09 0.04 0.04
query42 0.06 0.05 0.05
query43 0.05 0.04 0.05
Total cold run time: 110.89 s
Total hot run time: 31.26 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](MaxCompute)Refactoring maxCompute catalog using Storage API. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40225:
URL: https://github.com/apache/doris/pull/40225#issuecomment-2327147312
<details>
<summary>TPC-H: <b>Total hot run time: 38454 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 40fada4cdb82562d755cac8c2d74d64533a77c2a, data reload: false
------ Round 1 ----------------------------------
q1 17706 4613 4376 4376
q2 2024 191 174 174
q3 11222 970 1122 970
q4 10383 737 785 737
q5 7753 2921 2852 2852
q6 233 145 140 140
q7 971 615 616 615
q8 9329 2129 2129 2129
q9 7300 6570 6647 6570
q10 7014 2230 2168 2168
q11 448 240 249 240
q12 406 227 227 227
q13 17767 3117 3070 3070
q14 277 236 233 233
q15 538 506 506 506
q16 595 514 490 490
q17 999 679 737 679
q18 7583 6877 6782 6782
q19 1389 1078 1042 1042
q20 711 344 347 344
q21 3944 3184 3061 3061
q22 1138 1056 1049 1049
Total cold run time: 109730 ms
Total hot run time: 38454 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4432 4353 4351 4351
q2 389 273 272 272
q3 2953 2647 2681 2647
q4 1974 1654 1655 1654
q5 5690 5699 5793 5699
q6 231 144 141 141
q7 2235 1813 1845 1813
q8 3335 3445 3505 3445
q9 8948 8905 8847 8847
q10 3600 3434 3384 3384
q11 620 527 531 527
q12 828 636 654 636
q13 16399 3270 3291 3270
q14 338 311 316 311
q15 552 486 488 486
q16 633 577 581 577
q17 1868 1561 1591 1561
q18 8232 7915 7966 7915
q19 1758 1645 1582 1582
q20 2161 1953 1992 1953
q21 5818 5465 5490 5465
q22 1144 1037 1065 1037
Total cold run time: 74138 ms
Total hot run time: 57573 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](MaxCompute)Refactoring maxCompute catalog using Storage API. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40225:
URL: https://github.com/apache/doris/pull/40225#issuecomment-2337073824
<details>
<summary>ClickBench: <b>Total hot run time: 31.55 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 822a4f444e9b464f1cb62b0a04fab519da076acd, data reload: false
query1 0.04 0.04 0.04
query2 0.08 0.04 0.05
query3 0.23 0.06 0.05
query4 1.66 0.08 0.08
query5 0.51 0.51 0.49
query6 1.13 0.72 0.71
query7 0.02 0.02 0.01
query8 0.05 0.05 0.05
query9 0.54 0.48 0.48
query10 0.54 0.56 0.53
query11 0.16 0.12 0.12
query12 0.15 0.12 0.12
query13 0.59 0.59 0.58
query14 1.48 1.44 1.44
query15 0.86 0.81 0.83
query16 0.37 0.36 0.37
query17 1.02 0.99 1.01
query18 0.20 0.20 0.21
query19 1.89 1.89 1.78
query20 0.02 0.01 0.01
query21 15.40 0.69 0.67
query22 3.94 6.99 2.01
query23 18.27 1.49 1.39
query24 2.11 0.22 0.21
query25 0.14 0.09 0.08
query26 0.27 0.18 0.18
query27 0.07 0.08 0.07
query28 13.34 1.01 1.00
query29 12.58 3.33 3.28
query30 0.24 0.06 0.05
query31 2.88 0.39 0.40
query32 3.26 0.48 0.48
query33 2.98 3.04 3.01
query34 17.03 4.38 4.47
query35 4.43 4.44 4.39
query36 0.66 0.47 0.49
query37 0.19 0.15 0.15
query38 0.16 0.14 0.15
query39 0.06 0.04 0.04
query40 0.16 0.14 0.13
query41 0.09 0.05 0.05
query42 0.06 0.05 0.05
query43 0.04 0.04 0.05
Total cold run time: 109.9 s
Total hot run time: 31.55 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](MaxCompute)Refactoring maxCompute catalog using Storage API. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40225:
URL: https://github.com/apache/doris/pull/40225#issuecomment-2334122824
<details>
<summary>TPC-DS: <b>Total hot run time: 192840 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 0dd25050fa300530f5b1c949219d15a95dfe75cd, data reload: false
query1 1264 885 899 885
query2 6362 1933 1926 1926
query3 10618 3978 4084 3978
query4 59964 26199 23071 23071
query5 5446 500 500 500
query6 404 172 157 157
query7 5766 308 285 285
query8 321 213 209 209
query9 8686 2495 2482 2482
query10 492 260 270 260
query11 16894 15028 15292 15028
query12 166 100 100 100
query13 1558 396 415 396
query14 11220 7418 7368 7368
query15 231 178 178 178
query16 7677 471 492 471
query17 1099 565 562 562
query18 2100 317 288 288
query19 305 144 146 144
query20 115 112 114 112
query21 202 104 105 104
query22 4733 4343 4491 4343
query23 34195 33578 33368 33368
query24 5960 2857 2851 2851
query25 538 428 403 403
query26 687 160 153 153
query27 1785 273 280 273
query28 3847 2052 2030 2030
query29 681 429 429 429
query30 238 153 164 153
query31 934 747 773 747
query32 101 56 57 56
query33 466 289 289 289
query34 859 478 471 471
query35 831 714 733 714
query36 1081 937 912 912
query37 149 87 88 87
query38 3955 3815 3929 3815
query39 1431 1381 1422 1381
query40 200 121 116 116
query41 48 46 48 46
query42 124 100 96 96
query43 514 455 468 455
query44 1078 756 744 744
query45 197 166 173 166
query46 1075 739 734 734
query47 1874 1787 1814 1787
query48 370 296 295 295
query49 793 470 458 458
query50 825 433 418 418
query51 7091 6875 6987 6875
query52 100 88 85 85
query53 249 183 180 180
query54 572 457 460 457
query55 79 74 77 74
query56 282 257 280 257
query57 1178 1067 1094 1067
query58 244 244 243 243
query59 3120 2855 2754 2754
query60 308 274 283 274
query61 123 123 125 123
query62 769 673 650 650
query63 220 189 182 182
query64 2946 773 742 742
query65 3395 3144 3131 3131
query66 687 352 349 349
query67 15894 15350 15240 15240
query68 4380 574 555 555
query69 439 276 280 276
query70 1089 1140 1113 1113
query71 372 274 270 270
query72 6554 4069 4045 4045
query73 742 318 321 318
query74 9175 8849 9008 8849
query75 3377 2715 2711 2711
query76 1654 1049 972 972
query77 550 328 312 312
query78 10846 9279 9343 9279
query79 2140 538 533 533
query80 1071 509 534 509
query81 560 231 229 229
query82 853 140 139 139
query83 190 150 146 146
query84 251 73 84 73
query85 794 285 272 272
query86 442 278 297 278
query87 4286 4310 4252 4252
query88 3163 2285 2302 2285
query89 393 280 283 280
query90 1764 193 194 193
query91 126 121 103 103
query92 69 52 55 52
query93 2217 538 542 538
query94 808 298 286 286
query95 351 247 249 247
query96 596 266 267 266
query97 3215 3057 3066 3057
query98 255 202 195 195
query99 1560 1258 1230 1230
Total cold run time: 311998 ms
Total hot run time: 192840 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](MaxCompute)Refactoring maxCompute catalog using Storage API. [doris]
Posted by "hubgeter (via GitHub)" <gi...@apache.org>.
hubgeter commented on PR #40225:
URL: https://github.com/apache/doris/pull/40225#issuecomment-2333903701
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](MaxCompute)Refactoring maxCompute catalog using Storage API. [doris]
Posted by "morningman (via GitHub)" <gi...@apache.org>.
morningman commented on code in PR #40225:
URL: https://github.com/apache/doris/pull/40225#discussion_r1751444830
##########
regression-test/suites/external_table_p2/maxcompute/test_max_compute_complex_type.groovy:
##########
@@ -142,7 +151,7 @@
)
);
*/
-suite("test_max_compute_complex_type", "p0,external,doris,external_docker,external_docker_doris") {
+suite("test_max_compute_complex_type", "p2,external,doris,external_docker,external_docker_doris") {
Review Comment:
p2,external,maxcompute,external_remote,external_remote_maxcompute
##########
regression-test/suites/external_table_p2/maxcompute/test_external_catalog_maxcompute.groovy:
##########
@@ -14,8 +14,273 @@
// KIND, either express or implied. See the License for the
// specific language governing permissions and limitations
// under the License.
+/*
-suite("test_external_catalog_maxcompute", "p2,external,maxcompute,external_remote,external_remote_maxcompute") {
+
+ // Test DDL and Data:
+
+ drop table `multi_partitions`;
+ CREATE TABLE `multi_partitions` (
+ `city` string,
+ `gender` boolean,
+ `mnt` smallint,
+ `order_rate` float,
+ `amount` decimal(24,9),
+ `cut_date` date,
+ `create_time` datetime,
+ `finished_time` timestamp_ntz
+ ) PARTITIONED BY (
+ `yy` string,
+ `mm` string,
+ `dd` string,
+ `pt` bigint
+ );
+ INSERT INTO `multi_partitions` PARTITION (yy='2023', mm='08', dd='01', pt=1) VALUES
+ ('New York', TRUE, CAST(12 AS SMALLINT), CAST(0.75 AS FLOAT), CAST(1234.567890123 AS DECIMAL(24,9)),
+ CAST('2023-08-01' AS DATE), CAST('2023-08-01 10:30:00' AS DATETIME), CAST('2023-08-01 11:00:00' AS timestamp_ntz));
+ INSERT INTO `multi_partitions` PARTITION (yy='2023', mm='08', dd='01', pt=1) VALUES
+ ('Los Angeles', FALSE, CAST(10 AS SMALLINT), CAST(1.15 AS FLOAT), CAST(9876.543210987 AS DECIMAL(24,9)),
+ CAST('2023-08-01' AS DATE), CAST('2023-08-01 12:00:00' AS DATETIME), CAST('2023-08-01 12:30:00' AS timestamp_ntz));
+ INSERT INTO `multi_partitions` PARTITION (yy='2023', mm='08', dd='02', pt=2) VALUES
+ ('Chicago', TRUE, CAST(8 AS SMALLINT), CAST(0.90 AS FLOAT), CAST(5555.123456789 AS DECIMAL(24,9)),
+ CAST('2023-08-02' AS DATE), CAST('2023-08-02 08:15:00' AS DATETIME), CAST('2023-08-02 08:45:00' AS timestamp_ntz));
+ INSERT INTO `multi_partitions` PARTITION (yy='2023', mm='08', dd='02', pt=2) VALUES
+ ('Houston', FALSE, CAST(15 AS SMALLINT), CAST(1.25 AS FLOAT), CAST(2222.987654321 AS DECIMAL(24,9)),
+ CAST('2023-08-02' AS DATE), CAST('2023-08-02 14:45:00' AS DATETIME), CAST('2023-08-02 15:15:00' AS timestamp_ntz));
+ INSERT INTO `multi_partitions` PARTITION (yy='2023', mm='08', dd='03', pt=3) VALUES
+ ('Phoenix', TRUE, CAST(5 AS SMALLINT), CAST(0.50 AS FLOAT), CAST(7777.333333333 AS DECIMAL(24,9)),
+ CAST('2023-08-03' AS DATE), CAST('2023-08-03 09:00:00' AS DATETIME), CAST('2023-08-03 09:30:00' AS timestamp_ntz));
+ INSERT INTO `multi_partitions` PARTITION (yy='2023', mm='08', dd='03', pt=3) VALUES
+ ('Philadelphia', FALSE, CAST(7 AS SMALLINT), CAST(0.85 AS FLOAT), CAST(8888.222222222 AS DECIMAL(24,9)),
+ CAST('2023-08-03' AS DATE), CAST('2023-08-03 11:30:00' AS DATETIME), CAST('2023-08-03 12:00:00' AS timestamp_ntz));
+ INSERT INTO `multi_partitions` PARTITION (yy='2023', mm='08', dd='04', pt=4) VALUES
+ ('San Antonio', TRUE, CAST(9 AS SMALLINT), CAST(1.05 AS FLOAT), CAST(3333.666666666 AS DECIMAL(24,9)),
+ CAST('2023-08-04' AS DATE), CAST('2023-08-04 07:00:00' AS DATETIME), CAST('2023-08-04 07:30:00' AS timestamp_ntz));
+ INSERT INTO `multi_partitions` PARTITION (yy='2023', mm='08', dd='04', pt=4) VALUES
+ ('San Diego', FALSE, CAST(11 AS SMALLINT), CAST(1.10 AS FLOAT), CAST(4444.555555555 AS DECIMAL(24,9)),
+ CAST('2023-08-04' AS DATE), CAST('2023-08-04 10:00:00' AS DATETIME), CAST('2023-08-04 10:30:00' AS timestamp_ntz));
+ INSERT INTO `multi_partitions` PARTITION (yy='2023', mm='08', dd='05', pt=5) VALUES
+ ('Dallas', TRUE, CAST(6 AS SMALLINT), CAST(0.65 AS FLOAT), CAST(6666.444444444 AS DECIMAL(24,9)),
+ CAST('2023-08-05' AS DATE), CAST('2023-08-05 13:00:00' AS DATETIME), CAST('2023-08-05 13:30:00' AS timestamp_ntz));
+ INSERT INTO `multi_partitions` PARTITION (yy='2023', mm='08', dd='05', pt=5) VALUES
+ ('San Jose', FALSE, CAST(14 AS SMALLINT), CAST(1.20 AS FLOAT), CAST(9999.111111111 AS DECIMAL(24,9)),
+ CAST('2023-08-05' AS DATE), CAST('2023-08-05 15:30:00' AS DATETIME), CAST('2023-08-05 16:00:00' AS timestamp_ntz));
+ drop table mc_parts;
+ CREATE TABLE `mc_parts` (
+ `mc_bigint` bigint,
+ `mc_string` string
+ )PARTITIONED BY (
+ `dt` string
+ );
+ INSERT INTO `mc_parts` PARTITION (dt='2023-08-01') VALUES
+ (1001, 'Sample data 1');
+ INSERT INTO `mc_parts` PARTITION (dt='2023-08-02') VALUES
+ (1002, 'Sample data 2');
+ INSERT INTO `mc_parts` PARTITION (dt='2023-08-03') VALUES
+ (1003, 'Sample data 3');
+ INSERT INTO `mc_parts` PARTITION (dt='2023-08-04') VALUES
+ (1004, 'Sample data 4');
+ INSERT INTO `mc_parts` PARTITION (dt='2023-08-05') VALUES
+ (1005, 'Sample data 5');
+ CREATE TABLE int_types (
+ mc_boolean BOOLEAN,
+ mc_tinyint TINYINT,
+ mc_int SMALLINT,
+ mc_bigint BIGINT
+ );
+ INSERT INTO int_types VALUES (TRUE, CAST(-128 AS TINYINT), CAST(-32768 AS SMALLINT), CAST(-9223372036854775807 AS BIGINT));
+ INSERT INTO int_types VALUES (FALSE, CAST(127 AS TINYINT), CAST(32767 AS SMALLINT), CAST(9223372036854775807 AS BIGINT));
+ INSERT INTO int_types VALUES (TRUE, CAST(0 AS TINYINT), CAST(0 AS SMALLINT), CAST(0 AS BIGINT));
+ INSERT INTO int_types VALUES (FALSE, CAST(0 AS TINYINT), CAST(0 AS SMALLINT), CAST(0 AS BIGINT));
+ INSERT INTO int_types VALUES (TRUE, CAST(1 AS TINYINT), CAST(1 AS SMALLINT), CAST(1 AS BIGINT));
+ INSERT INTO int_types VALUES (FALSE, CAST(-1 AS TINYINT), CAST(-1 AS SMALLINT), CAST(-1 AS BIGINT));
+ INSERT INTO int_types VALUES
+ (TRUE, CAST(1 AS TINYINT), CAST(100 AS SMALLINT), CAST(1000 AS BIGINT)),
+ (FALSE, CAST(2 AS TINYINT), CAST(200 AS SMALLINT), CAST(2000 AS BIGINT)),
+ (TRUE, CAST(3 AS TINYINT), CAST(300 AS SMALLINT), CAST(3000 AS BIGINT)),
+ (FALSE, CAST(4 AS TINYINT), CAST(400 AS SMALLINT), CAST(4000 AS BIGINT)),
+ (TRUE, CAST(5 AS TINYINT), CAST(500 AS SMALLINT), CAST(5000 AS BIGINT)),
+ (FALSE, CAST(6 AS TINYINT), CAST(600 AS SMALLINT), CAST(6000 AS BIGINT)),
+ (TRUE, CAST(7 AS TINYINT), CAST(700 AS SMALLINT), CAST(7000 AS BIGINT)),
+ (FALSE, CAST(8 AS TINYINT), CAST(800 AS SMALLINT), CAST(8000 AS BIGINT)),
+ (TRUE, CAST(9 AS TINYINT), CAST(900 AS SMALLINT), CAST(9000 AS BIGINT)),
+ (FALSE, CAST(10 AS TINYINT), CAST(1000 AS SMALLINT), CAST(10000 AS BIGINT));
+ CREATE TABLE web_site (
+ web_site_sk BIGINT,
+ web_site_id STRING,
+ web_rec_start_date DATE,
+ web_rec_end_date DATE,
+ web_name STRING,
+ web_open_date_sk BIGINT,
+ web_close_date_sk BIGINT,
+ web_class STRING,
+ web_manager STRING,
+ web_mkt_id INT,
+ web_mkt_class STRING,
+ web_mkt_desc STRING,
+ web_market_manager STRING,
+ web_company_id INT,
+ web_company_name STRING,
+ web_street_number STRING,
+ web_street_name STRING,
+ web_street_type STRING,
+ web_suite_number STRING,
+ web_city STRING,
+ web_county STRING,
+ web_state STRING,
+ web_zip STRING,
+ web_country STRING,
+ web_gmt_offset DOUBLE,
+ web_tax_percentage DECIMAL(5,2)
+ );
+ INSERT INTO web_site VALUES
+ (
+ CAST(1 AS BIGINT),
+ 'WS0001',
+ CAST('2023-01-01' AS DATE),
+ CAST('2023-12-31' AS DATE),
+ 'Example Web Site 1',
+ CAST(20230101 AS BIGINT),
+ CAST(20231231 AS BIGINT),
+ 'E-commerce',
+ 'John Doe',
+ CAST(101 AS INT),
+ 'Retail',
+ 'Online retail website',
+ 'Jane Smith',
+ CAST(201 AS INT),
+ 'Example Company',
+ '1234',
+ 'Main Street',
+ 'Apt',
+ 'Unit 101',
+ 'Metropolis',
+ 'County',
+ 'NY',
+ '12345',
+ 'USA',
+ CAST(-5.0 AS DOUBLE),
+ CAST(8.25 AS DECIMAL(5,2))
+ );
+ INSERT INTO web_site VALUES
+ (
+ CAST(2 AS BIGINT),
+ 'WS0002',
+ CAST('2023-02-01' AS DATE),
+ CAST('2023-11-30' AS DATE),
+ 'Example Web Site 2',
+ CAST(20230201 AS BIGINT),
+ CAST(20231130 AS BIGINT),
+ 'Technology',
+ 'Alice Johnson',
+ CAST(102 AS INT),
+ 'Tech',
+ 'Tech news and reviews',
+ 'Bob Brown',
+ CAST(202 AS INT),
+ 'Tech Innovations',
+ '5678',
+ 'Tech Avenue',
+ 'Suite',
+ 'Suite 200',
+ 'Gotham',
+ 'County',
+ 'CA',
+ '67890',
+ 'USA',
+ CAST(-8.0 AS DOUBLE),
+ CAST(7.50 AS DECIMAL(5,2))
+ );
+ INSERT INTO web_site VALUES
+ (
+ CAST(3 AS BIGINT),
+ 'WS0003',
+ CAST('2023-03-01' AS DATE),
+ CAST('2023-10-31' AS DATE),
+ 'Example Web Site 3',
+ CAST(20230301 AS BIGINT),
+ CAST(20231031 AS BIGINT),
+ 'Healthcare',
+ 'Robert White',
+ CAST(103 AS INT),
+ 'Health',
+ 'Healthcare services and products',
+ 'Emily Green',
+ CAST(203 AS INT),
+ 'Health Corp',
+ '9101',
+ 'Health Drive',
+ 'Floor',
+ 'Floor 3',
+ 'Star City',
+ 'County',
+ 'TX',
+ '23456',
+ 'USA',
+ CAST(-6.0 AS DOUBLE),
+ CAST(6.75 AS DECIMAL(5,2))
+ );
+ INSERT INTO web_site VALUES
+ (
+ CAST(4 AS BIGINT),
+ 'WS0004',
+ CAST('2023-04-01' AS DATE),
+ CAST('2023-09-30' AS DATE),
+ 'Example Web Site 4',
+ CAST(20230401 AS BIGINT),
+ CAST(20230930 AS BIGINT),
+ 'Education',
+ 'David Black',
+ CAST(104 AS INT),
+ 'EdTech',
+ 'Educational technology platform',
+ 'Fiona Grey',
+ CAST(204 AS INT),
+ 'Edu Tech',
+ '1122',
+ 'Education Lane',
+ 'Building',
+ 'Building 1',
+ 'Smallville',
+ 'County',
+ 'FL',
+ '34567',
+ 'USA',
+ CAST(-4.0 AS DOUBLE),
+ CAST(5.00 AS DECIMAL(5,2))
+ );
+ INSERT INTO web_site VALUES
+ (
+ CAST(5 AS BIGINT),
+ 'WS0005',
+ CAST('2023-05-01' AS DATE),
+ CAST('2023-08-31' AS DATE),
+ 'Example Web Site 5',
+ CAST(20230501 AS BIGINT),
+ CAST(20230831 AS BIGINT),
+ 'Travel',
+ 'Sophia Blue',
+ CAST(105 AS INT),
+ 'Travel',
+ 'Travel and booking services',
+ 'Daniel Red',
+ CAST(205 AS INT),
+ 'Travel Inc',
+ '3344',
+ 'Tourist Street',
+ 'Unit',
+ 'Unit 5',
+ 'Metropolis',
+ 'County',
+ 'WA',
+ '45678',
+ 'USA',
+ CAST(-7.0 AS DOUBLE),
+ CAST(8.00 AS DECIMAL(5,2))
+ );
+
+ */
+suite("test_external_catalog_maxcompute", "p2,external,doris,external_docker,external_docker_doris") {
Review Comment:
p2,external,maxcompute,external_remote,external_remote_maxcompute
##########
regression-test/suites/external_table_p2/maxcompute/test_max_compute_all_type.groovy:
##########
@@ -0,0 +1,547 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+/*
+
+drop table mc_all_types;
+
+CREATE TABLE `mc_all_types`(
+ `id` int,
+ `boolean_col` boolean,
+ `tinyint_col` tinyint,
+ `smallint_col` smallint,
+ `int_col` int,
+ `bigint_col` bigint,
+ `float_col` float,
+ `double_col` double,
+ `decimal_col1` decimal(9,0),
+ `decimal_col2` decimal(8,4),
+ `decimal_col3` decimal(18,6),
+ `decimal_col4` decimal(38,12),
+ `string_col` string,
+ `date_col` date,
+ `datetime_col` datetime,
+ `timestamp_ntz_col2` timestamp_ntz,
+ `timestamp_ntz_col3` timestamp_ntz,
+ `char_col1` char(50),
+ `char_col2` char(100),
+ `char_col3` char(255),
+ `varchar_col1` varchar(50),
+ `varchar_col2` varchar(100),
+ `varchar_col3` varchar(255),
+ `t_map_string` map<string,string>,
+ `t_map_varchar` map<varchar(65535),varchar(65535)>,
+ `t_map_char` map<char(10),char(10)>,
+ `t_map_int` map<int,int>,
+ `t_map_bigint` map<bigint,bigint>,
+ `t_map_float` map<float,float>,
+ `t_map_double` map<double,double>,
+ `t_map_boolean` map<boolean,boolean>,
+ `t_array_string` array<string>,
+ `t_array_int` array<int>,
+ `t_array_bigint` array<bigint>,
+ `t_array_float` array<float>,
+ `t_array_double` array<double>,
+ `t_array_boolean` array<boolean>,
+ `t_array_varchar` array<varchar(65535)>,
+ `t_array_char` array<char(10)>,
+ `t_array_decimal_precision_2` array<decimal(2,1)>,
+ `t_array_decimal_precision_4` array<decimal(4,2)>,
+ `t_array_decimal_precision_8` array<decimal(8,4)>,
+ `t_array_decimal_precision_17` array<decimal(17,8)>,
+ `t_array_decimal_precision_18` array<decimal(18,8)>,
+ `t_array_decimal_precision_38` array<decimal(38,16)>,
+ `t_struct_bigint` struct<s_bigint:bigint>,
+ `t_complex` map<string,array<struct<s_int:int>>>,
+ `t_struct_nested` struct<struct_field:array<string>>,
+ `t_struct_non_nulls_after_nulls` struct<struct_non_nulls_after_nulls1:int,struct_non_nulls_after_nulls2:string>,
+ `t_nested_struct_non_nulls_after_nulls` struct<struct_field1:int,struct_field2:string,strict_field3:struct<nested_struct_field1:int,nested_struct_field2:string>>,
+ `t_array_string_starting_with_nulls` array<string>,
+ `t_array_string_with_nulls_in_between` array<string>,
+ `t_array_string_ending_with_nulls` array<string>
+);
+
+
+
+insert into mc_all_types
+VALUES (
+ 1,
+ cast(0 as boolean), -- boolean_col
+ cast(-7 as tinyint), -- tinyint_col
+ cast(-15 as smallint), -- smallint_col
+ 16, -- int_col
+ cast(-9223372036854775807 as bigint), -- bigint_col
+ CAST(-123.45 AS FLOAT), -- float_col
+ CAST(-123456.789 AS DOUBLE), -- double_col
+ CAST(123456789 AS DECIMAL(9,0)), -- decimal_col1
+ CAST(-1234.5678 AS DECIMAL(8,4)), -- decimal_col2
+ CAST(-123456.789012 AS DECIMAL(18,6)), -- decimal_col3
+ CAST(-123456789.012345678901 AS DECIMAL(38,12)), -- decimal_col4
+ 'str', -- string_col
+ cast('2024-03-25' as date), -- date_col
+ cast('2024-03-25 12:00:00' as datetime),
+ cast('2024-03-25 12:00:00.123456' as timestamp_ntz), -- timestamp_ntz_col2
+ cast('2024-03-25 12:00:00.123456789' as timestamp_ntz), -- timestamp_ntz_col3
+ 'char_value11111', -- char_col1
+ 'char_value22222', -- char_col2
+ 'char_value33333', -- char_col3
+ 'varchar_value11111', -- varchar_col1
+ 'varchar_value22222', -- varchar_col2
+ 'varchar_value33333', -- varchar_col3
+ MAP('key7', 'value1'), -- t_map_string
+ MAP('key7', 'value1'), -- t_map_varchar
+ MAP('x', 'y'), -- t_map_char
+ MAP(3, 20), -- t_map_int
+ MAP(3, 200000000000), -- t_map_bigint
+ MAP(CAST(3.2 AS FLOAT), CAST(20.2 AS FLOAT)), -- t_map_float
+ MAP(CAST(3.2 AS DOUBLE), CAST(20.2 AS DOUBLE)), -- t_map_double
+ MAP(false, true), -- t_map_boolean
+ ARRAY('string1', 'string2'), -- t_array_string
+ ARRAY(4, 5, 6), -- t_array_int
+ ARRAY(300000000000, 400000000000), -- t_array_bigint
+ ARRAY(CAST(3.3 AS FLOAT), CAST(4.4 AS FLOAT)), -- t_array_float
+ ARRAY(CAST(3.123456789 AS DOUBLE), CAST(4.123456789 AS DOUBLE)), -- t_array_double
+ ARRAY(false, true), -- t_array_boolean
+ ARRAY('varchar1', 'varchar2'), -- t_array_varchar
+ ARRAY('char1', 'char2'), -- t_array_char
+ ARRAY(CAST(3.3 AS DECIMAL(2,1)), CAST(4.4 AS DECIMAL(2,1))), -- t_array_decimal_precision_2
+ ARRAY(CAST(3.45 AS DECIMAL(4,2)), CAST(4.56 AS DECIMAL(4,2))), -- t_array_decimal_precision_4
+ ARRAY(CAST(9.4567 AS DECIMAL(8,4)), CAST(4.5678 AS DECIMAL(8,4))), -- t_array_decimal_precision_8
+ ARRAY(CAST(6.45678901 AS DECIMAL(17,8)), CAST(4.56789012 AS DECIMAL(17,8))), -- t_array_decimal_precision_17
+ ARRAY(CAST(3.45678901 AS DECIMAL(18,8)), CAST(4.56789012 AS DECIMAL(18,8))), -- t_array_decimal_precision_18
+ ARRAY(CAST(3.456789012345678901 AS DECIMAL(38,16)), CAST(4.567890123456789012 AS DECIMAL(38,16))), -- t_array_decimal_precision_38
+ NAMED_STRUCT('s_bigint', -1234567890), -- t_struct_bigint
+ MAP('key', ARRAY(NAMED_STRUCT('s_int', -123))), -- t_complex
+ NAMED_STRUCT('struct_field', ARRAY('value1', 'value2')), -- t_struct_nested
+ NAMED_STRUCT('struct_non_nulls_after_nulls1', -123, 'struct_non_nulls_after_nulls2', 'value'), -- t_struct_non_nulls_after_nulls
+ NAMED_STRUCT('struct_field1', -123, 'struct_field2', 'value', 'strict_field3', NAMED_STRUCT('nested_struct_field1', -123, 'nested_struct_field2', 'nested_value')), -- t_nested_struct_non_nulls_after_nulls
+ ARRAY(null, 'value1', 'value2'), -- t_array_string_starting_with_nulls
+ ARRAY('value1', null, 'value2'), -- t_array_string_with_nulls_in_between
+ ARRAY('value11', 'value2', null) -- t_array_string_ending_with_nulls
+);
+
+
+
+INSERT into mc_all_types
+VALUES (
+ 2,
+ CAST (1 as boolean), -- boolean_col
+ CAST (127 as tinyint), -- tinyint_col
+ cast(32767 as smallint), -- smallint_col
+ 2147483647, -- int_col
+ cast(9223372036854775807 as bigint), -- bigint_col
+ cast(123.45 as float), -- float_col
+ 123456.789, -- double_col
+ CAST(123456789 AS DECIMAL(9,0)), -- decimal_col1
+ CAST(1234.5678 AS DECIMAL(8,4)), -- decimal_col2
+ CAST(123456.789012 AS DECIMAL(18,6)), -- decimal_col3
+ CAST(123456789.012345678901 AS DECIMAL(38,12)), -- decimal_col4
+ 'string_value', -- string_col
+ cast('2024-03-20' as date), -- date_col
+ cast('2024-03-20 12:00:00' as datetime ),
+ cast('2024-03-20 12:00:00.123456' as TIMESTAMP_ntz ), -- timestamp_ntz_col2
+ cast('2024-03-20 12:00:00.123456789' as TIMESTAMP_ntz ), -- timestamp_ntz_col3
+ 'char_value1', -- char_col1
+ 'char_value2', -- char_col2
+ 'char_value3', -- char_col3
+ 'varchar_value1', -- varchar_col1
+ 'varchar_value2', -- varchar_col2
+ 'varchar_value3', -- varchar_col3
+ MAP('key1', 'value1'), -- t_map_string
+ MAP('key1', 'value1'), -- t_map_varchar
+ MAP('a', 'b'), -- t_map_char
+ MAP(1, 10), -- t_map_int
+ MAP(1, 100000000000), -- t_map_bigint
+ MAP(CAST(1.1 AS FLOAT), CAST(10.1 AS FLOAT)), -- t_map_float
+ MAP(CAST(1.1 AS DOUBLE), CAST(10.1 AS DOUBLE)), -- t_map_double
+ MAP(TRUE, FALSE), -- t_map_boolean
+ ARRAY('string1', 'string2'), -- t_array_string
+ ARRAY(1, 2, 3), -- t_array_int
+ ARRAY(100000000000, 200000000000), -- t_array_bigint
+ ARRAY(CAST(1.1 AS FLOAT), CAST(2.2 AS FLOAT)), -- t_array_float
+ ARRAY(CAST(1.123456789 AS DOUBLE), CAST(2.123456789 AS DOUBLE)), -- t_array_double
+ ARRAY(TRUE, FALSE), -- t_array_boolean
+ ARRAY('varchar1', 'varchar2'), -- t_array_varchar
+ ARRAY('char1', 'char2'), -- t_array_char
+ ARRAY(CAST(1.1 AS DECIMAL(2,1)), CAST(2.2 AS DECIMAL(2,1))), -- t_array_decimal_precision_2
+ ARRAY(CAST(1.23 AS DECIMAL(4,2)), CAST(2.34 AS DECIMAL(4,2))), -- t_array_decimal_precision_4
+ ARRAY(CAST(1.2345 AS DECIMAL(8,4)), CAST(2.3456 AS DECIMAL(8,4))), -- t_array_decimal_precision_8
+ ARRAY(CAST(1.23456789 AS DECIMAL(17,8)), CAST(2.34567891 AS DECIMAL(17,8))), -- t_array_decimal_precision_17
+ ARRAY(1.23456789, 2.34567891), -- t_array_decimal_precision_18
+ ARRAY(1.234567890123456789, 2.345678901234567890), -- t_array_decimal_precision_38
+ NAMED_STRUCT('s_bigint', 1234567890), -- t_struct_bigint
+ MAP('key', ARRAY(NAMED_STRUCT('s_int', 123))), -- t_complex
+ NAMED_STRUCT('struct_field', ARRAY('value1', 'value2')), -- t_struct_nested
+ NAMED_STRUCT('struct_non_nulls_after_nulls1', 123, 'struct_non_nulls_after_nulls2', 'value'), -- t_struct_non_nulls_after_nulls
+ NAMED_STRUCT('struct_field1', 123, 'struct_field2', 'value', 'strict_field3', NAMED_STRUCT('nested_struct_field1', 123, 'nested_struct_field2', 'nested_value')), -- t_nested_struct_non_nulls_after_nulls
+ ARRAY(null, 'value1', 'value2'), -- t_array_string_starting_with_nulls
+ ARRAY('value1', null, 'value2'), -- t_array_string_with_nulls_in_between
+ ARRAY('value1', 'value2', null) -- t_array_string_ending_with_nulls
+);
+
+INSERT into mc_all_types
+VALUES
+(
+ 3,
+ cast(0 as boolean), -- boolean_col
+ cast(-7 as tinyint), -- tinyint_col
+ cast(-15 as smallint), -- smallint_col
+ 16, -- int_col
+ cast(-9223372036854775807 as bigint), -- bigint_col
+ CAST(-123.45 AS FLOAT), -- float_col
+ CAST(-123456.789 AS DOUBLE), -- double_col
+ CAST(123456789 AS DECIMAL(9,0)), -- decimal_col1
+ CAST(-1234.5678 AS DECIMAL(8,4)), -- decimal_col2
+ CAST(-123456.789012 AS DECIMAL(18,6)), -- decimal_col3
+ CAST(-123456789.012345678901 AS DECIMAL(38,12)), -- decimal_col4
+ 'str', -- string_col
+ cast('2024-03-25' as date), -- date_col
+ cast('2024-03-25 12:00:00' as datetime),
+ cast('2024-03-25 12:00:00.123456' as timestamp_ntz), -- timestamp_ntz_col2
+ cast('2024-03-25 12:00:00.123456789' as timestamp_ntz), -- timestamp_ntz_col3
+ 'char_value11111', -- char_col1
+ 'char_value22222', -- char_col2
+ 'char_value33333', -- char_col3
+ 'varchar_value11111', -- varchar_col1
+ 'varchar_value22222', -- varchar_col2
+ 'varchar_value33333', -- varchar_col3
+ MAP('key7', 'value1'), -- t_map_string
+ MAP('key7', 'value1'), -- t_map_varchar
+ MAP('x', 'y'), -- t_map_char
+ MAP(3, 20), -- t_map_int
+ MAP(3, 200000000000), -- t_map_bigint
+ MAP(CAST(3.2 AS FLOAT), CAST(20.2 AS FLOAT)), -- t_map_float
+ MAP(CAST(3.2 AS DOUBLE), CAST(20.2 AS DOUBLE)), -- t_map_double
+ MAP(false, true), -- t_map_boolean
+ ARRAY('string1', 'string2'), -- t_array_string
+ ARRAY(4, 5, 6), -- t_array_int
+ ARRAY(300000000000, 400000000000), -- t_array_bigint
+ ARRAY(CAST(3.3 AS FLOAT), CAST(4.4 AS FLOAT)), -- t_array_float
+ ARRAY(CAST(3.123456789 AS DOUBLE), CAST(4.123456789 AS DOUBLE)), -- t_array_double
+ ARRAY(false, true), -- t_array_boolean
+ ARRAY('varchar1', 'varchar2'), -- t_array_varchar
+ ARRAY('char1', 'char2'), -- t_array_char
+ ARRAY(CAST(3.3 AS DECIMAL(2,1)), CAST(4.4 AS DECIMAL(2,1))), -- t_array_decimal_precision_2
+ ARRAY(CAST(3.45 AS DECIMAL(4,2)), CAST(4.56 AS DECIMAL(4,2))), -- t_array_decimal_precision_4
+ ARRAY(CAST(9.4567 AS DECIMAL(8,4)), CAST(4.5678 AS DECIMAL(8,4))), -- t_array_decimal_precision_8
+ ARRAY(CAST(6.45678901 AS DECIMAL(17,8)), CAST(4.56789012 AS DECIMAL(17,8))), -- t_array_decimal_precision_17
+ ARRAY(CAST(3.45678901 AS DECIMAL(18,8)), CAST(4.56789012 AS DECIMAL(18,8))), -- t_array_decimal_precision_18
+ ARRAY(CAST(3.456789012345678901 AS DECIMAL(38,16)), CAST(4.567890123456789012 AS DECIMAL(38,16))), -- t_array_decimal_precision_38
+ NAMED_STRUCT('s_bigint', -1234567890), -- t_struct_bigint
+ MAP('key', ARRAY(NAMED_STRUCT('s_int', -123))), -- t_complex
+ NAMED_STRUCT('struct_field', ARRAY('value1', 'value2')), -- t_struct_nested
+ NAMED_STRUCT('struct_non_nulls_after_nulls1', -123, 'struct_non_nulls_after_nulls2', 'value'), -- t_struct_non_nulls_after_nulls
+ NAMED_STRUCT('struct_field1', -123, 'struct_field2', 'value', 'strict_field3', NAMED_STRUCT('nested_struct_field1', -123, 'nested_struct_field2', 'nested_value')), -- t_nested_struct_non_nulls_after_nulls
+ ARRAY(null, 'value1', 'value2'), -- t_array_string_starting_with_nulls
+ ARRAY('value1', null, 'value2'), -- t_array_string_with_nulls_in_between
+ ARRAY('value11', 'value2', null) -- t_array_string_ending_with_nulls
+);
+INSERT into mc_all_types
+VALUES
+(
+ 4,
+ cast(0 as boolean), -- boolean_col
+ cast(-128 as tinyint), -- tinyint_col
+ cast(-32768 as smallint), -- smallint_col
+ -2147483647, -- int_col
+ cast(-9223372036854775807 as bigint), -- bigint_col
+ CAST(-123.45 AS FLOAT), -- float_col
+ CAST(-123456.789 AS DOUBLE), -- double_col
+ CAST(-123456789 AS DECIMAL(9,0)), -- decimal_col1
+ CAST(-1234.5678 AS DECIMAL(8,4)), -- decimal_col2
+ CAST(-123456.789012 AS DECIMAL(18,6)), -- decimal_col3
+ CAST(-123456789.012345678901 AS DECIMAL(38,12)), -- decimal_col4
+ 'string_value', -- string_col
+ cast('2024-03-21' as date), -- date_col
+ cast('2024-03-21 12:00:00' as datetime),
+ cast('2024-03-21 12:00:00.123456' as timestamp_ntz), -- timestamp_ntz_col2
+ cast('2024-03-21 12:00:00.123456789' as timestamp_ntz), -- timestamp_ntz_col3
+ 'char_value1', -- char_col1
+ 'char_value2', -- char_col2
+ 'char_value3', -- char_col3
+ 'varchar_value1', -- varchar_col1
+ 'varchar_value2', -- varchar_col2
+ 'varchar_value3', -- varchar_col3
+ MAP('key1', 'value1'), -- t_map_string
+ MAP('key1', 'value1'), -- t_map_varchar
+ MAP('x', 'y'), -- t_map_char
+ MAP(2, 20), -- t_map_int
+ MAP(2, 200000000000), -- t_map_bigint
+ MAP(CAST(2.2 AS FLOAT), CAST(20.2 AS FLOAT)), -- t_map_float
+ MAP(CAST(2.2 AS DOUBLE), CAST(20.2 AS DOUBLE)), -- t_map_double
+ MAP(false, true), -- t_map_boolean
+ ARRAY('string1', 'string2'), -- t_array_string
+ ARRAY(4, 5, 6), -- t_array_int
+ ARRAY(300000000000, 400000000000), -- t_array_bigint
+ ARRAY(CAST(3.3 AS FLOAT), CAST(4.4 AS FLOAT)), -- t_array_float
+ ARRAY(CAST(3.123456789 AS DOUBLE), CAST(4.123456789 AS DOUBLE)), -- t_array_double
+ ARRAY(false, true), -- t_array_boolean
+ ARRAY('varchar1', 'varchar2'), -- t_array_varchar
+ ARRAY('char1', 'char2'), -- t_array_char
+ ARRAY(CAST(3.3 AS DECIMAL(2,1)), CAST(4.4 AS DECIMAL(2,1))), -- t_array_decimal_precision_2
+ ARRAY(CAST(3.45 AS DECIMAL(4,2)), CAST(4.56 AS DECIMAL(4,2))), -- t_array_decimal_precision_4
+ ARRAY(CAST(3.4567 AS DECIMAL(8,4)), CAST(4.5678 AS DECIMAL(8,4))), -- t_array_decimal_precision_8
+ ARRAY(CAST(3.45678901 AS DECIMAL(17,8)), CAST(4.56789012 AS DECIMAL(17,8))), -- t_array_decimal_precision_17
+ ARRAY(CAST(3.45678901 AS DECIMAL(18,8)), CAST(4.56789012 AS DECIMAL(18,8))), -- t_array_decimal_precision_18
+ ARRAY(CAST(3.456789012345678901 AS DECIMAL(38,16)), CAST(4.567890123456789012 AS DECIMAL(38,16))), -- t_array_decimal_precision_38
+ NAMED_STRUCT('s_bigint', -1234567890), -- t_struct_bigint
+ MAP('key', ARRAY(NAMED_STRUCT('s_int', -123))), -- t_complex
+ NAMED_STRUCT('struct_field', ARRAY('value1', 'value2')), -- t_struct_nested
+ NAMED_STRUCT('struct_non_nulls_after_nulls1', -123, 'struct_non_nulls_after_nulls2', 'value'), -- t_struct_non_nulls_after_nulls
+ NAMED_STRUCT('struct_field1', -123, 'struct_field2', 'value', 'strict_field3', NAMED_STRUCT('nested_struct_field1', -123, 'nested_struct_field2', 'nested_value')), -- t_nested_struct_non_nulls_after_nulls
+ ARRAY(null, 'value1', 'value2'), -- t_array_string_starting_with_nulls
+ ARRAY('value1', null, 'value2'), -- t_array_string_with_nulls_in_between
+ ARRAY('value1', 'value2', null) -- t_array_string_ending_with_nulls
+);
+
+ALTER TABLE mc_all_types MERGE SMALLFILES;
+
+select * from mc_all_types;
+ */
+suite("test_max_compute_all_type", "p2,external,doris,external_docker,external_docker_doris") {
Review Comment:
p2,external,maxcompute,external_remote,external_remote_maxcompute
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](MaxCompute)Refactoring maxCompute catalog using Storage API. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40225:
URL: https://github.com/apache/doris/pull/40225#issuecomment-2329668954
<details>
<summary>TPC-H: <b>Total hot run time: 38524 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 8cb3173c7b8f8c032b956ab11d39828305edc1a1, data reload: false
------ Round 1 ----------------------------------
q1 18199 5431 4429 4429
q2 2890 189 177 177
q3 10745 1137 1111 1111
q4 10505 845 813 813
q5 7782 2965 2883 2883
q6 229 142 145 142
q7 982 620 603 603
q8 9438 2139 2109 2109
q9 7312 6526 6560 6526
q10 7028 2215 2181 2181
q11 447 249 253 249
q12 402 220 217 217
q13 17776 3062 3061 3061
q14 288 243 236 236
q15 534 494 494 494
q16 573 521 518 518
q17 984 637 666 637
q18 7478 6830 6863 6830
q19 1395 1077 1042 1042
q20 694 330 329 329
q21 4006 2917 3087 2917
q22 1120 1020 1032 1020
Total cold run time: 110807 ms
Total hot run time: 38524 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4374 4277 4323 4277
q2 385 292 276 276
q3 2943 2692 2647 2647
q4 1946 1711 1713 1711
q5 5418 5413 5434 5413
q6 220 133 131 131
q7 2157 1751 1779 1751
q8 3207 3414 3418 3414
q9 8499 8445 8463 8445
q10 3742 3241 3219 3219
q11 603 497 506 497
q12 784 620 623 620
q13 14063 3111 3110 3110
q14 314 274 282 274
q15 525 484 494 484
q16 629 581 585 581
q17 1804 1475 1497 1475
q18 7888 7477 7413 7413
q19 1686 1465 1645 1465
q20 2063 1814 1841 1814
q21 5460 5295 5151 5151
q22 1127 1017 1051 1017
Total cold run time: 69837 ms
Total hot run time: 55185 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](MaxCompute)Refactoring maxCompute catalog using Storage API. [doris]
Posted by "hubgeter (via GitHub)" <gi...@apache.org>.
hubgeter commented on PR #40225:
URL: https://github.com/apache/doris/pull/40225#issuecomment-2336976642
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](MaxCompute)Refactoring maxCompute catalog using Storage API. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40225:
URL: https://github.com/apache/doris/pull/40225#issuecomment-2327167051
<details>
<summary>TPC-DS: <b>Total hot run time: 192576 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 40fada4cdb82562d755cac8c2d74d64533a77c2a, data reload: false
query1 1270 881 887 881
query2 6311 1929 1897 1897
query3 10605 3903 3865 3865
query4 59704 26811 23327 23327
query5 5356 495 494 494
query6 400 166 176 166
query7 5771 294 288 288
query8 280 198 201 198
query9 8901 2473 2454 2454
query10 480 261 265 261
query11 18056 15085 15418 15085
query12 167 101 103 101
query13 1536 388 388 388
query14 10442 7633 7041 7041
query15 217 180 173 173
query16 7505 477 427 427
query17 1152 576 580 576
query18 1903 288 293 288
query19 288 151 140 140
query20 117 105 108 105
query21 215 104 102 102
query22 4789 4605 4474 4474
query23 34264 33474 33316 33316
query24 5971 2866 2843 2843
query25 529 407 400 400
query26 688 157 157 157
query27 1793 279 273 273
query28 3637 2040 2032 2032
query29 699 429 470 429
query30 240 152 157 152
query31 954 777 792 777
query32 90 50 59 50
query33 459 294 297 294
query34 870 474 479 474
query35 833 718 718 718
query36 1048 938 928 928
query37 151 90 94 90
query38 4005 3873 3835 3835
query39 1472 1416 1421 1416
query40 205 113 116 113
query41 47 47 44 44
query42 120 93 95 93
query43 519 477 474 474
query44 1084 723 732 723
query45 208 174 169 169
query46 1098 766 742 742
query47 1907 1815 1830 1815
query48 387 292 302 292
query49 780 432 447 432
query50 826 433 410 410
query51 7087 6961 6920 6920
query52 98 86 89 86
query53 252 182 184 182
query54 564 460 462 460
query55 75 74 76 74
query56 333 256 251 251
query57 1194 1040 1066 1040
query58 215 237 220 220
query59 3083 2798 2854 2798
query60 301 278 262 262
query61 103 99 98 98
query62 750 656 645 645
query63 216 178 180 178
query64 2858 674 678 674
query65 3193 3171 3164 3164
query66 629 337 335 335
query67 15673 15367 15227 15227
query68 3010 561 560 560
query69 407 286 294 286
query70 1135 1119 1025 1025
query71 357 280 284 280
query72 6235 4197 4019 4019
query73 770 330 325 325
query74 9140 8751 8927 8751
query75 3409 2667 2710 2667
query76 1486 982 1018 982
query77 569 315 314 314
query78 9724 9426 9139 9139
query79 1041 527 524 524
query80 698 501 497 497
query81 545 243 232 232
query82 249 147 146 146
query83 167 147 142 142
query84 271 76 74 74
query85 688 289 279 279
query86 312 289 300 289
query87 4443 4367 4352 4352
query88 3037 2350 2331 2331
query89 385 286 290 286
query90 1842 192 191 191
query91 119 99 100 99
query92 60 47 48 47
query93 1039 523 532 523
query94 613 296 286 286
query95 341 253 252 252
query96 582 268 271 268
query97 3227 3126 3157 3126
query98 222 201 195 195
query99 1551 1253 1282 1253
Total cold run time: 304647 ms
Total hot run time: 192576 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](MaxCompute)Refactoring maxCompute catalog using Storage API. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40225:
URL: https://github.com/apache/doris/pull/40225#issuecomment-2327175920
<details>
<summary>ClickBench: <b>Total hot run time: 32.31 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 40fada4cdb82562d755cac8c2d74d64533a77c2a, data reload: false
query1 0.04 0.04 0.04
query2 0.09 0.05 0.04
query3 0.22 0.05 0.06
query4 1.68 0.07 0.08
query5 0.51 0.50 0.50
query6 1.14 0.72 0.74
query7 0.02 0.01 0.01
query8 0.05 0.05 0.04
query9 0.56 0.49 0.50
query10 0.55 0.56 0.54
query11 0.16 0.12 0.12
query12 0.14 0.12 0.13
query13 0.62 0.58 0.58
query14 2.10 2.19 2.11
query15 0.83 0.84 0.84
query16 0.37 0.36 0.38
query17 1.01 1.05 1.06
query18 0.20 0.21 0.20
query19 1.92 1.75 1.76
query20 0.01 0.02 0.01
query21 15.40 0.66 0.66
query22 4.35 7.89 2.03
query23 18.24 1.34 1.22
query24 2.11 0.22 0.22
query25 0.15 0.08 0.08
query26 0.27 0.18 0.18
query27 0.08 0.08 0.08
query28 13.20 1.02 0.99
query29 12.64 3.40 3.39
query30 0.24 0.06 0.05
query31 2.88 0.41 0.41
query32 3.23 0.48 0.47
query33 2.94 3.02 3.04
query34 17.06 4.40 4.36
query35 4.49 4.46 4.48
query36 0.66 0.47 0.51
query37 0.18 0.16 0.15
query38 0.15 0.15 0.16
query39 0.04 0.04 0.04
query40 0.15 0.12 0.12
query41 0.09 0.04 0.05
query42 0.06 0.04 0.04
query43 0.04 0.04 0.04
Total cold run time: 110.87 s
Total hot run time: 32.31 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](MaxCompute)Refactoring maxCompute catalog using Storage API. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40225:
URL: https://github.com/apache/doris/pull/40225#issuecomment-2334097917
<details>
<summary>TPC-H: <b>Total hot run time: 38075 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 0dd25050fa300530f5b1c949219d15a95dfe75cd, data reload: false
------ Round 1 ----------------------------------
q1 17751 4467 4346 4346
q2 2036 195 193 193
q3 11666 981 1101 981
q4 10487 765 720 720
q5 7786 2877 2775 2775
q6 224 143 135 135
q7 961 618 610 610
q8 9331 2089 2087 2087
q9 6952 6575 6611 6575
q10 7015 2215 2204 2204
q11 477 241 242 241
q12 406 228 221 221
q13 17754 3056 3117 3056
q14 291 245 248 245
q15 543 493 482 482
q16 512 426 423 423
q17 986 616 774 616
q18 7447 6908 6826 6826
q19 1391 1003 1014 1003
q20 693 319 335 319
q21 4136 3029 3058 3029
q22 1101 988 1034 988
Total cold run time: 109946 ms
Total hot run time: 38075 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4404 4266 4294 4266
q2 390 274 273 273
q3 2908 2610 2630 2610
q4 1954 1660 1666 1660
q5 5603 5729 5675 5675
q6 230 134 135 134
q7 2234 1827 1853 1827
q8 3297 3442 3462 3442
q9 8763 8835 8851 8835
q10 3519 3371 3339 3339
q11 611 505 516 505
q12 831 655 676 655
q13 15653 3260 3264 3260
q14 334 305 284 284
q15 547 495 496 495
q16 545 486 496 486
q17 1836 1568 1529 1529
q18 8229 7845 7932 7845
q19 1761 1538 1587 1538
q20 2165 1916 1913 1913
q21 5749 5458 5442 5442
q22 1116 1049 1015 1015
Total cold run time: 72679 ms
Total hot run time: 57028 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](MaxCompute)Refactoring maxCompute catalog using Storage API. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40225:
URL: https://github.com/apache/doris/pull/40225#issuecomment-2334101928
TeamCity be ut coverage result:
Function Coverage: 36.84% (9380/25461)
Line Coverage: 28.23% (77361/274039)
Region Coverage: 27.63% (39955/144591)
Branch Coverage: 24.26% (20330/83794)
Coverage Report: http://coverage.selectdb-in.cc/coverage/0dd25050fa300530f5b1c949219d15a95dfe75cd_0dd25050fa300530f5b1c949219d15a95dfe75cd/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](MaxCompute)Refactoring maxCompute catalog using Storage API. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40225:
URL: https://github.com/apache/doris/pull/40225#issuecomment-2337068682
<details>
<summary>TPC-DS: <b>Total hot run time: 192429 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 822a4f444e9b464f1cb62b0a04fab519da076acd, data reload: false
query1 1282 900 909 900
query2 6327 1926 1888 1888
query3 10599 4075 4233 4075
query4 59728 25781 23318 23318
query5 5475 493 491 491
query6 419 172 161 161
query7 5759 297 292 292
query8 326 223 217 217
query9 8677 2508 2503 2503
query10 481 269 258 258
query11 17741 15173 15382 15173
query12 162 108 99 99
query13 1488 411 394 394
query14 11205 6878 7498 6878
query15 240 189 182 182
query16 7185 443 458 443
query17 1197 589 582 582
query18 1998 333 293 293
query19 288 151 148 148
query20 119 113 110 110
query21 207 109 104 104
query22 4770 4273 4301 4273
query23 34370 33602 33462 33462
query24 5956 2893 2856 2856
query25 568 415 404 404
query26 697 162 162 162
query27 1780 284 282 282
query28 3700 2053 2046 2046
query29 680 429 432 429
query30 233 162 152 152
query31 940 761 775 761
query32 87 57 62 57
query33 440 288 289 288
query34 871 493 469 469
query35 817 738 710 710
query36 1071 911 954 911
query37 151 88 93 88
query38 4004 3817 3910 3817
query39 1463 1399 1396 1396
query40 198 118 117 117
query41 48 46 47 46
query42 121 96 94 94
query43 495 471 470 470
query44 1115 759 743 743
query45 197 165 163 163
query46 1078 754 721 721
query47 1845 1766 1803 1766
query48 368 296 294 294
query49 782 456 472 456
query50 817 411 419 411
query51 7019 6875 6939 6875
query52 102 85 85 85
query53 251 178 179 178
query54 571 464 462 462
query55 79 76 72 72
query56 289 262 262 262
query57 1191 1074 1073 1073
query58 235 235 231 231
query59 3136 2794 2744 2744
query60 293 273 273 273
query61 142 126 126 126
query62 751 648 636 636
query63 217 186 182 182
query64 2991 760 763 760
query65 3201 3107 3125 3107
query66 702 353 365 353
query67 15330 15195 15427 15195
query68 4292 575 557 557
query69 421 271 274 271
query70 1067 1142 1107 1107
query71 373 267 270 267
query72 6589 4172 3803 3803
query73 754 322 329 322
query74 9193 8795 8815 8795
query75 3387 2704 2716 2704
query76 2019 992 961 961
query77 542 323 319 319
query78 9932 9177 9355 9177
query79 1414 547 530 530
query80 952 510 511 510
query81 559 231 228 228
query82 265 136 142 136
query83 256 149 147 147
query84 260 74 81 74
query85 956 327 272 272
query86 394 301 311 301
query87 4443 4275 4283 4275
query88 3476 2280 2270 2270
query89 392 286 280 280
query90 1775 192 193 192
query91 125 95 98 95
query92 63 50 49 49
query93 1781 536 533 533
query94 735 286 308 286
query95 349 251 251 251
query96 584 261 257 257
query97 3216 3074 3033 3033
query98 224 219 210 210
query99 1610 1303 1250 1250
Total cold run time: 309463 ms
Total hot run time: 192429 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](MaxCompute)Refactoring maxCompute catalog using Storage API. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40225:
URL: https://github.com/apache/doris/pull/40225#issuecomment-2337058632
<details>
<summary>TPC-H: <b>Total hot run time: 38188 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 822a4f444e9b464f1cb62b0a04fab519da076acd, data reload: false
------ Round 1 ----------------------------------
q1 17798 4566 4319 4319
q2 2042 195 189 189
q3 11357 1001 1238 1001
q4 10493 794 703 703
q5 7787 2874 2841 2841
q6 231 143 141 141
q7 981 618 614 614
q8 9361 2133 2115 2115
q9 7246 6566 6527 6527
q10 7001 2185 2205 2185
q11 447 253 252 252
q12 417 236 233 233
q13 17764 3101 3102 3101
q14 291 246 246 246
q15 542 513 486 486
q16 543 431 459 431
q17 980 656 728 656
q18 7513 6850 6742 6742
q19 1395 1082 1116 1082
q20 719 336 336 336
q21 4119 3128 2981 2981
q22 1101 1024 1007 1007
Total cold run time: 110128 ms
Total hot run time: 38188 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4370 4295 4300 4295
q2 385 286 278 278
q3 2895 2614 2623 2614
q4 1942 1720 1674 1674
q5 5663 5701 5771 5701
q6 233 141 144 141
q7 2254 1841 1795 1795
q8 3350 3471 3538 3471
q9 8924 8865 8841 8841
q10 3615 3361 3384 3361
q11 626 507 507 507
q12 844 664 660 660
q13 16692 3173 3321 3173
q14 322 321 304 304
q15 536 492 481 481
q16 544 500 490 490
q17 1834 1549 1541 1541
q18 8268 7853 7884 7853
q19 1802 1590 1516 1516
q20 2118 1940 1905 1905
q21 5819 5596 5654 5596
q22 1102 1034 1065 1034
Total cold run time: 74138 ms
Total hot run time: 57231 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](MaxCompute)Refactoring maxCompute catalog using Storage API. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40225:
URL: https://github.com/apache/doris/pull/40225#issuecomment-2337072685
TeamCity be ut coverage result:
Function Coverage: 36.82% (9382/25478)
Line Coverage: 28.21% (77346/274151)
Region Coverage: 27.63% (39962/144643)
Branch Coverage: 24.25% (20328/83814)
Coverage Report: http://coverage.selectdb-in.cc/coverage/822a4f444e9b464f1cb62b0a04fab519da076acd_822a4f444e9b464f1cb62b0a04fab519da076acd/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](MaxCompute)Refactoring maxCompute catalog using Storage API. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40225:
URL: https://github.com/apache/doris/pull/40225#issuecomment-2333259764
TeamCity be ut coverage result:
Function Coverage: 36.85% (9380/25452)
Line Coverage: 28.22% (77316/273950)
Region Coverage: 27.63% (39941/144546)
Branch Coverage: 24.26% (20327/83774)
Coverage Report: http://coverage.selectdb-in.cc/coverage/04f750caf7f73efeb0902ec2a15ea7b9a6bc4294_04f750caf7f73efeb0902ec2a15ea7b9a6bc4294/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](MaxCompute)Refactoring maxCompute catalog using Storage API. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40225:
URL: https://github.com/apache/doris/pull/40225#issuecomment-2343015698
<details>
<summary>TPC-H: <b>Total hot run time: 38584 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 9d37687020a10441967fc000292259bcbf7f75dd, data reload: false
------ Round 1 ----------------------------------
q1 17904 4644 4409 4409
q2 2039 189 187 187
q3 11264 979 1117 979
q4 10337 798 763 763
q5 7761 2889 2789 2789
q6 233 141 140 140
q7 980 626 624 624
q8 9323 2119 2126 2119
q9 7342 6632 6653 6632
q10 7019 2217 2261 2217
q11 465 249 260 249
q12 402 221 221 221
q13 17884 3104 3144 3104
q14 280 237 242 237
q15 547 499 486 486
q16 540 439 451 439
q17 1005 728 785 728
q18 7435 6870 6913 6870
q19 1386 1102 1130 1102
q20 667 346 337 337
q21 4007 3166 2945 2945
q22 1115 1037 1007 1007
Total cold run time: 109935 ms
Total hot run time: 38584 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4442 4351 4353 4351
q2 394 277 271 271
q3 2909 2680 2698 2680
q4 2000 1758 1724 1724
q5 5770 5751 5858 5751
q6 233 136 143 136
q7 2277 1843 1849 1843
q8 3330 3462 3537 3462
q9 8868 8927 8881 8881
q10 3655 3437 3392 3392
q11 620 525 503 503
q12 845 665 673 665
q13 15277 3238 3405 3238
q14 326 296 313 296
q15 552 485 497 485
q16 559 489 502 489
q17 1880 1566 1549 1549
q18 8153 7950 8053 7950
q19 1735 1716 1561 1561
q20 2175 1898 1947 1898
q21 5937 5610 5492 5492
q22 1139 998 1056 998
Total cold run time: 73076 ms
Total hot run time: 57615 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](MaxCompute)Refactoring maxCompute catalog using Storage API. [doris]
Posted by "hubgeter (via GitHub)" <gi...@apache.org>.
hubgeter commented on PR #40225:
URL: https://github.com/apache/doris/pull/40225#issuecomment-2323281519
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](MaxCompute)Refactoring maxCompute catalog using Storage API. [doris]
Posted by "hubgeter (via GitHub)" <gi...@apache.org>.
hubgeter commented on PR #40225:
URL: https://github.com/apache/doris/pull/40225#issuecomment-2326960078
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](MaxCompute)Refactoring maxCompute catalog using Storage API. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40225:
URL: https://github.com/apache/doris/pull/40225#issuecomment-2327125346
TeamCity be ut coverage result:
Function Coverage: 36.82% (9385/25488)
Line Coverage: 28.27% (77440/273905)
Region Coverage: 27.66% (39955/144425)
Branch Coverage: 24.32% (20340/83630)
Coverage Report: http://coverage.selectdb-in.cc/coverage/40fada4cdb82562d755cac8c2d74d64533a77c2a_40fada4cdb82562d755cac8c2d74d64533a77c2a/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org