You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "morningman (via GitHub)" <gi...@apache.org> on 2024/01/31 14:20:02 UTC
[PR] [opt](catalog) cache the converted properties [doris]
morningman opened a new pull request, #30668:
URL: https://github.com/apache/doris/pull/30668
## Proposed changes
convert properties may be a heavy operation, so we cache the result.
```
at sun.misc.URLClassPath.getNextLoader(URLClassPath.java:480)
- waiting to lock <0x00000005e99cca00> (a sun.misc.URLClassPath)
at sun.misc.URLClassPath.findResource(URLClassPath.java:225)
at java.net.URLClassLoader$2.run(URLClassLoader.java:577)
at java.net.URLClassLoader$2.run(URLClassLoader.java:575)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findResource(URLClassLoader.java:574)
at java.lang.ClassLoader.getResource(ClassLoader.java:1089)
at org.apache.hadoop.conf.Configuration.getResource(Configuration.java:2839)
at org.apache.hadoop.conf.Configuration.getStreamReader(Configuration.java:3113)
at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:3072)
at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:3045)
at org.apache.hadoop.conf.Configuration.loadProps(Configuration.java:2923)
- locked <0x000000065cfe78a8> (a org.apache.hadoop.hive.conf.HiveConf)
at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2905)
- locked <0x000000065cfe78a8> (a org.apache.hadoop.hive.conf.HiveConf)
at org.apache.hadoop.conf.Configuration.iterator(Configuration.java:2968)
at org.apache.hadoop.hive.conf.HiveConf.getProperties(HiveConf.java:5134)
at org.apache.hadoop.hive.conf.HiveConf.getAllProperties(HiveConf.java:5130)
at org.apache.hadoop.hive.conf.HiveConf.initialize(HiveConf.java:5147)
at org.apache.hadoop.hive.conf.HiveConf.<init>(HiveConf.java:5102)
at org.apache.doris.datasource.property.PropertyConverter.getPropertiesFromDLFConf(PropertyConverter.java:377)
at org.apache.doris.datasource.property.PropertyConverter.convertToDLFProperties(PropertyConverter.java:365)
at org.apache.doris.datasource.property.PropertyConverter.convertToMetaProperties(PropertyConverter.java:89)
at org.apache.doris.datasource.ExternalCatalog.getProperties(ExternalCatalog.java:407)
at org.apache.doris.datasource.hive.HiveMetaStoreCache.getFileCache(HiveMetaStoreCache.java:378)
```
## Further comments
If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](catalog) cache the converted properties [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30668:
URL: https://github.com/apache/doris/pull/30668#issuecomment-1925565947
Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
```
Load test result on commit a6c3b7920a2a9b742a1b2d85377ac1a34b26f1a7 with default session variables
Stream load json: 18 seconds loaded 2358488459 Bytes, about 124 MB/s
Stream load orc: 60 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
Insert into select: 13.7 seconds inserted 10000000 Rows, about 729K ops/s
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](catalog) cache the converted properties [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30668:
URL: https://github.com/apache/doris/pull/30668#issuecomment-1919430718
<details>
<summary>TPC-H: <b>Total hot run time: 36599 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 57362bd6a837720388ac1bf14bff6b6dc1d9c51d, data reload: false
------ Round 1 ----------------------------------
q1 17645 4401 4307 4307
q2 2035 140 131 131
q3 10598 898 891 891
q4 4685 765 751 751
q5 7660 2888 2718 2718
q6 184 117 117 117
q7 1127 712 698 698
q8 9253 1998 1985 1985
q9 7163 6340 6276 6276
q10 8115 2405 2419 2405
q11 415 192 208 192
q12 735 269 286 269
q13 18013 3291 3266 3266
q14 275 244 246 244
q15 537 488 483 483
q16 458 401 425 401
q17 921 556 500 500
q18 6756 6092 6003 6003
q19 1548 1379 1348 1348
q20 570 328 338 328
q21 6743 2985 2990 2985
q22 788 303 301 301
Total cold run time: 106224 ms
Total hot run time: 36599 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4359 4359 4396 4359
q2 325 237 234 234
q3 3034 2829 2817 2817
q4 1853 1610 1665 1610
q5 5211 5224 5226 5224
q6 187 111 115 111
q7 2090 1796 1821 1796
q8 3141 3223 3246 3223
q9 8304 8291 8260 8260
q10 5845 3570 3554 3554
q11 561 451 450 450
q12 725 572 583 572
q13 12557 3064 3070 3064
q14 274 243 265 243
q15 523 486 489 486
q16 513 480 459 459
q17 1821 1675 1702 1675
q18 8034 7722 7564 7564
q19 9263 1542 1556 1542
q20 2140 1898 1890 1890
q21 4825 4654 4531 4531
q22 540 463 459 459
Total cold run time: 76125 ms
Total hot run time: 54123 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](catalog) cache the converted properties [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30668:
URL: https://github.com/apache/doris/pull/30668#issuecomment-1925565250
<details>
<summary>ClickBench: <b>Total hot run time: 30.53 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit a6c3b7920a2a9b742a1b2d85377ac1a34b26f1a7, data reload: false
query1 0.04 0.02 0.03
query2 0.06 0.02 0.02
query3 0.24 0.06 0.06
query4 1.64 0.10 0.09
query5 0.52 0.51 0.53
query6 1.21 0.64 0.63
query7 0.02 0.01 0.01
query8 0.04 0.02 0.02
query9 0.57 0.52 0.48
query10 0.57 0.57 0.55
query11 0.12 0.08 0.08
query12 0.11 0.09 0.09
query13 0.60 0.60 0.61
query14 0.79 0.81 0.80
query15 0.79 0.78 0.78
query16 0.38 0.38 0.38
query17 1.02 1.03 1.03
query18 0.25 0.23 0.24
query19 1.81 1.81 1.78
query20 0.01 0.01 0.01
query21 15.40 0.57 0.58
query22 2.55 3.00 2.05
query23 17.33 0.77 0.72
query24 2.69 0.28 0.68
query25 0.40 0.19 0.08
query26 0.36 0.15 0.14
query27 0.04 0.05 0.05
query28 12.22 0.82 0.85
query29 12.55 3.20 3.20
query30 0.55 0.57 0.53
query31 2.78 0.35 0.36
query32 3.36 0.49 0.48
query33 3.21 3.26 3.21
query34 15.84 4.31 4.32
query35 4.40 4.36 4.27
query36 1.13 1.06 1.06
query37 0.06 0.05 0.04
query38 0.04 0.02 0.02
query39 0.03 0.01 0.01
query40 0.16 0.13 0.15
query41 0.07 0.02 0.01
query42 0.02 0.01 0.01
query43 0.03 0.02 0.02
Total cold run time: 106.01 s
Total hot run time: 30.53 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](catalog) cache the converted properties [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30668:
URL: https://github.com/apache/doris/pull/30668#issuecomment-1925564307
<details>
<summary>TPC-DS: <b>Total hot run time: 174449 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit a6c3b7920a2a9b742a1b2d85377ac1a34b26f1a7, data reload: false
query1 935 349 333 333
query2 6544 2071 1909 1909
query3 6700 206 200 200
query4 31690 22036 22022 22022
query5 4270 356 361 356
query6 258 172 177 172
query7 4613 285 277 277
query8 252 175 194 175
query9 8863 2313 2296 2296
query10 409 213 214 213
query11 18649 15467 15326 15326
query12 124 76 78 76
query13 1612 407 406 406
query14 9412 6886 6939 6886
query15 289 182 182 182
query16 8160 253 253 253
query17 2043 532 490 490
query18 2104 271 282 271
query19 368 142 137 137
query20 81 73 79 73
query21 199 120 123 120
query22 4901 4645 4680 4645
query23 31056 29983 29946 29946
query24 6430 2754 2753 2753
query25 545 347 343 343
query26 702 147 142 142
query27 2127 303 304 303
query28 4811 1884 1854 1854
query29 882 614 609 609
query30 282 130 142 130
query31 931 711 696 696
query32 89 59 49 49
query33 464 220 217 217
query34 815 451 468 451
query35 846 780 745 745
query36 992 914 946 914
query37 98 59 62 59
query38 3216 3120 3108 3108
query39 1290 1258 1220 1220
query40 181 93 91 91
query41 37 36 34 34
query42 95 87 90 87
query43 512 472 463 463
query44 1022 699 699 699
query45 187 179 176 176
query46 1033 649 647 647
query47 1515 1531 1498 1498
query48 428 353 361 353
query49 1044 289 286 286
query50 762 369 369 369
query51 5272 5177 5094 5094
query52 93 87 81 81
query53 337 275 259 259
query54 258 220 220 220
query55 79 77 86 77
query56 214 195 196 195
query57 949 892 902 892
query58 202 177 179 177
query59 2464 2449 2375 2375
query60 237 214 220 214
query61 87 86 84 84
query62 627 356 346 346
query63 286 275 274 274
query64 4648 3537 3715 3537
query65 3273 3229 3201 3201
query66 830 312 303 303
query67 14209 14266 14129 14129
query68 4398 534 525 525
query69 471 337 345 337
query70 1214 1229 1224 1224
query71 315 250 252 250
query72 6090 2868 2685 2685
query73 698 316 317 316
query74 6571 6342 6234 6234
query75 3060 2321 2279 2279
query76 2593 986 998 986
query77 343 230 223 223
query78 9257 8786 8589 8589
query79 2270 495 490 490
query80 1529 351 347 347
query81 534 194 198 194
query82 645 79 81 79
query83 240 120 121 120
query84 237 80 78 78
query85 2152 364 337 337
query86 494 277 308 277
query87 3392 3214 3166 3166
query88 3705 2307 2320 2307
query89 437 356 367 356
query90 1944 168 163 163
query91 168 146 124 124
query92 57 42 44 42
query93 2644 505 458 458
query94 1302 176 169 169
query95 468 360 354 354
query96 596 264 267 264
query97 4240 4104 4114 4104
query98 219 197 197 197
query99 1192 696 657 657
Total cold run time: 272408 ms
Total hot run time: 174449 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](catalog) cache the converted properties [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30668:
URL: https://github.com/apache/doris/pull/30668#issuecomment-1919469761
Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
```
Load test result on commit 57362bd6a837720388ac1bf14bff6b6dc1d9c51d with default session variables
Stream load json: 18 seconds loaded 2358488459 Bytes, about 124 MB/s
Stream load orc: 58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select: 13.6 seconds inserted 10000000 Rows, about 735K ops/s
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](catalog) cache the converted properties [doris]
Posted by "morningman (via GitHub)" <gi...@apache.org>.
morningman commented on PR #30668:
URL: https://github.com/apache/doris/pull/30668#issuecomment-1925555758
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](catalog) cache the converted properties [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30668:
URL: https://github.com/apache/doris/pull/30668#issuecomment-1921737735
Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
```
Load test result on commit ccf3fa1fa18bef86e69e3568943333c33a5a7539 with default session variables
Stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc: 58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select: 13.4 seconds inserted 10000000 Rows, about 746K ops/s
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](catalog) cache the converted properties [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30668:
URL: https://github.com/apache/doris/pull/30668#issuecomment-1919199449
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](catalog) cache the converted properties [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #30668:
URL: https://github.com/apache/doris/pull/30668#issuecomment-1926548736
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](catalog) cache the converted properties [doris]
Posted by "morningman (via GitHub)" <gi...@apache.org>.
morningman merged PR #30668:
URL: https://github.com/apache/doris/pull/30668
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](catalog) cache the converted properties [doris]
Posted by "morningman (via GitHub)" <gi...@apache.org>.
morningman commented on PR #30668:
URL: https://github.com/apache/doris/pull/30668#issuecomment-1919370516
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](catalog) cache the converted properties [doris]
Posted by "morningman (via GitHub)" <gi...@apache.org>.
morningman commented on PR #30668:
URL: https://github.com/apache/doris/pull/30668#issuecomment-1921608657
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](catalog) cache the converted properties [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #30668:
URL: https://github.com/apache/doris/pull/30668#issuecomment-1926564294
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](catalog) cache the converted properties [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30668:
URL: https://github.com/apache/doris/pull/30668#issuecomment-1925562208
<details>
<summary>TPC-H: <b>Total hot run time: 37589 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit a6c3b7920a2a9b742a1b2d85377ac1a34b26f1a7, data reload: false
------ Round 1 ----------------------------------
q1 17886 4502 4443 4443
q2 2044 145 136 136
q3 10590 935 952 935
q4 5148 756 746 746
q5 7739 3000 2918 2918
q6 183 121 122 121
q7 1117 750 733 733
q8 9312 2025 2010 2010
q9 7382 6511 6569 6511
q10 8299 2615 2696 2615
q11 415 222 207 207
q12 763 300 288 288
q13 18704 3355 3383 3355
q14 287 256 269 256
q15 547 509 500 500
q16 1297 417 418 417
q17 1734 561 540 540
q18 6840 5988 5895 5895
q19 1584 1499 1375 1375
q20 629 356 318 318
q21 6800 2972 3056 2972
q22 800 304 298 298
Total cold run time: 110100 ms
Total hot run time: 37589 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4538 4345 4494 4345
q2 330 240 237 237
q3 2982 2867 2801 2801
q4 1821 1629 1665 1629
q5 5171 5262 5231 5231
q6 194 115 117 115
q7 2145 1721 1762 1721
q8 3087 3221 3275 3221
q9 8320 8260 8282 8260
q10 5774 3507 3573 3507
q11 539 467 456 456
q12 767 579 559 559
q13 14665 3072 3089 3072
q14 274 251 262 251
q15 528 491 492 491
q16 537 471 470 470
q17 1864 1677 1691 1677
q18 7987 7816 7479 7479
q19 8611 1512 1544 1512
q20 2146 1906 1898 1898
q21 5085 4617 4543 4543
q22 565 464 442 442
Total cold run time: 77930 ms
Total hot run time: 53917 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](catalog) cache the converted properties [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30668:
URL: https://github.com/apache/doris/pull/30668#issuecomment-1921696264
<details>
<summary>TPC-H: <b>Total hot run time: 36977 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit ccf3fa1fa18bef86e69e3568943333c33a5a7539, data reload: false
------ Round 1 ----------------------------------
q1 17702 4665 4421 4421
q2 2042 135 131 131
q3 10678 920 896 896
q4 4676 711 715 711
q5 7672 2847 2843 2843
q6 181 117 117 117
q7 1134 725 717 717
q8 9310 2014 2026 2014
q9 7227 6327 6304 6304
q10 8084 2448 2435 2435
q11 409 201 209 201
q12 773 269 265 265
q13 18018 3276 3292 3276
q14 271 249 242 242
q15 530 480 495 480
q16 465 413 394 394
q17 934 576 569 569
q18 6849 5899 5830 5830
q19 1576 1373 1373 1373
q20 577 326 334 326
q21 6876 3177 3127 3127
q22 803 306 305 305
Total cold run time: 106787 ms
Total hot run time: 36977 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4428 4489 4372 4372
q2 328 239 245 239
q3 2943 2817 2807 2807
q4 1821 1740 1617 1617
q5 5238 5192 5251 5192
q6 188 113 113 113
q7 2170 1785 1750 1750
q8 3139 3221 3276 3221
q9 8274 8264 8364 8264
q10 5754 3581 3523 3523
q11 537 453 455 453
q12 750 574 590 574
q13 13522 3086 3071 3071
q14 272 246 257 246
q15 533 494 492 492
q16 497 471 471 471
q17 1804 1700 1690 1690
q18 8027 7566 7585 7566
q19 9281 1532 1482 1482
q20 2160 1903 1894 1894
q21 6312 4544 4537 4537
q22 541 468 430 430
Total cold run time: 78519 ms
Total hot run time: 54004 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](catalog) cache the converted properties [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30668:
URL: https://github.com/apache/doris/pull/30668#issuecomment-1921732039
<details>
<summary>ClickBench: <b>Total hot run time: 30.57 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit ccf3fa1fa18bef86e69e3568943333c33a5a7539, data reload: false
query1 0.04 0.03 0.03
query2 0.06 0.02 0.02
query3 0.23 0.06 0.06
query4 1.68 0.10 0.10
query5 0.53 0.51 0.51
query6 1.17 0.63 0.63
query7 0.02 0.01 0.01
query8 0.04 0.02 0.03
query9 0.55 0.50 0.50
query10 0.54 0.54 0.56
query11 0.12 0.08 0.08
query12 0.11 0.08 0.09
query13 0.60 0.61 0.61
query14 0.79 0.79 0.78
query15 0.78 0.77 0.77
query16 0.38 0.37 0.40
query17 0.99 0.98 1.01
query18 0.22 0.26 0.25
query19 1.79 1.74 1.76
query20 0.02 0.01 0.01
query21 15.41 0.58 0.57
query22 2.57 2.54 1.66
query23 16.87 0.89 0.80
query24 2.69 1.59 0.84
query25 0.26 0.31 0.08
query26 0.60 0.13 0.14
query27 0.05 0.05 0.05
query28 12.38 0.84 0.83
query29 12.66 3.10 3.22
query30 0.62 0.54 0.53
query31 2.79 0.33 0.36
query32 3.37 0.47 0.49
query33 3.23 3.22 3.22
query34 15.86 4.32 4.28
query35 4.29 4.27 4.30
query36 1.12 1.06 1.07
query37 0.07 0.05 0.05
query38 0.03 0.03 0.03
query39 0.02 0.01 0.02
query40 0.16 0.14 0.12
query41 0.07 0.01 0.01
query42 0.03 0.02 0.03
query43 0.03 0.02 0.02
Total cold run time: 105.84 s
Total hot run time: 30.57 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](catalog) cache the converted properties [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30668:
URL: https://github.com/apache/doris/pull/30668#issuecomment-1921720389
<details>
<summary>TPC-DS: <b>Total hot run time: 173261 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit ccf3fa1fa18bef86e69e3568943333c33a5a7539, data reload: false
query1 933 333 323 323
query2 6580 1927 1899 1899
query3 6696 201 195 195
query4 31961 21927 21955 21927
query5 4456 356 412 356
query6 249 158 153 153
query7 4597 265 261 261
query8 237 180 170 170
query9 9171 2300 2267 2267
query10 409 195 215 195
query11 18706 15471 15355 15355
query12 123 69 64 64
query13 1621 367 377 367
query14 9955 6914 7461 6914
query15 233 182 184 182
query16 7324 264 251 251
query17 1865 485 461 461
query18 1933 256 246 246
query19 199 127 134 127
query20 68 68 71 68
query21 196 147 124 124
query22 4962 4637 4566 4566
query23 31639 30184 30223 30184
query24 7315 2744 2726 2726
query25 526 309 316 309
query26 731 138 138 138
query27 2259 279 277 277
query28 5286 1848 1836 1836
query29 929 617 606 606
query30 284 133 138 133
query31 924 734 722 722
query32 95 58 56 56
query33 457 213 201 201
query34 814 442 466 442
query35 824 755 742 742
query36 950 920 921 920
query37 94 55 57 55
query38 3267 3131 3159 3131
query39 1310 1264 1235 1235
query40 193 87 88 87
query41 39 36 40 36
query42 91 82 79 79
query43 528 476 507 476
query44 1039 705 704 704
query45 192 178 174 174
query46 1028 619 612 612
query47 1531 1557 1489 1489
query48 385 305 311 305
query49 1122 285 284 284
query50 694 304 299 299
query51 5287 5152 5124 5124
query52 93 76 80 76
query53 319 258 263 258
query54 225 181 185 181
query55 79 74 78 74
query56 175 165 154 154
query57 975 894 911 894
query58 179 158 162 158
query59 2442 2298 2428 2298
query60 204 171 175 171
query61 84 87 84 84
query62 581 357 340 340
query63 281 255 260 255
query64 4681 3691 3335 3335
query65 3250 3228 3220 3220
query66 959 318 318 318
query67 14608 13974 13841 13841
query68 4025 497 502 497
query69 425 303 309 303
query70 1285 1218 1183 1183
query71 299 209 212 209
query72 5540 3074 2846 2846
query73 689 310 315 310
query74 6672 6223 6247 6223
query75 2942 2274 2264 2264
query76 2561 936 1017 936
query77 357 230 220 220
query78 9315 8596 8511 8511
query79 3068 481 482 481
query80 2034 325 309 309
query81 495 202 207 202
query82 763 80 81 80
query83 244 122 116 116
query84 280 72 73 72
query85 2006 345 336 336
query86 478 322 285 285
query87 3462 3271 3211 3211
query88 4127 2169 2152 2152
query89 445 360 352 352
query90 1912 185 184 184
query91 161 116 116 116
query92 53 44 40 40
query93 5362 452 443 443
query94 1210 157 164 157
query95 504 470 444 444
query96 630 310 309 309
query97 4229 4103 4115 4103
query98 207 189 184 184
query99 1177 673 673 673
Total cold run time: 278399 ms
Total hot run time: 173261 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](catalog) cache the converted properties [doris]
Posted by "morningman (via GitHub)" <gi...@apache.org>.
morningman commented on PR #30668:
URL: https://github.com/apache/doris/pull/30668#issuecomment-1926315121
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](catalog) cache the converted properties [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30668:
URL: https://github.com/apache/doris/pull/30668#issuecomment-1919452642
<details>
<summary>TPC-DS: <b>Total hot run time: 175700 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 57362bd6a837720388ac1bf14bff6b6dc1d9c51d, data reload: false
query1 925 336 329 329
query2 6574 2026 1901 1901
query3 6700 202 200 200
query4 31945 22209 22190 22190
query5 4452 432 352 352
query6 250 161 155 155
query7 4599 262 271 262
query8 245 174 175 174
query9 9270 2270 2270 2270
query10 423 210 217 210
query11 18706 15470 15380 15380
query12 121 70 67 67
query13 1628 403 385 385
query14 10221 7551 7350 7350
query15 247 177 181 177
query16 7528 267 251 251
query17 1984 501 459 459
query18 1968 258 249 249
query19 328 138 135 135
query20 75 75 68 68
query21 206 138 125 125
query22 5013 4830 4771 4771
query23 31739 30342 30379 30342
query24 10217 2755 2781 2755
query25 503 314 310 310
query26 709 138 139 138
query27 2095 286 300 286
query28 5538 1853 1835 1835
query29 901 624 611 611
query30 282 134 139 134
query31 900 702 710 702
query32 88 52 48 48
query33 606 206 212 206
query34 815 448 461 448
query35 861 765 738 738
query36 1360 1178 1212 1178
query37 90 58 57 57
query38 3320 3183 3146 3146
query39 1310 1265 1240 1240
query40 191 90 84 84
query41 38 35 36 35
query42 88 81 83 81
query43 513 493 497 493
query44 1032 683 689 683
query45 194 184 177 177
query46 1059 640 622 622
query47 1555 1406 1507 1406
query48 419 301 307 301
query49 1097 289 272 272
query50 683 317 315 315
query51 5273 5200 5175 5175
query52 102 73 79 73
query53 320 264 266 264
query54 276 187 182 182
query55 82 73 78 73
query56 179 162 159 159
query57 965 909 904 904
query58 182 163 158 158
query59 2553 2277 2380 2277
query60 202 176 187 176
query61 83 83 83 83
query62 602 361 361 361
query63 276 260 268 260
query64 4617 3711 3468 3468
query65 3240 3236 3217 3217
query66 940 324 318 318
query67 14353 14199 14236 14199
query68 4383 498 499 498
query69 458 310 315 310
query70 1591 1568 1539 1539
query71 282 214 212 212
query72 5867 3134 2845 2845
query73 677 317 318 317
query74 6639 6179 6270 6179
query75 3050 2252 2313 2252
query76 3201 982 958 958
query77 377 236 229 229
query78 9219 8676 8553 8553
query79 2802 502 502 502
query80 2122 325 319 319
query81 541 194 204 194
query82 709 85 86 85
query83 334 114 119 114
query84 275 82 76 76
query85 1692 330 339 330
query86 517 403 376 376
query87 3418 3223 3273 3223
query88 3773 2156 2174 2156
query89 420 366 359 359
query90 1945 186 180 180
query91 148 114 120 114
query92 52 45 42 42
query93 2462 429 418 418
query94 1420 159 161 159
query95 503 465 455 455
query96 618 315 308 308
query97 4235 4114 4116 4114
query98 220 198 176 176
query99 1142 679 658 658
Total cold run time: 280948 ms
Total hot run time: 175700 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](catalog) cache the converted properties [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30668:
URL: https://github.com/apache/doris/pull/30668#issuecomment-1919464100
<details>
<summary>ClickBench: <b>Total hot run time: 30.77 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 57362bd6a837720388ac1bf14bff6b6dc1d9c51d, data reload: false
query1 0.04 0.03 0.03
query2 0.05 0.02 0.02
query3 0.24 0.06 0.06
query4 1.69 0.10 0.10
query5 0.52 0.52 0.51
query6 1.20 0.67 0.64
query7 0.02 0.02 0.01
query8 0.03 0.02 0.02
query9 0.57 0.52 0.50
query10 0.56 0.55 0.56
query11 0.12 0.08 0.08
query12 0.12 0.10 0.09
query13 0.61 0.61 0.61
query14 0.78 0.80 0.81
query15 0.80 0.78 0.77
query16 0.37 0.39 0.39
query17 0.99 1.01 1.01
query18 0.24 0.25 0.26
query19 1.87 1.70 1.77
query20 0.01 0.01 0.01
query21 15.40 0.58 0.57
query22 2.67 2.32 1.81
query23 17.11 0.88 0.85
query24 2.51 0.85 1.11
query25 0.26 0.26 0.07
query26 0.51 0.15 0.15
query27 0.06 0.04 0.04
query28 11.40 0.86 0.84
query29 12.55 3.08 3.12
query30 0.63 0.47 0.51
query31 2.78 0.35 0.34
query32 3.37 0.48 0.49
query33 3.22 3.23 3.22
query34 15.74 4.27 4.29
query35 4.29 4.33 4.27
query36 1.11 1.06 1.07
query37 0.07 0.05 0.05
query38 0.04 0.03 0.02
query39 0.02 0.02 0.02
query40 0.16 0.12 0.12
query41 0.06 0.02 0.01
query42 0.03 0.01 0.02
query43 0.02 0.02 0.02
Total cold run time: 104.84 s
Total hot run time: 30.77 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org