You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "zddr (via GitHub)" <gi...@apache.org> on 2024/07/30 08:07:11 UTC
[PR] [enhance](mtmv)hive cache add partitionId to partitionName Map [doris]
zddr opened a new pull request, #38525:
URL: https://github.com/apache/doris/pull/38525
MTMV need get partitionName by partitionId
but now only have partitionName=>partitionId Map,when there are many partitions, obtaining the name from this map based on the ID can be slow,
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](mtmv)hive cache add partitionId to partitionName Map [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #38525:
URL: https://github.com/apache/doris/pull/38525#issuecomment-2258009963
<details>
<summary>TPC-H: <b>Total hot run time: 41738 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit f7f7d0d0e54b29d6bd70eea2fefd044cb0a003fb, data reload: false
------ Round 1 ----------------------------------
q1 18027 4198 4145 4145
q2 2359 203 198 198
q3 10479 1339 1348 1339
q4 10160 801 905 801
q5 7650 2914 2972 2914
q6 220 137 140 137
q7 1038 621 621 621
q8 9447 1883 1928 1883
q9 8484 6597 6654 6597
q10 8686 3855 3818 3818
q11 425 256 253 253
q12 413 231 231 231
q13 17755 2943 2917 2917
q14 270 248 240 240
q15 526 493 496 493
q16 486 398 389 389
q17 963 934 910 910
q18 8023 7314 7229 7229
q19 1391 1222 1228 1222
q20 567 313 341 313
q21 5334 4805 4836 4805
q22 365 291 283 283
Total cold run time: 113068 ms
Total hot run time: 41738 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4101 4026 4005 4005
q2 326 222 219 219
q3 3001 3001 3013 3001
q4 1863 1870 1885 1870
q5 5365 5325 5319 5319
q6 220 133 133 133
q7 2094 1738 1733 1733
q8 3255 3322 3304 3304
q9 8387 8305 8277 8277
q10 3761 3830 3837 3830
q11 536 442 452 442
q12 731 582 539 539
q13 14705 2952 2944 2944
q14 290 260 253 253
q15 519 476 481 476
q16 437 397 394 394
q17 1716 1696 1690 1690
q18 7688 7337 7292 7292
q19 1661 1667 1674 1667
q20 1988 1759 1733 1733
q21 5402 5193 5213 5193
q22 547 449 436 436
Total cold run time: 68593 ms
Total hot run time: 54750 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](mtmv)hive cache add partitionId to partitionName Map [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #38525:
URL: https://github.com/apache/doris/pull/38525#issuecomment-2277590680
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](mtmv)hive cache add partitionId to partitionName Map [doris]
Posted by "zddr (via GitHub)" <gi...@apache.org>.
zddr commented on PR #38525:
URL: https://github.com/apache/doris/pull/38525#issuecomment-2257976611
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](mtmv)hive cache add partitionId to partitionName Map [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #38525:
URL: https://github.com/apache/doris/pull/38525#issuecomment-2257849486
<details>
<summary>ClickBench: <b>Total hot run time: 30.16 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 3b401b073e35507eb202ab5c4057051da91289c8, data reload: false
query1 0.04 0.03 0.04
query2 0.07 0.04 0.04
query3 0.22 0.05 0.05
query4 1.68 0.07 0.07
query5 0.50 0.49 0.49
query6 1.15 0.73 0.71
query7 0.01 0.01 0.01
query8 0.05 0.05 0.05
query9 0.59 0.54 0.51
query10 0.57 0.61 0.57
query11 0.16 0.12 0.12
query12 0.15 0.13 0.12
query13 0.61 0.59 0.60
query14 0.79 0.80 0.79
query15 0.91 0.86 0.85
query16 0.35 0.35 0.35
query17 1.01 1.03 0.98
query18 0.22 0.21 0.21
query19 1.83 1.81 1.76
query20 0.01 0.01 0.01
query21 15.87 0.79 0.66
query22 3.97 8.16 1.36
query23 18.04 1.29 1.28
query24 2.24 0.21 0.22
query25 0.18 0.08 0.07
query26 0.32 0.21 0.21
query27 0.45 0.24 0.22
query28 13.17 0.99 0.97
query29 12.54 3.36 3.32
query30 0.25 0.06 0.06
query31 2.88 0.41 0.40
query32 3.26 0.49 0.49
query33 2.94 2.95 2.95
query34 15.46 4.28 4.27
query35 4.33 4.30 4.32
query36 0.68 0.48 0.50
query37 0.20 0.17 0.16
query38 0.16 0.15 0.15
query39 0.04 0.03 0.03
query40 0.17 0.14 0.13
query41 0.09 0.05 0.05
query42 0.05 0.05 0.04
query43 0.05 0.04 0.04
Total cold run time: 108.26 s
Total hot run time: 30.16 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](mtmv)hive cache add partitionId to partitionName Map [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #38525:
URL: https://github.com/apache/doris/pull/38525#issuecomment-2277590616
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](mtmv)hive cache add partitionId to partitionName Map [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #38525:
URL: https://github.com/apache/doris/pull/38525#issuecomment-2268737387
<details>
<summary>ClickBench: <b>Total hot run time: 29.69 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 26271ce27b64657f154a67953f07ebdf2894dc27, data reload: false
query1 0.04 0.04 0.04
query2 0.08 0.04 0.03
query3 0.23 0.05 0.05
query4 1.68 0.07 0.07
query5 0.49 0.48 0.49
query6 1.13 0.71 0.72
query7 0.02 0.01 0.02
query8 0.04 0.04 0.04
query9 0.56 0.50 0.52
query10 0.56 0.57 0.55
query11 0.16 0.12 0.11
query12 0.15 0.13 0.12
query13 0.61 0.61 0.60
query14 0.78 0.78 0.80
query15 0.90 0.86 0.86
query16 0.36 0.35 0.35
query17 0.99 1.00 1.00
query18 0.22 0.22 0.22
query19 1.80 1.75 1.75
query20 0.00 0.01 0.01
query21 15.43 0.74 0.65
query22 4.28 8.60 1.04
query23 17.80 1.38 1.32
query24 2.26 0.22 0.22
query25 0.19 0.09 0.08
query26 0.31 0.21 0.22
query27 0.45 0.23 0.23
query28 13.18 0.99 0.97
query29 12.53 3.29 3.25
query30 0.26 0.06 0.06
query31 2.86 0.41 0.41
query32 3.26 0.48 0.48
query33 2.94 2.94 2.94
query34 15.40 4.26 4.23
query35 4.31 4.26 4.33
query36 0.68 0.48 0.48
query37 0.19 0.17 0.16
query38 0.16 0.14 0.16
query39 0.05 0.03 0.04
query40 0.16 0.13 0.13
query41 0.11 0.05 0.05
query42 0.06 0.05 0.04
query43 0.04 0.03 0.05
Total cold run time: 107.71 s
Total hot run time: 29.69 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](mtmv)hive cache add partitionId to partitionName Map [doris]
Posted by "zddr (via GitHub)" <gi...@apache.org>.
zddr commented on PR #38525:
URL: https://github.com/apache/doris/pull/38525#issuecomment-2257732486
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](mtmv)hive cache add partitionId to partitionName Map [doris]
Posted by "zddr (via GitHub)" <gi...@apache.org>.
zddr commented on PR #38525:
URL: https://github.com/apache/doris/pull/38525#issuecomment-2268666030
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](mtmv)hive cache add partitionId to partitionName Map [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #38525:
URL: https://github.com/apache/doris/pull/38525#issuecomment-2257732293
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR)
Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website).
See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](mtmv)hive cache add partitionId to partitionName Map [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #38525:
URL: https://github.com/apache/doris/pull/38525#issuecomment-2258029374
<details>
<summary>TPC-DS: <b>Total hot run time: 169289 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit f7f7d0d0e54b29d6bd70eea2fefd044cb0a003fb, data reload: false
query1 915 374 375 374
query2 6472 1794 1775 1775
query3 6693 213 235 213
query4 20150 17525 17309 17309
query5 4304 536 534 534
query6 284 171 175 171
query7 4610 301 291 291
query8 259 200 197 197
query9 8495 2393 2384 2384
query10 442 289 284 284
query11 10502 10085 9924 9924
query12 143 92 92 92
query13 1647 393 370 370
query14 9201 7247 6969 6969
query15 205 164 166 164
query16 7118 443 454 443
query17 967 572 552 552
query18 1912 287 283 283
query19 200 150 144 144
query20 100 86 86 86
query21 220 100 105 100
query22 4134 4079 4190 4079
query23 33716 32837 32924 32837
query24 10766 3127 3029 3029
query25 695 412 403 403
query26 1893 153 153 153
query27 3025 274 280 274
query28 7057 1956 1955 1955
query29 1402 423 419 419
query30 286 155 153 153
query31 939 743 756 743
query32 98 55 58 55
query33 700 310 327 310
query34 918 484 489 484
query35 860 727 729 727
query36 1006 889 874 874
query37 304 82 80 80
query38 2871 2773 2807 2773
query39 874 809 812 809
query40 290 124 124 124
query41 49 50 47 47
query42 131 101 107 101
query43 500 435 455 435
query44 1226 732 733 732
query45 207 177 178 177
query46 1107 832 792 792
query47 1772 1687 1716 1687
query48 371 299 293 293
query49 1187 443 444 443
query50 950 456 448 448
query51 6786 6747 6637 6637
query52 101 94 91 91
query53 264 186 190 186
query54 656 462 461 461
query55 81 77 76 76
query56 279 262 270 262
query57 1167 1067 1064 1064
query58 269 286 286 286
query59 2733 2375 2368 2368
query60 293 291 289 289
query61 104 99 120 99
query62 908 656 666 656
query63 224 198 193 193
query64 5950 2037 1910 1910
query65 3199 3119 3115 3115
query66 1464 340 341 340
query67 15393 14754 14626 14626
query68 4363 561 582 561
query69 446 332 313 313
query70 1150 1061 1118 1061
query71 369 285 278 278
query72 7195 2762 2510 2510
query73 779 330 325 325
query74 6017 5666 5583 5583
query75 3379 2781 2764 2764
query76 2186 1400 1457 1400
query77 451 330 317 317
query78 9467 8961 8987 8961
query79 1308 535 546 535
query80 1022 549 533 533
query81 548 226 231 226
query82 1137 141 138 138
query83 246 172 177 172
query84 276 81 81 81
query85 1299 321 317 317
query86 405 316 308 308
query87 3270 3080 3087 3080
query88 3008 2448 2422 2422
query89 393 295 300 295
query90 1773 201 197 197
query91 134 105 105 105
query92 73 51 56 51
query93 1391 630 622 622
query94 906 309 367 309
query95 398 286 274 274
query96 614 289 283 283
query97 3193 3066 3022 3022
query98 222 216 196 196
query99 1686 1296 1313 1296
Total cold run time: 263148 ms
Total hot run time: 169289 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](mtmv)hive cache add partitionId to partitionName Map [doris]
Posted by "zddr (via GitHub)" <gi...@apache.org>.
zddr commented on PR #38525:
URL: https://github.com/apache/doris/pull/38525#issuecomment-2268864243
run p1
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](mtmv)hive cache add partitionId to partitionName Map [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #38525:
URL: https://github.com/apache/doris/pull/38525#issuecomment-2268707205
<details>
<summary>TPC-H: <b>Total hot run time: 41653 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 26271ce27b64657f154a67953f07ebdf2894dc27, data reload: false
------ Round 1 ----------------------------------
q1 19725 4127 4125 4125
q2 2436 208 213 208
q3 11047 1318 1335 1318
q4 10733 861 930 861
q5 7764 3009 2952 2952
q6 223 134 135 134
q7 1042 611 610 610
q8 9420 1874 1913 1874
q9 8375 6565 6644 6565
q10 8719 3805 3851 3805
q11 423 255 263 255
q12 407 231 226 226
q13 17754 2932 2960 2932
q14 276 239 245 239
q15 530 482 495 482
q16 491 416 407 407
q17 968 894 909 894
q18 8042 7304 7274 7274
q19 1395 1218 1213 1213
q20 560 323 347 323
q21 5253 4671 4714 4671
q22 360 285 291 285
Total cold run time: 115943 ms
Total hot run time: 41653 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4097 3991 4012 3991
q2 333 221 232 221
q3 2985 2990 2974 2974
q4 1877 1851 1836 1836
q5 5274 5224 5206 5206
q6 214 128 132 128
q7 2074 1725 1695 1695
q8 3155 3271 3227 3227
q9 8273 8263 8246 8246
q10 3750 3853 3807 3807
q11 544 450 456 450
q12 749 590 550 550
q13 13239 2896 2938 2896
q14 292 269 253 253
q15 519 479 484 479
q16 453 398 390 390
q17 1735 1705 1692 1692
q18 7883 7284 7173 7173
q19 1679 1682 1678 1678
q20 1984 1762 1771 1762
q21 5310 5108 5186 5108
q22 540 466 471 466
Total cold run time: 66959 ms
Total hot run time: 54228 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](mtmv)hive cache add partitionId to partitionName Map [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #38525:
URL: https://github.com/apache/doris/pull/38525#issuecomment-2258039250
<details>
<summary>ClickBench: <b>Total hot run time: 30.9 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit f7f7d0d0e54b29d6bd70eea2fefd044cb0a003fb, data reload: false
query1 0.04 0.03 0.03
query2 0.07 0.04 0.04
query3 0.23 0.05 0.05
query4 1.68 0.07 0.08
query5 0.49 0.48 0.48
query6 1.13 0.72 0.72
query7 0.02 0.01 0.01
query8 0.05 0.04 0.04
query9 0.56 0.52 0.51
query10 0.57 0.57 0.56
query11 0.16 0.11 0.12
query12 0.15 0.13 0.13
query13 0.62 0.60 0.60
query14 0.78 0.80 0.79
query15 0.89 0.86 0.87
query16 0.36 0.36 0.36
query17 0.97 0.98 1.00
query18 0.22 0.22 0.22
query19 1.81 1.81 1.82
query20 0.01 0.01 0.02
query21 15.41 0.74 0.64
query22 4.15 7.42 2.13
query23 18.04 1.22 1.14
query24 2.24 0.23 0.22
query25 0.17 0.08 0.08
query26 0.33 0.21 0.22
query27 0.46 0.22 0.23
query28 13.18 1.00 0.98
query29 12.58 3.33 3.32
query30 0.26 0.07 0.06
query31 2.86 0.41 0.40
query32 3.25 0.50 0.51
query33 2.94 2.97 2.99
query34 15.44 4.27 4.28
query35 4.28 4.29 4.29
query36 0.68 0.47 0.49
query37 0.19 0.17 0.16
query38 0.15 0.16 0.16
query39 0.04 0.04 0.03
query40 0.15 0.14 0.13
query41 0.10 0.06 0.05
query42 0.06 0.05 0.06
query43 0.05 0.04 0.04
Total cold run time: 107.82 s
Total hot run time: 30.9 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](mtmv)hive cache add partitionId to partitionName Map [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #38525:
URL: https://github.com/apache/doris/pull/38525#issuecomment-2268727249
<details>
<summary>TPC-DS: <b>Total hot run time: 168314 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 26271ce27b64657f154a67953f07ebdf2894dc27, data reload: false
query1 919 371 382 371
query2 6495 1697 1592 1592
query3 6698 216 222 216
query4 20122 17610 17265 17265
query5 4320 510 513 510
query6 288 166 167 166
query7 4594 296 295 295
query8 246 196 198 196
query9 8531 2385 2364 2364
query10 459 292 270 270
query11 10402 10014 10037 10014
query12 144 89 88 88
query13 1639 386 377 377
query14 8472 6771 7284 6771
query15 202 162 169 162
query16 7088 449 457 449
query17 955 577 553 553
query18 1869 290 289 289
query19 193 148 148 148
query20 100 87 89 87
query21 202 107 107 107
query22 4045 3928 3886 3886
query23 33617 33073 32904 32904
query24 10360 3123 2979 2979
query25 700 410 405 405
query26 1799 151 154 151
query27 2947 274 275 274
query28 6948 1983 1980 1980
query29 1324 431 432 431
query30 291 154 163 154
query31 931 750 759 750
query32 104 61 57 57
query33 735 320 332 320
query34 907 480 481 480
query35 873 736 729 729
query36 1011 879 885 879
query37 293 81 79 79
query38 2862 2794 2783 2783
query39 846 826 808 808
query40 292 116 114 114
query41 49 49 46 46
query42 123 107 107 107
query43 465 435 439 435
query44 1193 734 735 734
query45 213 178 177 177
query46 1086 813 811 811
query47 1795 1701 1699 1699
query48 373 293 301 293
query49 1295 425 431 425
query50 898 441 439 439
query51 6840 6701 6664 6664
query52 104 114 98 98
query53 253 183 174 174
query54 655 451 465 451
query55 75 76 75 75
query56 272 259 263 259
query57 1148 1021 1005 1005
query58 276 273 262 262
query59 2473 2423 2310 2310
query60 316 271 295 271
query61 101 99 96 96
query62 933 658 661 658
query63 225 185 183 183
query64 5868 1910 1884 1884
query65 3145 3091 3106 3091
query66 1458 335 327 327
query67 15642 14847 14823 14823
query68 4748 573 584 573
query69 691 345 314 314
query70 1064 1075 1080 1075
query71 477 287 285 285
query72 8046 2696 2492 2492
query73 775 331 337 331
query74 6014 5622 5661 5622
query75 4071 2828 2749 2749
query76 2941 1142 1233 1142
query77 715 313 316 313
query78 9511 8843 8920 8843
query79 2738 527 538 527
query80 1043 503 491 491
query81 557 227 229 227
query82 827 133 128 128
query83 348 172 169 169
query84 267 81 80 80
query85 1523 317 300 300
query86 454 322 289 289
query87 3242 3142 3075 3075
query88 3815 2417 2436 2417
query89 385 284 293 284
query90 2032 194 194 194
query91 127 99 101 99
query92 64 52 55 52
query93 2190 620 618 618
query94 1030 312 291 291
query95 385 273 276 273
query96 605 284 286 284
query97 3241 3053 3100 3053
query98 223 200 201 200
query99 1699 1292 1274 1274
Total cold run time: 267945 ms
Total hot run time: 168314 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](mtmv)hive cache add partitionId to partitionName Map [doris]
Posted by "zddr (via GitHub)" <gi...@apache.org>.
zddr commented on PR #38525:
URL: https://github.com/apache/doris/pull/38525#issuecomment-2259505642
run cloud_p0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](mtmv)hive cache add partitionId to partitionName Map [doris]
Posted by "morrySnow (via GitHub)" <gi...@apache.org>.
morrySnow commented on code in PR #38525:
URL: https://github.com/apache/doris/pull/38525#discussion_r1698220761
##########
fe/fe-core/src/main/java/org/apache/doris/datasource/hive/HiveMetaStoreCache.java:
##########
@@ -636,6 +636,7 @@ public void addPartitionsCache(String dbName, String tblName, List<String> parti
HivePartitionValues copy = partitionValues.copy();
Map<Long, PartitionItem> idToPartitionItemBefore = copy.getIdToPartitionItem();
Map<String, Long> partitionNameToIdMapBefore = copy.getPartitionNameToIdMap();
+ Map<Long, String> partitionIdToNameMapBefore = copy.getPartitionIdToNameMap();
Review Comment:
both id and name should be unique, may be better to use bimap to avoid inconsistent bewteen them
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](mtmv)hive cache add partitionId to partitionName Map [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #38525:
URL: https://github.com/apache/doris/pull/38525#issuecomment-2257835333
<details>
<summary>TPC-DS: <b>Total hot run time: 169944 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 3b401b073e35507eb202ab5c4057051da91289c8, data reload: false
query1 902 373 361 361
query2 6454 1821 1779 1779
query3 6654 219 224 219
query4 19953 17475 17199 17199
query5 3671 510 509 509
query6 277 172 179 172
query7 4607 302 294 294
query8 251 203 191 191
query9 8526 2360 2354 2354
query10 416 272 260 260
query11 10467 9983 10085 9983
query12 120 87 84 84
query13 1622 362 366 362
query14 8932 7953 7080 7080
query15 203 158 163 158
query16 6814 443 428 428
query17 933 559 553 553
query18 1900 280 276 276
query19 202 145 141 141
query20 93 86 86 86
query21 199 121 97 97
query22 4276 4171 4028 4028
query23 33726 33773 33663 33663
query24 9355 3100 3118 3100
query25 678 445 424 424
query26 1533 163 155 155
query27 3266 280 279 279
query28 7605 1999 1979 1979
query29 1061 453 437 437
query30 236 160 158 158
query31 984 774 774 774
query32 98 58 67 58
query33 680 330 352 330
query34 939 482 507 482
query35 894 772 781 772
query36 1036 927 913 913
query37 227 82 81 81
query38 2898 2851 2782 2782
query39 885 819 837 819
query40 260 115 117 115
query41 47 47 50 47
query42 117 101 100 100
query43 487 430 426 426
query44 1173 736 742 736
query45 211 180 182 180
query46 1090 806 771 771
query47 1808 1712 1709 1709
query48 377 291 294 291
query49 895 419 409 409
query50 914 428 433 428
query51 6820 6686 6686 6686
query52 100 91 87 87
query53 262 182 181 181
query54 623 451 442 442
query55 76 71 75 71
query56 288 261 251 251
query57 1126 1061 1019 1019
query58 267 276 274 274
query59 2767 2634 2429 2429
query60 295 272 273 272
query61 101 97 99 97
query62 883 669 681 669
query63 217 177 191 177
query64 5776 1917 1896 1896
query65 3151 3092 3101 3092
query66 1026 359 332 332
query67 15483 14808 14770 14770
query68 5448 547 558 547
query69 683 393 307 307
query70 1102 1121 1083 1083
query71 440 273 278 273
query72 7753 2710 2500 2500
query73 912 321 327 321
query74 6014 5702 5591 5591
query75 3528 2753 2706 2706
query76 3120 1380 1344 1344
query77 562 306 307 306
query78 9410 8958 8927 8927
query79 1733 531 514 514
query80 910 510 507 507
query81 564 232 225 225
query82 729 127 126 126
query83 251 165 165 165
query84 264 79 78 78
query85 1380 330 298 298
query86 469 307 284 284
query87 3253 3102 3101 3101
query88 3698 2404 2396 2396
query89 387 290 300 290
query90 1777 188 192 188
query91 129 102 104 102
query92 62 50 50 50
query93 1999 602 603 602
query94 797 286 307 286
query95 373 318 266 266
query96 602 277 277 277
query97 3188 3088 3022 3022
query98 231 202 202 202
query99 1664 1297 1283 1283
Total cold run time: 263645 ms
Total hot run time: 169944 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](mtmv)hive cache add partitionId to partitionName Map [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #38525:
URL: https://github.com/apache/doris/pull/38525#issuecomment-2257813872
<details>
<summary>TPC-H: <b>Total hot run time: 41694 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 3b401b073e35507eb202ab5c4057051da91289c8, data reload: false
------ Round 1 ----------------------------------
q1 17604 4071 4103 4071
q2 2011 198 197 197
q3 10489 1317 1316 1316
q4 10160 851 939 851
q5 7624 3020 2944 2944
q6 220 139 138 138
q7 1036 611 619 611
q8 9444 1916 1918 1916
q9 8553 6605 6593 6593
q10 8775 3812 3802 3802
q11 433 249 250 249
q12 404 227 232 227
q13 17763 2962 2975 2962
q14 265 246 249 246
q15 532 489 500 489
q16 539 385 388 385
q17 954 926 918 918
q18 8006 7199 7201 7199
q19 1482 1216 1209 1209
q20 561 314 348 314
q21 5372 4777 4890 4777
q22 351 280 283 280
Total cold run time: 112578 ms
Total hot run time: 41694 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4066 4017 4033 4017
q2 325 226 229 226
q3 3046 2996 3132 2996
q4 2001 2032 1981 1981
q5 5559 5518 5468 5468
q6 216 139 134 134
q7 2132 1775 1879 1775
q8 3324 3367 3358 3358
q9 8663 8668 8778 8668
q10 4004 4042 4011 4011
q11 557 450 447 447
q12 754 585 624 585
q13 16434 3134 3166 3134
q14 301 272 272 272
q15 541 487 495 487
q16 455 417 412 412
q17 1751 1711 1701 1701
q18 8215 7852 7650 7650
q19 2016 1718 1716 1716
q20 2142 1882 1844 1844
q21 5903 5600 5315 5315
q22 535 469 468 468
Total cold run time: 72940 ms
Total hot run time: 56665 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](mtmv)hive cache add partitionId to partitionName Map [doris]
Posted by "morrySnow (via GitHub)" <gi...@apache.org>.
morrySnow merged PR #38525:
URL: https://github.com/apache/doris/pull/38525
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org