You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "freemandealer (via GitHub)" <gi...@apache.org> on 2024/10/29 12:18:31 UTC
[PR] [enhancement](cloud) expose file cache capacity for each disk [doris]
freemandealer opened a new pull request, #42867:
URL: https://github.com/apache/doris/pull/42867
expose file cache capacity for better monitoring and easier to calculate file cache space usage using prometheus.
## Proposed changes
Issue Number: close #xxx
<!--Describe your changes.-->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhancement](cloud) expose file cache capacity for each disk [doris]
Posted by "freemandealer (via GitHub)" <gi...@apache.org>.
freemandealer commented on PR #42867:
URL: https://github.com/apache/doris/pull/42867#issuecomment-2444055752
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhancement](cloud) expose file cache capacity for each disk [doris]
Posted by "dataroaring (via GitHub)" <gi...@apache.org>.
dataroaring merged PR #42867:
URL: https://github.com/apache/doris/pull/42867
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhancement](cloud) expose file cache capacity for each disk [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #42867:
URL: https://github.com/apache/doris/pull/42867#issuecomment-2444063005
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhancement](cloud) expose file cache capacity for each disk [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #42867:
URL: https://github.com/apache/doris/pull/42867#issuecomment-2444054238
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR)
Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website).
See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhancement](cloud) expose file cache capacity for each disk [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #42867:
URL: https://github.com/apache/doris/pull/42867#issuecomment-2444165851
<details>
<summary>TPC-DS: <b>Total hot run time: 197502 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 81fb0dd4fec5aed841d162a18e8a9eee94f91b49, data reload: false
query1 1226 1002 1032 1002
query2 6240 2093 2059 2059
query3 11519 4936 4791 4791
query4 33584 23591 23642 23591
query5 4694 448 437 437
query6 280 182 174 174
query7 3980 291 307 291
query8 295 237 231 231
query9 9570 2752 2740 2740
query10 502 256 251 251
query11 18275 15248 15381 15248
query12 153 103 102 102
query13 1568 431 404 404
query14 10265 7426 7753 7426
query15 259 183 191 183
query16 8171 483 480 480
query17 1580 601 575 575
query18 2161 315 326 315
query19 370 152 155 152
query20 125 116 120 116
query21 212 111 109 109
query22 4790 4638 4332 4332
query23 35048 34311 34150 34150
query24 11465 2725 2777 2725
query25 655 386 402 386
query26 1651 154 156 154
query27 2567 282 301 282
query28 7774 2462 2471 2462
query29 1001 422 414 414
query30 260 158 160 158
query31 1021 816 797 797
query32 90 55 56 55
query33 762 264 265 264
query34 988 511 511 511
query35 1045 875 881 875
query36 1114 938 946 938
query37 197 81 76 76
query38 4475 4220 4341 4220
query39 1534 1427 1428 1427
query40 255 97 97 97
query41 49 48 47 47
query42 111 103 100 100
query43 518 479 510 479
query44 1279 827 824 824
query45 184 166 169 166
query46 1134 699 698 698
query47 1927 1855 1842 1842
query48 432 323 339 323
query49 1020 404 403 403
query50 801 397 413 397
query51 7144 7040 6975 6975
query52 99 93 85 85
query53 254 181 175 175
query54 1250 412 392 392
query55 83 75 79 75
query56 251 233 240 233
query57 1330 1180 1143 1143
query58 219 200 197 197
query59 3220 2991 2882 2882
query60 273 247 250 247
query61 100 95 99 95
query62 868 715 675 675
query63 210 189 177 177
query64 5040 608 623 608
query65 3280 3236 3211 3211
query66 1337 313 297 297
query67 16450 15947 15729 15729
query68 4844 570 573 570
query69 426 250 255 250
query70 1230 1168 1151 1151
query71 382 263 257 257
query72 6295 3971 3956 3956
query73 770 363 369 363
query74 10181 9032 8979 8979
query75 3437 2658 2672 2658
query76 2712 929 937 929
query77 375 277 279 277
query78 10686 9571 9551 9551
query79 1134 613 588 588
query80 766 463 419 419
query81 528 246 237 237
query82 1328 115 113 113
query83 224 147 132 132
query84 218 70 68 68
query85 1105 287 282 282
query86 319 266 301 266
query87 4838 4737 4616 4616
query88 3489 2194 2247 2194
query89 410 285 290 285
query90 2017 183 184 183
query91 132 104 97 97
query92 57 46 49 46
query93 1190 561 547 547
query94 755 284 288 284
query95 348 250 244 244
query96 619 276 284 276
query97 2984 2721 2716 2716
query98 210 196 197 196
query99 1519 1344 1297 1297
Total cold run time: 306112 ms
Total hot run time: 197502 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhancement](cloud) expose file cache capacity for each disk [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #42867:
URL: https://github.com/apache/doris/pull/42867#issuecomment-2444134574
TeamCity be ut coverage result:
Function Coverage: 37.47% (9718/25938)
Line Coverage: 28.72% (80591/280619)
Region Coverage: 28.14% (41660/148038)
Branch Coverage: 24.71% (21165/85666)
Coverage Report: http://coverage.selectdb-in.cc/coverage/81fb0dd4fec5aed841d162a18e8a9eee94f91b49_81fb0dd4fec5aed841d162a18e8a9eee94f91b49/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhancement](cloud) expose file cache capacity for each disk [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #42867:
URL: https://github.com/apache/doris/pull/42867#issuecomment-2444181673
<details>
<summary>ClickBench: <b>Total hot run time: 32.36 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 81fb0dd4fec5aed841d162a18e8a9eee94f91b49, data reload: false
query1 0.04 0.04 0.03
query2 0.07 0.03 0.04
query3 0.23 0.06 0.06
query4 1.64 0.10 0.10
query5 0.41 0.39 0.40
query6 1.17 0.68 0.66
query7 0.02 0.02 0.01
query8 0.04 0.03 0.03
query9 0.57 0.50 0.49
query10 0.55 0.56 0.55
query11 0.13 0.10 0.11
query12 0.13 0.11 0.11
query13 0.63 0.59 0.59
query14 2.70 2.76 2.71
query15 0.89 0.82 0.82
query16 0.38 0.40 0.38
query17 1.06 1.00 1.01
query18 0.19 0.20 0.20
query19 1.96 1.84 1.87
query20 0.02 0.01 0.01
query21 15.36 0.59 0.59
query22 2.33 3.05 1.43
query23 17.03 0.78 0.87
query24 3.25 1.27 1.48
query25 0.18 0.12 0.08
query26 0.68 0.13 0.13
query27 0.04 0.04 0.06
query28 10.12 1.09 1.08
query29 12.63 3.30 3.30
query30 0.24 0.06 0.05
query31 2.97 0.38 0.38
query32 3.31 0.46 0.46
query33 2.93 3.02 3.03
query34 17.11 4.46 4.41
query35 4.45 4.46 4.50
query36 0.66 0.48 0.48
query37 0.08 0.06 0.06
query38 0.05 0.03 0.04
query39 0.03 0.02 0.02
query40 0.16 0.12 0.12
query41 0.08 0.02 0.02
query42 0.03 0.02 0.02
query43 0.03 0.03 0.02
Total cold run time: 106.58 s
Total hot run time: 32.36 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhancement](cloud) expose file cache capacity for each disk [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #42867:
URL: https://github.com/apache/doris/pull/42867#issuecomment-2444128386
<details>
<summary>TPC-H: <b>Total hot run time: 41547 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 81fb0dd4fec5aed841d162a18e8a9eee94f91b49, data reload: false
------ Round 1 ----------------------------------
q1 17600 7730 7348 7348
q2 2073 169 162 162
q3 10955 1148 1185 1148
q4 10577 813 901 813
q5 7757 3109 3078 3078
q6 234 152 150 150
q7 1026 609 614 609
q8 9351 1992 2020 1992
q9 6687 6457 6473 6457
q10 7070 2394 2439 2394
q11 440 251 247 247
q12 399 220 213 213
q13 17807 3036 3004 3004
q14 242 214 210 210
q15 565 508 507 507
q16 646 586 602 586
q17 981 553 568 553
q18 7445 6718 6716 6716
q19 1333 1115 984 984
q20 464 187 183 183
q21 4017 3200 3440 3200
q22 1122 993 997 993
Total cold run time: 108791 ms
Total hot run time: 41547 ms
----- Round 2, with runtime_filter_mode=off -----
q1 7305 7286 7335 7286
q2 329 229 227 227
q3 3057 2989 3052 2989
q4 2181 1890 1863 1863
q5 6119 5809 5848 5809
q6 237 142 145 142
q7 2221 1845 1855 1845
q8 3395 3577 3557 3557
q9 9017 8963 8968 8963
q10 3583 3576 3900 3576
q11 589 493 488 488
q12 867 642 600 600
q13 6400 3207 3208 3207
q14 327 286 285 285
q15 581 523 527 523
q16 684 636 665 636
q17 1892 1661 1636 1636
q18 8302 7888 7773 7773
q19 1712 1566 1569 1566
q20 2103 1899 1893 1893
q21 5570 5450 5535 5450
q22 1121 1036 1062 1036
Total cold run time: 67592 ms
Total hot run time: 61350 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhancement](cloud) expose file cache capacity for each disk [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #42867:
URL: https://github.com/apache/doris/pull/42867#issuecomment-2444212145
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhancement](cloud) expose file cache capacity for each disk [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #42867:
URL: https://github.com/apache/doris/pull/42867#issuecomment-2444212231
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org