You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "ByteYue (via GitHub)" <gi...@apache.org> on 2024/07/09 10:38:52 UTC
[PR] [enhance](HDFS) Add one config to control hdfs memory limiter [doris]
ByteYue opened a new pull request, #37547:
URL: https://github.com/apache/doris/pull/37547
## Proposed changes
Issue Number: close #xxx
<!--Describe your changes.-->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](HDFS) Add one config to control hdfs memory limiter [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #37547:
URL: https://github.com/apache/doris/pull/37547#issuecomment-2217959279
<details>
<summary>ClickBench: <b>Total hot run time: 30.84 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 15ae2c83c3316892a541eadfd42785b87b036994, data reload: false
query1 0.03 0.03 0.03
query2 0.08 0.04 0.04
query3 0.22 0.05 0.06
query4 1.67 0.06 0.07
query5 0.51 0.49 0.48
query6 1.14 0.73 0.73
query7 0.02 0.01 0.01
query8 0.05 0.04 0.04
query9 0.54 0.49 0.49
query10 0.54 0.55 0.55
query11 0.14 0.11 0.12
query12 0.14 0.11 0.12
query13 0.60 0.59 0.58
query14 0.74 0.78 0.77
query15 0.85 0.81 0.82
query16 0.37 0.36 0.38
query17 0.96 1.04 1.03
query18 0.23 0.23 0.22
query19 1.88 1.75 1.78
query20 0.01 0.01 0.01
query21 15.38 0.78 0.66
query22 4.24 6.69 2.02
query23 18.33 1.40 1.31
query24 2.19 0.23 0.22
query25 0.17 0.09 0.09
query26 0.31 0.21 0.21
query27 0.46 0.24 0.23
query28 13.19 1.00 1.00
query29 12.62 3.28 3.30
query30 0.27 0.06 0.06
query31 2.86 0.41 0.38
query32 3.28 0.48 0.48
query33 2.90 2.87 2.90
query34 16.95 4.34 4.35
query35 4.41 4.42 4.44
query36 0.64 0.46 0.47
query37 0.18 0.16 0.17
query38 0.17 0.15 0.15
query39 0.04 0.04 0.03
query40 0.15 0.11 0.13
query41 0.09 0.05 0.05
query42 0.05 0.04 0.04
query43 0.05 0.04 0.04
Total cold run time: 109.65 s
Total hot run time: 30.84 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](HDFS) Add one config to control hdfs memory limiter [doris]
Posted by "ByteYue (via GitHub)" <gi...@apache.org>.
ByteYue commented on PR #37547:
URL: https://github.com/apache/doris/pull/37547#issuecomment-2217730627
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](HDFS) Add one config to control hdfs memory limiter [doris]
Posted by "ByteYue (via GitHub)" <gi...@apache.org>.
ByteYue commented on PR #37547:
URL: https://github.com/apache/doris/pull/37547#issuecomment-2217331570
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](HDFS) Add one config to control hdfs memory limiter [doris]
Posted by "dataroaring (via GitHub)" <gi...@apache.org>.
dataroaring merged PR #37547:
URL: https://github.com/apache/doris/pull/37547
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](HDFS) Add one config to control hdfs memory limiter [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #37547:
URL: https://github.com/apache/doris/pull/37547#issuecomment-2217342971
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](HDFS) Add one config to control hdfs memory limiter [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #37547:
URL: https://github.com/apache/doris/pull/37547#issuecomment-2217739506
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](HDFS) Add one config to control hdfs memory limiter [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #37547:
URL: https://github.com/apache/doris/pull/37547#issuecomment-2217293322
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR)
Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website).
See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](HDFS) Add one config to control hdfs memory limiter [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #37547:
URL: https://github.com/apache/doris/pull/37547#issuecomment-2217921796
<details>
<summary>TPC-H: <b>Total hot run time: 40099 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 15ae2c83c3316892a541eadfd42785b87b036994, data reload: false
------ Round 1 ----------------------------------
q1 17615 5055 4344 4344
q2 2009 195 186 186
q3 10510 1232 1117 1117
q4 10200 862 768 768
q5 7498 2706 2634 2634
q6 225 140 139 139
q7 960 606 609 606
q8 9228 2104 2100 2100
q9 8898 6517 6479 6479
q10 9070 3737 3702 3702
q11 513 243 239 239
q12 424 243 241 241
q13 17778 2984 2987 2984
q14 281 226 230 226
q15 519 486 499 486
q16 506 375 378 375
q17 972 694 744 694
q18 8006 7508 7435 7435
q19 6559 1630 1456 1456
q20 694 333 331 331
q21 4934 3214 4098 3214
q22 408 343 353 343
Total cold run time: 117807 ms
Total hot run time: 40099 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4430 4303 4279 4279
q2 387 275 269 269
q3 3091 2958 2926 2926
q4 1993 1747 1758 1747
q5 5542 5476 5566 5476
q6 237 137 139 137
q7 2253 1884 1897 1884
q8 3341 3440 3438 3438
q9 8739 8904 8735 8735
q10 3992 3814 3886 3814
q11 603 497 490 490
q12 808 642 650 642
q13 16362 3152 3163 3152
q14 318 287 277 277
q15 538 495 479 479
q16 492 422 430 422
q17 1824 1517 1495 1495
q18 8152 7944 8022 7944
q19 2125 1716 1738 1716
q20 2980 1873 1841 1841
q21 5181 4903 4868 4868
q22 680 570 560 560
Total cold run time: 74068 ms
Total hot run time: 56591 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](HDFS) Add one config to control hdfs memory limiter [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #37547:
URL: https://github.com/apache/doris/pull/37547#issuecomment-2219568336
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](HDFS) Add one config to control hdfs memory limiter [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #37547:
URL: https://github.com/apache/doris/pull/37547#issuecomment-2217304214
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](HDFS) Add one config to control hdfs memory limiter [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #37547:
URL: https://github.com/apache/doris/pull/37547#issuecomment-2217338921
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](HDFS) Add one config to control hdfs memory limiter [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #37547:
URL: https://github.com/apache/doris/pull/37547#issuecomment-2217946168
<details>
<summary>TPC-DS: <b>Total hot run time: 174328 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 15ae2c83c3316892a541eadfd42785b87b036994, data reload: false
query1 915 375 378 375
query2 6379 2478 2449 2449
query3 6630 205 215 205
query4 27892 17493 17161 17161
query5 3588 519 478 478
query6 266 174 170 170
query7 4591 293 279 279
query8 328 305 309 305
query9 8408 2388 2355 2355
query10 434 276 274 274
query11 10554 10010 10129 10010
query12 120 88 86 86
query13 1651 388 373 373
query14 10278 8569 7622 7622
query15 241 190 189 189
query16 7768 314 306 306
query17 1722 545 522 522
query18 1898 277 273 273
query19 201 151 150 150
query20 92 85 82 82
query21 209 131 124 124
query22 4273 4216 4031 4031
query23 33984 33601 33595 33595
query24 9442 2903 2834 2834
query25 584 404 441 404
query26 715 149 148 148
query27 2261 285 283 283
query28 5964 2129 2131 2129
query29 923 638 640 638
query30 243 149 150 149
query31 980 779 769 769
query32 104 55 54 54
query33 667 313 322 313
query34 880 493 513 493
query35 698 572 569 569
query36 1125 1012 966 966
query37 149 83 80 80
query38 2931 2861 2816 2816
query39 885 840 845 840
query40 218 125 121 121
query41 57 51 53 51
query42 115 97 99 97
query43 617 538 544 538
query44 1101 713 717 713
query45 193 167 170 167
query46 1074 756 711 711
query47 1848 1795 1753 1753
query48 365 315 293 293
query49 836 408 411 408
query50 776 397 393 393
query51 6955 6707 6827 6707
query52 109 94 93 93
query53 358 299 291 291
query54 863 445 440 440
query55 75 73 71 71
query56 292 271 281 271
query57 1112 1040 1027 1027
query58 246 248 263 248
query59 3542 3262 3248 3248
query60 296 300 282 282
query61 100 94 106 94
query62 788 660 632 632
query63 328 303 294 294
query64 9231 2155 7432 2155
query65 3153 3115 3140 3115
query66 729 338 332 332
query67 15408 15008 14849 14849
query68 5089 533 536 533
query69 666 424 361 361
query70 1182 1132 1086 1086
query71 429 286 269 269
query72 7451 5482 5102 5102
query73 757 321 316 316
query74 6061 5593 5454 5454
query75 3519 2686 2687 2686
query76 3028 961 971 961
query77 608 298 325 298
query78 9446 8901 8905 8901
query79 2618 519 503 503
query80 2239 485 479 479
query81 595 222 229 222
query82 1406 133 139 133
query83 295 173 166 166
query84 264 96 88 88
query85 1492 328 297 297
query86 468 314 320 314
query87 3252 3167 3081 3081
query88 3553 2355 2340 2340
query89 483 393 383 383
query90 1960 190 190 190
query91 131 104 103 103
query92 58 51 50 50
query93 2465 506 508 506
query94 1385 218 294 218
query95 415 313 318 313
query96 597 264 267 264
query97 3205 3022 2963 2963
query98 219 204 196 196
query99 1556 1282 1259 1259
Total cold run time: 278650 ms
Total hot run time: 174328 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](HDFS) Add one config to control hdfs memory limiter [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #37547:
URL: https://github.com/apache/doris/pull/37547#issuecomment-2219568430
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org