You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "cambyzju (via GitHub)" <gi...@apache.org> on 2026/01/15 07:20:45 UTC
[PR] [feat](libhdfs3) support dfs.client.use.datanode.hostname on libhdfs3 [doris]
cambyzju opened a new pull request, #59915:
URL: https://github.com/apache/doris/pull/59915
### What problem does this PR solve?
support `dfs.client.use.datanode.hostname` on libhdfs3, then we could connect hdfs inside docker.
### Release note
None
### Check List (For Author)
- Test <!-- At least one of them must be included. -->
- [ ] Regression test
- [ ] Unit Test
- [ ] Manual test (add detailed scripts or steps below)
- [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
- [ ] Previous test can cover this change.
- [ ] No code files have been changed.
- [ ] Other reason <!-- Add your reason? -->
- Behavior changed:
- [ ] No.
- [ ] Yes. <!-- Explain the behavior change -->
- Does this need documentation?
- [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg: https://github.com/apache/doris-website/pull/1214 -->
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR should merge into -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](libhdfs3) support dfs.client.use.datanode.hostname for libhdfs3 [doris]
Posted by "cambyzju (via GitHub)" <gi...@apache.org>.
cambyzju commented on PR #59915:
URL: https://github.com/apache/doris/pull/59915#issuecomment-3760195975
run p0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](libhdfs3) support dfs.client.use.datanode.hostname on libhdfs3 [doris]
Posted by "Thearas (via GitHub)" <gi...@apache.org>.
Thearas commented on PR #59915:
URL: https://github.com/apache/doris/pull/59915#issuecomment-3753215055
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR).
Please clearly describe your PR:
1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
3. What features were added. Why was this function added?
4. Which code was refactored and why was this part of the code refactored?
5. Which functions were optimized and what is the difference before and after the optimization?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](libhdfs3) support dfs.client.use.datanode.hostname for libhdfs3 [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #59915:
URL: https://github.com/apache/doris/pull/59915#issuecomment-3760636820
# BE Regression && UT Coverage Report
Increment line coverage `100% (0/0)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/59915_b61312845f29672ca2e71b834fba34369bfd3f35_merge/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/59915_b61312845f29672ca2e71b834fba34369bfd3f35_merge/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 73.85% (25936/35122) |
| Line Coverage | 61.26% (275795/450183) |
| Region Coverage | 56.10% (229947/409885) |
| Branch Coverage | 57.99% (99004/170735) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](libhdfs3) support dfs.client.use.datanode.hostname on libhdfs3 [doris]
Posted by "cambyzju (via GitHub)" <gi...@apache.org>.
cambyzju commented on PR #59915:
URL: https://github.com/apache/doris/pull/59915#issuecomment-3753215321
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](libhdfs3) support dfs.client.use.datanode.hostname for libhdfs3 [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #59915:
URL: https://github.com/apache/doris/pull/59915#issuecomment-3754244835
# BE Regression && UT Coverage Report
Increment line coverage `100% (0/0)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/59915_b61312845f29672ca2e71b834fba34369bfd3f35_merge/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/59915_b61312845f29672ca2e71b834fba34369bfd3f35_merge/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 73.85% (25936/35122) |
| Line Coverage | 61.26% (275795/450183) |
| Region Coverage | 56.10% (229947/409885) |
| Branch Coverage | 57.99% (99004/170735) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](libhdfs3) support dfs.client.use.datanode.hostname for libhdfs3 [doris]
Posted by "cambyzju (via GitHub)" <gi...@apache.org>.
cambyzju commented on PR #59915:
URL: https://github.com/apache/doris/pull/59915#issuecomment-3760197098
run cloud_p0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](libhdfs3) support dfs.client.use.datanode.hostname for libhdfs3 [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59915:
URL: https://github.com/apache/doris/pull/59915#issuecomment-3753388200
<details>
<summary>TPC-H: <b>Total hot run time: 31327 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit b61312845f29672ca2e71b834fba34369bfd3f35, data reload: false
------ Round 1 ----------------------------------
q1 17600 4221 4138 4138
q2 2039 366 249 249
q3 10144 1254 697 697
q4 10205 827 299 299
q5 7517 2088 1771 1771
q6 193 174 139 139
q7 912 785 656 656
q8 9271 1394 1095 1095
q9 5100 4608 4650 4608
q10 6770 1773 1382 1382
q11 517 314 272 272
q12 679 755 642 642
q13 17784 3796 3105 3105
q14 281 305 279 279
q15 588 513 507 507
q16 684 662 649 649
q17 636 775 537 537
q18 6616 6351 6234 6234
q19 1091 965 626 626
q20 401 366 241 241
q21 2980 2519 2266 2266
q22 1060 1016 935 935
Total cold run time: 103068 ms
Total hot run time: 31327 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4101 4058 4049 4049
q2 317 397 322 322
q3 2061 2572 2252 2252
q4 1313 1761 1370 1370
q5 4079 3998 4081 3998
q6 208 167 133 133
q7 1878 1811 1670 1670
q8 2729 2524 2471 2471
q9 7136 7209 7165 7165
q10 2400 2801 2304 2304
q11 563 487 468 468
q12 706 789 610 610
q13 3621 4032 3608 3608
q14 412 357 282 282
q15 551 504 502 502
q16 640 685 619 619
q17 1141 1383 1421 1383
q18 8389 7657 7768 7657
q19 839 861 855 855
q20 1945 2052 1909 1909
q21 4911 4591 4302 4302
q22 1087 1003 958 958
Total cold run time: 51027 ms
Total hot run time: 48887 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](libhdfs3) support dfs.client.use.datanode.hostname for libhdfs3 [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59915:
URL: https://github.com/apache/doris/pull/59915#issuecomment-3753440920
<details>
<summary>TPC-DS: <b>Total hot run time: 173948 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit b61312845f29672ca2e71b834fba34369bfd3f35, data reload: false
query5 4406 614 483 483
query6 348 221 216 216
query7 4224 454 250 250
query8 353 244 240 240
query9 8735 2894 2907 2894
query10 552 380 338 338
query11 15362 15136 14885 14885
query12 171 113 111 111
query13 1239 455 370 370
query14 5569 3107 2853 2853
query14_1 2725 2713 2714 2713
query15 199 192 175 175
query16 989 468 455 455
query17 993 686 586 586
query18 2444 444 343 343
query19 230 217 201 201
query20 119 116 115 115
query21 218 143 126 126
query22 3944 4038 3750 3750
query23 15988 15399 15053 15053
query23_1 15403 15437 15419 15419
query24 7193 1576 1224 1224
query24_1 1178 1192 1195 1192
query25 570 479 434 434
query26 1229 262 150 150
query27 2772 442 273 273
query28 4590 2120 2123 2120
query29 795 548 462 462
query30 316 232 207 207
query31 794 636 562 562
query32 92 74 80 74
query33 548 368 323 323
query34 904 877 539 539
query35 732 765 668 668
query36 878 921 834 834
query37 134 99 90 90
query38 2741 2695 2651 2651
query39 786 745 744 744
query39_1 709 706 708 706
query40 227 143 124 124
query41 77 70 71 70
query42 111 106 105 105
query43 499 491 450 450
query44 1373 734 739 734
query45 187 187 182 182
query46 832 966 575 575
query47 1356 1438 1348 1348
query48 330 335 242 242
query49 634 432 363 363
query50 619 271 206 206
query51 3821 3811 3816 3811
query52 100 108 102 102
query53 291 334 280 280
query54 309 290 273 273
query55 84 79 82 79
query56 323 325 320 320
query57 1038 1046 889 889
query58 260 254 251 251
query59 2051 2114 2032 2032
query60 341 321 315 315
query61 162 158 154 154
query62 384 342 302 302
query63 293 266 266 266
query64 4969 1295 985 985
query65 3837 3729 3700 3700
query66 1462 415 317 317
query67 15546 15576 15376 15376
query68 2691 1087 724 724
query69 455 358 332 332
query70 1017 925 942 925
query71 329 306 296 296
query72 5546 3346 3295 3295
query73 591 714 318 318
query74 8678 8767 8482 8482
query75 2757 2815 2479 2479
query76 2363 1059 687 687
query77 366 367 303 303
query78 9775 9862 9164 9164
query79 1083 912 587 587
query80 1267 563 491 491
query81 549 259 233 233
query82 1019 140 106 106
query83 335 253 240 240
query84 287 111 91 91
query85 944 509 446 446
query86 443 300 312 300
query87 2889 2911 2737 2737
query88 3524 2589 2551 2551
query89 399 360 330 330
query90 2004 170 168 168
query91 174 161 134 134
query92 73 72 70 70
query93 1015 897 528 528
query94 647 324 310 310
query95 579 334 376 334
query96 653 509 230 230
query97 2337 2357 2281 2281
query98 214 206 194 194
query99 582 566 504 504
Total cold run time: 247226 ms
Total hot run time: 173948 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](libhdfs3) support dfs.client.use.datanode.hostname for libhdfs3 [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59915:
URL: https://github.com/apache/doris/pull/59915#issuecomment-3753484060
<details>
<summary>ClickBench: <b>Total hot run time: 26.75 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit b61312845f29672ca2e71b834fba34369bfd3f35, data reload: false
query1 0.06 0.04 0.05
query2 0.09 0.05 0.05
query3 0.26 0.09 0.08
query4 1.60 0.11 0.12
query5 0.27 0.27 0.24
query6 1.14 0.67 0.64
query7 0.03 0.03 0.03
query8 0.05 0.04 0.04
query9 0.58 0.51 0.50
query10 0.53 0.55 0.55
query11 0.14 0.10 0.10
query12 0.15 0.12 0.10
query13 0.61 0.59 0.60
query14 0.95 0.96 0.94
query15 0.79 0.77 0.76
query16 0.41 0.40 0.40
query17 1.06 1.06 0.98
query18 0.23 0.21 0.22
query19 1.97 1.86 1.87
query20 0.02 0.01 0.01
query21 15.44 0.27 0.14
query22 4.98 0.06 0.05
query23 15.82 0.28 0.11
query24 1.57 0.25 0.69
query25 0.08 0.06 0.06
query26 0.14 0.13 0.13
query27 0.06 0.04 0.05
query28 4.28 1.08 0.88
query29 12.54 3.85 3.12
query30 0.28 0.14 0.13
query31 2.82 0.62 0.40
query32 3.23 0.56 0.46
query33 2.99 2.99 3.04
query34 15.78 5.04 4.49
query35 4.42 4.46 4.42
query36 0.66 0.50 0.50
query37 0.10 0.06 0.07
query38 0.07 0.04 0.03
query39 0.04 0.03 0.03
query40 0.17 0.14 0.13
query41 0.08 0.03 0.04
query42 0.04 0.03 0.03
query43 0.05 0.04 0.04
Total cold run time: 96.58 s
Total hot run time: 26.75 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](libhdfs3) support dfs.client.use.datanode.hostname for libhdfs3 [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #59915:
URL: https://github.com/apache/doris/pull/59915#issuecomment-3874924880
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](libhdfs3) support dfs.client.use.datanode.hostname for libhdfs3 [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #59915:
URL: https://github.com/apache/doris/pull/59915#issuecomment-3760766517
# BE Regression && UT Coverage Report
Increment line coverage `100% (0/0)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/59915_b61312845f29672ca2e71b834fba34369bfd3f35_merge/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/59915_b61312845f29672ca2e71b834fba34369bfd3f35_merge/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 73.82% (25928/35122) |
| Line Coverage | 61.24% (275682/450183) |
| Region Coverage | 56.09% (229895/409885) |
| Branch Coverage | 57.97% (98973/170735) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](libhdfs3) support dfs.client.use.datanode.hostname for libhdfs3 [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #59915:
URL: https://github.com/apache/doris/pull/59915#issuecomment-3753722990
# BE UT Coverage Report
Increment line coverage ` ` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/b61312845f29672ca2e71b834fba34369bfd3f35_b61312845f29672ca2e71b834fba34369bfd3f35/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/b61312845f29672ca2e71b834fba34369bfd3f35_b61312845f29672ca2e71b834fba34369bfd3f35/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 52.98% (18986/35839) |
| Line Coverage | 39.01% (175854/450768) |
| Region Coverage | 33.60% (136196/405377) |
| Branch Coverage | 34.64% (58877/169974) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](libhdfs3) support dfs.client.use.datanode.hostname for libhdfs3 [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #59915:
URL: https://github.com/apache/doris/pull/59915#issuecomment-3874925017
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](libhdfs3) support dfs.client.use.datanode.hostname for libhdfs3 [doris]
Posted by "cambyzju (via GitHub)" <gi...@apache.org>.
cambyzju merged PR #59915:
URL: https://github.com/apache/doris/pull/59915
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org