You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "CalvinKirs (via GitHub)" <gi...@apache.org> on 2025/10/21 13:08:48 UTC
[PR] [feat](thirdparty) Refactor HDFS dependency management: move HDFS client jars to FE and slim down BE thirdparty [doris]
CalvinKirs opened a new pull request, #57202:
URL: https://github.com/apache/doris/pull/57202
### Background
Previously, BE depended on the HDFS official distribution through the thirdparty module. This approach directly compiled and packaged Hadoop’s full dependencies, including large and unnecessary modules such as YARN, MapReduce, etc. As a result:
The build artifact contained many duplicate jars.
The total size was unnecessarily large (hundreds of MB).
BE only acted as an HDFS client, not a Hadoop runtime, so most of these dependencies were not needed.
### Changes in this PR
Move all HDFS-related jars to FE for unified dependency management, since FE already uses HDFS extensively.
Retain necessary native/C++ dependencies under thirdparty for BE.
Do not build an uber-jar for HDFS (unlike other BE thirdparty libs).
Exclude unused Hadoop modules (e.g., YARN, MR, tools).
The resulting target structure now includes:
```
target/
├── classes
├── lib/
│ └── (~57M total)
├── hadoop-deps.jar
└── ...
```
Compared with the previous build, the total artifact size has been reduced by ~150 MB.
### Rationale
Since BE only functions as a client to access HDFS, managing Java-side HDFS dependencies in FE is cleaner and more maintainable. If BE needs custom logic (e.g., optimized HDFS client), we can extend or override the relevant classes and control class loading order.
Impact
Build size reduced by ~150 MB
Simpler dependency graph
### Notes
Native/C++ HDFS dependencies remain managed under thirdparty.
FE now provides necessary jars for BE at runtime.
No need to modify existing C++ build logic.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](thirdparty) Refactor HDFS dependency management: move HDFS client jars to FE and slim down BE thirdparty [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #57202:
URL: https://github.com/apache/doris/pull/57202#issuecomment-3450123886
<details>
<summary>TPC-DS: <b>Total hot run time: 189151 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 1f63c0c51665a1c0c71a3fce78fb9fc69747b0ef, data reload: false
query1 1081 412 398 398
query2 6556 1689 1687 1687
query3 6763 224 221 221
query4 26726 24148 23006 23006
query5 5912 656 476 476
query6 380 252 253 252
query7 4645 502 297 297
query8 307 257 252 252
query9 8719 2563 2572 2563
query10 549 342 282 282
query11 15797 15102 14892 14892
query12 180 122 115 115
query13 1688 549 439 439
query14 11635 9232 9143 9143
query15 216 190 185 185
query16 7674 698 506 506
query17 1612 792 641 641
query18 2115 486 355 355
query19 236 223 192 192
query20 139 154 131 131
query21 224 139 123 123
query22 4606 4649 4545 4545
query23 35079 33825 33954 33825
query24 9096 2504 2520 2504
query25 610 542 468 468
query26 1290 283 176 176
query27 3183 529 373 373
query28 4669 2248 2198 2198
query29 842 660 514 514
query30 311 243 203 203
query31 904 832 783 783
query32 86 92 73 73
query33 702 410 344 344
query34 800 846 516 516
query35 809 835 726 726
query36 942 984 905 905
query37 140 106 85 85
query38 3496 3521 3484 3484
query39 1489 1413 1447 1413
query40 224 124 117 117
query41 67 65 67 65
query42 122 112 119 112
query43 495 487 476 476
query44 1213 738 734 734
query45 184 178 174 174
query46 882 990 626 626
query47 1769 1802 1759 1759
query48 412 435 313 313
query49 778 507 425 425
query50 654 689 400 400
query51 3936 3957 3852 3852
query52 110 105 99 99
query53 237 266 201 201
query54 601 580 550 550
query55 85 88 86 86
query56 325 314 309 309
query57 1186 1206 1125 1125
query58 293 281 278 278
query59 2533 2572 2559 2559
query60 361 361 361 361
query61 169 156 152 152
query62 813 725 693 693
query63 234 195 188 188
query64 4428 1152 848 848
query65 4039 3925 3958 3925
query66 1022 429 335 335
query67 15447 15037 14975 14975
query68 9624 887 604 604
query69 517 319 289 289
query70 1338 1217 1302 1217
query71 527 350 320 320
query72 5506 4777 4769 4769
query73 723 562 356 356
query74 8899 9308 8715 8715
query75 4592 3340 2874 2874
query76 4021 1149 727 727
query77 1047 394 315 315
query78 9626 9906 8900 8900
query79 1103 821 595 595
query80 624 589 495 495
query81 486 266 232 232
query82 265 156 135 135
query83 273 272 248 248
query84 257 115 93 93
query85 855 466 423 423
query86 333 302 294 294
query87 3688 3797 3622 3622
query88 2859 2274 2245 2245
query89 378 325 304 304
query90 2021 218 216 216
query91 162 167 139 139
query92 83 75 68 68
query93 1105 971 642 642
query94 695 447 338 338
query95 414 330 317 317
query96 500 574 283 283
query97 2978 2988 2864 2864
query98 235 212 211 211
query99 1448 1418 1273 1273
Total cold run time: 282009 ms
Total hot run time: 189151 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](thirdparty) Refactor HDFS dependency management: move HDFS client jars to FE and slim down BE thirdparty [doris]
Posted by "CalvinKirs (via GitHub)" <gi...@apache.org>.
CalvinKirs commented on PR #57202:
URL: https://github.com/apache/doris/pull/57202#issuecomment-3442871536
run p0
run external
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](thirdparty) Refactor HDFS dependency management: move HDFS client jars to FE and slim down BE thirdparty [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #57202:
URL: https://github.com/apache/doris/pull/57202#issuecomment-3455113302
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](thirdparty) Refactor HDFS dependency management: move HDFS client jars to FE and slim down BE thirdparty [doris]
Posted by "morningman (via GitHub)" <gi...@apache.org>.
morningman commented on PR #57202:
URL: https://github.com/apache/doris/pull/57202#issuecomment-3465795871
run performance
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](thirdparty) Refactor HDFS dependency management: move HDFS client jars to FE and slim down BE thirdparty [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #57202:
URL: https://github.com/apache/doris/pull/57202#issuecomment-3454401310
<details>
<summary>ClickBench: <b>Total hot run time: 27.69 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 841b6cf07c117dd03471f939cec8ed24265ffa38, data reload: false
query1 0.06 0.05 0.05
query2 0.09 0.06 0.05
query3 0.25 0.08 0.08
query4 1.60 0.12 0.12
query5 0.28 0.27 0.26
query6 1.17 0.68 0.65
query7 0.03 0.03 0.03
query8 0.06 0.04 0.04
query9 0.63 0.52 0.52
query10 0.58 0.58 0.57
query11 0.18 0.11 0.11
query12 0.16 0.12 0.13
query13 0.64 0.61 0.60
query14 1.00 1.02 1.01
query15 0.85 0.86 0.87
query16 0.38 0.39 0.40
query17 1.01 1.05 1.05
query18 0.22 0.20 0.20
query19 1.95 1.80 1.84
query20 0.02 0.01 0.02
query21 15.44 0.20 0.15
query22 4.83 0.06 0.05
query23 15.66 0.27 0.10
query24 2.88 1.10 0.37
query25 0.07 0.05 0.06
query26 0.15 0.13 0.13
query27 0.07 0.06 0.06
query28 3.93 1.14 0.93
query29 12.62 4.03 3.28
query30 0.31 0.13 0.11
query31 2.82 0.58 0.38
query32 3.22 0.54 0.47
query33 3.12 3.08 3.11
query34 15.76 5.17 4.59
query35 4.60 4.58 4.67
query36 0.68 0.51 0.49
query37 0.10 0.07 0.07
query38 0.06 0.05 0.03
query39 0.03 0.03 0.03
query40 0.16 0.14 0.15
query41 0.09 0.04 0.03
query42 0.04 0.03 0.02
query43 0.04 0.03 0.04
Total cold run time: 97.84 s
Total hot run time: 27.69 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](thirdparty) Refactor HDFS dependency management: move HDFS client jars to FE and slim down BE thirdparty [doris]
Posted by "morningman (via GitHub)" <gi...@apache.org>.
morningman commented on code in PR #57202:
URL: https://github.com/apache/doris/pull/57202#discussion_r2467809638
##########
fe/pom.xml:
##########
@@ -167,6 +167,7 @@ under the License.
<configLocation>check/checkstyle/checkstyle.xml</configLocation>
<suppressionsLocation>check/checkstyle/suppressions.xml</suppressionsLocation>
<encoding>UTF-8</encoding>
+ <skip>true</skip>
Review Comment:
Why skip this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](thirdparty) Refactor HDFS dependency management: move HDFS client jars to FE and slim down BE thirdparty [doris]
Posted by "morningman (via GitHub)" <gi...@apache.org>.
morningman commented on PR #57202:
URL: https://github.com/apache/doris/pull/57202#issuecomment-3455111070
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](thirdparty) Refactor HDFS dependency management: move HDFS client jars to FE and slim down BE thirdparty [doris]
Posted by "CalvinKirs (via GitHub)" <gi...@apache.org>.
CalvinKirs commented on PR #57202:
URL: https://github.com/apache/doris/pull/57202#issuecomment-3442901375
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](thirdparty) Refactor HDFS dependency management: move HDFS client jars to FE and slim down BE thirdparty [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #57202:
URL: https://github.com/apache/doris/pull/57202#issuecomment-3443142761
<details>
<summary>ClickBench: <b>Total hot run time: 29.86 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 0d9a629c7b2a3d437f77e2c5a3626d7e1a4bb2a3, data reload: false
query1 0.07 0.06 0.05
query2 0.11 0.06 0.06
query3 0.27 0.09 0.09
query4 1.61 0.13 0.13
query5 0.31 0.29 0.28
query6 1.19 0.70 0.68
query7 0.04 0.03 0.04
query8 0.07 0.06 0.05
query9 0.67 0.58 0.56
query10 0.62 0.62 0.63
query11 0.19 0.13 0.13
query12 0.18 0.14 0.14
query13 0.64 0.62 0.61
query14 1.07 1.03 1.04
query15 0.91 0.90 0.90
query16 0.42 0.42 0.44
query17 1.13 1.22 1.09
query18 0.24 0.22 0.22
query19 2.06 1.89 1.92
query20 0.02 0.01 0.01
query21 15.40 0.23 0.16
query22 5.04 0.08 0.06
query23 15.59 0.31 0.12
query24 3.36 0.98 1.43
query25 0.10 0.06 0.06
query26 0.17 0.18 0.17
query27 0.07 0.06 0.06
query28 5.63 1.23 0.97
query29 12.64 4.61 3.85
query30 0.31 0.15 0.13
query31 2.85 0.68 0.43
query32 3.24 0.59 0.49
query33 3.09 3.19 3.25
query34 15.76 5.26 4.66
query35 4.67 4.66 4.61
query36 0.73 0.53 0.54
query37 0.11 0.08 0.07
query38 0.08 0.05 0.04
query39 0.04 0.03 0.04
query40 0.19 0.16 0.15
query41 0.10 0.04 0.04
query42 0.05 0.04 0.03
query43 0.05 0.04 0.04
Total cold run time: 101.09 s
Total hot run time: 29.86 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](thirdparty) Refactor HDFS dependency management: move HDFS client jars to FE and slim down BE thirdparty [doris]
Posted by "CalvinKirs (via GitHub)" <gi...@apache.org>.
CalvinKirs commented on PR #57202:
URL: https://github.com/apache/doris/pull/57202#issuecomment-3465897268
run performance
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](thirdparty) Refactor HDFS dependency management: move HDFS client jars to FE and slim down BE thirdparty [doris]
Posted by "gavinchou (via GitHub)" <gi...@apache.org>.
gavinchou commented on code in PR #57202:
URL: https://github.com/apache/doris/pull/57202#discussion_r2471563205
##########
build.sh:
##########
@@ -958,8 +977,12 @@ fi
if [[ ${BUILD_CLOUD} -eq 1 ]]; then
rm -rf "${DORIS_HOME}/output/ms"
rm -rf "${DORIS_HOME}/cloud/output/lib/hadoop_hdfs"
- if [[ -d "${DORIS_THIRDPARTY}/installed/lib/hadoop_hdfs/" ]]; then
- cp -r -p "${DORIS_THIRDPARTY}/installed/lib/hadoop_hdfs/" "${DORIS_HOME}/cloud/output/lib"
+ # FIXME: If hadoop dependencies are required, building cloud module must be done after building FE first
Review Comment:
consider using new var `ENABLE_CLOUD_HDFS` for force check installing hdfs jars.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](thirdparty) Refactor HDFS dependency management: move HDFS client jars to FE and slim down BE thirdparty [doris]
Posted by "CalvinKirs (via GitHub)" <gi...@apache.org>.
CalvinKirs commented on PR #57202:
URL: https://github.com/apache/doris/pull/57202#issuecomment-3460101288
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](thirdparty) Refactor HDFS dependency management: move HDFS client jars to FE and slim down BE thirdparty [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #57202:
URL: https://github.com/apache/doris/pull/57202#issuecomment-3460642884
# BE UT Coverage Report
Increment line coverage ` ` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/f23d78d5ebbe57e753bd2bf9c0a92c86dd4a9605_f23d78d5ebbe57e753bd2bf9c0a92c86dd4a9605/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/f23d78d5ebbe57e753bd2bf9c0a92c86dd4a9605_f23d78d5ebbe57e753bd2bf9c0a92c86dd4a9605/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 52.75% (18064/34243) |
| Line Coverage | 37.98% (163797/431224) |
| Region Coverage | 32.36% (124874/385846) |
| Branch Coverage | 33.72% (54606/161927) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](thirdparty) Refactor HDFS dependency management: move HDFS client jars to FE and slim down BE thirdparty [doris]
Posted by "CalvinKirs (via GitHub)" <gi...@apache.org>.
CalvinKirs commented on PR #57202:
URL: https://github.com/apache/doris/pull/57202#issuecomment-3441317169
run compile
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](thirdparty) Refactor HDFS dependency management: move HDFS client jars to FE and slim down BE thirdparty [doris]
Posted by "CalvinKirs (via GitHub)" <gi...@apache.org>.
CalvinKirs commented on PR #57202:
URL: https://github.com/apache/doris/pull/57202#issuecomment-3442441795
run compile
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](thirdparty) Refactor HDFS dependency management: move HDFS client jars to FE and slim down BE thirdparty [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #57202:
URL: https://github.com/apache/doris/pull/57202#issuecomment-3454380320
# FE UT Coverage Report
Increment line coverage `` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/57202_841b6cf07c117dd03471f939cec8ed24265ffa38/fe_increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/57202_841b6cf07c117dd03471f939cec8ed24265ffa38/fe_report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](thirdparty) Refactor HDFS dependency management: move HDFS client jars to FE and slim down BE thirdparty [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #57202:
URL: https://github.com/apache/doris/pull/57202#issuecomment-3454653052
# BE UT Coverage Report
Increment line coverage ` ` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/841b6cf07c117dd03471f939cec8ed24265ffa38_841b6cf07c117dd03471f939cec8ed24265ffa38/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/841b6cf07c117dd03471f939cec8ed24265ffa38_841b6cf07c117dd03471f939cec8ed24265ffa38/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 52.76% (18056/34221) |
| Line Coverage | 37.99% (163707/430865) |
| Region Coverage | 32.37% (124763/385456) |
| Branch Coverage | 33.73% (54553/161713) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](thirdparty) Refactor HDFS dependency management: move HDFS client jars to FE and slim down BE thirdparty [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #57202:
URL: https://github.com/apache/doris/pull/57202#issuecomment-3455864308
# BE Regression && UT Coverage Report
Increment line coverage `100% (0/0)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/57202_1234fa993fcb42ab9149fbd30d3e5cad278b722a_merge/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/57202_1234fa993fcb42ab9149fbd30d3e5cad278b722a_merge/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 71.38% (23932/33526) |
| Line Coverage | 57.79% (248904/430690) |
| Region Coverage | 52.94% (206557/390199) |
| Branch Coverage | 54.62% (88747/162490) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](thirdparty) Refactor HDFS dependency management: move HDFS client jars to FE and slim down BE thirdparty [doris]
Posted by "CalvinKirs (via GitHub)" <gi...@apache.org>.
CalvinKirs commented on code in PR #57202:
URL: https://github.com/apache/doris/pull/57202#discussion_r2471919964
##########
build.sh:
##########
@@ -958,8 +977,12 @@ fi
if [[ ${BUILD_CLOUD} -eq 1 ]]; then
rm -rf "${DORIS_HOME}/output/ms"
rm -rf "${DORIS_HOME}/cloud/output/lib/hadoop_hdfs"
- if [[ -d "${DORIS_THIRDPARTY}/installed/lib/hadoop_hdfs/" ]]; then
- cp -r -p "${DORIS_THIRDPARTY}/installed/lib/hadoop_hdfs/" "${DORIS_HOME}/cloud/output/lib"
+ # FIXME: If hadoop dependencies are required, building cloud module must be done after building FE first
Review Comment:
We can handle this automatically — when building cloud alone, it will also trigger be-java-extensions just like BE does.
If cloud doesn’t need those dependencies, users can disable it with DISABLE_BE_JAVA_EXTENSIONS.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](thirdparty) Refactor HDFS dependency management: move HDFS client jars to FE and slim down BE thirdparty [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #57202:
URL: https://github.com/apache/doris/pull/57202#issuecomment-3455689043
# BE UT Coverage Report
Increment line coverage ` ` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/1234fa993fcb42ab9149fbd30d3e5cad278b722a_1234fa993fcb42ab9149fbd30d3e5cad278b722a/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/1234fa993fcb42ab9149fbd30d3e5cad278b722a_1234fa993fcb42ab9149fbd30d3e5cad278b722a/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 52.76% (18055/34221) |
| Line Coverage | 37.99% (163697/430889) |
| Region Coverage | 32.36% (124753/385484) |
| Branch Coverage | 33.74% (54557/161717) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](thirdparty) Refactor HDFS dependency management: move HDFS client jars to FE and slim down BE thirdparty [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #57202:
URL: https://github.com/apache/doris/pull/57202#issuecomment-3465972744
<details>
<summary>ClickBench: <b>Total hot run time: 28.32 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit f23d78d5ebbe57e753bd2bf9c0a92c86dd4a9605, data reload: false
query1 0.06 0.06 0.06
query2 0.10 0.06 0.05
query3 0.26 0.09 0.08
query4 1.60 0.12 0.12
query5 0.29 0.28 0.25
query6 1.18 0.66 0.66
query7 0.03 0.02 0.03
query8 0.05 0.05 0.05
query9 0.64 0.55 0.53
query10 0.61 0.61 0.58
query11 0.18 0.12 0.12
query12 0.16 0.12 0.12
query13 0.62 0.61 0.60
query14 1.04 1.03 1.02
query15 0.86 0.86 0.87
query16 0.42 0.39 0.41
query17 1.06 1.09 1.04
query18 0.23 0.21 0.21
query19 1.96 1.86 1.88
query20 0.02 0.01 0.02
query21 15.42 0.21 0.14
query22 4.98 0.07 0.05
query23 15.63 0.28 0.11
query24 2.71 0.52 1.01
query25 0.09 0.08 0.07
query26 0.15 0.15 0.14
query27 0.07 0.06 0.05
query28 5.69 1.23 0.94
query29 12.57 4.18 3.45
query30 0.28 0.15 0.12
query31 2.84 0.59 0.40
query32 3.25 0.56 0.49
query33 3.16 3.18 3.11
query34 15.86 5.23 4.57
query35 4.58 4.66 4.65
query36 0.68 0.51 0.50
query37 0.11 0.07 0.06
query38 0.07 0.05 0.04
query39 0.04 0.03 0.04
query40 0.18 0.14 0.14
query41 0.09 0.04 0.04
query42 0.05 0.04 0.03
query43 0.06 0.04 0.04
Total cold run time: 99.93 s
Total hot run time: 28.32 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](thirdparty) Refactor HDFS dependency management: move HDFS client jars to FE and slim down BE thirdparty [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #57202:
URL: https://github.com/apache/doris/pull/57202#issuecomment-3460949714
# BE Regression && UT Coverage Report
Increment line coverage `100% (0/0)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/57202_f23d78d5ebbe57e753bd2bf9c0a92c86dd4a9605_merge/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/57202_f23d78d5ebbe57e753bd2bf9c0a92c86dd4a9605_merge/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 71.35% (23938/33548) |
| Line Coverage | 57.79% (249080/431036) |
| Region Coverage | 52.91% (206681/390610) |
| Branch Coverage | 54.62% (88872/162712) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](thirdparty) Refactor HDFS dependency management: move HDFS client jars to FE and slim down BE thirdparty [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #57202:
URL: https://github.com/apache/doris/pull/57202#issuecomment-3455262344
<details>
<summary>ClickBench: <b>Total hot run time: 29.25 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 1234fa993fcb42ab9149fbd30d3e5cad278b722a, data reload: false
query1 0.07 0.06 0.06
query2 0.10 0.06 0.06
query3 0.26 0.09 0.09
query4 1.61 0.12 0.12
query5 0.30 0.28 0.27
query6 1.20 0.69 0.67
query7 0.04 0.03 0.03
query8 0.06 0.05 0.05
query9 0.67 0.57 0.57
query10 0.63 0.64 0.64
query11 0.20 0.13 0.13
query12 0.19 0.14 0.15
query13 0.64 0.62 0.62
query14 1.04 1.02 1.06
query15 0.91 0.89 0.88
query16 0.41 0.42 0.42
query17 1.12 1.17 1.22
query18 0.24 0.23 0.23
query19 2.05 1.89 1.86
query20 0.02 0.01 0.02
query21 15.37 0.21 0.15
query22 5.08 0.08 0.05
query23 15.62 0.31 0.12
query24 2.81 0.56 0.31
query25 0.07 0.07 0.07
query26 0.16 0.16 0.17
query27 0.10 0.07 0.06
query28 3.41 1.19 0.96
query29 12.71 4.59 3.89
query30 0.30 0.15 0.16
query31 2.82 0.66 0.41
query32 3.26 0.58 0.49
query33 3.26 3.16 3.13
query34 15.92 5.29 4.68
query35 4.68 4.64 4.64
query36 0.72 0.55 0.53
query37 0.11 0.07 0.07
query38 0.07 0.04 0.04
query39 0.05 0.04 0.03
query40 0.19 0.16 0.16
query41 0.10 0.04 0.04
query42 0.04 0.04 0.03
query43 0.06 0.04 0.05
Total cold run time: 98.67 s
Total hot run time: 29.25 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](thirdparty) Refactor HDFS dependency management: move HDFS client jars to FE and slim down BE thirdparty [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #57202:
URL: https://github.com/apache/doris/pull/57202#issuecomment-3426856832
<details>
<summary>ClickBench: <b>Total hot run time: 30.7 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit de1e6c293ece514062b07a598feb058c658e21f9, data reload: false
query1 0.07 0.06 0.06
query2 0.10 0.06 0.06
query3 0.26 0.09 0.10
query4 1.61 0.12 0.13
query5 0.29 0.28 0.29
query6 1.20 0.69 0.69
query7 0.04 0.04 0.03
query8 0.06 0.05 0.05
query9 0.67 0.57 0.56
query10 0.63 0.64 0.63
query11 0.19 0.13 0.13
query12 0.18 0.14 0.13
query13 0.66 0.63 0.62
query14 1.07 1.06 1.06
query15 0.92 0.90 0.95
query16 0.46 0.44 0.46
query17 1.20 1.28 1.15
query18 0.24 0.22 0.24
query19 2.07 1.92 1.95
query20 0.02 0.02 0.01
query21 15.36 0.20 0.15
query22 5.07 0.08 0.05
query23 15.60 0.29 0.11
query24 2.79 0.85 0.86
query25 0.09 0.07 0.06
query26 0.18 0.17 0.16
query27 0.07 0.07 0.06
query28 4.01 1.20 0.98
query29 12.62 4.73 3.88
query30 0.29 0.14 0.13
query31 2.85 0.67 0.43
query32 3.25 0.60 0.54
query33 3.23 3.20 3.25
query34 16.14 5.59 4.99
query35 5.01 5.02 5.06
query36 0.71 0.55 0.53
query37 0.12 0.08 0.08
query38 0.08 0.05 0.05
query39 0.05 0.04 0.03
query40 0.19 0.17 0.15
query41 0.10 0.04 0.03
query42 0.05 0.04 0.03
query43 0.05 0.04 0.04
Total cold run time: 99.85 s
Total hot run time: 30.7 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](thirdparty) Refactor HDFS dependency management: move HDFS client jars to FE and slim down BE thirdparty [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #57202:
URL: https://github.com/apache/doris/pull/57202#issuecomment-3454394595
<details>
<summary>TPC-DS: <b>Total hot run time: 190878 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 841b6cf07c117dd03471f939cec8ed24265ffa38, data reload: false
query1 1083 443 409 409
query2 6577 1684 1702 1684
query3 6767 222 223 222
query4 26445 23525 23072 23072
query5 4414 640 498 498
query6 347 274 253 253
query7 4651 498 307 307
query8 315 291 278 278
query9 8696 2574 2596 2574
query10 495 349 282 282
query11 15720 15686 15580 15580
query12 199 126 119 119
query13 2215 588 473 473
query14 12431 9472 9696 9472
query15 266 205 189 189
query16 8571 702 572 572
query17 2109 1043 634 634
query18 2429 446 379 379
query19 301 209 180 180
query20 149 134 145 134
query21 233 131 122 122
query22 4380 4689 4704 4689
query23 34531 33893 33403 33403
query24 8195 2431 2453 2431
query25 608 533 486 486
query26 1232 273 160 160
query27 2690 509 350 350
query28 4299 2235 2177 2177
query29 810 617 477 477
query30 300 220 196 196
query31 914 822 738 738
query32 85 74 71 71
query33 584 397 315 315
query34 792 859 506 506
query35 782 841 744 744
query36 952 975 915 915
query37 142 115 83 83
query38 3523 3584 3480 3480
query39 1483 1439 1460 1439
query40 220 126 114 114
query41 61 57 58 57
query42 125 119 111 111
query43 496 521 470 470
query44 1233 749 741 741
query45 195 186 172 172
query46 879 991 630 630
query47 1774 1793 1714 1714
query48 405 437 333 333
query49 781 508 438 438
query50 651 700 405 405
query51 3936 4108 3942 3942
query52 114 107 103 103
query53 239 257 204 204
query54 609 616 533 533
query55 90 91 83 83
query56 349 335 317 317
query57 1201 1202 1123 1123
query58 280 274 287 274
query59 2608 2587 2655 2587
query60 356 338 335 335
query61 189 165 161 161
query62 796 759 664 664
query63 230 196 198 196
query64 4493 1164 878 878
query65 4014 3969 3933 3933
query66 1085 442 333 333
query67 15280 15256 15050 15050
query68 8307 901 609 609
query69 499 332 297 297
query70 1336 1308 1271 1271
query71 515 341 316 316
query72 5946 4936 5084 4936
query73 702 603 358 358
query74 9030 9246 8969 8969
query75 4098 3346 2872 2872
query76 3728 1150 730 730
query77 820 414 325 325
query78 9593 9652 8974 8974
query79 3073 796 588 588
query80 718 565 524 524
query81 494 269 234 234
query82 479 162 137 137
query83 306 272 257 257
query84 302 113 102 102
query85 921 501 442 442
query86 343 304 286 286
query87 3747 3730 3691 3691
query88 3415 2252 2231 2231
query89 424 331 295 295
query90 2045 216 234 216
query91 187 183 136 136
query92 90 68 62 62
query93 2586 973 636 636
query94 705 444 340 340
query95 416 319 325 319
query96 488 595 280 280
query97 2937 2968 2888 2888
query98 243 222 213 213
query99 1452 1421 1350 1350
Total cold run time: 283349 ms
Total hot run time: 190878 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](thirdparty) Refactor HDFS dependency management: move HDFS client jars to FE and slim down BE thirdparty [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #57202:
URL: https://github.com/apache/doris/pull/57202#issuecomment-3450221276
# FE UT Coverage Report
Increment line coverage `` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/57202_1f63c0c51665a1c0c71a3fce78fb9fc69747b0ef/fe_increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/57202_1f63c0c51665a1c0c71a3fce78fb9fc69747b0ef/fe_report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](thirdparty) Refactor HDFS dependency management: move HDFS client jars to FE and slim down BE thirdparty [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #57202:
URL: https://github.com/apache/doris/pull/57202#issuecomment-3454303946
# Cloud UT Coverage Report
Increment line coverage ` ` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/841b6cf07c117dd03471f939cec8ed24265ffa38_841b6cf07c117dd03471f939cec8ed24265ffa38_cloud/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/841b6cf07c117dd03471f939cec8ed24265ffa38_841b6cf07c117dd03471f939cec8ed24265ffa38_cloud/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 80.64% (1649/2045) |
| Line Coverage | 67.00% (29103/43437) |
| Region Coverage | 67.36% (14428/21418) |
| Branch Coverage | 57.73% (7673/13292) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](thirdparty) Refactor HDFS dependency management: move HDFS client jars to FE and slim down BE thirdparty [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #57202:
URL: https://github.com/apache/doris/pull/57202#issuecomment-3460175784
# Cloud UT Coverage Report
Increment line coverage ` ` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/f23d78d5ebbe57e753bd2bf9c0a92c86dd4a9605_f23d78d5ebbe57e753bd2bf9c0a92c86dd4a9605_cloud/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/f23d78d5ebbe57e753bd2bf9c0a92c86dd4a9605_f23d78d5ebbe57e753bd2bf9c0a92c86dd4a9605_cloud/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 80.66% (1656/2053) |
| Line Coverage | 67.00% (29226/43619) |
| Region Coverage | 67.37% (14498/21519) |
| Branch Coverage | 57.71% (7711/13362) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](thirdparty) Refactor HDFS dependency management: move HDFS client jars to FE and slim down BE thirdparty [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #57202:
URL: https://github.com/apache/doris/pull/57202#issuecomment-3450692247
# BE Regression && UT Coverage Report
Increment line coverage `100% (0/0)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/57202_1f63c0c51665a1c0c71a3fce78fb9fc69747b0ef_merge/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/57202_1f63c0c51665a1c0c71a3fce78fb9fc69747b0ef_merge/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 71.39% (23921/33507) |
| Line Coverage | 57.77% (248611/430334) |
| Region Coverage | 52.89% (206227/389943) |
| Branch Coverage | 54.57% (88595/162339) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](thirdparty) Refactor HDFS dependency management: move HDFS client jars to FE and slim down BE thirdparty [doris]
Posted by "CalvinKirs (via GitHub)" <gi...@apache.org>.
CalvinKirs commented on PR #57202:
URL: https://github.com/apache/doris/pull/57202#issuecomment-3441834638
run compile
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](thirdparty) Refactor HDFS dependency management: move HDFS client jars to FE and slim down BE thirdparty [doris]
Posted by "CalvinKirs (via GitHub)" <gi...@apache.org>.
CalvinKirs commented on PR #57202:
URL: https://github.com/apache/doris/pull/57202#issuecomment-3449986647
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](thirdparty) Refactor HDFS dependency management: move HDFS client jars to FE and slim down BE thirdparty [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #57202:
URL: https://github.com/apache/doris/pull/57202#issuecomment-3455113437
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](thirdparty) Refactor HDFS dependency management: move HDFS client jars to FE and slim down BE thirdparty [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #57202:
URL: https://github.com/apache/doris/pull/57202#issuecomment-3455242454
# Cloud UT Coverage Report
Increment line coverage ` ` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/1234fa993fcb42ab9149fbd30d3e5cad278b722a_1234fa993fcb42ab9149fbd30d3e5cad278b722a_cloud/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/1234fa993fcb42ab9149fbd30d3e5cad278b722a_1234fa993fcb42ab9149fbd30d3e5cad278b722a_cloud/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 80.64% (1649/2045) |
| Line Coverage | 67.00% (29101/43437) |
| Region Coverage | 67.30% (14415/21418) |
| Branch Coverage | 57.67% (7666/13292) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](thirdparty) Refactor HDFS dependency management: move HDFS client jars to FE and slim down BE thirdparty [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #57202:
URL: https://github.com/apache/doris/pull/57202#issuecomment-3460201542
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](thirdparty) Refactor HDFS dependency management: move HDFS client jars to FE and slim down BE thirdparty [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #57202:
URL: https://github.com/apache/doris/pull/57202#issuecomment-3450149996
<details>
<summary>ClickBench: <b>Total hot run time: 28.25 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 1f63c0c51665a1c0c71a3fce78fb9fc69747b0ef, data reload: false
query1 0.06 0.05 0.05
query2 0.09 0.05 0.05
query3 0.26 0.08 0.08
query4 1.61 0.11 0.12
query5 0.28 0.27 0.26
query6 1.18 0.66 0.64
query7 0.03 0.03 0.04
query8 0.06 0.05 0.06
query9 0.61 0.51 0.52
query10 0.58 0.58 0.57
query11 0.17 0.12 0.11
query12 0.17 0.12 0.13
query13 0.63 0.60 0.60
query14 1.01 1.01 1.01
query15 0.86 0.85 0.85
query16 0.41 0.39 0.40
query17 1.05 1.03 1.01
query18 0.22 0.20 0.20
query19 1.92 1.84 1.85
query20 0.01 0.02 0.01
query21 15.43 0.20 0.13
query22 5.10 0.07 0.04
query23 15.63 0.27 0.10
query24 2.86 1.25 0.85
query25 0.08 0.07 0.06
query26 0.14 0.13 0.14
query27 0.07 0.06 0.05
query28 4.72 1.16 0.94
query29 12.55 4.02 3.37
query30 0.29 0.14 0.12
query31 2.82 0.60 0.39
query32 3.22 0.55 0.49
query33 3.07 3.09 3.08
query34 15.96 5.20 4.54
query35 4.59 4.62 4.59
query36 0.67 0.50 0.50
query37 0.10 0.07 0.07
query38 0.06 0.05 0.04
query39 0.04 0.03 0.03
query40 0.19 0.14 0.16
query41 0.09 0.03 0.03
query42 0.04 0.03 0.03
query43 0.04 0.04 0.04
Total cold run time: 98.97 s
Total hot run time: 28.25 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](thirdparty) Refactor HDFS dependency management: move HDFS client jars to FE and slim down BE thirdparty [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #57202:
URL: https://github.com/apache/doris/pull/57202#issuecomment-3455410154
# FE UT Coverage Report
Increment line coverage `` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/57202_1234fa993fcb42ab9149fbd30d3e5cad278b722a/fe_increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/57202_1234fa993fcb42ab9149fbd30d3e5cad278b722a/fe_report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](thirdparty) Refactor HDFS dependency management: move HDFS client jars to FE and slim down BE thirdparty [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #57202:
URL: https://github.com/apache/doris/pull/57202#issuecomment-3450339801
# BE UT Coverage Report
Increment line coverage ` ` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/1f63c0c51665a1c0c71a3fce78fb9fc69747b0ef_1f63c0c51665a1c0c71a3fce78fb9fc69747b0ef/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/1f63c0c51665a1c0c71a3fce78fb9fc69747b0ef_1f63c0c51665a1c0c71a3fce78fb9fc69747b0ef/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 52.64% (17966/34129) |
| Line Coverage | 37.91% (163024/429992) |
| Region Coverage | 32.31% (124307/384679) |
| Branch Coverage | 33.69% (54360/161346) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](thirdparty) Refactor HDFS dependency management: move HDFS client jars to FE and slim down BE thirdparty [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #57202:
URL: https://github.com/apache/doris/pull/57202#issuecomment-3454790852
# BE Regression && UT Coverage Report
Increment line coverage `100% (0/0)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/57202_841b6cf07c117dd03471f939cec8ed24265ffa38_merge/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/57202_841b6cf07c117dd03471f939cec8ed24265ffa38_merge/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 71.47% (23961/33526) |
| Line Coverage | 57.88% (249274/430666) |
| Region Coverage | 53.02% (206886/390171) |
| Branch Coverage | 54.73% (88921/162486) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](thirdparty) Refactor HDFS dependency management: move HDFS client jars to FE and slim down BE thirdparty [doris]
Posted by "CalvinKirs (via GitHub)" <gi...@apache.org>.
CalvinKirs commented on PR #57202:
URL: https://github.com/apache/doris/pull/57202#issuecomment-3454155229
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](thirdparty) Refactor HDFS dependency management: move HDFS client jars to FE and slim down BE thirdparty [doris]
Posted by "CalvinKirs (via GitHub)" <gi...@apache.org>.
CalvinKirs merged PR #57202:
URL: https://github.com/apache/doris/pull/57202
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](thirdparty) Refactor HDFS dependency management: move HDFS client jars to FE and slim down BE thirdparty [doris]
Posted by "CalvinKirs (via GitHub)" <gi...@apache.org>.
CalvinKirs commented on PR #57202:
URL: https://github.com/apache/doris/pull/57202#issuecomment-3442173861
run compile
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](thirdparty) Refactor HDFS dependency management: move HDFS client jars to FE and slim down BE thirdparty [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #57202:
URL: https://github.com/apache/doris/pull/57202#issuecomment-3426550472
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR).
Please clearly describe your PR:
1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
3. What features were added. Why was this function added?
4. Which code was refactored and why was this part of the code refactored?
5. Which functions were optimized and what is the difference before and after the optimization?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](thirdparty) Refactor HDFS dependency management: move HDFS client jars to FE and slim down BE thirdparty [doris]
Posted by "CalvinKirs (via GitHub)" <gi...@apache.org>.
CalvinKirs commented on PR #57202:
URL: https://github.com/apache/doris/pull/57202#issuecomment-3426572738
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org