You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "morningman (via GitHub)" <gi...@apache.org> on 2024/02/19 16:12:50 UTC
[PR] [opt](deps) reduce the size of FE dependencies [doris]
morningman opened a new pull request, #31129:
URL: https://github.com/apache/doris/pull/31129
## Proposed changes
the `ranger-plugins-common` depends `aws-java-sdk-bundle`, which is too large and unnecessary.
exclude it and add aws-java-sdk-logs alone.
## Further comments
If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](deps) reduce the size of FE dependencies [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31129:
URL: https://github.com/apache/doris/pull/31129#issuecomment-1952790663
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](deps) reduce the size of FE dependencies [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31129:
URL: https://github.com/apache/doris/pull/31129#issuecomment-1952911246
Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
```
Load test result on commit fa88d09a26040c2a42ea3bba89e9f2c38db65d9b with default session variables
Stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc: 59 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select: 13.5 seconds inserted 10000000 Rows, about 740K ops/s
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](deps) reduce the size of FE dependencies [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31129:
URL: https://github.com/apache/doris/pull/31129#issuecomment-1952899761
<details>
<summary>TPC-DS: <b>Total hot run time: 178981 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit fa88d09a26040c2a42ea3bba89e9f2c38db65d9b, data reload: false
query1 915 355 349 349
query2 6536 1761 1658 1658
query3 6691 206 201 201
query4 23116 21177 21325 21177
query5 4231 391 387 387
query6 254 171 169 169
query7 4607 294 292 292
query8 255 213 202 202
query9 8821 2880 2863 2863
query10 416 224 225 224
query11 15153 14779 14647 14647
query12 162 84 84 84
query13 1703 435 426 426
query14 10167 8462 8817 8462
query15 790 212 217 212
query16 9787 314 289 289
query17 5266 1379 557 557
query18 3645 287 291 287
query19 981 154 168 154
query20 359 85 89 85
query21 908 129 140 129
query22 4916 4705 4698 4698
query23 32452 31607 31501 31501
query24 11535 3405 3508 3405
query25 627 372 382 372
query26 1570 156 161 156
query27 3036 311 317 311
query28 6495 1867 1853 1853
query29 1096 623 621 621
query30 275 138 145 138
query31 921 757 760 757
query32 95 60 56 56
query33 749 240 250 240
query34 1045 491 502 491
query35 948 855 849 849
query36 979 881 876 876
query37 129 60 65 60
query38 3252 3208 3143 3143
query39 1372 1342 1322 1322
query40 278 105 106 105
query41 37 35 35 35
query42 105 100 100 100
query43 468 441 446 441
query44 1059 698 717 698
query45 201 189 175 175
query46 1057 773 751 751
query47 1599 1551 1573 1551
query48 416 338 344 338
query49 1095 308 305 305
query50 783 378 372 372
query51 5295 5149 5219 5149
query52 107 99 92 92
query53 393 305 293 293
query54 308 225 234 225
query55 84 86 77 77
query56 227 204 198 198
query57 1075 935 954 935
query58 220 206 202 202
query59 2279 2223 2198 2198
query60 252 225 230 225
query61 92 83 82 82
query62 582 365 370 365
query63 337 298 287 287
query64 6306 3067 3149 3067
query65 3303 3245 3258 3245
query66 1351 330 321 321
query67 14680 14346 14240 14240
query68 5136 561 567 561
query69 529 370 360 360
query70 1226 1290 1255 1255
query71 443 285 257 257
query72 6343 2804 2640 2640
query73 707 313 310 310
query74 6909 6429 6358 6358
query75 3227 2580 2567 2567
query76 3290 1102 1215 1102
query77 370 239 249 239
query78 9452 8867 8740 8740
query79 966 515 523 515
query80 507 353 347 347
query81 439 217 209 209
query82 165 92 90 90
query83 143 128 130 128
query84 231 83 79 79
query85 1033 354 343 343
query86 304 308 312 308
query87 3439 3322 3325 3322
query88 2733 2274 2268 2268
query89 452 369 368 368
query90 2000 171 169 169
query91 159 130 132 130
query92 54 53 48 48
query93 1021 500 513 500
query94 1307 186 178 178
query95 8639 375 8486 375
query96 588 269 264 264
query97 4433 4259 4268 4259
query98 221 211 200 200
query99 1068 743 717 717
Total cold run time: 287778 ms
Total hot run time: 178981 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](deps) reduce the size of FE dependencies [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #31129:
URL: https://github.com/apache/doris/pull/31129#issuecomment-1953334290
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](deps) reduce the size of FE dependencies [doris]
Posted by "morningman (via GitHub)" <gi...@apache.org>.
morningman commented on PR #31129:
URL: https://github.com/apache/doris/pull/31129#issuecomment-1952820679
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](deps) reduce the size of FE dependencies [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31129:
URL: https://github.com/apache/doris/pull/31129#issuecomment-1952907191
<details>
<summary>ClickBench: <b>Total hot run time: 30.94 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit fa88d09a26040c2a42ea3bba89e9f2c38db65d9b, data reload: false
query1 0.04 0.03 0.02
query2 0.06 0.02 0.02
query3 0.23 0.07 0.08
query4 1.64 0.10 0.08
query5 0.49 0.48 0.48
query6 1.36 0.60 0.62
query7 0.02 0.02 0.01
query8 0.04 0.02 0.02
query9 0.54 0.46 0.46
query10 0.49 0.50 0.51
query11 0.12 0.10 0.10
query12 0.12 0.10 0.11
query13 0.60 0.58 0.59
query14 0.76 0.80 0.78
query15 0.82 0.81 0.80
query16 0.33 0.36 0.35
query17 0.91 0.87 0.93
query18 0.20 0.18 0.17
query19 1.77 1.63 1.67
query20 0.02 0.01 0.02
query21 15.41 0.66 0.58
query22 3.67 3.97 2.41
query23 17.37 1.07 1.01
query24 2.05 0.71 0.20
query25 0.64 0.05 0.07
query26 0.16 0.14 0.16
query27 0.06 0.06 0.05
query28 11.95 0.85 0.84
query29 12.59 3.35 3.32
query30 0.51 0.50 0.44
query31 2.79 0.37 0.38
query32 3.33 0.48 0.48
query33 3.14 3.14 3.17
query34 15.36 4.54 4.51
query35 4.50 4.53 4.52
query36 1.09 0.96 0.96
query37 0.07 0.05 0.06
query38 0.04 0.03 0.04
query39 0.02 0.02 0.01
query40 0.16 0.14 0.14
query41 0.07 0.02 0.01
query42 0.03 0.01 0.01
query43 0.03 0.02 0.02
Total cold run time: 105.6 s
Total hot run time: 30.94 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](deps) reduce the size of FE dependencies [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31129:
URL: https://github.com/apache/doris/pull/31129#issuecomment-1952881422
<details>
<summary>TPC-H: <b>Total hot run time: 41299 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit fa88d09a26040c2a42ea3bba89e9f2c38db65d9b, data reload: false
------ Round 1 ----------------------------------
q1 17701 4927 4861 4861
q2 2041 143 131 131
q3 10580 1034 994 994
q4 4649 964 971 964
q5 7693 3209 3260 3209
q6 194 134 135 134
q7 1257 788 761 761
q8 9228 2063 2069 2063
q9 7618 6686 6689 6686
q10 8310 2652 2644 2644
q11 405 220 224 220
q12 729 336 338 336
q13 18001 3680 3670 3670
q14 288 259 255 255
q15 590 518 509 509
q16 491 408 408 408
q17 931 876 850 850
q18 7387 6756 6521 6521
q19 1539 1486 1485 1485
q20 659 324 344 324
q21 6481 3931 3946 3931
q22 888 358 343 343
Total cold run time: 107660 ms
Total hot run time: 41299 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4895 4921 4947 4921
q2 314 198 197 197
q3 3610 3614 3627 3614
q4 2547 2565 2554 2554
q5 5748 5745 5746 5745
q6 210 128 130 128
q7 2282 1664 1697 1664
q8 3076 3136 3200 3136
q9 8705 8699 8681 8681
q10 6793 4253 4226 4226
q11 573 424 432 424
q12 798 559 559 559
q13 4342 3408 3423 3408
q14 264 248 254 248
q15 607 510 529 510
q16 509 451 442 442
q17 1825 1767 1787 1767
q18 8454 7908 7819 7819
q19 1626 1625 1628 1625
q20 2127 1843 1853 1843
q21 6735 6405 6317 6317
q22 609 530 537 530
Total cold run time: 66649 ms
Total hot run time: 60358 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](deps) reduce the size of FE dependencies [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #31129:
URL: https://github.com/apache/doris/pull/31129#issuecomment-1953334306
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](deps) reduce the size of FE dependencies [doris]
Posted by "yiguolei (via GitHub)" <gi...@apache.org>.
yiguolei merged PR #31129:
URL: https://github.com/apache/doris/pull/31129
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org