You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "ByteYue (via GitHub)" <gi...@apache.org> on 2024/03/21 10:57:34 UTC
[PR] [feature](Resource) Support to specify the root path for hdfs resource [doris]
ByteYue opened a new pull request, #32632:
URL: https://github.com/apache/doris/pull/32632
## Proposed changes
Issue Number: close #xxx
<!--Describe your changes.-->
## Further comments
If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](Resource) Support to specify the root path for hdfs resource [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #32632:
URL: https://github.com/apache/doris/pull/32632#issuecomment-2011954173
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](Resource) Support to specify the root path for hdfs resource [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32632:
URL: https://github.com/apache/doris/pull/32632#issuecomment-2014853422
Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
```
Load test result on commit c00225bef8cb45d35daa16e817ae589b4d099fc6 with default session variables
Stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc: 59 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select: 19.4 seconds inserted 10000000 Rows, about 515K ops/s
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](Resource) Support to specify the root path for hdfs resource [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #32632:
URL: https://github.com/apache/doris/pull/32632#issuecomment-2014200213
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](Resource) Support to specify the root path for hdfs resource [doris]
Posted by "dataroaring (via GitHub)" <gi...@apache.org>.
dataroaring commented on PR #32632:
URL: https://github.com/apache/doris/pull/32632#issuecomment-2014746841
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](Resource) Support to specify the root path for hdfs resource [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #32632:
URL: https://github.com/apache/doris/pull/32632#issuecomment-2014198795
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](Resource) Support to specify the root path for hdfs resource [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32632:
URL: https://github.com/apache/doris/pull/32632#issuecomment-2011937471
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR)
Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website).
See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](Resource) Support to specify the root path for hdfs resource [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #32632:
URL: https://github.com/apache/doris/pull/32632#issuecomment-2018243414
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](Resource) Support to specify the root path for hdfs resource [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32632:
URL: https://github.com/apache/doris/pull/32632#issuecomment-2014848423
<details>
<summary>TPC-DS: <b>Total hot run time: 181503 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit c00225bef8cb45d35daa16e817ae589b4d099fc6, data reload: false
query1 930 364 348 348
query2 7461 2059 1949 1949
query3 6720 221 216 216
query4 31284 20768 20772 20768
query5 4419 426 406 406
query6 276 184 169 169
query7 4629 307 295 295
query8 245 180 176 176
query9 9256 2366 2402 2366
query10 571 252 260 252
query11 15497 14245 14230 14230
query12 143 88 91 88
query13 1623 416 434 416
query14 12389 10366 11180 10366
query15 357 199 197 197
query16 8145 270 265 265
query17 2093 601 552 552
query18 2096 307 281 281
query19 368 154 155 154
query20 98 83 90 83
query21 201 126 130 126
query22 4602 4423 4389 4389
query23 32032 31086 31203 31086
query24 10697 2835 2852 2835
query25 639 381 383 381
query26 1381 161 161 161
query27 2986 354 359 354
query28 7543 1947 1924 1924
query29 898 662 620 620
query30 298 152 150 150
query31 978 754 743 743
query32 94 57 54 54
query33 770 343 255 255
query34 963 478 487 478
query35 836 615 619 615
query36 993 861 904 861
query37 129 79 76 76
query38 3531 3488 3458 3458
query39 1421 1399 1359 1359
query40 208 114 113 113
query41 52 50 50 50
query42 110 93 101 93
query43 503 461 453 453
query44 1234 717 730 717
query45 283 264 264 264
query46 1099 722 695 695
query47 1639 1582 1594 1582
query48 475 359 370 359
query49 1117 357 346 346
query50 785 387 378 378
query51 6680 6668 6629 6629
query52 105 97 91 91
query53 348 275 286 275
query54 310 254 253 253
query55 84 80 79 79
query56 249 240 238 238
query57 1090 1006 1008 1006
query58 241 214 218 214
query59 2805 2644 2551 2551
query60 281 259 267 259
query61 118 116 110 110
query62 596 400 394 394
query63 301 279 280 279
query64 5761 3902 3756 3756
query65 3058 3016 3018 3016
query66 1001 341 336 336
query67 14782 14372 14334 14334
query68 9199 539 547 539
query69 676 377 382 377
query70 1433 1147 1161 1147
query71 550 290 266 266
query72 6814 2639 2521 2521
query73 1606 321 318 318
query74 6991 6574 6511 6511
query75 5227 2828 2811 2811
query76 5832 934 804 804
query77 869 276 263 263
query78 10368 9739 9636 9636
query79 10799 530 535 530
query80 1119 400 395 395
query81 522 211 213 211
query82 350 201 202 201
query83 206 148 160 148
query84 290 78 79 78
query85 1126 343 313 313
query86 369 285 263 263
query87 3781 3581 3622 3581
query88 5206 2364 2375 2364
query89 490 383 378 378
query90 2139 180 177 177
query91 172 136 141 136
query92 59 51 48 48
query93 6358 520 496 496
query94 1399 184 179 179
query95 431 341 344 341
query96 594 275 270 270
query97 3021 2899 2871 2871
query98 226 220 213 213
query99 990 745 749 745
Total cold run time: 314119 ms
Total hot run time: 181503 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](Resource) Support to specify the root path for hdfs resource [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32632:
URL: https://github.com/apache/doris/pull/32632#issuecomment-2014898715
TeamCity be ut coverage result:
Function Coverage: 35.26% (8733/24764)
Line Coverage: 27.06% (71513/264273)
Region Coverage: 26.30% (37103/141090)
Branch Coverage: 23.19% (18975/81814)
Coverage Report: http://coverage.selectdb-in.cc/coverage/c00225bef8cb45d35daa16e817ae589b4d099fc6_c00225bef8cb45d35daa16e817ae589b4d099fc6/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](Resource) Support to specify the root path for hdfs resource [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32632:
URL: https://github.com/apache/doris/pull/32632#issuecomment-2014830113
<details>
<summary>TPC-H: <b>Total hot run time: 37901 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit c00225bef8cb45d35daa16e817ae589b4d099fc6, data reload: false
------ Round 1 ----------------------------------
q1 17882 4410 4190 4190
q2 2537 162 161 161
q3 11484 1171 1226 1171
q4 10651 781 803 781
q5 8562 3032 2976 2976
q6 202 125 128 125
q7 1062 576 560 560
q8 9359 2055 2010 2010
q9 7027 6520 6514 6514
q10 8417 3406 3501 3406
q11 442 230 217 217
q12 370 212 198 198
q13 17798 2867 2884 2867
q14 231 201 213 201
q15 513 465 447 447
q16 481 373 376 373
q17 978 543 572 543
q18 7406 6503 6415 6415
q19 4835 1411 1445 1411
q20 546 261 246 246
q21 3536 2874 2809 2809
q22 351 280 296 280
Total cold run time: 114670 ms
Total hot run time: 37901 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4146 4091 4092 4091
q2 339 232 227 227
q3 2984 2886 2835 2835
q4 1881 1576 1555 1555
q5 5230 5274 5244 5244
q6 194 115 117 115
q7 2240 1873 1874 1873
q8 3174 3310 3310 3310
q9 8635 8586 8597 8586
q10 3670 3717 3700 3700
q11 543 455 438 438
q12 733 535 540 535
q13 16920 2882 2845 2845
q14 277 251 259 251
q15 492 457 473 457
q16 478 421 421 421
q17 1747 1506 1478 1478
q18 7443 7283 7070 7070
q19 1624 1526 1525 1525
q20 1940 1739 1716 1716
q21 4811 4767 4654 4654
q22 525 444 434 434
Total cold run time: 70026 ms
Total hot run time: 53360 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](Resource) Support to specify the root path for hdfs resource [doris]
Posted by "dataroaring (via GitHub)" <gi...@apache.org>.
dataroaring merged PR #32632:
URL: https://github.com/apache/doris/pull/32632
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](Resource) Support to specify the root path for hdfs resource [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #32632:
URL: https://github.com/apache/doris/pull/32632#issuecomment-2011948047
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org