You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "cambyzju (via GitHub)" <gi...@apache.org> on 2023/11/16 12:23:32 UTC
[PR] [improve](streamload) Explicitly judge the return value of close [doris]
cambyzju opened a new pull request, #27134:
URL: https://github.com/apache/doris/pull/27134
## Proposed changes
Nowadays, we use network disks on the cloud, but sometimes closing them may fail.
Here `ctx->body_sink.reset()` will close the file, but it will not check the return value of `::close`.
We should better call `finish` to close the file, and check the return value of `::close`.
## Further comments
If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improve](streamload) Explicitly judge the return value of close [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #27134:
URL: https://github.com/apache/doris/pull/27134#issuecomment-1815689602
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 47.77 seconds
stream load tsv: 583 seconds loaded 74807831229 Bytes, about 122 MB/s
stream load json: 18 seconds loaded 2358488459 Bytes, about 124 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 29.1 seconds inserted 10000000 Rows, about 343K ops/s
storage size: 17099511457 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improve](streamload) Explicitly judge the return value of close [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #27134:
URL: https://github.com/apache/doris/pull/27134#issuecomment-1815663372
TeamCity be ut coverage result:
Function Coverage: 36.76% (8411/22883)
Line Coverage: 29.28% (68417/233643)
Region Coverage: 27.88% (35377/126871)
Branch Coverage: 24.64% (18073/73358)
Coverage Report: http://coverage.selectdb-in.cc/coverage/4c75d5d052d7f7751c2265a988107e995eca29f9_4c75d5d052d7f7751c2265a988107e995eca29f9/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improve](streamload) Explicitly judge the return value of close [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #27134:
URL: https://github.com/apache/doris/pull/27134#issuecomment-1814350232
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improve](streamload) Explicitly judge the return value of close [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #27134:
URL: https://github.com/apache/doris/pull/27134#issuecomment-1815583753
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improve](streamload) Explicitly judge the return value of close [doris]
Posted by "yiguolei (via GitHub)" <gi...@apache.org>.
yiguolei commented on PR #27134:
URL: https://github.com/apache/doris/pull/27134#issuecomment-1815583295
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improve](streamload) Explicitly judge the return value of close [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #27134:
URL: https://github.com/apache/doris/pull/27134#issuecomment-1815681373
<details>
<summary>TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'</summary>
```
Tpch sf100 test result on commit 4c75d5d052d7f7751c2265a988107e995eca29f9, data reload: false
run tpch-sf100 query with default conf and session variables
q1 4922 4652 4663 4652
q2 359 147 171 147
q3 2016 1891 1884 1884
q4 1388 1229 1213 1213
q5 3980 3966 4069 3966
q6 249 131 133 131
q7 1409 879 889 879
q8 2792 2817 2779 2779
q9 9688 9660 9578 9578
q10 3473 3504 3560 3504
q11 371 254 245 245
q12 441 297 292 292
q13 4601 3822 3782 3782
q14 317 278 285 278
q15 594 530 530 530
q16 675 588 579 579
q17 1141 994 918 918
q18 7846 7374 7571 7374
q19 1703 1700 1649 1649
q20 569 308 310 308
q21 4386 3953 3965 3953
q22 478 374 375 374
Total cold run time: 53398 ms
Total hot run time: 49015 ms
run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1 4597 4566 4590 4566
q2 333 219 260 219
q3 4007 3999 3992 3992
q4 2720 2735 2711 2711
q5 9585 9609 9573 9573
q6 245 123 127 123
q7 2620 2261 2358 2261
q8 4406 4413 4424 4413
q9 13192 13235 13124 13124
q10 4082 4182 4178 4178
q11 764 679 638 638
q12 973 811 806 806
q13 4286 3572 3593 3572
q14 380 340 340 340
q15 563 523 521 521
q16 733 682 664 664
q17 3847 3984 3927 3927
q18 9663 9021 9231 9021
q19 1766 1784 1768 1768
q20 2386 2067 2046 2046
q21 8810 8480 8432 8432
q22 898 797 799 797
Total cold run time: 80856 ms
Total hot run time: 77692 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improve](streamload) Explicitly judge the return value of close [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #27134:
URL: https://github.com/apache/doris/pull/27134#issuecomment-1815583786
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improve](streamload) Explicitly judge the return value of close [doris]
Posted by "yiguolei (via GitHub)" <gi...@apache.org>.
yiguolei merged PR #27134:
URL: https://github.com/apache/doris/pull/27134
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org