You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "HappenLee (via GitHub)" <gi...@apache.org> on 2023/11/28 12:17:43 UTC
[PR] [Pipeline](load) fix may oom in pipeline load [doris]
HappenLee opened a new pull request, #27714:
URL: https://github.com/apache/doris/pull/27714
## Proposed changes
Issue Number: close #xxx
<!--Describe your changes.-->
## Further comments
If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Pipeline](load) fix may oom in pipeline load [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #27714:
URL: https://github.com/apache/doris/pull/27714#issuecomment-1830700397
<details>
<summary>TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'</summary>
```
Tpch sf100 test result on commit 11c73d5182ca35a9e179e4fc70b1bbf6e2225ac6, data reload: false
run tpch-sf100 query with default conf and session variables
q1 4909 4666 4660 4660
q2 358 166 158 158
q3 1517 1322 1274 1274
q4 1155 1001 972 972
q5 3263 3214 3230 3214
q6 258 137 133 133
q7 1016 521 546 521
q8 2238 2229 2228 2228
q9 6947 6923 6958 6923
q10 3295 3377 3346 3346
q11 344 214 209 209
q12 356 216 219 216
q13 4652 3860 3842 3842
q14 245 211 215 211
q15 592 541 524 524
q16 439 379 411 379
q17 1029 651 607 607
q18 7924 7546 7964 7546
q19 1559 1517 1530 1517
q20 594 330 319 319
q21 3403 2993 2996 2993
q22 370 295 302 295
Total cold run time: 46463 ms
Total hot run time: 42087 ms
run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1 4598 4590 4572 4572
q2 319 220 209 209
q3 3735 3740 3739 3739
q4 2537 2531 2520 2520
q5 6203 6186 6171 6171
q6 250 122 124 122
q7 2570 1980 1977 1977
q8 3758 3680 3697 3680
q9 9435 9356 9401 9356
q10 4069 4145 4147 4145
q11 643 529 503 503
q12 804 616 660 616
q13 4371 3636 3654 3636
q14 286 250 254 250
q15 581 513 532 513
q16 508 442 503 442
q17 2076 2075 2099 2075
q18 9441 8814 9018 8814
q19 1794 1768 1761 1761
q20 2327 1995 1954 1954
q21 7367 6940 7072 6940
q22 684 570 554 554
Total cold run time: 68356 ms
Total hot run time: 64549 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Pipeline](load) fix may oom in pipeline load [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #27714:
URL: https://github.com/apache/doris/pull/27714#issuecomment-1829745754
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Pipeline](load) fix may oom in pipeline load [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #27714:
URL: https://github.com/apache/doris/pull/27714#issuecomment-1829738616
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Pipeline](load) fix may oom in pipeline load [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #27714:
URL: https://github.com/apache/doris/pull/27714#issuecomment-1830486053
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Pipeline](load) fix may oom in pipeline load [doris]
Posted by "dataroaring (via GitHub)" <gi...@apache.org>.
dataroaring commented on PR #27714:
URL: https://github.com/apache/doris/pull/27714#issuecomment-1830484075
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Pipeline](load) fix may oom in pipeline load [doris]
Posted by "HappenLee (via GitHub)" <gi...@apache.org>.
HappenLee merged PR #27714:
URL: https://github.com/apache/doris/pull/27714
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Pipeline](load) fix may oom in pipeline load [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #27714:
URL: https://github.com/apache/doris/pull/27714#issuecomment-1830716534
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 43.85 seconds
stream load tsv: 579 seconds loaded 74807831229 Bytes, about 123 MB/s
stream load json: 18 seconds loaded 2358488459 Bytes, about 124 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 28.3 seconds inserted 10000000 Rows, about 353K ops/s
storage size: 17101030398 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org