You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "Jibing-Li (via GitHub)" <gi...@apache.org> on 2023/12/22 14:38:11 UTC
[PR] [improvement](statistics)Remove retry load when load stats cache fail. [doris]
Jibing-Li opened a new pull request, #28904:
URL: https://github.com/apache/doris/pull/28904
Remove retry load when load stats cache fail. This case usually happens when BE is down or BE OOM, retry doesn't work in these cases and may increase BE work load.
<!--Describe your changes.-->
## Further comments
If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](statistics)Remove retry load when load stats cache fail. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #28904:
URL: https://github.com/apache/doris/pull/28904#issuecomment-1867854003
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 45.89 seconds
stream load tsv: 571 seconds loaded 74807831229 Bytes, about 124 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 33 seconds loaded 861443392 Bytes, about 24 MB/s
insert into select: 29.0 seconds inserted 10000000 Rows, about 344K ops/s
storage size: 17184122209 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](statistics)Remove retry load when load stats cache fail. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #28904:
URL: https://github.com/apache/doris/pull/28904#issuecomment-1867846499
TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
```
Tpch sf100 test result on commit 980031cebfdbc6da1184fdad0ba917a502a912ea, data reload: false
run tpch-sf100 query with default conf and session variables
q1 4862 4651 4637 4637
q2 377 151 158 151
q3 1496 1260 1259 1259
q4 1174 938 953 938
q5 3239 3225 3239 3225
q6 268 130 128 128
q7 1018 484 544 484
q8 2234 2311 2288 2288
q9 6824 6817 6799 6799
q10 3220 3288 3327 3288
q11 323 188 180 180
q12 383 215 208 208
q13 4575 3834 3822 3822
q14 248 211 216 211
q15 579 530 524 524
q16 439 391 387 387
q17 1016 586 511 511
q18 7059 6878 6864 6864
q19 1617 1395 1506 1395
q20 544 295 301 295
q21 3150 2763 2752 2752
q22 350 286 288 286
Total cold run time: 44995 ms
Total hot run time: 40632 ms
run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1 4631 4638 4570 4570
q2 290 164 170 164
q3 3577 3589 3540 3540
q4 2467 2434 2441 2434
q5 5811 5957 5829 5829
q6 261 124 125 124
q7 2370 1892 1872 1872
q8 3699 3713 3744 3713
q9 8996 9153 9036 9036
q10 4023 4059 4031 4031
q11 495 359 373 359
q12 777 602 598 598
q13 4333 3647 3642 3642
q14 304 261 252 252
q15 570 517 521 517
q16 503 440 465 440
q17 1948 1933 1932 1932
q18 8707 8370 8279 8279
q19 1867 1902 1872 1872
q20 2319 1983 1931 1931
q21 6670 6342 6267 6267
q22 508 422 433 422
Total cold run time: 65126 ms
Total hot run time: 61824 ms
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](statistics)Remove retry load when load stats cache fail. [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #28904:
URL: https://github.com/apache/doris/pull/28904#issuecomment-1869323048
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](statistics)Remove retry load when load stats cache fail. [doris]
Posted by "Jibing-Li (via GitHub)" <gi...@apache.org>.
Jibing-Li commented on PR #28904:
URL: https://github.com/apache/doris/pull/28904#issuecomment-1867760225
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](statistics)Remove retry load when load stats cache fail. [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #28904:
URL: https://github.com/apache/doris/pull/28904#issuecomment-1868288646
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](statistics)Remove retry load when load stats cache fail. [doris]
Posted by "morningman (via GitHub)" <gi...@apache.org>.
morningman merged PR #28904:
URL: https://github.com/apache/doris/pull/28904
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org