You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "Jibing-Li (via GitHub)" <gi...@apache.org> on 2023/12/22 14:38:11 UTC

[PR] [improvement](statistics)Remove retry load when load stats cache fail. [doris]

Jibing-Li opened a new pull request, #28904:
URL: https://github.com/apache/doris/pull/28904

   Remove retry load when load stats cache fail. This case usually happens when BE is down or BE OOM, retry doesn't work in these cases and may increase BE work load.
   
   <!--Describe your changes.-->
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [improvement](statistics)Remove retry load when load stats cache fail. [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #28904:
URL: https://github.com/apache/doris/pull/28904#issuecomment-1867854003

   (From new machine)TeamCity pipeline, clickbench performance test result:
    the sum of best hot time: 45.89 seconds
    stream load tsv:          571 seconds loaded 74807831229 Bytes, about 124 MB/s
    stream load json:         19 seconds loaded 2358488459 Bytes, about 118 MB/s
    stream load orc:          66 seconds loaded 1101869774 Bytes, about 15 MB/s
    stream load parquet:          33 seconds loaded 861443392 Bytes, about 24 MB/s
    insert into select:          29.0 seconds inserted 10000000 Rows, about 344K ops/s
    storage size: 17184122209 Bytes


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [improvement](statistics)Remove retry load when load stats cache fail. [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #28904:
URL: https://github.com/apache/doris/pull/28904#issuecomment-1867846499

   
   TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   ```
   Tpch sf100 test result on commit 980031cebfdbc6da1184fdad0ba917a502a912ea, data reload: false
   
   run tpch-sf100 query with default conf and session variables
   q1	4862	4651	4637	4637
   q2	377	151	158	151
   q3	1496	1260	1259	1259
   q4	1174	938	953	938
   q5	3239	3225	3239	3225
   q6	268	130	128	128
   q7	1018	484	544	484
   q8	2234	2311	2288	2288
   q9	6824	6817	6799	6799
   q10	3220	3288	3327	3288
   q11	323	188	180	180
   q12	383	215	208	208
   q13	4575	3834	3822	3822
   q14	248	211	216	211
   q15	579	530	524	524
   q16	439	391	387	387
   q17	1016	586	511	511
   q18	7059	6878	6864	6864
   q19	1617	1395	1506	1395
   q20	544	295	301	295
   q21	3150	2763	2752	2752
   q22	350	286	288	286
   Total cold run time: 44995 ms
   Total hot run time: 40632 ms
   
   run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
   q1	4631	4638	4570	4570
   q2	290	164	170	164
   q3	3577	3589	3540	3540
   q4	2467	2434	2441	2434
   q5	5811	5957	5829	5829
   q6	261	124	125	124
   q7	2370	1892	1872	1872
   q8	3699	3713	3744	3713
   q9	8996	9153	9036	9036
   q10	4023	4059	4031	4031
   q11	495	359	373	359
   q12	777	602	598	598
   q13	4333	3647	3642	3642
   q14	304	261	252	252
   q15	570	517	521	517
   q16	503	440	465	440
   q17	1948	1933	1932	1932
   q18	8707	8370	8279	8279
   q19	1867	1902	1872	1872
   q20	2319	1983	1931	1931
   q21	6670	6342	6267	6267
   q22	508	422	433	422
   Total cold run time: 65126 ms
   Total hot run time: 61824 ms
   ```
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [improvement](statistics)Remove retry load when load stats cache fail. [doris]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #28904:
URL: https://github.com/apache/doris/pull/28904#issuecomment-1869323048

   PR approved by at least one committer and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [improvement](statistics)Remove retry load when load stats cache fail. [doris]

Posted by "Jibing-Li (via GitHub)" <gi...@apache.org>.
Jibing-Li commented on PR #28904:
URL: https://github.com/apache/doris/pull/28904#issuecomment-1867760225

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [improvement](statistics)Remove retry load when load stats cache fail. [doris]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #28904:
URL: https://github.com/apache/doris/pull/28904#issuecomment-1868288646

   PR approved by anyone and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [improvement](statistics)Remove retry load when load stats cache fail. [doris]

Posted by "morningman (via GitHub)" <gi...@apache.org>.
morningman merged PR #28904:
URL: https://github.com/apache/doris/pull/28904


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org