You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "Yukang-Lian (via GitHub)" <gi...@apache.org> on 2023/11/20 16:52:25 UTC

[PR] [Fix](core) Fix wal space back pressure core [doris]

Yukang-Lian opened a new pull request, #27311:
URL: https://github.com/apache/doris/pull/27311

   ## Proposed changes
   
   Issue Number: close #xxx
   
   <!--Describe your changes.-->
   
   Relative PR #26907 
   
   Tests will be added soon.
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [Fix](core) Fix wal space back pressure core [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #27311:
URL: https://github.com/apache/doris/pull/27311#issuecomment-1826398041

   (From new machine)TeamCity pipeline, clickbench performance test result:
    the sum of best hot time: 46.66 seconds
    stream load tsv:          566 seconds loaded 74807831229 Bytes, about 126 MB/s
    stream load json:         25 seconds loaded 2358488459 Bytes, about 89 MB/s
    stream load orc:          71 seconds loaded 1101869774 Bytes, about 14 MB/s
    stream load parquet:          32 seconds loaded 861443392 Bytes, about 25 MB/s
    insert into select:          29.1 seconds inserted 10000000 Rows, about 343K ops/s
    storage size: 17097447726 Bytes


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [Fix](core) Fix wal space back pressure core and add regression test [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #27311:
URL: https://github.com/apache/doris/pull/27311#issuecomment-1826738553

   (From new machine)TeamCity pipeline, clickbench performance test result:
    the sum of best hot time: 45.53 seconds
    stream load tsv:          566 seconds loaded 74807831229 Bytes, about 126 MB/s
    stream load json:         25 seconds loaded 2358488459 Bytes, about 89 MB/s
    stream load orc:          71 seconds loaded 1101869774 Bytes, about 14 MB/s
    stream load parquet:          32 seconds loaded 861443392 Bytes, about 25 MB/s
    insert into select:          29.0 seconds inserted 10000000 Rows, about 344K ops/s
    storage size: 17099111032 Bytes


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [Fix](core) Fix wal space back pressure core [doris]

Posted by "Yukang-Lian (via GitHub)" <gi...@apache.org>.
Yukang-Lian commented on PR #27311:
URL: https://github.com/apache/doris/pull/27311#issuecomment-1826487190

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [Fix](core) Fix wal space back pressure core [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #27311:
URL: https://github.com/apache/doris/pull/27311#issuecomment-1819631666

   (From new machine)TeamCity pipeline, clickbench performance test result:
    the sum of best hot time: 45.09 seconds
    stream load tsv:          568 seconds loaded 74807831229 Bytes, about 125 MB/s
    stream load json:         18 seconds loaded 2358488459 Bytes, about 124 MB/s
    stream load orc:          65 seconds loaded 1101869774 Bytes, about 16 MB/s
    stream load parquet:          33 seconds loaded 861443392 Bytes, about 24 MB/s
    insert into select:          28.4 seconds inserted 10000000 Rows, about 352K ops/s
    storage size: 17097957666 Bytes


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [Fix](core) Fix wal space back pressure core [doris]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #27311:
URL: https://github.com/apache/doris/pull/27311#issuecomment-1820406986

   clang-tidy review says "All clean, LGTM! :+1:"


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [Fix](core) Fix wal space back pressure core and add regression test [doris]

Posted by "dataroaring (via GitHub)" <gi...@apache.org>.
dataroaring merged PR #27311:
URL: https://github.com/apache/doris/pull/27311


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [Fix](core) Fix wal space back pressure core [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #27311:
URL: https://github.com/apache/doris/pull/27311#issuecomment-1819548580

   
   <details>
   <summary>TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'</summary>
   
   ```
   Tpch sf100 test result on commit 2caec72cccf933945f17faeea22101657406d2f6, data reload: false
   
   run tpch-sf100 query with default conf and session variables
   q1	4937	4650	4669	4650
   q2	359	158	160	158
   q3	2039	1915	1916	1915
   q4	1387	1235	1240	1235
   q5	4007	3941	3994	3941
   q6	243	129	135	129
   q7	1419	878	907	878
   q8	2728	2773	2772	2772
   q9	9738	9734	10063	9734
   q10	3494	3555	3543	3543
   q11	371	251	239	239
   q12	441	299	297	297
   q13	4610	3811	3839	3811
   q14	317	285	282	282
   q15	597	548	524	524
   q16	681	585	583	583
   q17	1134	966	929	929
   q18	7748	7383	7432	7383
   q19	1656	1707	1650	1650
   q20	582	322	275	275
   q21	4425	3946	3940	3940
   q22	473	370	371	370
   Total cold run time: 53386 ms
   Total hot run time: 49238 ms
   
   run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
   q1	4597	4568	4599	4568
   q2	338	228	246	228
   q3	4015	3985	3989	3985
   q4	2697	2700	2697	2697
   q5	9602	9597	9582	9582
   q6	244	125	125	125
   q7	3019	2475	2447	2447
   q8	4485	4496	4483	4483
   q9	13261	13138	13029	13029
   q10	4097	4184	4171	4171
   q11	771	700	677	677
   q12	980	829	833	829
   q13	4324	3636	3547	3547
   q14	394	348	342	342
   q15	581	518	525	518
   q16	753	700	701	700
   q17	3868	3868	3892	3868
   q18	9403	9083	8983	8983
   q19	1810	1786	1751	1751
   q20	2396	2059	2026	2026
   q21	8783	8370	8555	8370
   q22	850	779	858	779
   Total cold run time: 81268 ms
   Total hot run time: 77705 ms
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [Fix](core) Fix wal space back pressure core [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #27311:
URL: https://github.com/apache/doris/pull/27311#issuecomment-1820995962

   TeamCity be ut coverage result:
    Function Coverage: 36.52% (8445/23125) 
    Line Coverage: 28.84% (68651/238006)
    Region Coverage: 27.81% (35503/127677)
    Branch Coverage: 24.56% (18104/73714)
    Coverage Report: http://coverage.selectdb-in.cc/coverage/00efadcd9b6c05f4e3743d1264e8f828ae974469_00efadcd9b6c05f4e3743d1264e8f828ae974469/report/index.html


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [Fix](core) Fix wal space back pressure core [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #27311:
URL: https://github.com/apache/doris/pull/27311#issuecomment-1821066641

   
   <details>
   <summary>TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'</summary>
   
   ```
   Tpch sf100 test result on commit 00efadcd9b6c05f4e3743d1264e8f828ae974469, data reload: false
   
   run tpch-sf100 query with default conf and session variables
   q1	4906	4655	4651	4651
   q2	356	142	151	142
   q3	2035	1895	1855	1855
   q4	1383	1262	1241	1241
   q5	3952	3953	4009	3953
   q6	250	133	134	133
   q7	1434	897	892	892
   q8	2739	2786	2764	2764
   q9	9679	9866	9542	9542
   q10	3460	3570	3544	3544
   q11	380	246	247	246
   q12	439	293	293	293
   q13	4588	3819	3847	3819
   q14	316	303	283	283
   q15	590	542	525	525
   q16	666	580	574	574
   q17	1123	967	939	939
   q18	7784	7384	7292	7292
   q19	1667	1660	1675	1660
   q20	550	299	320	299
   q21	4382	3953	3945	3945
   q22	479	377	370	370
   Total cold run time: 53158 ms
   Total hot run time: 48962 ms
   
   run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
   q1	4588	4595	4592	4592
   q2	344	227	255	227
   q3	3995	3981	3979	3979
   q4	2705	2677	2690	2677
   q5	9723	9703	9693	9693
   q6	245	124	126	124
   q7	2988	2502	2468	2468
   q8	4444	4433	4426	4426
   q9	13160	13121	13088	13088
   q10	4100	4182	4183	4182
   q11	802	653	712	653
   q12	967	811	812	811
   q13	4272	3589	3560	3560
   q14	386	344	358	344
   q15	578	532	529	529
   q16	740	670	654	654
   q17	3973	3855	3915	3855
   q18	9522	8994	9087	8994
   q19	1793	1762	1803	1762
   q20	2406	2063	2063	2063
   q21	8764	8568	8724	8568
   q22	901	767	813	767
   Total cold run time: 81396 ms
   Total hot run time: 78016 ms
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [Fix](core) Fix wal space back pressure core and add regression test [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #27311:
URL: https://github.com/apache/doris/pull/27311#issuecomment-1826539780

   (From new machine)TeamCity pipeline, clickbench performance test result:
    the sum of best hot time: 46.69 seconds
    stream load tsv:          565 seconds loaded 74807831229 Bytes, about 126 MB/s
    stream load json:         27 seconds loaded 2358488459 Bytes, about 83 MB/s
    stream load orc:          71 seconds loaded 1101869774 Bytes, about 14 MB/s
    stream load parquet:          33 seconds loaded 861443392 Bytes, about 24 MB/s
    insert into select:          28.8 seconds inserted 10000000 Rows, about 347K ops/s
    storage size: 17099430555 Bytes


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [Fix](core) Fix wal space back pressure core and add regression test [doris]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #27311:
URL: https://github.com/apache/doris/pull/27311#issuecomment-1826763448

   PR approved by at least one committer and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [Fix](core) Fix wal space back pressure core [doris]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #27311:
URL: https://github.com/apache/doris/pull/27311#issuecomment-1826361094

   clang-tidy review says "All clean, LGTM! :+1:"


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [Fix](core) Fix wal space back pressure core and add regression test [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #27311:
URL: https://github.com/apache/doris/pull/27311#issuecomment-1826525301

   
   <details>
   <summary>TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'</summary>
   
   ```
   Tpch sf100 test result on commit cfc6af8c36ca7c99f046fbb047120ac593917d1f, data reload: false
   
   run tpch-sf100 query with default conf and session variables
   q1	4955	4643	4624	4624
   q2	354	146	137	137
   q3	2047	1964	1904	1904
   q4	1379	1267	1238	1238
   q5	3943	3961	4027	3961
   q6	257	132	133	132
   q7	1441	896	879	879
   q8	2775	2784	2772	2772
   q9	9683	9435	9430	9430
   q10	3446	3501	3524	3501
   q11	383	248	245	245
   q12	438	285	283	283
   q13	4559	3787	3832	3787
   q14	333	290	281	281
   q15	584	537	521	521
   q16	669	592	580	580
   q17	1141	967	926	926
   q18	7769	7341	7260	7260
   q19	1669	1674	1689	1674
   q20	534	322	299	299
   q21	4430	3978	3945	3945
   q22	480	370	376	370
   Total cold run time: 53269 ms
   Total hot run time: 48749 ms
   
   run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
   q1	4614	4640	4616	4616
   q2	340	236	261	236
   q3	3978	3969	3966	3966
   q4	2673	2672	2668	2668
   q5	9642	9510	9385	9385
   q6	242	127	123	123
   q7	3004	2480	2480	2480
   q8	4502	4445	4492	4445
   q9	12854	12850	12780	12780
   q10	4098	4149	4191	4149
   q11	792	637	659	637
   q12	982	808	805	805
   q13	4297	3554	3564	3554
   q14	384	345	356	345
   q15	588	528	530	528
   q16	728	672	671	671
   q17	3929	3888	3980	3888
   q18	9590	9076	8998	8998
   q19	1830	1789	1786	1786
   q20	2405	2086	2043	2043
   q21	8769	8556	8441	8441
   q22	877	836	783	783
   Total cold run time: 81118 ms
   Total hot run time: 77327 ms
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [Fix](core) Fix wal space back pressure core [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #27311:
URL: https://github.com/apache/doris/pull/27311#issuecomment-1826395944

   
   <details>
   <summary>TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'</summary>
   
   ```
   Tpch sf100 test result on commit 5709d33c6933d2b65a81946c26d1a360d483a9a4, data reload: false
   
   run tpch-sf100 query with default conf and session variables
   q1	4938	4637	4659	4637
   q2	376	160	161	160
   q3	2029	1933	1832	1832
   q4	1392	1227	1218	1218
   q5	3928	3944	3977	3944
   q6	242	128	131	128
   q7	1408	885	885	885
   q8	2767	2791	2769	2769
   q9	9701	9570	9478	9478
   q10	3479	3538	3521	3521
   q11	376	245	245	245
   q12	446	294	291	291
   q13	4532	3811	3790	3790
   q14	329	290	291	290
   q15	589	556	528	528
   q16	661	602	585	585
   q17	1130	989	947	947
   q18	7722	7324	7256	7256
   q19	1676	1664	1679	1664
   q20	555	297	281	281
   q21	4417	3938	3963	3938
   q22	471	369	367	367
   Total cold run time: 53164 ms
   Total hot run time: 48754 ms
   
   run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
   q1	4590	4596	4581	4581
   q2	325	233	263	233
   q3	3987	3993	3969	3969
   q4	2678	2660	2671	2660
   q5	9612	9503	9486	9486
   q6	241	123	126	123
   q7	3002	2473	2479	2473
   q8	4441	4443	4432	4432
   q9	12931	12688	12769	12688
   q10	4055	4159	4215	4159
   q11	773	647	644	644
   q12	989	806	800	800
   q13	4271	3600	3581	3581
   q14	384	359	369	359
   q15	593	524	527	524
   q16	730	659	690	659
   q17	3880	3845	3839	3839
   q18	9395	8979	8887	8887
   q19	1825	1784	1791	1784
   q20	2387	2067	2031	2031
   q21	8806	8395	8502	8395
   q22	896	777	816	777
   Total cold run time: 80791 ms
   Total hot run time: 77084 ms
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [Fix](core) Fix wal space back pressure core and add regression test [doris]

Posted by "Yukang-Lian (via GitHub)" <gi...@apache.org>.
Yukang-Lian commented on PR #27311:
URL: https://github.com/apache/doris/pull/27311#issuecomment-1826735154

   run pipelinex_p0


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [Fix](core) Fix wal space back pressure core [doris]

Posted by "Yukang-Lian (via GitHub)" <gi...@apache.org>.
Yukang-Lian commented on PR #27311:
URL: https://github.com/apache/doris/pull/27311#issuecomment-1819449859

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [Fix](core) Fix wal space back pressure core [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #27311:
URL: https://github.com/apache/doris/pull/27311#issuecomment-1819564505

   (From new machine)TeamCity pipeline, clickbench performance test result:
    the sum of best hot time: 44.6 seconds
    stream load tsv:          569 seconds loaded 74807831229 Bytes, about 125 MB/s
    stream load json:         18 seconds loaded 2358488459 Bytes, about 124 MB/s
    stream load orc:          65 seconds loaded 1101869774 Bytes, about 16 MB/s
    stream load parquet:          34 seconds loaded 861443392 Bytes, about 24 MB/s
    insert into select:          28.6 seconds inserted 10000000 Rows, about 349K ops/s
    storage size: 17098943923 Bytes


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [Fix](core) Fix wal space back pressure core and add regression test [doris]

Posted by "Yukang-Lian (via GitHub)" <gi...@apache.org>.
Yukang-Lian commented on PR #27311:
URL: https://github.com/apache/doris/pull/27311#issuecomment-1826761773

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [Fix](core) Fix wal space back pressure core and add regression test [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #27311:
URL: https://github.com/apache/doris/pull/27311#issuecomment-1826787777

   
   <details>
   <summary>TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'</summary>
   
   ```
   Tpch sf100 test result on commit fd2c967300228552d33e9ebf67da77522a61be09, data reload: false
   
   run tpch-sf100 query with default conf and session variables
   q1	4926	4685	4660	4660
   q2	363	159	156	156
   q3	2047	1877	1862	1862
   q4	1386	1262	1259	1259
   q5	3933	3939	4060	3939
   q6	252	134	136	134
   q7	1460	883	876	876
   q8	2782	2813	2796	2796
   q9	9867	9490	9465	9465
   q10	3473	3537	3523	3523
   q11	374	248	235	235
   q12	447	292	287	287
   q13	4561	3758	3767	3758
   q14	318	286	282	282
   q15	587	531	529	529
   q16	672	585	581	581
   q17	1157	1000	932	932
   q18	7855	7356	7344	7344
   q19	1680	1697	1696	1696
   q20	552	330	288	288
   q21	4346	3966	3969	3966
   q22	473	369	377	369
   Total cold run time: 53511 ms
   Total hot run time: 48937 ms
   
   run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
   q1	4614	4606	4590	4590
   q2	333	242	270	242
   q3	4003	3985	3974	3974
   q4	2770	2673	2673	2673
   q5	9667	9579	9536	9536
   q6	240	125	125	125
   q7	3003	2464	2456	2456
   q8	4474	4443	4466	4443
   q9	12988	12794	12790	12790
   q10	4073	4153	4194	4153
   q11	809	736	704	704
   q12	970	813	799	799
   q13	4285	3559	3542	3542
   q14	383	345	349	345
   q15	587	520	517	517
   q16	728	676	666	666
   q17	3984	3913	3855	3855
   q18	9655	8971	9082	8971
   q19	1842	1780	1798	1780
   q20	2409	2057	2073	2057
   q21	8742	8481	8646	8481
   q22	879	819	774	774
   Total cold run time: 81438 ms
   Total hot run time: 77473 ms
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [Fix](core) Fix wal space back pressure core [doris]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #27311:
URL: https://github.com/apache/doris/pull/27311#issuecomment-1819460459

   clang-tidy review says "All clean, LGTM! :+1:"


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [Fix](core) Fix wal space back pressure core [doris]

Posted by "Yukang-Lian (via GitHub)" <gi...@apache.org>.
Yukang-Lian commented on PR #27311:
URL: https://github.com/apache/doris/pull/27311#issuecomment-1820870278

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [Fix](core) Fix wal space back pressure core [doris]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #27311:
URL: https://github.com/apache/doris/pull/27311#issuecomment-1822029761

   PR approved by anyone and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [Fix](core) Fix wal space back pressure core [doris]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on code in PR #27311:
URL: https://github.com/apache/doris/pull/27311#discussion_r1405152032


##########
be/src/olap/wal_manager.h:
##########
@@ -17,6 +17,7 @@
 
 #include <gen_cpp/PaloInternalService_types.h>

Review Comment:
   warning: 'gen_cpp/PaloInternalService_types.h' file not found [clang-diagnostic-error]
   ```cpp
   #include <gen_cpp/PaloInternalService_types.h>
            ^
   ```
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [Fix](core) Fix wal space back pressure core and add regression test [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #27311:
URL: https://github.com/apache/doris/pull/27311#issuecomment-1826736381

   
   <details>
   <summary>TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'</summary>
   
   ```
   Tpch sf100 test result on commit c52983ff246da42608ed8dad04e09e791c1a5a91, data reload: false
   
   run tpch-sf100 query with default conf and session variables
   q1	4902	4687	4635	4635
   q2	353	151	172	151
   q3	2067	1988	1915	1915
   q4	1389	1266	1277	1266
   q5	3950	3936	4014	3936
   q6	250	135	137	135
   q7	1446	881	875	875
   q8	2761	2781	2799	2781
   q9	9627	9501	9622	9501
   q10	3459	3520	3534	3520
   q11	388	247	239	239
   q12	435	293	287	287
   q13	4552	3837	3803	3803
   q14	323	296	290	290
   q15	591	544	525	525
   q16	665	582	582	582
   q17	1156	947	929	929
   q18	7757	7258	7327	7258
   q19	1675	1698	1683	1683
   q20	555	312	294	294
   q21	4382	4008	3974	3974
   q22	477	381	364	364
   Total cold run time: 53160 ms
   Total hot run time: 48943 ms
   
   run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
   q1	4596	4602	4554	4554
   q2	340	228	265	228
   q3	4014	3995	3978	3978
   q4	2701	2662	2665	2662
   q5	9592	9611	9520	9520
   q6	242	125	124	124
   q7	3002	2447	2467	2447
   q8	4496	4484	4471	4471
   q9	12922	12884	12905	12884
   q10	4112	4169	4185	4169
   q11	760	658	659	658
   q12	974	836	801	801
   q13	4276	3634	3605	3605
   q14	383	344	352	344
   q15	580	526	523	523
   q16	727	653	658	653
   q17	3950	3950	3909	3909
   q18	9510	8823	8923	8823
   q19	1787	1804	1798	1798
   q20	2388	2071	2050	2050
   q21	8831	8514	8536	8514
   q22	885	795	806	795
   Total cold run time: 81068 ms
   Total hot run time: 77510 ms
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [Fix](core) Fix wal space back pressure core [doris]

Posted by "Yukang-Lian (via GitHub)" <gi...@apache.org>.
Yukang-Lian commented on PR #27311:
URL: https://github.com/apache/doris/pull/27311#issuecomment-1826375347

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [Fix](core) Fix wal space back pressure core and add regression test [doris]

Posted by "Yukang-Lian (via GitHub)" <gi...@apache.org>.
Yukang-Lian commented on PR #27311:
URL: https://github.com/apache/doris/pull/27311#issuecomment-1826708663

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [Fix](core) Fix wal space back pressure core and add regression test [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #27311:
URL: https://github.com/apache/doris/pull/27311#issuecomment-1826790304

   (From new machine)TeamCity pipeline, clickbench performance test result:
    the sum of best hot time: 46.18 seconds
    stream load tsv:          567 seconds loaded 74807831229 Bytes, about 125 MB/s
    stream load json:         26 seconds loaded 2358488459 Bytes, about 86 MB/s
    stream load orc:          70 seconds loaded 1101869774 Bytes, about 15 MB/s
    stream load parquet:          32 seconds loaded 861443392 Bytes, about 25 MB/s
    insert into select:          28.5 seconds inserted 10000000 Rows, about 350K ops/s
    storage size: 17098833145 Bytes


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [Fix](core) Fix wal space back pressure core [doris]

Posted by "Yukang-Lian (via GitHub)" <gi...@apache.org>.
Yukang-Lian commented on PR #27311:
URL: https://github.com/apache/doris/pull/27311#issuecomment-1819465865

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [Fix](core) Fix wal space back pressure core [doris]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #27311:
URL: https://github.com/apache/doris/pull/27311#issuecomment-1819476592

   clang-tidy review says "All clean, LGTM! :+1:"


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [Fix](core) Fix wal space back pressure core [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #27311:
URL: https://github.com/apache/doris/pull/27311#issuecomment-1821067121

   (From new machine)TeamCity pipeline, clickbench performance test result:
    the sum of best hot time: 45.63 seconds
    stream load tsv:          565 seconds loaded 74807831229 Bytes, about 126 MB/s
    stream load json:         18 seconds loaded 2358488459 Bytes, about 124 MB/s
    stream load orc:          65 seconds loaded 1101869774 Bytes, about 16 MB/s
    stream load parquet:          32 seconds loaded 861443392 Bytes, about 25 MB/s
    insert into select:          28.9 seconds inserted 10000000 Rows, about 346K ops/s
    storage size: 17098115979 Bytes


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org