You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "Yukang-Lian (via GitHub)" <gi...@apache.org> on 2023/11/20 16:52:25 UTC
[PR] [Fix](core) Fix wal space back pressure core [doris]
Yukang-Lian opened a new pull request, #27311:
URL: https://github.com/apache/doris/pull/27311
## Proposed changes
Issue Number: close #xxx
<!--Describe your changes.-->
Relative PR #26907
Tests will be added soon.
## Further comments
If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](core) Fix wal space back pressure core [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #27311:
URL: https://github.com/apache/doris/pull/27311#issuecomment-1826398041
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 46.66 seconds
stream load tsv: 566 seconds loaded 74807831229 Bytes, about 126 MB/s
stream load json: 25 seconds loaded 2358488459 Bytes, about 89 MB/s
stream load orc: 71 seconds loaded 1101869774 Bytes, about 14 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 29.1 seconds inserted 10000000 Rows, about 343K ops/s
storage size: 17097447726 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](core) Fix wal space back pressure core and add regression test [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #27311:
URL: https://github.com/apache/doris/pull/27311#issuecomment-1826738553
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 45.53 seconds
stream load tsv: 566 seconds loaded 74807831229 Bytes, about 126 MB/s
stream load json: 25 seconds loaded 2358488459 Bytes, about 89 MB/s
stream load orc: 71 seconds loaded 1101869774 Bytes, about 14 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 29.0 seconds inserted 10000000 Rows, about 344K ops/s
storage size: 17099111032 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](core) Fix wal space back pressure core [doris]
Posted by "Yukang-Lian (via GitHub)" <gi...@apache.org>.
Yukang-Lian commented on PR #27311:
URL: https://github.com/apache/doris/pull/27311#issuecomment-1826487190
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](core) Fix wal space back pressure core [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #27311:
URL: https://github.com/apache/doris/pull/27311#issuecomment-1819631666
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 45.09 seconds
stream load tsv: 568 seconds loaded 74807831229 Bytes, about 125 MB/s
stream load json: 18 seconds loaded 2358488459 Bytes, about 124 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 33 seconds loaded 861443392 Bytes, about 24 MB/s
insert into select: 28.4 seconds inserted 10000000 Rows, about 352K ops/s
storage size: 17097957666 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](core) Fix wal space back pressure core [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #27311:
URL: https://github.com/apache/doris/pull/27311#issuecomment-1820406986
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](core) Fix wal space back pressure core and add regression test [doris]
Posted by "dataroaring (via GitHub)" <gi...@apache.org>.
dataroaring merged PR #27311:
URL: https://github.com/apache/doris/pull/27311
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](core) Fix wal space back pressure core [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #27311:
URL: https://github.com/apache/doris/pull/27311#issuecomment-1819548580
<details>
<summary>TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'</summary>
```
Tpch sf100 test result on commit 2caec72cccf933945f17faeea22101657406d2f6, data reload: false
run tpch-sf100 query with default conf and session variables
q1 4937 4650 4669 4650
q2 359 158 160 158
q3 2039 1915 1916 1915
q4 1387 1235 1240 1235
q5 4007 3941 3994 3941
q6 243 129 135 129
q7 1419 878 907 878
q8 2728 2773 2772 2772
q9 9738 9734 10063 9734
q10 3494 3555 3543 3543
q11 371 251 239 239
q12 441 299 297 297
q13 4610 3811 3839 3811
q14 317 285 282 282
q15 597 548 524 524
q16 681 585 583 583
q17 1134 966 929 929
q18 7748 7383 7432 7383
q19 1656 1707 1650 1650
q20 582 322 275 275
q21 4425 3946 3940 3940
q22 473 370 371 370
Total cold run time: 53386 ms
Total hot run time: 49238 ms
run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1 4597 4568 4599 4568
q2 338 228 246 228
q3 4015 3985 3989 3985
q4 2697 2700 2697 2697
q5 9602 9597 9582 9582
q6 244 125 125 125
q7 3019 2475 2447 2447
q8 4485 4496 4483 4483
q9 13261 13138 13029 13029
q10 4097 4184 4171 4171
q11 771 700 677 677
q12 980 829 833 829
q13 4324 3636 3547 3547
q14 394 348 342 342
q15 581 518 525 518
q16 753 700 701 700
q17 3868 3868 3892 3868
q18 9403 9083 8983 8983
q19 1810 1786 1751 1751
q20 2396 2059 2026 2026
q21 8783 8370 8555 8370
q22 850 779 858 779
Total cold run time: 81268 ms
Total hot run time: 77705 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](core) Fix wal space back pressure core [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #27311:
URL: https://github.com/apache/doris/pull/27311#issuecomment-1820995962
TeamCity be ut coverage result:
Function Coverage: 36.52% (8445/23125)
Line Coverage: 28.84% (68651/238006)
Region Coverage: 27.81% (35503/127677)
Branch Coverage: 24.56% (18104/73714)
Coverage Report: http://coverage.selectdb-in.cc/coverage/00efadcd9b6c05f4e3743d1264e8f828ae974469_00efadcd9b6c05f4e3743d1264e8f828ae974469/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](core) Fix wal space back pressure core [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #27311:
URL: https://github.com/apache/doris/pull/27311#issuecomment-1821066641
<details>
<summary>TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'</summary>
```
Tpch sf100 test result on commit 00efadcd9b6c05f4e3743d1264e8f828ae974469, data reload: false
run tpch-sf100 query with default conf and session variables
q1 4906 4655 4651 4651
q2 356 142 151 142
q3 2035 1895 1855 1855
q4 1383 1262 1241 1241
q5 3952 3953 4009 3953
q6 250 133 134 133
q7 1434 897 892 892
q8 2739 2786 2764 2764
q9 9679 9866 9542 9542
q10 3460 3570 3544 3544
q11 380 246 247 246
q12 439 293 293 293
q13 4588 3819 3847 3819
q14 316 303 283 283
q15 590 542 525 525
q16 666 580 574 574
q17 1123 967 939 939
q18 7784 7384 7292 7292
q19 1667 1660 1675 1660
q20 550 299 320 299
q21 4382 3953 3945 3945
q22 479 377 370 370
Total cold run time: 53158 ms
Total hot run time: 48962 ms
run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1 4588 4595 4592 4592
q2 344 227 255 227
q3 3995 3981 3979 3979
q4 2705 2677 2690 2677
q5 9723 9703 9693 9693
q6 245 124 126 124
q7 2988 2502 2468 2468
q8 4444 4433 4426 4426
q9 13160 13121 13088 13088
q10 4100 4182 4183 4182
q11 802 653 712 653
q12 967 811 812 811
q13 4272 3589 3560 3560
q14 386 344 358 344
q15 578 532 529 529
q16 740 670 654 654
q17 3973 3855 3915 3855
q18 9522 8994 9087 8994
q19 1793 1762 1803 1762
q20 2406 2063 2063 2063
q21 8764 8568 8724 8568
q22 901 767 813 767
Total cold run time: 81396 ms
Total hot run time: 78016 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](core) Fix wal space back pressure core and add regression test [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #27311:
URL: https://github.com/apache/doris/pull/27311#issuecomment-1826539780
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 46.69 seconds
stream load tsv: 565 seconds loaded 74807831229 Bytes, about 126 MB/s
stream load json: 27 seconds loaded 2358488459 Bytes, about 83 MB/s
stream load orc: 71 seconds loaded 1101869774 Bytes, about 14 MB/s
stream load parquet: 33 seconds loaded 861443392 Bytes, about 24 MB/s
insert into select: 28.8 seconds inserted 10000000 Rows, about 347K ops/s
storage size: 17099430555 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](core) Fix wal space back pressure core and add regression test [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #27311:
URL: https://github.com/apache/doris/pull/27311#issuecomment-1826763448
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](core) Fix wal space back pressure core [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #27311:
URL: https://github.com/apache/doris/pull/27311#issuecomment-1826361094
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](core) Fix wal space back pressure core and add regression test [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #27311:
URL: https://github.com/apache/doris/pull/27311#issuecomment-1826525301
<details>
<summary>TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'</summary>
```
Tpch sf100 test result on commit cfc6af8c36ca7c99f046fbb047120ac593917d1f, data reload: false
run tpch-sf100 query with default conf and session variables
q1 4955 4643 4624 4624
q2 354 146 137 137
q3 2047 1964 1904 1904
q4 1379 1267 1238 1238
q5 3943 3961 4027 3961
q6 257 132 133 132
q7 1441 896 879 879
q8 2775 2784 2772 2772
q9 9683 9435 9430 9430
q10 3446 3501 3524 3501
q11 383 248 245 245
q12 438 285 283 283
q13 4559 3787 3832 3787
q14 333 290 281 281
q15 584 537 521 521
q16 669 592 580 580
q17 1141 967 926 926
q18 7769 7341 7260 7260
q19 1669 1674 1689 1674
q20 534 322 299 299
q21 4430 3978 3945 3945
q22 480 370 376 370
Total cold run time: 53269 ms
Total hot run time: 48749 ms
run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1 4614 4640 4616 4616
q2 340 236 261 236
q3 3978 3969 3966 3966
q4 2673 2672 2668 2668
q5 9642 9510 9385 9385
q6 242 127 123 123
q7 3004 2480 2480 2480
q8 4502 4445 4492 4445
q9 12854 12850 12780 12780
q10 4098 4149 4191 4149
q11 792 637 659 637
q12 982 808 805 805
q13 4297 3554 3564 3554
q14 384 345 356 345
q15 588 528 530 528
q16 728 672 671 671
q17 3929 3888 3980 3888
q18 9590 9076 8998 8998
q19 1830 1789 1786 1786
q20 2405 2086 2043 2043
q21 8769 8556 8441 8441
q22 877 836 783 783
Total cold run time: 81118 ms
Total hot run time: 77327 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](core) Fix wal space back pressure core [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #27311:
URL: https://github.com/apache/doris/pull/27311#issuecomment-1826395944
<details>
<summary>TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'</summary>
```
Tpch sf100 test result on commit 5709d33c6933d2b65a81946c26d1a360d483a9a4, data reload: false
run tpch-sf100 query with default conf and session variables
q1 4938 4637 4659 4637
q2 376 160 161 160
q3 2029 1933 1832 1832
q4 1392 1227 1218 1218
q5 3928 3944 3977 3944
q6 242 128 131 128
q7 1408 885 885 885
q8 2767 2791 2769 2769
q9 9701 9570 9478 9478
q10 3479 3538 3521 3521
q11 376 245 245 245
q12 446 294 291 291
q13 4532 3811 3790 3790
q14 329 290 291 290
q15 589 556 528 528
q16 661 602 585 585
q17 1130 989 947 947
q18 7722 7324 7256 7256
q19 1676 1664 1679 1664
q20 555 297 281 281
q21 4417 3938 3963 3938
q22 471 369 367 367
Total cold run time: 53164 ms
Total hot run time: 48754 ms
run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1 4590 4596 4581 4581
q2 325 233 263 233
q3 3987 3993 3969 3969
q4 2678 2660 2671 2660
q5 9612 9503 9486 9486
q6 241 123 126 123
q7 3002 2473 2479 2473
q8 4441 4443 4432 4432
q9 12931 12688 12769 12688
q10 4055 4159 4215 4159
q11 773 647 644 644
q12 989 806 800 800
q13 4271 3600 3581 3581
q14 384 359 369 359
q15 593 524 527 524
q16 730 659 690 659
q17 3880 3845 3839 3839
q18 9395 8979 8887 8887
q19 1825 1784 1791 1784
q20 2387 2067 2031 2031
q21 8806 8395 8502 8395
q22 896 777 816 777
Total cold run time: 80791 ms
Total hot run time: 77084 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](core) Fix wal space back pressure core and add regression test [doris]
Posted by "Yukang-Lian (via GitHub)" <gi...@apache.org>.
Yukang-Lian commented on PR #27311:
URL: https://github.com/apache/doris/pull/27311#issuecomment-1826735154
run pipelinex_p0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](core) Fix wal space back pressure core [doris]
Posted by "Yukang-Lian (via GitHub)" <gi...@apache.org>.
Yukang-Lian commented on PR #27311:
URL: https://github.com/apache/doris/pull/27311#issuecomment-1819449859
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](core) Fix wal space back pressure core [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #27311:
URL: https://github.com/apache/doris/pull/27311#issuecomment-1819564505
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 44.6 seconds
stream load tsv: 569 seconds loaded 74807831229 Bytes, about 125 MB/s
stream load json: 18 seconds loaded 2358488459 Bytes, about 124 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 34 seconds loaded 861443392 Bytes, about 24 MB/s
insert into select: 28.6 seconds inserted 10000000 Rows, about 349K ops/s
storage size: 17098943923 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](core) Fix wal space back pressure core and add regression test [doris]
Posted by "Yukang-Lian (via GitHub)" <gi...@apache.org>.
Yukang-Lian commented on PR #27311:
URL: https://github.com/apache/doris/pull/27311#issuecomment-1826761773
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](core) Fix wal space back pressure core and add regression test [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #27311:
URL: https://github.com/apache/doris/pull/27311#issuecomment-1826787777
<details>
<summary>TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'</summary>
```
Tpch sf100 test result on commit fd2c967300228552d33e9ebf67da77522a61be09, data reload: false
run tpch-sf100 query with default conf and session variables
q1 4926 4685 4660 4660
q2 363 159 156 156
q3 2047 1877 1862 1862
q4 1386 1262 1259 1259
q5 3933 3939 4060 3939
q6 252 134 136 134
q7 1460 883 876 876
q8 2782 2813 2796 2796
q9 9867 9490 9465 9465
q10 3473 3537 3523 3523
q11 374 248 235 235
q12 447 292 287 287
q13 4561 3758 3767 3758
q14 318 286 282 282
q15 587 531 529 529
q16 672 585 581 581
q17 1157 1000 932 932
q18 7855 7356 7344 7344
q19 1680 1697 1696 1696
q20 552 330 288 288
q21 4346 3966 3969 3966
q22 473 369 377 369
Total cold run time: 53511 ms
Total hot run time: 48937 ms
run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1 4614 4606 4590 4590
q2 333 242 270 242
q3 4003 3985 3974 3974
q4 2770 2673 2673 2673
q5 9667 9579 9536 9536
q6 240 125 125 125
q7 3003 2464 2456 2456
q8 4474 4443 4466 4443
q9 12988 12794 12790 12790
q10 4073 4153 4194 4153
q11 809 736 704 704
q12 970 813 799 799
q13 4285 3559 3542 3542
q14 383 345 349 345
q15 587 520 517 517
q16 728 676 666 666
q17 3984 3913 3855 3855
q18 9655 8971 9082 8971
q19 1842 1780 1798 1780
q20 2409 2057 2073 2057
q21 8742 8481 8646 8481
q22 879 819 774 774
Total cold run time: 81438 ms
Total hot run time: 77473 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](core) Fix wal space back pressure core [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #27311:
URL: https://github.com/apache/doris/pull/27311#issuecomment-1819460459
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](core) Fix wal space back pressure core [doris]
Posted by "Yukang-Lian (via GitHub)" <gi...@apache.org>.
Yukang-Lian commented on PR #27311:
URL: https://github.com/apache/doris/pull/27311#issuecomment-1820870278
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](core) Fix wal space back pressure core [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #27311:
URL: https://github.com/apache/doris/pull/27311#issuecomment-1822029761
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](core) Fix wal space back pressure core [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on code in PR #27311:
URL: https://github.com/apache/doris/pull/27311#discussion_r1405152032
##########
be/src/olap/wal_manager.h:
##########
@@ -17,6 +17,7 @@
#include <gen_cpp/PaloInternalService_types.h>
Review Comment:
warning: 'gen_cpp/PaloInternalService_types.h' file not found [clang-diagnostic-error]
```cpp
#include <gen_cpp/PaloInternalService_types.h>
^
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](core) Fix wal space back pressure core and add regression test [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #27311:
URL: https://github.com/apache/doris/pull/27311#issuecomment-1826736381
<details>
<summary>TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'</summary>
```
Tpch sf100 test result on commit c52983ff246da42608ed8dad04e09e791c1a5a91, data reload: false
run tpch-sf100 query with default conf and session variables
q1 4902 4687 4635 4635
q2 353 151 172 151
q3 2067 1988 1915 1915
q4 1389 1266 1277 1266
q5 3950 3936 4014 3936
q6 250 135 137 135
q7 1446 881 875 875
q8 2761 2781 2799 2781
q9 9627 9501 9622 9501
q10 3459 3520 3534 3520
q11 388 247 239 239
q12 435 293 287 287
q13 4552 3837 3803 3803
q14 323 296 290 290
q15 591 544 525 525
q16 665 582 582 582
q17 1156 947 929 929
q18 7757 7258 7327 7258
q19 1675 1698 1683 1683
q20 555 312 294 294
q21 4382 4008 3974 3974
q22 477 381 364 364
Total cold run time: 53160 ms
Total hot run time: 48943 ms
run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1 4596 4602 4554 4554
q2 340 228 265 228
q3 4014 3995 3978 3978
q4 2701 2662 2665 2662
q5 9592 9611 9520 9520
q6 242 125 124 124
q7 3002 2447 2467 2447
q8 4496 4484 4471 4471
q9 12922 12884 12905 12884
q10 4112 4169 4185 4169
q11 760 658 659 658
q12 974 836 801 801
q13 4276 3634 3605 3605
q14 383 344 352 344
q15 580 526 523 523
q16 727 653 658 653
q17 3950 3950 3909 3909
q18 9510 8823 8923 8823
q19 1787 1804 1798 1798
q20 2388 2071 2050 2050
q21 8831 8514 8536 8514
q22 885 795 806 795
Total cold run time: 81068 ms
Total hot run time: 77510 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](core) Fix wal space back pressure core [doris]
Posted by "Yukang-Lian (via GitHub)" <gi...@apache.org>.
Yukang-Lian commented on PR #27311:
URL: https://github.com/apache/doris/pull/27311#issuecomment-1826375347
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](core) Fix wal space back pressure core and add regression test [doris]
Posted by "Yukang-Lian (via GitHub)" <gi...@apache.org>.
Yukang-Lian commented on PR #27311:
URL: https://github.com/apache/doris/pull/27311#issuecomment-1826708663
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](core) Fix wal space back pressure core and add regression test [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #27311:
URL: https://github.com/apache/doris/pull/27311#issuecomment-1826790304
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 46.18 seconds
stream load tsv: 567 seconds loaded 74807831229 Bytes, about 125 MB/s
stream load json: 26 seconds loaded 2358488459 Bytes, about 86 MB/s
stream load orc: 70 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 28.5 seconds inserted 10000000 Rows, about 350K ops/s
storage size: 17098833145 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](core) Fix wal space back pressure core [doris]
Posted by "Yukang-Lian (via GitHub)" <gi...@apache.org>.
Yukang-Lian commented on PR #27311:
URL: https://github.com/apache/doris/pull/27311#issuecomment-1819465865
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](core) Fix wal space back pressure core [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #27311:
URL: https://github.com/apache/doris/pull/27311#issuecomment-1819476592
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](core) Fix wal space back pressure core [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #27311:
URL: https://github.com/apache/doris/pull/27311#issuecomment-1821067121
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 45.63 seconds
stream load tsv: 565 seconds loaded 74807831229 Bytes, about 126 MB/s
stream load json: 18 seconds loaded 2358488459 Bytes, about 124 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 28.9 seconds inserted 10000000 Rows, about 346K ops/s
storage size: 17098115979 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org