You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "hust-hhb (via GitHub)" <gi...@apache.org> on 2024/03/01 04:27:57 UTC
[PR] [improvement](group_comit) Add bvar to monitor the total wal count on disk [doris]
hust-hhb opened a new pull request, #31646:
URL: https://github.com/apache/doris/pull/31646
## Proposed changes
Use a bvar to monitor the total wal count on disk.
Issue Number: close #xxx
<!--Describe your changes.-->
## Further comments
If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](group_comit) Add bvar to monitor the total wal count on disk [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31646:
URL: https://github.com/apache/doris/pull/31646#issuecomment-1972485814
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](group_comit) Add bvar to monitor the total wal count on disk [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31646:
URL: https://github.com/apache/doris/pull/31646#issuecomment-1975557956
TeamCity be ut coverage result:
Function Coverage: 35.43% (8555/24146)
Line Coverage: 27.20% (69427/255230)
Region Coverage: 26.41% (36010/136337)
Branch Coverage: 23.25% (18394/79100)
Coverage Report: http://coverage.selectdb-in.cc/coverage/06caa9c4594cdd9df9468c456eaecf885312d3a3_06caa9c4594cdd9df9468c456eaecf885312d3a3/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](group_comit) Add bvar to monitor the total wal count on disk [doris]
Posted by "hust-hhb (via GitHub)" <gi...@apache.org>.
hust-hhb commented on PR #31646:
URL: https://github.com/apache/doris/pull/31646#issuecomment-2009082779
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](group_comit) Add bvar to monitor the total wal count on disk [doris]
Posted by "dataroaring (via GitHub)" <gi...@apache.org>.
dataroaring merged PR #31646:
URL: https://github.com/apache/doris/pull/31646
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](group_comit) Add bvar to monitor the total wal count on disk [doris]
Posted by "hust-hhb (via GitHub)" <gi...@apache.org>.
hust-hhb commented on code in PR #31646:
URL: https://github.com/apache/doris/pull/31646#discussion_r1508713110
##########
be/src/olap/wal/wal_manager.cpp:
##########
@@ -380,6 +386,8 @@ Status WalManager::_replay_background() {
if (_first_replay.compare_exchange_strong(expected, false)) {
RETURN_IF_ERROR(_load_wals());
}
+ auto _total_wal_size = get_wal_queue_size(-1);
+ g_wal_total_count.set_value(_total_wal_size);
Review Comment:
@mymeiyi already change
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](group_comit) Add bvar to monitor the total wal count on disk [doris]
Posted by "hust-hhb (via GitHub)" <gi...@apache.org>.
hust-hhb commented on PR #31646:
URL: https://github.com/apache/doris/pull/31646#issuecomment-2008721657
run p0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](group_comit) Add bvar to monitor the total wal count on disk [doris]
Posted by "hust-hhb (via GitHub)" <gi...@apache.org>.
hust-hhb commented on PR #31646:
URL: https://github.com/apache/doris/pull/31646#issuecomment-2009023731
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](group_comit) Add bvar to monitor the total wal count on disk [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31646:
URL: https://github.com/apache/doris/pull/31646#issuecomment-1972540347
TeamCity be ut coverage result:
Function Coverage: 35.64% (8550/23988)
Line Coverage: 27.45% (69397/252802)
Region Coverage: 26.62% (36014/135265)
Branch Coverage: 23.44% (18397/78498)
Coverage Report: http://coverage.selectdb-in.cc/coverage/e024be77ac0e23072b660138ad9e7ec3e565d450_e024be77ac0e23072b660138ad9e7ec3e565d450/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](group_comit) Add bvar to monitor the total wal count on disk [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #31646:
URL: https://github.com/apache/doris/pull/31646#issuecomment-1972856513
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](group_comit) Add bvar to monitor the total wal count on disk [doris]
Posted by "hust-hhb (via GitHub)" <gi...@apache.org>.
hust-hhb commented on PR #31646:
URL: https://github.com/apache/doris/pull/31646#issuecomment-2006493506
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](group_comit) Add bvar to monitor the total wal count on disk [doris]
Posted by "mymeiyi (via GitHub)" <gi...@apache.org>.
mymeiyi commented on code in PR #31646:
URL: https://github.com/apache/doris/pull/31646#discussion_r1508697219
##########
be/src/olap/wal/wal_manager.cpp:
##########
@@ -380,6 +386,8 @@ Status WalManager::_replay_background() {
if (_first_replay.compare_exchange_strong(expected, false)) {
RETURN_IF_ERROR(_load_wals());
}
+ auto _total_wal_size = get_wal_queue_size(-1);
+ g_wal_total_count.set_value(_total_wal_size);
Review Comment:
can write to one line:
g_wal_total_count.set_value(get_wal_queue_size(-1));
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](group_comit) Add bvar to monitor the total wal count on disk [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #31646:
URL: https://github.com/apache/doris/pull/31646#issuecomment-2006328782
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](group_comit) Add bvar to monitor the total wal count on disk [doris]
Posted by "hust-hhb (via GitHub)" <gi...@apache.org>.
hust-hhb commented on PR #31646:
URL: https://github.com/apache/doris/pull/31646#issuecomment-1975538908
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](group_comit) Add bvar to monitor the total wal count on disk [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31646:
URL: https://github.com/apache/doris/pull/31646#issuecomment-2009111398
TeamCity be ut coverage result:
Function Coverage: 35.31% (8731/24730)
Line Coverage: 27.10% (71423/263531)
Region Coverage: 26.35% (37071/140705)
Branch Coverage: 23.25% (18951/81512)
Coverage Report: http://coverage.selectdb-in.cc/coverage/06caa9c4594cdd9df9468c456eaecf885312d3a3_06caa9c4594cdd9df9468c456eaecf885312d3a3/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](group_comit) Add bvar to monitor the total wal count on disk [doris]
Posted by "hust-hhb (via GitHub)" <gi...@apache.org>.
hust-hhb commented on PR #31646:
URL: https://github.com/apache/doris/pull/31646#issuecomment-2008564257
run p0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](group_comit) Add bvar to monitor the total wal count on disk [doris]
Posted by "hust-hhb (via GitHub)" <gi...@apache.org>.
hust-hhb commented on PR #31646:
URL: https://github.com/apache/doris/pull/31646#issuecomment-1972485929
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](group_comit) Add bvar to monitor the total wal count on disk [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31646:
URL: https://github.com/apache/doris/pull/31646#issuecomment-2009061474
TeamCity be ut coverage result:
Function Coverage: 35.31% (8731/24730)
Line Coverage: 27.10% (71419/263531)
Region Coverage: 26.35% (37070/140705)
Branch Coverage: 23.25% (18952/81512)
Coverage Report: http://coverage.selectdb-in.cc/coverage/06caa9c4594cdd9df9468c456eaecf885312d3a3_06caa9c4594cdd9df9468c456eaecf885312d3a3/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](group_comit) Add bvar to monitor the total wal count on disk [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31646:
URL: https://github.com/apache/doris/pull/31646#issuecomment-2006589453
TeamCity be ut coverage result:
Function Coverage: 35.32% (8727/24710)
Line Coverage: 27.13% (71385/263144)
Region Coverage: 26.38% (37036/140410)
Branch Coverage: 23.27% (18935/81360)
Coverage Report: http://coverage.selectdb-in.cc/coverage/06caa9c4594cdd9df9468c456eaecf885312d3a3_06caa9c4594cdd9df9468c456eaecf885312d3a3/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](group_comit) Add bvar to monitor the total wal count on disk [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #31646:
URL: https://github.com/apache/doris/pull/31646#issuecomment-1972835213
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](group_comit) Add bvar to monitor the total wal count on disk [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #31646:
URL: https://github.com/apache/doris/pull/31646#issuecomment-1972492512
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](group_comit) Add bvar to monitor the total wal count on disk [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31646:
URL: https://github.com/apache/doris/pull/31646#issuecomment-1972519479
<details>
<summary>TPC-H: <b>Total hot run time: 38165 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit e024be77ac0e23072b660138ad9e7ec3e565d450, data reload: false
------ Round 1 ----------------------------------
q1 17662 4302 4053 4053
q2 2052 150 141 141
q3 10675 1006 958 958
q4 4686 955 952 952
q5 7624 2931 3608 2931
q6 183 126 124 124
q7 1307 842 824 824
q8 9572 2109 2109 2109
q9 7471 6590 6496 6496
q10 8258 2593 2627 2593
q11 421 220 216 216
q12 794 316 318 316
q13 17995 2989 2937 2937
q14 276 254 250 250
q15 476 443 432 432
q16 472 392 393 392
q17 955 878 842 842
q18 6825 5815 5907 5815
q19 1571 1535 1537 1535
q20 559 302 283 283
q21 7569 3676 3696 3676
q22 812 290 295 290
Total cold run time: 108215 ms
Total hot run time: 38165 ms
----- Round 2, with runtime_filter_mode=off -----
q1 3991 4013 3971 3971
q2 308 218 222 218
q3 2975 2904 2931 2904
q4 1831 1806 1798 1798
q5 5240 5251 5248 5248
q6 201 115 117 115
q7 2242 1797 1808 1797
q8 3215 3268 3295 3268
q9 8496 8531 8566 8531
q10 6126 3763 3740 3740
q11 535 432 441 432
q12 686 542 537 537
q13 7109 2821 2789 2789
q14 286 263 247 247
q15 473 447 440 440
q16 454 426 399 399
q17 1691 1676 1671 1671
q18 7926 7533 7243 7243
q19 3206 1638 1625 1625
q20 1946 1769 1731 1731
q21 5060 4810 4715 4715
q22 529 477 467 467
Total cold run time: 64526 ms
Total hot run time: 53886 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org