You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "koarz (via GitHub)" <gi...@apache.org> on 2025/12/24 07:55:04 UTC
[PR] [enhance](meta-service)add bvar for fdb process status [doris]
koarz opened a new pull request, #59329:
URL: https://github.com/apache/doris/pull/59329
### What problem does this PR solve?
Issue Number: close #xxx
Related PR: #xxx
Problem Summary:
### Release note
None
### Check List (For Author)
- Test <!-- At least one of them must be included. -->
- [ ] Regression test
- [ ] Unit Test
- [ ] Manual test (add detailed scripts or steps below)
- [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
- [ ] Previous test can cover this change.
- [ ] No code files have been changed.
- [ ] Other reason <!-- Add your reason? -->
- Behavior changed:
- [ ] No.
- [ ] Yes. <!-- Explain the behavior change -->
- Does this need documentation?
- [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg: https://github.com/apache/doris-website/pull/1214 -->
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR should merge into -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](meta-service)add bvar for fdb process status [doris]
Posted by "Thearas (via GitHub)" <gi...@apache.org>.
Thearas commented on PR #59329:
URL: https://github.com/apache/doris/pull/59329#issuecomment-3688990644
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR).
Please clearly describe your PR:
1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
3. What features were added. Why was this function added?
4. Which code was refactored and why was this part of the code refactored?
5. Which functions were optimized and what is the difference before and after the optimization?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](meta-service)add bvar for fdb process status [doris]
Posted by "koarz (via GitHub)" <gi...@apache.org>.
koarz commented on PR #59329:
URL: https://github.com/apache/doris/pull/59329#issuecomment-3689360375
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](meta-service)add bvar for fdb process status [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59329:
URL: https://github.com/apache/doris/pull/59329#issuecomment-3698260006
# Cloud UT Coverage Report
Increment line coverage `92.59% (75/81)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/8a9359873b6c56a26ac418e15468f18654c0a70f_8a9359873b6c56a26ac418e15468f18654c0a70f_cloud/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/8a9359873b6c56a26ac418e15468f18654c0a70f_8a9359873b6c56a26ac418e15468f18654c0a70f_cloud/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 79.54% (1773/2229) |
| Line Coverage | 64.85% (31367/48367) |
| Region Coverage | 65.38% (15596/23856) |
| Branch Coverage | 56.03% (8298/14810) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](meta-service)add bvar for fdb process status [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #59329:
URL: https://github.com/apache/doris/pull/59329#issuecomment-3701291802
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](meta-service)add bvar for fdb process status [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #59329:
URL: https://github.com/apache/doris/pull/59329#issuecomment-3689062625
# Cloud UT Coverage Report
Increment line coverage `92.68% (76/82)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/be1143964adb1e1f5cd313f707bcadb89dadcf8a_be1143964adb1e1f5cd313f707bcadb89dadcf8a_cloud/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/be1143964adb1e1f5cd313f707bcadb89dadcf8a_be1143964adb1e1f5cd313f707bcadb89dadcf8a_cloud/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 79.54% (1773/2229) |
| Line Coverage | 64.88% (31351/48325) |
| Region Coverage | 65.40% (15581/23825) |
| Branch Coverage | 56.00% (8282/14790) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](meta-service)add bvar for fdb process status [doris]
Posted by "gavinchou (via GitHub)" <gi...@apache.org>.
gavinchou commented on code in PR #59329:
URL: https://github.com/apache/doris/pull/59329#discussion_r2645388305
##########
cloud/src/common/metric.cpp:
##########
@@ -296,10 +234,117 @@ static void export_fdb_status_details(const std::string& status_str) {
}
}
+ // Helper function for recursive name construction
+ // such as: {"disk": {"reads": {"counter": 123, "hz": 0}}}
+ // component is "disk", the names of these two values should be "reads_counter" and "reads_hz"
+ auto recursive_name_helper = [](std::string& origin_name,
+ const char* next_level_name) -> std::string {
+ return origin_name + '_' + next_level_name;
+ };
+
+ // Generic recursive function to traverse JSON node and set bvar values
+ // There are three cases here: int64, double, and object.
+ // If it is double or int64, put it directly into the bvar.
+ // If it is an object, recursively obtain the full name and corresponding value.
+ auto recursive_traverse_and_set = [&recursive_name_helper](auto&& set_value_callback,
+ auto&& self, std::string name,
+ auto temp_node) -> void {
+ // if the node is an object, then get Member(iter) and recursive with iter as arg
+ if (temp_node->value.IsObject()) {
+ for (auto iter = temp_node->value.MemberBegin(); iter != temp_node->value.MemberEnd();
+ iter++) {
+ self(set_value_callback, self, recursive_name_helper(name, iter->name.GetString()),
+ iter);
+ }
+ return;
+ }
+ // if not object, set bvar value
+ set_value_callback(name, temp_node);
+ };
+
+ auto get_process_metric = [&](std::string component) {
+ auto node = document.FindMember("cluster");
+ if (!node->value.HasMember("processes")) return;
+ node = node->value.FindMember("processes");
+ // process
+ for (auto process_node = node->value.MemberBegin(); process_node != node->value.MemberEnd();
+ process_node++) {
+ const char* process_id = process_node->name.GetString();
+ decltype(process_node) component_node;
+ // get component iter
+ if (!process_node->value.HasMember(component.data())) return;
Review Comment:
seems should be `continue` instead of `return` if there is no such a member
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](meta-service)add bvar for fdb process status [doris]
Posted by "koarz (via GitHub)" <gi...@apache.org>.
koarz commented on PR #59329:
URL: https://github.com/apache/doris/pull/59329#issuecomment-3698204685
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](meta-service)add bvar for fdb process status [doris]
Posted by "gavinchou (via GitHub)" <gi...@apache.org>.
gavinchou commented on PR #59329:
URL: https://github.com/apache/doris/pull/59329#issuecomment-3704770198
run cloud_p0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](meta-service)add bvar for fdb process status [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen merged PR #59329:
URL: https://github.com/apache/doris/pull/59329
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](meta-service)add bvar for fdb process status [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #59329:
URL: https://github.com/apache/doris/pull/59329#issuecomment-3689474000
# Cloud UT Coverage Report
Increment line coverage `92.59% (75/81)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/682be9d59627f18d947a4c7cf259956f66371e28_682be9d59627f18d947a4c7cf259956f66371e28_cloud/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/682be9d59627f18d947a4c7cf259956f66371e28_682be9d59627f18d947a4c7cf259956f66371e28_cloud/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 79.54% (1773/2229) |
| Line Coverage | 64.87% (31348/48322) |
| Region Coverage | 65.41% (15585/23825) |
| Branch Coverage | 56.02% (8285/14790) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](meta-service)add bvar for fdb process status [doris]
Posted by "gavinchou (via GitHub)" <gi...@apache.org>.
gavinchou commented on PR #59329:
URL: https://github.com/apache/doris/pull/59329#issuecomment-3704770334
run vault_p0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](meta-service)add bvar for fdb process status [doris]
Posted by "koarz (via GitHub)" <gi...@apache.org>.
koarz commented on PR #59329:
URL: https://github.com/apache/doris/pull/59329#issuecomment-3688990979
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](meta-service)add bvar for fdb process status [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #59329:
URL: https://github.com/apache/doris/pull/59329#issuecomment-3707548986
run external
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](meta-service)add bvar for fdb process status [doris]
Posted by "gavinchou (via GitHub)" <gi...@apache.org>.
gavinchou commented on code in PR #59329:
URL: https://github.com/apache/doris/pull/59329#discussion_r2645105914
##########
cloud/src/common/bvars.cpp:
##########
@@ -257,8 +257,8 @@ bvar::Status<int64_t> g_bvar_fdb_workload_transactions_started_hz("fdb_workload_
bvar::Status<int64_t> g_bvar_fdb_workload_transactions_committed_hz("fdb_workload_transactions_committed_hz", BVAR_FDB_INVALID_VALUE);
bvar::Status<int64_t> g_bvar_fdb_workload_transactions_rejected_hz("fdb_workload_transactions_rejected_hz", BVAR_FDB_INVALID_VALUE);
bvar::Status<int64_t> g_bvar_fdb_client_thread_busyness_percent("fdb_client_thread_busyness_percent", BVAR_FDB_INVALID_VALUE);
-mBvarStatus<int64_t> g_bvar_fdb_process_status_int("fdb_process_status_int", {"process_id", "component", "metric"});
-mBvarStatus<double> g_bvar_fdb_process_status_float("fdb_process_status_float", {"process_id", "component", "metric"});
+mBvarStatus<double> g_bvar_fdb_cluster_process_status("fdb_cluster_process_status", {"process_id", "component", "metric"});
Review Comment:
label naming
fdb_cluster_process_status -> fdb_cluster_processes
fdb_cluster_workload_status -> fdb_cluster_workload
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](meta-service)add bvar for fdb process status [doris]
Posted by "koarz (via GitHub)" <gi...@apache.org>.
koarz commented on PR #59329:
URL: https://github.com/apache/doris/pull/59329#issuecomment-3692698923
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](meta-service)add bvar for fdb process status [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59329:
URL: https://github.com/apache/doris/pull/59329#issuecomment-3692734853
# Cloud UT Coverage Report
Increment line coverage `92.59% (75/81)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/ebb9e89088f7def15f466d513c6640accac6bbed_ebb9e89088f7def15f466d513c6640accac6bbed_cloud/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/ebb9e89088f7def15f466d513c6640accac6bbed_ebb9e89088f7def15f466d513c6640accac6bbed_cloud/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 79.54% (1773/2229) |
| Line Coverage | 64.83% (31340/48341) |
| Region Coverage | 65.39% (15595/23848) |
| Branch Coverage | 55.98% (8288/14806) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](meta-service)add bvar for fdb process status [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #59329:
URL: https://github.com/apache/doris/pull/59329#issuecomment-3701291766
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org