You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by GitBox <gi...@apache.org> on 2022/10/25 08:16:21 UTC
[GitHub] [doris] ByteYue opened a new pull request, #13646: [enhancement](metric)Add metric for exec_state prepare function
ByteYue opened a new pull request, #13646:
URL: https://github.com/apache/doris/pull/13646
# Proposed changes
Issue Number: close #xxx
## Problem summary
Previously, we found out that sometimes the rpc service might timeout due to the extremely time consuming operation during exec_state prepare, thus this pr adds one metric to monitor the prepare time cost of each single fragment.
After compiling with this commit, BE would be able to generate data compatible with grafana, and you can watch the data using dashboard like below.
As for the total time consumption of all the fragments prepare, you can refer to the brpc port to watch the rpc metric for more detailed message.
## Checklist(Required)
1. Does it affect the original behavior:
- [ ] Yes
- [x] No
- [ ] I don't know
2. Has unit tests been added:
- [x] Yes
- [ ] No
- [ ] No Need
3. Has document been added or modified:
- [x] Yes
- [ ] No
- [ ] No Need
4. Does it need to update dependencies:
- [ ] Yes
- [x] No
5. Are there any changes that cannot be rolled back:
- [ ] Yes (If Yes, please explain WHY)
- [x] No
## Further comments
If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] ByteYue commented on pull request #13646: [enhancement](metric)Add metric for exec_state prepare function
Posted by GitBox <gi...@apache.org>.
ByteYue commented on PR #13646:
URL: https://github.com/apache/doris/pull/13646#issuecomment-1290686548
> LGTM, and maybe we can move all metrics to bvar in future.
Looking forward to it😂
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] dataroaring commented on a diff in pull request #13646: [enhancement](metric)Add metric for exec_state prepare function
Posted by GitBox <gi...@apache.org>.
dataroaring commented on code in PR #13646:
URL: https://github.com/apache/doris/pull/13646#discussion_r1006845262
##########
be/src/runtime/fragment_mgr.cpp:
##########
@@ -61,6 +63,7 @@ namespace doris {
DEFINE_GAUGE_METRIC_PROTOTYPE_2ARG(plan_fragment_count, MetricUnit::NOUNIT);
DEFINE_GAUGE_METRIC_PROTOTYPE_2ARG(timeout_canceled_fragment_count, MetricUnit::NOUNIT);
DEFINE_GAUGE_METRIC_PROTOTYPE_2ARG(fragment_thread_pool_queue_size, MetricUnit::NOUNIT);
+bvar::LatencyRecorder g_fragmentmgr_prepare_latency("doris_FragmentMgr", "prepare");
Review Comment:
In my memory, latency of bvar should be used with window.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #13646: [enhancement](metric)Add metric for exec_state prepare function
Posted by GitBox <gi...@apache.org>.
hello-stephen commented on PR #13646:
URL: https://github.com/apache/doris/pull/13646#issuecomment-1294445898
TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 38.53 seconds
load time: 558 seconds
storage size: 17154699358 Bytes
https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20221028043958_clickbench_pr_35107.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #13646: [enhancement](metric)Add metric for exec_state prepare function
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #13646:
URL: https://github.com/apache/doris/pull/13646#issuecomment-1291142747
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #13646: [enhancement](metric)Add metric for exec_state prepare function
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #13646:
URL: https://github.com/apache/doris/pull/13646#issuecomment-1291142799
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] ByteYue commented on a diff in pull request #13646: [enhancement](metric)Add metric for exec_state prepare function
Posted by GitBox <gi...@apache.org>.
ByteYue commented on code in PR #13646:
URL: https://github.com/apache/doris/pull/13646#discussion_r1007039627
##########
be/src/runtime/fragment_mgr.cpp:
##########
@@ -61,6 +63,7 @@ namespace doris {
DEFINE_GAUGE_METRIC_PROTOTYPE_2ARG(plan_fragment_count, MetricUnit::NOUNIT);
DEFINE_GAUGE_METRIC_PROTOTYPE_2ARG(timeout_canceled_fragment_count, MetricUnit::NOUNIT);
DEFINE_GAUGE_METRIC_PROTOTYPE_2ARG(fragment_thread_pool_queue_size, MetricUnit::NOUNIT);
+bvar::LatencyRecorder g_fragmentmgr_prepare_latency("doris_FragmentMgr", "prepare");
Review Comment:
I did one test using this LatencyRecorder, it seemed that it could give us a whole view of latency and percentiles.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] morningman commented on a diff in pull request #13646: [enhancement](metric)Add metric for exec_state prepare function
Posted by GitBox <gi...@apache.org>.
morningman commented on code in PR #13646:
URL: https://github.com/apache/doris/pull/13646#discussion_r1004315541
##########
docs/zh-CN/docs/admin-manual/maint-monitor/monitor-metrics/metrics.md:
##########
@@ -210,6 +210,7 @@ curl http://be_host:webserver_port/metrics?type=json
|| `{status="skip",type="report_all_tablets"}` | Num | xxx 类型任务被跳过执行的次数的累计值 | |
|`doris_be_fragment_endpoint_count`| | Num| 同 | FIXME: 同 `doris_be_data_stream_receiver_count` 数目。并且向量化引擎缺失 |
|`doris_be_fragment_request_duration_us`| | 微秒| 所有 fragment intance 的执行时间累计 | 通过斜率观测 instance 的执行耗时 | P0 |
+|`doris_be_fragment_prepare_duration_ns`| | 纳秒| 单个 fragment intance 的prepare时间累计 | 通过斜率观测 instance 的prepare耗时 | P0 |
Review Comment:
How about use same unit as `doris_be_fragment_request_duration_us`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] dataroaring commented on pull request #13646: [enhancement](metric)Add metric for exec_state prepare function
Posted by GitBox <gi...@apache.org>.
dataroaring commented on PR #13646:
URL: https://github.com/apache/doris/pull/13646#issuecomment-1293494778
please update the figure above.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #13646: [enhancement](metric)Add metric for exec_state prepare function
Posted by GitBox <gi...@apache.org>.
hello-stephen commented on PR #13646:
URL: https://github.com/apache/doris/pull/13646#issuecomment-1294903443
TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 39.13 seconds
load time: 572 seconds
storage size: 17154644815 Bytes
https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20221028115203_clickbench_pr_35384.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] yiguolei merged pull request #13646: [enhancement](metric)Add metric for exec_state prepare function
Posted by GitBox <gi...@apache.org>.
yiguolei merged PR #13646:
URL: https://github.com/apache/doris/pull/13646
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org