You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by GitBox <gi...@apache.org> on 2021/04/16 07:44:48 UTC

[GitHub] [incubator-doris] weizuo93 opened a new pull request #5665: [Metrics] Add metrics to show the distribution of tablets version count

weizuo93 opened a new pull request #5665:
URL: https://github.com/apache/incubator-doris/pull/5665


   ## Proposed changes
   
    Add metrics (min, max, median, average, standard_deviation, P50, P75, P90, P95, P99, etc.) to show the distribution of tablets version count.
   
   ```
   # TYPE doris_be_tablet_version_num_distribution gauge
   doris_be_tablet_version_num_distribution{type="P99"} 11.000000
   doris_be_tablet_version_num_distribution{type="P95"} 11.000000
   doris_be_tablet_version_num_distribution{type="P90"} 11.000000
   doris_be_tablet_version_num_distribution{type="P75"} 10.833333
   doris_be_tablet_version_num_distribution{type="min"} 8
   doris_be_tablet_version_num_distribution{type="max"} 11
   doris_be_tablet_version_num_distribution{type="sum"} 970
   doris_be_tablet_version_num_distribution{type="median"} 8.857143
   doris_be_tablet_version_num_distribution{type="average"} 9.700000
   doris_be_tablet_version_num_distribution{type="standard_deviation"} 1.187434
   doris_be_tablet_version_num_distribution{type="P50"} 8.857143
   ```
   
   ## Types of changes
   
   What types of changes does your code introduce to Doris?
   _Put an `x` in the boxes that apply_
   
   - [ ] Bugfix (non-breaking change which fixes an issue)
   - [x] New feature (non-breaking change which adds functionality)
   - [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
   - [ ] Documentation Update (if none of the other choices apply)
   - [ ] Code refactor (Modify the code structure, format the code, etc...)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org

[GitHub] [incubator-doris] morningman merged pull request #5665: [Metrics] Add metrics to show the distribution of tablets version count

Posted by GitBox <gi...@apache.org>.
morningman merged pull request #5665:
URL: https://github.com/apache/incubator-doris/pull/5665


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org

[GitHub] [incubator-doris] acelyc111 commented on a change in pull request #5665: [Metrics] Add metrics to show the distribution of tablets version count

Posted by GitBox <gi...@apache.org>.
acelyc111 commented on a change in pull request #5665:
URL: https://github.com/apache/incubator-doris/pull/5665#discussion_r616305639



##########
File path: be/src/util/doris_metrics.h
##########
@@ -137,6 +137,20 @@ class DorisMetrics {
     // permits required by the compaction task which is waitting for permits
     IntGauge* compaction_waitting_permits;
 
+    UIntGauge* tablet_version_num_distribution_num;
+    UIntGauge* tablet_version_num_distribution_min;
+    UIntGauge* tablet_version_num_distribution_max;
+    UIntGauge* tablet_version_num_distribution_sum;
+    DoubleGauge* tablet_version_num_distribution_median;
+    DoubleGauge* tablet_version_num_distribution_average;
+    DoubleGauge* tablet_version_num_distribution_standard_deviation;
+    DoubleGauge* tablet_version_num_distribution_p50;
+    DoubleGauge* tablet_version_num_distribution_p75;
+    DoubleGauge* tablet_version_num_distribution_p90;
+    DoubleGauge* tablet_version_num_distribution_p95;
+    DoubleGauge* tablet_version_num_distribution_p99;
+

Review comment:
       Use HistogramMetric directly, its output is standard now.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org

[GitHub] [incubator-doris] acelyc111 commented on a change in pull request #5665: [Metrics] Add metrics to show the distribution of tablets version count

Posted by GitBox <gi...@apache.org>.
acelyc111 commented on a change in pull request #5665:
URL: https://github.com/apache/incubator-doris/pull/5665#discussion_r616770113



##########
File path: be/src/olap/tablet_manager.cpp
##########
@@ -976,13 +978,17 @@ OLAPStatus TabletManager::build_all_report_tablets_info(std::map<TTabletId, TTab
                     expire_txn_map.erase(find);
                 }
                 t_tablet.tablet_infos.push_back(tablet_info);
+                if (tablet_ptr->tablet_id() == tablet_id) {
+                    DorisMetrics::instance()->tablet_version_num_distribution->add(tablet_ptr->version_count());

Review comment:
       It's not atomic and may last a long time before all values added, that means we may get a middle state which is not very correct.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org

[GitHub] [incubator-doris] weizuo93 commented on a change in pull request #5665: [Metrics] Add metrics to show the distribution of tablets version count

Posted by GitBox <gi...@apache.org>.
weizuo93 commented on a change in pull request #5665:
URL: https://github.com/apache/incubator-doris/pull/5665#discussion_r616483167



##########
File path: be/src/util/doris_metrics.h
##########
@@ -137,6 +137,20 @@ class DorisMetrics {
     // permits required by the compaction task which is waitting for permits
     IntGauge* compaction_waitting_permits;
 
+    UIntGauge* tablet_version_num_distribution_num;
+    UIntGauge* tablet_version_num_distribution_min;
+    UIntGauge* tablet_version_num_distribution_max;
+    UIntGauge* tablet_version_num_distribution_sum;
+    DoubleGauge* tablet_version_num_distribution_median;
+    DoubleGauge* tablet_version_num_distribution_average;
+    DoubleGauge* tablet_version_num_distribution_standard_deviation;
+    DoubleGauge* tablet_version_num_distribution_p50;
+    DoubleGauge* tablet_version_num_distribution_p75;
+    DoubleGauge* tablet_version_num_distribution_p90;
+    DoubleGauge* tablet_version_num_distribution_p95;
+    DoubleGauge* tablet_version_num_distribution_p99;
+

Review comment:
       > Use HistogramMetric directly, its output is standard now.
   
   OK,  I see it too. PR #5671 has been merged.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org

[GitHub] [incubator-doris] weizuo93 commented on a change in pull request #5665: [Metrics] Add metrics to show the distribution of tablets version count

Posted by GitBox <gi...@apache.org>.
weizuo93 commented on a change in pull request #5665:
URL: https://github.com/apache/incubator-doris/pull/5665#discussion_r617184124



##########
File path: be/src/olap/tablet_manager.cpp
##########
@@ -976,13 +978,17 @@ OLAPStatus TabletManager::build_all_report_tablets_info(std::map<TTabletId, TTab
                     expire_txn_map.erase(find);
                 }
                 t_tablet.tablet_infos.push_back(tablet_info);
+                if (tablet_ptr->tablet_id() == tablet_id) {
+                    DorisMetrics::instance()->tablet_version_num_distribution->add(tablet_ptr->version_count());

Review comment:
       > It's not atomic and may last a long time before all values added, that means we may get a middle state which is not very correct.
   
   Thank you, you are right.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org