You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2022/10/15 12:56:05 UTC

[GitHub] [pulsar] yyj8 opened a new issue, #18056: Add broker level metrics statistics and print

yyj8 opened a new issue, #18056:
URL: https://github.com/apache/pulsar/issues/18056

   ### Search before asking
   
   - [X] I searched in the [issues](https://github.com/apache/pulsar/issues) and found nothing similar.
   
   
   ### Motivation
   
   Currently, pulsar does not statistics broker level metrics, and all relevant metrics are 0 by default.
   
   When the number of topic partitions reaches more than 100000 or even millions, and the topic level metrics are exposed (exposeTopicLevelMetricsInPrometheus=true), if you want to query the metrics of the broker dimension, you need to summarize all topics, and the performance becomes very poor, or even the results cannot be queried from the promethus. Common broker level metrics include:
   ```
   pulsar_ topics_ count
   pulsar_ subscriptions_ count
   pulsar_ producers_ count
   pulsar_ consumers_ count
   pulsar_ rate_ in
   pulsar_ rate_ out
   pulsar_ throughput_ in
   pulsar_ throughput_ out
   pulsar_ storage_ size
   pulsar_ storage_ logical_ size
   pulsar_ storage_ write_ rate
   pulsar_ storage_ read_ rate
   pulsar_ msg_ backlog
   ```
   
   We need to statistics the metrics of the broker dimension and expose them to prometheus to improve the performance of the monitoring  of the broker dimension. Modify the original metrics name as follows:
   ```
   pulsar_broker_topics_count
   pulsar_broker_subscriptions_count
   pulsar_broker_producers_count
   pulsar_broker_consumers_count
   pulsar_broker_rate_in
   pulsar_broker_rate_out
   pulsar_broker_throughput_in
   pulsar_broker_throughput_out
   pulsar_broker_storage_size
   pulsar_broker_storage_logical_size
   pulsar_broker_storage_write_rate
   pulsar_broker_storage_read_rate
   pulsar_broker_msg_backlog
   ```
   
   ### Solution
   
   _No response_
   
   ### Alternatives
   
   _No response_
   
   ### Anything else?
   
   _No response_
   
   ### Are you willing to submit a PR?
   
   - [X] I'm willing to submit a PR!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar] github-actions[bot] commented on issue #18056: [PIP-214][broker]Add broker level metrics statistics and expose to prometheus

Posted by GitBox <gi...@apache.org>.

github-actions[bot] commented on issue #18056:
URL: https://github.com/apache/pulsar/issues/18056#issuecomment-1324458504

   The issue had no activity for 30 days, mark with Stale label.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar] github-actions[bot] commented on issue #18056: [PIP-214][broker]Add broker level metrics statistics and expose to prometheus

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.

github-actions[bot] commented on issue #18056:
URL: https://github.com/apache/pulsar/issues/18056#issuecomment-1566384953

   The issue had no activity for 30 days, mark with Stale label.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar] poorbarcode commented on issue #18056: [PIP-214][broker]Add broker level metrics statistics and expose to prometheus

Posted by "poorbarcode (via GitHub)" <gi...@apache.org>.

poorbarcode commented on issue #18056:
URL: https://github.com/apache/pulsar/issues/18056#issuecomment-1526962956

   Hi @yyj8 
   
   What is the status of this PIP? I see https://github.com/apache/pulsar/pull/19047 is merged. So my understanding is that all things are completed, right? If yes, Could we send an email to close this PIP?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar] asafm commented on issue #18056: [PIP-214][broker]Add broker level metrics statistics and expose to prometheus

Posted by GitBox <gi...@apache.org>.

asafm commented on issue #18056:
URL: https://github.com/apache/pulsar/issues/18056#issuecomment-1288077347

   I'm not sure a PIP is formatted as you did. It has a certain process to it.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org