You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@rocketmq.apache.org by wl...@apache.org on 2020/03/08 13:16:20 UTC
[rocketmq-exporter] branch master updated (81c90f3 -> 5d25982)
This is an automated email from the ASF dual-hosted git repository.
wlliqipeng pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/rocketmq-exporter.git.
from 81c90f3 Update README.md
new f0e1125 add all metrics
new 95153a0 format codes
new fd564d9 add client consume runtime info
new 5d25982 Merge pull request #9 from francisoliverlee/master
The 43 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
pom.xml | 192 +--
.../exporter/aspect/admin/MQAdminAspect.java | 71 --
.../admin/annotation/MultiMQAdminCmdMethod.java | 30 -
.../exporter/collector/RMQMetricsCollector.java | 1277 ++++++++++++++++++--
.../config/CollectClientMetricExecutorConfig.java | 46 +
.../rocketmq/exporter/config/RMQConfigure.java | 4 +-
.../rocketmq/exporter/config/ScheduleConfig.java | 26 +
.../exporter/controller/RMQMetricsController.java | 9 +-
.../exporter/exception/ServiceException.java | 31 -
.../exporter/model/BrokerRuntimeStats.java | 574 +++++++++
.../rocketmq/exporter/model/common/TwoTuple.java | 19 +
.../exporter/model/metrics/BrokerMetric.java | 43 +-
.../model/metrics/ConsumerCountMetric.java | 58 +
.../exporter/model/metrics/ConsumerMetric.java | 45 +-
.../model/metrics/ConsumerQueueMetric.java | 93 --
.../model/metrics/ConsumerTopicDiffMetric.java | 71 ++
.../model/metrics/DLQTopicOffsetMetric.java | 72 ++
.../exporter/model/metrics/ProducerMetric.java | 61 +-
.../exporter/model/metrics/TopicPutNumMetric.java | 83 ++
.../metrics/brokerruntime/BrokerRuntimeMetric.java | 91 ++
.../ConsumerRuntimeConsumeFailedMsgsMetric.java | 73 ++
.../ConsumerRuntimeConsumeFailedTPSMetric.java | 21 +
.../ConsumerRuntimeConsumeOKTPSMetric.java | 21 +
.../ConsumerRuntimeConsumeRTMetric.java | 21 +
.../clientrunime/ConsumerRuntimePullRTMetric.java | 21 +
.../clientrunime/ConsumerRuntimePullTPSMetric.java | 21 +
.../exporter/service/AbstractCommonService.java | 50 -
.../exporter/service/RMQMetricsService.java | 6 +-
.../exporter/service/client/MQAdminExtImpl.java | 279 +++--
.../exporter/service/client/MQAdminInstance.java | 118 +-
.../service/impl/RMQMetricsServiceImpl.java | 90 +-
...ientMetricCollectorFixedThreadPoolExecutor.java | 24 +
.../exporter/task/ClientMetricTaskRunnable.java | 124 ++
.../rocketmq/exporter/task/MetricsCollectTask.java | 685 +++++++----
.../apache/rocketmq/exporter/util/JsonUtil.java | 39 +-
src/main/resources/application.properties | 14 -
src/main/resources/application.yml | 38 +
src/main/resources/logback.xml | 54 +-
38 files changed, 3567 insertions(+), 1028 deletions(-)
delete mode 100644 src/main/java/org/apache/rocketmq/exporter/aspect/admin/MQAdminAspect.java
delete mode 100644 src/main/java/org/apache/rocketmq/exporter/aspect/admin/annotation/MultiMQAdminCmdMethod.java
create mode 100644 src/main/java/org/apache/rocketmq/exporter/config/CollectClientMetricExecutorConfig.java
create mode 100644 src/main/java/org/apache/rocketmq/exporter/config/ScheduleConfig.java
delete mode 100644 src/main/java/org/apache/rocketmq/exporter/exception/ServiceException.java
create mode 100644 src/main/java/org/apache/rocketmq/exporter/model/BrokerRuntimeStats.java
create mode 100644 src/main/java/org/apache/rocketmq/exporter/model/common/TwoTuple.java
create mode 100644 src/main/java/org/apache/rocketmq/exporter/model/metrics/ConsumerCountMetric.java
delete mode 100644 src/main/java/org/apache/rocketmq/exporter/model/metrics/ConsumerQueueMetric.java
create mode 100644 src/main/java/org/apache/rocketmq/exporter/model/metrics/ConsumerTopicDiffMetric.java
create mode 100644 src/main/java/org/apache/rocketmq/exporter/model/metrics/DLQTopicOffsetMetric.java
create mode 100644 src/main/java/org/apache/rocketmq/exporter/model/metrics/TopicPutNumMetric.java
create mode 100644 src/main/java/org/apache/rocketmq/exporter/model/metrics/brokerruntime/BrokerRuntimeMetric.java
create mode 100644 src/main/java/org/apache/rocketmq/exporter/model/metrics/clientrunime/ConsumerRuntimeConsumeFailedMsgsMetric.java
create mode 100644 src/main/java/org/apache/rocketmq/exporter/model/metrics/clientrunime/ConsumerRuntimeConsumeFailedTPSMetric.java
create mode 100644 src/main/java/org/apache/rocketmq/exporter/model/metrics/clientrunime/ConsumerRuntimeConsumeOKTPSMetric.java
create mode 100644 src/main/java/org/apache/rocketmq/exporter/model/metrics/clientrunime/ConsumerRuntimeConsumeRTMetric.java
create mode 100644 src/main/java/org/apache/rocketmq/exporter/model/metrics/clientrunime/ConsumerRuntimePullRTMetric.java
create mode 100644 src/main/java/org/apache/rocketmq/exporter/model/metrics/clientrunime/ConsumerRuntimePullTPSMetric.java
delete mode 100644 src/main/java/org/apache/rocketmq/exporter/service/AbstractCommonService.java
create mode 100644 src/main/java/org/apache/rocketmq/exporter/task/ClientMetricCollectorFixedThreadPoolExecutor.java
create mode 100644 src/main/java/org/apache/rocketmq/exporter/task/ClientMetricTaskRunnable.java
delete mode 100644 src/main/resources/application.properties
create mode 100644 src/main/resources/application.yml
[rocketmq-exporter] 22/43: add setup example
Posted by wl...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
wlliqipeng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/rocketmq-exporter.git
commit 8285146b21e3bd5200ea1b407b54dfcce71f23da
Author: breezecoolyang <br...@163.com>
AuthorDate: Sun Jul 21 15:40:18 2019 +0800
add setup example
---
rocketmq_exporter_setup_example.md | 101 +++++++++++++++++++++++++++++++++++++
1 file changed, 101 insertions(+)
diff --git a/rocketmq_exporter_setup_example.md b/rocketmq_exporter_setup_example.md
new file mode 100644
index 0000000..a550185
--- /dev/null
+++ b/rocketmq_exporter_setup_example.md
@@ -0,0 +1,101 @@
+## RocketMQ-Exporter使用示例
+
+1 启动NameServer和Broker
+
+要验证RocketMQ的Spring-Boot客户端，首先要确保RocketMQ服务正确的下载并启动。可以参考RocketMQ主站的快速开始来进行操作。确保启动NameServer和Broker已经正确启动。
+
+2 编译RocketMQ-Exporter
+用户当前使用，需要自行下载git源码编译
+
+```
+git clone https://github.com/apache/rocketmq-exporter
+cd rocketmq-exporter
+mvn clean install
+```
+3 配置和运行
+RocketMQ-Exporter 有如下的运行选项
+选项 | 默认值 | 含义
+---|---|---
+rocketmq.config.namesrvAddr | 127.0.0.1:9876 | MQ集群的nameSrv地址
+rocketmq.config.webTelemetryPath | /metrics | 指标搜集路径
+server.port | 5557 | HTTP服务暴露端口
+以上的运行选项既可以在下载代码后在配置文件中更改，也可以通过命令行来设置。
+编译出来的jar包就叫rocketmq-exporter-0.0.1-SNAPSHOT.jar，可以通过如下的方式来运行。
+
+```
+java -jar rocketmq-exporter-0.0.1-SNAPSHOT.jar [--rocketmq.config.namesrvAddr="127.0.0.1:9876" ...]
+```
+
+4 安装Prometheus
+首先到Prometheus官方下载地址:[https://prometheus.io/download/](https://prometheus.io/download/)去下载Prometheus安装包，当前以linux的安装为例，选择的安装包为
+prometheus-2.7.0-rc.1.linux-amd64.tar.gz，经过如下的操作步骤就可以启动prometheus进程。
+
+```
+tar -xzf prometheus-2.7.0-rc.1.linux-amd64.tar.gz
+cd prometheus-2.7.0-rc.1.linux-amd64/
+./prometheus --config.file=prometheus.yml --web.listen-address=:5555
+```
+prometheus 默认监听端口号为9090，为了不与系统上的其它进程监听端口冲突，我们在启动参数里面重新设置了监听端口号为5555。然后通过浏览器访问http://<服务器IP地址>:5555,就可以验证prometheus是否已成功安装。由于RocketMQ-Exporter进程已启动，这个时候可以通过prometheus来抓取RocketMQ-Exporter的数据，这个时候只需要更改prometheus启动的配置文件即可。
+整体配置文件如下：
+
+```
+# my global config
+global:
+ scrape_interval: 15s # Set the scrape interval to every 15 seconds. Default is every 1 minute.
+ evaluation_interval: 15s # Evaluate rules every 15 seconds. The default is every 1 minute.
+ # scrape_timeout is set to the global default (10s).
+
+
+ # Load rules once and periodically evaluate them according to the global 'evaluation_interval'.
+ rule_files:
+ # - "first_rules.yml"
+ # - "second_rules.yml"
+
+
+ scrape_configs:
+ - job_name: 'prometheus'
+ static_configs:
+ - targets: ['localhost:5555']
+
+
+ - job_name: 'exporter'
+ static_configs:
+ - targets: ['localhost:5557']
+```
+更改配置文件后，重启即可。
+
+5 Grafana dashboard for RocketMQ
+
+Prometheus自身的指标展示平台没有当前流行的展示平台Grafana好， 为了更好的展示RocketMQ的指标，可以使用Grafana来展示Prometheus获取的指标。首先到官网去下载[https://grafana.com/grafana/download](https://grafana.com/grafana/download), 这里仍以二进制文件安装为例进行介绍。
+
+```
+wget https://dl.grafana.com/oss/release/grafana-6.2.5.linux-amd64.tar.gz
+tar -zxvf grafana-6.2.5.linux-amd64.tar.gz
+cd grafana-5.4.3/
+```
+同样为了不与其它进程的端口冲突，可以修改conf目录下的defaults.ini文件的监听端口，当前将这个grafana的监听端口改为55555，然后使用如下的命令启动即可
+
+```
+./bin/grafana-server web
+```
+然后通过浏览器访问http://<服务器IP地址>:55555,就可以验证grafana是否已成功安装。系统默认用户名和密码为admin/admin，第一次登陆系统会要求修改密码，修改密码后登陆。为了便于用户使用，当前已将RocketMQ的dashboard配置文件上传到Grafana的官网，地址为[https://grafana.com/dashboards/10477/revisions](https://grafana.com/dashboards/10477/revisions)去下载dashboard配置文件，用户只需要到下载配置文件，并将配置文件导入dashboard即可创建RocketMQ的dashboard
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
[rocketmq-exporter] 36/43: Polish the doc
Posted by wl...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
wlliqipeng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/rocketmq-exporter.git
commit 86f9313b9e47b7d169b6d89626ef101b19156686
Author: vongosling <vo...@apache.org>
AuthorDate: Mon Jul 22 20:19:11 2019 +0800
Polish the doc
---
README.md | 4 ++--
rocketmq_exporter_quickstart.md | 6 +++---
2 files changed, 5 insertions(+), 5 deletions(-)
diff --git a/README.md b/README.md
index 463d366..2b87f1e 100644
--- a/README.md
+++ b/README.md
@@ -29,7 +29,7 @@ Support [Apache RocketMQ](https://rocketmq.apache.org) version 4.3.2 (and later)
Configuration
---
-This image is configurable using different properties, See ``application.properties`` for a configuration example.
+This image is configurable using different properties, see ``application.properties`` for a configuration example.
| name | Default | Description |
| -----------------------------------|--------------------|----------------------------------------------------|
@@ -208,4 +208,4 @@ For details of the dashboard please see [RocketMQ Exporter Overview](https://gra
Quick Start
-------------
-This [guide]((./rocketmq_exporter_quickstart.md)) will teach you how to build and run rocketmq exporter from scratch.
+This [guide](./rocketmq_exporter_quickstart.md) will teach you how to build and run rocketmq exporter with Prometheus and Grafana Integration from scratch.
diff --git a/rocketmq_exporter_quickstart.md b/rocketmq_exporter_quickstart.md
index 4ba2863..2f218fe 100644
--- a/rocketmq_exporter_quickstart.md
+++ b/rocketmq_exporter_quickstart.md
@@ -7,7 +7,7 @@ To use RocketMQ Exporter, first make sure the RocketMQ is downloaded and started
## Install Prometheus
-Firstly go to Prometheus official download address: https://prometheus.io/download/ to download the Prometheus installation package, currently using linux installation as an example, the selected installation package is Prometheus-2.7.0-rc.1.linux-amd64.tar.gz, the Prometheus process can be started after the following steps.
+Download [Prometheus installation package](https://prometheus.io/download/) and install it.
```
tar -xzf prometheus-2.7.0-rc.1.linux-amd64.tar.gz
@@ -18,7 +18,7 @@ cd prometheus-2.7.0-rc.1.linux-amd64/
The default listening port number of Prometheus is 9090. In order not conflict with other processes on the system, we reset the listening port number to 5555 in the startup parameters. Then go to website http:// sever ip:5555 through browser and users can verify whether the Prometheus has been successfully installed. Since the RocketMQ-Exporter process has been started, the data of RocketMQ-Exporter can be retrieved by Prometheus at this time. At this time, users only need to change th [...]
```
-# my global config
+# Global config
global:
scrape_interval: 15s # Set the scrape interval to every 15 seconds. Default is every 1 minute.
evaluation_interval: 15s # Evaluate rules every 15 seconds. The default is every 1 minute.
@@ -46,7 +46,7 @@ global:
## Create Grafana dashboard for RocketMQ
-Prometheus' own metric display platform is not as good as Grafana. In order to better show RocketMQ's metrics, Grafana can be used to show the metrics that Prometheus gets. Firstly go to the official website https://grafana.com/grafana/download to download installation file. Here is a an example for binary file installation.
+Prometheus' own metric display platform is not as good as Grafana. In order to better show RocketMQ's metrics, Grafana can be used to show the metrics that Prometheus gets. Download and install it as the following.
```
wget https://dl.grafana.com/oss/release/grafana-6.2.5.linux-amd64.tar.gz
[rocketmq-exporter] 33/43: Merge pull request #4 from
breezecoolyang/master
Posted by wl...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
wlliqipeng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/rocketmq-exporter.git
commit 0821128e0cf5050c1d31c78a790af86408c1534e
Merge: d5cb076 0415f4a
Author: von gosling <vo...@apache.org>
AuthorDate: Mon Jul 22 17:50:53 2019 +0800
Merge pull request #4 from breezecoolyang/master
Add Use example for RocketMQ Exporter
README.md | 425 ++++++++++++++++++++-------------------
rocketmq_exporter_use_example.md | 119 +++++++++++
2 files changed, 334 insertions(+), 210 deletions(-)
[rocketmq-exporter] 05/43: Update README.md
Posted by wl...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
wlliqipeng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/rocketmq-exporter.git
commit c936cfec50716f242981e7af00668da3d5adfa1d
Author: von gosling <vo...@apache.org>
AuthorDate: Mon Apr 8 11:36:31 2019 +0800
Update README.md
---
rocketmq-prometheus-exporter/README.md | 5 -----
1 file changed, 5 deletions(-)
diff --git a/rocketmq-prometheus-exporter/README.md b/rocketmq-prometheus-exporter/README.md
index 570cd33..c70f170 100644
--- a/rocketmq-prometheus-exporter/README.md
+++ b/rocketmq-prometheus-exporter/README.md
@@ -31,11 +31,6 @@ Dependency
- [Prometheus](https://prometheus.io)
-Download
---------
-
-source code can be downloaded from [github](https://github.com/hdchen/rocketmq-exporter ) page.
-
Compile
-------
[rocketmq-exporter] 09/43: add grafana dashboard for rocketmq
exporter
Posted by wl...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
wlliqipeng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/rocketmq-exporter.git
commit a39f97db5c44780872822767c7703997397d2933
Author: fengqing <fe...@sunlands.com>
AuthorDate: Fri Jun 21 16:28:37 2019 +0800
add grafana dashboard for rocketmq exporter
---
README.md | 6 +
rocketmq_exporter_overview.json | 777 ++++++++++++++++++++++++++++++++++++++++
2 files changed, 783 insertions(+)
diff --git a/README.md b/README.md
index c70f170..c9d3c42 100644
--- a/README.md
+++ b/README.md
@@ -19,6 +19,7 @@ Table of Contents
- [Brokers](#brokers)
- [Topics](#topics)
- [Consumer Groups](#consumer-groups)
+- [Grafana Dashboard](#Grafana Dashboard)
- [Contribute](#contribute)
Compatibility
@@ -202,3 +203,8 @@ rocketmq_group_get_latency_by_storetime{cluster="MQCluster",broker="broker-a",to
rocketmq_group_get_latency_by_storetime{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",} 3232.0
rocketmq_group_get_latency_by_storetime{cluster="MQCluster",broker="broker-b",topic="DEV_TID_tfq",group="DEV_CID_cfq",} 0.0
```
+
+Grafana Dashboard
+-------
+Grafana Dashboard ID: 10405, name: RocketMQ Exporter Overview.
+For details of the dashboard please see [RocketMQ Exporter Overview](https://grafana.com/dashboards/10405).
\ No newline at end of file
diff --git a/rocketmq_exporter_overview.json b/rocketmq_exporter_overview.json
new file mode 100644
index 0000000..537b21d
--- /dev/null
+++ b/rocketmq_exporter_overview.json
@@ -0,0 +1,777 @@
+{
+ "__inputs": [
+ {
+ "name": "DS_PROMETHEUS",
+ "label": "Prometheus",
+ "description": "",
+ "type": "datasource",
+ "pluginId": "prometheus",
+ "pluginName": "Prometheus"
+ }
+ ],
+ "__requires": [
+ {
+ "type": "grafana",
+ "id": "grafana",
+ "name": "Grafana",
+ "version": "5.4.3"
+ },
+ {
+ "type": "panel",
+ "id": "graph",
+ "name": "Graph",
+ "version": "5.0.0"
+ },
+ {
+ "type": "datasource",
+ "id": "prometheus",
+ "name": "Prometheus",
+ "version": "5.0.0"
+ }
+ ],
+ "annotations": {
+ "list": [
+ {
+ "builtIn": 1,
+ "datasource": "-- Grafana --",
+ "enable": true,
+ "hide": true,
+ "iconColor": "rgba(0, 211, 255, 1)",
+ "name": "Annotations & Alerts",
+ "type": "dashboard"
+ }
+ ]
+ },
+ "editable": true,
+ "gnetId": null,
+ "graphTooltip": 0,
+ "id": null,
+ "links": [],
+ "panels": [
+ {
+ "aliasColors": {},
+ "bars": false,
+ "dashLength": 10,
+ "dashes": false,
+ "fill": 1,
+ "gridPos": {
+ "h": 6,
+ "w": 7,
+ "x": 0,
+ "y": 0
+ },
+ "id": 12,
+ "legend": {
+ "avg": false,
+ "current": false,
+ "max": false,
+ "min": false,
+ "rightSide": true,
+ "show": false,
+ "total": false,
+ "values": false
+ },
+ "lines": true,
+ "linewidth": 1,
+ "links": [],
+ "nullPointMode": "null",
+ "percentage": false,
+ "pointradius": 5,
+ "points": false,
+ "renderer": "flot",
+ "seriesOverrides": [],
+ "spaceLength": 10,
+ "stack": false,
+ "steppedLine": false,
+ "targets": [
+ {
+ "expr": "rocketmq_broker_tps",
+ "format": "time_series",
+ "intervalFactor": 1,
+ "refId": "A"
+ },
+ {
+ "expr": "rocketmq_broker_qps",
+ "format": "time_series",
+ "intervalFactor": 1,
+ "refId": "B"
+ }
+ ],
+ "thresholds": [],
+ "timeFrom": null,
+ "timeRegions": [],
+ "timeShift": null,
+ "title": "broker tps & broker qps",
+ "tooltip": {
+ "shared": true,
+ "sort": 0,
+ "value_type": "individual"
+ },
+ "type": "graph",
+ "xaxis": {
+ "buckets": null,
+ "mode": "time",
+ "name": null,
+ "show": true,
+ "values": []
+ },
+ "yaxes": [
+ {
+ "format": "short",
+ "label": null,
+ "logBase": 1,
+ "max": null,
+ "min": null,
+ "show": true
+ },
+ {
+ "format": "short",
+ "label": null,
+ "logBase": 1,
+ "max": null,
+ "min": null,
+ "show": true
+ }
+ ],
+ "yaxis": {
+ "align": false,
+ "alignLevel": null
+ }
+ },
+ {
+ "aliasColors": {},
+ "bars": false,
+ "dashLength": 10,
+ "dashes": false,
+ "fill": 1,
+ "gridPos": {
+ "h": 6,
+ "w": 9,
+ "x": 7,
+ "y": 0
+ },
+ "id": 8,
+ "legend": {
+ "avg": false,
+ "current": false,
+ "max": false,
+ "min": false,
+ "rightSide": true,
+ "show": false,
+ "total": false,
+ "values": false
+ },
+ "lines": true,
+ "linewidth": 1,
+ "links": [],
+ "nullPointMode": "null",
+ "percentage": false,
+ "pointradius": 5,
+ "points": false,
+ "renderer": "flot",
+ "seriesOverrides": [],
+ "spaceLength": 10,
+ "stack": false,
+ "steppedLine": false,
+ "targets": [
+ {
+ "expr": "rocketmq_consumer_offset",
+ "format": "time_series",
+ "intervalFactor": 1,
+ "refId": "A"
+ }
+ ],
+ "thresholds": [],
+ "timeFrom": null,
+ "timeRegions": [],
+ "timeShift": null,
+ "title": "rocketmq_consumer_offset",
+ "tooltip": {
+ "shared": true,
+ "sort": 0,
+ "value_type": "individual"
+ },
+ "type": "graph",
+ "xaxis": {
+ "buckets": null,
+ "mode": "time",
+ "name": null,
+ "show": true,
+ "values": []
+ },
+ "yaxes": [
+ {
+ "format": "short",
+ "label": null,
+ "logBase": 1,
+ "max": null,
+ "min": null,
+ "show": true
+ },
+ {
+ "format": "short",
+ "label": null,
+ "logBase": 1,
+ "max": null,
+ "min": null,
+ "show": true
+ }
+ ],
+ "yaxis": {
+ "align": false,
+ "alignLevel": null
+ }
+ },
+ {
+ "aliasColors": {},
+ "bars": false,
+ "dashLength": 10,
+ "dashes": false,
+ "fill": 1,
+ "gridPos": {
+ "h": 6,
+ "w": 8,
+ "x": 16,
+ "y": 0
+ },
+ "id": 16,
+ "legend": {
+ "avg": false,
+ "current": false,
+ "max": false,
+ "min": false,
+ "rightSide": true,
+ "show": false,
+ "total": false,
+ "values": false
+ },
+ "lines": true,
+ "linewidth": 1,
+ "links": [],
+ "nullPointMode": "null",
+ "percentage": false,
+ "pointradius": 5,
+ "points": false,
+ "renderer": "flot",
+ "seriesOverrides": [],
+ "spaceLength": 10,
+ "stack": false,
+ "steppedLine": false,
+ "targets": [
+ {
+ "expr": "rocketmq_consumer_get_size",
+ "format": "time_series",
+ "intervalFactor": 1,
+ "refId": "A"
+ }
+ ],
+ "thresholds": [],
+ "timeFrom": null,
+ "timeRegions": [],
+ "timeShift": null,
+ "title": "rocketmq_consumer_get_size",
+ "tooltip": {
+ "shared": true,
+ "sort": 0,
+ "value_type": "individual"
+ },
+ "type": "graph",
+ "xaxis": {
+ "buckets": null,
+ "mode": "time",
+ "name": null,
+ "show": true,
+ "values": []
+ },
+ "yaxes": [
+ {
+ "format": "short",
+ "label": null,
+ "logBase": 1,
+ "max": null,
+ "min": null,
+ "show": true
+ },
+ {
+ "format": "short",
+ "label": null,
+ "logBase": 1,
+ "max": null,
+ "min": null,
+ "show": true
+ }
+ ],
+ "yaxis": {
+ "align": false,
+ "alignLevel": null
+ }
+ },
+ {
+ "aliasColors": {},
+ "bars": false,
+ "dashLength": 10,
+ "dashes": false,
+ "decimals": null,
+ "fill": 1,
+ "gridPos": {
+ "h": 7,
+ "w": 7,
+ "x": 0,
+ "y": 6
+ },
+ "id": 6,
+ "legend": {
+ "avg": false,
+ "current": false,
+ "max": false,
+ "min": false,
+ "rightSide": true,
+ "show": false,
+ "total": false,
+ "values": false
+ },
+ "lines": true,
+ "linewidth": 1,
+ "links": [],
+ "nullPointMode": "null",
+ "percentage": false,
+ "pointradius": 5,
+ "points": false,
+ "renderer": "flot",
+ "seriesOverrides": [],
+ "spaceLength": 10,
+ "stack": false,
+ "steppedLine": false,
+ "targets": [
+ {
+ "expr": "rocketmq_producer_offset",
+ "format": "time_series",
+ "intervalFactor": 1,
+ "refId": "A"
+ }
+ ],
+ "thresholds": [],
+ "timeFrom": null,
+ "timeRegions": [],
+ "timeShift": null,
+ "title": "rocketmq_producer_offset",
+ "tooltip": {
+ "shared": true,
+ "sort": 0,
+ "value_type": "individual"
+ },
+ "type": "graph",
+ "xaxis": {
+ "buckets": null,
+ "mode": "time",
+ "name": null,
+ "show": true,
+ "values": []
+ },
+ "yaxes": [
+ {
+ "format": "short",
+ "label": null,
+ "logBase": 1,
+ "max": null,
+ "min": null,
+ "show": true
+ },
+ {
+ "format": "short",
+ "label": null,
+ "logBase": 1,
+ "max": null,
+ "min": null,
+ "show": true
+ }
+ ],
+ "yaxis": {
+ "align": false,
+ "alignLevel": null
+ }
+ },
+ {
+ "aliasColors": {},
+ "bars": false,
+ "dashLength": 10,
+ "dashes": false,
+ "datasource": "${DS_PROMETHEUS}",
+ "description": "消费tps",
+ "fill": 1,
+ "gridPos": {
+ "h": 7,
+ "w": 9,
+ "x": 7,
+ "y": 6
+ },
+ "id": 4,
+ "legend": {
+ "avg": false,
+ "current": false,
+ "max": false,
+ "min": false,
+ "rightSide": true,
+ "show": false,
+ "total": false,
+ "values": false
+ },
+ "lines": true,
+ "linewidth": 1,
+ "links": [],
+ "nullPointMode": "null",
+ "percentage": false,
+ "pointradius": 5,
+ "points": false,
+ "renderer": "flot",
+ "seriesOverrides": [],
+ "spaceLength": 10,
+ "stack": false,
+ "steppedLine": false,
+ "targets": [
+ {
+ "expr": "rocketmq_consumer_tps",
+ "format": "time_series",
+ "intervalFactor": 1,
+ "refId": "A"
+ }
+ ],
+ "thresholds": [],
+ "timeFrom": null,
+ "timeRegions": [],
+ "timeShift": null,
+ "title": "rocketmq_consumer_tps",
+ "tooltip": {
+ "shared": true,
+ "sort": 0,
+ "value_type": "individual"
+ },
+ "type": "graph",
+ "xaxis": {
+ "buckets": null,
+ "mode": "time",
+ "name": null,
+ "show": true,
+ "values": []
+ },
+ "yaxes": [
+ {
+ "format": "short",
+ "label": null,
+ "logBase": 1,
+ "max": null,
+ "min": null,
+ "show": true
+ },
+ {
+ "format": "short",
+ "label": null,
+ "logBase": 1,
+ "max": null,
+ "min": null,
+ "show": true
+ }
+ ],
+ "yaxis": {
+ "align": false,
+ "alignLevel": null
+ }
+ },
+ {
+ "aliasColors": {},
+ "bars": false,
+ "dashLength": 10,
+ "dashes": false,
+ "fill": 1,
+ "gridPos": {
+ "h": 7,
+ "w": 8,
+ "x": 16,
+ "y": 6
+ },
+ "id": 14,
+ "legend": {
+ "avg": false,
+ "current": false,
+ "max": false,
+ "min": false,
+ "rightSide": true,
+ "show": false,
+ "total": false,
+ "values": false
+ },
+ "lines": true,
+ "linewidth": 1,
+ "links": [],
+ "nullPointMode": "null",
+ "percentage": false,
+ "pointradius": 5,
+ "points": false,
+ "renderer": "flot",
+ "seriesOverrides": [],
+ "spaceLength": 10,
+ "stack": false,
+ "steppedLine": false,
+ "targets": [
+ {
+ "expr": "rocketmq_producer_put_size",
+ "format": "time_series",
+ "intervalFactor": 1,
+ "refId": "A"
+ }
+ ],
+ "thresholds": [],
+ "timeFrom": null,
+ "timeRegions": [],
+ "timeShift": null,
+ "title": "rocketmq_producer_put_size",
+ "tooltip": {
+ "shared": true,
+ "sort": 0,
+ "value_type": "individual"
+ },
+ "type": "graph",
+ "xaxis": {
+ "buckets": null,
+ "mode": "time",
+ "name": null,
+ "show": true,
+ "values": []
+ },
+ "yaxes": [
+ {
+ "format": "short",
+ "label": null,
+ "logBase": 1,
+ "max": null,
+ "min": null,
+ "show": true
+ },
+ {
+ "format": "short",
+ "label": null,
+ "logBase": 1,
+ "max": null,
+ "min": null,
+ "show": true
+ }
+ ],
+ "yaxis": {
+ "align": false,
+ "alignLevel": null
+ }
+ },
+ {
+ "aliasColors": {},
+ "bars": false,
+ "dashLength": 10,
+ "dashes": false,
+ "datasource": "${DS_PROMETHEUS}",
+ "fill": 1,
+ "gridPos": {
+ "h": 7,
+ "w": 7,
+ "x": 0,
+ "y": 13
+ },
+ "id": 2,
+ "legend": {
+ "alignAsTable": false,
+ "avg": false,
+ "current": false,
+ "max": false,
+ "min": false,
+ "rightSide": true,
+ "show": false,
+ "total": false,
+ "values": false
+ },
+ "lines": true,
+ "linewidth": 1,
+ "links": [],
+ "nullPointMode": "null",
+ "percentage": false,
+ "pointradius": 5,
+ "points": false,
+ "renderer": "flot",
+ "seriesOverrides": [],
+ "spaceLength": 10,
+ "stack": false,
+ "steppedLine": false,
+ "targets": [
+ {
+ "expr": "rocketmq_producer_tps",
+ "format": "time_series",
+ "hide": false,
+ "instant": false,
+ "intervalFactor": 1,
+ "refId": "A"
+ }
+ ],
+ "thresholds": [],
+ "timeFrom": null,
+ "timeRegions": [],
+ "timeShift": null,
+ "title": "rocketmq_producer_tps",
+ "tooltip": {
+ "shared": true,
+ "sort": 0,
+ "value_type": "individual"
+ },
+ "type": "graph",
+ "xaxis": {
+ "buckets": null,
+ "mode": "time",
+ "name": null,
+ "show": true,
+ "values": []
+ },
+ "yaxes": [
+ {
+ "format": "short",
+ "label": null,
+ "logBase": 1,
+ "max": null,
+ "min": null,
+ "show": true
+ },
+ {
+ "format": "short",
+ "label": null,
+ "logBase": 1,
+ "max": null,
+ "min": null,
+ "show": true
+ }
+ ],
+ "yaxis": {
+ "align": false,
+ "alignLevel": null
+ }
+ },
+ {
+ "aliasColors": {},
+ "bars": false,
+ "dashLength": 10,
+ "dashes": false,
+ "fill": 1,
+ "gridPos": {
+ "h": 7,
+ "w": 9,
+ "x": 7,
+ "y": 13
+ },
+ "id": 10,
+ "legend": {
+ "avg": false,
+ "current": false,
+ "max": false,
+ "min": false,
+ "rightSide": true,
+ "show": false,
+ "total": false,
+ "values": false
+ },
+ "lines": true,
+ "linewidth": 1,
+ "links": [],
+ "nullPointMode": "null",
+ "percentage": false,
+ "pointradius": 5,
+ "points": false,
+ "renderer": "flot",
+ "seriesOverrides": [],
+ "spaceLength": 10,
+ "stack": false,
+ "steppedLine": false,
+ "targets": [
+ {
+ "expr": "rocketmq_group_get_latency_by_storetime",
+ "format": "time_series",
+ "intervalFactor": 1,
+ "refId": "A"
+ }
+ ],
+ "thresholds": [],
+ "timeFrom": null,
+ "timeRegions": [],
+ "timeShift": null,
+ "title": "rocketmq_group_get_latency_by_storetime",
+ "tooltip": {
+ "shared": true,
+ "sort": 0,
+ "value_type": "individual"
+ },
+ "type": "graph",
+ "xaxis": {
+ "buckets": null,
+ "mode": "time",
+ "name": null,
+ "show": true,
+ "values": []
+ },
+ "yaxes": [
+ {
+ "format": "short",
+ "label": null,
+ "logBase": 1,
+ "max": null,
+ "min": null,
+ "show": true
+ },
+ {
+ "format": "short",
+ "label": null,
+ "logBase": 1,
+ "max": null,
+ "min": null,
+ "show": true
+ }
+ ],
+ "yaxis": {
+ "align": false,
+ "alignLevel": null
+ }
+ }
+ ],
+ "refresh": false,
+ "schemaVersion": 16,
+ "style": "dark",
+ "tags": [],
+ "templating": {
+ "list": []
+ },
+ "time": {
+ "from": "now-1h",
+ "to": "now"
+ },
+ "timepicker": {
+ "refresh_intervals": [
+ "5s",
+ "10s",
+ "30s",
+ "1m",
+ "5m",
+ "15m",
+ "30m",
+ "1h",
+ "2h",
+ "1d"
+ ],
+ "time_options": [
+ "5m",
+ "15m",
+ "1h",
+ "6h",
+ "12h",
+ "24h",
+ "2d",
+ "7d",
+ "30d"
+ ]
+ },
+ "timezone": "",
+ "title": "Rocketmq_dashboard",
+ "uid": "zkVx1w_iz",
+ "version": 11
+}
\ No newline at end of file
[rocketmq-exporter] 20/43: Polish
Posted by wl...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
wlliqipeng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/rocketmq-exporter.git
commit 026437c1de496fa0d6a812877bec332d598ba2a3
Author: vongosling <vo...@apache.org>
AuthorDate: Fri Jul 5 16:14:15 2019 +0800
Polish
---
README.md | 3 +--
1 file changed, 1 insertion(+), 2 deletions(-)
diff --git a/README.md b/README.md
index 531878e..5313655 100644
--- a/README.md
+++ b/README.md
@@ -20,7 +20,6 @@ Table of Contents
- [Topics](#topics)
- [Consumer Groups](#consumer-groups)
- [Grafana Dashboard](#Grafana-Dashboard)
-- [Contribute](#contribute)
Compatibility
-------------
@@ -160,7 +159,7 @@ rocketmq_producer_offset{cluster="MQCluster",broker="broker-a",topic="DEV_TID_20
| `rocketmq_consumer_offset` | Progress of consumption message for a consumer group |
| `rocketmq_group_get_latency` | Consumer latency on some topic for one queue |
| `rocketmq_group_get_latency_by_storetime ` | Consumption delay time of a consumer group |
-| 消息堆积量（需要通过proSQL聚合）（rocketmq_producer_offset-rocketmq_consumer_offset） | Message accumulation (sum(rocketmq_producer_offset) by (topic) - on(topic) group_right sum(rocketmq_consumer_offset) by (group,topic)) |
+| `rocketmq_message_accumulation`| How far Consumer offset lag behind |
**Metrics output example**
[rocketmq-exporter] 07/43: Avoid crash when get consumer offsets as
a group of topic consuming
Posted by wl...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
wlliqipeng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/rocketmq-exporter.git
commit 378184de875742e83cbfc1e92eda98d25c85ef26
Author: breezecoolyang <br...@users.noreply.github.com>
AuthorDate: Mon Jun 3 10:47:05 2019 +0800
Avoid crash when get consumer offsets as a group of topic consuming
---
.../rocketmq/exporter/task/MetricsCollectTask.java | 23 ++++++++++++----------
1 file changed, 13 insertions(+), 10 deletions(-)
diff --git a/rocketmq-prometheus-exporter/src/main/java/org/apache/rocketmq/exporter/task/MetricsCollectTask.java b/rocketmq-prometheus-exporter/src/main/java/org/apache/rocketmq/exporter/task/MetricsCollectTask.java
index 3f359e9..6ae442d 100644
--- a/rocketmq-prometheus-exporter/src/main/java/org/apache/rocketmq/exporter/task/MetricsCollectTask.java
+++ b/rocketmq-prometheus-exporter/src/main/java/org/apache/rocketmq/exporter/task/MetricsCollectTask.java
@@ -106,17 +106,20 @@ public class MetricsCollectTask {
GroupList groupList = mqAdminExt.queryTopicConsumeByWho(topic);
if (groupList != null && !groupList.getGroupList().isEmpty()) {
for (String group : groupList.getGroupList()) {
- ConsumeStats consumeStatus = mqAdminExt.examineConsumeStats(group,topic);
- Set<Map.Entry<MessageQueue, OffsetWrapper>> consumeStatusEntries = consumeStatus.getOffsetTable().entrySet();
- for (Map.Entry<MessageQueue, OffsetWrapper> consumeStatusEntry : consumeStatusEntries) {
- MessageQueue q = consumeStatusEntry.getKey();
- OffsetWrapper offset = consumeStatusEntry.getValue();
- if (consumeOffsetMap.containsKey(q.getBrokerName())) {
- consumeOffsetMap.put(q.getBrokerName(), consumeOffsetMap.get(q.getBrokerName()) + offset.getConsumerOffset());
- }
- else {
- consumeOffsetMap.put(q.getBrokerName(), offset.getConsumerOffset());
+ try {
+ ConsumeStats consumeStatus = mqAdminExt.examineConsumeStats(group, topic);
+ Set<Map.Entry<MessageQueue, OffsetWrapper>> consumeStatusEntries = consumeStatus.getOffsetTable().entrySet();
+ for (Map.Entry<MessageQueue, OffsetWrapper> consumeStatusEntry : consumeStatusEntries) {
+ MessageQueue q = consumeStatusEntry.getKey();
+ OffsetWrapper offset = consumeStatusEntry.getValue();
+ if (consumeOffsetMap.containsKey(q.getBrokerName())) {
+ consumeOffsetMap.put(q.getBrokerName(), consumeOffsetMap.get(q.getBrokerName()) + offset.getConsumerOffset());
+ } else {
+ consumeOffsetMap.put(q.getBrokerName(), offset.getConsumerOffset());
+ }
}
+ } catch (Exception e) {
+ log.info("ignore this consumer", e.getMessage());
}
Set<Map.Entry<String, Long>> consumeOffsetEntries = consumeOffsetMap.entrySet();
for (Map.Entry<String, Long> consumeOffsetEntry : consumeOffsetEntries) {
[rocketmq-exporter] 13/43: make metrics value integer if the value
is greater than one
Posted by wl...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
wlliqipeng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/rocketmq-exporter.git
commit bd58583749e3bfbccc6cae128a6c55a09d2774cc
Author: fengqing <fe...@sunlands.com>
AuthorDate: Tue Jul 2 20:45:24 2019 +0800
make metrics value integer if the value is greater than one
---
README.md | 24 ++++++++++++------------
1 file changed, 12 insertions(+), 12 deletions(-)
diff --git a/README.md b/README.md
index 9e04be5..b61319d 100644
--- a/README.md
+++ b/README.md
@@ -93,12 +93,12 @@ Documents about exposed Prometheus metrics.
```txt
# HELP rocketmq_broker_tps BrokerPutNums
# TYPE rocketmq_broker_tps gauge
-rocketmq_broker_tps{cluster="MQCluster",broker="broker-a",} 7.93
-rocketmq_broker_tps{cluster="MQCluster",broker="broker-b",} 7.91
+rocketmq_broker_tps{cluster="MQCluster",broker="broker-a",} 7.0
+rocketmq_broker_tps{cluster="MQCluster",broker="broker-b",} 7.0
# HELP rocketmq_broker_qps BrokerGetNums
# TYPE rocketmq_broker_qps gauge
-rocketmq_broker_qps{cluster="MQCluster",broker="broker-a",} 8.2
-rocketmq_broker_qps{cluster="MQCluster",broker="broker-b",} 8.15
+rocketmq_broker_qps{cluster="MQCluster",broker="broker-a",} 8.0
+rocketmq_broker_qps{cluster="MQCluster",broker="broker-b",} 8.0
```
### Topics
@@ -116,12 +116,12 @@ rocketmq_broker_qps{cluster="MQCluster",broker="broker-b",} 8.15
```txt
# HELP rocketmq_producer_tps TopicPutNums
# TYPE rocketmq_producer_tps gauge
-rocketmq_producer_tps{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",} 7.93
-rocketmq_producer_tps{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",} 7.91
+rocketmq_producer_tps{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",} 7.0
+rocketmq_producer_tps{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",} 7.0
# HELP rocketmq_producer_message_size TopicPutMessageSize
# TYPE rocketmq_producer_message_size gauge
-rocketmq_producer_message_size{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",} 1642.2
-rocketmq_producer_message_size{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",} 1638.75
+rocketmq_producer_message_size{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",} 1642.0
+rocketmq_producer_message_size{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",} 1638.0
# HELP rocketmq_producer_offset TopicOffset
# TYPE rocketmq_producer_offset counter
rocketmq_producer_offset{cluster="MQCluster",broker="broker-a",topic="TBW102",} 0.0
@@ -166,12 +166,12 @@ rocketmq_producer_offset{cluster="MQCluster",broker="broker-a",topic="DEV_TID_20
```txt
# HELP rocketmq_consumer_tps GroupGetNums
# TYPE rocketmq_consumer_tps gauge
-rocketmq_consumer_tps{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",} 7.91
-rocketmq_consumer_tps{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",} 7.93
+rocketmq_consumer_tps{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",} 7.0
+rocketmq_consumer_tps{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",} 7.0
# HELP rocketmq_consumer_message_size GroupGetMessageSize
# TYPE rocketmq_consumer_message_size gauge
-rocketmq_consumer_message_size{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",} 1638.75
-rocketmq_consumer_message_size{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",} 1642.2
+rocketmq_consumer_message_size{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",} 1638.0
+rocketmq_consumer_message_size{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",} 1642.0
# HELP rocketmq_consumer_offset GroupOffset
# TYPE rocketmq_consumer_offset counter
rocketmq_consumer_offset{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",} 1462030.0
[rocketmq-exporter] 03/43: RocketMQ Exporter (#241)
Posted by wl...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
wlliqipeng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/rocketmq-exporter.git
commit 022794dfcda453d52f534ff768ef3aa1baacc744
Author: hdchen <14...@qq.com>
AuthorDate: Thu Apr 4 21:59:12 2019 +0800
RocketMQ Exporter (#241)
* First commit
---
rocketmq-prometheus-exporter/.gitignore | 13 +
rocketmq-prometheus-exporter/LICENSE | 201 ++++++++
rocketmq-prometheus-exporter/README.md | 211 ++++++++-
rocketmq-prometheus-exporter/example.rules | 67 +++
rocketmq-prometheus-exporter/pom.xml | 137 ++++++
.../src/main/docker/Dockerfile | 5 +
.../exporter/RocketMQExporterApplication.java | 35 ++
.../exporter/aspect/admin/MQAdminAspect.java | 71 +++
.../admin/annotation/MultiMQAdminCmdMethod.java | 30 ++
.../exporter/collector/RMQMetricsCollector.java | 190 ++++++++
.../rocketmq/exporter/config/RMQConfigure.java | 102 ++++
.../exporter/controller/RMQMetricsController.java | 52 +++
.../exporter/exception/ServiceException.java | 31 ++
.../exporter/model/metrics/BrokerMetric.java | 70 +++
.../exporter/model/metrics/ConsumerMetric.java | 86 ++++
.../model/metrics/ConsumerQueueMetric.java | 93 ++++
.../exporter/model/metrics/ProducerMetric.java | 73 +++
.../exporter/service/AbstractCommonService.java | 50 ++
.../exporter/service/RMQMetricsService.java | 29 ++
.../exporter/service/client/MQAdminExtImpl.java | 515 +++++++++++++++++++++
.../exporter/service/client/MQAdminInstance.java | 119 +++++
.../service/impl/RMQMetricsServiceImpl.java | 53 +++
.../rocketmq/exporter/task/MetricsCollectTask.java | 300 ++++++++++++
.../apache/rocketmq/exporter/util/JsonUtil.java | 157 +++++++
.../src/main/resources/application.properties | 14 +
.../src/main/resources/logback.xml | 33 ++
.../style/copyright/Apache.xml | 23 +
.../style/copyright/profiles_settings.xml | 64 +++
.../style/rmq_checkstyle.xml | 135 ++++++
.../style/rmq_codeStyle.xml | 157 +++++++
30 files changed, 3114 insertions(+), 2 deletions(-)
diff --git a/rocketmq-prometheus-exporter/.gitignore b/rocketmq-prometheus-exporter/.gitignore
new file mode 100644
index 0000000..06bb326
--- /dev/null
+++ b/rocketmq-prometheus-exporter/.gitignore
@@ -0,0 +1,13 @@
+.idea
+.classpath
+.project
+.settings/
+target/
+*.log*
+*.iml
+.idea/
+*.versionsBackup
+!NOTICE-BIN
+!LICENSE-BIN
+.DS_Store
+.vscode
diff --git a/rocketmq-prometheus-exporter/LICENSE b/rocketmq-prometheus-exporter/LICENSE
new file mode 100644
index 0000000..261eeb9
--- /dev/null
+++ b/rocketmq-prometheus-exporter/LICENSE
@@ -0,0 +1,201 @@
+ Apache License
+ Version 2.0, January 2004
+ http://www.apache.org/licenses/
+
+ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+ 1. Definitions.
+
+ "License" shall mean the terms and conditions for use, reproduction,
+ and distribution as defined by Sections 1 through 9 of this document.
+
+ "Licensor" shall mean the copyright owner or entity authorized by
+ the copyright owner that is granting the License.
+
+ "Legal Entity" shall mean the union of the acting entity and all
+ other entities that control, are controlled by, or are under common
+ control with that entity. For the purposes of this definition,
+ "control" means (i) the power, direct or indirect, to cause the
+ direction or management of such entity, whether by contract or
+ otherwise, or (ii) ownership of fifty percent (50%) or more of the
+ outstanding shares, or (iii) beneficial ownership of such entity.
+
+ "You" (or "Your") shall mean an individual or Legal Entity
+ exercising permissions granted by this License.
+
+ "Source" form shall mean the preferred form for making modifications,
+ including but not limited to software source code, documentation
+ source, and configuration files.
+
+ "Object" form shall mean any form resulting from mechanical
+ transformation or translation of a Source form, including but
+ not limited to compiled object code, generated documentation,
+ and conversions to other media types.
+
+ "Work" shall mean the work of authorship, whether in Source or
+ Object form, made available under the License, as indicated by a
+ copyright notice that is included in or attached to the work
+ (an example is provided in the Appendix below).
+
+ "Derivative Works" shall mean any work, whether in Source or Object
+ form, that is based on (or derived from) the Work and for which the
+ editorial revisions, annotations, elaborations, or other modifications
+ represent, as a whole, an original work of authorship. For the purposes
+ of this License, Derivative Works shall not include works that remain
+ separable from, or merely link (or bind by name) to the interfaces of,
+ the Work and Derivative Works thereof.
+
+ "Contribution" shall mean any work of authorship, including
+ the original version of the Work and any modifications or additions
+ to that Work or Derivative Works thereof, that is intentionally
+ submitted to Licensor for inclusion in the Work by the copyright owner
+ or by an individual or Legal Entity authorized to submit on behalf of
+ the copyright owner. For the purposes of this definition, "submitted"
+ means any form of electronic, verbal, or written communication sent
+ to the Licensor or its representatives, including but not limited to
+ communication on electronic mailing lists, source code control systems,
+ and issue tracking systems that are managed by, or on behalf of, the
+ Licensor for the purpose of discussing and improving the Work, but
+ excluding communication that is conspicuously marked or otherwise
+ designated in writing by the copyright owner as "Not a Contribution."
+
+ "Contributor" shall mean Licensor and any individual or Legal Entity
+ on behalf of whom a Contribution has been received by Licensor and
+ subsequently incorporated within the Work.
+
+ 2. Grant of Copyright License. Subject to the terms and conditions of
+ this License, each Contributor hereby grants to You a perpetual,
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+ copyright license to reproduce, prepare Derivative Works of,
+ publicly display, publicly perform, sublicense, and distribute the
+ Work and such Derivative Works in Source or Object form.
+
+ 3. Grant of Patent License. Subject to the terms and conditions of
+ this License, each Contributor hereby grants to You a perpetual,
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+ (except as stated in this section) patent license to make, have made,
+ use, offer to sell, sell, import, and otherwise transfer the Work,
+ where such license applies only to those patent claims licensable
+ by such Contributor that are necessarily infringed by their
+ Contribution(s) alone or by combination of their Contribution(s)
+ with the Work to which such Contribution(s) was submitted. If You
+ institute patent litigation against any entity (including a
+ cross-claim or counterclaim in a lawsuit) alleging that the Work
+ or a Contribution incorporated within the Work constitutes direct
+ or contributory patent infringement, then any patent licenses
+ granted to You under this License for that Work shall terminate
+ as of the date such litigation is filed.
+
+ 4. Redistribution. You may reproduce and distribute copies of the
+ Work or Derivative Works thereof in any medium, with or without
+ modifications, and in Source or Object form, provided that You
+ meet the following conditions:
+
+ (a) You must give any other recipients of the Work or
+ Derivative Works a copy of this License; and
+
+ (b) You must cause any modified files to carry prominent notices
+ stating that You changed the files; and
+
+ (c) You must retain, in the Source form of any Derivative Works
+ that You distribute, all copyright, patent, trademark, and
+ attribution notices from the Source form of the Work,
+ excluding those notices that do not pertain to any part of
+ the Derivative Works; and
+
+ (d) If the Work includes a "NOTICE" text file as part of its
+ distribution, then any Derivative Works that You distribute must
+ include a readable copy of the attribution notices contained
+ within such NOTICE file, excluding those notices that do not
+ pertain to any part of the Derivative Works, in at least one
+ of the following places: within a NOTICE text file distributed
+ as part of the Derivative Works; within the Source form or
+ documentation, if provided along with the Derivative Works; or,
+ within a display generated by the Derivative Works, if and
+ wherever such third-party notices normally appear. The contents
+ of the NOTICE file are for informational purposes only and
+ do not modify the License. You may add Your own attribution
+ notices within Derivative Works that You distribute, alongside
+ or as an addendum to the NOTICE text from the Work, provided
+ that such additional attribution notices cannot be construed
+ as modifying the License.
+
+ You may add Your own copyright statement to Your modifications and
+ may provide additional or different license terms and conditions
+ for use, reproduction, or distribution of Your modifications, or
+ for any such Derivative Works as a whole, provided Your use,
+ reproduction, and distribution of the Work otherwise complies with
+ the conditions stated in this License.
+
+ 5. Submission of Contributions. Unless You explicitly state otherwise,
+ any Contribution intentionally submitted for inclusion in the Work
+ by You to the Licensor shall be under the terms and conditions of
+ this License, without any additional terms or conditions.
+ Notwithstanding the above, nothing herein shall supersede or modify
+ the terms of any separate license agreement you may have executed
+ with Licensor regarding such Contributions.
+
+ 6. Trademarks. This License does not grant permission to use the trade
+ names, trademarks, service marks, or product names of the Licensor,
+ except as required for reasonable and customary use in describing the
+ origin of the Work and reproducing the content of the NOTICE file.
+
+ 7. Disclaimer of Warranty. Unless required by applicable law or
+ agreed to in writing, Licensor provides the Work (and each
+ Contributor provides its Contributions) on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+ implied, including, without limitation, any warranties or conditions
+ of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+ PARTICULAR PURPOSE. You are solely responsible for determining the
+ appropriateness of using or redistributing the Work and assume any
+ risks associated with Your exercise of permissions under this License.
+
+ 8. Limitation of Liability. In no event and under no legal theory,
+ whether in tort (including negligence), contract, or otherwise,
+ unless required by applicable law (such as deliberate and grossly
+ negligent acts) or agreed to in writing, shall any Contributor be
+ liable to You for damages, including any direct, indirect, special,
+ incidental, or consequential damages of any character arising as a
+ result of this License or out of the use or inability to use the
+ Work (including but not limited to damages for loss of goodwill,
+ work stoppage, computer failure or malfunction, or any and all
+ other commercial damages or losses), even if such Contributor
+ has been advised of the possibility of such damages.
+
+ 9. Accepting Warranty or Additional Liability. While redistributing
+ the Work or Derivative Works thereof, You may choose to offer,
+ and charge a fee for, acceptance of support, warranty, indemnity,
+ or other liability obligations and/or rights consistent with this
+ License. However, in accepting such obligations, You may act only
+ on Your own behalf and on Your sole responsibility, not on behalf
+ of any other Contributor, and only if You agree to indemnify,
+ defend, and hold each Contributor harmless for any liability
+ incurred by, or claims asserted against, such Contributor by reason
+ of your accepting any such warranty or additional liability.
+
+ END OF TERMS AND CONDITIONS
+
+ APPENDIX: How to apply the Apache License to your work.
+
+ To apply the Apache License to your work, attach the following
+ boilerplate notice, with the fields enclosed by brackets "[]"
+ replaced with your own identifying information. (Don't include
+ the brackets!) The text should be enclosed in the appropriate
+ comment syntax for the file format. We also recommend that a
+ file or class name and description of purpose be included on the
+ same "printed page" as the copyright notice for easier
+ identification within third-party archives.
+
+ Copyright [yyyy] [name of copyright owner]
+
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
diff --git a/rocketmq-prometheus-exporter/README.md b/rocketmq-prometheus-exporter/README.md
index a9dc2f8..268e445 100644
--- a/rocketmq-prometheus-exporter/README.md
+++ b/rocketmq-prometheus-exporter/README.md
@@ -1,5 +1,212 @@
-# RocketMQ-prometheus-exporter
+RocketMQ_exporter
+==============
-## Overview
+RocketMQ exporter for Prometheus.
+Table of Contents
+-----------------
+- [Compatibility](#compatibility)
+- [Dependency](#dependency)
+- [Download](#download)
+- [Compile](#compile)
+ - [Build Binary](#build-binary)
+ - [Build Docker Image](#build-docker-image)
+- [Run](#run)
+ - [Run Binary](#run-binary)
+ - [Run Docker Image](#run-docker-image)
+- [Flags](#flags)
+- [Metrics](#metrics)
+ - [Brokers](#brokers)
+ - [Topics](#topics)
+ - [Consumer Groups](#consumer-groups)
+- [Contribute](#contribute)
+Compatibility
+-------------
+
+Support [Apache RocketMQ](https://rocketmq.apache.org) version 4.3.2 (and later).
+
+Dependency
+----------
+
+- [Prometheus](https://prometheus.io)
+
+Download
+--------
+
+source code can be downloaded from [github](https://github.com/hdchen/rocketmq-exporter ) page.
+
+Compile
+-------
+
+### Build Binary
+
+```shell
+mvn clean install
+```
+
+### Build Docker Image
+
+```shell
+mvn package -Dmaven.test.skip=true docker:build
+```
+
+
+It can be used directly instead of having to build the image yourself. ([Docker Hub breezecoolyang/rocketmq-exporter](https://cloud.docker.com/repository/docker/breezecoolyang/rocketmq-exporter)\)
+
+Run
+---
+
+### Run Binary
+
+```shell
+java -jar rocketmq-exporter-0.0.1-SNAPSHOT.jar [--rocketmq.config.namesrvAddr="127.0.0.1:9876" ...]
+```
+
+### Run Docker Image
+
+```
+docker container run -itd --rm -p 5557:5557 breezecoolyang/rocketmq-exporter [--rocketmq.config.namesrvAddr="127.0.0.1:9876" ...]
+```
+
+Flags
+---
+
+This image is configurable using different flags
+
+|Flag name | Default | Description |
+| -----------------------------------|--------------------|----------------------------------------------------|
+| `rocketmq.config.namesrvAddr` | 127.0.0.1:9876 |name server address for broker cluster |
+| `rocketmq.config.webTelemetryPath` | /metrics |Path under which to expose metrics |
+| `server.port` | 5557 |Address to listen on for web interface and telemetry|
+| `rocketmq.config.rocketmqVersion` | V4_3_2 |rocketmq broker version |
+
+Metrics
+-------
+
+Documents about exposed Prometheus metrics.
+
+### Broker
+
+**Metrics details**
+
+| Name | Exposed information |
+| ------------ | ---------------------------------------------------- |
+| `rocketmq_broker_tps` | total put message numbers per second for this broker |
+| `rocketmq_broker_qps` | total get message numbers per second for this broker |
+
+**Metrics output example**
+
+```txt
+# HELP rocketmq_broker_tps BrokerPutNums
+# TYPE rocketmq_broker_tps gauge
+rocketmq_broker_tps{cluster="MQCluster",broker="broker-a",} 7.933333333333334
+rocketmq_broker_tps{cluster="MQCluster",broker="broker-b",} 7.916666666666667
+# HELP rocketmq_broker_qps BrokerGetNums
+# TYPE rocketmq_broker_qps gauge
+rocketmq_broker_qps{cluster="MQCluster",broker="broker-a",} 8.2
+rocketmq_broker_qps{cluster="MQCluster",broker="broker-b",} 8.15
+```
+
+### Topics
+
+**Metrics details**
+
+| Name | Exposed information |
+| ------------------- | -------------------------------------------------- |
+| `rocketmq_producer_tps` | sending messages number per second for this topic |
+| `rocketmq_producer_put_size` | sending messages size per second for this topic |
+| `rocketmq_producer_offset` | Current Offset of a Broker for this topic |
+
+**Metrics output example**
+
+```txt
+# HELP rocketmq_producer_tps TopicPutNums
+# TYPE rocketmq_producer_tps gauge
+rocketmq_producer_tps{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",} 7.933333333333334
+rocketmq_producer_tps{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",} 7.916666666666667
+# HELP rocketmq_producer_put_size TopicPutSize
+# TYPE rocketmq_producer_put_size gauge
+rocketmq_producer_put_size{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",} 1642.2
+rocketmq_producer_put_size{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",} 1638.75
+# HELP rocketmq_producer_offset TopicOffset
+# TYPE rocketmq_producer_offset counter
+rocketmq_producer_offset{cluster="MQCluster",broker="broker-a",topic="TBW102",} 0.0
+rocketmq_producer_offset{cluster="MQCluster",broker="broker-b",topic="DEV_TID_tfq",} 1878633.0
+rocketmq_producer_offset{cluster="MQCluster",broker="broker-a",topic="DEV_TID_tfq",} 3843787.0
+rocketmq_producer_offset{cluster="MQCluster",broker="broker-b",topic="DEV_TID_20190304",} 0.0
+rocketmq_producer_offset{cluster="MQCluster",broker="broker-a",topic="BenchmarkTest",} 0.0
+rocketmq_producer_offset{cluster="MQCluster",broker="broker-b",topic="DEV_TID_20190305",} 0.0
+rocketmq_producer_offset{cluster="MQCluster",broker="broker-b",topic="MQCluster",} 0.0
+rocketmq_producer_offset{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",} 2798195.0
+rocketmq_producer_offset{cluster="MQCluster",broker="broker-b",topic="BenchmarkTest",} 0.0
+rocketmq_producer_offset{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",} 1459666.0
+rocketmq_producer_offset{cluster="MQCluster",broker="broker-a",topic="MQCluster",} 0.0
+rocketmq_producer_offset{cluster="MQCluster",broker="broker-a",topic="SELF_TEST_TOPIC",} 0.0
+rocketmq_producer_offset{cluster="MQCluster",broker="broker-a",topic="OFFSET_MOVED_EVENT",} 0.0
+rocketmq_producer_offset{cluster="MQCluster",broker="broker-b",topic="broker-b",} 0.0
+rocketmq_producer_offset{cluster="MQCluster",broker="broker-a",topic="broker-a",} 0.0
+rocketmq_producer_offset{cluster="MQCluster",broker="broker-b",topic="SELF_TEST_TOPIC",} 0.0
+rocketmq_producer_offset{cluster="MQCluster",broker="broker-b",topic="RMQ_SYS_TRANS_HALF_TOPIC",} 0.0
+rocketmq_producer_offset{cluster="MQCluster",broker="broker-a",topic="DEV_TID_20190305",} 0.0
+rocketmq_producer_offset{cluster="MQCluster",broker="broker-b",topic="OFFSET_MOVED_EVENT",} 0.0
+rocketmq_producer_offset{cluster="MQCluster",broker="broker-a",topic="RMQ_SYS_TRANS_HALF_TOPIC",} 0.0
+rocketmq_producer_offset{cluster="MQCluster",broker="broker-b",topic="TBW102",} 0.0
+rocketmq_producer_offset{cluster="MQCluster",broker="broker-a",topic="DEV_TID_20190304",} 0.0
+
+```
+
+### Consumer Groups
+
+**Metrics details**
+
+| Name | Exposed information |
+| --------------------------------- | ------------------------------------------------------------ |
+| `rocketmq_consumer_tps` | consumer message numbers per second for this Topic |
+| `rocketmq_consumer_get_size` | consumer message size per second for this Topic |
+| `rocketmq_consumer_offset` | consumer offset for this topic |
+| `rocketmq_group_get_latency` | consumer latency on some topic for one queue |
+| `rocketmq_group_get_latency_by_storetime ` | consumer latency between message consume time and message store time on some topic |
+
+**Metrics output example**
+
+```txt
+# HELP rocketmq_consumer_tps GroupGetNums
+# TYPE rocketmq_consumer_tps gauge
+rocketmq_consumer_tps{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",} 7.916666666666667
+rocketmq_consumer_tps{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",} 7.933333333333334
+# HELP rocketmq_consumer_get_size GroupGetSize
+# TYPE rocketmq_consumer_get_size gauge
+rocketmq_consumer_get_size{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",} 1638.75
+rocketmq_consumer_get_size{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",} 1642.2
+# HELP rocketmq_consumer_offset GroupOffset
+# TYPE rocketmq_consumer_offset counter
+rocketmq_consumer_offset{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",} 1462030.0
+rocketmq_consumer_offset{cluster="MQCluster",broker="broker-a",topic="DEV_TID_tfq",group="DEV_CID_cfq",} 3843787.0
+rocketmq_consumer_offset{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",} 2800569.0
+rocketmq_consumer_offset{cluster="MQCluster",broker="broker-b",topic="DEV_TID_tfq",group="DEV_CID_cfq",} 1878633.0
+# HELP rocketmq_group_get_latency GroupGetLatency
+# TYPE rocketmq_group_get_latency gauge
+rocketmq_group_get_latency{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="0",} 0.05
+rocketmq_group_get_latency{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="1",} 0.0
+rocketmq_group_get_latency{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="7",} 0.05
+rocketmq_group_get_latency{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="6",} 0.016666666666666666
+rocketmq_group_get_latency{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="3",} 0.0
+rocketmq_group_get_latency{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="7",} 0.03333333333333333
+rocketmq_group_get_latency{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="4",} 0.0
+rocketmq_group_get_latency{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="5",} 0.03333333333333333
+rocketmq_group_get_latency{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="6",} 0.016666666666666666
+rocketmq_group_get_latency{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="2",} 0.0
+rocketmq_group_get_latency{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="3",} 0.0
+rocketmq_group_get_latency{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="0",} 0.0
+rocketmq_group_get_latency{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="4",} 0.0
+rocketmq_group_get_latency{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="1",} 0.03333333333333333
+rocketmq_group_get_latency{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="5",} 0.0
+rocketmq_group_get_latency{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="2",} 0.0
+# HELP rocketmq_group_get_latency_by_storetime GroupGetLatencyByStoreTime
+# TYPE rocketmq_group_get_latency_by_storetime gauge
+rocketmq_group_get_latency_by_storetime{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",} 3215.0
+rocketmq_group_get_latency_by_storetime{cluster="MQCluster",broker="broker-a",topic="DEV_TID_tfq",group="DEV_CID_cfq",} 0.0
+rocketmq_group_get_latency_by_storetime{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",} 3232.0
+rocketmq_group_get_latency_by_storetime{cluster="MQCluster",broker="broker-b",topic="DEV_TID_tfq",group="DEV_CID_cfq",} 0.0
+```
diff --git a/rocketmq-prometheus-exporter/example.rules b/rocketmq-prometheus-exporter/example.rules
new file mode 100644
index 0000000..e30c909
--- /dev/null
+++ b/rocketmq-prometheus-exporter/example.rules
@@ -0,0 +1,67 @@
+###
+# Sample prometheus rules/alerts for rocketmq.
+#
+# NOTE: Please review these carefully as thresholds and behavior may not meet
+# your SLOs or labels.
+#
+
+###
+# Recording Rules
+
+
+###
+# Galera Alerts
+
+groups:
+- name: GaleraAlerts
+ rules:
+ - alert: RocketMQClusterProduceHigh
+ expr: sum(rocketmq_producer_tps) by (cluster) >= 10
+ for: 3m
+ labels:
+ severity: warning
+ annotations:
+ description: '{{$labels.cluster}} Sending tps too high.'
+ summary: cluster send tps too high
+ - alert: RocketMQClusterProduceLow
+ expr: sum(rocketmq_producer_tps) by (cluster) < 1
+ for: 3m
+ labels:
+ severity: warning
+ annotations:
+ description: '{{$labels.cluster}} Sending tps too low.'
+ summary: cluster send tps too low
+ - alert: RocketMQClusterConsumeHigh
+ expr: sum(rocketmq_consumer_tps) by (cluster) >= 10
+ for: 3m
+ labels:
+ severity: warning
+ annotations:
+ description: '{{$labels.cluster}} consuming tps too high.'
+ summary: cluster consume tps too high
+ - alert: RocketMQClusterConsumeLow
+ expr: sum(rocketmq_consumer_tps) by (cluster) < 1
+ for: 3m
+ labels:
+ severity: warning
+ annotations:
+ description: '{{$labels.cluster}} consuming tps too low.'
+ summary: cluster consume tps too low
+ - alert: ConsumerFallingBehind
+ expr: (sum(rocketmq_producer_offset) by (topic) - on(topic) group_right sum(rocketmq_consumer_offset) by (group,topic)) - ignoring(group) group_left sum (avg_over_time(rocketmq_producer_tps[5m])) by (topic)*5*60 > 0
+ for: 3m
+ labels:
+ severity: warning
+ annotations:
+ description: 'consumer {{$labels.group}} on {{$labels.topic}} lag behind
+ and is falling behind (behind value {{$value}}).'
+ summary: consumer lag behind
+ - alert: GroupGetLatencyByStoretime
+ expr: rocketmq_group_get_latency_by_storetime > 1000
+ for: 3m
+ labels:
+ severity: warning
+ annotations:
+ description: 'consumer {{$labels.group}} on {{$labels.broker}}, {{$labels.topic}} consume time lag behind message store time
+ and (behind value is {{$value}}).'
+ summary: message consumes time lag behind message store time too much
diff --git a/rocketmq-prometheus-exporter/pom.xml b/rocketmq-prometheus-exporter/pom.xml
new file mode 100644
index 0000000..cda549d
--- /dev/null
+++ b/rocketmq-prometheus-exporter/pom.xml
@@ -0,0 +1,137 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <modelVersion>4.0.0</modelVersion>
+ <parent>
+ <groupId>org.springframework.boot</groupId>
+ <artifactId>spring-boot-starter-parent</artifactId>
+ <version>2.1.2.RELEASE</version>
+ <relativePath/> <!-- lookup parent from repository -->
+ </parent>
+ <groupId>org.apache</groupId>
+ <artifactId>rocketmq-exporter</artifactId>
+ <version>0.0.1-SNAPSHOT</version>
+ <name>rocketmq-exporter</name>
+
+ <description>Demo project for Spring Boot</description>
+
+ <properties>
+ <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
+ <java.version>1.7</java.version>
+ <guava.version>16.0.1</guava.version>
+ <commons-digester.version>2.1</commons-digester.version>
+ <commons-lang.version>2.6</commons-lang.version>
+ <commons-io.version>2.4</commons-io.version>
+ <commons-cli.version>1.2</commons-cli.version>
+ <rocketmq.version>4.4.0</rocketmq.version>
+ <surefire.version>2.19.1</surefire.version>
+ <aspectj.version>1.8.9</aspectj.version>
+ <main.basedir>${basedir}/../..</main.basedir>
+ <docker.image.prefix>docker.io</docker.image.prefix>
+ </properties>
+
+ <dependencies>
+ <dependency>
+ <groupId>org.springframework.boot</groupId>
+ <artifactId>spring-boot-starter-web</artifactId>
+ </dependency>
+
+ <dependency>
+ <groupId>org.springframework.boot</groupId>
+ <artifactId>spring-boot-starter-test</artifactId>
+ <scope>test</scope>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.rocketmq</groupId>
+ <artifactId>rocketmq-tools</artifactId>
+ <version>${rocketmq.version}</version>
+ </dependency>
+ <dependency>
+ <groupId>commons-collections</groupId>
+ <artifactId>commons-collections</artifactId>
+ <version>3.2.2</version>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.rocketmq</groupId>
+ <artifactId>rocketmq-namesrv</artifactId>
+ <version>${rocketmq.version}</version>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.rocketmq</groupId>
+ <artifactId>rocketmq-broker</artifactId>
+ <version>${rocketmq.version}</version>
+ </dependency>
+ <dependency>
+ <groupId>org.aspectj</groupId>
+ <artifactId>aspectjrt</artifactId>
+ <version>${aspectj.version}</version>
+ </dependency>
+ <dependency>
+ <groupId>org.aspectj</groupId>
+ <artifactId>aspectjweaver</artifactId>
+ <version>${aspectj.version}</version>
+ </dependency>
+ <dependency>
+ <groupId>org.jooq</groupId>
+ <artifactId>joor</artifactId>
+ <version>0.9.6</version>
+ </dependency>
+ <dependency>
+ <groupId>io.prometheus</groupId>
+ <artifactId>simpleclient</artifactId>
+ <version>0.6.0</version>
+ </dependency>
+ <dependency>
+ <groupId>io.prometheus</groupId>
+ <artifactId>simpleclient_common</artifactId>
+ <version>0.6.0</version>
+ </dependency>
+ </dependencies>
+
+ <build>
+ <plugins>
+ <plugin>
+ <groupId>org.springframework.boot</groupId>
+ <artifactId>spring-boot-maven-plugin</artifactId>
+ </plugin>
+ <plugin>
+ <artifactId>maven-checkstyle-plugin</artifactId>
+ <version>2.17</version>
+ <executions>
+ <execution>
+ <id>verify</id>
+ <phase>verify</phase>
+ <configuration>
+ <configLocation>style/rmq_checkstyle.xml</configLocation>
+ <encoding>UTF-8</encoding>
+ <consoleOutput>true</consoleOutput>
+ <failsOnError>true</failsOnError>
+ <includeTestSourceDirectory>false</includeTestSourceDirectory>
+ </configuration>
+ <goals>
+ <goal>check</goal>
+ </goals>
+ </execution>
+ </executions>
+ </plugin>
+ <plugin>
+ <groupId>com.spotify</groupId>
+ <artifactId>docker-maven-plugin</artifactId>
+ <version>0.4.11</version>
+ <configuration>
+ <imageName>${docker.image.prefix}/${project.artifactId}</imageName>
+ <dockerDirectory>src/main/docker</dockerDirectory>
+ <resources>
+ <resource>
+ <targetPath>/</targetPath>
+ <directory>${project.build.directory}</directory>
+ <include>${project.build.finalName}.jar</include>
+ </resource>
+ </resources>
+ </configuration>
+ </plugin>
+ </plugins>
+
+ </build>
+
+</project>
diff --git a/rocketmq-prometheus-exporter/src/main/docker/Dockerfile b/rocketmq-prometheus-exporter/src/main/docker/Dockerfile
new file mode 100644
index 0000000..27f209c
--- /dev/null
+++ b/rocketmq-prometheus-exporter/src/main/docker/Dockerfile
@@ -0,0 +1,5 @@
+FROM java:8
+MAINTAINER breeze
+ADD rocketmq-exporter-0.0.1-SNAPSHOT.jar demo.jar
+EXPOSE 5557
+ENTRYPOINT ["java","-jar","demo.jar"]
diff --git a/rocketmq-prometheus-exporter/src/main/java/org/apache/rocketmq/exporter/RocketMQExporterApplication.java b/rocketmq-prometheus-exporter/src/main/java/org/apache/rocketmq/exporter/RocketMQExporterApplication.java
new file mode 100644
index 0000000..0866aef
--- /dev/null
+++ b/rocketmq-prometheus-exporter/src/main/java/org/apache/rocketmq/exporter/RocketMQExporterApplication.java
@@ -0,0 +1,35 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.rocketmq.exporter;
+
+import org.springframework.boot.SpringApplication;
+import org.springframework.boot.autoconfigure.EnableAutoConfiguration;
+import org.springframework.boot.autoconfigure.SpringBootApplication;
+import org.springframework.boot.web.servlet.ServletComponentScan;
+import org.springframework.scheduling.annotation.EnableScheduling;
+
+@EnableAutoConfiguration
+@SpringBootApplication
+@ServletComponentScan
+@EnableScheduling
+public class RocketMQExporterApplication {
+ public static void main(String[] args) {
+ SpringApplication.run(RocketMQExporterApplication.class, args);
+ }
+}
+
+
diff --git a/rocketmq-prometheus-exporter/src/main/java/org/apache/rocketmq/exporter/aspect/admin/MQAdminAspect.java b/rocketmq-prometheus-exporter/src/main/java/org/apache/rocketmq/exporter/aspect/admin/MQAdminAspect.java
new file mode 100644
index 0000000..bc3c38e
--- /dev/null
+++ b/rocketmq-prometheus-exporter/src/main/java/org/apache/rocketmq/exporter/aspect/admin/MQAdminAspect.java
@@ -0,0 +1,71 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.rocketmq.exporter.aspect.admin;
+
+import java.lang.reflect.Method;
+import org.apache.rocketmq.exporter.aspect.admin.annotation.MultiMQAdminCmdMethod;
+import org.apache.rocketmq.exporter.service.client.MQAdminInstance;
+import org.aspectj.lang.ProceedingJoinPoint;
+import org.aspectj.lang.annotation.Around;
+import org.aspectj.lang.annotation.Aspect;
+import org.aspectj.lang.annotation.Pointcut;
+import org.aspectj.lang.reflect.MethodSignature;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import org.springframework.stereotype.Service;
+
+@Aspect
+@Service
+public class MQAdminAspect {
+ private Logger logger = LoggerFactory.getLogger(MQAdminAspect.class);
+
+ public MQAdminAspect() {
+ }
+
+ @Pointcut("execution(* org.apache.rocketmq.exporter.service.client.MQAdminExtImpl..*(..))")
+ public void mQAdminMethodPointCut() {
+
+ }
+
+ @Pointcut("@annotation(org.apache.rocketmq.exporter.aspect.admin.annotation.MultiMQAdminCmdMethod)")
+ public void multiMQAdminMethodPointCut() {
+
+ }
+
+ @Around(value = "mQAdminMethodPointCut() || multiMQAdminMethodPointCut()")
+ public Object aroundMQAdminMethod(ProceedingJoinPoint joinPoint) throws Throwable {
+ long start = System.currentTimeMillis();
+ Object obj = null;
+ try {
+ MethodSignature signature = (MethodSignature)joinPoint.getSignature();
+ Method method = signature.getMethod();
+ MultiMQAdminCmdMethod multiMQAdminCmdMethod = method.getAnnotation(MultiMQAdminCmdMethod.class);
+ if (multiMQAdminCmdMethod != null && multiMQAdminCmdMethod.timeoutMillis() > 0) {
+ MQAdminInstance.initMQAdminInstance(multiMQAdminCmdMethod.timeoutMillis());
+ }
+ else {
+ MQAdminInstance.initMQAdminInstance(0);
+ }
+ obj = joinPoint.proceed();
+ }
+ finally {
+ MQAdminInstance.destroyMQAdminInstance();
+ logger.debug("op=look method={} cost={}", joinPoint.getSignature().getName(), System.currentTimeMillis() - start);
+ }
+ return obj;
+ }
+}
diff --git a/rocketmq-prometheus-exporter/src/main/java/org/apache/rocketmq/exporter/aspect/admin/annotation/MultiMQAdminCmdMethod.java b/rocketmq-prometheus-exporter/src/main/java/org/apache/rocketmq/exporter/aspect/admin/annotation/MultiMQAdminCmdMethod.java
new file mode 100644
index 0000000..7953fb3
--- /dev/null
+++ b/rocketmq-prometheus-exporter/src/main/java/org/apache/rocketmq/exporter/aspect/admin/annotation/MultiMQAdminCmdMethod.java
@@ -0,0 +1,30 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.rocketmq.exporter.aspect.admin.annotation;
+
+import java.lang.annotation.Documented;
+import java.lang.annotation.ElementType;
+import java.lang.annotation.Retention;
+import java.lang.annotation.RetentionPolicy;
+import java.lang.annotation.Target;
+
+@Target({ElementType.METHOD})
+@Retention(RetentionPolicy.RUNTIME)
+@Documented
+public @interface MultiMQAdminCmdMethod {
+ long timeoutMillis() default 0;
+}
diff --git a/rocketmq-prometheus-exporter/src/main/java/org/apache/rocketmq/exporter/collector/RMQMetricsCollector.java b/rocketmq-prometheus-exporter/src/main/java/org/apache/rocketmq/exporter/collector/RMQMetricsCollector.java
new file mode 100644
index 0000000..dbaf4dd
--- /dev/null
+++ b/rocketmq-prometheus-exporter/src/main/java/org/apache/rocketmq/exporter/collector/RMQMetricsCollector.java
@@ -0,0 +1,190 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.rocketmq.exporter.collector;
+
+import io.prometheus.client.Collector;
+import io.prometheus.client.CounterMetricFamily;
+import io.prometheus.client.GaugeMetricFamily;
+import org.apache.rocketmq.exporter.model.metrics.BrokerMetric;
+import org.apache.rocketmq.exporter.model.metrics.ConsumerMetric;
+import org.apache.rocketmq.exporter.model.metrics.ConsumerQueueMetric;
+import org.apache.rocketmq.exporter.model.metrics.ProducerMetric;
+
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.ConcurrentHashMap;
+
+public class RMQMetricsCollector extends Collector {
+
+ private ConcurrentHashMap<ProducerMetric, Double> topicPutNums = new ConcurrentHashMap<>();
+ private ConcurrentHashMap<ProducerMetric, Double> topicPutSize = new ConcurrentHashMap<>();
+
+ private ConcurrentHashMap<ProducerMetric, Double> topicOffset = new ConcurrentHashMap<>();
+
+ private ConcurrentHashMap<BrokerMetric, Double> brokerPutNums = new ConcurrentHashMap<>();
+ private ConcurrentHashMap<BrokerMetric, Double> brokerGetNums = new ConcurrentHashMap<>();
+
+ private ConcurrentHashMap<ConsumerMetric, Double> groupGetNums = new ConcurrentHashMap<>();
+ private ConcurrentHashMap<ConsumerMetric, Double> groupGetSize = new ConcurrentHashMap<>();
+
+ private ConcurrentHashMap<ConsumerMetric, Double> sendBackNums = new ConcurrentHashMap<>();
+ private ConcurrentHashMap<ConsumerMetric, Double> groupOffset = new ConcurrentHashMap<>();
+
+ private ConcurrentHashMap<ConsumerQueueMetric, Double> groupGetLatency = new ConcurrentHashMap<>();
+
+ private ConcurrentHashMap<ConsumerMetric, Double> groupGetLatencyByStoreTime = new ConcurrentHashMap<>();
+
+ @Override
+ public List<MetricFamilySamples> collect() {
+
+ List<MetricFamilySamples> mfs = new ArrayList<MetricFamilySamples>();
+
+ GaugeMetricFamily topicPutNumsGauge = new GaugeMetricFamily("rocketmq_producer_tps", "TopicPutNums", Arrays.asList("cluster","broker","topic"));
+ for (Map.Entry<ProducerMetric,Double> entry:topicPutNums.entrySet()) {
+ topicPutNumsGauge.addMetric(Arrays.asList(entry.getKey().getClusterName(),entry.getKey().getBrokerName(),entry.getKey().getTopicName()), entry.getValue());
+ }
+ mfs.add(topicPutNumsGauge);
+
+
+ GaugeMetricFamily topicPutSizeGauge = new GaugeMetricFamily("rocketmq_producer_put_size", "TopicPutSize", Arrays.asList("cluster","broker","topic"));
+ for (Map.Entry<ProducerMetric, Double> entry: topicPutSize.entrySet()) {
+ topicPutSizeGauge.addMetric(Arrays.asList(entry.getKey().getClusterName(),entry.getKey().getBrokerName(),entry.getKey().getTopicName()), entry.getValue());
+ }
+ mfs.add(topicPutSizeGauge);
+
+
+ CounterMetricFamily topicOffsetGauge = new CounterMetricFamily("rocketmq_producer_offset", "TopicOffset", Arrays.asList("cluster","broker","topic"));
+ for (Map.Entry<ProducerMetric, Double> entry: topicOffset.entrySet()) {
+ topicOffsetGauge.addMetric(Arrays.asList(entry.getKey().getClusterName(),entry.getKey().getBrokerName(),entry.getKey().getTopicName()), entry.getValue());
+ }
+ mfs.add(topicOffsetGauge);
+
+
+ GaugeMetricFamily brokerPutNumsGauge = new GaugeMetricFamily("rocketmq_broker_tps", "BrokerPutNums", Arrays.asList("cluster","broker"));
+ for (Map.Entry<BrokerMetric, Double> entry: brokerPutNums.entrySet()) {
+ brokerPutNumsGauge.addMetric(Arrays.asList(entry.getKey().getClusterName(),entry.getKey().getBrokerName()), entry.getValue());
+ }
+ mfs.add(brokerPutNumsGauge);
+
+
+ GaugeMetricFamily brokerGetNumsGauge = new GaugeMetricFamily("rocketmq_broker_qps", "BrokerGetNums", Arrays.asList("cluster","broker"));
+ for (Map.Entry<BrokerMetric, Double> entry: brokerGetNums.entrySet()) {
+ brokerGetNumsGauge.addMetric(Arrays.asList(entry.getKey().getClusterName(),entry.getKey().getBrokerName()), entry.getValue());
+ }
+ mfs.add(brokerGetNumsGauge);
+
+
+ GaugeMetricFamily groupGetNumsGauge = new GaugeMetricFamily("rocketmq_consumer_tps", "GroupGetNums", Arrays.asList("cluster","broker","topic","group"));
+ for (Map.Entry<ConsumerMetric, Double> entry: groupGetNums.entrySet()) {
+ groupGetNumsGauge.addMetric(Arrays.asList(entry.getKey().getClusterName(),entry.getKey().getBrokerName(),entry.getKey().getTopicName(),entry.getKey().getConsumerGroupName()), entry.getValue());
+ }
+
+ mfs.add(groupGetNumsGauge);
+
+
+ GaugeMetricFamily groupGetSizeGauge = new GaugeMetricFamily("rocketmq_consumer_get_size", "GroupGetSize", Arrays.asList("cluster","broker","topic","group"));
+ for (Map.Entry<ConsumerMetric, Double> entry: groupGetSize.entrySet()) {
+ groupGetSizeGauge.addMetric(Arrays.asList(entry.getKey().getClusterName(),entry.getKey().getBrokerName(),entry.getKey().getTopicName(),entry.getKey().getConsumerGroupName()), entry.getValue());
+ }
+ mfs.add(groupGetSizeGauge);
+
+ CounterMetricFamily groupOffsetGauge = new CounterMetricFamily("rocketmq_consumer_offset", "GroupOffset", Arrays.asList("cluster","broker","topic","group"));
+ for (Map.Entry<ConsumerMetric, Double> entry: groupOffset.entrySet()) {
+ groupOffsetGauge.addMetric(Arrays.asList(entry.getKey().getClusterName(),entry.getKey().getBrokerName(),entry.getKey().getTopicName(),entry.getKey().getConsumerGroupName()), entry.getValue());
+ }
+ mfs.add(groupOffsetGauge);
+
+
+ GaugeMetricFamily sendBackNumsGauge = new GaugeMetricFamily("rocketmq_send_back_nums", "SendBackNums", Arrays.asList("cluster","broker","topic","group"));
+ for (Map.Entry<ConsumerMetric, Double> entry: sendBackNums.entrySet()) {
+ sendBackNumsGauge.addMetric(Arrays.asList(entry.getKey().getClusterName(),entry.getKey().getBrokerName(),entry.getKey().getTopicName(),entry.getKey().getConsumerGroupName()), entry.getValue());
+ }
+ mfs.add(sendBackNumsGauge);
+
+
+ GaugeMetricFamily groupGetLatencyGauge = new GaugeMetricFamily("rocketmq_group_get_latency", "GroupGetLatency", Arrays.asList("cluster","broker","topic","group","queueid"));
+ for (Map.Entry<ConsumerQueueMetric, Double> entry: groupGetLatency.entrySet()) {
+ groupGetLatencyGauge.addMetric(Arrays.asList(entry.getKey().getClusterName(),entry.getKey().getBrokerName(),entry.getKey().getTopicName(),entry.getKey().getConsumerGroupName(),entry.getKey().getQueueId()), entry.getValue());
+ }
+ mfs.add(groupGetLatencyGauge);
+
+ GaugeMetricFamily groupGetLatencyByStoretimeGauge = new GaugeMetricFamily("rocketmq_group_get_latency_by_storetime", "GroupGetLatencyByStoreTime", Arrays.asList("cluster","broker","topic","group"));
+ for (Map.Entry<ConsumerMetric, Double> entry: groupGetLatencyByStoreTime.entrySet()) {
+ groupGetLatencyByStoretimeGauge.addMetric(Arrays.asList(entry.getKey().getClusterName(),entry.getKey().getBrokerName(),entry.getKey().getTopicName(),entry.getKey().getConsumerGroupName()), entry.getValue());
+ }
+ mfs.add(groupGetLatencyByStoretimeGauge);
+
+ return mfs;
+ }
+ public void AddTopicPutNumsMetric(String clusterName, String brokerName, String topic, double value)
+ {
+ topicPutNums.put(new ProducerMetric(clusterName,brokerName,topic),value);
+ }
+
+ public void AddTopicPutSizeMetric(String clusterName, String brokerName, String topic, double value)
+ {
+ topicPutSize.put(new ProducerMetric(clusterName,brokerName,topic),value);
+ }
+
+ public void AddTopicOffsetMetric(String clusterName, String brokerName, String topic, double value)
+ {
+ topicOffset.put(new ProducerMetric(clusterName,brokerName,topic),value);
+ }
+
+ public void AddBrokerPutNumsMetric(String clusterName, String brokerName, double value)
+ {
+ brokerPutNums.put(new BrokerMetric(clusterName,brokerName),value);
+ }
+
+ public void AddBrokerGetNumsMetric(String clusterName, String brokerName, double value)
+ {
+ brokerGetNums.put(new BrokerMetric(clusterName,brokerName),value);
+ }
+
+ public void AddGroupGetNumsMetric(String clusterName, String brokerName, String topic, String group, double value)
+ {
+ groupGetNums.put(new ConsumerMetric(clusterName,brokerName,topic,group),value);
+ }
+
+ public void AddGroupGetSizeMetric(String clusterName, String brokerName, String topic, String group, double value)
+ {
+ groupGetSize.put(new ConsumerMetric(clusterName,brokerName,topic,group),value);
+ }
+
+ public void AddGroupOffsetMetric(String clusterName, String brokerName, String topic, String group, double value)
+ {
+ groupOffset.put(new ConsumerMetric(clusterName,brokerName,topic,group),value);
+ }
+
+
+ public void AddsendBackNumsMetric(String clusterName, String brokerName, String topic, String group, double value)
+ {
+ sendBackNums.put(new ConsumerMetric(clusterName,brokerName,topic,group),value);
+ }
+
+ public void AddGroupGetLatencyMetric(String clusterName, String brokerName, String topic, String group, String queueId,double value) {
+
+ groupGetLatency.put(new ConsumerQueueMetric(clusterName,brokerName,topic,group,queueId),value);
+ }
+
+ public void AddGroupGetLatencyByStoreTimeMetric(String clusterName, String brokerName, String topic, String group,double value) {
+
+ groupGetLatencyByStoreTime.put(new ConsumerMetric(clusterName,brokerName,topic,group),value);
+ }
+}
\ No newline at end of file
diff --git a/rocketmq-prometheus-exporter/src/main/java/org/apache/rocketmq/exporter/config/RMQConfigure.java b/rocketmq-prometheus-exporter/src/main/java/org/apache/rocketmq/exporter/config/RMQConfigure.java
new file mode 100644
index 0000000..2b3956b
--- /dev/null
+++ b/rocketmq-prometheus-exporter/src/main/java/org/apache/rocketmq/exporter/config/RMQConfigure.java
@@ -0,0 +1,102 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.rocketmq.exporter.config;
+
+import org.apache.commons.lang3.StringUtils;
+import org.apache.rocketmq.common.MixAll;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import org.springframework.boot.context.properties.ConfigurationProperties;
+import org.springframework.context.annotation.Configuration;
+
+
+
+import static org.apache.rocketmq.client.ClientConfig.SEND_MESSAGE_WITH_VIP_CHANNEL_PROPERTY;
+
+@Configuration
+@ConfigurationProperties(prefix = "rocketmq.config")
+public class RMQConfigure {
+
+ public static final String ROCKETMQ_CONFIG_WEB_TELEMETRY_PATH = "rocketmq.config.webTelemetryPath";
+ public static final String ROCKETMQ_CONFIG_ROCKETMQ_VERSION = "rocketmq.config.rocketmqVersion";
+
+ private Logger logger = LoggerFactory.getLogger(RMQConfigure.class);
+ //use rocketmq.namesrv.addr first,if it is empty,than use system proerty or system env
+ private volatile String namesrvAddr = System.getProperty(MixAll.NAMESRV_ADDR_PROPERTY, System.getenv(MixAll.NAMESRV_ADDR_ENV));
+
+ private volatile String isVIPChannel = System.getProperty(SEND_MESSAGE_WITH_VIP_CHANNEL_PROPERTY, "true");
+
+ private boolean enableCollect;
+
+ private volatile String webTelemetryPath = System.getProperty(ROCKETMQ_CONFIG_WEB_TELEMETRY_PATH, "/metrics");
+
+ private volatile String rocketmqVersion = System.getProperty(ROCKETMQ_CONFIG_ROCKETMQ_VERSION, "V4_3_2");
+
+ public String getNamesrvAddr() {
+ return namesrvAddr;
+ }
+
+ public void setNamesrvAddr(String namesrvAddr) {
+ if (StringUtils.isNotBlank(namesrvAddr)) {
+ this.namesrvAddr = namesrvAddr;
+ System.setProperty(MixAll.NAMESRV_ADDR_PROPERTY, namesrvAddr);
+ logger.info("setNameSrvAddrByProperty nameSrvAddr={}", namesrvAddr);
+ }
+ }
+
+ public String getIsVIPChannel() {
+ return isVIPChannel;
+ }
+
+ public void setIsVIPChannel(String isVIPChannel) {
+ if (StringUtils.isNotBlank(isVIPChannel)) {
+ this.isVIPChannel = isVIPChannel;
+ System.setProperty(SEND_MESSAGE_WITH_VIP_CHANNEL_PROPERTY, isVIPChannel);
+ logger.info("setIsVIPChannel isVIPChannel={}", isVIPChannel);
+ }
+ }
+ public boolean isEnableCollect() {
+ return enableCollect;
+ }
+ public void setEnableCollect(boolean enableCollect) {
+ this.enableCollect = enableCollect;
+ }
+
+ public void setWebTelemetryPath(String webTelemetryPath) {
+ if (StringUtils.isNotBlank(webTelemetryPath)) {
+ this.webTelemetryPath = webTelemetryPath;
+ System.setProperty(ROCKETMQ_CONFIG_WEB_TELEMETRY_PATH, webTelemetryPath);
+ logger.info("setWebTelemetryPath webTelemetryPath={}", webTelemetryPath);
+ }
+ }
+
+ public String getWebTelemetryPath() {
+ return webTelemetryPath;
+ }
+
+ public void setRocketmqVersion(String rocketmqVersion) {
+ if (StringUtils.isNotBlank(rocketmqVersion)) {
+ this.rocketmqVersion = rocketmqVersion;
+ System.setProperty(ROCKETMQ_CONFIG_ROCKETMQ_VERSION, rocketmqVersion);
+ logger.info("setRocketmqVersion rocketmqVersion={}", rocketmqVersion);
+ }
+ }
+
+ public String getRocketmqVersion() {
+ return rocketmqVersion;
+ }
+}
diff --git a/rocketmq-prometheus-exporter/src/main/java/org/apache/rocketmq/exporter/controller/RMQMetricsController.java b/rocketmq-prometheus-exporter/src/main/java/org/apache/rocketmq/exporter/controller/RMQMetricsController.java
new file mode 100644
index 0000000..4df69d1
--- /dev/null
+++ b/rocketmq-prometheus-exporter/src/main/java/org/apache/rocketmq/exporter/controller/RMQMetricsController.java
@@ -0,0 +1,52 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.rocketmq.exporter.controller;
+
+
+import org.apache.rocketmq.exporter.service.RMQMetricsService;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import org.springframework.boot.autoconfigure.EnableAutoConfiguration;
+import org.springframework.web.bind.annotation.RequestMapping;
+import org.springframework.web.bind.annotation.ResponseBody;
+import org.springframework.web.bind.annotation.RestController;
+
+import javax.annotation.Resource;
+import javax.servlet.http.HttpServletResponse;
+import java.io.IOException;
+import java.io.StringWriter;
+
+@RestController
+@EnableAutoConfiguration
+public class RMQMetricsController {
+
+ private final static Logger log = LoggerFactory.getLogger(RMQMetricsController.class);
+
+ @Resource
+ RMQMetricsService metricsService;
+
+ @RequestMapping(value = "${rocketmq.config.webTelemetryPath}")
+ @ResponseBody
+ private void metrics(HttpServletResponse response) throws IOException {
+
+ StringWriter writer = new StringWriter();
+ metricsService.Metrics(writer);
+
+ response.setHeader("Content-Type", "text/plain; version=0.0.4; charset=utf-8");
+ response.getOutputStream().print(writer.toString());
+ }
+}
diff --git a/rocketmq-prometheus-exporter/src/main/java/org/apache/rocketmq/exporter/exception/ServiceException.java b/rocketmq-prometheus-exporter/src/main/java/org/apache/rocketmq/exporter/exception/ServiceException.java
new file mode 100644
index 0000000..ccf3fdc
--- /dev/null
+++ b/rocketmq-prometheus-exporter/src/main/java/org/apache/rocketmq/exporter/exception/ServiceException.java
@@ -0,0 +1,31 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.rocketmq.exporter.exception;
+
+public class ServiceException extends RuntimeException {
+ private static final long serialVersionUID = 9213584003139969215L;
+ private int code;
+
+ public ServiceException(int code, String message) {
+ super(message);
+ this.code = code;
+ }
+
+ public int getCode() {
+ return code;
+ }
+}
diff --git a/rocketmq-prometheus-exporter/src/main/java/org/apache/rocketmq/exporter/model/metrics/BrokerMetric.java b/rocketmq-prometheus-exporter/src/main/java/org/apache/rocketmq/exporter/model/metrics/BrokerMetric.java
new file mode 100644
index 0000000..c7a0727
--- /dev/null
+++ b/rocketmq-prometheus-exporter/src/main/java/org/apache/rocketmq/exporter/model/metrics/BrokerMetric.java
@@ -0,0 +1,70 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.rocketmq.exporter.model.metrics;
+
+public class BrokerMetric {
+
+ private String clusterName;
+ private String brokerName;
+
+
+ public void setClusterName(String cluster) {
+
+ clusterName = cluster;
+ }
+ public String getClusterName() {
+
+ return clusterName;
+ }
+ void setBrokerName(String broker) {
+
+ brokerName = broker;
+ }
+
+ public String getBrokerName() {
+
+ return brokerName;
+ }
+
+ public BrokerMetric(String cluster, String broker) {
+ clusterName = cluster;
+ brokerName = broker;
+ }
+
+ @Override
+ public boolean equals(Object obj) {
+ if (!(obj instanceof BrokerMetric)) {
+ return false;
+ }
+ BrokerMetric other = (BrokerMetric) obj;
+
+ return other.clusterName.equals(clusterName) && other.brokerName.equals(brokerName);
+ }
+
+ @Override
+ public int hashCode() {
+ int hash = 1;
+ hash = 37 * hash + clusterName.hashCode();
+ hash = 37 * hash + brokerName.hashCode();
+ return hash;
+ }
+
+ @Override
+ public String toString() {
+ return "ClusterName: " + clusterName + " BrokerName: " + brokerName;
+ }
+}
diff --git a/rocketmq-prometheus-exporter/src/main/java/org/apache/rocketmq/exporter/model/metrics/ConsumerMetric.java b/rocketmq-prometheus-exporter/src/main/java/org/apache/rocketmq/exporter/model/metrics/ConsumerMetric.java
new file mode 100644
index 0000000..9530fff
--- /dev/null
+++ b/rocketmq-prometheus-exporter/src/main/java/org/apache/rocketmq/exporter/model/metrics/ConsumerMetric.java
@@ -0,0 +1,86 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.rocketmq.exporter.model.metrics;
+
+public class ConsumerMetric {
+
+ private String clusterName;
+ private String brokerName;
+ private String topicName;
+ private String consumerGroupName;
+
+ public void setClusterName(String cluster) {
+ clusterName = cluster;
+ }
+ public String getClusterName() {
+ return clusterName;
+ }
+ void setBrokerName(String broker) {
+ brokerName = broker;
+ }
+
+ public String getBrokerName() {
+ return brokerName;
+ }
+
+ public void setTopicName(String topic) {
+ topicName = topic;
+ }
+ public String getTopicName() {
+ return topicName;
+ }
+ public String getConsumerGroupName() {
+ return consumerGroupName;
+ }
+
+ public void setConsumerGroupName(String consumerGroupName) {
+ this.consumerGroupName = consumerGroupName;
+ }
+
+ public ConsumerMetric(String cluster, String broker, String topic,String consumerGroup) {
+ clusterName = cluster;
+ brokerName = broker;
+ topicName = topic;
+ consumerGroupName = consumerGroup;
+ }
+
+ @Override
+ public boolean equals(Object obj) {
+ if (!(obj instanceof ConsumerMetric)) {
+ return false;
+ }
+ ConsumerMetric other = (ConsumerMetric) obj;
+
+ return other.clusterName.equals(clusterName) && other.brokerName.equals(brokerName)
+ && other.topicName.equals(topicName) && other.consumerGroupName.equals(consumerGroupName);
+ }
+
+ @Override
+ public int hashCode() {
+ int hash = 1;
+ hash = 37 * hash + clusterName.hashCode();
+ hash = 37 * hash + brokerName.hashCode();
+ hash = 37 * hash + topicName.hashCode();
+ hash = 37 * hash + consumerGroupName.hashCode();
+ return hash;
+ }
+
+ @Override
+ public String toString() {
+ return "ClusterName: " + clusterName + " BrokerName: " + brokerName + " topicName: " + topicName + " ConsumeGroupName: " + consumerGroupName;
+ }
+}
diff --git a/rocketmq-prometheus-exporter/src/main/java/org/apache/rocketmq/exporter/model/metrics/ConsumerQueueMetric.java b/rocketmq-prometheus-exporter/src/main/java/org/apache/rocketmq/exporter/model/metrics/ConsumerQueueMetric.java
new file mode 100644
index 0000000..a6453fc
--- /dev/null
+++ b/rocketmq-prometheus-exporter/src/main/java/org/apache/rocketmq/exporter/model/metrics/ConsumerQueueMetric.java
@@ -0,0 +1,93 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.rocketmq.exporter.model.metrics;
+
+public class ConsumerQueueMetric {
+
+ private String clusterName;
+ private String brokerName;
+ private String topicName;
+ private String consumerGroupName;
+ private String queueId;
+
+ public void setClusterName(String cluster) {
+ clusterName = cluster;
+ }
+ public String getClusterName() {
+ return clusterName;
+ }
+ void setBrokerName(String broker) {
+ brokerName = broker;
+ }
+ public String getBrokerName() {
+ return brokerName;
+ }
+ public void setTopicName(String topic) {
+ topicName = topic;
+ }
+ public String getTopicName() {
+ return topicName;
+ }
+ public String getConsumerGroupName() {
+ return consumerGroupName;
+ }
+
+ public void setConsumerGroupName(String consumerGroupName) {
+ this.consumerGroupName = consumerGroupName;
+ }
+ public String getQueueId() {
+ return queueId;
+ }
+ public void setQueueId(String queueId) {
+ this.queueId = queueId;
+ }
+ public ConsumerQueueMetric(String cluster, String broker, String topic, String consumerGroup,String queue) {
+ clusterName = cluster;
+ brokerName = broker;
+ topicName = topic;
+ consumerGroupName = consumerGroup;
+ queueId = queue;
+ }
+
+ @Override
+ public boolean equals(Object obj) {
+ if (!(obj instanceof ConsumerQueueMetric)) {
+ return false;
+ }
+ ConsumerQueueMetric other = (ConsumerQueueMetric) obj;
+
+ return other.clusterName.equals(clusterName) && other.brokerName.equals(brokerName)
+ && other.topicName.equals(topicName) && other.consumerGroupName.equals(consumerGroupName)
+ && other.queueId.equals(queueId);
+ }
+
+ @Override
+ public int hashCode() {
+ int hash = 1;
+ hash = 37 * hash + clusterName.hashCode();
+ hash = 37 * hash + brokerName.hashCode();
+ hash = 37 * hash + topicName.hashCode();
+ hash = 37 * hash + consumerGroupName.hashCode();
+ hash = 37 * hash + queueId.hashCode();
+ return hash;
+ }
+
+ @Override
+ public String toString() {
+ return "ClusterName: " + clusterName + " BrokerName: " + brokerName + " topicName: " + topicName + " ConsumeGroupName: " + consumerGroupName + "queueId: " + queueId;
+ }
+}
diff --git a/rocketmq-prometheus-exporter/src/main/java/org/apache/rocketmq/exporter/model/metrics/ProducerMetric.java b/rocketmq-prometheus-exporter/src/main/java/org/apache/rocketmq/exporter/model/metrics/ProducerMetric.java
new file mode 100644
index 0000000..72baa73
--- /dev/null
+++ b/rocketmq-prometheus-exporter/src/main/java/org/apache/rocketmq/exporter/model/metrics/ProducerMetric.java
@@ -0,0 +1,73 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.rocketmq.exporter.model.metrics;
+
+public class ProducerMetric {
+
+ private String clusterName;
+ private String brokerName;
+ private String topicName;
+
+ public void setClusterName(String cluster) {
+ clusterName = cluster;
+ }
+ public String getClusterName() {
+ return clusterName;
+ }
+ void setBrokerName(String broker) {
+ brokerName = broker;
+ }
+ public String getBrokerName() {
+ return brokerName;
+ }
+ public void setTopicName(String topic) {
+ topicName = topic;
+ }
+ public String getTopicName() {
+ return topicName;
+ }
+ public ProducerMetric(String cluster,String broker,String topic) {
+ clusterName = cluster;
+ brokerName = broker;
+ topicName = topic;
+ }
+
+ @Override
+ public boolean equals(Object obj) {
+ if (!(obj instanceof ProducerMetric)) {
+ return false;
+ }
+ ProducerMetric other = (ProducerMetric) obj;
+
+ return other.clusterName.equals(clusterName) && other.brokerName.equals(brokerName)
+ && other.topicName.equals(topicName);
+ }
+
+ @Override
+ public int hashCode() {
+ int hash = 1;
+ hash = 37 * hash + clusterName.hashCode();
+ hash = 37 * hash + brokerName.hashCode();
+ hash = 37 * hash + topicName.hashCode();
+ return hash;
+ }
+
+ @Override
+ public String toString() {
+ return "ClusterName: " + clusterName + " BrokerName: " + brokerName + " topicName: " + topicName;
+ }
+}
diff --git a/rocketmq-prometheus-exporter/src/main/java/org/apache/rocketmq/exporter/service/AbstractCommonService.java b/rocketmq-prometheus-exporter/src/main/java/org/apache/rocketmq/exporter/service/AbstractCommonService.java
new file mode 100644
index 0000000..50f5b0a
--- /dev/null
+++ b/rocketmq-prometheus-exporter/src/main/java/org/apache/rocketmq/exporter/service/AbstractCommonService.java
@@ -0,0 +1,50 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.rocketmq.exporter.service;
+
+import com.google.common.base.Throwables;
+import com.google.common.collect.Sets;
+import org.apache.commons.collections.CollectionUtils;
+import org.apache.rocketmq.tools.admin.MQAdminExt;
+
+import javax.annotation.Resource;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Set;
+
+public abstract class AbstractCommonService {
+ @Resource
+ protected MQAdminExt mqAdminExt;
+ protected final Set<String> changeToBrokerNameSet(HashMap<String, Set<String>> clusterAddrTable,
+ List<String> clusterNameList, List<String> brokerNameList) {
+ Set<String> finalBrokerNameList = Sets.newHashSet();
+ if (CollectionUtils.isNotEmpty(clusterNameList)) {
+ try {
+ for (String clusterName : clusterNameList) {
+ finalBrokerNameList.addAll(clusterAddrTable.get(clusterName));
+ }
+ }
+ catch (Exception e) {
+ throw Throwables.propagate(e);
+ }
+ }
+ if (CollectionUtils.isNotEmpty(brokerNameList)) {
+ finalBrokerNameList.addAll(brokerNameList);
+ }
+ return finalBrokerNameList;
+ }
+}
diff --git a/rocketmq-prometheus-exporter/src/main/java/org/apache/rocketmq/exporter/service/RMQMetricsService.java b/rocketmq-prometheus-exporter/src/main/java/org/apache/rocketmq/exporter/service/RMQMetricsService.java
new file mode 100644
index 0000000..3b2e403
--- /dev/null
+++ b/rocketmq-prometheus-exporter/src/main/java/org/apache/rocketmq/exporter/service/RMQMetricsService.java
@@ -0,0 +1,29 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.rocketmq.exporter.service;
+
+import org.apache.rocketmq.exporter.collector.RMQMetricsCollector;
+
+
+import java.io.IOException;
+import java.io.StringWriter;
+
+
+public interface RMQMetricsService {
+ public RMQMetricsCollector getCollector();
+ public void Metrics(StringWriter writer) throws IOException;
+}
diff --git a/rocketmq-prometheus-exporter/src/main/java/org/apache/rocketmq/exporter/service/client/MQAdminExtImpl.java b/rocketmq-prometheus-exporter/src/main/java/org/apache/rocketmq/exporter/service/client/MQAdminExtImpl.java
new file mode 100644
index 0000000..17ec7d5
--- /dev/null
+++ b/rocketmq-prometheus-exporter/src/main/java/org/apache/rocketmq/exporter/service/client/MQAdminExtImpl.java
@@ -0,0 +1,515 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.rocketmq.exporter.service.client;
+
+import com.google.common.base.Throwables;
+import org.apache.rocketmq.client.QueryResult;
+import org.apache.rocketmq.client.consumer.PullResult;
+import org.apache.rocketmq.client.exception.MQBrokerException;
+import org.apache.rocketmq.client.exception.MQClientException;
+import org.apache.rocketmq.client.impl.MQAdminImpl;
+import org.apache.rocketmq.common.TopicConfig;
+import org.apache.rocketmq.common.admin.ConsumeStats;
+import org.apache.rocketmq.common.admin.RollbackStats;
+import org.apache.rocketmq.common.admin.TopicStatsTable;
+import org.apache.rocketmq.common.message.MessageClientIDSetter;
+import org.apache.rocketmq.common.message.MessageExt;
+import org.apache.rocketmq.common.message.MessageQueue;
+import org.apache.rocketmq.common.protocol.RequestCode;
+import org.apache.rocketmq.common.protocol.ResponseCode;
+import org.apache.rocketmq.common.protocol.body.BrokerStatsData;
+import org.apache.rocketmq.common.protocol.body.ClusterInfo;
+import org.apache.rocketmq.common.protocol.body.ConsumeMessageDirectlyResult;
+import org.apache.rocketmq.common.protocol.body.ConsumeStatsList;
+import org.apache.rocketmq.common.protocol.body.ConsumerConnection;
+import org.apache.rocketmq.common.protocol.body.ConsumerRunningInfo;
+import org.apache.rocketmq.common.protocol.body.GroupList;
+import org.apache.rocketmq.common.protocol.body.KVTable;
+import org.apache.rocketmq.common.protocol.body.ProducerConnection;
+import org.apache.rocketmq.common.protocol.body.QueryConsumeQueueResponseBody;
+import org.apache.rocketmq.common.protocol.body.QueueTimeSpan;
+import org.apache.rocketmq.common.protocol.body.SubscriptionGroupWrapper;
+import org.apache.rocketmq.common.protocol.body.TopicConfigSerializeWrapper;
+import org.apache.rocketmq.common.protocol.body.TopicList;
+import org.apache.rocketmq.common.protocol.route.TopicRouteData;
+import org.apache.rocketmq.common.subscription.SubscriptionGroupConfig;
+import org.apache.rocketmq.exporter.util.JsonUtil;
+import org.apache.rocketmq.remoting.RemotingClient;
+import org.apache.rocketmq.remoting.exception.RemotingCommandException;
+import org.apache.rocketmq.remoting.exception.RemotingConnectException;
+import org.apache.rocketmq.remoting.exception.RemotingException;
+import org.apache.rocketmq.remoting.exception.RemotingSendRequestException;
+import org.apache.rocketmq.remoting.exception.RemotingTimeoutException;
+import org.apache.rocketmq.remoting.protocol.RemotingCommand;
+import org.apache.rocketmq.tools.admin.MQAdminExt;
+import org.apache.rocketmq.tools.admin.api.MessageTrack;
+import org.joor.Reflect;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import org.springframework.stereotype.Service;
+
+import java.io.UnsupportedEncodingException;
+import java.util.List;
+import java.util.Map;
+import java.util.Properties;
+import java.util.Set;
+
+import static org.apache.rocketmq.remoting.protocol.RemotingSerializable.decode;
+
+@Service
+public class MQAdminExtImpl implements MQAdminExt {
+ private Logger logger = LoggerFactory.getLogger(MQAdminExtImpl.class);
+
+ public MQAdminExtImpl() {
+ }
+
+
+ public PullResult queryMsgByOffset(MessageQueue mq, long offset) throws Exception {
+ return MQAdminInstance.threadLocalMQPullConsumer().pull(mq, "*", offset, 1);
+ }
+
+ @Override
+ public void updateBrokerConfig(String brokerAddr, Properties properties)
+ throws RemotingConnectException, RemotingSendRequestException, RemotingTimeoutException,
+ UnsupportedEncodingException, InterruptedException, MQBrokerException {
+ MQAdminInstance.threadLocalMQAdminExt().updateBrokerConfig(brokerAddr, properties);
+ }
+
+ @Override
+ public void createAndUpdateTopicConfig(String addr, TopicConfig config)
+ throws RemotingException, MQBrokerException, InterruptedException, MQClientException {
+ MQAdminInstance.threadLocalMQAdminExt().createAndUpdateTopicConfig(addr, config);
+ }
+
+ @Override
+ public void createAndUpdateSubscriptionGroupConfig(String addr, SubscriptionGroupConfig config)
+ throws RemotingException, MQBrokerException, InterruptedException, MQClientException {
+ MQAdminInstance.threadLocalMQAdminExt().createAndUpdateSubscriptionGroupConfig(addr, config);
+ }
+
+ @Override
+ public SubscriptionGroupConfig examineSubscriptionGroupConfig(String addr, String group) {
+ RemotingClient remotingClient = MQAdminInstance.threadLocalRemotingClient();
+ RemotingCommand request = RemotingCommand.createRequestCommand(RequestCode.GET_ALL_SUBSCRIPTIONGROUP_CONFIG, null);
+ RemotingCommand response = null;
+ try {
+ response = remotingClient.invokeSync(addr, request, 3000);
+ }
+ catch (Exception err) {
+ throw Throwables.propagate(err);
+ }
+ assert response != null;
+ switch (response.getCode()) {
+ case ResponseCode.SUCCESS: {
+ SubscriptionGroupWrapper subscriptionGroupWrapper = decode(response.getBody(), SubscriptionGroupWrapper.class);
+ return subscriptionGroupWrapper.getSubscriptionGroupTable().get(group);
+ }
+ default:
+ throw Throwables.propagate(new MQBrokerException(response.getCode(), response.getRemark()));
+ }
+ }
+
+ @Override
+ public TopicConfig examineTopicConfig(String addr, String topic) {
+ RemotingClient remotingClient = MQAdminInstance.threadLocalRemotingClient();
+ RemotingCommand request = RemotingCommand.createRequestCommand(RequestCode.GET_ALL_TOPIC_CONFIG, null);
+ RemotingCommand response = null;
+ try {
+ response = remotingClient.invokeSync(addr, request, 3000);
+ }
+ catch (Exception err) {
+ throw Throwables.propagate(err);
+ }
+ switch (response.getCode()) {
+ case ResponseCode.SUCCESS: {
+ TopicConfigSerializeWrapper topicConfigSerializeWrapper = decode(response.getBody(), TopicConfigSerializeWrapper.class);
+ return topicConfigSerializeWrapper.getTopicConfigTable().get(topic);
+ }
+ default:
+ throw Throwables.propagate(new MQBrokerException(response.getCode(), response.getRemark()));
+ }
+ }
+
+ @Override
+ public TopicStatsTable examineTopicStats(String topic)
+ throws RemotingException, MQClientException, InterruptedException, MQBrokerException {
+ return MQAdminInstance.threadLocalMQAdminExt().examineTopicStats(topic);
+ }
+
+ @Override
+ public TopicList fetchAllTopicList() throws RemotingException, MQClientException, InterruptedException {
+ TopicList topicList = MQAdminInstance.threadLocalMQAdminExt().fetchAllTopicList();
+ logger.debug("op=look={}", JsonUtil.obj2String(topicList.getTopicList()));
+ return topicList;
+ }
+
+ @Override
+ public KVTable fetchBrokerRuntimeStats(String brokerAddr)
+ throws RemotingConnectException, RemotingSendRequestException, RemotingTimeoutException,
+ InterruptedException, MQBrokerException {
+ return MQAdminInstance.threadLocalMQAdminExt().fetchBrokerRuntimeStats(brokerAddr);
+ }
+
+ @Override
+ public ConsumeStats examineConsumeStats(String consumerGroup)
+ throws RemotingException, MQClientException, InterruptedException, MQBrokerException {
+ return MQAdminInstance.threadLocalMQAdminExt().examineConsumeStats(consumerGroup);
+ }
+
+ @Override
+ public ConsumeStats examineConsumeStats(String consumerGroup, String topic)
+ throws RemotingException, MQClientException, InterruptedException, MQBrokerException {
+ return MQAdminInstance.threadLocalMQAdminExt().examineConsumeStats(consumerGroup, topic);
+ }
+
+ @Override
+ public ClusterInfo examineBrokerClusterInfo()
+ throws InterruptedException, MQBrokerException, RemotingTimeoutException, RemotingSendRequestException,
+ RemotingConnectException {
+ return MQAdminInstance.threadLocalMQAdminExt().examineBrokerClusterInfo();
+ }
+
+ @Override
+ public TopicRouteData examineTopicRouteInfo(String topic)
+ throws RemotingException, MQClientException, InterruptedException {
+ return MQAdminInstance.threadLocalMQAdminExt().examineTopicRouteInfo(topic);
+ }
+
+ @Override
+ public ConsumerConnection examineConsumerConnectionInfo(String consumerGroup)
+ throws RemotingConnectException, RemotingSendRequestException, RemotingTimeoutException,
+ InterruptedException, MQBrokerException, RemotingException, MQClientException {
+ return MQAdminInstance.threadLocalMQAdminExt().examineConsumerConnectionInfo(consumerGroup);
+ }
+
+ @Override
+ public ProducerConnection examineProducerConnectionInfo(String producerGroup, String topic)
+ throws RemotingException, MQClientException, InterruptedException, MQBrokerException {
+ return MQAdminInstance.threadLocalMQAdminExt().examineProducerConnectionInfo(producerGroup, topic);
+ }
+
+ @Override
+ public List<String> getNameServerAddressList() {
+ return MQAdminInstance.threadLocalMQAdminExt().getNameServerAddressList();
+ }
+
+ @Override
+ public int wipeWritePermOfBroker(String namesrvAddr, String brokerName)
+ throws RemotingCommandException, RemotingConnectException, RemotingSendRequestException,
+ RemotingTimeoutException, InterruptedException, MQClientException {
+ return MQAdminInstance.threadLocalMQAdminExt().wipeWritePermOfBroker(namesrvAddr, brokerName);
+ }
+
+ @Override
+ public void putKVConfig(String namespace, String key, String value) {
+ MQAdminInstance.threadLocalMQAdminExt().putKVConfig(namespace, key, value);
+ }
+
+ @Override
+ public String getKVConfig(String namespace, String key)
+ throws RemotingException, MQClientException, InterruptedException {
+ return MQAdminInstance.threadLocalMQAdminExt().getKVConfig(namespace, key);
+ }
+
+ @Override
+ public KVTable getKVListByNamespace(String namespace)
+ throws RemotingException, MQClientException, InterruptedException {
+ return MQAdminInstance.threadLocalMQAdminExt().getKVListByNamespace(namespace);
+ }
+
+ @Override
+ public void deleteTopicInBroker(Set<String> addrs, String topic)
+ throws RemotingException, MQBrokerException, InterruptedException, MQClientException {
+ logger.info("addrs={} topic={}", JsonUtil.obj2String(addrs), topic);
+ MQAdminInstance.threadLocalMQAdminExt().deleteTopicInBroker(addrs, topic);
+ }
+
+ @Override
+ public void deleteTopicInNameServer(Set<String> addrs, String topic)
+ throws RemotingException, MQBrokerException, InterruptedException, MQClientException {
+ MQAdminInstance.threadLocalMQAdminExt().deleteTopicInNameServer(addrs, topic);
+ }
+
+ @Override
+ public void deleteSubscriptionGroup(String addr, String groupName)
+ throws RemotingException, MQBrokerException, InterruptedException, MQClientException {
+ MQAdminInstance.threadLocalMQAdminExt().deleteSubscriptionGroup(addr, groupName);
+ }
+
+ @Override
+ public void createAndUpdateKvConfig(String namespace, String key, String value)
+ throws RemotingException, MQBrokerException, InterruptedException, MQClientException {
+ MQAdminInstance.threadLocalMQAdminExt().createAndUpdateKvConfig(namespace, key, value);
+ }
+
+ @Override
+ public void deleteKvConfig(String namespace, String key)
+ throws RemotingException, MQBrokerException, InterruptedException, MQClientException {
+ MQAdminInstance.threadLocalMQAdminExt().deleteKvConfig(namespace, key);
+ }
+
+ @Override
+ public List<RollbackStats> resetOffsetByTimestampOld(String consumerGroup, String topic, long timestamp,
+ boolean force) throws RemotingException, MQBrokerException, InterruptedException, MQClientException {
+ return MQAdminInstance.threadLocalMQAdminExt().resetOffsetByTimestampOld(consumerGroup, topic, timestamp, force);
+ }
+
+ @Override
+ public Map<MessageQueue, Long> resetOffsetByTimestamp(String topic, String group, long timestamp,
+ boolean isForce) throws RemotingException, MQBrokerException, InterruptedException, MQClientException {
+ return MQAdminInstance.threadLocalMQAdminExt().resetOffsetByTimestamp(topic, group, timestamp, isForce);
+ }
+
+ @Override
+ public void resetOffsetNew(String consumerGroup, String topic, long timestamp)
+ throws RemotingException, MQBrokerException, InterruptedException, MQClientException {
+ MQAdminInstance.threadLocalMQAdminExt().resetOffsetNew(consumerGroup, topic, timestamp);
+ }
+
+ @Override
+ public Map<String, Map<MessageQueue, Long>> getConsumeStatus(String topic, String group,
+ String clientAddr) throws RemotingException, MQBrokerException, InterruptedException, MQClientException {
+ return MQAdminInstance.threadLocalMQAdminExt().getConsumeStatus(topic, group, clientAddr);
+ }
+
+ @Override
+ public void createOrUpdateOrderConf(String key, String value, boolean isCluster)
+ throws RemotingException, MQBrokerException, InterruptedException, MQClientException {
+ MQAdminInstance.threadLocalMQAdminExt().createOrUpdateOrderConf(key, value, isCluster);
+ }
+
+ @Override
+ public GroupList queryTopicConsumeByWho(String topic)
+ throws RemotingConnectException, RemotingSendRequestException, RemotingTimeoutException,
+ InterruptedException, MQBrokerException, RemotingException, MQClientException {
+ return MQAdminInstance.threadLocalMQAdminExt().queryTopicConsumeByWho(topic);
+ }
+
+ @Override
+ public boolean cleanExpiredConsumerQueue(String cluster)
+ throws RemotingConnectException, RemotingSendRequestException, RemotingTimeoutException, MQClientException,
+ InterruptedException {
+ return MQAdminInstance.threadLocalMQAdminExt().cleanExpiredConsumerQueue(cluster);
+ }
+
+ @Override
+ public boolean cleanExpiredConsumerQueueByAddr(String addr)
+ throws RemotingConnectException, RemotingSendRequestException, RemotingTimeoutException, MQClientException,
+ InterruptedException {
+ return MQAdminInstance.threadLocalMQAdminExt().cleanExpiredConsumerQueueByAddr(addr);
+ }
+
+ @Override
+ public ConsumerRunningInfo getConsumerRunningInfo(String consumerGroup, String clientId, boolean jstack)
+ throws RemotingException, MQClientException, InterruptedException {
+ return MQAdminInstance.threadLocalMQAdminExt().getConsumerRunningInfo(consumerGroup, clientId, jstack);
+ }
+
+ @Override
+ public ConsumeMessageDirectlyResult consumeMessageDirectly(String consumerGroup, String clientId,
+ String msgId) throws RemotingException, MQClientException, InterruptedException, MQBrokerException {
+ return MQAdminInstance.threadLocalMQAdminExt().consumeMessageDirectly(consumerGroup, clientId, msgId);
+ }
+
+ @Override
+ public List<MessageTrack> messageTrackDetail(MessageExt msg)
+ throws RemotingException, MQClientException, InterruptedException, MQBrokerException {
+ return MQAdminInstance.threadLocalMQAdminExt().messageTrackDetail(msg);
+ }
+
+ @Override
+ public void cloneGroupOffset(String srcGroup, String destGroup, String topic, boolean isOffline)
+ throws RemotingException, MQClientException, InterruptedException, MQBrokerException {
+ MQAdminInstance.threadLocalMQAdminExt().cloneGroupOffset(srcGroup, destGroup, topic, isOffline);
+ }
+
+ @Override
+ public void createTopic(String key, String newTopic, int queueNum) throws MQClientException {
+ MQAdminInstance.threadLocalMQAdminExt().createTopic(key, newTopic, queueNum);
+ }
+
+ @Override
+ public void createTopic(String key, String newTopic, int queueNum, int topicSysFlag)
+ throws MQClientException {
+ MQAdminInstance.threadLocalMQAdminExt().createTopic(key, newTopic, queueNum, topicSysFlag);
+ }
+
+ @Override
+ public long searchOffset(MessageQueue mq, long timestamp) throws MQClientException {
+ return MQAdminInstance.threadLocalMQAdminExt().searchOffset(mq, timestamp);
+ }
+
+ @Override
+ public long maxOffset(MessageQueue mq) throws MQClientException {
+ return MQAdminInstance.threadLocalMQAdminExt().maxOffset(mq);
+ }
+
+ @Override
+ public long minOffset(MessageQueue mq) throws MQClientException {
+ return MQAdminInstance.threadLocalMQAdminExt().minOffset(mq);
+ }
+
+ @Override
+ public long earliestMsgStoreTime(MessageQueue mq) throws MQClientException {
+ return MQAdminInstance.threadLocalMQAdminExt().earliestMsgStoreTime(mq);
+ }
+
+ @Override
+ public MessageExt viewMessage(String msgId)
+ throws RemotingException, MQBrokerException, InterruptedException, MQClientException {
+ return MQAdminInstance.threadLocalMQAdminExt().viewMessage(msgId);
+ }
+
+ @Override
+ public QueryResult queryMessage(String topic, String key, int maxNum, long begin, long end)
+ throws MQClientException, InterruptedException {
+ return MQAdminInstance.threadLocalMQAdminExt().queryMessage(topic, key, maxNum, begin, end);
+ }
+
+ @Override
+ @Deprecated
+ public void start() throws MQClientException {
+ throw new IllegalStateException("thisMethod is deprecated.use org.apache.rocketmq.console.aspect.admin.MQAdminAspect instead of this");
+ }
+
+ @Override
+ @Deprecated
+ public void shutdown() {
+ throw new IllegalStateException("thisMethod is deprecated.use org.apache.rocketmq.console.aspect.admin.MQAdminAspect instead of this");
+ }
+
+ // below is 3.2.6->3.5.8 updated
+
+ @Override
+ public List<QueueTimeSpan> queryConsumeTimeSpan(String topic,
+ String group) throws InterruptedException, MQBrokerException, RemotingException, MQClientException {
+ return MQAdminInstance.threadLocalMQAdminExt().queryConsumeTimeSpan(topic, group);
+ }
+
+ //MessageClientIDSetter.getNearlyTimeFromID has bug,so we subtract half a day
+ //next version we will remove it
+ //https://issues.apache.org/jira/browse/ROCKETMQ-111
+ //https://github.com/apache/incubator-rocketmq/pull/69
+ @Override
+ public MessageExt viewMessage(String topic,
+ String msgId) throws RemotingException, MQBrokerException, InterruptedException, MQClientException {
+ logger.info("MessageClientIDSetter.getNearlyTimeFromID(msgId)={} msgId={}", MessageClientIDSetter.getNearlyTimeFromID(msgId), msgId);
+ try {
+ return viewMessage(msgId);
+ }
+ catch (Exception e) {
+ }
+ MQAdminImpl mqAdminImpl = MQAdminInstance.threadLocalMqClientInstance().getMQAdminImpl();
+ QueryResult qr = Reflect.on(mqAdminImpl).call("queryMessage", topic, msgId, 32,
+ MessageClientIDSetter.getNearlyTimeFromID(msgId).getTime() - 1000 * 60 * 60 * 13L, Long.MAX_VALUE, true).get();
+ if (qr != null && qr.getMessageList() != null && qr.getMessageList().size() > 0) {
+ return qr.getMessageList().get(0);
+ }
+ else {
+ return null;
+ }
+ }
+
+ @Override
+ public ConsumeMessageDirectlyResult consumeMessageDirectly(String consumerGroup, String clientId, String topic,
+ String msgId) throws RemotingException, MQClientException, InterruptedException, MQBrokerException {
+ return MQAdminInstance.threadLocalMQAdminExt().consumeMessageDirectly(consumerGroup, clientId, topic, msgId);
+ }
+
+ @Override
+ public Properties getBrokerConfig(
+ String brokerAddr) throws RemotingConnectException, RemotingSendRequestException, RemotingTimeoutException, UnsupportedEncodingException, InterruptedException, MQBrokerException {
+ return MQAdminInstance.threadLocalMQAdminExt().getBrokerConfig(brokerAddr);
+ }
+
+ @Override
+ public TopicList fetchTopicsByCLuster(
+ String clusterName) throws RemotingException, MQClientException, InterruptedException {
+ return MQAdminInstance.threadLocalMQAdminExt().fetchTopicsByCLuster(clusterName);
+ }
+
+ @Override
+ public boolean cleanUnusedTopic(
+ String cluster) throws RemotingConnectException, RemotingSendRequestException, RemotingTimeoutException, MQClientException, InterruptedException {
+ return MQAdminInstance.threadLocalMQAdminExt().cleanUnusedTopic(cluster);
+ }
+
+ @Override
+ public boolean cleanUnusedTopicByAddr(
+ String addr) throws RemotingConnectException, RemotingSendRequestException, RemotingTimeoutException, MQClientException, InterruptedException {
+ return MQAdminInstance.threadLocalMQAdminExt().cleanUnusedTopicByAddr(addr);
+ }
+
+ @Override
+ public BrokerStatsData viewBrokerStatsData(String brokerAddr, String statsName,
+ String statsKey) throws RemotingConnectException, RemotingSendRequestException, RemotingTimeoutException, MQClientException, InterruptedException {
+ return MQAdminInstance.threadLocalMQAdminExt().viewBrokerStatsData(brokerAddr, statsName, statsKey);
+ }
+
+ @Override
+ public Set<String> getClusterList(
+ String topic) throws RemotingConnectException, RemotingSendRequestException, RemotingTimeoutException, MQClientException, InterruptedException {
+ return MQAdminInstance.threadLocalMQAdminExt().getClusterList(topic);
+ }
+
+ @Override
+ public ConsumeStatsList fetchConsumeStatsInBroker(String brokerAddr, boolean isOrder,
+ long timeoutMillis) throws RemotingConnectException, RemotingSendRequestException, RemotingTimeoutException, MQClientException, InterruptedException {
+ return MQAdminInstance.threadLocalMQAdminExt().fetchConsumeStatsInBroker(brokerAddr, isOrder, timeoutMillis);
+ }
+
+ @Override
+ public Set<String> getTopicClusterList(
+ String topic) throws InterruptedException, MQBrokerException, MQClientException, RemotingException {
+ return MQAdminInstance.threadLocalMQAdminExt().getTopicClusterList(topic);
+ }
+
+ @Override
+ public SubscriptionGroupWrapper getAllSubscriptionGroup(String brokerAddr,
+ long timeoutMillis) throws InterruptedException, RemotingTimeoutException, RemotingSendRequestException, RemotingConnectException, MQBrokerException {
+ return MQAdminInstance.threadLocalMQAdminExt().getAllSubscriptionGroup(brokerAddr, timeoutMillis);
+ }
+
+ @Override
+ public TopicConfigSerializeWrapper getAllTopicGroup(String brokerAddr,
+ long timeoutMillis) throws InterruptedException, RemotingTimeoutException, RemotingSendRequestException, RemotingConnectException, MQBrokerException {
+ return MQAdminInstance.threadLocalMQAdminExt().getAllTopicGroup(brokerAddr, timeoutMillis);
+ }
+
+ @Override
+ public void updateConsumeOffset(String brokerAddr, String consumeGroup, MessageQueue mq,
+ long offset) throws RemotingException, InterruptedException, MQBrokerException {
+ MQAdminInstance.threadLocalMQAdminExt().updateConsumeOffset(brokerAddr, consumeGroup, mq, offset);
+ }
+
+ // 4.0.0 added
+ @Override public void updateNameServerConfig(Properties properties,
+ List<String> list) throws InterruptedException, RemotingConnectException, UnsupportedEncodingException, RemotingSendRequestException, RemotingTimeoutException, MQClientException, MQBrokerException {
+
+ }
+
+ @Override public Map<String, Properties> getNameServerConfig(
+ List<String> list) throws InterruptedException, RemotingTimeoutException, RemotingSendRequestException, RemotingConnectException, MQClientException, UnsupportedEncodingException {
+ return null;
+ }
+
+ @Override public QueryConsumeQueueResponseBody queryConsumeQueue(String brokerAddr, String topic,
+ int queueId, long index, int count,
+ String consumerGroup) throws InterruptedException, RemotingTimeoutException, RemotingSendRequestException, RemotingConnectException, MQClientException {
+ return null;
+ }
+}
diff --git a/rocketmq-prometheus-exporter/src/main/java/org/apache/rocketmq/exporter/service/client/MQAdminInstance.java b/rocketmq-prometheus-exporter/src/main/java/org/apache/rocketmq/exporter/service/client/MQAdminInstance.java
new file mode 100644
index 0000000..6994c23
--- /dev/null
+++ b/rocketmq-prometheus-exporter/src/main/java/org/apache/rocketmq/exporter/service/client/MQAdminInstance.java
@@ -0,0 +1,119 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.rocketmq.exporter.service.client;
+
+import org.apache.rocketmq.client.consumer.DefaultMQPullConsumer;
+import org.apache.rocketmq.client.exception.MQClientException;
+import org.apache.rocketmq.client.impl.MQClientAPIImpl;
+import org.apache.rocketmq.client.impl.factory.MQClientInstance;
+import org.apache.rocketmq.common.MixAll;
+import org.apache.rocketmq.remoting.RemotingClient;
+import org.apache.rocketmq.tools.admin.DefaultMQAdminExt;
+import org.apache.rocketmq.tools.admin.DefaultMQAdminExtImpl;
+import org.apache.rocketmq.tools.admin.MQAdminExt;
+import org.joor.Reflect;
+
+import static org.apache.rocketmq.common.MixAll.TOOLS_CONSUMER_GROUP;
+
+
+public class MQAdminInstance {
+
+ private static final ThreadLocal<DefaultMQAdminExt> MQ_ADMIN_EXT_THREAD_LOCAL = new ThreadLocal<DefaultMQAdminExt>();
+
+ private static final ThreadLocal<DefaultMQPullConsumer> MQ_PULL_CONSUMER_THREAD_LOCAL = new ThreadLocal<DefaultMQPullConsumer>();
+
+ private static final ThreadLocal<Integer> INIT_COUNTER = new ThreadLocal<Integer>();
+
+ public static MQAdminExt threadLocalMQAdminExt() {
+ DefaultMQAdminExt defaultMQAdminExt = MQ_ADMIN_EXT_THREAD_LOCAL.get();
+ if (defaultMQAdminExt == null) {
+ throw new IllegalStateException("defaultMQAdminExt should be init before you get this");
+ }
+ return defaultMQAdminExt;
+ }
+
+
+ public static DefaultMQPullConsumer threadLocalMQPullConsumer() {
+ DefaultMQPullConsumer pullConsumer = MQ_PULL_CONSUMER_THREAD_LOCAL.get();
+ if (pullConsumer == null) {
+ throw new IllegalStateException("pullConsumer should be init before you get this");
+ }
+ return pullConsumer;
+ }
+
+
+ public static RemotingClient threadLocalRemotingClient() {
+ MQClientInstance mqClientInstance = threadLocalMqClientInstance();
+ MQClientAPIImpl mQClientAPIImpl = Reflect.on(mqClientInstance).get("mQClientAPIImpl");
+ return Reflect.on(mQClientAPIImpl).get("remotingClient");
+ }
+
+ public static MQClientInstance threadLocalMqClientInstance() {
+ DefaultMQAdminExtImpl defaultMQAdminExtImpl = Reflect.on(MQAdminInstance.threadLocalMQAdminExt()).get("defaultMQAdminExtImpl");
+ return Reflect.on(defaultMQAdminExtImpl).get("mqClientInstance");
+ }
+
+ public static void initMQAdminInstance(long timeoutMillis) throws MQClientException {
+ Integer nowCount = INIT_COUNTER.get();
+ if (nowCount == null) {
+ DefaultMQAdminExt defaultMQAdminExt;
+ if (timeoutMillis > 0) {
+ defaultMQAdminExt = new DefaultMQAdminExt(timeoutMillis);
+ }
+ else {
+ defaultMQAdminExt = new DefaultMQAdminExt();
+ }
+ defaultMQAdminExt.setInstanceName("admin-" + Long.toString(System.currentTimeMillis()));
+ defaultMQAdminExt.start();
+ MQ_ADMIN_EXT_THREAD_LOCAL.set(defaultMQAdminExt);
+
+
+ DefaultMQPullConsumer pullConsumer;
+ pullConsumer = new DefaultMQPullConsumer(TOOLS_CONSUMER_GROUP,null);
+ pullConsumer.setInstanceName("consumer-" + Long.toString(System.currentTimeMillis()));
+ pullConsumer.setNamesrvAddr(System.getProperty(MixAll.NAMESRV_ADDR_PROPERTY, System.getenv(MixAll.NAMESRV_ADDR_ENV)));
+ pullConsumer.start();
+ pullConsumer.getDefaultMQPullConsumerImpl().getPullAPIWrapper().setConnectBrokerByUser(true);
+
+ MQ_PULL_CONSUMER_THREAD_LOCAL.set(pullConsumer);
+ INIT_COUNTER.set(1);
+ }
+ else {
+ INIT_COUNTER.set(nowCount + 1);
+ }
+
+ }
+
+ public static void destroyMQAdminInstance() {
+ Integer nowCount = INIT_COUNTER.get() - 1;
+ if (nowCount > 0) {
+ INIT_COUNTER.set(nowCount);
+ return;
+ }
+ MQAdminExt mqAdminExt = MQ_ADMIN_EXT_THREAD_LOCAL.get();
+ if (mqAdminExt != null) {
+ DefaultMQPullConsumer consumer = MQ_PULL_CONSUMER_THREAD_LOCAL.get();
+ if (consumer != null) {
+ consumer.shutdown();
+ MQ_PULL_CONSUMER_THREAD_LOCAL.remove();
+ }
+ mqAdminExt.shutdown();
+ MQ_ADMIN_EXT_THREAD_LOCAL.remove();
+ INIT_COUNTER.remove();
+ }
+ }
+}
diff --git a/rocketmq-prometheus-exporter/src/main/java/org/apache/rocketmq/exporter/service/impl/RMQMetricsServiceImpl.java b/rocketmq-prometheus-exporter/src/main/java/org/apache/rocketmq/exporter/service/impl/RMQMetricsServiceImpl.java
new file mode 100644
index 0000000..5dd008b
--- /dev/null
+++ b/rocketmq-prometheus-exporter/src/main/java/org/apache/rocketmq/exporter/service/impl/RMQMetricsServiceImpl.java
@@ -0,0 +1,53 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.rocketmq.exporter.service.impl;
+
+import io.prometheus.client.CollectorRegistry;
+import io.prometheus.client.exporter.common.TextFormat;
+import org.apache.rocketmq.exporter.collector.RMQMetricsCollector;
+import org.apache.rocketmq.exporter.service.AbstractCommonService;
+import org.apache.rocketmq.exporter.service.RMQMetricsService;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import org.springframework.stereotype.Service;
+
+import java.io.IOException;
+import java.io.StringWriter;
+
+@Service
+public class RMQMetricsServiceImpl extends AbstractCommonService implements RMQMetricsService {
+
+ private Logger logger = LoggerFactory.getLogger(RMQMetricsServiceImpl.class);
+
+ private CollectorRegistry registry = new CollectorRegistry();
+
+ private final RMQMetricsCollector rmqMetricsCollector;
+
+
+ public RMQMetricsCollector getCollector() {
+ return rmqMetricsCollector;
+ }
+
+ public RMQMetricsServiceImpl() {
+ rmqMetricsCollector = new RMQMetricsCollector();
+ rmqMetricsCollector.register(registry);
+ }
+ public void Metrics(StringWriter writer) throws IOException {
+ TextFormat.write004(writer, registry.metricFamilySamples());
+ logger.info(writer.toString());
+ }
+}
diff --git a/rocketmq-prometheus-exporter/src/main/java/org/apache/rocketmq/exporter/task/MetricsCollectTask.java b/rocketmq-prometheus-exporter/src/main/java/org/apache/rocketmq/exporter/task/MetricsCollectTask.java
new file mode 100644
index 0000000..504ac88
--- /dev/null
+++ b/rocketmq-prometheus-exporter/src/main/java/org/apache/rocketmq/exporter/task/MetricsCollectTask.java
@@ -0,0 +1,300 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.rocketmq.exporter.task;
+
+import com.google.common.base.Throwables;
+import org.apache.rocketmq.client.consumer.PullResult;
+import org.apache.rocketmq.client.consumer.PullStatus;
+import org.apache.rocketmq.common.MixAll;
+import org.apache.rocketmq.common.admin.ConsumeStats;
+import org.apache.rocketmq.common.admin.OffsetWrapper;
+import org.apache.rocketmq.common.admin.TopicOffset;
+import org.apache.rocketmq.common.admin.TopicStatsTable;
+import org.apache.rocketmq.common.message.MessageQueue;
+import org.apache.rocketmq.common.protocol.body.BrokerStatsData;
+import org.apache.rocketmq.common.protocol.body.ClusterInfo;
+import org.apache.rocketmq.common.protocol.body.GroupList;
+import org.apache.rocketmq.common.protocol.body.TopicList;
+import org.apache.rocketmq.common.protocol.route.BrokerData;
+import org.apache.rocketmq.common.protocol.route.TopicRouteData;
+import org.apache.rocketmq.exporter.aspect.admin.annotation.MultiMQAdminCmdMethod;
+import org.apache.rocketmq.exporter.config.RMQConfigure;
+import org.apache.rocketmq.exporter.service.RMQMetricsService;
+import org.apache.rocketmq.exporter.service.client.MQAdminExtImpl;
+import org.apache.rocketmq.store.stats.BrokerStatsManager;
+import org.apache.rocketmq.tools.admin.MQAdminExt;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import org.springframework.scheduling.annotation.Scheduled;
+import org.springframework.stereotype.Component;
+
+import javax.annotation.Resource;
+import java.util.Date;
+import java.util.HashMap;
+import java.util.Map;
+import java.util.Set;
+
+@Component
+public class MetricsCollectTask {
+
+ @Resource
+ private MQAdminExt mqAdminExt;
+ @Resource
+ private RMQConfigure rmqConfigure;
+
+ @Resource
+ private RMQMetricsService metricsService;
+
+ private final static Logger log = LoggerFactory.getLogger(MetricsCollectTask.class);
+
+ @Scheduled(cron = "15 0/1 * * * ?")
+ @MultiMQAdminCmdMethod(timeoutMillis = 5000)
+ public void collectOffset() {
+ if (!rmqConfigure.isEnableCollect()) {
+ return;
+ }
+ Date date = new Date();
+ try {
+ TopicList topicList = mqAdminExt.fetchAllTopicList();
+ Set<String> topicSet = topicList.getTopicList();
+ for (String topic : topicSet) {
+ if (topic.startsWith(MixAll.RETRY_GROUP_TOPIC_PREFIX) || topic.startsWith(MixAll.DLQ_GROUP_TOPIC_PREFIX)) {
+ continue;
+ }
+ String clusterName = null;
+ ClusterInfo clusterInfo = mqAdminExt.examineBrokerClusterInfo();
+ Set<Map.Entry<String, BrokerData>> clusterEntries = clusterInfo.getBrokerAddrTable().entrySet();
+ for (Map.Entry<String, BrokerData> clusterEntry : clusterEntries) {
+ clusterName = clusterEntry.getValue().getCluster();
+ break;
+ }
+ if (clusterName != null) {
+ HashMap<String,Long> brokerOffsetMap = new HashMap<>();
+ TopicStatsTable topicStatus = mqAdminExt.examineTopicStats(topic);
+ Set<Map.Entry<MessageQueue, TopicOffset>> topicStatusEntries = topicStatus.getOffsetTable().entrySet();
+ for (Map.Entry<MessageQueue, TopicOffset> topicStatusEntry : topicStatusEntries) {
+ MessageQueue q = topicStatusEntry.getKey();
+ TopicOffset offset = topicStatusEntry.getValue();
+ if (brokerOffsetMap.containsKey(q.getBrokerName())) {
+ brokerOffsetMap.put(q.getBrokerName(),brokerOffsetMap.get(q.getBrokerName()) + offset.getMaxOffset());
+ }
+ else {
+ brokerOffsetMap.put(q.getBrokerName(),offset.getMaxOffset());
+ }
+ }
+ Set<Map.Entry<String, Long>> brokerOffsetEntries = brokerOffsetMap.entrySet();
+ for (Map.Entry<String, Long> brokerOffsetEntry : brokerOffsetEntries) {
+ metricsService.getCollector().AddTopicOffsetMetric(clusterName,brokerOffsetEntry.getKey(), topic, brokerOffsetEntry.getValue());
+ }
+ }
+
+ HashMap<String,Long> consumeOffsetMap = new HashMap<>();
+ GroupList groupList = mqAdminExt.queryTopicConsumeByWho(topic);
+ if (groupList != null && !groupList.getGroupList().isEmpty()) {
+ for (String group : groupList.getGroupList()) {
+ ConsumeStats consumeStatus = mqAdminExt.examineConsumeStats(group,topic);
+ Set<Map.Entry<MessageQueue, OffsetWrapper>> consumeStatusEntries = consumeStatus.getOffsetTable().entrySet();
+ for (Map.Entry<MessageQueue, OffsetWrapper> consumeStatusEntry : consumeStatusEntries) {
+ MessageQueue q = consumeStatusEntry.getKey();
+ OffsetWrapper offset = consumeStatusEntry.getValue();
+ if (consumeOffsetMap.containsKey(q.getBrokerName())) {
+ consumeOffsetMap.put(q.getBrokerName(), consumeOffsetMap.get(q.getBrokerName()) + offset.getConsumerOffset());
+ }
+ else {
+ consumeOffsetMap.put(q.getBrokerName(), offset.getConsumerOffset());
+ }
+ }
+ Set<Map.Entry<String, Long>> consumeOffsetEntries = consumeOffsetMap.entrySet();
+ for (Map.Entry<String, Long> consumeOffsetEntry : consumeOffsetEntries) {
+ metricsService.getCollector().AddGroupOffsetMetric(clusterName,consumeOffsetEntry.getKey(), topic, group, consumeOffsetEntry.getValue());
+ }
+ }
+ }
+ }
+ } catch (Exception e) {
+ log.info("error is " + e.getMessage());
+ }
+ }
+
+ @Scheduled(cron = "15 0/1 * * * ?")
+ @MultiMQAdminCmdMethod(timeoutMillis = 5000)
+ public void collectTopic() {
+ if (!rmqConfigure.isEnableCollect()) {
+ return;
+ }
+ Date date = new Date();
+ try {
+ TopicList topicList = mqAdminExt.fetchAllTopicList();
+ Set<String> topicSet = topicList.getTopicList();
+ for (String topic : topicSet) {
+ if (topic.startsWith(MixAll.RETRY_GROUP_TOPIC_PREFIX) || topic.startsWith(MixAll.DLQ_GROUP_TOPIC_PREFIX)) {
+ continue;
+ }
+ TopicRouteData topicRouteData = mqAdminExt.examineTopicRouteInfo(topic);
+ GroupList groupList = mqAdminExt.queryTopicConsumeByWho(topic);
+
+ for (BrokerData bd : topicRouteData.getBrokerDatas()) {
+ String masterAddr = bd.getBrokerAddrs().get(MixAll.MASTER_ID);
+ if (masterAddr != null) {
+ try {
+ BrokerStatsData bsd = null;
+ try {
+ bsd = mqAdminExt.viewBrokerStatsData(masterAddr, BrokerStatsManager.TOPIC_PUT_NUMS, topic);
+ metricsService.getCollector().AddTopicPutNumsMetric(bd.getCluster(), bd.getBrokerName(), topic, bsd.getStatsMinute().getTps());
+ }
+ catch (Exception e) {
+ log.info("error is " + e.getMessage());
+ }
+ try {
+ bsd = mqAdminExt.viewBrokerStatsData(masterAddr, BrokerStatsManager.TOPIC_PUT_SIZE, topic);
+ metricsService.getCollector().AddTopicPutSizeMetric(bd.getCluster(), bd.getBrokerName(), topic, bsd.getStatsMinute().getTps());
+ }
+ catch (Exception e) {
+ log.info("error is " + e.getMessage());
+ }
+ } catch (Exception e) {
+ log.info("error is " + e.getMessage());
+ }
+ }
+ }
+ if (groupList != null && !groupList.getGroupList().isEmpty()) {
+ for (String group : groupList.getGroupList()) {
+ for (BrokerData bd : topicRouteData.getBrokerDatas()) {
+ String masterAddr = bd.getBrokerAddrs().get(MixAll.MASTER_ID);
+ if (masterAddr != null) {
+ try {
+ String statsKey = String.format("%s@%s", topic, group);
+ BrokerStatsData bsd = null;
+ try {
+ bsd = mqAdminExt.viewBrokerStatsData(masterAddr, BrokerStatsManager.GROUP_GET_NUMS, statsKey);
+ metricsService.getCollector().AddGroupGetNumsMetric(bd.getCluster(), bd.getBrokerName(), topic, group, bsd.getStatsMinute().getTps());
+ } catch (Exception e) {
+ log.info("error is " + e.getMessage());
+ }
+ try {
+ bsd = mqAdminExt.viewBrokerStatsData(masterAddr, BrokerStatsManager.GROUP_GET_SIZE, statsKey);
+ metricsService.getCollector().AddGroupGetSizeMetric(bd.getCluster(), bd.getBrokerName(), topic, group, bsd.getStatsMinute().getTps());
+ } catch (Exception e) {
+ log.info("error is " + e.getMessage());
+ }
+ try {
+
+ bsd = mqAdminExt.viewBrokerStatsData(masterAddr, BrokerStatsManager.SNDBCK_PUT_NUMS, statsKey);
+ metricsService.getCollector().AddsendBackNumsMetric(bd.getCluster(), bd.getBrokerName(), topic, group, bsd.getStatsMinute().getTps());
+ } catch (Exception e) {
+ log.info("error is " + e.getMessage());
+ }
+ try {
+ collectLatencyMetrcisInner(topic, group, masterAddr, bd);
+ } catch (Exception e) {
+ log.info("error is " + e.getMessage());
+ }
+ } catch (Exception e) {
+ log.info("error is " + e.getMessage());
+ }
+ }
+ }
+ }
+ }
+ }
+ }
+ catch (Exception err) {
+ throw Throwables.propagate(err);
+ }
+ }
+
+ @Scheduled(cron = "15 0/1 * * * ?")
+ @MultiMQAdminCmdMethod(timeoutMillis = 5000)
+ public void collectBroker() {
+ if (!rmqConfigure.isEnableCollect()) {
+ return;
+ }
+ try {
+ Date date = new Date();
+ ClusterInfo clusterInfo = mqAdminExt.examineBrokerClusterInfo();
+ Set<Map.Entry<String, BrokerData>> clusterEntries = clusterInfo.getBrokerAddrTable().entrySet();
+ for (Map.Entry<String, BrokerData> clusterEntry : clusterEntries) {
+ String masterAddr = clusterEntry.getValue().getBrokerAddrs().get(MixAll.MASTER_ID);
+ if (masterAddr != null) {
+ try {
+ BrokerStatsData bsd = null;
+ try {
+ bsd = mqAdminExt.viewBrokerStatsData(masterAddr, BrokerStatsManager.BROKER_PUT_NUMS,clusterEntry.getValue().getCluster());
+ metricsService.getCollector().AddBrokerPutNumsMetric(clusterEntry.getValue().getCluster(), clusterEntry.getValue().getBrokerName(), bsd.getStatsMinute().getTps());
+ }
+ catch (Exception e) {
+ log.info("error is " + e.getMessage());
+ }
+ try {
+ bsd = mqAdminExt.viewBrokerStatsData(masterAddr, BrokerStatsManager.BROKER_GET_NUMS, clusterEntry.getValue().getCluster());
+ metricsService.getCollector().AddBrokerGetNumsMetric(clusterEntry.getValue().getCluster(), clusterEntry.getValue().getBrokerName(), bsd.getStatsMinute().getTps());
+ }
+ catch (Exception e) {
+ log.info("error is " + e.getMessage());
+ }
+ } catch (Exception e) {
+ log.info("error is " + e.getMessage());
+ }
+ }
+ }
+ }
+ catch (Exception err) {
+ throw Throwables.propagate(err);
+ }
+ }
+ private void collectLatencyMetrcisInner(String topic,String group,String masterAddr, BrokerData bd) throws Exception {
+ long maxLagTime = 0;
+ String statsKey;
+ BrokerStatsData bsd = null;
+ ConsumeStats consumeStatus = mqAdminExt.examineConsumeStats(group, topic);
+ Set<Map.Entry<MessageQueue, OffsetWrapper>> consumeStatusEntries = consumeStatus.getOffsetTable().entrySet();
+ for (Map.Entry<MessageQueue, OffsetWrapper> consumeStatusEntry : consumeStatusEntries) {
+ MessageQueue q = consumeStatusEntry.getKey();
+ OffsetWrapper offset = consumeStatusEntry.getValue();
+ int queueId = q.getQueueId();
+ statsKey = String.format("%d@%s@%s", queueId, topic, group);
+ try {
+ bsd = mqAdminExt.viewBrokerStatsData(masterAddr, BrokerStatsManager.GROUP_GET_LATENCY, statsKey);
+ metricsService.getCollector().AddGroupGetLatencyMetric(bd.getCluster(), bd.getBrokerName(), topic, group, String.format("%d", queueId), bsd.getStatsMinute().getTps());
+ } catch (Exception e) {
+ log.info("error is " + e.getMessage());
+ }
+ MQAdminExtImpl mqAdminImpl = (MQAdminExtImpl) mqAdminExt;
+ PullResult consumePullResult = mqAdminImpl.queryMsgByOffset(q, offset.getConsumerOffset());
+ long lagTime = 0;
+ if (consumePullResult != null && consumePullResult.getPullStatus() == PullStatus.FOUND) {
+ lagTime = System.currentTimeMillis() - consumePullResult.getMsgFoundList().get(0).getStoreTimestamp();
+ if (offset.getBrokerOffset() == offset.getConsumerOffset()) {
+ lagTime = 0;
+ }
+ } else if (consumePullResult.getPullStatus() == PullStatus.NO_MATCHED_MSG) {
+ lagTime = 0;
+ } else if (consumePullResult.getPullStatus() == PullStatus.OFFSET_ILLEGAL) {
+ PullResult pullResult = mqAdminImpl.queryMsgByOffset(q, consumePullResult.getMinOffset());
+ if (pullResult != null && pullResult.getPullStatus() == PullStatus.FOUND) {
+ lagTime = System.currentTimeMillis() - consumePullResult.getMsgFoundList().get(0).getStoreTimestamp();
+ }
+ } else {
+ lagTime = 0;
+ }
+ if (lagTime > maxLagTime) {
+ maxLagTime = lagTime;
+ }
+ }
+ metricsService.getCollector().AddGroupGetLatencyByStoreTimeMetric(bd.getCluster(), bd.getBrokerName(), topic, group, maxLagTime);
+ }
+}
diff --git a/rocketmq-prometheus-exporter/src/main/java/org/apache/rocketmq/exporter/util/JsonUtil.java b/rocketmq-prometheus-exporter/src/main/java/org/apache/rocketmq/exporter/util/JsonUtil.java
new file mode 100644
index 0000000..a80e998
--- /dev/null
+++ b/rocketmq-prometheus-exporter/src/main/java/org/apache/rocketmq/exporter/util/JsonUtil.java
@@ -0,0 +1,157 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.rocketmq.exporter.util;
+
+import com.fasterxml.jackson.annotation.JsonInclude;
+import com.fasterxml.jackson.core.type.TypeReference;
+import com.fasterxml.jackson.databind.DeserializationFeature;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import com.fasterxml.jackson.databind.SerializationFeature;
+import com.fasterxml.jackson.databind.ser.impl.SimpleFilterProvider;
+import com.google.common.base.Strings;
+import com.google.common.base.Throwables;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.IOException;
+import java.io.Writer;
+import java.text.SimpleDateFormat;
+import java.util.Map;
+
+@SuppressWarnings("unchecked")
+public class JsonUtil {
+
+ private static Logger logger = LoggerFactory.getLogger(JsonUtil.class);
+ private static ObjectMapper objectMapper = new ObjectMapper();
+
+ private JsonUtil() {
+ }
+
+ static {
+ objectMapper.configure(SerializationFeature.FAIL_ON_EMPTY_BEANS, false);
+ objectMapper.configure(SerializationFeature.WRITE_ENUMS_USING_TO_STRING, true);
+ objectMapper.configure(DeserializationFeature.ACCEPT_SINGLE_VALUE_AS_ARRAY, true);
+ objectMapper.disable(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES);
+ objectMapper.setFilters(new SimpleFilterProvider().setFailOnUnknownId(false));
+ objectMapper.setSerializationInclusion(JsonInclude.Include.NON_EMPTY);
+ objectMapper.setDateFormat(new SimpleDateFormat("yyyy-MM-dd HH:mm:ss"));
+ }
+
+ public static void writeValue(Writer writer, Object obj) {
+ try {
+ objectMapper.writeValue(writer, obj);
+ }
+ catch (IOException e) {
+ Throwables.propagateIfPossible(e);
+ }
+ }
+
+ public static <T> String obj2String(T src) {
+ if (src == null) {
+ return null;
+ }
+
+ try {
+ return src instanceof String ? (String)src : objectMapper.writeValueAsString(src);
+ }
+ catch (Exception e) {
+ logger.error("Parse Object to String error src=" + src, e);
+ return null;
+ }
+ }
+
+ public static <T> byte[] obj2Byte(T src) {
+ if (src == null) {
+ return null;
+ }
+
+ try {
+ return src instanceof byte[] ? (byte[])src : objectMapper.writeValueAsBytes(src);
+ }
+ catch (Exception e) {
+ logger.error("Parse Object to byte[] error", e);
+ return null;
+ }
+ }
+
+ public static <T> T string2Obj(String str, Class<T> clazz) {
+ if (Strings.isNullOrEmpty(str) || clazz == null) {
+ return null;
+ }
+ str = escapesSpecialChar(str);
+ try {
+ return clazz.equals(String.class) ? (T)str : objectMapper.readValue(str, clazz);
+ }
+ catch (Exception e) {
+ logger.error("Parse String to Object error\nString: {}\nClass<T>: {}\nError: {}", str, clazz.getName(), e);
+ return null;
+ }
+ }
+
+ public static <T> T byte2Obj(byte[] bytes, Class<T> clazz) {
+ if (bytes == null || clazz == null) {
+ return null;
+ }
+ try {
+ return clazz.equals(byte[].class) ? (T)bytes : objectMapper.readValue(bytes, clazz);
+ }
+ catch (Exception e) {
+ logger.error("Parse byte[] to Object error\nbyte[]: {}\nClass<T>: {}\nError: {}", bytes, clazz.getName(), e);
+ return null;
+ }
+ }
+
+ public static <T> T string2Obj(String str, TypeReference<T> typeReference) {
+ if (Strings.isNullOrEmpty(str) || typeReference == null) {
+ return null;
+ }
+ str = escapesSpecialChar(str);
+ try {
+ return (T)(typeReference.getType().equals(String.class) ? str : objectMapper.readValue(str, typeReference));
+ }
+ catch (Exception e) {
+ logger.error("Parse String to Object error\nString: {}\nTypeReference<T>: {}\nError: {}", str,
+ typeReference.getType(), e);
+ return null;
+ }
+ }
+
+ public static <T> T byte2Obj(byte[] bytes, TypeReference<T> typeReference) {
+ if (bytes == null || typeReference == null) {
+ return null;
+ }
+ try {
+ return (T)(typeReference.getType().equals(byte[].class) ? bytes : objectMapper.readValue(bytes,
+ typeReference));
+ }
+ catch (Exception e) {
+ logger.error("Parse byte[] to Object error\nbyte[]: {}\nTypeReference<T>: {}\nError: {}", bytes,
+ typeReference.getType(), e);
+ return null;
+ }
+ }
+
+ public static <T> T map2Obj(Map<String, String> map, Class<T> clazz) {
+ String str = obj2String(map);
+ return string2Obj(str, clazz);
+ }
+
+ private static String escapesSpecialChar(String str) {
+ return str.replace("\n", "\\n").replace("\r", "\\r");
+ }
+}
diff --git a/rocketmq-prometheus-exporter/src/main/resources/application.properties b/rocketmq-prometheus-exporter/src/main/resources/application.properties
new file mode 100644
index 0000000..ad1d563
--- /dev/null
+++ b/rocketmq-prometheus-exporter/src/main/resources/application.properties
@@ -0,0 +1,14 @@
+server.port=5557
+
+spring.application.name=rocketmq-exporter
+spring.http.encoding.charset=UTF-8
+spring.http.encoding.enabled=true
+spring.http.encoding.force=true
+logging.config=classpath:logback.xml
+#if this value is empty,use env value rocketmq.config.namesrvAddr NAMESRV_ADDR
+rocketmq.config.namesrvAddr=192.168.0.48:9876
+
+
+rocketmq.config.enableCollect=true
+rocketmq.config.webTelemetryPath=/metrics
+rocketmq.config.rocketmqVersion=4_3_2
\ No newline at end of file
diff --git a/rocketmq-prometheus-exporter/src/main/resources/logback.xml b/rocketmq-prometheus-exporter/src/main/resources/logback.xml
new file mode 100644
index 0000000..b5291d3
--- /dev/null
+++ b/rocketmq-prometheus-exporter/src/main/resources/logback.xml
@@ -0,0 +1,33 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<configuration>
+ <appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
+ <encoder charset="UTF-8">
+ <pattern>[%d{yyyy-MM-dd HH:mm:ss.SSS}] %5p %m%n</pattern>
+ </encoder>
+ </appender>
+
+ <appender name="FILE"
+ class="ch.qos.logback.core.rolling.RollingFileAppender">
+ <file>${user.home}/logs/exporterlogs/rocketmq-exporter.log</file>
+ <append>true</append>
+ <rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
+ <fileNamePattern>${user.home}/logs/exporterlogs/rocketmq-exporter-%d{yyyy-MM-dd}.%i.log
+ </fileNamePattern>
+ <timeBasedFileNamingAndTriggeringPolicy
+ class="ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP">
+ <maxFileSize>104857600</maxFileSize>
+ </timeBasedFileNamingAndTriggeringPolicy>
+ <MaxHistory>10</MaxHistory>
+ </rollingPolicy>
+ <encoder>
+ <pattern>[%d{yyyy-MM-dd HH:mm:ss.SSS}] %5p %m%n</pattern>
+ <charset class="java.nio.charset.Charset">UTF-8</charset>
+ </encoder>
+ </appender>
+
+ <root level="INFO">
+ <appender-ref ref="STDOUT" />
+ <appender-ref ref="FILE" />
+ </root>
+
+</configuration>
\ No newline at end of file
diff --git a/rocketmq-prometheus-exporter/style/copyright/Apache.xml b/rocketmq-prometheus-exporter/style/copyright/Apache.xml
new file mode 100644
index 0000000..e3e3dec
--- /dev/null
+++ b/rocketmq-prometheus-exporter/style/copyright/Apache.xml
@@ -0,0 +1,23 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+ -->
+
+<component name="CopyrightManager">
+ <copyright>
+ <option name="myName" value="Apache" />
+ <option name="notice" value="Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this work for additional information regarding copyright ownership. The ASF licenses this file to You under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.a [...]
+ </copyright>
+</component>
\ No newline at end of file
diff --git a/rocketmq-prometheus-exporter/style/copyright/profiles_settings.xml b/rocketmq-prometheus-exporter/style/copyright/profiles_settings.xml
new file mode 100644
index 0000000..747c7e2
--- /dev/null
+++ b/rocketmq-prometheus-exporter/style/copyright/profiles_settings.xml
@@ -0,0 +1,64 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+ -->
+
+<component name="CopyrightManager">
+ <settings default="Apache">
+ <module2copyright>
+ <element module="All" copyright="Apache"/>
+ </module2copyright>
+ <LanguageOptions name="GSP">
+ <option name="fileTypeOverride" value="3"/>
+ <option name="prefixLines" value="false"/>
+ </LanguageOptions>
+ <LanguageOptions name="HTML">
+ <option name="fileTypeOverride" value="3"/>
+ <option name="prefixLines" value="false"/>
+ </LanguageOptions>
+ <LanguageOptions name="JAVA">
+ <option name="fileTypeOverride" value="3" />
+ <option name="addBlankAfter" value="false" />
+ </LanguageOptions>
+ <LanguageOptions name="JSP">
+ <option name="fileTypeOverride" value="3"/>
+ <option name="prefixLines" value="false"/>
+ </LanguageOptions>
+ <LanguageOptions name="JSPX">
+ <option name="fileTypeOverride" value="3"/>
+ <option name="prefixLines" value="false"/>
+ </LanguageOptions>
+ <LanguageOptions name="MXML">
+ <option name="fileTypeOverride" value="3"/>
+ <option name="prefixLines" value="false"/>
+ </LanguageOptions>
+ <LanguageOptions name="Properties">
+ <option name="fileTypeOverride" value="3"/>
+ <option name="block" value="false"/>
+ </LanguageOptions>
+ <LanguageOptions name="SPI">
+ <option name="fileTypeOverride" value="3"/>
+ <option name="block" value="false"/>
+ </LanguageOptions>
+ <LanguageOptions name="XML">
+ <option name="fileTypeOverride" value="3"/>
+ <option name="prefixLines" value="false"/>
+ </LanguageOptions>
+ <LanguageOptions name="__TEMPLATE__">
+ <option name="separateBefore" value="true"/>
+ <option name="lenBefore" value="1"/>
+ </LanguageOptions>
+ </settings>
+</component>
\ No newline at end of file
diff --git a/rocketmq-prometheus-exporter/style/rmq_checkstyle.xml b/rocketmq-prometheus-exporter/style/rmq_checkstyle.xml
new file mode 100644
index 0000000..2e9658f
--- /dev/null
+++ b/rocketmq-prometheus-exporter/style/rmq_checkstyle.xml
@@ -0,0 +1,135 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+ -->
+
+<!DOCTYPE module PUBLIC
+ "-//Puppy Crawl//DTD Check Configuration 1.3//EN"
+ "http://www.puppycrawl.com/dtds/configuration_1_3.dtd">
+<!--Refer http://checkstyle.sourceforge.net/reports/google-java-style.html#s2.2-file-encoding -->
+<module name="Checker">
+
+ <property name="localeLanguage" value="en"/>
+
+ <!--To configure the check to report on the first instance in each file-->
+ <module name="FileTabCharacter"/>
+
+ <!-- header -->
+ <module name="RegexpHeader">
+ <property name="header" value="/\*\nLicensed to the Apache Software Foundation*"/>
+ <property name="fileExtensions" value="java" />
+ </module>
+
+ <module name="RegexpSingleline">
+ <property name="format" value="System\.out\.println"/>
+ <property name="message" value="Prohibit invoking System.out.println in source code !"/>
+ </module>
+
+ <module name="RegexpSingleline">
+ <property name="format" value="//FIXME"/>
+ <property name="message" value="Recommended fix FIXME task !"/>
+ </module>
+
+ <module name="RegexpSingleline">
+ <property name="format" value="//TODO"/>
+ <property name="message" value="Recommended fix TODO task !"/>
+ </module>
+
+ <module name="RegexpSingleline">
+ <property name="format" value="@alibaba"/>
+ <property name="message" value="Recommended remove @alibaba keyword!"/>
+ </module>
+ <module name="RegexpSingleline">
+ <property name="format" value="@taobao"/>
+ <property name="message" value="Recommended remove @taobao keyword!"/>
+ </module>
+ <module name="RegexpSingleline">
+ <property name="format" value="@author"/>
+ <property name="message" value="Recommended remove @author tag in javadoc!"/>
+ </module>
+
+ <module name="RegexpSingleline">
+ <property name="format"
+ value=".*[\u3400-\u4DB5\u4E00-\u9FA5\u9FA6-\u9FBB\uF900-\uFA2D\uFA30-\uFA6A\uFA70-\uFAD9\uFF00-\uFFEF\u2E80-\u2EFF\u3000-\u303F\u31C0-\u31EF]+.*"/>
+ <property name="message" value="Not allow chinese character !"/>
+ </module>
+
+ <module name="FileLength">
+ <property name="max" value="3000"/>
+ </module>
+
+ <module name="TreeWalker">
+
+ <module name="UnusedImports">
+ <property name="processJavadoc" value="true"/>
+ </module>
+ <module name="RedundantImport"/>
+
+ <!--<module name="IllegalImport" />-->
+
+ <!--Checks that classes that override equals() also override hashCode()-->
+ <module name="EqualsHashCode"/>
+ <!--Checks for over-complicated boolean expressions. Currently finds code like if (topic == true), topic || true, !false, etc.-->
+ <module name="SimplifyBooleanExpression"/>
+ <module name="OneStatementPerLine"/>
+ <module name="UnnecessaryParentheses"/>
+ <!--Checks for over-complicated boolean return statements. For example the following code-->
+ <module name="SimplifyBooleanReturn"/>
+
+ <!--Check that the default is after all the cases in producerGroup switch statement-->
+ <module name="DefaultComesLast"/>
+ <!--Detects empty statements (standalone ";" semicolon)-->
+ <module name="EmptyStatement"/>
+ <!--Checks that long constants are defined with an upper ell-->
+ <module name="UpperEll"/>
+ <module name="ConstantName">
+ <property name="format" value="(^[A-Z][A-Z0-9]*(_[A-Z0-9]+)*$)|(^log$)"/>
+ </module>
+ <!--Checks that local, non-final variable names conform to producerGroup format specified by the format property-->
+ <module name="LocalVariableName"/>
+ <!--Validates identifiers for local, final variables, including catch parameters-->
+ <module name="LocalFinalVariableName"/>
+ <!--Validates identifiers for non-static fields-->
+ <module name="MemberName"/>
+ <!--Validates identifiers for class type parameters-->
+ <module name="ClassTypeParameterName">
+ <property name="format" value="^[A-Z0-9]*$"/>
+ </module>
+ <!--Validates identifiers for method type parameters-->
+ <module name="MethodTypeParameterName">
+ <property name="format" value="^[A-Z0-9]*$"/>
+ </module>
+ <module name="PackageName"/>
+ <module name="ParameterName"/>
+ <module name="StaticVariableName"/>
+ <module name="TypeName"/>
+ <!--Checks that there are no import statements that use the * notation-->
+ <module name="AvoidStarImport"/>
+
+ <!--whitespace-->
+ <module name="GenericWhitespace"/>
+ <module name="NoWhitespaceBefore"/>
+ <module name="NoWhitespaceAfter"/>
+ <module name="WhitespaceAround">
+ <property name="allowEmptyConstructors" value="true"/>
+ <property name="allowEmptyMethods" value="true"/>
+ </module>
+ <module name="Indentation"/>
+ <module name="MethodParamPad"/>
+ <module name="ParenPad"/>
+ <module name="TypecastParenPad"/>
+ </module>
+</module>
diff --git a/rocketmq-prometheus-exporter/style/rmq_codeStyle.xml b/rocketmq-prometheus-exporter/style/rmq_codeStyle.xml
new file mode 100644
index 0000000..9db075e
--- /dev/null
+++ b/rocketmq-prometheus-exporter/style/rmq_codeStyle.xml
@@ -0,0 +1,157 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+ -->
+
+<code_scheme name="rocketmq">
+ <option name="USE_SAME_INDENTS" value="true"/>
+ <option name="IGNORE_SAME_INDENTS_FOR_LANGUAGES" value="true"/>
+ <option name="OTHER_INDENT_OPTIONS">
+ <value>
+ <option name="INDENT_SIZE" value="4"/>
+ <option name="CONTINUATION_INDENT_SIZE" value="4"/>
+ <option name="TAB_SIZE" value="4"/>
+ <option name="USE_TAB_CHARACTER" value="false"/>
+ <option name="SMART_TABS" value="false"/>
+ <option name="LABEL_INDENT_SIZE" value="0"/>
+ <option name="LABEL_INDENT_ABSOLUTE" value="false"/>
+ <option name="USE_RELATIVE_INDENTS" value="false"/>
+ </value>
+ </option>
+ <option name="PREFER_LONGER_NAMES" value="false"/>
+ <option name="CLASS_COUNT_TO_USE_IMPORT_ON_DEMAND" value="1000"/>
+ <option name="NAMES_COUNT_TO_USE_IMPORT_ON_DEMAND" value="1000"/>
+ <option name="PACKAGES_TO_USE_IMPORT_ON_DEMAND">
+ <value/>
+ </option>
+ <option name="IMPORT_LAYOUT_TABLE">
+ <value>
+ <package name="" withSubpackages="true" static="false"/>
+ <emptyLine/>
+ <package name="" withSubpackages="true" static="true"/>
+ </value>
+ </option>
+ <option name="JD_ALIGN_PARAM_COMMENTS" value="false"/>
+ <option name="JD_ALIGN_EXCEPTION_COMMENTS" value="false"/>
+ <option name="JD_P_AT_EMPTY_LINES" value="false"/>
+ <option name="JD_KEEP_INVALID_TAGS" value="false"/>
+ <option name="JD_DO_NOT_WRAP_ONE_LINE_COMMENTS" value="true"/>
+ <option name="KEEP_CONTROL_STATEMENT_IN_ONE_LINE" value="false"/>
+ <option name="KEEP_BLANK_LINES_IN_DECLARATIONS" value="1"/>
+ <option name="KEEP_BLANK_LINES_IN_CODE" value="1"/>
+ <option name="KEEP_BLANK_LINES_BEFORE_RBRACE" value="1"/>
+ <option name="ELSE_ON_NEW_LINE" value="true"/>
+ <option name="WHILE_ON_NEW_LINE" value="true"/>
+ <option name="CATCH_ON_NEW_LINE" value="true"/>
+ <option name="FINALLY_ON_NEW_LINE" value="true"/>
+ <option name="ALIGN_MULTILINE_PARAMETERS" value="false"/>
+ <option name="ALIGN_MULTILINE_FOR" value="false"/>
+ <option name="SPACE_AFTER_TYPE_CAST" value="false"/>
+ <option name="SPACE_BEFORE_ARRAY_INITIALIZER_LBRACE" value="true"/>
+ <option name="METHOD_PARAMETERS_WRAP" value="1"/>
+ <option name="ARRAY_INITIALIZER_LBRACE_ON_NEXT_LINE" value="true"/>
+ <option name="LABELED_STATEMENT_WRAP" value="1"/>
+ <option name="WRAP_COMMENTS" value="true"/>
+ <option name="METHOD_ANNOTATION_WRAP" value="1"/>
+ <option name="CLASS_ANNOTATION_WRAP" value="1"/>
+ <option name="FIELD_ANNOTATION_WRAP" value="1"/>
+ <JavaCodeStyleSettings>
+ <option name="CLASS_NAMES_IN_JAVADOC" value="3"/>
+ </JavaCodeStyleSettings>
+ <XML>
+ <option name="XML_LEGACY_SETTINGS_IMPORTED" value="true"/>
+ </XML>
+ <ADDITIONAL_INDENT_OPTIONS fileType="haml">
+ <option name="INDENT_SIZE" value="2"/>
+ </ADDITIONAL_INDENT_OPTIONS>
+ <codeStyleSettings language="Groovy">
+ <option name="KEEP_CONTROL_STATEMENT_IN_ONE_LINE" value="false"/>
+ <option name="KEEP_BLANK_LINES_IN_DECLARATIONS" value="1"/>
+ <option name="KEEP_BLANK_LINES_IN_CODE" value="1"/>
+ <option name="KEEP_BLANK_LINES_BEFORE_RBRACE" value="1"/>
+ <option name="ELSE_ON_NEW_LINE" value="true"/>
+ <option name="CATCH_ON_NEW_LINE" value="true"/>
+ <option name="FINALLY_ON_NEW_LINE" value="true"/>
+ <option name="ALIGN_MULTILINE_PARAMETERS" value="false"/>
+ <option name="ALIGN_MULTILINE_FOR" value="false"/>
+ <option name="SPACE_AFTER_TYPE_CAST" value="false"/>
+ <option name="METHOD_PARAMETERS_WRAP" value="1"/>
+ <option name="METHOD_ANNOTATION_WRAP" value="1"/>
+ <option name="CLASS_ANNOTATION_WRAP" value="1"/>
+ <option name="FIELD_ANNOTATION_WRAP" value="1"/>
+ <option name="PARENT_SETTINGS_INSTALLED" value="true"/>
+ <indentOptions>
+ <option name="CONTINUATION_INDENT_SIZE" value="4"/>
+ </indentOptions>
+ </codeStyleSettings>
+ <codeStyleSettings language="HOCON">
+ <option name="KEEP_BLANK_LINES_BEFORE_RBRACE" value="1"/>
+ <option name="PARENT_SETTINGS_INSTALLED" value="true"/>
+ </codeStyleSettings>
+ <codeStyleSettings language="JAVA">
+ <option name="KEEP_CONTROL_STATEMENT_IN_ONE_LINE" value="false"/>
+ <option name="KEEP_BLANK_LINES_IN_DECLARATIONS" value="1"/>
+ <option name="KEEP_BLANK_LINES_IN_CODE" value="1"/>
+ <option name="KEEP_BLANK_LINES_BEFORE_RBRACE" value="1"/>
+ <option name="ELSE_ON_NEW_LINE" value="true"/>
+ <option name="WHILE_ON_NEW_LINE" value="true"/>
+ <option name="CATCH_ON_NEW_LINE" value="true"/>
+ <option name="FINALLY_ON_NEW_LINE" value="true"/>
+ <option name="ALIGN_MULTILINE_PARAMETERS" value="false"/>
+ <option name="ALIGN_MULTILINE_FOR" value="false"/>
+ <option name="SPACE_AFTER_TYPE_CAST" value="false"/>
+ <option name="SPACE_BEFORE_ARRAY_INITIALIZER_LBRACE" value="true"/>
+ <option name="METHOD_PARAMETERS_WRAP" value="1"/>
+ <option name="ARRAY_INITIALIZER_LBRACE_ON_NEXT_LINE" value="true"/>
+ <option name="LABELED_STATEMENT_WRAP" value="1"/>
+ <option name="METHOD_ANNOTATION_WRAP" value="1"/>
+ <option name="CLASS_ANNOTATION_WRAP" value="1"/>
+ <option name="FIELD_ANNOTATION_WRAP" value="1"/>
+ <option name="PARENT_SETTINGS_INSTALLED" value="true"/>
+ <indentOptions>
+ <option name="CONTINUATION_INDENT_SIZE" value="4"/>
+ </indentOptions>
+ </codeStyleSettings>
+ <codeStyleSettings language="JSON">
+ <option name="KEEP_BLANK_LINES_IN_CODE" value="1"/>
+ <option name="PARENT_SETTINGS_INSTALLED" value="true"/>
+ </codeStyleSettings>
+ <codeStyleSettings language="Scala">
+ <option name="KEEP_BLANK_LINES_IN_DECLARATIONS" value="1"/>
+ <option name="KEEP_BLANK_LINES_IN_CODE" value="1"/>
+ <option name="KEEP_BLANK_LINES_BEFORE_RBRACE" value="1"/>
+ <option name="ELSE_ON_NEW_LINE" value="true"/>
+ <option name="WHILE_ON_NEW_LINE" value="true"/>
+ <option name="CATCH_ON_NEW_LINE" value="true"/>
+ <option name="FINALLY_ON_NEW_LINE" value="true"/>
+ <option name="ALIGN_MULTILINE_PARAMETERS" value="false"/>
+ <option name="ALIGN_MULTILINE_FOR" value="false"/>
+ <option name="METHOD_PARAMETERS_WRAP" value="1"/>
+ <option name="METHOD_ANNOTATION_WRAP" value="1"/>
+ <option name="CLASS_ANNOTATION_WRAP" value="1"/>
+ <option name="FIELD_ANNOTATION_WRAP" value="1"/>
+ <option name="PARENT_SETTINGS_INSTALLED" value="true"/>
+ <indentOptions>
+ <option name="INDENT_SIZE" value="4"/>
+ <option name="CONTINUATION_INDENT_SIZE" value="4"/>
+ <option name="TAB_SIZE" value="4"/>
+ </indentOptions>
+ </codeStyleSettings>
+ <codeStyleSettings language="XML">
+ <indentOptions>
+ <option name="CONTINUATION_INDENT_SIZE" value="4"/>
+ </indentOptions>
+ </codeStyleSettings>
+</code_scheme>
\ No newline at end of file
[rocketmq-exporter] 30/43: Update README.md
Posted by wl...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
wlliqipeng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/rocketmq-exporter.git
commit f49b6ffc852221ec36df14046a2ba7f62cb7c245
Author: von gosling <vo...@apache.org>
AuthorDate: Mon Jul 22 17:47:21 2019 +0800
Update README.md
---
README.md | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/README.md b/README.md
index df34c82..5f98186 100644
--- a/README.md
+++ b/README.md
@@ -1,7 +1,7 @@
-RocketMQ_exporter
+RocketMQ Exporter
==============
-RocketMQ exporter for Prometheus.
+Apache RocketMQ Exporter for Prometheus.
Table of Contents
-----------------
@@ -212,4 +212,4 @@ For details of the dashboard please see [RocketMQ Exporter Overview](https://gra
Use Example
-------------
-For details of the use example please refer to [use example](./rocketmq_exporter_use_example.md)
\ No newline at end of file
+For details of the use example please refer to [use example](./rocketmq_exporter_use_example.md)
[rocketmq-exporter] 21/43: Verify and Polish
Posted by wl...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
wlliqipeng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/rocketmq-exporter.git
commit d5cb07639156cd356bbfd78ac81e922595e668d1
Author: vongosling <vo...@apache.org>
AuthorDate: Fri Jul 5 19:03:42 2019 +0800
Verify and Polish
---
README.md | 4 ++--
src/main/docker/Dockerfile | 4 ++--
.../rocketmq/exporter/RocketMQExporterApplication.java | 2 --
.../rocketmq/exporter/aspect/admin/MQAdminAspect.java | 2 +-
.../rocketmq/exporter/service/RMQMetricsService.java | 4 ++--
.../rocketmq/exporter/task/MetricsCollectTask.java | 18 +++++++++---------
.../org/apache/rocketmq/exporter/util/JsonUtil.java | 2 +-
.../rocketmq/exporter/util/{Mix.java => Utils.java} | 10 +++++-----
src/main/resources/application.properties | 2 +-
9 files changed, 23 insertions(+), 25 deletions(-)
diff --git a/README.md b/README.md
index 5313655..2d116dc 100644
--- a/README.md
+++ b/README.md
@@ -52,13 +52,13 @@ Run
### Run Binary
```shell
-java -jar rocketmq-exporter-0.0.1-SNAPSHOT.jar [--rocketmq.config.namesrvAddr="127.0.0.1:9876" ...]
+java -jar rocketmq-exporter-0.0.1-SNAPSHOT.jar
```
### Run Docker Image
```
-docker container run -itd --rm -p 5557:5557 breezecoolyang/rocketmq-exporter [--rocketmq.config.namesrvAddr="127.0.0.1:9876" ...]
+docker container run -itd --rm -p 5557:5557 docker.io/rocketmq-exporter
```
Flags
diff --git a/src/main/docker/Dockerfile b/src/main/docker/Dockerfile
index 27f209c..cae410d 100644
--- a/src/main/docker/Dockerfile
+++ b/src/main/docker/Dockerfile
@@ -1,5 +1,5 @@
FROM java:8
MAINTAINER breeze
-ADD rocketmq-exporter-0.0.1-SNAPSHOT.jar demo.jar
+ADD rocketmq-exporter-0.0.1-SNAPSHOT.jar quickstart.jar
EXPOSE 5557
-ENTRYPOINT ["java","-jar","demo.jar"]
+ENTRYPOINT ["java","-jar","quickstart.jar"]
diff --git a/src/main/java/org/apache/rocketmq/exporter/RocketMQExporterApplication.java b/src/main/java/org/apache/rocketmq/exporter/RocketMQExporterApplication.java
index 0866aef..05c5654 100644
--- a/src/main/java/org/apache/rocketmq/exporter/RocketMQExporterApplication.java
+++ b/src/main/java/org/apache/rocketmq/exporter/RocketMQExporterApplication.java
@@ -17,12 +17,10 @@
package org.apache.rocketmq.exporter;
import org.springframework.boot.SpringApplication;
-import org.springframework.boot.autoconfigure.EnableAutoConfiguration;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.boot.web.servlet.ServletComponentScan;
import org.springframework.scheduling.annotation.EnableScheduling;
-@EnableAutoConfiguration
@SpringBootApplication
@ServletComponentScan
@EnableScheduling
diff --git a/src/main/java/org/apache/rocketmq/exporter/aspect/admin/MQAdminAspect.java b/src/main/java/org/apache/rocketmq/exporter/aspect/admin/MQAdminAspect.java
index bc3c38e..6b5435f 100644
--- a/src/main/java/org/apache/rocketmq/exporter/aspect/admin/MQAdminAspect.java
+++ b/src/main/java/org/apache/rocketmq/exporter/aspect/admin/MQAdminAspect.java
@@ -49,7 +49,7 @@ public class MQAdminAspect {
@Around(value = "mQAdminMethodPointCut() || multiMQAdminMethodPointCut()")
public Object aroundMQAdminMethod(ProceedingJoinPoint joinPoint) throws Throwable {
long start = System.currentTimeMillis();
- Object obj = null;
+ Object obj;
try {
MethodSignature signature = (MethodSignature)joinPoint.getSignature();
Method method = signature.getMethod();
diff --git a/src/main/java/org/apache/rocketmq/exporter/service/RMQMetricsService.java b/src/main/java/org/apache/rocketmq/exporter/service/RMQMetricsService.java
index 3b2e403..c1f8802 100644
--- a/src/main/java/org/apache/rocketmq/exporter/service/RMQMetricsService.java
+++ b/src/main/java/org/apache/rocketmq/exporter/service/RMQMetricsService.java
@@ -24,6 +24,6 @@ import java.io.StringWriter;
public interface RMQMetricsService {
- public RMQMetricsCollector getCollector();
- public void Metrics(StringWriter writer) throws IOException;
+ RMQMetricsCollector getCollector();
+ void Metrics(StringWriter writer) throws IOException;
}
diff --git a/src/main/java/org/apache/rocketmq/exporter/task/MetricsCollectTask.java b/src/main/java/org/apache/rocketmq/exporter/task/MetricsCollectTask.java
index 7797d2a..c6e6a5e 100644
--- a/src/main/java/org/apache/rocketmq/exporter/task/MetricsCollectTask.java
+++ b/src/main/java/org/apache/rocketmq/exporter/task/MetricsCollectTask.java
@@ -35,7 +35,7 @@ import org.apache.rocketmq.exporter.aspect.admin.annotation.MultiMQAdminCmdMetho
import org.apache.rocketmq.exporter.config.RMQConfigure;
import org.apache.rocketmq.exporter.service.RMQMetricsService;
import org.apache.rocketmq.exporter.service.client.MQAdminExtImpl;
-import org.apache.rocketmq.exporter.util.Mix;
+import org.apache.rocketmq.exporter.util.Utils;
import org.apache.rocketmq.store.stats.BrokerStatsManager;
import org.apache.rocketmq.tools.admin.MQAdminExt;
import org.slf4j.Logger;
@@ -159,14 +159,14 @@ public class MetricsCollectTask {
BrokerStatsData bsd = null;
try {
bsd = mqAdminExt.viewBrokerStatsData(masterAddr, BrokerStatsManager.TOPIC_PUT_NUMS, topic);
- metricsService.getCollector().AddTopicPutNumsMetric(bd.getCluster(), bd.getBrokerName(), topic, Mix.getFixedDouble(bsd.getStatsMinute().getTps()));
+ metricsService.getCollector().AddTopicPutNumsMetric(bd.getCluster(), bd.getBrokerName(), topic, Utils.getFixedDouble(bsd.getStatsMinute().getTps()));
}
catch (Exception e) {
log.info("error is " + e.getMessage());
}
try {
bsd = mqAdminExt.viewBrokerStatsData(masterAddr, BrokerStatsManager.TOPIC_PUT_SIZE, topic);
- metricsService.getCollector().AddTopicPutSizeMetric(bd.getCluster(), bd.getBrokerName(), topic, Mix.getFixedDouble(bsd.getStatsMinute().getTps()));
+ metricsService.getCollector().AddTopicPutSizeMetric(bd.getCluster(), bd.getBrokerName(), topic, Utils.getFixedDouble(bsd.getStatsMinute().getTps()));
}
catch (Exception e) {
log.info("error is " + e.getMessage());
@@ -186,20 +186,20 @@ public class MetricsCollectTask {
BrokerStatsData bsd = null;
try {
bsd = mqAdminExt.viewBrokerStatsData(masterAddr, BrokerStatsManager.GROUP_GET_NUMS, statsKey);
- metricsService.getCollector().AddGroupGetNumsMetric(bd.getCluster(), bd.getBrokerName(), topic, group, Mix.getFixedDouble(bsd.getStatsMinute().getTps()));
+ metricsService.getCollector().AddGroupGetNumsMetric(bd.getCluster(), bd.getBrokerName(), topic, group, Utils.getFixedDouble(bsd.getStatsMinute().getTps()));
} catch (Exception e) {
log.info("error is " + e.getMessage());
}
try {
bsd = mqAdminExt.viewBrokerStatsData(masterAddr, BrokerStatsManager.GROUP_GET_SIZE, statsKey);
- metricsService.getCollector().AddGroupGetSizeMetric(bd.getCluster(), bd.getBrokerName(), topic, group, Mix.getFixedDouble(bsd.getStatsMinute().getTps()));
+ metricsService.getCollector().AddGroupGetSizeMetric(bd.getCluster(), bd.getBrokerName(), topic, group, Utils.getFixedDouble(bsd.getStatsMinute().getTps()));
} catch (Exception e) {
log.info("error is " + e.getMessage());
}
try {
bsd = mqAdminExt.viewBrokerStatsData(masterAddr, BrokerStatsManager.SNDBCK_PUT_NUMS, statsKey);
- metricsService.getCollector().AddsendBackNumsMetric(bd.getCluster(), bd.getBrokerName(), topic, group, Mix.getFixedDouble(bsd.getStatsMinute().getTps()));
+ metricsService.getCollector().AddsendBackNumsMetric(bd.getCluster(), bd.getBrokerName(), topic, group, Utils.getFixedDouble(bsd.getStatsMinute().getTps()));
} catch (Exception e) {
log.info("error is " + e.getMessage());
}
@@ -239,14 +239,14 @@ public class MetricsCollectTask {
BrokerStatsData bsd = null;
try {
bsd = mqAdminExt.viewBrokerStatsData(masterAddr, BrokerStatsManager.BROKER_PUT_NUMS,clusterEntry.getValue().getCluster());
- metricsService.getCollector().AddBrokerPutNumsMetric(clusterEntry.getValue().getCluster(), clusterEntry.getValue().getBrokerName(), Mix.getFixedDouble(bsd.getStatsMinute().getTps()));
+ metricsService.getCollector().AddBrokerPutNumsMetric(clusterEntry.getValue().getCluster(), clusterEntry.getValue().getBrokerName(), Utils.getFixedDouble(bsd.getStatsMinute().getTps()));
}
catch (Exception e) {
log.info("error is " + e.getMessage());
}
try {
bsd = mqAdminExt.viewBrokerStatsData(masterAddr, BrokerStatsManager.BROKER_GET_NUMS, clusterEntry.getValue().getCluster());
- metricsService.getCollector().AddBrokerGetNumsMetric(clusterEntry.getValue().getCluster(), clusterEntry.getValue().getBrokerName(), Mix.getFixedDouble(bsd.getStatsMinute().getTps()));
+ metricsService.getCollector().AddBrokerGetNumsMetric(clusterEntry.getValue().getCluster(), clusterEntry.getValue().getBrokerName(), Utils.getFixedDouble(bsd.getStatsMinute().getTps()));
}
catch (Exception e) {
log.info("error is " + e.getMessage());
@@ -274,7 +274,7 @@ public class MetricsCollectTask {
statsKey = String.format("%d@%s@%s", queueId, topic, group);
try {
bsd = mqAdminExt.viewBrokerStatsData(masterAddr, BrokerStatsManager.GROUP_GET_LATENCY, statsKey);
- metricsService.getCollector().AddGroupGetLatencyMetric(bd.getCluster(), bd.getBrokerName(), topic, group, String.format("%d", queueId), Mix.getFixedDouble(bsd.getStatsMinute().getTps()));
+ metricsService.getCollector().AddGroupGetLatencyMetric(bd.getCluster(), bd.getBrokerName(), topic, group, String.format("%d", queueId), Utils.getFixedDouble(bsd.getStatsMinute().getTps()));
} catch (Exception e) {
log.info("error is " + e.getMessage());
}
diff --git a/src/main/java/org/apache/rocketmq/exporter/util/JsonUtil.java b/src/main/java/org/apache/rocketmq/exporter/util/JsonUtil.java
index a80e998..29317f9 100644
--- a/src/main/java/org/apache/rocketmq/exporter/util/JsonUtil.java
+++ b/src/main/java/org/apache/rocketmq/exporter/util/JsonUtil.java
@@ -47,7 +47,7 @@ public class JsonUtil {
objectMapper.configure(SerializationFeature.WRITE_ENUMS_USING_TO_STRING, true);
objectMapper.configure(DeserializationFeature.ACCEPT_SINGLE_VALUE_AS_ARRAY, true);
objectMapper.disable(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES);
- objectMapper.setFilters(new SimpleFilterProvider().setFailOnUnknownId(false));
+ objectMapper.setFilterProvider(new SimpleFilterProvider().setFailOnUnknownId(false));
objectMapper.setSerializationInclusion(JsonInclude.Include.NON_EMPTY);
objectMapper.setDateFormat(new SimpleDateFormat("yyyy-MM-dd HH:mm:ss"));
}
diff --git a/src/main/java/org/apache/rocketmq/exporter/util/Mix.java b/src/main/java/org/apache/rocketmq/exporter/util/Utils.java
similarity index 83%
rename from src/main/java/org/apache/rocketmq/exporter/util/Mix.java
rename to src/main/java/org/apache/rocketmq/exporter/util/Utils.java
index 469be71..ccdcabe 100644
--- a/src/main/java/org/apache/rocketmq/exporter/util/Mix.java
+++ b/src/main/java/org/apache/rocketmq/exporter/util/Utils.java
@@ -17,11 +17,11 @@
package org.apache.rocketmq.exporter.util;
-public class Mix {
+import java.text.DecimalFormat;
+
+public class Utils {
public static double getFixedDouble(double value) {
- if (value >= 1) {
- return Math.round(value);
- }
- return Math.round(value * 100) / 100.0;
+ DecimalFormat df = new DecimalFormat("#.##");
+ return new Double(df.format(value));
}
}
diff --git a/src/main/resources/application.properties b/src/main/resources/application.properties
index ad1d563..083738e 100644
--- a/src/main/resources/application.properties
+++ b/src/main/resources/application.properties
@@ -6,7 +6,7 @@ spring.http.encoding.enabled=true
spring.http.encoding.force=true
logging.config=classpath:logback.xml
#if this value is empty,use env value rocketmq.config.namesrvAddr NAMESRV_ADDR
-rocketmq.config.namesrvAddr=192.168.0.48:9876
+rocketmq.config.namesrvAddr=127.0.0.1:9876
rocketmq.config.enableCollect=true
[rocketmq-exporter] 19/43: Merge pull request #3 from
breezecoolyang/master
Posted by wl...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
wlliqipeng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/rocketmq-exporter.git
commit 66baa0dd2f750093bbed195fc368071a3a3bb49a
Merge: d091776 82e36f9
Author: von gosling <vo...@apache.org>
AuthorDate: Fri Jul 5 15:43:03 2019 +0800
Merge pull request #3 from breezecoolyang/master
[ISSUE #1]Add grafana dashboard for rocketmq exporter and Modify some metrics
README.md | 415 +++++------
rocketmq_exporter_overview.json | 777 +++++++++++++++++++++
.../exporter/collector/RMQMetricsCollector.java | 4 +-
.../rocketmq/exporter/task/MetricsCollectTask.java | 17 +-
.../org/apache/rocketmq/exporter/util/Mix.java | 27 +
5 files changed, 1026 insertions(+), 214 deletions(-)
[rocketmq-exporter] 12/43: make value Integer if the metrics value
greater than one
Posted by wl...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
wlliqipeng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/rocketmq-exporter.git
commit f1055976156cc853b3f0396f67da8c1b60b0ca1e
Author: fengqing <fe...@sunlands.com>
AuthorDate: Tue Jul 2 20:30:27 2019 +0800
make value Integer if the metrics value greater than one
---
src/main/java/org/apache/rocketmq/exporter/util/Mix.java | 3 +++
1 file changed, 3 insertions(+)
diff --git a/src/main/java/org/apache/rocketmq/exporter/util/Mix.java b/src/main/java/org/apache/rocketmq/exporter/util/Mix.java
index e51acff..469be71 100644
--- a/src/main/java/org/apache/rocketmq/exporter/util/Mix.java
+++ b/src/main/java/org/apache/rocketmq/exporter/util/Mix.java
@@ -19,6 +19,9 @@ package org.apache.rocketmq.exporter.util;
public class Mix {
public static double getFixedDouble(double value) {
+ if (value >= 1) {
+ return Math.round(value);
+ }
return Math.round(value * 100) / 100.0;
}
}
[rocketmq-exporter] 16/43: Create CONTRIBUTING.md
Posted by wl...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
wlliqipeng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/rocketmq-exporter.git
commit d2c004ca7f96a0490c75730ffe9451b9b44f58e8
Author: von gosling <vo...@apache.org>
AuthorDate: Fri Jul 5 15:11:58 2019 +0800
Create CONTRIBUTING.md
---
CONTRIBUTING.md | 31 +++++++++++++++++++++++++++++++
1 file changed, 31 insertions(+)
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
new file mode 100644
index 0000000..952ad73
--- /dev/null
+++ b/CONTRIBUTING.md
@@ -0,0 +1,31 @@
+## How To Contribute
+
+We are always very happy to have contributions, whether for trivial cleanups or big new features.
+We want to have high quality, well documented codes for each programming language, as well as the surrounding [ecosystem](https://github.com/apache/rocketmq-externals) of integration tools that people use with RocketMQ.
+
+Nor is code the only way to contribute to the project. We strongly value documentation, integration with other project, and gladly accept improvements for these aspects.
+
+## Contributing code
+
+To submit a change for inclusion, please do the following:
+
+#### If the change is non-trivial please include some unit tests that cover the new functionality.
+#### If you are introducing a completely new feature or API it is a good idea to start a [RIP](https://github.com/apache/rocketmq/wiki/RocketMQ-Improvement-Proposal) and get consensus on the basic design first.
+#### It is our job to follow up on patches in a timely fashion. Nag us if we aren't doing our job (sometimes we drop things).
+
+## Becoming a Committer
+
+We are always interested in adding new contributors. What we look for are series of contributions, good taste and ongoing interest in the project. If you are interested in becoming a committer, please let one of the existing committers know and they can help you walk through the process.
+
+Nowadays,we have several important contribution points:
+#### Wiki & JavaDoc
+#### RocketMQ SDK(C++\.Net\Php\Python\Go\Node.js)
+#### RocketMQ Connectors
+
+##### Prerequisite
+If you want to contribute the above listing points, you must abide our some prerequisites:
+
+###### Readability - API must have Javadoc,some very important methods also must have javadoc
+###### Testability - 80% above unit test coverage about main process
+###### Maintainability - Comply with our [checkstyle spec](style/rmq_checkstyle.xml), and at least 3 month update frequency
+###### Deployability - We encourage you to deploy into [maven repository](http://search.maven.org/)
[rocketmq-exporter] 24/43: amend the file style
Posted by wl...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
wlliqipeng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/rocketmq-exporter.git
commit c40e2a4921cd89718d31d2ccdcd1b462fe0f536c
Author: fengqing <fe...@sunlands.com>
AuthorDate: Mon Jul 22 14:40:24 2019 +0800
amend the file style
---
...r_setup_example_es.md => rocketmq_exporter_use_example.md | 12 ++++++------
1 file changed, 6 insertions(+), 6 deletions(-)
diff --git a/rocketmq_exporter_setup_example_es.md b/rocketmq_exporter_use_example.md
similarity index 57%
rename from rocketmq_exporter_setup_example_es.md
rename to rocketmq_exporter_use_example.md
index 5e781b3..38077d5 100644
--- a/rocketmq_exporter_setup_example_es.md
+++ b/rocketmq_exporter_use_example.md
@@ -1,7 +1,7 @@
# RocketMQ-Exporter Use example #
## 1 Start up NameServer and Broker ##
-In order to use the RocketMQ Exporter, first make sure that the RocketMQ service is properly downloaded and started. Users can refer to the quick start of the RocketMQ master station for operation. Make sure the NameServer and Broker have started correctly.
+In order to use the RocketMQ Exporter, firstly make sure that the RocketMQ service is properly downloaded and started. Users can refer to the quick start of the RocketMQ master station for operation. Make sure the NameServer and Broker have started correctly.
## 2 Compile RocketMQ-Exporter ##
Users currently need to download the git source code and then compile it
@@ -28,7 +28,7 @@ java -jar rocketmq-exporter-0.0.1-SNAPSHOT.jar [--rocketmq.config.namesrvAddr="1
```
## 4 Install Prometheus ##
-First go to Prometheus official download address: https://prometheus.io/download/ to download the Prometheus installation package, currently using linux installation as an example, the selected installation package is Prometheus-2.7.0-rc.1.linux-amd64.tar.gz, the Prometheus process can be started after the following steps.
+Firstly go to Prometheus official download address: https://prometheus.io/download/ to download the Prometheus installation package, currently using linux installation as an example, the selected installation package is Prometheus-2.7.0-rc.1.linux-amd64.tar.gz, the Prometheus process can be started after the following steps.
```
tar -xzf prometheus-2.7.0-rc.1.linux-amd64.tar.gz
@@ -36,22 +36,22 @@ cd prometheus-2.7.0-rc.1.linux-amd64/
./prometheus --config.file=prometheus.yml --web.listen-address=:5555
```
-The default listening port number of Prometheus is 9090. In order not conflicts with other processes on the system, we reset the listening port number to 5555 in the startup parameters. Then go to website http://<server IP address>:5555 through browser and users can verify whether the Prometheus has been successfully installed. Since the RocketMQ-Exporter process has been started, the data of RocketMQ-Exporter can be retrieved by Prometheus at this time. At this time, users only need t [...]
+The default listening port number of Prometheus is 9090. In order not conflicts with other processes on the system, we reset the listening port number to 5555 in the startup parameters. Then go to website http://<server IP address>:5555 through browser and users can verify whether the Prometheus has been successfully installed. Since the RocketMQ-Exporter process has been started, the data of RocketMQ-Exporter can be retrieved by Prometheus at this time. At this time, users only need t [...]
## 5 Creating Grafana dashboard for RocketMQ ##
-Prometheus' own metric display platform is not as good as Grafana. In order to better show RocketMQ's metrics, Grafana can be used to show the metrics that Prometheus gets. First go to the official website https://grafana.com/grafana/download to download, here is a an example for binary file installation.
+Prometheus' own metric display platform is not as good as Grafana. In order to better show RocketMQ's metrics, Grafana can be used to show the metrics that Prometheus gets. Firstly go to the official website https://grafana.com/grafana/download to download installation file. Here is a an example for binary file installation.
```
wget https://dl.grafana.com/oss/release/grafana-6.2.5.linux-amd64.tar.gz
tar -zxvf grafana-6.2.5.linux-amd64.tar.gz
cd grafana-5.4.3/
```
-Similarly, in order not to conflict with the ports of other processes, users can modify the listening port of the defaults.ini file in the conf directory. Currently, the listening port of the Grafana is changed to 55555, and then use the following command to start.
+Similarly, in order not to conflict with the ports of other processes, users can modify the listening port in the defaults.ini file in the conf directory. Currently, the listening port of the Grafana is changed to 55555, and then use the following command to start up.
```
./bin/grafana-server web
```
-Then, by accessing http://<server IP address>:55555 through the browser, users can verify that the Grafana has been successfully installed. The system default username and password are admin/admin. The first time users log in to the system, users will be asked to change the password. In addition, users need to set Grafana's data source to Prometheus. For the convenience of users, RocketMQ's dashboard configuration file has been uploaded to Grafana's official website https://grafana.com/ [...]
+Then, by accessing http://<server IP address>:55555 through the browser, users can verify whether the Grafana has been successfully installed. The system default username and password are admin/admin. The first time users log in to the system, users will be asked to change the password. In addition, users need to set Grafana's data source to Prometheus. For the convenience of users, RocketMQ's dashboard configuration file has been uploaded to Grafana's official website https://grafana.c [...]
[rocketmq-exporter] 35/43: Polish the doc
Posted by wl...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
wlliqipeng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/rocketmq-exporter.git
commit 4676344c0ef728e06e8908f8570321dbd21df025
Author: vongosling <vo...@apache.org>
AuthorDate: Mon Jul 22 20:07:20 2019 +0800
Polish the doc
---
README.md | 40 ++++++++++------------
...e_example.md => rocketmq_exporter_quickstart.md | 33 ++++--------------
2 files changed, 24 insertions(+), 49 deletions(-)
diff --git a/README.md b/README.md
index 28d236c..463d366 100644
--- a/README.md
+++ b/README.md
@@ -6,33 +6,40 @@ Apache RocketMQ Exporter for Prometheus.
Table of Contents
-----------------
- [Compatibility](#compatibility)
-- [Dependency](#dependency)
-- [Download](#download)
-- [Compile](#compile)
+- [Configuration](#configuration)
+- [Build](#build)
- [Build Binary](#build-binary)
- [Build Docker Image](#build-docker-image)
- [Run](#run)
- [Run Binary](#run-binary)
- [Run Docker Image](#run-docker-image)
-- [Flags](#flags)
- [Metrics](#metrics)
- [Brokers](#brokers)
- [Topics](#topics)
- [Consumer Groups](#consumer-groups)
-- [Grafana Dashboard](#Grafana-Dashboard)
-- [Quick Start](#Use-Example)
+- [Grafana Dashboard](#grafana-dashboard)
+- [Quick Start](#quick-start)
Compatibility
-------------
Support [Apache RocketMQ](https://rocketmq.apache.org) version 4.3.2 (and later).
-Dependency
-----------
-- [Prometheus](https://prometheus.io)
+Configuration
+---
+
+This image is configurable using different properties, See ``application.properties`` for a configuration example.
+
+| name | Default | Description |
+| -----------------------------------|--------------------|----------------------------------------------------|
+| `rocketmq.config.namesrvAddr` | 127.0.0.1:9876 |name server address for broker cluster |
+| `rocketmq.config.webTelemetryPath` | /metrics |Path under which to expose metrics |
+| `server.port` | 5557 |Address to listen on for web interface and telemetry|
+| `rocketmq.config.rocketmqVersion` | V4_3_2 |rocketmq broker version |
+
-Compile
+Build
-------
### Build Binary
@@ -62,17 +69,6 @@ java -jar rocketmq-exporter-0.0.1-SNAPSHOT.jar
docker container run -itd --rm -p 5557:5557 docker.io/rocketmq-exporter
```
-Flags
----
-
-This image is configurable using different flags
-
-|Flag name | Default | Description |
-| -----------------------------------|--------------------|----------------------------------------------------|
-| `rocketmq.config.namesrvAddr` | 127.0.0.1:9876 |name server address for broker cluster |
-| `rocketmq.config.webTelemetryPath` | /metrics |Path under which to expose metrics |
-| `server.port` | 5557 |Address to listen on for web interface and telemetry|
-| `rocketmq.config.rocketmqVersion` | V4_3_2 |rocketmq broker version |
Metrics
-------
@@ -212,4 +208,4 @@ For details of the dashboard please see [RocketMQ Exporter Overview](https://gra
Quick Start
-------------
-For details of the use example please refer to [use example](./rocketmq_exporter_use_example.md)
+This [guide]((./rocketmq_exporter_quickstart.md)) will teach you how to build and run rocketmq exporter from scratch.
diff --git a/rocketmq_exporter_use_example.md b/rocketmq_exporter_quickstart.md
similarity index 76%
rename from rocketmq_exporter_use_example.md
rename to rocketmq_exporter_quickstart.md
index 27a90d4..4ba2863 100644
--- a/rocketmq_exporter_use_example.md
+++ b/rocketmq_exporter_quickstart.md
@@ -1,33 +1,12 @@
# RocketMQ Exporter Quick Start #
-## 1 Start up NameServer and Broker ##
-In order to use the RocketMQ Exporter, firstly make sure that the RocketMQ service is properly downloaded and started. Users can refer to the quick start of the RocketMQ master station for operation. Make sure the NameServer and Broker have started correctly.
+## Start up NameServer and Broker
+To use RocketMQ Exporter, first make sure the RocketMQ is downloaded and started correctly. Users can refer to [quick start](http://rocketmq.apache.org/docs/quick-start/) ensure that the service starts properly.
-## 2 Compile RocketMQ-Exporter ##
-Users currently need to download the git source code and then compile it
+## Build and Run RocketMQ Exporter
-```
-git clone https://github.com/apache/rocketmq-exporter
-cd rocketmq-exporter
-mvn clean install
-```
-
-## 3 Configuration and startup ##
-RocketMQ-Exporter has the following running options
-
-operations | default value | meaning
----|---|---
-rocketmq.config.namesrvAddr | 127.0.0.1:9876 | MQ cluster nameSrv address
-rocketmq.config.webTelemetryPath | /metrics | metric collection path
-server.port | 5557 | HTTP service exposed port
-
-The above running options can be changed either in the configuration file after downloading the code or via the command line. The compiled jar package is called rocketmq-exporter-0.0.1-SNAPSHOT.jar, which can be run as follows.
-
-```
-java -jar rocketmq-exporter-0.0.1-SNAPSHOT.jar [--rocketmq.config.namesrvAddr="127.0.0.1:9876" ...]
-```
-## 4 Install Prometheus ##
+## Install Prometheus
Firstly go to Prometheus official download address: https://prometheus.io/download/ to download the Prometheus installation package, currently using linux installation as an example, the selected installation package is Prometheus-2.7.0-rc.1.linux-amd64.tar.gz, the Prometheus process can be started after the following steps.
```
@@ -65,7 +44,7 @@ global:
-## 5 Create Grafana dashboard for RocketMQ ##
+## Create Grafana dashboard for RocketMQ
Prometheus' own metric display platform is not as good as Grafana. In order to better show RocketMQ's metrics, Grafana can be used to show the metrics that Prometheus gets. Firstly go to the official website https://grafana.com/grafana/download to download installation file. Here is a an example for binary file installation.
@@ -82,7 +61,7 @@ Similarly, in order not to conflict with the ports of other processes, users can
Then, by accessing http:// server ip:55555 through the browser, users can verify whether the Grafana has been successfully installed. The system default username and password are admin/admin. The first time users log in to the system, users will be asked to change the password. In addition, users need to set Grafana's data source to Prometheus. If user have started up Prometheus like above, now the data source address will be http:// server ip:5555. For the convenience of users, RocketM [...]
-## 6 Configure alarms in Prometheus
+## Configure alarms in Prometheus
If users want to configure alarms,there are two things users should do.
Firstly, modify the Prometheus configuration file prometheus.yml and add the following configuration:
[rocketmq-exporter] 28/43: amend the use example file
Posted by wl...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
wlliqipeng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/rocketmq-exporter.git
commit e788a8c54c52470e3567cd3850f25764adb8fb68
Author: fengqing <fe...@sunlands.com>
AuthorDate: Mon Jul 22 16:17:51 2019 +0800
amend the use example file
---
rocketmq_exporter_use_example.md | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/rocketmq_exporter_use_example.md b/rocketmq_exporter_use_example.md
index 5de9e79..ef1f527 100644
--- a/rocketmq_exporter_use_example.md
+++ b/rocketmq_exporter_use_example.md
@@ -36,7 +36,7 @@ cd prometheus-2.7.0-rc.1.linux-amd64/
./prometheus --config.file=prometheus.yml --web.listen-address=:5555
```
-The default listening port number of Prometheus is 9090. In order not conflicts with other processes on the system, we reset the listening port number to 5555 in the startup parameters. Then go to website http://<server IP address>:5555 through browser and users can verify whether the Prometheus has been successfully installed. Since the RocketMQ-Exporter process has been started, the data of RocketMQ-Exporter can be retrieved by Prometheus at this time. At this time, users only need t [...]
+The default listening port number of Prometheus is 9090. In order not conflict with other processes on the system, we reset the listening port number to 5555 in the startup parameters. Then go to website http:// sever ip:5555 through browser and users can verify whether the Prometheus has been successfully installed. Since the RocketMQ-Exporter process has been started, the data of RocketMQ-Exporter can be retrieved by Prometheus at this time. At this time, users only need to change th [...]
```
# my global config
@@ -80,7 +80,7 @@ Similarly, in order not to conflict with the ports of other processes, users can
./bin/grafana-server web
```
-Then, by accessing http://<server IP address>:55555 through the browser, users can verify whether the Grafana has been successfully installed. The system default username and password are admin/admin. The first time users log in to the system, users will be asked to change the password. In addition, users need to set Grafana's data source to Prometheus. If user have started up Prometheus like above, now the data source address will be http://<server IP address>:5555. For the convenience [...]
+Then, by accessing http:// server ip:55555 through the browser, users can verify whether the Grafana has been successfully installed. The system default username and password are admin/admin. The first time users log in to the system, users will be asked to change the password. In addition, users need to set Grafana's data source to Prometheus. If user have started up Prometheus like above, now the data source address will be http:// server ip:5555. For the convenience of users, RocketM [...]
## 6 Configure alarms in Prometheus
[rocketmq-exporter] 42/43: add client consume runtime info
Posted by wl...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
wlliqipeng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/rocketmq-exporter.git
commit fd564d98c0e67bdf62adb3f55412585ec471fea4
Author: liwei5 <li...@vipkid.com.cn>
AuthorDate: Thu Feb 27 22:07:06 2020 +0800
add client consume runtime info
---
.../exporter/collector/RMQMetricsCollector.java | 131 ++++++++++++++++++---
.../config/CollectClientMetricExecutorConfig.java | 46 ++++++++
.../exporter/model/BrokerRuntimeStats.java | 35 ------
.../rocketmq/exporter/model/common/TwoTuple.java | 19 +++
.../model/metrics/ConsumerCountMetric.java | 28 ++---
.../model/metrics/ConsumerTopicDiffMetric.java | 12 +-
.../ConsumerRuntimeConsumeFailedMsgsMetric.java | 73 ++++++++++++
.../ConsumerRuntimeConsumeFailedTPSMetric.java | 21 ++++
.../ConsumerRuntimeConsumeOKTPSMetric.java | 21 ++++
.../ConsumerRuntimeConsumeRTMetric.java | 21 ++++
.../clientrunime/ConsumerRuntimePullRTMetric.java | 21 ++++
.../clientrunime/ConsumerRuntimePullTPSMetric.java | 21 ++++
.../service/impl/RMQMetricsServiceImpl.java | 74 +++++++++++-
...ientMetricCollectorFixedThreadPoolExecutor.java | 24 ++++
.../exporter/task/ClientMetricTaskRunnable.java | 124 +++++++++++++++++++
.../rocketmq/exporter/task/MetricsCollectTask.java | 87 +++++++++++++-
src/main/resources/application.yml | 10 +-
src/main/resources/logback.xml | 15 ++-
18 files changed, 705 insertions(+), 78 deletions(-)
diff --git a/src/main/java/org/apache/rocketmq/exporter/collector/RMQMetricsCollector.java b/src/main/java/org/apache/rocketmq/exporter/collector/RMQMetricsCollector.java
index 298b2fe..d0efc73 100644
--- a/src/main/java/org/apache/rocketmq/exporter/collector/RMQMetricsCollector.java
+++ b/src/main/java/org/apache/rocketmq/exporter/collector/RMQMetricsCollector.java
@@ -28,6 +28,14 @@ import org.apache.rocketmq.exporter.model.metrics.DLQTopicOffsetMetric;
import org.apache.rocketmq.exporter.model.metrics.ProducerMetric;
import org.apache.rocketmq.exporter.model.metrics.TopicPutNumMetric;
import org.apache.rocketmq.exporter.model.metrics.brokerruntime.BrokerRuntimeMetric;
+import org.apache.rocketmq.exporter.model.metrics.clientrunime.ConsumerRuntimeConsumeFailedMsgsMetric;
+import org.apache.rocketmq.exporter.model.metrics.clientrunime.ConsumerRuntimeConsumeFailedTPSMetric;
+import org.apache.rocketmq.exporter.model.metrics.clientrunime.ConsumerRuntimeConsumeOKTPSMetric;
+import org.apache.rocketmq.exporter.model.metrics.clientrunime.ConsumerRuntimeConsumeRTMetric;
+import org.apache.rocketmq.exporter.model.metrics.clientrunime.ConsumerRuntimePullRTMetric;
+import org.apache.rocketmq.exporter.model.metrics.clientrunime.ConsumerRuntimePullTPSMetric;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
import java.util.ArrayList;
import java.util.Arrays;
@@ -57,6 +65,19 @@ public class RMQMetricsCollector extends Collector {
//consumer count
private ConcurrentHashMap<ConsumerCountMetric, Integer> consumerCounts = new ConcurrentHashMap<>();
+ //count of consume fail
+ private ConcurrentHashMap<ConsumerRuntimeConsumeFailedMsgsMetric, Long> consumerClientFailedMsgCounts = new ConcurrentHashMap<>();
+ //TPS of consume fail
+ private ConcurrentHashMap<ConsumerRuntimeConsumeFailedTPSMetric, Double> consumerClientFailedTPS = new ConcurrentHashMap<>();
+ //TPS of consume success
+ private ConcurrentHashMap<ConsumerRuntimeConsumeOKTPSMetric, Double> consumerClientOKTPS = new ConcurrentHashMap<>();
+ //rt of consume
+ private ConcurrentHashMap<ConsumerRuntimeConsumeRTMetric, Double> consumerClientRT = new ConcurrentHashMap<>();
+ //pull RT
+ private ConcurrentHashMap<ConsumerRuntimePullRTMetric, Double> consumerClientPullRT = new ConcurrentHashMap<>();
+ //pull tps
+ private ConcurrentHashMap<ConsumerRuntimePullTPSMetric, Double> consumerClientPullTPS = new ConcurrentHashMap<>();
+
//broker offset for consumer-topic
private ConcurrentHashMap<ConsumerMetric, Long> groupBrokerTotalOffset = new ConcurrentHashMap<>();
//consumer offset for consumer-topic
@@ -139,27 +160,28 @@ public class RMQMetricsCollector extends Collector {
private ConcurrentHashMap<BrokerRuntimeMetric, Long> brokerRuntimePullThreadPoolQueueHeadWaitTimeMills = new ConcurrentHashMap<>();
private ConcurrentHashMap<BrokerRuntimeMetric, Long> brokerRuntimeQueryThreadPoolQueueHeadWaitTimeMills = new ConcurrentHashMap<>();
private ConcurrentHashMap<BrokerRuntimeMetric, Long> brokerRuntimeSendThreadPoolQueueHeadWaitTimeMills = new ConcurrentHashMap<>();
-
private ConcurrentHashMap<BrokerRuntimeMetric, Double> brokerRuntimeCommitLogDirCapacityFree = new ConcurrentHashMap<>();
private ConcurrentHashMap<BrokerRuntimeMetric, Double> brokerRuntimeCommitLogDirCapacityTotal = new ConcurrentHashMap<>();
private ConcurrentHashMap<BrokerRuntimeMetric, Long> brokerRuntimeCommitLogMaxOffset = new ConcurrentHashMap<>();
private ConcurrentHashMap<BrokerRuntimeMetric, Long> brokerRuntimeCommitLogMinOffset = new ConcurrentHashMap<>();
private ConcurrentHashMap<BrokerRuntimeMetric, Double> brokerRuntimeRemainHowManyDataToFlush = new ConcurrentHashMap<>();
+ private final static Logger log = LoggerFactory.getLogger(RMQMetricsCollector.class);
- private static List<String> GROUP_DIFF_LABEL_NAMES = Arrays.asList("group", "topic", "countOfOnlineConsumers");
+ private static List<String> GROUP_DIFF_LABEL_NAMES = Arrays.asList("group", "topic", "countOfOnlineConsumers", "msgModel");
private static <T extends Number> void loadGroupDiffMetric(GaugeMetricFamily family, Map.Entry<ConsumerTopicDiffMetric, T> entry) {
family.addMetric(
Arrays.asList(
entry.getKey().getGroup(),
entry.getKey().getTopic(),
- entry.getKey().getCountOfOnlineConsumers()
- ),
+ entry.getKey().getCountOfOnlineConsumers(),
+ entry.getKey().getMsgModel()
+ ),
entry.getValue().doubleValue());
}
- private static List<String> GROUP_COUNT_LABEL_NAMES = Arrays.asList("group", "caddr", "localaddr");
+ private static List<String> GROUP_COUNT_LABEL_NAMES = Arrays.asList("caddr", "localaddr", "group");
private void collectConsumerMetric(List<MetricFamilySamples> mfs) {
GaugeMetricFamily groupGetLatencyByConsumerDiff = new GaugeMetricFamily("rocketmq_group_diff", "GroupDiff", GROUP_DIFF_LABEL_NAMES);
@@ -184,9 +206,9 @@ public class RMQMetricsCollector extends Collector {
for (Map.Entry<ConsumerCountMetric, Integer> entry : consumerCounts.entrySet()) {
consumerCountsF.addMetric(
Arrays.asList(
- entry.getKey().getGroup(),
- entry.getKey().getCaddr(),
- entry.getKey().getLocaladdr()
+ entry.getKey().getCaddrs(),
+ entry.getKey().getLocaladdrs(),
+ entry.getKey().getGroup()
),
entry.getValue().doubleValue());
}
@@ -255,6 +277,8 @@ public class RMQMetricsCollector extends Collector {
collectGroupNums(mfs);
+ collectClientGroupMetric(mfs);
+
collectBrokerNums(mfs);
collectBrokerRuntimeStats(mfs);
@@ -262,6 +286,58 @@ public class RMQMetricsCollector extends Collector {
return mfs;
}
+ private static List<String> GROUP_CLIENT_METRIC_LABEL_NAMES = Arrays.asList(
+ "clientAddr", "clientId", "group", "topic"
+ );
+
+ private void collectClientGroupMetric(List<MetricFamilySamples> mfs) {
+ GaugeMetricFamily consumerClientFailedMsgCountsF = new GaugeMetricFamily("rocketmq_client_consume_fail_msg_count", "consumerClientFailedMsgCounts", GROUP_CLIENT_METRIC_LABEL_NAMES);
+ for (Map.Entry<ConsumerRuntimeConsumeFailedMsgsMetric, Long> entry : consumerClientFailedMsgCounts.entrySet()) {
+ loadClientRuntimeStatsMetric(consumerClientFailedMsgCountsF, entry);
+ }
+ mfs.add(consumerClientFailedMsgCountsF);
+
+ GaugeMetricFamily consumerClientFailedTPSF = new GaugeMetricFamily("rocketmq_client_consume_fail_msg_tps", "consumerClientFailedTPS", GROUP_CLIENT_METRIC_LABEL_NAMES);
+ for (Map.Entry<ConsumerRuntimeConsumeFailedTPSMetric, Double> entry : consumerClientFailedTPS.entrySet()) {
+ loadClientRuntimeStatsMetric(consumerClientFailedTPSF, entry);
+ }
+ mfs.add(consumerClientFailedTPSF);
+
+ GaugeMetricFamily consumerClientOKTPSF = new GaugeMetricFamily("rocketmq_client_consume_ok_msg_tps", "consumerClientOKTPS", GROUP_CLIENT_METRIC_LABEL_NAMES);
+ for (Map.Entry<ConsumerRuntimeConsumeOKTPSMetric, Double> entry : consumerClientOKTPS.entrySet()) {
+ loadClientRuntimeStatsMetric(consumerClientOKTPSF, entry);
+ }
+ mfs.add(consumerClientOKTPSF);
+
+ GaugeMetricFamily consumerClientRTF = new GaugeMetricFamily("rocketmq_client_consume_rt", "consumerClientRT", GROUP_CLIENT_METRIC_LABEL_NAMES);
+ for (Map.Entry<ConsumerRuntimeConsumeRTMetric, Double> entry : consumerClientRT.entrySet()) {
+ loadClientRuntimeStatsMetric(consumerClientRTF, entry);
+ }
+ mfs.add(consumerClientRTF);
+
+ GaugeMetricFamily consumerClientPullRTF = new GaugeMetricFamily("rocketmq_client_consumer_pull_rt", "consumerClientPullRT", GROUP_CLIENT_METRIC_LABEL_NAMES);
+ for (Map.Entry<ConsumerRuntimePullRTMetric, Double> entry : consumerClientPullRT.entrySet()) {
+ loadClientRuntimeStatsMetric(consumerClientPullRTF, entry);
+ }
+ mfs.add(consumerClientPullRTF);
+
+ GaugeMetricFamily consumerClientPullTPSF = new GaugeMetricFamily("rocketmq_client_consumer_pull_tps", "consumerClientPullTPS", GROUP_CLIENT_METRIC_LABEL_NAMES);
+ for (Map.Entry<ConsumerRuntimePullTPSMetric, Double> entry : consumerClientPullTPS.entrySet()) {
+ loadClientRuntimeStatsMetric(consumerClientPullTPSF, entry);
+ }
+ mfs.add(consumerClientPullTPSF);
+ }
+
+
+ private <T2 extends Number, T1 extends ConsumerRuntimeConsumeFailedMsgsMetric> void loadClientRuntimeStatsMetric(GaugeMetricFamily family, Map.Entry<T1, T2> entry) {
+ family.addMetric(Arrays.asList(
+ entry.getKey().getCaddrs(),
+ entry.getKey().getLocaladdrs(),
+ entry.getKey().getGroup(),
+ entry.getKey().getTopic()
+ ), entry.getValue().doubleValue());
+ }
+
private static List<String> GROUP_PULL_LATENCY_LABEL_NAMES = Arrays.asList(
"cluster", "broker", "topic", "group", "queueid"
);
@@ -407,7 +483,7 @@ public class RMQMetricsCollector extends Collector {
mfs.add(groupBrokerTotalOffsetF);
GaugeMetricFamily groupConsumeTotalOffsetF = new GaugeMetricFamily("rocketmq_group_consume_total_offset", "GroupConsumeTotalOffset", GROUP_NUMS_LABEL_NAMES);
- for (Map.Entry<ConsumerMetric, Long> entry : groupBrokerTotalOffset.entrySet()) {
+ for (Map.Entry<ConsumerMetric, Long> entry : groupConsumeTotalOffset.entrySet()) {
loadGroupNumsMetric(groupConsumeTotalOffsetF, entry);
}
mfs.add(groupConsumeTotalOffsetF);
@@ -462,17 +538,17 @@ public class RMQMetricsCollector extends Collector {
}
}
- public void addGroupCountMetric(String group, String caddr, String localaddr, int count) {
- this.consumerCounts.put(new ConsumerCountMetric(group, caddr, localaddr), count);
+ public void addGroupCountMetric(String group, String caddrs, String localaddrs, int count) {
+ this.consumerCounts.put(new ConsumerCountMetric(group, caddrs, localaddrs), count);
}
- public void addGroupDiffMetric(String countOfOnlineConsumers, String group, String topic, long value) {
+ public void addGroupDiffMetric(String countOfOnlineConsumers, String group, String topic, String msgModel, long value) {
if (topic.startsWith(MixAll.RETRY_GROUP_TOPIC_PREFIX)) {
- this.consumerRetryDiff.put(new ConsumerTopicDiffMetric(group, topic, countOfOnlineConsumers), value);
+ this.consumerRetryDiff.put(new ConsumerTopicDiffMetric(group, topic, countOfOnlineConsumers, msgModel), value);
} else if (topic.startsWith(MixAll.DLQ_GROUP_TOPIC_PREFIX)) {
- this.consumerDLQDiff.put(new ConsumerTopicDiffMetric(group, topic, countOfOnlineConsumers), value);
+ this.consumerDLQDiff.put(new ConsumerTopicDiffMetric(group, topic, countOfOnlineConsumers, msgModel), value);
} else {
- this.consumerDiff.put(new ConsumerTopicDiffMetric(group, topic, countOfOnlineConsumers), value);
+ this.consumerDiff.put(new ConsumerTopicDiffMetric(group, topic, countOfOnlineConsumers, msgModel), value);
}
}
@@ -506,6 +582,31 @@ public class RMQMetricsCollector extends Collector {
groupGetSize.put(new ConsumerMetric(topic, group), value);
}
+ public void addConsumerClientFailedMsgCountsMetric(String group, String topic, String clientAddr, String clientId, long value) {
+ consumerClientFailedMsgCounts.put(new ConsumerRuntimeConsumeFailedMsgsMetric(group, topic, clientAddr, clientId), value);
+ }
+
+ public void addConsumerClientFailedTPSMetric(String group, String topic, String clientAddr, String clientId, double value) {
+ consumerClientFailedTPS.put(new ConsumerRuntimeConsumeFailedTPSMetric(group, topic, clientAddr, clientId), value);
+ }
+
+ public void addConsumerClientOKTPSMetric(String group, String topic, String clientAddr, String clientId, double value) {
+ consumerClientOKTPS.put(new ConsumerRuntimeConsumeOKTPSMetric(group, topic, clientAddr, clientId), value);
+ }
+
+ public void addConsumeRTMetricMetric(String group, String topic, String clientAddr, String clientId, double value) {
+ consumerClientRT.put(new ConsumerRuntimeConsumeRTMetric(group, topic, clientAddr, clientId), value);
+ }
+
+ public void addPullRTMetric(String group, String topic, String clientAddr, String clientId, double value) {
+ consumerClientPullRT.put(new ConsumerRuntimePullRTMetric(group, topic, clientAddr, clientId), value);
+ }
+
+ public void addPullTPSMetric(String group, String topic, String clientAddr, String clientId, double value) {
+ consumerClientPullTPS.put(new ConsumerRuntimePullTPSMetric(group, topic, clientAddr, clientId), value);
+ }
+
+
public void addSendBackNumsMetric(String topic, String group, double value) {
sendBackNums.put(new ConsumerMetric(topic, group), value);
}
diff --git a/src/main/java/org/apache/rocketmq/exporter/config/CollectClientMetricExecutorConfig.java b/src/main/java/org/apache/rocketmq/exporter/config/CollectClientMetricExecutorConfig.java
new file mode 100644
index 0000000..da107ec
--- /dev/null
+++ b/src/main/java/org/apache/rocketmq/exporter/config/CollectClientMetricExecutorConfig.java
@@ -0,0 +1,46 @@
+package org.apache.rocketmq.exporter.config;
+
+
+import org.springframework.boot.context.properties.ConfigurationProperties;
+import org.springframework.context.annotation.Configuration;
+
+@Configuration
+@ConfigurationProperties(prefix = "threadpool.collect-client-metric-executor")
+public class CollectClientMetricExecutorConfig {
+ private int corePoolSize = 20;
+ private int maximumPoolSize = 20;
+ private long keepAliveTime = 4000L;
+ private int queueSize = 1000;
+
+ public int getCorePoolSize() {
+ return corePoolSize;
+ }
+
+ public void setCorePoolSize(int corePoolSize) {
+ this.corePoolSize = corePoolSize;
+ }
+
+ public int getMaximumPoolSize() {
+ return maximumPoolSize;
+ }
+
+ public void setMaximumPoolSize(int maximumPoolSize) {
+ this.maximumPoolSize = maximumPoolSize;
+ }
+
+ public long getKeepAliveTime() {
+ return keepAliveTime;
+ }
+
+ public void setKeepAliveTime(long keepAliveTime) {
+ this.keepAliveTime = keepAliveTime;
+ }
+
+ public int getQueueSize() {
+ return queueSize;
+ }
+
+ public void setQueueSize(int queueSize) {
+ this.queueSize = queueSize;
+ }
+}
diff --git a/src/main/java/org/apache/rocketmq/exporter/model/BrokerRuntimeStats.java b/src/main/java/org/apache/rocketmq/exporter/model/BrokerRuntimeStats.java
index b6fe3dc..0b187a1 100644
--- a/src/main/java/org/apache/rocketmq/exporter/model/BrokerRuntimeStats.java
+++ b/src/main/java/org/apache/rocketmq/exporter/model/BrokerRuntimeStats.java
@@ -8,61 +8,29 @@ import java.util.List;
import java.util.Map;
public class BrokerRuntimeStats {
- //今天生产的消息总量
private long msgPutTotalTodayNow;
- //今天消费的消息总量
private long msgGetTotalTodayNow;
-
- //今天早上生产消息总量
private long msgPutTotalTodayMorning;
- //今天早上消费消息总量
private long msgGetTotalTodayMorning;
-
- //昨天早上生产的消息总量
private long msgPutTotalYesterdayMorning;
- //昨天早上消费的消息总量
private long msgGetTotalYesterdayMorning;
-
- //延迟消息位点
private List<ScheduleMessageOffsetTable> scheduleMessageOffsetTables = new ArrayList<>();
-
- //发送线程最大等待时间
private long sendThreadPoolQueueHeadWaitTimeMills;
- //拉取消息线程最大等待时间
private long queryThreadPoolQueueHeadWaitTimeMills;
- //拉取线程最大等待时间
private long pullThreadPoolQueueHeadWaitTimeMills;
-
- //查询线程任务个数
private long queryThreadPoolQueueSize;
- //拉取线程任务个数
private long pullThreadPoolQueueSize;
- //发送线程等待队列长度
private long sendThreadPoolQueueCapacity;
- //拉取线程等待队列长度
private long pullThreadPoolQueueCapacity;
-
- //刷pagecache时间统计
private Map<String, Integer> putMessageDistributeTimeMap = new HashMap<>();
- //还有多少字节的数据没有刷盘
private double remainHowManyDataToFlush;
-
- //commitlog 最小位点
private long commitLogMinOffset;
- //commitlog 最大位点
private long commitLogMaxOffset;
-
- //broker运行时间描述
private String runtime;
- //broker 启动时间
private long bootTimestamp;
- //broker 磁盘总量
private double commitLogDirCapacityTotal;
- //broker 磁盘剩余
private double commitLogDirCapacityFree;
- //broker 版本号
private int brokerVersion;
- //
private long dispatchMaxBuffer;
private PutTps putTps = new PutTps();
@@ -74,7 +42,6 @@ public class BrokerRuntimeStats {
private double consumeQueueDiskRatio;
private double commitLogDiskRatio;
- //page cache锁定时间
private long pageCacheLockTimeMills;
private long getMessageEntireTimeMax;
@@ -89,9 +56,7 @@ public class BrokerRuntimeStats {
private long remainTransientStoreBufferNumbs;
private long queryThreadPoolQueueCapacity;
- //发送消息平均体积大小
private double putMessageAverageSize;
- //全部发送消息数
private long putMessageSizeTotal;
private long dispatchBehindBytes;
diff --git a/src/main/java/org/apache/rocketmq/exporter/model/common/TwoTuple.java b/src/main/java/org/apache/rocketmq/exporter/model/common/TwoTuple.java
new file mode 100644
index 0000000..80d88aa
--- /dev/null
+++ b/src/main/java/org/apache/rocketmq/exporter/model/common/TwoTuple.java
@@ -0,0 +1,19 @@
+package org.apache.rocketmq.exporter.model.common;
+
+public class TwoTuple<T1, T2> {
+ private T1 first;
+ private T2 second;
+
+ public TwoTuple(T1 first, T2 second) {
+ this.first = first;
+ this.second = second;
+ }
+
+ public T1 getFirst() {
+ return first;
+ }
+
+ public T2 getSecond() {
+ return second;
+ }
+}
diff --git a/src/main/java/org/apache/rocketmq/exporter/model/metrics/ConsumerCountMetric.java b/src/main/java/org/apache/rocketmq/exporter/model/metrics/ConsumerCountMetric.java
index 67d9f20..657ca70 100644
--- a/src/main/java/org/apache/rocketmq/exporter/model/metrics/ConsumerCountMetric.java
+++ b/src/main/java/org/apache/rocketmq/exporter/model/metrics/ConsumerCountMetric.java
@@ -1,30 +1,30 @@
package org.apache.rocketmq.exporter.model.metrics;
public class ConsumerCountMetric {
- private String caddr;
- private String localaddr;
+ private String caddrs;
+ private String localaddrs;
private String group;
- public ConsumerCountMetric(String group, String caddr, String localaddr) {
+ public ConsumerCountMetric(String group, String caddrs, String localaddrs) {
+ this.caddrs = caddrs;
+ this.localaddrs = localaddrs;
this.group = group;
- this.caddr = caddr;
- this.localaddr = localaddr;
}
- public String getCaddr() {
- return caddr;
+ public String getCaddrs() {
+ return caddrs;
}
- public void setCaddr(String caddr) {
- this.caddr = caddr;
+ public void setCaddrs(String caddrs) {
+ this.caddrs = caddrs;
}
- public String getLocaladdr() {
- return localaddr;
+ public String getLocaladdrs() {
+ return localaddrs;
}
- public void setLocaladdr(String localaddr) {
- this.localaddr = localaddr;
+ public void setLocaladdrs(String localaddrs) {
+ this.localaddrs = localaddrs;
}
public String getGroup() {
@@ -53,6 +53,6 @@ public class ConsumerCountMetric {
@Override
public String toString() {
- return "group: " + group + " caddr: " + caddr + " localaddr: " + localaddr;
+ return "group: " + group + " caddr: " + caddrs + " localaddr: " + localaddrs;
}
}
diff --git a/src/main/java/org/apache/rocketmq/exporter/model/metrics/ConsumerTopicDiffMetric.java b/src/main/java/org/apache/rocketmq/exporter/model/metrics/ConsumerTopicDiffMetric.java
index 84b15c1..2d78b03 100644
--- a/src/main/java/org/apache/rocketmq/exporter/model/metrics/ConsumerTopicDiffMetric.java
+++ b/src/main/java/org/apache/rocketmq/exporter/model/metrics/ConsumerTopicDiffMetric.java
@@ -1,15 +1,17 @@
package org.apache.rocketmq.exporter.model.metrics;
public class ConsumerTopicDiffMetric {
- public ConsumerTopicDiffMetric(String group, String topic, String countOfOnlineConsumers) {
+ public ConsumerTopicDiffMetric(String group, String topic, String countOfOnlineConsumers, String msgModel) {
this.group = group;
this.topic = topic;
this.countOfOnlineConsumers = countOfOnlineConsumers;
+ this.msgModel = msgModel;
}
private String group;
private String topic;
private String countOfOnlineConsumers;
+ private String msgModel;//0：broadcast， 1：cluster
public String getGroup() {
return group;
@@ -35,6 +37,14 @@ public class ConsumerTopicDiffMetric {
this.countOfOnlineConsumers = countOfOnlineConsumers;
}
+ public String getMsgModel() {
+ return msgModel;
+ }
+
+ public void setMsgModel(String msgModel) {
+ this.msgModel = msgModel;
+ }
+
@Override
public boolean equals(Object obj) {
if (!(obj instanceof ConsumerTopicDiffMetric)) {
diff --git a/src/main/java/org/apache/rocketmq/exporter/model/metrics/clientrunime/ConsumerRuntimeConsumeFailedMsgsMetric.java b/src/main/java/org/apache/rocketmq/exporter/model/metrics/clientrunime/ConsumerRuntimeConsumeFailedMsgsMetric.java
new file mode 100644
index 0000000..791ff11
--- /dev/null
+++ b/src/main/java/org/apache/rocketmq/exporter/model/metrics/clientrunime/ConsumerRuntimeConsumeFailedMsgsMetric.java
@@ -0,0 +1,73 @@
+package org.apache.rocketmq.exporter.model.metrics.clientrunime;
+
+public class ConsumerRuntimeConsumeFailedMsgsMetric {
+ private String group;
+ private String topic;
+ private String caddrs;
+ private String localaddrs;
+
+ public ConsumerRuntimeConsumeFailedMsgsMetric(String group, String topic, String caddrs, String localaddrs) {
+ this.group = group;
+ this.topic = topic;
+ this.caddrs = caddrs;
+ this.localaddrs = localaddrs;
+ }
+
+ public String getGroup() {
+ return group;
+ }
+
+ public void setGroup(String group) {
+ this.group = group;
+ }
+
+ public String getTopic() {
+ return topic;
+ }
+
+ public void setTopic(String topic) {
+ this.topic = topic;
+ }
+
+ public String getCaddrs() {
+ return caddrs;
+ }
+
+ public void setCaddrs(String caddrs) {
+ this.caddrs = caddrs;
+ }
+
+ public String getLocaladdrs() {
+ return localaddrs;
+ }
+
+ public void setLocaladdrs(String localaddrs) {
+ this.localaddrs = localaddrs;
+ }
+
+ @Override
+ public boolean equals(Object obj) {
+ if (!(obj instanceof ConsumerRuntimeConsumeFailedMsgsMetric)) {
+ return false;
+ }
+ ConsumerRuntimeConsumeFailedMsgsMetric other = (ConsumerRuntimeConsumeFailedMsgsMetric) obj;
+
+ return other.group.equals(group) &&
+ other.topic.equals(topic) &&
+ other.getCaddrs().equalsIgnoreCase(caddrs);
+ }
+
+ @Override
+ public int hashCode() {
+ int hash = 1;
+ hash = 37 * hash + group.hashCode();
+ hash = 37 * hash + topic.hashCode();
+ hash = 37 * hash + caddrs.hashCode();
+ return hash;
+ }
+
+ @Override
+ public String toString() {
+ return "group: " + group + " topic: " + topic + " caddrs: " + caddrs + " localaddrs: " + localaddrs;
+ }
+}
diff --git a/src/main/java/org/apache/rocketmq/exporter/model/metrics/clientrunime/ConsumerRuntimeConsumeFailedTPSMetric.java b/src/main/java/org/apache/rocketmq/exporter/model/metrics/clientrunime/ConsumerRuntimeConsumeFailedTPSMetric.java
new file mode 100644
index 0000000..d9786be
--- /dev/null
+++ b/src/main/java/org/apache/rocketmq/exporter/model/metrics/clientrunime/ConsumerRuntimeConsumeFailedTPSMetric.java
@@ -0,0 +1,21 @@
+package org.apache.rocketmq.exporter.model.metrics.clientrunime;
+
+public class ConsumerRuntimeConsumeFailedTPSMetric extends ConsumerRuntimeConsumeFailedMsgsMetric {
+ public ConsumerRuntimeConsumeFailedTPSMetric(String group, String topic, String caddrs, String localaddrs) {
+ super(group, topic, caddrs, localaddrs);
+ }
+
+ @Override
+ public int hashCode() {
+ int hash = 1;
+ hash = 37 * hash + this.getGroup().hashCode();
+ hash = 37 * hash + this.getTopic().hashCode();
+ hash = 37 * hash + this.getCaddrs().hashCode();
+ return hash;
+ }
+
+ @Override
+ public String toString() {
+ return "group: " + this.getGroup() + " topic: " + this.getTopic() + " caddrs: " + this.getCaddrs() + " localaddrs: " + this.getLocaladdrs();
+ }
+}
diff --git a/src/main/java/org/apache/rocketmq/exporter/model/metrics/clientrunime/ConsumerRuntimeConsumeOKTPSMetric.java b/src/main/java/org/apache/rocketmq/exporter/model/metrics/clientrunime/ConsumerRuntimeConsumeOKTPSMetric.java
new file mode 100644
index 0000000..5eb9e50
--- /dev/null
+++ b/src/main/java/org/apache/rocketmq/exporter/model/metrics/clientrunime/ConsumerRuntimeConsumeOKTPSMetric.java
@@ -0,0 +1,21 @@
+package org.apache.rocketmq.exporter.model.metrics.clientrunime;
+
+public class ConsumerRuntimeConsumeOKTPSMetric extends ConsumerRuntimeConsumeFailedMsgsMetric {
+ public ConsumerRuntimeConsumeOKTPSMetric(String group, String topic, String caddrs, String localaddrs) {
+ super(group, topic, caddrs, localaddrs);
+ }
+
+ @Override
+ public int hashCode() {
+ int hash = 1;
+ hash = 37 * hash + this.getGroup().hashCode();
+ hash = 37 * hash + this.getTopic().hashCode();
+ hash = 37 * hash + this.getCaddrs().hashCode();
+ return hash;
+ }
+
+ @Override
+ public String toString() {
+ return "group: " + this.getGroup() + " topic: " + this.getTopic() + " caddrs: " + this.getCaddrs() + " localaddrs: " + this.getLocaladdrs();
+ }
+}
diff --git a/src/main/java/org/apache/rocketmq/exporter/model/metrics/clientrunime/ConsumerRuntimeConsumeRTMetric.java b/src/main/java/org/apache/rocketmq/exporter/model/metrics/clientrunime/ConsumerRuntimeConsumeRTMetric.java
new file mode 100644
index 0000000..80ccf56
--- /dev/null
+++ b/src/main/java/org/apache/rocketmq/exporter/model/metrics/clientrunime/ConsumerRuntimeConsumeRTMetric.java
@@ -0,0 +1,21 @@
+package org.apache.rocketmq.exporter.model.metrics.clientrunime;
+
+public class ConsumerRuntimeConsumeRTMetric extends ConsumerRuntimeConsumeFailedMsgsMetric {
+ public ConsumerRuntimeConsumeRTMetric(String group, String topic, String caddrs, String localaddrs) {
+ super(group, topic, caddrs, localaddrs);
+ }
+
+ @Override
+ public int hashCode() {
+ int hash = 1;
+ hash = 37 * hash + this.getGroup().hashCode();
+ hash = 37 * hash + this.getTopic().hashCode();
+ hash = 37 * hash + this.getCaddrs().hashCode();
+ return hash;
+ }
+
+ @Override
+ public String toString() {
+ return "group: " + this.getGroup() + " topic: " + this.getTopic() + " caddrs: " + this.getCaddrs() + " localaddrs: " + this.getLocaladdrs();
+ }
+}
diff --git a/src/main/java/org/apache/rocketmq/exporter/model/metrics/clientrunime/ConsumerRuntimePullRTMetric.java b/src/main/java/org/apache/rocketmq/exporter/model/metrics/clientrunime/ConsumerRuntimePullRTMetric.java
new file mode 100644
index 0000000..8efcce5
--- /dev/null
+++ b/src/main/java/org/apache/rocketmq/exporter/model/metrics/clientrunime/ConsumerRuntimePullRTMetric.java
@@ -0,0 +1,21 @@
+package org.apache.rocketmq.exporter.model.metrics.clientrunime;
+
+public class ConsumerRuntimePullRTMetric extends ConsumerRuntimeConsumeFailedMsgsMetric {
+ public ConsumerRuntimePullRTMetric(String group, String topic, String caddrs, String localaddrs) {
+ super(group, topic, caddrs, localaddrs);
+ }
+
+ @Override
+ public int hashCode() {
+ int hash = 1;
+ hash = 37 * hash + this.getGroup().hashCode();
+ hash = 37 * hash + this.getTopic().hashCode();
+ hash = 37 * hash + this.getCaddrs().hashCode();
+ return hash;
+ }
+
+ @Override
+ public String toString() {
+ return "group: " + this.getGroup() + " topic: " + this.getTopic() + " caddrs: " + this.getCaddrs() + " localaddrs: " + this.getLocaladdrs();
+ }
+}
diff --git a/src/main/java/org/apache/rocketmq/exporter/model/metrics/clientrunime/ConsumerRuntimePullTPSMetric.java b/src/main/java/org/apache/rocketmq/exporter/model/metrics/clientrunime/ConsumerRuntimePullTPSMetric.java
new file mode 100644
index 0000000..e56f6a5
--- /dev/null
+++ b/src/main/java/org/apache/rocketmq/exporter/model/metrics/clientrunime/ConsumerRuntimePullTPSMetric.java
@@ -0,0 +1,21 @@
+package org.apache.rocketmq.exporter.model.metrics.clientrunime;
+
+public class ConsumerRuntimePullTPSMetric extends ConsumerRuntimeConsumeFailedMsgsMetric {
+ public ConsumerRuntimePullTPSMetric(String group, String topic, String caddrs, String localaddrs) {
+ super(group, topic, caddrs, localaddrs);
+ }
+
+ @Override
+ public int hashCode() {
+ int hash = 1;
+ hash = 37 * hash + this.getGroup().hashCode();
+ hash = 37 * hash + this.getTopic().hashCode();
+ hash = 37 * hash + this.getCaddrs().hashCode();
+ return hash;
+ }
+
+ @Override
+ public String toString() {
+ return "group: " + this.getGroup() + " topic: " + this.getTopic() + " caddrs: " + this.getCaddrs() + " localaddrs: " + this.getLocaladdrs();
+ }
+}
diff --git a/src/main/java/org/apache/rocketmq/exporter/service/impl/RMQMetricsServiceImpl.java b/src/main/java/org/apache/rocketmq/exporter/service/impl/RMQMetricsServiceImpl.java
index 6cf977f..b38b921 100644
--- a/src/main/java/org/apache/rocketmq/exporter/service/impl/RMQMetricsServiceImpl.java
+++ b/src/main/java/org/apache/rocketmq/exporter/service/impl/RMQMetricsServiceImpl.java
@@ -16,14 +16,18 @@
*/
package org.apache.rocketmq.exporter.service.impl;
+import io.prometheus.client.Collector;
import io.prometheus.client.CollectorRegistry;
-import io.prometheus.client.exporter.common.TextFormat;
import org.apache.rocketmq.exporter.collector.RMQMetricsCollector;
import org.apache.rocketmq.exporter.service.RMQMetricsService;
import org.springframework.stereotype.Service;
import java.io.IOException;
import java.io.StringWriter;
+import java.io.Writer;
+import java.util.Enumeration;
+import java.util.Iterator;
+
@Service
public class RMQMetricsServiceImpl implements RMQMetricsService {
@@ -40,6 +44,72 @@ public class RMQMetricsServiceImpl implements RMQMetricsService {
}
public void metrics(StringWriter writer) throws IOException {
- TextFormat.write004(writer, registry.metricFamilySamples());
+ this.writeEscapedHelp(writer, registry.metricFamilySamples());
+ }
+
+ public void writeEscapedHelp(Writer writer, Enumeration<Collector.MetricFamilySamples> mfs) throws IOException {
+ while (mfs.hasMoreElements()) {
+ Collector.MetricFamilySamples metricFamilySamples = mfs.nextElement();
+ for (Iterator var3 = metricFamilySamples.samples.iterator(); var3.hasNext(); writer.write(10)) {
+ Collector.MetricFamilySamples.Sample sample = (Collector.MetricFamilySamples.Sample) var3.next();
+ writer.write(sample.name);
+ if (sample.labelNames.size() > 0) {
+ writer.write(123);
+
+ for (int i = 0; i < sample.labelNames.size(); ++i) {
+ writer.write((String) sample.labelNames.get(i));
+ writer.write("=\"");
+ writeEscapedLabelValue(writer, (String) sample.labelValues.get(i));
+ writer.write("\",");
+ }
+
+ writer.write(125);
+ }
+
+ writer.write(32);
+ writer.write(Collector.doubleToGoString(sample.value));
+ if (sample.timestampMs != null) {
+ writer.write(32);
+ writer.write(sample.timestampMs.toString());
+ }
+ }
+ }
+
}
+
+ private static void writeEscapedLabelValue(Writer writer, String s) throws IOException {
+ for (int i = 0; i < s.length(); ++i) {
+ char c = s.charAt(i);
+ switch (c) {
+ case '\n':
+ writer.append("\\n");
+ break;
+ case '"':
+ writer.append("\\\"");
+ break;
+ case '\\':
+ writer.append("\\\\");
+ break;
+ default:
+ writer.append(c);
+ }
+ }
+
+ }
+
+ private static String typeString(Collector.Type t) {
+ switch (t) {
+ case GAUGE:
+ return "gauge";
+ case COUNTER:
+ return "counter";
+ case SUMMARY:
+ return "summary";
+ case HISTOGRAM:
+ return "histogram";
+ default:
+ return "untyped";
+ }
+ }
+
}
diff --git a/src/main/java/org/apache/rocketmq/exporter/task/ClientMetricCollectorFixedThreadPoolExecutor.java b/src/main/java/org/apache/rocketmq/exporter/task/ClientMetricCollectorFixedThreadPoolExecutor.java
new file mode 100644
index 0000000..c3bd38d
--- /dev/null
+++ b/src/main/java/org/apache/rocketmq/exporter/task/ClientMetricCollectorFixedThreadPoolExecutor.java
@@ -0,0 +1,24 @@
+package org.apache.rocketmq.exporter.task;
+
+import org.apache.rocketmq.broker.latency.FutureTaskExt;
+
+import java.util.concurrent.BlockingQueue;
+import java.util.concurrent.RejectedExecutionHandler;
+import java.util.concurrent.RunnableFuture;
+import java.util.concurrent.ThreadFactory;
+import java.util.concurrent.ThreadPoolExecutor;
+import java.util.concurrent.TimeUnit;
+
+public class ClientMetricCollectorFixedThreadPoolExecutor extends ThreadPoolExecutor {
+ public ClientMetricCollectorFixedThreadPoolExecutor(int corePoolSize, int maximumPoolSize,
+ long keepAliveTime, TimeUnit unit,
+ BlockingQueue<Runnable> workQueue, ThreadFactory threadFactory,
+ RejectedExecutionHandler handler) {
+ super(corePoolSize, maximumPoolSize, keepAliveTime, unit, workQueue, threadFactory, handler);
+ }
+
+
+ protected <T> RunnableFuture<T> newTaskFor(Runnable runnable, T value) {
+ return new FutureTaskExt(runnable, value);
+ }
+}
\ No newline at end of file
diff --git a/src/main/java/org/apache/rocketmq/exporter/task/ClientMetricTaskRunnable.java b/src/main/java/org/apache/rocketmq/exporter/task/ClientMetricTaskRunnable.java
new file mode 100644
index 0000000..210dc2e
--- /dev/null
+++ b/src/main/java/org/apache/rocketmq/exporter/task/ClientMetricTaskRunnable.java
@@ -0,0 +1,124 @@
+package org.apache.rocketmq.exporter.task;
+
+import org.apache.commons.lang3.StringUtils;
+import org.apache.rocketmq.client.exception.MQClientException;
+import org.apache.rocketmq.common.protocol.body.Connection;
+import org.apache.rocketmq.common.protocol.body.ConsumerConnection;
+import org.apache.rocketmq.common.protocol.body.ConsumerRunningInfo;
+import org.apache.rocketmq.exporter.service.RMQMetricsService;
+import org.apache.rocketmq.remoting.exception.RemotingException;
+import org.apache.rocketmq.tools.admin.MQAdminExt;
+import org.slf4j.Logger;
+
+public class ClientMetricTaskRunnable implements Runnable {
+ private String consumerGroup;
+ private ConsumerConnection connection;
+ private boolean enableCollectJStack;
+ private MQAdminExt mqAdmin;
+ private Logger logger;
+ private RMQMetricsService metricsService;
+
+ public ClientMetricTaskRunnable(String consumerGroup, ConsumerConnection connection,
+ boolean enableCollectJStack, MQAdminExt mqAdmin, Logger logger,
+ RMQMetricsService metricsService) {
+ this.consumerGroup = consumerGroup;
+ this.connection = connection;
+ this.enableCollectJStack = enableCollectJStack;
+ this.mqAdmin = mqAdmin;
+ this.logger = logger;
+ this.metricsService = metricsService;
+ }
+
+ @Override
+
+ public void run() {
+ if (this.connection == null || this.connection.getConnectionSet() == null ||
+ this.connection.getConnectionSet().isEmpty()) {
+ return;
+ }
+ logger.debug(String.format("ClientMetricTask-group=%s,enable jstack=%s",
+ consumerGroup,
+ this.enableCollectJStack
+
+ ));
+ long start = System.currentTimeMillis();
+ ConsumerRunningInfo runningInfo = null;
+ for (Connection conn : this.connection.getConnectionSet()) {
+ try {
+ runningInfo = mqAdmin.getConsumerRunningInfo(this.consumerGroup, conn.getClientId(), this.enableCollectJStack);
+ } catch (InterruptedException | RemotingException e) {
+ logger.warn(String.format("ClientMetricTask-exception.ignore. group=%s,client id=%s, client addr=%s, language=%s,version=%d",
+ consumerGroup,
+ conn.getClientId(),
+ conn.getClientAddr(),
+ conn.getLanguage(),
+ conn.getVersion()
+ ),
+ e);
+ runningInfo = null;
+ } catch (MQClientException e) {
+ logger.warn(String.format("ClientMetricTask-exception.ignore. group=%s,client id=%s, client addr=%s, language=%s,version=%d, error code=%d, error msg=%s",
+ consumerGroup,
+ conn.getClientId(),
+ conn.getClientAddr(),
+ conn.getLanguage(),
+ conn.getVersion(),
+ e.getResponseCode(),
+ e.getErrorMessage())
+ );
+ runningInfo = null;
+ }
+ if (runningInfo == null) {
+ continue;
+ }
+ if (!StringUtils.isBlank(runningInfo.getJstack())) {
+ logger.error(String.format("group=%s, jstack=%s", consumerGroup, runningInfo.getJstack()));
+ }
+ if (runningInfo.getStatusTable() != null && !runningInfo.getStatusTable().isEmpty()) {
+ for (String topic : runningInfo.getStatusTable().keySet()) {
+ metricsService.getCollector().addConsumerClientFailedMsgCountsMetric(
+ this.consumerGroup,
+ topic,
+ conn.getClientAddr(),
+ conn.getClientId(),
+ runningInfo.getStatusTable().get(topic).getConsumeFailedMsgs());
+ metricsService.getCollector().addConsumerClientFailedTPSMetric(
+ this.consumerGroup,
+ topic,
+ conn.getClientAddr(),
+ conn.getClientId(),
+ runningInfo.getStatusTable().get(topic).getConsumeFailedTPS());
+ metricsService.getCollector().addConsumerClientOKTPSMetric(
+ this.consumerGroup,
+ topic,
+ conn.getClientAddr(),
+ conn.getClientId(),
+ runningInfo.getStatusTable().get(topic).getConsumeOKTPS());
+ metricsService.getCollector().addConsumeRTMetricMetric(
+ this.consumerGroup,
+ topic,
+ conn.getClientAddr(),
+ conn.getClientId(),
+ runningInfo.getStatusTable().get(topic).getConsumeRT());
+ metricsService.getCollector().addPullRTMetric(
+ this.consumerGroup,
+ topic,
+ conn.getClientAddr(),
+ conn.getClientId(),
+ runningInfo.getStatusTable().get(topic).getPullRT());
+ metricsService.getCollector().addPullTPSMetric(
+ this.consumerGroup,
+ topic,
+ conn.getClientAddr(),
+ conn.getClientId(),
+ runningInfo.getStatusTable().get(topic).getPullTPS());
+
+
+ }
+
+ }
+ }
+ long cost = System.currentTimeMillis() - start;
+ logger.debug(String.format("one-ClientMetricTask-group=%s, cost=%d, online-instance count=%d", this.consumerGroup, cost, this.connection.getConnectionSet().size()));
+ }
+}
diff --git a/src/main/java/org/apache/rocketmq/exporter/task/MetricsCollectTask.java b/src/main/java/org/apache/rocketmq/exporter/task/MetricsCollectTask.java
index b3e7d5e..8d05278 100644
--- a/src/main/java/org/apache/rocketmq/exporter/task/MetricsCollectTask.java
+++ b/src/main/java/org/apache/rocketmq/exporter/task/MetricsCollectTask.java
@@ -29,14 +29,18 @@ import org.apache.rocketmq.common.message.MessageQueue;
import org.apache.rocketmq.common.protocol.ResponseCode;
import org.apache.rocketmq.common.protocol.body.BrokerStatsData;
import org.apache.rocketmq.common.protocol.body.ClusterInfo;
+import org.apache.rocketmq.common.protocol.body.Connection;
import org.apache.rocketmq.common.protocol.body.ConsumerConnection;
import org.apache.rocketmq.common.protocol.body.GroupList;
import org.apache.rocketmq.common.protocol.body.KVTable;
import org.apache.rocketmq.common.protocol.body.TopicList;
+import org.apache.rocketmq.common.protocol.heartbeat.MessageModel;
import org.apache.rocketmq.common.protocol.route.BrokerData;
import org.apache.rocketmq.common.protocol.route.TopicRouteData;
+import org.apache.rocketmq.exporter.config.CollectClientMetricExecutorConfig;
import org.apache.rocketmq.exporter.config.RMQConfigure;
import org.apache.rocketmq.exporter.model.BrokerRuntimeStats;
+import org.apache.rocketmq.exporter.model.common.TwoTuple;
import org.apache.rocketmq.exporter.service.RMQMetricsService;
import org.apache.rocketmq.exporter.util.Utils;
import org.apache.rocketmq.remoting.exception.RemotingConnectException;
@@ -48,13 +52,24 @@ import org.apache.rocketmq.tools.admin.MQAdminExt;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Qualifier;
+import org.springframework.context.annotation.Bean;
import org.springframework.scheduling.annotation.Scheduled;
import org.springframework.stereotype.Component;
import javax.annotation.PostConstruct;
import javax.annotation.Resource;
+import java.util.ArrayList;
+import java.util.HashSet;
+import java.util.List;
import java.util.Map;
import java.util.Set;
+import java.util.concurrent.BlockingQueue;
+import java.util.concurrent.ExecutorService;
+import java.util.concurrent.LinkedBlockingDeque;
+import java.util.concurrent.ThreadFactory;
+import java.util.concurrent.ThreadPoolExecutor;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.atomic.AtomicLong;
@Component
public class MetricsCollectTask {
@@ -64,9 +79,36 @@ public class MetricsCollectTask {
@Resource
private RMQConfigure rmqConfigure;
@Resource
+ @Qualifier("collectClientMetricExecutor")
+ private ExecutorService collectClientMetricExecutor;
+ @Resource
private RMQMetricsService metricsService;
private final static Logger log = LoggerFactory.getLogger(MetricsCollectTask.class);
+ private BlockingQueue<Runnable> collectClientTaskBlockQueue;
+
+ @Bean(name = "collectClientMetricExecutor")
+ private ExecutorService collectClientMetricExecutor(CollectClientMetricExecutorConfig collectClientMetricExecutorConfig) {
+ collectClientTaskBlockQueue = new LinkedBlockingDeque<Runnable>(collectClientMetricExecutorConfig.getQueueSize());
+ ExecutorService executorService = new ClientMetricCollectorFixedThreadPoolExecutor(
+ collectClientMetricExecutorConfig.getCorePoolSize(),
+ collectClientMetricExecutorConfig.getMaximumPoolSize(),
+ collectClientMetricExecutorConfig.getKeepAliveTime(),
+ TimeUnit.MILLISECONDS,
+ this.collectClientTaskBlockQueue,
+ new ThreadFactory() {
+ private final AtomicLong threadIndex = new AtomicLong(0);
+
+ @Override
+ public Thread newThread(Runnable r) {
+ return new Thread(r, "collectClientMetricThread_" + this.threadIndex.incrementAndGet());
+ }
+ },
+ new ThreadPoolExecutor.DiscardOldestPolicy()
+ );
+ return executorService;
+ }
+
@PostConstruct
public void init() throws InterruptedException, RemotingConnectException, RemotingTimeoutException, RemotingSendRequestException, MQBrokerException {
log.info("MetricsCollectTask init starting....");
@@ -180,8 +222,10 @@ public class MetricsCollectTask {
int countOfOnlineConsumers = 0;
double consumeTPS = 0F;
+ MessageModel messageModel = MessageModel.CLUSTERING;
try {
onlineConsumers = mqAdminExt.examineConsumerConnectionInfo(group);
+ messageModel = onlineConsumers.getMessageModel();
} catch (InterruptedException | RemotingException ex) {
log.error(String.format("get topic's(%s) online consumers(%s) exception", topic, group), ex);
} catch (MQClientException ex) {
@@ -195,6 +239,25 @@ public class MetricsCollectTask {
} else {
countOfOnlineConsumers = onlineConsumers.getConnectionSet().size();
}
+ {
+ String cAddrs = "", localAddrs = "";
+ if (countOfOnlineConsumers > 0) {
+ TwoTuple<String, String> addresses = buildClientAddresses(onlineConsumers.getConnectionSet());
+ cAddrs = addresses.getFirst();
+ localAddrs = addresses.getSecond();
+ }
+ metricsService.getCollector().addGroupCountMetric(group, cAddrs, localAddrs, countOfOnlineConsumers);
+ }
+ if (countOfOnlineConsumers > 0) {
+ collectClientMetricExecutor.submit(new ClientMetricTaskRunnable(
+ group,
+ onlineConsumers,
+ false,
+ this.mqAdminExt,
+ log,
+ this.metricsService
+ ));
+ }
try {
consumeStats = mqAdminExt.examineConsumeStats(group, topic);
} catch (InterruptedException | RemotingException ex) {
@@ -211,7 +274,13 @@ public class MetricsCollectTask {
{
diff = consumeStats.computeTotalDiff();
consumeTPS = consumeStats.getConsumeTps();
- metricsService.getCollector().addGroupDiffMetric(String.valueOf(countOfOnlineConsumers), group, topic, diff);
+ metricsService.getCollector().addGroupDiffMetric(
+ String.valueOf(countOfOnlineConsumers),
+ group,
+ topic,
+ String.valueOf(messageModel.ordinal()),
+ diff
+ );
metricsService.getCollector().addGroupConsumeTPSMetric(topic, group, consumeTPS);
}
Set<Map.Entry<MessageQueue, OffsetWrapper>> consumeStatusEntries = consumeStats.getOffsetTable().entrySet();
@@ -489,6 +558,22 @@ public class MetricsCollectTask {
log.info("broker runtime stats collection task finished...." + (System.currentTimeMillis() - start));
}
+ private static TwoTuple<String, String> buildClientAddresses(HashSet<Connection> connectionSet) {
+ if (connectionSet == null || connectionSet.isEmpty()) {
+ return new TwoTuple<>("", "");
+ }
+ List<String> clientAddresses = new ArrayList<>();
+ List<String> clientIdAddresses = new ArrayList<>();
+
+ for (Connection connection : connectionSet) {
+ clientAddresses.add(connection.getClientAddr());//tcp连接地址
+ clientIdAddresses.add(connection.getClientId());//本地ip组成的id
+ }
+ String str1 = String.join(",", clientAddresses);
+ String str2 = String.join(",", clientIdAddresses);
+ return new TwoTuple<>(str1, str2);
+ }
+
private void handleTopicNotExistException(int responseCode, Exception ex, String topic, String group) {
if (responseCode == ResponseCode.TOPIC_NOT_EXIST || responseCode == ResponseCode.CONSUMER_NOT_ONLINE) {
log.error(String.format("get topic's(%s) consumer-stats(%s) exception, detail: %s", topic, group, ex.getMessage()));
diff --git a/src/main/resources/application.yml b/src/main/resources/application.yml
index 8ff9e07..ac716bb 100644
--- a/src/main/resources/application.yml
+++ b/src/main/resources/application.yml
@@ -15,11 +15,17 @@ rocketmq:
config:
webTelemetryPath: /metrics
rocketmqVersion: 4_2_0
- namesrvAddr: 127.0.0.1:9876 #
+ namesrvAddr: 127.0.0.1:9876
enableCollect: true
+threadpool:
+ collect-client-metric-executor:
+ core-pool-size: 10
+ maximum-pool-size: 10
+ keep-alive-time: 3000
+ queueSize: 5000
task:
- count: 5
+ count: 5 # num of scheduled-tasks
collectTopicOffset:
cron: 15 0/1 * * * ?
collectConsumerOffset:
diff --git a/src/main/resources/logback.xml b/src/main/resources/logback.xml
index 8032516..e54fd39 100644
--- a/src/main/resources/logback.xml
+++ b/src/main/resources/logback.xml
@@ -8,15 +8,14 @@
<appender name="FILE"
class="ch.qos.logback.core.rolling.RollingFileAppender">
- <file>./logs/exporterlogs/rocketmq-exporter.log</file>
+ <file>${user.home}/logs/exporterlogs/rocketmq-exporter.log</file>
<append>true</append>
- <rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
- <fileNamePattern>./logs/exporterlogs/rocketmq-exporter-%d{yyyy-MM-dd}.%i.log
- </fileNamePattern>
- <timeBasedFileNamingAndTriggeringPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP">
- <maxFileSize>104857600</maxFileSize>
- </timeBasedFileNamingAndTriggeringPolicy>
- <totalSizeCap>20gb</totalSizeCap>
+ <rollingPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedRollingPolicy">
+ <fileNamePattern>${user.home}/logs/exporterlogs/rocketmq-exporter-%d{yyyy-MM-dd}.%i.log</fileNamePattern>
+ <maxFileSize>50MB</maxFileSize>
+ <totalSizeCap>10GB</totalSizeCap>
+ <maxHistory>3</maxHistory>
+ <cleanHistoryOnStart>true</cleanHistoryOnStart>
</rollingPolicy>
<encoder>
<pattern>[%d{yyyy-MM-dd HH:mm:ss.SSS}] %5p %m%n</pattern>
[rocketmq-exporter] 26/43: amend the style of use example
Posted by wl...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
wlliqipeng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/rocketmq-exporter.git
commit e0f22e7abfda08f7ae04e347d779b3cffd1ea8be
Author: fengqing <fe...@sunlands.com>
AuthorDate: Mon Jul 22 16:02:54 2019 +0800
amend the style of use example
---
rocketmq_exporter_use_example.md | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/rocketmq_exporter_use_example.md b/rocketmq_exporter_use_example.md
index 17892d9..2bebafe 100644
--- a/rocketmq_exporter_use_example.md
+++ b/rocketmq_exporter_use_example.md
@@ -92,7 +92,7 @@ rule_files:
- /etc/prometheus/rules/*.rules
```
-Secondly, create an alert file in the directory /etc/prometheus/rules/. The content will be like as follows. For more details, please refer to the file [example.rules](./example.rules)
+Secondly, create an alert file in the directory /etc/prometheus/rules/. The content of rules file will be like as follows. For more details, please refer to the file [example.rules](./example.rules)
```
groups:
[rocketmq-exporter] 32/43: Update rocketmq_exporter_use_example.md
Posted by wl...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
wlliqipeng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/rocketmq-exporter.git
commit 0415f4a1aab85bac01cb62d92474ca8a3c6d7924
Author: von gosling <vo...@apache.org>
AuthorDate: Mon Jul 22 17:50:13 2019 +0800
Update rocketmq_exporter_use_example.md
---
rocketmq_exporter_use_example.md | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/rocketmq_exporter_use_example.md b/rocketmq_exporter_use_example.md
index ef1f527..27a90d4 100644
--- a/rocketmq_exporter_use_example.md
+++ b/rocketmq_exporter_use_example.md
@@ -1,4 +1,4 @@
-# RocketMQ-Exporter Use example #
+# RocketMQ Exporter Quick Start #
## 1 Start up NameServer and Broker ##
In order to use the RocketMQ Exporter, firstly make sure that the RocketMQ service is properly downloaded and started. Users can refer to the quick start of the RocketMQ master station for operation. Make sure the NameServer and Broker have started correctly.
[rocketmq-exporter] 43/43: Merge pull request #9 from
francisoliverlee/master
Posted by wl...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
wlliqipeng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/rocketmq-exporter.git
commit 5d25982183c283e62ca63d6cd859dfaa74363b29
Merge: 81c90f3 fd564d9
Author: wlliqipeng <wl...@163.com>
AuthorDate: Sun Mar 8 21:16:10 2020 +0800
Merge pull request #9 from francisoliverlee/master
Collect all metrics
pom.xml | 192 +--
.../exporter/aspect/admin/MQAdminAspect.java | 71 --
.../admin/annotation/MultiMQAdminCmdMethod.java | 30 -
.../exporter/collector/RMQMetricsCollector.java | 1277 ++++++++++++++++++--
.../config/CollectClientMetricExecutorConfig.java | 46 +
.../rocketmq/exporter/config/RMQConfigure.java | 4 +-
.../rocketmq/exporter/config/ScheduleConfig.java | 26 +
.../exporter/controller/RMQMetricsController.java | 9 +-
.../exporter/exception/ServiceException.java | 31 -
.../exporter/model/BrokerRuntimeStats.java | 574 +++++++++
.../rocketmq/exporter/model/common/TwoTuple.java | 19 +
.../exporter/model/metrics/BrokerMetric.java | 43 +-
.../model/metrics/ConsumerCountMetric.java | 58 +
.../exporter/model/metrics/ConsumerMetric.java | 45 +-
.../model/metrics/ConsumerQueueMetric.java | 93 --
.../model/metrics/ConsumerTopicDiffMetric.java | 71 ++
.../model/metrics/DLQTopicOffsetMetric.java | 72 ++
.../exporter/model/metrics/ProducerMetric.java | 61 +-
.../exporter/model/metrics/TopicPutNumMetric.java | 83 ++
.../metrics/brokerruntime/BrokerRuntimeMetric.java | 91 ++
.../ConsumerRuntimeConsumeFailedMsgsMetric.java | 73 ++
.../ConsumerRuntimeConsumeFailedTPSMetric.java | 21 +
.../ConsumerRuntimeConsumeOKTPSMetric.java | 21 +
.../ConsumerRuntimeConsumeRTMetric.java | 21 +
.../clientrunime/ConsumerRuntimePullRTMetric.java | 21 +
.../clientrunime/ConsumerRuntimePullTPSMetric.java | 21 +
.../exporter/service/AbstractCommonService.java | 50 -
.../exporter/service/RMQMetricsService.java | 6 +-
.../exporter/service/client/MQAdminExtImpl.java | 279 +++--
.../exporter/service/client/MQAdminInstance.java | 118 +-
.../service/impl/RMQMetricsServiceImpl.java | 90 +-
...ientMetricCollectorFixedThreadPoolExecutor.java | 24 +
.../exporter/task/ClientMetricTaskRunnable.java | 124 ++
.../rocketmq/exporter/task/MetricsCollectTask.java | 685 +++++++----
.../apache/rocketmq/exporter/util/JsonUtil.java | 39 +-
src/main/resources/application.properties | 14 -
src/main/resources/application.yml | 38 +
src/main/resources/logback.xml | 54 +-
38 files changed, 3567 insertions(+), 1028 deletions(-)
[rocketmq-exporter] 37/43: Polish the user guide
Posted by wl...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
wlliqipeng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/rocketmq-exporter.git
commit ebfa5d4a45316a07d8d2a39ddd2c4eb46d875e27
Author: vongosling <vo...@apache.org>
AuthorDate: Tue Jul 23 14:00:24 2019 +0800
Polish the user guide
---
rocketmq_exporter_quickstart.md | 17 +++++++++--------
1 file changed, 9 insertions(+), 8 deletions(-)
diff --git a/rocketmq_exporter_quickstart.md b/rocketmq_exporter_quickstart.md
index 2f218fe..c75954c 100644
--- a/rocketmq_exporter_quickstart.md
+++ b/rocketmq_exporter_quickstart.md
@@ -1,4 +1,4 @@
-# RocketMQ Exporter Quick Start #
+# Quick Start #
## Start up NameServer and Broker
To use RocketMQ Exporter, first make sure the RocketMQ is downloaded and started correctly. Users can refer to [quick start](http://rocketmq.apache.org/docs/quick-start/) ensure that the service starts properly.
@@ -15,7 +15,7 @@ cd prometheus-2.7.0-rc.1.linux-amd64/
./prometheus --config.file=prometheus.yml --web.listen-address=:5555
```
-The default listening port number of Prometheus is 9090. In order not conflict with other processes on the system, we reset the listening port number to 5555 in the startup parameters. Then go to website http:// sever ip:5555 through browser and users can verify whether the Prometheus has been successfully installed. Since the RocketMQ-Exporter process has been started, the data of RocketMQ-Exporter can be retrieved by Prometheus at this time. At this time, users only need to change th [...]
+The content of prometheus.yml is as follows.
```
# Global config
@@ -44,27 +44,28 @@ global:
-## Create Grafana dashboard for RocketMQ
+## Integration with Grafana Dashboard
+
+Download Grafana installation package and install it.
-Prometheus' own metric display platform is not as good as Grafana. In order to better show RocketMQ's metrics, Grafana can be used to show the metrics that Prometheus gets. Download and install it as the following.
```
wget https://dl.grafana.com/oss/release/grafana-6.2.5.linux-amd64.tar.gz
tar -zxvf grafana-6.2.5.linux-amd64.tar.gz
cd grafana-5.4.3/
```
-Similarly, in order not to conflict with the ports of other processes, users can modify the listening port in the defaults.ini file in the conf directory. Currently, the listening port of the Grafana is changed to 55555, and then use the following command to start up.
+The user can modify the listener port in the default.ini file in the conf directory. Currently, the Grafana listener port changes to 55555 and starts with the following command.
```
./bin/grafana-server web
```
-Then, by accessing http:// server ip:55555 through the browser, users can verify whether the Grafana has been successfully installed. The system default username and password are admin/admin. The first time users log in to the system, users will be asked to change the password. In addition, users need to set Grafana's data source to Prometheus. If user have started up Prometheus like above, now the data source address will be http:// server ip:5555. For the convenience of users, RocketM [...]
+The user can verify that Grafana has been successfully installed by visiting http:// server IP :55555 in a browser. The default system username and password is admin/admin. The first time the user logs into the system, the system will ask the user to change the password. In addition, the user needs to set the data source of Grafana as Prometheus. If Prometheus was launched by the user as above, the data source address will now be http:// server IP :5555. For the convenience of users, Roc [...]
## Configure alarms in Prometheus
-If users want to configure alarms,there are two things users should do.
-Firstly, modify the Prometheus configuration file prometheus.yml and add the following configuration:
+The user wants to configure monitoring alerts, there are two things the user should do.
+First, modify the Prometheus configuration file. And add the following configuration.
```
rule_files:
[rocketmq-exporter] 04/43: Update README.md
Posted by wl...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
wlliqipeng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/rocketmq-exporter.git
commit 58e0eec1ed4531ac05a630d588fdb5c5e0522847
Author: von gosling <vo...@apache.org>
AuthorDate: Thu Apr 4 22:04:47 2019 +0800
Update README.md
---
rocketmq-prometheus-exporter/README.md | 3 ---
1 file changed, 3 deletions(-)
diff --git a/rocketmq-prometheus-exporter/README.md b/rocketmq-prometheus-exporter/README.md
index 268e445..570cd33 100644
--- a/rocketmq-prometheus-exporter/README.md
+++ b/rocketmq-prometheus-exporter/README.md
@@ -51,9 +51,6 @@ mvn clean install
mvn package -Dmaven.test.skip=true docker:build
```
-
-It can be used directly instead of having to build the image yourself. ([Docker Hub breezecoolyang/rocketmq-exporter](https://cloud.docker.com/repository/docker/breezecoolyang/rocketmq-exporter)\)
-
Run
---
[rocketmq-exporter] 41/43: Update README.md
Posted by wl...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
wlliqipeng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/rocketmq-exporter.git
commit 81c90f3d76ee2e8884c8e46ee87ddedfe7126f08
Author: wlliqipeng <wl...@163.com>
AuthorDate: Wed Jan 8 15:19:24 2020 +0800
Update README.md
update maintain information
---
README.md | 8 +++++---
1 file changed, 5 insertions(+), 3 deletions(-)
diff --git a/README.md b/README.md
index afc45e2..55cbb8b 100644
--- a/README.md
+++ b/README.md
@@ -1,7 +1,9 @@
-RocketMQ Exporter
-==============
+# Apache RocketMQ Exporter for Prometheus.
+[](https://www.apache.org/licenses/LICENSE-2.0.html)
+[](http://isitmaintained.com/project/apache/rocketmq-exporter "Average time to resolve an issue")
+[](http://isitmaintained.com/project/apache/rocketmq-exporter "Percentage of issues still open")
+
-Apache RocketMQ Exporter for Prometheus.
Table of Contents
-----------------
[rocketmq-exporter] 18/43: Create PULL_REQUEST_TEMPLATE.md
Posted by wl...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
wlliqipeng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/rocketmq-exporter.git
commit d091776a85b374192f5dc1ef3f23f402ab3f359d
Author: von gosling <vo...@apache.org>
AuthorDate: Fri Jul 5 15:38:44 2019 +0800
Create PULL_REQUEST_TEMPLATE.md
---
.github/PULL_REQUEST_TEMPLATE.md | 20 ++++++++++++++++++++
1 file changed, 20 insertions(+)
diff --git a/.github/PULL_REQUEST_TEMPLATE.md b/.github/PULL_REQUEST_TEMPLATE.md
new file mode 100644
index 0000000..143dac8
--- /dev/null
+++ b/.github/PULL_REQUEST_TEMPLATE.md
@@ -0,0 +1,20 @@
+## What is the purpose of the change
+
+XXXXX
+
+## Brief changelog
+
+XX
+
+## Verifying this change
+
+XXXX
+
+Follow this checklist to help us incorporate your contribution quickly and easily. Notice, `it would be helpful if you could finish the following 5 checklist(the last one is not necessary)before request the community to review your PR`.
+
+- [x] Make sure there is a [Github issue](https://github.com/apache/rocketmq/issues) filed for the change (usually before you start working on it). Trivial changes like typos do not require a Github issue. Your pull request should address just this issue, without pulling in other changes - one PR resolves one issue.
+- [x] Format the pull request title like `[ISSUE #123] Fix UnknownException when host config not exist`. Each commit in the pull request should have a meaningful subject line and body.
+- [x] Write a pull request description that is detailed enough to understand what the pull request does, how, and why.
+- [x] Write necessary unit-test(over 80% coverage) to verify your logic correction, more mock a little better when cross module dependency exist. If the new feature or significant change is committed, please remember to add integration-test in [test module](https://github.com/apache/rocketmq/tree/master/test).
+- [x] Run `mvn -B clean apache-rat:check findbugs:findbugs checkstyle:checkstyle` to make sure basic checks pass. Run `mvn clean install -DskipITs` to make sure unit-test pass. Run `mvn clean test-compile failsafe:integration-test` to make sure integration-test pass.
+- [ ] If this contribution is large, please file an [Apache Individual Contributor License Agreement](http://www.apache.org/licenses/#clas).
[rocketmq-exporter] 01/43: Initial Project
Posted by wl...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
wlliqipeng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/rocketmq-exporter.git
commit 9f0b38a5b9a8cb9b96c20456eb39763990b29602
Author: ShannonDing <li...@163.com>
AuthorDate: Mon Jun 3 15:17:34 2019 +0800
Initial Project
---
README.md | 2 ++
1 file changed, 2 insertions(+)
diff --git a/README.md b/README.md
new file mode 100644
index 0000000..6ae4e8f
--- /dev/null
+++ b/README.md
@@ -0,0 +1,2 @@
+## RocketMQ Exporter
+[](https://www.apache.org/licenses/LICENSE-2.0.html)
[rocketmq-exporter] 40/43: format codes
Posted by wl...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
wlliqipeng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/rocketmq-exporter.git
commit 95153a0f4c19ce6272d4f8d28b75e05d5e15c40a
Author: liwei5 <li...@vipkid.com.cn>
AuthorDate: Thu Dec 12 12:34:20 2019 +0800
format codes
---
pom.xml | 182 ++++++++++-----------
.../exporter/collector/RMQMetricsCollector.java | 2 +-
.../rocketmq/exporter/task/MetricsCollectTask.java | 15 +-
3 files changed, 100 insertions(+), 99 deletions(-)
diff --git a/pom.xml b/pom.xml
index 8a84c73..6a465b3 100644
--- a/pom.xml
+++ b/pom.xml
@@ -1,46 +1,46 @@
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
- xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
- <modelVersion>4.0.0</modelVersion>
- <parent>
- <groupId>org.springframework.boot</groupId>
- <artifactId>spring-boot-starter-parent</artifactId>
- <version>2.1.2.RELEASE</version>
- <relativePath/> <!-- lookup parent from repository -->
- </parent>
- <groupId>org.apache</groupId>
- <artifactId>rocketmq-exporter</artifactId>
- <version>0.0.1-SNAPSHOT</version>
- <name>rocketmq-exporter</name>
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <modelVersion>4.0.0</modelVersion>
+ <parent>
+ <groupId>org.springframework.boot</groupId>
+ <artifactId>spring-boot-starter-parent</artifactId>
+ <version>2.1.2.RELEASE</version>
+ <relativePath/> <!-- lookup parent from repository -->
+ </parent>
+ <groupId>org.apache</groupId>
+ <artifactId>rocketmq-exporter</artifactId>
+ <version>0.0.2-SNAPSHOT</version>
+ <name>rocketmq-exporter</name>
- <description>Demo project for Spring Boot</description>
+ <description>Demo project for Spring Boot</description>
- <properties>
- <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
- <java.version>1.7</java.version>
- <guava.version>16.0.1</guava.version>
- <commons-digester.version>2.1</commons-digester.version>
- <commons-lang.version>2.6</commons-lang.version>
- <commons-io.version>2.4</commons-io.version>
- <commons-cli.version>1.2</commons-cli.version>
- <rocketmq.version>4.4.0</rocketmq.version>
- <surefire.version>2.19.1</surefire.version>
- <aspectj.version>1.8.9</aspectj.version>
- <main.basedir>${basedir}/../..</main.basedir>
- <docker.image.prefix>docker.io</docker.image.prefix>
- </properties>
+ <properties>
+ <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
+ <java.version>1.7</java.version>
+ <guava.version>16.0.1</guava.version>
+ <commons-digester.version>2.1</commons-digester.version>
+ <commons-lang.version>2.6</commons-lang.version>
+ <commons-io.version>2.4</commons-io.version>
+ <commons-cli.version>1.2</commons-cli.version>
+ <rocketmq.version>4.4.0</rocketmq.version>
+ <surefire.version>2.19.1</surefire.version>
+ <aspectj.version>1.8.9</aspectj.version>
+ <main.basedir>${basedir}/../..</main.basedir>
+ <docker.image.prefix>docker.io</docker.image.prefix>
+ </properties>
- <dependencies>
- <dependency>
- <groupId>org.springframework.boot</groupId>
- <artifactId>spring-boot-starter-web</artifactId>
- </dependency>
+ <dependencies>
+ <dependency>
+ <groupId>org.springframework.boot</groupId>
+ <artifactId>spring-boot-starter-web</artifactId>
+ </dependency>
- <dependency>
- <groupId>org.springframework.boot</groupId>
- <artifactId>spring-boot-starter-test</artifactId>
- <scope>test</scope>
- </dependency>
+ <dependency>
+ <groupId>org.springframework.boot</groupId>
+ <artifactId>spring-boot-starter-test</artifactId>
+ <scope>test</scope>
+ </dependency>
<dependency>
<groupId>org.apache.rocketmq</groupId>
<artifactId>rocketmq-tools</artifactId>
@@ -61,16 +61,16 @@
<artifactId>rocketmq-broker</artifactId>
<version>${rocketmq.version}</version>
</dependency>
- <dependency>
- <groupId>org.aspectj</groupId>
- <artifactId>aspectjrt</artifactId>
- <version>${aspectj.version}</version>
- </dependency>
- <dependency>
- <groupId>org.aspectj</groupId>
- <artifactId>aspectjweaver</artifactId>
- <version>${aspectj.version}</version>
- </dependency>
+ <dependency>
+ <groupId>org.aspectj</groupId>
+ <artifactId>aspectjrt</artifactId>
+ <version>${aspectj.version}</version>
+ </dependency>
+ <dependency>
+ <groupId>org.aspectj</groupId>
+ <artifactId>aspectjweaver</artifactId>
+ <version>${aspectj.version}</version>
+ </dependency>
<dependency>
<groupId>org.jooq</groupId>
<artifactId>joor</artifactId>
@@ -86,50 +86,50 @@
<artifactId>simpleclient_common</artifactId>
<version>0.6.0</version>
</dependency>
- </dependencies>
+ </dependencies>
- <build>
- <plugins>
- <plugin>
- <groupId>org.springframework.boot</groupId>
- <artifactId>spring-boot-maven-plugin</artifactId>
- </plugin>
- <plugin>
- <artifactId>maven-checkstyle-plugin</artifactId>
- <version>2.17</version>
- <executions>
- <execution>
- <id>verify</id>
- <phase>verify</phase>
- <configuration>
- <configLocation>style/rmq_checkstyle.xml</configLocation>
- <encoding>UTF-8</encoding>
- <consoleOutput>true</consoleOutput>
- <failsOnError>true</failsOnError>
- <includeTestSourceDirectory>false</includeTestSourceDirectory>
- </configuration>
- <goals>
- <goal>check</goal>
- </goals>
- </execution>
- </executions>
- </plugin>
- <plugin>
- <groupId>com.spotify</groupId>
- <artifactId>docker-maven-plugin</artifactId>
- <version>0.4.11</version>
- <configuration>
- <imageName>${docker.image.prefix}/${project.artifactId}</imageName>
- <dockerDirectory>src/main/docker</dockerDirectory>
- <resources>
- <resource>
- <targetPath>/</targetPath>
- <directory>${project.build.directory}</directory>
- <include>${project.build.finalName}.jar</include>
- </resource>
- </resources>
- </configuration>
- </plugin>
+ <build>
+ <plugins>
+ <plugin>
+ <groupId>org.springframework.boot</groupId>
+ <artifactId>spring-boot-maven-plugin</artifactId>
+ </plugin>
+ <plugin>
+ <artifactId>maven-checkstyle-plugin</artifactId>
+ <version>2.17</version>
+ <executions>
+ <execution>
+ <id>verify</id>
+ <phase>verify</phase>
+ <configuration>
+ <configLocation>style/rmq_checkstyle.xml</configLocation>
+ <encoding>UTF-8</encoding>
+ <consoleOutput>true</consoleOutput>
+ <failsOnError>true</failsOnError>
+ <includeTestSourceDirectory>false</includeTestSourceDirectory>
+ </configuration>
+ <goals>
+ <goal>check</goal>
+ </goals>
+ </execution>
+ </executions>
+ </plugin>
+ <plugin>
+ <groupId>com.spotify</groupId>
+ <artifactId>docker-maven-plugin</artifactId>
+ <version>0.4.11</version>
+ <configuration>
+ <imageName>${docker.image.prefix}/${project.artifactId}</imageName>
+ <dockerDirectory>src/main/docker</dockerDirectory>
+ <resources>
+ <resource>
+ <targetPath>/</targetPath>
+ <directory>${project.build.directory}</directory>
+ <include>${project.build.finalName}.jar</include>
+ </resource>
+ </resources>
+ </configuration>
+ </plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
@@ -140,6 +140,6 @@
</plugin>
</plugins>
- </build>
+ </build>
</project>
diff --git a/src/main/java/org/apache/rocketmq/exporter/collector/RMQMetricsCollector.java b/src/main/java/org/apache/rocketmq/exporter/collector/RMQMetricsCollector.java
index 253e9f9..298b2fe 100644
--- a/src/main/java/org/apache/rocketmq/exporter/collector/RMQMetricsCollector.java
+++ b/src/main/java/org/apache/rocketmq/exporter/collector/RMQMetricsCollector.java
@@ -52,7 +52,7 @@ public class RMQMetricsCollector extends Collector {
private ConcurrentHashMap<ConsumerTopicDiffMetric, Long> consumerDiff = new ConcurrentHashMap<>();
//retry diff for consumer group
private ConcurrentHashMap<ConsumerTopicDiffMetric, Long> consumerRetryDiff = new ConcurrentHashMap<>();
- //死信堆积 todo 检查是否存在这个数据 应该不存在
+ //dlq diff for consumer group
private ConcurrentHashMap<ConsumerTopicDiffMetric, Long> consumerDLQDiff = new ConcurrentHashMap<>();
//consumer count
private ConcurrentHashMap<ConsumerCountMetric, Integer> consumerCounts = new ConcurrentHashMap<>();
diff --git a/src/main/java/org/apache/rocketmq/exporter/task/MetricsCollectTask.java b/src/main/java/org/apache/rocketmq/exporter/task/MetricsCollectTask.java
index f1777ec..b3e7d5e 100644
--- a/src/main/java/org/apache/rocketmq/exporter/task/MetricsCollectTask.java
+++ b/src/main/java/org/apache/rocketmq/exporter/task/MetricsCollectTask.java
@@ -219,13 +219,14 @@ public class MetricsCollectTask {
MessageQueue q = consumeStatusEntry.getKey();
OffsetWrapper offset = consumeStatusEntry.getValue();
- //topic + consumer group 生产offset
+ //topic + consumer group
totalBrokerOffset += totalBrokerOffset + offset.getBrokerOffset();
- //topic + consumer group 消费offset
+ //topic + consumer group
totalConsumerOffset += offset.getConsumerOffset();
}
metricsService.getCollector().addGroupBrokerTotalOffsetMetric(topic, group, totalBrokerOffset);
metricsService.getCollector().addGroupConsumerTotalOffsetMetric(topic, group, totalBrokerOffset);
+
}
}
log.info("consumer offset collection task finished...." + (System.currentTimeMillis() - start));
@@ -276,7 +277,7 @@ public class MetricsCollectTask {
if (!StringUtils.isBlank(masterAddr)) {
BrokerStatsData bsd = null;
try {
- //topic发了多少条消息
+ //how many messages has sent for the topic
bsd = mqAdminExt.viewBrokerStatsData(masterAddr, BrokerStatsManager.TOPIC_PUT_NUMS, topic);
String brokerIP = clusterInfo.getBrokerAddrTable().get(bd.getBrokerName()).getBrokerAddrs().get(MixAll.MASTER_ID);
metricsService.getCollector().addTopicPutNumsMetric(
@@ -297,7 +298,7 @@ public class MetricsCollectTask {
log.error(String.format("TOPIC_PUT_NUMS-error, topic=%s, master broker=%s", topic, masterAddr), ex1);
}
try {
- //topic总共发了多少字节
+ //how many bytes has sent for the topic
bsd = mqAdminExt.viewBrokerStatsData(masterAddr, BrokerStatsManager.TOPIC_PUT_SIZE, topic);
String brokerIP = clusterInfo.getBrokerAddrTable().get(bd.getBrokerName()).getBrokerAddrs().get(MixAll.MASTER_ID);
metricsService.getCollector().addTopicPutSizeMetric(
@@ -338,7 +339,7 @@ public class MetricsCollectTask {
String statsKey = String.format("%s@%s", topic, group);
BrokerStatsData bsd = null;
try {
- //消费者消费了多少条消息
+ //how many messages the consumer has get for the topic
bsd = mqAdminExt.viewBrokerStatsData(masterAddr, BrokerStatsManager.GROUP_GET_NUMS, statsKey);
metricsService.getCollector().addGroupGetNumsMetric(
topic,
@@ -354,7 +355,7 @@ public class MetricsCollectTask {
log.error(String.format("GROUP_GET_NUMS-error, topic=%s, group=%s,master broker=%s", topic, group, masterAddr), ex);
}
try {
- //消费者消费了多少字节
+ //how many bytes the consumer has get for the topic
bsd = mqAdminExt.viewBrokerStatsData(masterAddr, BrokerStatsManager.GROUP_GET_SIZE, statsKey);
metricsService.getCollector().addGroupGetSizeMetric(
topic,
@@ -370,7 +371,7 @@ public class MetricsCollectTask {
log.error(String.format("GROUP_GET_SIZE-error, topic=%s, group=%s, master broker=%s", topic, group, masterAddr), ex);
}
try {
- //消费者重新消费topic的次数
+ ////how many re-send times the consumer did for the topic
bsd = mqAdminExt.viewBrokerStatsData(masterAddr, BrokerStatsManager.SNDBCK_PUT_NUMS, statsKey);
metricsService.getCollector().addSendBackNumsMetric(
topic,
[rocketmq-exporter] 29/43: Merge branch 'master' into master
Posted by wl...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
wlliqipeng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/rocketmq-exporter.git
commit ad60f293df419f586b285fd90b2ea6876ffbaea4
Merge: e788a8c d5cb076
Author: breezecoolyang <br...@users.noreply.github.com>
AuthorDate: Mon Jul 22 17:17:51 2019 +0800
Merge branch 'master' into master
.github/ISSUE_TEMPLATE/bug_report.md | 38 ++
.github/ISSUE_TEMPLATE/feature_request.md | 20 +
.github/PULL_REQUEST_TEMPLATE.md | 20 +
CONTRIBUTING.md | 31 ++
README.md | 428 ++++++++++-----------
src/main/docker/Dockerfile | 4 +-
.../exporter/RocketMQExporterApplication.java | 2 -
.../exporter/aspect/admin/MQAdminAspect.java | 2 +-
.../exporter/service/RMQMetricsService.java | 4 +-
.../rocketmq/exporter/task/MetricsCollectTask.java | 18 +-
.../apache/rocketmq/exporter/util/JsonUtil.java | 2 +-
.../exporter/util/{Mix.java => Utils.java} | 10 +-
src/main/resources/application.properties | 2 +-
13 files changed, 344 insertions(+), 237 deletions(-)
diff --cc README.md
index f97a09b,2d116dc..df34c82
--- a/README.md
+++ b/README.md
@@@ -1,215 -1,210 +1,215 @@@
--RocketMQ_exporter
--==============
--
--RocketMQ exporter for Prometheus.
--
--Table of Contents
-------------------
--- [Compatibility](#compatibility)
--- [Dependency](#dependency)
--- [Download](#download)
--- [Compile](#compile)
-- - [Build Binary](#build-binary)
-- - [Build Docker Image](#build-docker-image)
--- [Run](#run)
-- - [Run Binary](#run-binary)
-- - [Run Docker Image](#run-docker-image)
--- [Flags](#flags)
--- [Metrics](#metrics)
-- - [Brokers](#brokers)
-- - [Topics](#topics)
-- - [Consumer Groups](#consumer-groups)
--- [Grafana Dashboard](#Grafana-Dashboard)
- - [Use Example](#Use-Example)
--
--Compatibility
---------------
--
--Support [Apache RocketMQ](https://rocketmq.apache.org) version 4.3.2 (and later).
--
--Dependency
------------
--
--- [Prometheus](https://prometheus.io)
--
--Compile
---------
--
--### Build Binary
--
--```shell
--mvn clean install
--```
--
--### Build Docker Image
--
--```shell
--mvn package -Dmaven.test.skip=true docker:build
--```
--
--Run
-----
--
--### Run Binary
--
--```shell
- java -jar rocketmq-exporter-0.0.1-SNAPSHOT.jar [--rocketmq.config.namesrvAddr="127.0.0.1:9876" ...]
-java -jar rocketmq-exporter-0.0.1-SNAPSHOT.jar
--```
--
--### Run Docker Image
--
--```
- docker container run -itd --rm -p 5557:5557 breezecoolyang/rocketmq-exporter [--rocketmq.config.namesrvAddr="127.0.0.1:9876" ...]
-docker container run -itd --rm -p 5557:5557 docker.io/rocketmq-exporter
--```
--
--Flags
-----
--
--This image is configurable using different flags
--
--|Flag name | Default | Description |
--| -----------------------------------|--------------------|----------------------------------------------------|
--| `rocketmq.config.namesrvAddr` | 127.0.0.1:9876 |name server address for broker cluster |
--| `rocketmq.config.webTelemetryPath` | /metrics |Path under which to expose metrics |
--| `server.port` | 5557 |Address to listen on for web interface and telemetry|
--| `rocketmq.config.rocketmqVersion` | V4_3_2 |rocketmq broker version |
--
--Metrics
---------
--
--Documents about exposed Prometheus metrics.
--
--### Broker
--
--**Metrics details**
--
--| Name | Exposed information |
--| ------------ | ---------------------------------------------------- |
--| `rocketmq_broker_tps` | Broker produces the number of messages per second |
--| `rocketmq_broker_qps` | Broker consumes messages per second |
--
--**Metrics output example**
--
--```txt
--# HELP rocketmq_broker_tps BrokerPutNums
--# TYPE rocketmq_broker_tps gauge
--rocketmq_broker_tps{cluster="MQCluster",broker="broker-a",} 7.0
--rocketmq_broker_tps{cluster="MQCluster",broker="broker-b",} 7.0
--# HELP rocketmq_broker_qps BrokerGetNums
--# TYPE rocketmq_broker_qps gauge
--rocketmq_broker_qps{cluster="MQCluster",broker="broker-a",} 8.0
--rocketmq_broker_qps{cluster="MQCluster",broker="broker-b",} 8.0
--```
--
--### Topics
--
--**Metrics details**
--
--| Name | Exposed information |
--| ------------------- | -------------------------------------------------- |
--| `rocketmq_producer_tps` | The number of messages produced per second per topic |
--| `rocketmq_producer_message_size` | The size of a message produced per second by a topic (in bytes) |
--| `rocketmq_producer_offset` | The progress of a topic's production message |
--
--**Metrics output example**
--
--```txt
--# HELP rocketmq_producer_tps TopicPutNums
--# TYPE rocketmq_producer_tps gauge
--rocketmq_producer_tps{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",} 7.0
--rocketmq_producer_tps{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",} 7.0
--# HELP rocketmq_producer_message_size TopicPutMessageSize
--# TYPE rocketmq_producer_message_size gauge
--rocketmq_producer_message_size{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",} 1642.0
--rocketmq_producer_message_size{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",} 1638.0
--# HELP rocketmq_producer_offset TopicOffset
--# TYPE rocketmq_producer_offset counter
--rocketmq_producer_offset{cluster="MQCluster",broker="broker-a",topic="TBW102",} 0.0
--rocketmq_producer_offset{cluster="MQCluster",broker="broker-b",topic="DEV_TID_tfq",} 1878633.0
--rocketmq_producer_offset{cluster="MQCluster",broker="broker-a",topic="DEV_TID_tfq",} 3843787.0
--rocketmq_producer_offset{cluster="MQCluster",broker="broker-b",topic="DEV_TID_20190304",} 0.0
--rocketmq_producer_offset{cluster="MQCluster",broker="broker-a",topic="BenchmarkTest",} 0.0
--rocketmq_producer_offset{cluster="MQCluster",broker="broker-b",topic="DEV_TID_20190305",} 0.0
--rocketmq_producer_offset{cluster="MQCluster",broker="broker-b",topic="MQCluster",} 0.0
--rocketmq_producer_offset{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",} 2798195.0
--rocketmq_producer_offset{cluster="MQCluster",broker="broker-b",topic="BenchmarkTest",} 0.0
--rocketmq_producer_offset{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",} 1459666.0
--rocketmq_producer_offset{cluster="MQCluster",broker="broker-a",topic="MQCluster",} 0.0
--rocketmq_producer_offset{cluster="MQCluster",broker="broker-a",topic="SELF_TEST_TOPIC",} 0.0
--rocketmq_producer_offset{cluster="MQCluster",broker="broker-a",topic="OFFSET_MOVED_EVENT",} 0.0
--rocketmq_producer_offset{cluster="MQCluster",broker="broker-b",topic="broker-b",} 0.0
--rocketmq_producer_offset{cluster="MQCluster",broker="broker-a",topic="broker-a",} 0.0
--rocketmq_producer_offset{cluster="MQCluster",broker="broker-b",topic="SELF_TEST_TOPIC",} 0.0
--rocketmq_producer_offset{cluster="MQCluster",broker="broker-b",topic="RMQ_SYS_TRANS_HALF_TOPIC",} 0.0
--rocketmq_producer_offset{cluster="MQCluster",broker="broker-a",topic="DEV_TID_20190305",} 0.0
--rocketmq_producer_offset{cluster="MQCluster",broker="broker-b",topic="OFFSET_MOVED_EVENT",} 0.0
--rocketmq_producer_offset{cluster="MQCluster",broker="broker-a",topic="RMQ_SYS_TRANS_HALF_TOPIC",} 0.0
--rocketmq_producer_offset{cluster="MQCluster",broker="broker-b",topic="TBW102",} 0.0
--rocketmq_producer_offset{cluster="MQCluster",broker="broker-a",topic="DEV_TID_20190304",} 0.0
--
--```
--
--### Consumer Groups
--
--**Metrics details**
--
--| Name | Exposed information |
--| ------------------------------------------------------------ | ------------------------------------------------------------ |
--| `rocketmq_consumer_tps` | The number of messages consumed per second by a consumer group |
--| `rocketmq_consumer_message_size` | The size of the message consumed by the consumer group per second (in bytes) |
--| `rocketmq_consumer_offset` | Progress of consumption message for a consumer group |
--| `rocketmq_group_get_latency` | Consumer latency on some topic for one queue |
--| `rocketmq_group_get_latency_by_storetime ` | Consumption delay time of a consumer group |
- | 消息堆积量（需要通过proSQL聚合）（rocketmq_producer_offset-rocketmq_consumer_offset） | Message accumulation (sum(rocketmq_producer_offset) by (topic) - on(topic) group_right sum(rocketmq_consumer_offset) by (group,topic)) |
-| `rocketmq_message_accumulation`| How far Consumer offset lag behind |
--
--**Metrics output example**
--
--```txt
--# HELP rocketmq_consumer_tps GroupGetNums
--# TYPE rocketmq_consumer_tps gauge
--rocketmq_consumer_tps{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",} 7.0
--rocketmq_consumer_tps{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",} 7.0
--# HELP rocketmq_consumer_message_size GroupGetMessageSize
--# TYPE rocketmq_consumer_message_size gauge
--rocketmq_consumer_message_size{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",} 1638.0
--rocketmq_consumer_message_size{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",} 1642.0
--# HELP rocketmq_consumer_offset GroupOffset
--# TYPE rocketmq_consumer_offset counter
--rocketmq_consumer_offset{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",} 1462030.0
--rocketmq_consumer_offset{cluster="MQCluster",broker="broker-a",topic="DEV_TID_tfq",group="DEV_CID_cfq",} 3843787.0
--rocketmq_consumer_offset{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",} 2800569.0
--rocketmq_consumer_offset{cluster="MQCluster",broker="broker-b",topic="DEV_TID_tfq",group="DEV_CID_cfq",} 1878633.0
--# HELP rocketmq_group_get_latency GroupGetLatency
--# TYPE rocketmq_group_get_latency gauge
--rocketmq_group_get_latency{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="0",} 0.05
--rocketmq_group_get_latency{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="1",} 0.0
--rocketmq_group_get_latency{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="7",} 0.05
--rocketmq_group_get_latency{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="6",} 0.01
--rocketmq_group_get_latency{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="3",} 0.0
--rocketmq_group_get_latency{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="7",} 0.03
--rocketmq_group_get_latency{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="4",} 0.0
--rocketmq_group_get_latency{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="5",} 0.03
--rocketmq_group_get_latency{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="6",} 0.01
--rocketmq_group_get_latency{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="2",} 0.0
--rocketmq_group_get_latency{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="3",} 0.0
--rocketmq_group_get_latency{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="0",} 0.0
--rocketmq_group_get_latency{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="4",} 0.0
--rocketmq_group_get_latency{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="1",} 0.03
--rocketmq_group_get_latency{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="5",} 0.0
--rocketmq_group_get_latency{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="2",} 0.0
--# HELP rocketmq_group_get_latency_by_storetime GroupGetLatencyByStoreTime
--# TYPE rocketmq_group_get_latency_by_storetime gauge
--rocketmq_group_get_latency_by_storetime{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",} 3215.0
--rocketmq_group_get_latency_by_storetime{cluster="MQCluster",broker="broker-a",topic="DEV_TID_tfq",group="DEV_CID_cfq",} 0.0
--rocketmq_group_get_latency_by_storetime{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",} 3232.0
--rocketmq_group_get_latency_by_storetime{cluster="MQCluster",broker="broker-b",topic="DEV_TID_tfq",group="DEV_CID_cfq",} 0.0
--```
--
--Grafana Dashboard
---------
--Grafana Dashboard ID: 10477, name: RocketMQ Exporter Overview.
- For details of the dashboard please see [RocketMQ Exporter Overview](https://grafana.com/dashboards/10477).
-
- Use Example
- -------------
-For details of the dashboard please see [RocketMQ Exporter Overview](https://grafana.com/dashboards/10477).
++RocketMQ_exporter
++==============
++
++RocketMQ exporter for Prometheus.
++
++Table of Contents
++-----------------
++- [Compatibility](#compatibility)
++- [Dependency](#dependency)
++- [Download](#download)
++- [Compile](#compile)
++ - [Build Binary](#build-binary)
++ - [Build Docker Image](#build-docker-image)
++- [Run](#run)
++ - [Run Binary](#run-binary)
++ - [Run Docker Image](#run-docker-image)
++- [Flags](#flags)
++- [Metrics](#metrics)
++ - [Brokers](#brokers)
++ - [Topics](#topics)
++ - [Consumer Groups](#consumer-groups)
++- [Grafana Dashboard](#Grafana-Dashboard)
++- [Use Example](#Use-Example)
++
++Compatibility
++-------------
++
++Support [Apache RocketMQ](https://rocketmq.apache.org) version 4.3.2 (and later).
++
++Dependency
++----------
++
++- [Prometheus](https://prometheus.io)
++
++Compile
++-------
++
++### Build Binary
++
++```shell
++mvn clean install
++```
++
++### Build Docker Image
++
++```shell
++mvn package -Dmaven.test.skip=true docker:build
++```
++
++Run
++---
++
++### Run Binary
++
++```shell
++java -jar rocketmq-exporter-0.0.1-SNAPSHOT.jar
++```
++
++### Run Docker Image
++
++```
++docker container run -itd --rm -p 5557:5557 docker.io/rocketmq-exporter
++```
++
++Flags
++---
++
++This image is configurable using different flags
++
++|Flag name | Default | Description |
++| -----------------------------------|--------------------|----------------------------------------------------|
++| `rocketmq.config.namesrvAddr` | 127.0.0.1:9876 |name server address for broker cluster |
++| `rocketmq.config.webTelemetryPath` | /metrics |Path under which to expose metrics |
++| `server.port` | 5557 |Address to listen on for web interface and telemetry|
++| `rocketmq.config.rocketmqVersion` | V4_3_2 |rocketmq broker version |
++
++Metrics
++-------
++
++Documents about exposed Prometheus metrics.
++
++### Broker
++
++**Metrics details**
++
++| Name | Exposed information |
++| ------------ | ---------------------------------------------------- |
++| `rocketmq_broker_tps` | Broker produces the number of messages per second |
++| `rocketmq_broker_qps` | Broker consumes messages per second |
++
++**Metrics output example**
++
++```txt
++# HELP rocketmq_broker_tps BrokerPutNums
++# TYPE rocketmq_broker_tps gauge
++rocketmq_broker_tps{cluster="MQCluster",broker="broker-a",} 7.0
++rocketmq_broker_tps{cluster="MQCluster",broker="broker-b",} 7.0
++# HELP rocketmq_broker_qps BrokerGetNums
++# TYPE rocketmq_broker_qps gauge
++rocketmq_broker_qps{cluster="MQCluster",broker="broker-a",} 8.0
++rocketmq_broker_qps{cluster="MQCluster",broker="broker-b",} 8.0
++```
++
++### Topics
++
++**Metrics details**
++
++| Name | Exposed information |
++| ------------------- | -------------------------------------------------- |
++| `rocketmq_producer_tps` | The number of messages produced per second per topic |
++| `rocketmq_producer_message_size` | The size of a message produced per second by a topic (in bytes) |
++| `rocketmq_producer_offset` | The progress of a topic's production message |
++
++**Metrics output example**
++
++```txt
++# HELP rocketmq_producer_tps TopicPutNums
++# TYPE rocketmq_producer_tps gauge
++rocketmq_producer_tps{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",} 7.0
++rocketmq_producer_tps{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",} 7.0
++# HELP rocketmq_producer_message_size TopicPutMessageSize
++# TYPE rocketmq_producer_message_size gauge
++rocketmq_producer_message_size{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",} 1642.0
++rocketmq_producer_message_size{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",} 1638.0
++# HELP rocketmq_producer_offset TopicOffset
++# TYPE rocketmq_producer_offset counter
++rocketmq_producer_offset{cluster="MQCluster",broker="broker-a",topic="TBW102",} 0.0
++rocketmq_producer_offset{cluster="MQCluster",broker="broker-b",topic="DEV_TID_tfq",} 1878633.0
++rocketmq_producer_offset{cluster="MQCluster",broker="broker-a",topic="DEV_TID_tfq",} 3843787.0
++rocketmq_producer_offset{cluster="MQCluster",broker="broker-b",topic="DEV_TID_20190304",} 0.0
++rocketmq_producer_offset{cluster="MQCluster",broker="broker-a",topic="BenchmarkTest",} 0.0
++rocketmq_producer_offset{cluster="MQCluster",broker="broker-b",topic="DEV_TID_20190305",} 0.0
++rocketmq_producer_offset{cluster="MQCluster",broker="broker-b",topic="MQCluster",} 0.0
++rocketmq_producer_offset{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",} 2798195.0
++rocketmq_producer_offset{cluster="MQCluster",broker="broker-b",topic="BenchmarkTest",} 0.0
++rocketmq_producer_offset{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",} 1459666.0
++rocketmq_producer_offset{cluster="MQCluster",broker="broker-a",topic="MQCluster",} 0.0
++rocketmq_producer_offset{cluster="MQCluster",broker="broker-a",topic="SELF_TEST_TOPIC",} 0.0
++rocketmq_producer_offset{cluster="MQCluster",broker="broker-a",topic="OFFSET_MOVED_EVENT",} 0.0
++rocketmq_producer_offset{cluster="MQCluster",broker="broker-b",topic="broker-b",} 0.0
++rocketmq_producer_offset{cluster="MQCluster",broker="broker-a",topic="broker-a",} 0.0
++rocketmq_producer_offset{cluster="MQCluster",broker="broker-b",topic="SELF_TEST_TOPIC",} 0.0
++rocketmq_producer_offset{cluster="MQCluster",broker="broker-b",topic="RMQ_SYS_TRANS_HALF_TOPIC",} 0.0
++rocketmq_producer_offset{cluster="MQCluster",broker="broker-a",topic="DEV_TID_20190305",} 0.0
++rocketmq_producer_offset{cluster="MQCluster",broker="broker-b",topic="OFFSET_MOVED_EVENT",} 0.0
++rocketmq_producer_offset{cluster="MQCluster",broker="broker-a",topic="RMQ_SYS_TRANS_HALF_TOPIC",} 0.0
++rocketmq_producer_offset{cluster="MQCluster",broker="broker-b",topic="TBW102",} 0.0
++rocketmq_producer_offset{cluster="MQCluster",broker="broker-a",topic="DEV_TID_20190304",} 0.0
++
++```
++
++### Consumer Groups
++
++**Metrics details**
++
++| Name | Exposed information |
++| ------------------------------------------------------------ | ------------------------------------------------------------ |
++| `rocketmq_consumer_tps` | The number of messages consumed per second by a consumer group |
++| `rocketmq_consumer_message_size` | The size of the message consumed by the consumer group per second (in bytes) |
++| `rocketmq_consumer_offset` | Progress of consumption message for a consumer group |
++| `rocketmq_group_get_latency` | Consumer latency on some topic for one queue |
++| `rocketmq_group_get_latency_by_storetime ` | Consumption delay time of a consumer group |
++| `rocketmq_message_accumulation`| How far Consumer offset lag behind |
++
++**Metrics output example**
++
++```txt
++# HELP rocketmq_consumer_tps GroupGetNums
++# TYPE rocketmq_consumer_tps gauge
++rocketmq_consumer_tps{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",} 7.0
++rocketmq_consumer_tps{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",} 7.0
++# HELP rocketmq_consumer_message_size GroupGetMessageSize
++# TYPE rocketmq_consumer_message_size gauge
++rocketmq_consumer_message_size{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",} 1638.0
++rocketmq_consumer_message_size{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",} 1642.0
++# HELP rocketmq_consumer_offset GroupOffset
++# TYPE rocketmq_consumer_offset counter
++rocketmq_consumer_offset{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",} 1462030.0
++rocketmq_consumer_offset{cluster="MQCluster",broker="broker-a",topic="DEV_TID_tfq",group="DEV_CID_cfq",} 3843787.0
++rocketmq_consumer_offset{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",} 2800569.0
++rocketmq_consumer_offset{cluster="MQCluster",broker="broker-b",topic="DEV_TID_tfq",group="DEV_CID_cfq",} 1878633.0
++# HELP rocketmq_group_get_latency GroupGetLatency
++# TYPE rocketmq_group_get_latency gauge
++rocketmq_group_get_latency{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="0",} 0.05
++rocketmq_group_get_latency{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="1",} 0.0
++rocketmq_group_get_latency{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="7",} 0.05
++rocketmq_group_get_latency{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="6",} 0.01
++rocketmq_group_get_latency{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="3",} 0.0
++rocketmq_group_get_latency{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="7",} 0.03
++rocketmq_group_get_latency{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="4",} 0.0
++rocketmq_group_get_latency{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="5",} 0.03
++rocketmq_group_get_latency{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="6",} 0.01
++rocketmq_group_get_latency{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="2",} 0.0
++rocketmq_group_get_latency{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="3",} 0.0
++rocketmq_group_get_latency{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="0",} 0.0
++rocketmq_group_get_latency{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="4",} 0.0
++rocketmq_group_get_latency{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="1",} 0.03
++rocketmq_group_get_latency{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="5",} 0.0
++rocketmq_group_get_latency{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="2",} 0.0
++# HELP rocketmq_group_get_latency_by_storetime GroupGetLatencyByStoreTime
++# TYPE rocketmq_group_get_latency_by_storetime gauge
++rocketmq_group_get_latency_by_storetime{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",} 3215.0
++rocketmq_group_get_latency_by_storetime{cluster="MQCluster",broker="broker-a",topic="DEV_TID_tfq",group="DEV_CID_cfq",} 0.0
++rocketmq_group_get_latency_by_storetime{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",} 3232.0
++rocketmq_group_get_latency_by_storetime{cluster="MQCluster",broker="broker-b",topic="DEV_TID_tfq",group="DEV_CID_cfq",} 0.0
++```
++
++Grafana Dashboard
++-------
++Grafana Dashboard ID: 10477, name: RocketMQ Exporter Overview.
++For details of the dashboard please see [RocketMQ Exporter Overview](https://grafana.com/dashboards/10477).
++
++Use Example
++-------------
+For details of the use example please refer to [use example](./rocketmq_exporter_use_example.md)
[rocketmq-exporter] 27/43: amend the use example
Posted by wl...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
wlliqipeng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/rocketmq-exporter.git
commit 0133e730277a9f4ca3676f2fcb369cfc1f8cf720
Author: fengqing <fe...@sunlands.com>
AuthorDate: Mon Jul 22 16:08:37 2019 +0800
amend the use example
---
rocketmq_exporter_use_example.md | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/rocketmq_exporter_use_example.md b/rocketmq_exporter_use_example.md
index 2bebafe..5de9e79 100644
--- a/rocketmq_exporter_use_example.md
+++ b/rocketmq_exporter_use_example.md
@@ -80,7 +80,7 @@ Similarly, in order not to conflict with the ports of other processes, users can
./bin/grafana-server web
```
-Then, by accessing http://<server IP address>:55555 through the browser, users can verify whether the Grafana has been successfully installed. The system default username and password are admin/admin. The first time users log in to the system, users will be asked to change the password. In addition, users need to set Grafana's data source to Prometheus. If user have start up Prometheus like above, now the data source address will be http://<server IP address>:5555. For the convenience o [...]
+Then, by accessing http://<server IP address>:55555 through the browser, users can verify whether the Grafana has been successfully installed. The system default username and password are admin/admin. The first time users log in to the system, users will be asked to change the password. In addition, users need to set Grafana's data source to Prometheus. If user have started up Prometheus like above, now the data source address will be http://<server IP address>:5555. For the convenience [...]
## 6 Configure alarms in Prometheus
[rocketmq-exporter] 14/43: update grafana dashboard
Posted by wl...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
wlliqipeng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/rocketmq-exporter.git
commit f53e24f3ca32bcc6c1fb35a691b83fb1f610e009
Author: fengqing <fe...@sunlands.com>
AuthorDate: Wed Jul 3 14:53:27 2019 +0800
update grafana dashboard
---
README.md | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/README.md b/README.md
index b61319d..5fadb98 100644
--- a/README.md
+++ b/README.md
@@ -206,5 +206,5 @@ rocketmq_group_get_latency_by_storetime{cluster="MQCluster",broker="broker-b",to
Grafana Dashboard
-------
-Grafana Dashboard ID: 10405, name: RocketMQ Exporter Overview.
-For details of the dashboard please see [RocketMQ Exporter Overview](https://grafana.com/dashboards/10405).
\ No newline at end of file
+Grafana Dashboard ID: 10476, name: RocketMQ Exporter Overview.
+For details of the dashboard please see [RocketMQ Exporter Overview](https://grafana.com/dashboards/10476).
\ No newline at end of file
[rocketmq-exporter] 06/43: Fix consumer offset repeated calc (#269)
Posted by wl...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
wlliqipeng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/rocketmq-exporter.git
commit 02ef916eb61bd1886a509326dd1e247797f56260
Author: Sip <ea...@hotmail.com>
AuthorDate: Tue May 7 11:48:12 2019 +0800
Fix consumer offset repeated calc (#269)
---
.../main/java/org/apache/rocketmq/exporter/task/MetricsCollectTask.java | 1 +
1 file changed, 1 insertion(+)
diff --git a/rocketmq-prometheus-exporter/src/main/java/org/apache/rocketmq/exporter/task/MetricsCollectTask.java b/rocketmq-prometheus-exporter/src/main/java/org/apache/rocketmq/exporter/task/MetricsCollectTask.java
index 504ac88..3f359e9 100644
--- a/rocketmq-prometheus-exporter/src/main/java/org/apache/rocketmq/exporter/task/MetricsCollectTask.java
+++ b/rocketmq-prometheus-exporter/src/main/java/org/apache/rocketmq/exporter/task/MetricsCollectTask.java
@@ -122,6 +122,7 @@ public class MetricsCollectTask {
for (Map.Entry<String, Long> consumeOffsetEntry : consumeOffsetEntries) {
metricsService.getCollector().AddGroupOffsetMetric(clusterName,consumeOffsetEntry.getKey(), topic, group, consumeOffsetEntry.getValue());
}
+ consumeOffsetMap.clear();
}
}
}
[rocketmq-exporter] 23/43: add set up example for rocketmq exporter
Posted by wl...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
wlliqipeng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/rocketmq-exporter.git
commit 8f4449b75ad0b925b7bdd40486ce0616889d894b
Author: fengqing <fe...@sunlands.com>
AuthorDate: Mon Jul 22 14:24:16 2019 +0800
add set up example for rocketmq exporter
---
rocketmq_exporter_setup_example.md | 101 ----------------------------------
rocketmq_exporter_setup_example_es.md | 57 +++++++++++++++++++
2 files changed, 57 insertions(+), 101 deletions(-)
diff --git a/rocketmq_exporter_setup_example.md b/rocketmq_exporter_setup_example.md
deleted file mode 100644
index a550185..0000000
--- a/rocketmq_exporter_setup_example.md
+++ /dev/null
@@ -1,101 +0,0 @@
-## RocketMQ-Exporter使用示例
-
-1 启动NameServer和Broker
-
-要验证RocketMQ的Spring-Boot客户端，首先要确保RocketMQ服务正确的下载并启动。可以参考RocketMQ主站的快速开始来进行操作。确保启动NameServer和Broker已经正确启动。
-
-2 编译RocketMQ-Exporter
-用户当前使用，需要自行下载git源码编译
-
-```
-git clone https://github.com/apache/rocketmq-exporter
-cd rocketmq-exporter
-mvn clean install
-```
-3 配置和运行
-RocketMQ-Exporter 有如下的运行选项
-选项 | 默认值 | 含义
----|---|---
-rocketmq.config.namesrvAddr | 127.0.0.1:9876 | MQ集群的nameSrv地址
-rocketmq.config.webTelemetryPath | /metrics | 指标搜集路径
-server.port | 5557 | HTTP服务暴露端口
-以上的运行选项既可以在下载代码后在配置文件中更改，也可以通过命令行来设置。
-编译出来的jar包就叫rocketmq-exporter-0.0.1-SNAPSHOT.jar，可以通过如下的方式来运行。
-
-```
-java -jar rocketmq-exporter-0.0.1-SNAPSHOT.jar [--rocketmq.config.namesrvAddr="127.0.0.1:9876" ...]
-```
-
-4 安装Prometheus
-首先到Prometheus官方下载地址:[https://prometheus.io/download/](https://prometheus.io/download/)去下载Prometheus安装包，当前以linux的安装为例，选择的安装包为
-prometheus-2.7.0-rc.1.linux-amd64.tar.gz，经过如下的操作步骤就可以启动prometheus进程。
-
-```
-tar -xzf prometheus-2.7.0-rc.1.linux-amd64.tar.gz
-cd prometheus-2.7.0-rc.1.linux-amd64/
-./prometheus --config.file=prometheus.yml --web.listen-address=:5555
-```
-prometheus 默认监听端口号为9090，为了不与系统上的其它进程监听端口冲突，我们在启动参数里面重新设置了监听端口号为5555。然后通过浏览器访问http://<服务器IP地址>:5555,就可以验证prometheus是否已成功安装。由于RocketMQ-Exporter进程已启动，这个时候可以通过prometheus来抓取RocketMQ-Exporter的数据，这个时候只需要更改prometheus启动的配置文件即可。
-整体配置文件如下：
-
-```
-# my global config
-global:
- scrape_interval: 15s # Set the scrape interval to every 15 seconds. Default is every 1 minute.
- evaluation_interval: 15s # Evaluate rules every 15 seconds. The default is every 1 minute.
- # scrape_timeout is set to the global default (10s).
-
-
- # Load rules once and periodically evaluate them according to the global 'evaluation_interval'.
- rule_files:
- # - "first_rules.yml"
- # - "second_rules.yml"
-
-
- scrape_configs:
- - job_name: 'prometheus'
- static_configs:
- - targets: ['localhost:5555']
-
-
- - job_name: 'exporter'
- static_configs:
- - targets: ['localhost:5557']
-```
-更改配置文件后，重启即可。
-
-5 Grafana dashboard for RocketMQ
-
-Prometheus自身的指标展示平台没有当前流行的展示平台Grafana好， 为了更好的展示RocketMQ的指标，可以使用Grafana来展示Prometheus获取的指标。首先到官网去下载[https://grafana.com/grafana/download](https://grafana.com/grafana/download), 这里仍以二进制文件安装为例进行介绍。
-
-```
-wget https://dl.grafana.com/oss/release/grafana-6.2.5.linux-amd64.tar.gz
-tar -zxvf grafana-6.2.5.linux-amd64.tar.gz
-cd grafana-5.4.3/
-```
-同样为了不与其它进程的端口冲突，可以修改conf目录下的defaults.ini文件的监听端口，当前将这个grafana的监听端口改为55555，然后使用如下的命令启动即可
-
-```
-./bin/grafana-server web
-```
-然后通过浏览器访问http://<服务器IP地址>:55555,就可以验证grafana是否已成功安装。系统默认用户名和密码为admin/admin，第一次登陆系统会要求修改密码，修改密码后登陆。为了便于用户使用，当前已将RocketMQ的dashboard配置文件上传到Grafana的官网，地址为[https://grafana.com/dashboards/10477/revisions](https://grafana.com/dashboards/10477/revisions)去下载dashboard配置文件，用户只需要到下载配置文件，并将配置文件导入dashboard即可创建RocketMQ的dashboard
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
diff --git a/rocketmq_exporter_setup_example_es.md b/rocketmq_exporter_setup_example_es.md
new file mode 100644
index 0000000..5e781b3
--- /dev/null
+++ b/rocketmq_exporter_setup_example_es.md
@@ -0,0 +1,57 @@
+# RocketMQ-Exporter Use example #
+
+## 1 Start up NameServer and Broker ##
+In order to use the RocketMQ Exporter, first make sure that the RocketMQ service is properly downloaded and started. Users can refer to the quick start of the RocketMQ master station for operation. Make sure the NameServer and Broker have started correctly.
+
+## 2 Compile RocketMQ-Exporter ##
+Users currently need to download the git source code and then compile it
+
+```
+git clone https://github.com/apache/rocketmq-exporter
+cd rocketmq-exporter
+mvn clean install
+```
+
+## 3 Configuration and startup ##
+RocketMQ-Exporter has the following running options
+
+operations | default value | meaning
+---|---|---
+rocketmq.config.namesrvAddr | 127.0.0.1:9876 | MQ cluster nameSrv address
+rocketmq.config.webTelemetryPath | /metrics | metric collection path
+server.port | 5557 | HTTP service exposed port
+
+The above running options can be changed either in the configuration file after downloading the code or via the command line. The compiled jar package is called rocketmq-exporter-0.0.1-SNAPSHOT.jar, which can be run as follows.
+
+```
+java -jar rocketmq-exporter-0.0.1-SNAPSHOT.jar [--rocketmq.config.namesrvAddr="127.0.0.1:9876" ...]
+```
+
+## 4 Install Prometheus ##
+First go to Prometheus official download address: https://prometheus.io/download/ to download the Prometheus installation package, currently using linux installation as an example, the selected installation package is Prometheus-2.7.0-rc.1.linux-amd64.tar.gz, the Prometheus process can be started after the following steps.
+
+```
+tar -xzf prometheus-2.7.0-rc.1.linux-amd64.tar.gz
+cd prometheus-2.7.0-rc.1.linux-amd64/
+./prometheus --config.file=prometheus.yml --web.listen-address=:5555
+```
+
+The default listening port number of Prometheus is 9090. In order not conflicts with other processes on the system, we reset the listening port number to 5555 in the startup parameters. Then go to website http://<server IP address>:5555 through browser and users can verify whether the Prometheus has been successfully installed. Since the RocketMQ-Exporter process has been started, the data of RocketMQ-Exporter can be retrieved by Prometheus at this time. At this time, users only need t [...]
+
+## 5 Creating Grafana dashboard for RocketMQ ##
+
+Prometheus' own metric display platform is not as good as Grafana. In order to better show RocketMQ's metrics, Grafana can be used to show the metrics that Prometheus gets. First go to the official website https://grafana.com/grafana/download to download, here is a an example for binary file installation.
+
+```
+wget https://dl.grafana.com/oss/release/grafana-6.2.5.linux-amd64.tar.gz
+tar -zxvf grafana-6.2.5.linux-amd64.tar.gz
+cd grafana-5.4.3/
+```
+Similarly, in order not to conflict with the ports of other processes, users can modify the listening port of the defaults.ini file in the conf directory. Currently, the listening port of the Grafana is changed to 55555, and then use the following command to start.
+
+```
+./bin/grafana-server web
+```
+
+Then, by accessing http://<server IP address>:55555 through the browser, users can verify that the Grafana has been successfully installed. The system default username and password are admin/admin. The first time users log in to the system, users will be asked to change the password. In addition, users need to set Grafana's data source to Prometheus. For the convenience of users, RocketMQ's dashboard configuration file has been uploaded to Grafana's official website https://grafana.com/ [...]
+
[rocketmq-exporter] 15/43: update dashboard id and add message
accmulation metric
Posted by wl...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
wlliqipeng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/rocketmq-exporter.git
commit 82e36f9f795fc4ee0e4cd0d160513e21eecb83b4
Author: fengqing <fe...@sunlands.com>
AuthorDate: Wed Jul 3 15:41:39 2019 +0800
update dashboard id and add message accmulation metric
---
README.md | 19 ++++++++++---------
1 file changed, 10 insertions(+), 9 deletions(-)
diff --git a/README.md b/README.md
index 5fadb98..531878e 100644
--- a/README.md
+++ b/README.md
@@ -153,13 +153,14 @@ rocketmq_producer_offset{cluster="MQCluster",broker="broker-a",topic="DEV_TID_20
**Metrics details**
-| Name | Exposed information |
-| ------------------------------------------ | ------------------------------------------------------------ |
-| `rocketmq_consumer_tps` | The number of messages consumed per second by a consumer group |
-| `rocketmq_consumer_message_size` | The size of the message consumed by the consumer group per second (in bytes) |
-| `rocketmq_consumer_offset` | Progress of consumption message for a consumer group |
-| `rocketmq_group_get_latency` | Consumer latency on some topic for one queue |
-| `rocketmq_group_get_latency_by_storetime ` | Consumption delay time of a consumer group |
+| Name | Exposed information |
+| ------------------------------------------------------------ | ------------------------------------------------------------ |
+| `rocketmq_consumer_tps` | The number of messages consumed per second by a consumer group |
+| `rocketmq_consumer_message_size` | The size of the message consumed by the consumer group per second (in bytes) |
+| `rocketmq_consumer_offset` | Progress of consumption message for a consumer group |
+| `rocketmq_group_get_latency` | Consumer latency on some topic for one queue |
+| `rocketmq_group_get_latency_by_storetime ` | Consumption delay time of a consumer group |
+| 消息堆积量（需要通过proSQL聚合）（rocketmq_producer_offset-rocketmq_consumer_offset） | Message accumulation (sum(rocketmq_producer_offset) by (topic) - on(topic) group_right sum(rocketmq_consumer_offset) by (group,topic)) |
**Metrics output example**
@@ -206,5 +207,5 @@ rocketmq_group_get_latency_by_storetime{cluster="MQCluster",broker="broker-b",to
Grafana Dashboard
-------
-Grafana Dashboard ID: 10476, name: RocketMQ Exporter Overview.
-For details of the dashboard please see [RocketMQ Exporter Overview](https://grafana.com/dashboards/10476).
\ No newline at end of file
+Grafana Dashboard ID: 10477, name: RocketMQ Exporter Overview.
+For details of the dashboard please see [RocketMQ Exporter Overview](https://grafana.com/dashboards/10477).
\ No newline at end of file
[rocketmq-exporter] 11/43: amend some metric name and make the metric have fixed Precision“
Posted by wl...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
wlliqipeng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/rocketmq-exporter.git
commit 2d22503d2cd3722ff11db1f8e61fbe73586a61be
Author: fengqing <fe...@sunlands.com>
AuthorDate: Tue Jul 2 17:42:51 2019 +0800
amend some metric name and make the metric have fixed Precision“
---
README.md | 62 +++++++++++-----------
rocketmq_exporter_overview.json | 8 +--
.../exporter/collector/RMQMetricsCollector.java | 4 +-
.../rocketmq/exporter/task/MetricsCollectTask.java | 17 +++---
.../org/apache/rocketmq/exporter/util/Mix.java | 24 +++++++++
5 files changed, 70 insertions(+), 45 deletions(-)
diff --git a/README.md b/README.md
index d1eda8f..9e04be5 100644
--- a/README.md
+++ b/README.md
@@ -85,16 +85,16 @@ Documents about exposed Prometheus metrics.
| Name | Exposed information |
| ------------ | ---------------------------------------------------- |
-| `rocketmq_broker_tps` | total put message numbers per second for this broker |
-| `rocketmq_broker_qps` | total get message numbers per second for this broker |
+| `rocketmq_broker_tps` | Broker produces the number of messages per second |
+| `rocketmq_broker_qps` | Broker consumes messages per second |
**Metrics output example**
```txt
# HELP rocketmq_broker_tps BrokerPutNums
# TYPE rocketmq_broker_tps gauge
-rocketmq_broker_tps{cluster="MQCluster",broker="broker-a",} 7.933333333333334
-rocketmq_broker_tps{cluster="MQCluster",broker="broker-b",} 7.916666666666667
+rocketmq_broker_tps{cluster="MQCluster",broker="broker-a",} 7.93
+rocketmq_broker_tps{cluster="MQCluster",broker="broker-b",} 7.91
# HELP rocketmq_broker_qps BrokerGetNums
# TYPE rocketmq_broker_qps gauge
rocketmq_broker_qps{cluster="MQCluster",broker="broker-a",} 8.2
@@ -107,21 +107,21 @@ rocketmq_broker_qps{cluster="MQCluster",broker="broker-b",} 8.15
| Name | Exposed information |
| ------------------- | -------------------------------------------------- |
-| `rocketmq_producer_tps` | sending messages number per second for this topic |
-| `rocketmq_producer_put_size` | sending messages size per second for this topic |
-| `rocketmq_producer_offset` | Current Offset of a Broker for this topic |
+| `rocketmq_producer_tps` | The number of messages produced per second per topic |
+| `rocketmq_producer_message_size` | The size of a message produced per second by a topic (in bytes) |
+| `rocketmq_producer_offset` | The progress of a topic's production message |
**Metrics output example**
```txt
# HELP rocketmq_producer_tps TopicPutNums
# TYPE rocketmq_producer_tps gauge
-rocketmq_producer_tps{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",} 7.933333333333334
-rocketmq_producer_tps{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",} 7.916666666666667
-# HELP rocketmq_producer_put_size TopicPutSize
-# TYPE rocketmq_producer_put_size gauge
-rocketmq_producer_put_size{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",} 1642.2
-rocketmq_producer_put_size{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",} 1638.75
+rocketmq_producer_tps{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",} 7.93
+rocketmq_producer_tps{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",} 7.91
+# HELP rocketmq_producer_message_size TopicPutMessageSize
+# TYPE rocketmq_producer_message_size gauge
+rocketmq_producer_message_size{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",} 1642.2
+rocketmq_producer_message_size{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",} 1638.75
# HELP rocketmq_producer_offset TopicOffset
# TYPE rocketmq_producer_offset counter
rocketmq_producer_offset{cluster="MQCluster",broker="broker-a",topic="TBW102",} 0.0
@@ -153,25 +153,25 @@ rocketmq_producer_offset{cluster="MQCluster",broker="broker-a",topic="DEV_TID_20
**Metrics details**
-| Name | Exposed information |
-| --------------------------------- | ------------------------------------------------------------ |
-| `rocketmq_consumer_tps` | consumer message numbers per second for this Topic |
-| `rocketmq_consumer_get_size` | consumer message size per second for this Topic |
-| `rocketmq_consumer_offset` | consumer offset for this topic |
-| `rocketmq_group_get_latency` | consumer latency on some topic for one queue |
-| `rocketmq_group_get_latency_by_storetime ` | consumer latency between message consume time and message store time on some topic |
+| Name | Exposed information |
+| ------------------------------------------ | ------------------------------------------------------------ |
+| `rocketmq_consumer_tps` | The number of messages consumed per second by a consumer group |
+| `rocketmq_consumer_message_size` | The size of the message consumed by the consumer group per second (in bytes) |
+| `rocketmq_consumer_offset` | Progress of consumption message for a consumer group |
+| `rocketmq_group_get_latency` | Consumer latency on some topic for one queue |
+| `rocketmq_group_get_latency_by_storetime ` | Consumption delay time of a consumer group |
**Metrics output example**
```txt
# HELP rocketmq_consumer_tps GroupGetNums
# TYPE rocketmq_consumer_tps gauge
-rocketmq_consumer_tps{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",} 7.916666666666667
-rocketmq_consumer_tps{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",} 7.933333333333334
-# HELP rocketmq_consumer_get_size GroupGetSize
-# TYPE rocketmq_consumer_get_size gauge
-rocketmq_consumer_get_size{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",} 1638.75
-rocketmq_consumer_get_size{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",} 1642.2
+rocketmq_consumer_tps{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",} 7.91
+rocketmq_consumer_tps{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",} 7.93
+# HELP rocketmq_consumer_message_size GroupGetMessageSize
+# TYPE rocketmq_consumer_message_size gauge
+rocketmq_consumer_message_size{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",} 1638.75
+rocketmq_consumer_message_size{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",} 1642.2
# HELP rocketmq_consumer_offset GroupOffset
# TYPE rocketmq_consumer_offset counter
rocketmq_consumer_offset{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",} 1462030.0
@@ -183,17 +183,17 @@ rocketmq_consumer_offset{cluster="MQCluster",broker="broker-b",topic="DEV_TID_tf
rocketmq_group_get_latency{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="0",} 0.05
rocketmq_group_get_latency{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="1",} 0.0
rocketmq_group_get_latency{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="7",} 0.05
-rocketmq_group_get_latency{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="6",} 0.016666666666666666
+rocketmq_group_get_latency{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="6",} 0.01
rocketmq_group_get_latency{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="3",} 0.0
-rocketmq_group_get_latency{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="7",} 0.03333333333333333
+rocketmq_group_get_latency{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="7",} 0.03
rocketmq_group_get_latency{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="4",} 0.0
-rocketmq_group_get_latency{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="5",} 0.03333333333333333
-rocketmq_group_get_latency{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="6",} 0.016666666666666666
+rocketmq_group_get_latency{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="5",} 0.03
+rocketmq_group_get_latency{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="6",} 0.01
rocketmq_group_get_latency{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="2",} 0.0
rocketmq_group_get_latency{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="3",} 0.0
rocketmq_group_get_latency{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="0",} 0.0
rocketmq_group_get_latency{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="4",} 0.0
-rocketmq_group_get_latency{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="1",} 0.03333333333333333
+rocketmq_group_get_latency{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="1",} 0.03
rocketmq_group_get_latency{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="5",} 0.0
rocketmq_group_get_latency{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="2",} 0.0
# HELP rocketmq_group_get_latency_by_storetime GroupGetLatencyByStoreTime
diff --git a/rocketmq_exporter_overview.json b/rocketmq_exporter_overview.json
index 537b21d..0bd1691 100644
--- a/rocketmq_exporter_overview.json
+++ b/rocketmq_exporter_overview.json
@@ -259,7 +259,7 @@
"steppedLine": false,
"targets": [
{
- "expr": "rocketmq_consumer_get_size",
+ "expr": "rocketmq_consumer_message_size",
"format": "time_series",
"intervalFactor": 1,
"refId": "A"
@@ -269,7 +269,7 @@
"timeFrom": null,
"timeRegions": [],
"timeShift": null,
- "title": "rocketmq_consumer_get_size",
+ "title": "rocketmq_consumer_message_size",
"tooltip": {
"shared": true,
"sort": 0,
@@ -514,7 +514,7 @@
"steppedLine": false,
"targets": [
{
- "expr": "rocketmq_producer_put_size",
+ "expr": "rocketmq_producer_message_size",
"format": "time_series",
"intervalFactor": 1,
"refId": "A"
@@ -524,7 +524,7 @@
"timeFrom": null,
"timeRegions": [],
"timeShift": null,
- "title": "rocketmq_producer_put_size",
+ "title": "rocketmq_producer_message_size",
"tooltip": {
"shared": true,
"sort": 0,
diff --git a/src/main/java/org/apache/rocketmq/exporter/collector/RMQMetricsCollector.java b/src/main/java/org/apache/rocketmq/exporter/collector/RMQMetricsCollector.java
index dbaf4dd..2854712 100644
--- a/src/main/java/org/apache/rocketmq/exporter/collector/RMQMetricsCollector.java
+++ b/src/main/java/org/apache/rocketmq/exporter/collector/RMQMetricsCollector.java
@@ -62,7 +62,7 @@ public class RMQMetricsCollector extends Collector {
mfs.add(topicPutNumsGauge);
- GaugeMetricFamily topicPutSizeGauge = new GaugeMetricFamily("rocketmq_producer_put_size", "TopicPutSize", Arrays.asList("cluster","broker","topic"));
+ GaugeMetricFamily topicPutSizeGauge = new GaugeMetricFamily("rocketmq_producer_message_size", "TopicPutMessageSize", Arrays.asList("cluster","broker","topic"));
for (Map.Entry<ProducerMetric, Double> entry: topicPutSize.entrySet()) {
topicPutSizeGauge.addMetric(Arrays.asList(entry.getKey().getClusterName(),entry.getKey().getBrokerName(),entry.getKey().getTopicName()), entry.getValue());
}
@@ -98,7 +98,7 @@ public class RMQMetricsCollector extends Collector {
mfs.add(groupGetNumsGauge);
- GaugeMetricFamily groupGetSizeGauge = new GaugeMetricFamily("rocketmq_consumer_get_size", "GroupGetSize", Arrays.asList("cluster","broker","topic","group"));
+ GaugeMetricFamily groupGetSizeGauge = new GaugeMetricFamily("rocketmq_consumer_message_size", "GroupGetMessageSize", Arrays.asList("cluster","broker","topic","group"));
for (Map.Entry<ConsumerMetric, Double> entry: groupGetSize.entrySet()) {
groupGetSizeGauge.addMetric(Arrays.asList(entry.getKey().getClusterName(),entry.getKey().getBrokerName(),entry.getKey().getTopicName(),entry.getKey().getConsumerGroupName()), entry.getValue());
}
diff --git a/src/main/java/org/apache/rocketmq/exporter/task/MetricsCollectTask.java b/src/main/java/org/apache/rocketmq/exporter/task/MetricsCollectTask.java
index 6ae442d..7797d2a 100644
--- a/src/main/java/org/apache/rocketmq/exporter/task/MetricsCollectTask.java
+++ b/src/main/java/org/apache/rocketmq/exporter/task/MetricsCollectTask.java
@@ -35,6 +35,7 @@ import org.apache.rocketmq.exporter.aspect.admin.annotation.MultiMQAdminCmdMetho
import org.apache.rocketmq.exporter.config.RMQConfigure;
import org.apache.rocketmq.exporter.service.RMQMetricsService;
import org.apache.rocketmq.exporter.service.client.MQAdminExtImpl;
+import org.apache.rocketmq.exporter.util.Mix;
import org.apache.rocketmq.store.stats.BrokerStatsManager;
import org.apache.rocketmq.tools.admin.MQAdminExt;
import org.slf4j.Logger;
@@ -158,14 +159,14 @@ public class MetricsCollectTask {
BrokerStatsData bsd = null;
try {
bsd = mqAdminExt.viewBrokerStatsData(masterAddr, BrokerStatsManager.TOPIC_PUT_NUMS, topic);
- metricsService.getCollector().AddTopicPutNumsMetric(bd.getCluster(), bd.getBrokerName(), topic, bsd.getStatsMinute().getTps());
+ metricsService.getCollector().AddTopicPutNumsMetric(bd.getCluster(), bd.getBrokerName(), topic, Mix.getFixedDouble(bsd.getStatsMinute().getTps()));
}
catch (Exception e) {
log.info("error is " + e.getMessage());
}
try {
bsd = mqAdminExt.viewBrokerStatsData(masterAddr, BrokerStatsManager.TOPIC_PUT_SIZE, topic);
- metricsService.getCollector().AddTopicPutSizeMetric(bd.getCluster(), bd.getBrokerName(), topic, bsd.getStatsMinute().getTps());
+ metricsService.getCollector().AddTopicPutSizeMetric(bd.getCluster(), bd.getBrokerName(), topic, Mix.getFixedDouble(bsd.getStatsMinute().getTps()));
}
catch (Exception e) {
log.info("error is " + e.getMessage());
@@ -185,20 +186,20 @@ public class MetricsCollectTask {
BrokerStatsData bsd = null;
try {
bsd = mqAdminExt.viewBrokerStatsData(masterAddr, BrokerStatsManager.GROUP_GET_NUMS, statsKey);
- metricsService.getCollector().AddGroupGetNumsMetric(bd.getCluster(), bd.getBrokerName(), topic, group, bsd.getStatsMinute().getTps());
+ metricsService.getCollector().AddGroupGetNumsMetric(bd.getCluster(), bd.getBrokerName(), topic, group, Mix.getFixedDouble(bsd.getStatsMinute().getTps()));
} catch (Exception e) {
log.info("error is " + e.getMessage());
}
try {
bsd = mqAdminExt.viewBrokerStatsData(masterAddr, BrokerStatsManager.GROUP_GET_SIZE, statsKey);
- metricsService.getCollector().AddGroupGetSizeMetric(bd.getCluster(), bd.getBrokerName(), topic, group, bsd.getStatsMinute().getTps());
+ metricsService.getCollector().AddGroupGetSizeMetric(bd.getCluster(), bd.getBrokerName(), topic, group, Mix.getFixedDouble(bsd.getStatsMinute().getTps()));
} catch (Exception e) {
log.info("error is " + e.getMessage());
}
try {
bsd = mqAdminExt.viewBrokerStatsData(masterAddr, BrokerStatsManager.SNDBCK_PUT_NUMS, statsKey);
- metricsService.getCollector().AddsendBackNumsMetric(bd.getCluster(), bd.getBrokerName(), topic, group, bsd.getStatsMinute().getTps());
+ metricsService.getCollector().AddsendBackNumsMetric(bd.getCluster(), bd.getBrokerName(), topic, group, Mix.getFixedDouble(bsd.getStatsMinute().getTps()));
} catch (Exception e) {
log.info("error is " + e.getMessage());
}
@@ -238,14 +239,14 @@ public class MetricsCollectTask {
BrokerStatsData bsd = null;
try {
bsd = mqAdminExt.viewBrokerStatsData(masterAddr, BrokerStatsManager.BROKER_PUT_NUMS,clusterEntry.getValue().getCluster());
- metricsService.getCollector().AddBrokerPutNumsMetric(clusterEntry.getValue().getCluster(), clusterEntry.getValue().getBrokerName(), bsd.getStatsMinute().getTps());
+ metricsService.getCollector().AddBrokerPutNumsMetric(clusterEntry.getValue().getCluster(), clusterEntry.getValue().getBrokerName(), Mix.getFixedDouble(bsd.getStatsMinute().getTps()));
}
catch (Exception e) {
log.info("error is " + e.getMessage());
}
try {
bsd = mqAdminExt.viewBrokerStatsData(masterAddr, BrokerStatsManager.BROKER_GET_NUMS, clusterEntry.getValue().getCluster());
- metricsService.getCollector().AddBrokerGetNumsMetric(clusterEntry.getValue().getCluster(), clusterEntry.getValue().getBrokerName(), bsd.getStatsMinute().getTps());
+ metricsService.getCollector().AddBrokerGetNumsMetric(clusterEntry.getValue().getCluster(), clusterEntry.getValue().getBrokerName(), Mix.getFixedDouble(bsd.getStatsMinute().getTps()));
}
catch (Exception e) {
log.info("error is " + e.getMessage());
@@ -273,7 +274,7 @@ public class MetricsCollectTask {
statsKey = String.format("%d@%s@%s", queueId, topic, group);
try {
bsd = mqAdminExt.viewBrokerStatsData(masterAddr, BrokerStatsManager.GROUP_GET_LATENCY, statsKey);
- metricsService.getCollector().AddGroupGetLatencyMetric(bd.getCluster(), bd.getBrokerName(), topic, group, String.format("%d", queueId), bsd.getStatsMinute().getTps());
+ metricsService.getCollector().AddGroupGetLatencyMetric(bd.getCluster(), bd.getBrokerName(), topic, group, String.format("%d", queueId), Mix.getFixedDouble(bsd.getStatsMinute().getTps()));
} catch (Exception e) {
log.info("error is " + e.getMessage());
}
diff --git a/src/main/java/org/apache/rocketmq/exporter/util/Mix.java b/src/main/java/org/apache/rocketmq/exporter/util/Mix.java
new file mode 100644
index 0000000..e51acff
--- /dev/null
+++ b/src/main/java/org/apache/rocketmq/exporter/util/Mix.java
@@ -0,0 +1,24 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.rocketmq.exporter.util;
+
+public class Mix {
+ public static double getFixedDouble(double value) {
+ return Math.round(value * 100) / 100.0;
+ }
+}
[rocketmq-exporter] 17/43: Update issue templates
Posted by wl...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
wlliqipeng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/rocketmq-exporter.git
commit d4fdb8deda5b3dbc9c452860b5600a10e9af720a
Author: von gosling <vo...@apache.org>
AuthorDate: Fri Jul 5 15:14:30 2019 +0800
Update issue templates
---
.github/ISSUE_TEMPLATE/bug_report.md | 38 +++++++++++++++++++++++++++++++
.github/ISSUE_TEMPLATE/feature_request.md | 20 ++++++++++++++++
2 files changed, 58 insertions(+)
diff --git a/.github/ISSUE_TEMPLATE/bug_report.md b/.github/ISSUE_TEMPLATE/bug_report.md
new file mode 100644
index 0000000..dd84ea7
--- /dev/null
+++ b/.github/ISSUE_TEMPLATE/bug_report.md
@@ -0,0 +1,38 @@
+---
+name: Bug report
+about: Create a report to help us improve
+title: ''
+labels: ''
+assignees: ''
+
+---
+
+**Describe the bug**
+A clear and concise description of what the bug is.
+
+**To Reproduce**
+Steps to reproduce the behavior:
+1. Go to '...'
+2. Click on '....'
+3. Scroll down to '....'
+4. See error
+
+**Expected behavior**
+A clear and concise description of what you expected to happen.
+
+**Screenshots**
+If applicable, add screenshots to help explain your problem.
+
+**Desktop (please complete the following information):**
+ - OS: [e.g. iOS]
+ - Browser [e.g. chrome, safari]
+ - Version [e.g. 22]
+
+**Smartphone (please complete the following information):**
+ - Device: [e.g. iPhone6]
+ - OS: [e.g. iOS8.1]
+ - Browser [e.g. stock browser, safari]
+ - Version [e.g. 22]
+
+**Additional context**
+Add any other context about the problem here.
diff --git a/.github/ISSUE_TEMPLATE/feature_request.md b/.github/ISSUE_TEMPLATE/feature_request.md
new file mode 100644
index 0000000..bbcbbe7
--- /dev/null
+++ b/.github/ISSUE_TEMPLATE/feature_request.md
@@ -0,0 +1,20 @@
+---
+name: Feature request
+about: Suggest an idea for this project
+title: ''
+labels: ''
+assignees: ''
+
+---
+
+**Is your feature request related to a problem? Please describe.**
+A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
+
+**Describe the solution you'd like**
+A clear and concise description of what you want to happen.
+
+**Describe alternatives you've considered**
+A clear and concise description of any alternative solutions or features you've considered.
+
+**Additional context**
+Add any other context or screenshots about the feature request here.
[rocketmq-exporter] 34/43: Update README.md
Posted by wl...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
wlliqipeng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/rocketmq-exporter.git
commit 4b313ceb66921a1141846fad04d03a149dcfef74
Author: von gosling <vo...@apache.org>
AuthorDate: Mon Jul 22 19:31:39 2019 +0800
Update README.md
---
README.md | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/README.md b/README.md
index 5f98186..28d236c 100644
--- a/README.md
+++ b/README.md
@@ -20,7 +20,7 @@ Table of Contents
- [Topics](#topics)
- [Consumer Groups](#consumer-groups)
- [Grafana Dashboard](#Grafana-Dashboard)
-- [Use Example](#Use-Example)
+- [Quick Start](#Use-Example)
Compatibility
-------------
@@ -210,6 +210,6 @@ Grafana Dashboard
Grafana Dashboard ID: 10477, name: RocketMQ Exporter Overview.
For details of the dashboard please see [RocketMQ Exporter Overview](https://grafana.com/dashboards/10477).
-Use Example
+Quick Start
-------------
For details of the use example please refer to [use example](./rocketmq_exporter_use_example.md)
[rocketmq-exporter] 08/43: Cherry pick from the external codes
Posted by wl...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
wlliqipeng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/rocketmq-exporter.git
commit 43e8bd2ac16272731c93476ed1265454ca0921fc
Author: vongosling <vo...@apache.org>
AuthorDate: Mon Jun 3 18:56:46 2019 +0800
Cherry pick from the external codes
---
.../.gitignore => .gitignore | 0
rocketmq-prometheus-exporter/LICENSE => LICENSE | 0
README.md | 206 ++++++++++++++++++++-
.../example.rules => example.rules | 0
rocketmq-prometheus-exporter/pom.xml => pom.xml | 0
rocketmq-prometheus-exporter/README.md | 204 --------------------
.../src => src}/main/docker/Dockerfile | 0
.../exporter/RocketMQExporterApplication.java | 0
.../exporter/aspect/admin/MQAdminAspect.java | 0
.../admin/annotation/MultiMQAdminCmdMethod.java | 0
.../exporter/collector/RMQMetricsCollector.java | 0
.../rocketmq/exporter/config/RMQConfigure.java | 0
.../exporter/controller/RMQMetricsController.java | 0
.../exporter/exception/ServiceException.java | 0
.../exporter/model/metrics/BrokerMetric.java | 0
.../exporter/model/metrics/ConsumerMetric.java | 0
.../model/metrics/ConsumerQueueMetric.java | 0
.../exporter/model/metrics/ProducerMetric.java | 0
.../exporter/service/AbstractCommonService.java | 0
.../exporter/service/RMQMetricsService.java | 0
.../exporter/service/client/MQAdminExtImpl.java | 0
.../exporter/service/client/MQAdminInstance.java | 0
.../service/impl/RMQMetricsServiceImpl.java | 0
.../rocketmq/exporter/task/MetricsCollectTask.java | 0
.../apache/rocketmq/exporter/util/JsonUtil.java | 0
.../main/resources/application.properties | 0
.../src => src}/main/resources/logback.xml | 0
.../style => style}/copyright/Apache.xml | 0
.../copyright/profiles_settings.xml | 0
.../style => style}/rmq_checkstyle.xml | 0
.../style => style}/rmq_codeStyle.xml | 0
31 files changed, 204 insertions(+), 206 deletions(-)
diff --git a/rocketmq-prometheus-exporter/.gitignore b/.gitignore
similarity index 100%
rename from rocketmq-prometheus-exporter/.gitignore
rename to .gitignore
diff --git a/rocketmq-prometheus-exporter/LICENSE b/LICENSE
similarity index 100%
rename from rocketmq-prometheus-exporter/LICENSE
rename to LICENSE
diff --git a/README.md b/README.md
index 6ae4e8f..c70f170 100644
--- a/README.md
+++ b/README.md
@@ -1,2 +1,204 @@
-## RocketMQ Exporter
-[](https://www.apache.org/licenses/LICENSE-2.0.html)
+RocketMQ_exporter
+==============
+
+RocketMQ exporter for Prometheus.
+
+Table of Contents
+-----------------
+- [Compatibility](#compatibility)
+- [Dependency](#dependency)
+- [Download](#download)
+- [Compile](#compile)
+ - [Build Binary](#build-binary)
+ - [Build Docker Image](#build-docker-image)
+- [Run](#run)
+ - [Run Binary](#run-binary)
+ - [Run Docker Image](#run-docker-image)
+- [Flags](#flags)
+- [Metrics](#metrics)
+ - [Brokers](#brokers)
+ - [Topics](#topics)
+ - [Consumer Groups](#consumer-groups)
+- [Contribute](#contribute)
+
+Compatibility
+-------------
+
+Support [Apache RocketMQ](https://rocketmq.apache.org) version 4.3.2 (and later).
+
+Dependency
+----------
+
+- [Prometheus](https://prometheus.io)
+
+Compile
+-------
+
+### Build Binary
+
+```shell
+mvn clean install
+```
+
+### Build Docker Image
+
+```shell
+mvn package -Dmaven.test.skip=true docker:build
+```
+
+Run
+---
+
+### Run Binary
+
+```shell
+java -jar rocketmq-exporter-0.0.1-SNAPSHOT.jar [--rocketmq.config.namesrvAddr="127.0.0.1:9876" ...]
+```
+
+### Run Docker Image
+
+```
+docker container run -itd --rm -p 5557:5557 breezecoolyang/rocketmq-exporter [--rocketmq.config.namesrvAddr="127.0.0.1:9876" ...]
+```
+
+Flags
+---
+
+This image is configurable using different flags
+
+|Flag name | Default | Description |
+| -----------------------------------|--------------------|----------------------------------------------------|
+| `rocketmq.config.namesrvAddr` | 127.0.0.1:9876 |name server address for broker cluster |
+| `rocketmq.config.webTelemetryPath` | /metrics |Path under which to expose metrics |
+| `server.port` | 5557 |Address to listen on for web interface and telemetry|
+| `rocketmq.config.rocketmqVersion` | V4_3_2 |rocketmq broker version |
+
+Metrics
+-------
+
+Documents about exposed Prometheus metrics.
+
+### Broker
+
+**Metrics details**
+
+| Name | Exposed information |
+| ------------ | ---------------------------------------------------- |
+| `rocketmq_broker_tps` | total put message numbers per second for this broker |
+| `rocketmq_broker_qps` | total get message numbers per second for this broker |
+
+**Metrics output example**
+
+```txt
+# HELP rocketmq_broker_tps BrokerPutNums
+# TYPE rocketmq_broker_tps gauge
+rocketmq_broker_tps{cluster="MQCluster",broker="broker-a",} 7.933333333333334
+rocketmq_broker_tps{cluster="MQCluster",broker="broker-b",} 7.916666666666667
+# HELP rocketmq_broker_qps BrokerGetNums
+# TYPE rocketmq_broker_qps gauge
+rocketmq_broker_qps{cluster="MQCluster",broker="broker-a",} 8.2
+rocketmq_broker_qps{cluster="MQCluster",broker="broker-b",} 8.15
+```
+
+### Topics
+
+**Metrics details**
+
+| Name | Exposed information |
+| ------------------- | -------------------------------------------------- |
+| `rocketmq_producer_tps` | sending messages number per second for this topic |
+| `rocketmq_producer_put_size` | sending messages size per second for this topic |
+| `rocketmq_producer_offset` | Current Offset of a Broker for this topic |
+
+**Metrics output example**
+
+```txt
+# HELP rocketmq_producer_tps TopicPutNums
+# TYPE rocketmq_producer_tps gauge
+rocketmq_producer_tps{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",} 7.933333333333334
+rocketmq_producer_tps{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",} 7.916666666666667
+# HELP rocketmq_producer_put_size TopicPutSize
+# TYPE rocketmq_producer_put_size gauge
+rocketmq_producer_put_size{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",} 1642.2
+rocketmq_producer_put_size{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",} 1638.75
+# HELP rocketmq_producer_offset TopicOffset
+# TYPE rocketmq_producer_offset counter
+rocketmq_producer_offset{cluster="MQCluster",broker="broker-a",topic="TBW102",} 0.0
+rocketmq_producer_offset{cluster="MQCluster",broker="broker-b",topic="DEV_TID_tfq",} 1878633.0
+rocketmq_producer_offset{cluster="MQCluster",broker="broker-a",topic="DEV_TID_tfq",} 3843787.0
+rocketmq_producer_offset{cluster="MQCluster",broker="broker-b",topic="DEV_TID_20190304",} 0.0
+rocketmq_producer_offset{cluster="MQCluster",broker="broker-a",topic="BenchmarkTest",} 0.0
+rocketmq_producer_offset{cluster="MQCluster",broker="broker-b",topic="DEV_TID_20190305",} 0.0
+rocketmq_producer_offset{cluster="MQCluster",broker="broker-b",topic="MQCluster",} 0.0
+rocketmq_producer_offset{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",} 2798195.0
+rocketmq_producer_offset{cluster="MQCluster",broker="broker-b",topic="BenchmarkTest",} 0.0
+rocketmq_producer_offset{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",} 1459666.0
+rocketmq_producer_offset{cluster="MQCluster",broker="broker-a",topic="MQCluster",} 0.0
+rocketmq_producer_offset{cluster="MQCluster",broker="broker-a",topic="SELF_TEST_TOPIC",} 0.0
+rocketmq_producer_offset{cluster="MQCluster",broker="broker-a",topic="OFFSET_MOVED_EVENT",} 0.0
+rocketmq_producer_offset{cluster="MQCluster",broker="broker-b",topic="broker-b",} 0.0
+rocketmq_producer_offset{cluster="MQCluster",broker="broker-a",topic="broker-a",} 0.0
+rocketmq_producer_offset{cluster="MQCluster",broker="broker-b",topic="SELF_TEST_TOPIC",} 0.0
+rocketmq_producer_offset{cluster="MQCluster",broker="broker-b",topic="RMQ_SYS_TRANS_HALF_TOPIC",} 0.0
+rocketmq_producer_offset{cluster="MQCluster",broker="broker-a",topic="DEV_TID_20190305",} 0.0
+rocketmq_producer_offset{cluster="MQCluster",broker="broker-b",topic="OFFSET_MOVED_EVENT",} 0.0
+rocketmq_producer_offset{cluster="MQCluster",broker="broker-a",topic="RMQ_SYS_TRANS_HALF_TOPIC",} 0.0
+rocketmq_producer_offset{cluster="MQCluster",broker="broker-b",topic="TBW102",} 0.0
+rocketmq_producer_offset{cluster="MQCluster",broker="broker-a",topic="DEV_TID_20190304",} 0.0
+
+```
+
+### Consumer Groups
+
+**Metrics details**
+
+| Name | Exposed information |
+| --------------------------------- | ------------------------------------------------------------ |
+| `rocketmq_consumer_tps` | consumer message numbers per second for this Topic |
+| `rocketmq_consumer_get_size` | consumer message size per second for this Topic |
+| `rocketmq_consumer_offset` | consumer offset for this topic |
+| `rocketmq_group_get_latency` | consumer latency on some topic for one queue |
+| `rocketmq_group_get_latency_by_storetime ` | consumer latency between message consume time and message store time on some topic |
+
+**Metrics output example**
+
+```txt
+# HELP rocketmq_consumer_tps GroupGetNums
+# TYPE rocketmq_consumer_tps gauge
+rocketmq_consumer_tps{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",} 7.916666666666667
+rocketmq_consumer_tps{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",} 7.933333333333334
+# HELP rocketmq_consumer_get_size GroupGetSize
+# TYPE rocketmq_consumer_get_size gauge
+rocketmq_consumer_get_size{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",} 1638.75
+rocketmq_consumer_get_size{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",} 1642.2
+# HELP rocketmq_consumer_offset GroupOffset
+# TYPE rocketmq_consumer_offset counter
+rocketmq_consumer_offset{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",} 1462030.0
+rocketmq_consumer_offset{cluster="MQCluster",broker="broker-a",topic="DEV_TID_tfq",group="DEV_CID_cfq",} 3843787.0
+rocketmq_consumer_offset{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",} 2800569.0
+rocketmq_consumer_offset{cluster="MQCluster",broker="broker-b",topic="DEV_TID_tfq",group="DEV_CID_cfq",} 1878633.0
+# HELP rocketmq_group_get_latency GroupGetLatency
+# TYPE rocketmq_group_get_latency gauge
+rocketmq_group_get_latency{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="0",} 0.05
+rocketmq_group_get_latency{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="1",} 0.0
+rocketmq_group_get_latency{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="7",} 0.05
+rocketmq_group_get_latency{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="6",} 0.016666666666666666
+rocketmq_group_get_latency{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="3",} 0.0
+rocketmq_group_get_latency{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="7",} 0.03333333333333333
+rocketmq_group_get_latency{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="4",} 0.0
+rocketmq_group_get_latency{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="5",} 0.03333333333333333
+rocketmq_group_get_latency{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="6",} 0.016666666666666666
+rocketmq_group_get_latency{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="2",} 0.0
+rocketmq_group_get_latency{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="3",} 0.0
+rocketmq_group_get_latency{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="0",} 0.0
+rocketmq_group_get_latency{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="4",} 0.0
+rocketmq_group_get_latency{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="1",} 0.03333333333333333
+rocketmq_group_get_latency{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="5",} 0.0
+rocketmq_group_get_latency{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="2",} 0.0
+# HELP rocketmq_group_get_latency_by_storetime GroupGetLatencyByStoreTime
+# TYPE rocketmq_group_get_latency_by_storetime gauge
+rocketmq_group_get_latency_by_storetime{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",} 3215.0
+rocketmq_group_get_latency_by_storetime{cluster="MQCluster",broker="broker-a",topic="DEV_TID_tfq",group="DEV_CID_cfq",} 0.0
+rocketmq_group_get_latency_by_storetime{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",} 3232.0
+rocketmq_group_get_latency_by_storetime{cluster="MQCluster",broker="broker-b",topic="DEV_TID_tfq",group="DEV_CID_cfq",} 0.0
+```
diff --git a/rocketmq-prometheus-exporter/example.rules b/example.rules
similarity index 100%
rename from rocketmq-prometheus-exporter/example.rules
rename to example.rules
diff --git a/rocketmq-prometheus-exporter/pom.xml b/pom.xml
similarity index 100%
rename from rocketmq-prometheus-exporter/pom.xml
rename to pom.xml
diff --git a/rocketmq-prometheus-exporter/README.md b/rocketmq-prometheus-exporter/README.md
deleted file mode 100644
index c70f170..0000000
--- a/rocketmq-prometheus-exporter/README.md
+++ /dev/null
@@ -1,204 +0,0 @@
-RocketMQ_exporter
-==============
-
-RocketMQ exporter for Prometheus.
-
-Table of Contents
------------------
-- [Compatibility](#compatibility)
-- [Dependency](#dependency)
-- [Download](#download)
-- [Compile](#compile)
- - [Build Binary](#build-binary)
- - [Build Docker Image](#build-docker-image)
-- [Run](#run)
- - [Run Binary](#run-binary)
- - [Run Docker Image](#run-docker-image)
-- [Flags](#flags)
-- [Metrics](#metrics)
- - [Brokers](#brokers)
- - [Topics](#topics)
- - [Consumer Groups](#consumer-groups)
-- [Contribute](#contribute)
-
-Compatibility
--------------
-
-Support [Apache RocketMQ](https://rocketmq.apache.org) version 4.3.2 (and later).
-
-Dependency
-----------
-
-- [Prometheus](https://prometheus.io)
-
-Compile
--------
-
-### Build Binary
-
-```shell
-mvn clean install
-```
-
-### Build Docker Image
-
-```shell
-mvn package -Dmaven.test.skip=true docker:build
-```
-
-Run
----
-
-### Run Binary
-
-```shell
-java -jar rocketmq-exporter-0.0.1-SNAPSHOT.jar [--rocketmq.config.namesrvAddr="127.0.0.1:9876" ...]
-```
-
-### Run Docker Image
-
-```
-docker container run -itd --rm -p 5557:5557 breezecoolyang/rocketmq-exporter [--rocketmq.config.namesrvAddr="127.0.0.1:9876" ...]
-```
-
-Flags
----
-
-This image is configurable using different flags
-
-|Flag name | Default | Description |
-| -----------------------------------|--------------------|----------------------------------------------------|
-| `rocketmq.config.namesrvAddr` | 127.0.0.1:9876 |name server address for broker cluster |
-| `rocketmq.config.webTelemetryPath` | /metrics |Path under which to expose metrics |
-| `server.port` | 5557 |Address to listen on for web interface and telemetry|
-| `rocketmq.config.rocketmqVersion` | V4_3_2 |rocketmq broker version |
-
-Metrics
--------
-
-Documents about exposed Prometheus metrics.
-
-### Broker
-
-**Metrics details**
-
-| Name | Exposed information |
-| ------------ | ---------------------------------------------------- |
-| `rocketmq_broker_tps` | total put message numbers per second for this broker |
-| `rocketmq_broker_qps` | total get message numbers per second for this broker |
-
-**Metrics output example**
-
-```txt
-# HELP rocketmq_broker_tps BrokerPutNums
-# TYPE rocketmq_broker_tps gauge
-rocketmq_broker_tps{cluster="MQCluster",broker="broker-a",} 7.933333333333334
-rocketmq_broker_tps{cluster="MQCluster",broker="broker-b",} 7.916666666666667
-# HELP rocketmq_broker_qps BrokerGetNums
-# TYPE rocketmq_broker_qps gauge
-rocketmq_broker_qps{cluster="MQCluster",broker="broker-a",} 8.2
-rocketmq_broker_qps{cluster="MQCluster",broker="broker-b",} 8.15
-```
-
-### Topics
-
-**Metrics details**
-
-| Name | Exposed information |
-| ------------------- | -------------------------------------------------- |
-| `rocketmq_producer_tps` | sending messages number per second for this topic |
-| `rocketmq_producer_put_size` | sending messages size per second for this topic |
-| `rocketmq_producer_offset` | Current Offset of a Broker for this topic |
-
-**Metrics output example**
-
-```txt
-# HELP rocketmq_producer_tps TopicPutNums
-# TYPE rocketmq_producer_tps gauge
-rocketmq_producer_tps{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",} 7.933333333333334
-rocketmq_producer_tps{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",} 7.916666666666667
-# HELP rocketmq_producer_put_size TopicPutSize
-# TYPE rocketmq_producer_put_size gauge
-rocketmq_producer_put_size{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",} 1642.2
-rocketmq_producer_put_size{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",} 1638.75
-# HELP rocketmq_producer_offset TopicOffset
-# TYPE rocketmq_producer_offset counter
-rocketmq_producer_offset{cluster="MQCluster",broker="broker-a",topic="TBW102",} 0.0
-rocketmq_producer_offset{cluster="MQCluster",broker="broker-b",topic="DEV_TID_tfq",} 1878633.0
-rocketmq_producer_offset{cluster="MQCluster",broker="broker-a",topic="DEV_TID_tfq",} 3843787.0
-rocketmq_producer_offset{cluster="MQCluster",broker="broker-b",topic="DEV_TID_20190304",} 0.0
-rocketmq_producer_offset{cluster="MQCluster",broker="broker-a",topic="BenchmarkTest",} 0.0
-rocketmq_producer_offset{cluster="MQCluster",broker="broker-b",topic="DEV_TID_20190305",} 0.0
-rocketmq_producer_offset{cluster="MQCluster",broker="broker-b",topic="MQCluster",} 0.0
-rocketmq_producer_offset{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",} 2798195.0
-rocketmq_producer_offset{cluster="MQCluster",broker="broker-b",topic="BenchmarkTest",} 0.0
-rocketmq_producer_offset{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",} 1459666.0
-rocketmq_producer_offset{cluster="MQCluster",broker="broker-a",topic="MQCluster",} 0.0
-rocketmq_producer_offset{cluster="MQCluster",broker="broker-a",topic="SELF_TEST_TOPIC",} 0.0
-rocketmq_producer_offset{cluster="MQCluster",broker="broker-a",topic="OFFSET_MOVED_EVENT",} 0.0
-rocketmq_producer_offset{cluster="MQCluster",broker="broker-b",topic="broker-b",} 0.0
-rocketmq_producer_offset{cluster="MQCluster",broker="broker-a",topic="broker-a",} 0.0
-rocketmq_producer_offset{cluster="MQCluster",broker="broker-b",topic="SELF_TEST_TOPIC",} 0.0
-rocketmq_producer_offset{cluster="MQCluster",broker="broker-b",topic="RMQ_SYS_TRANS_HALF_TOPIC",} 0.0
-rocketmq_producer_offset{cluster="MQCluster",broker="broker-a",topic="DEV_TID_20190305",} 0.0
-rocketmq_producer_offset{cluster="MQCluster",broker="broker-b",topic="OFFSET_MOVED_EVENT",} 0.0
-rocketmq_producer_offset{cluster="MQCluster",broker="broker-a",topic="RMQ_SYS_TRANS_HALF_TOPIC",} 0.0
-rocketmq_producer_offset{cluster="MQCluster",broker="broker-b",topic="TBW102",} 0.0
-rocketmq_producer_offset{cluster="MQCluster",broker="broker-a",topic="DEV_TID_20190304",} 0.0
-
-```
-
-### Consumer Groups
-
-**Metrics details**
-
-| Name | Exposed information |
-| --------------------------------- | ------------------------------------------------------------ |
-| `rocketmq_consumer_tps` | consumer message numbers per second for this Topic |
-| `rocketmq_consumer_get_size` | consumer message size per second for this Topic |
-| `rocketmq_consumer_offset` | consumer offset for this topic |
-| `rocketmq_group_get_latency` | consumer latency on some topic for one queue |
-| `rocketmq_group_get_latency_by_storetime ` | consumer latency between message consume time and message store time on some topic |
-
-**Metrics output example**
-
-```txt
-# HELP rocketmq_consumer_tps GroupGetNums
-# TYPE rocketmq_consumer_tps gauge
-rocketmq_consumer_tps{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",} 7.916666666666667
-rocketmq_consumer_tps{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",} 7.933333333333334
-# HELP rocketmq_consumer_get_size GroupGetSize
-# TYPE rocketmq_consumer_get_size gauge
-rocketmq_consumer_get_size{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",} 1638.75
-rocketmq_consumer_get_size{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",} 1642.2
-# HELP rocketmq_consumer_offset GroupOffset
-# TYPE rocketmq_consumer_offset counter
-rocketmq_consumer_offset{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",} 1462030.0
-rocketmq_consumer_offset{cluster="MQCluster",broker="broker-a",topic="DEV_TID_tfq",group="DEV_CID_cfq",} 3843787.0
-rocketmq_consumer_offset{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",} 2800569.0
-rocketmq_consumer_offset{cluster="MQCluster",broker="broker-b",topic="DEV_TID_tfq",group="DEV_CID_cfq",} 1878633.0
-# HELP rocketmq_group_get_latency GroupGetLatency
-# TYPE rocketmq_group_get_latency gauge
-rocketmq_group_get_latency{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="0",} 0.05
-rocketmq_group_get_latency{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="1",} 0.0
-rocketmq_group_get_latency{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="7",} 0.05
-rocketmq_group_get_latency{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="6",} 0.016666666666666666
-rocketmq_group_get_latency{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="3",} 0.0
-rocketmq_group_get_latency{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="7",} 0.03333333333333333
-rocketmq_group_get_latency{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="4",} 0.0
-rocketmq_group_get_latency{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="5",} 0.03333333333333333
-rocketmq_group_get_latency{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="6",} 0.016666666666666666
-rocketmq_group_get_latency{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="2",} 0.0
-rocketmq_group_get_latency{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="3",} 0.0
-rocketmq_group_get_latency{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="0",} 0.0
-rocketmq_group_get_latency{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="4",} 0.0
-rocketmq_group_get_latency{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="1",} 0.03333333333333333
-rocketmq_group_get_latency{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="5",} 0.0
-rocketmq_group_get_latency{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="2",} 0.0
-# HELP rocketmq_group_get_latency_by_storetime GroupGetLatencyByStoreTime
-# TYPE rocketmq_group_get_latency_by_storetime gauge
-rocketmq_group_get_latency_by_storetime{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",} 3215.0
-rocketmq_group_get_latency_by_storetime{cluster="MQCluster",broker="broker-a",topic="DEV_TID_tfq",group="DEV_CID_cfq",} 0.0
-rocketmq_group_get_latency_by_storetime{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",} 3232.0
-rocketmq_group_get_latency_by_storetime{cluster="MQCluster",broker="broker-b",topic="DEV_TID_tfq",group="DEV_CID_cfq",} 0.0
-```
diff --git a/rocketmq-prometheus-exporter/src/main/docker/Dockerfile b/src/main/docker/Dockerfile
similarity index 100%
rename from rocketmq-prometheus-exporter/src/main/docker/Dockerfile
rename to src/main/docker/Dockerfile
diff --git a/rocketmq-prometheus-exporter/src/main/java/org/apache/rocketmq/exporter/RocketMQExporterApplication.java b/src/main/java/org/apache/rocketmq/exporter/RocketMQExporterApplication.java
similarity index 100%
rename from rocketmq-prometheus-exporter/src/main/java/org/apache/rocketmq/exporter/RocketMQExporterApplication.java
rename to src/main/java/org/apache/rocketmq/exporter/RocketMQExporterApplication.java
diff --git a/rocketmq-prometheus-exporter/src/main/java/org/apache/rocketmq/exporter/aspect/admin/MQAdminAspect.java b/src/main/java/org/apache/rocketmq/exporter/aspect/admin/MQAdminAspect.java
similarity index 100%
rename from rocketmq-prometheus-exporter/src/main/java/org/apache/rocketmq/exporter/aspect/admin/MQAdminAspect.java
rename to src/main/java/org/apache/rocketmq/exporter/aspect/admin/MQAdminAspect.java
diff --git a/rocketmq-prometheus-exporter/src/main/java/org/apache/rocketmq/exporter/aspect/admin/annotation/MultiMQAdminCmdMethod.java b/src/main/java/org/apache/rocketmq/exporter/aspect/admin/annotation/MultiMQAdminCmdMethod.java
similarity index 100%
rename from rocketmq-prometheus-exporter/src/main/java/org/apache/rocketmq/exporter/aspect/admin/annotation/MultiMQAdminCmdMethod.java
rename to src/main/java/org/apache/rocketmq/exporter/aspect/admin/annotation/MultiMQAdminCmdMethod.java
diff --git a/rocketmq-prometheus-exporter/src/main/java/org/apache/rocketmq/exporter/collector/RMQMetricsCollector.java b/src/main/java/org/apache/rocketmq/exporter/collector/RMQMetricsCollector.java
similarity index 100%
rename from rocketmq-prometheus-exporter/src/main/java/org/apache/rocketmq/exporter/collector/RMQMetricsCollector.java
rename to src/main/java/org/apache/rocketmq/exporter/collector/RMQMetricsCollector.java
diff --git a/rocketmq-prometheus-exporter/src/main/java/org/apache/rocketmq/exporter/config/RMQConfigure.java b/src/main/java/org/apache/rocketmq/exporter/config/RMQConfigure.java
similarity index 100%
rename from rocketmq-prometheus-exporter/src/main/java/org/apache/rocketmq/exporter/config/RMQConfigure.java
rename to src/main/java/org/apache/rocketmq/exporter/config/RMQConfigure.java
diff --git a/rocketmq-prometheus-exporter/src/main/java/org/apache/rocketmq/exporter/controller/RMQMetricsController.java b/src/main/java/org/apache/rocketmq/exporter/controller/RMQMetricsController.java
similarity index 100%
rename from rocketmq-prometheus-exporter/src/main/java/org/apache/rocketmq/exporter/controller/RMQMetricsController.java
rename to src/main/java/org/apache/rocketmq/exporter/controller/RMQMetricsController.java
diff --git a/rocketmq-prometheus-exporter/src/main/java/org/apache/rocketmq/exporter/exception/ServiceException.java b/src/main/java/org/apache/rocketmq/exporter/exception/ServiceException.java
similarity index 100%
rename from rocketmq-prometheus-exporter/src/main/java/org/apache/rocketmq/exporter/exception/ServiceException.java
rename to src/main/java/org/apache/rocketmq/exporter/exception/ServiceException.java
diff --git a/rocketmq-prometheus-exporter/src/main/java/org/apache/rocketmq/exporter/model/metrics/BrokerMetric.java b/src/main/java/org/apache/rocketmq/exporter/model/metrics/BrokerMetric.java
similarity index 100%
rename from rocketmq-prometheus-exporter/src/main/java/org/apache/rocketmq/exporter/model/metrics/BrokerMetric.java
rename to src/main/java/org/apache/rocketmq/exporter/model/metrics/BrokerMetric.java
diff --git a/rocketmq-prometheus-exporter/src/main/java/org/apache/rocketmq/exporter/model/metrics/ConsumerMetric.java b/src/main/java/org/apache/rocketmq/exporter/model/metrics/ConsumerMetric.java
similarity index 100%
rename from rocketmq-prometheus-exporter/src/main/java/org/apache/rocketmq/exporter/model/metrics/ConsumerMetric.java
rename to src/main/java/org/apache/rocketmq/exporter/model/metrics/ConsumerMetric.java
diff --git a/rocketmq-prometheus-exporter/src/main/java/org/apache/rocketmq/exporter/model/metrics/ConsumerQueueMetric.java b/src/main/java/org/apache/rocketmq/exporter/model/metrics/ConsumerQueueMetric.java
similarity index 100%
rename from rocketmq-prometheus-exporter/src/main/java/org/apache/rocketmq/exporter/model/metrics/ConsumerQueueMetric.java
rename to src/main/java/org/apache/rocketmq/exporter/model/metrics/ConsumerQueueMetric.java
diff --git a/rocketmq-prometheus-exporter/src/main/java/org/apache/rocketmq/exporter/model/metrics/ProducerMetric.java b/src/main/java/org/apache/rocketmq/exporter/model/metrics/ProducerMetric.java
similarity index 100%
rename from rocketmq-prometheus-exporter/src/main/java/org/apache/rocketmq/exporter/model/metrics/ProducerMetric.java
rename to src/main/java/org/apache/rocketmq/exporter/model/metrics/ProducerMetric.java
diff --git a/rocketmq-prometheus-exporter/src/main/java/org/apache/rocketmq/exporter/service/AbstractCommonService.java b/src/main/java/org/apache/rocketmq/exporter/service/AbstractCommonService.java
similarity index 100%
rename from rocketmq-prometheus-exporter/src/main/java/org/apache/rocketmq/exporter/service/AbstractCommonService.java
rename to src/main/java/org/apache/rocketmq/exporter/service/AbstractCommonService.java
diff --git a/rocketmq-prometheus-exporter/src/main/java/org/apache/rocketmq/exporter/service/RMQMetricsService.java b/src/main/java/org/apache/rocketmq/exporter/service/RMQMetricsService.java
similarity index 100%
rename from rocketmq-prometheus-exporter/src/main/java/org/apache/rocketmq/exporter/service/RMQMetricsService.java
rename to src/main/java/org/apache/rocketmq/exporter/service/RMQMetricsService.java
diff --git a/rocketmq-prometheus-exporter/src/main/java/org/apache/rocketmq/exporter/service/client/MQAdminExtImpl.java b/src/main/java/org/apache/rocketmq/exporter/service/client/MQAdminExtImpl.java
similarity index 100%
rename from rocketmq-prometheus-exporter/src/main/java/org/apache/rocketmq/exporter/service/client/MQAdminExtImpl.java
rename to src/main/java/org/apache/rocketmq/exporter/service/client/MQAdminExtImpl.java
diff --git a/rocketmq-prometheus-exporter/src/main/java/org/apache/rocketmq/exporter/service/client/MQAdminInstance.java b/src/main/java/org/apache/rocketmq/exporter/service/client/MQAdminInstance.java
similarity index 100%
rename from rocketmq-prometheus-exporter/src/main/java/org/apache/rocketmq/exporter/service/client/MQAdminInstance.java
rename to src/main/java/org/apache/rocketmq/exporter/service/client/MQAdminInstance.java
diff --git a/rocketmq-prometheus-exporter/src/main/java/org/apache/rocketmq/exporter/service/impl/RMQMetricsServiceImpl.java b/src/main/java/org/apache/rocketmq/exporter/service/impl/RMQMetricsServiceImpl.java
similarity index 100%
rename from rocketmq-prometheus-exporter/src/main/java/org/apache/rocketmq/exporter/service/impl/RMQMetricsServiceImpl.java
rename to src/main/java/org/apache/rocketmq/exporter/service/impl/RMQMetricsServiceImpl.java
diff --git a/rocketmq-prometheus-exporter/src/main/java/org/apache/rocketmq/exporter/task/MetricsCollectTask.java b/src/main/java/org/apache/rocketmq/exporter/task/MetricsCollectTask.java
similarity index 100%
rename from rocketmq-prometheus-exporter/src/main/java/org/apache/rocketmq/exporter/task/MetricsCollectTask.java
rename to src/main/java/org/apache/rocketmq/exporter/task/MetricsCollectTask.java
diff --git a/rocketmq-prometheus-exporter/src/main/java/org/apache/rocketmq/exporter/util/JsonUtil.java b/src/main/java/org/apache/rocketmq/exporter/util/JsonUtil.java
similarity index 100%
rename from rocketmq-prometheus-exporter/src/main/java/org/apache/rocketmq/exporter/util/JsonUtil.java
rename to src/main/java/org/apache/rocketmq/exporter/util/JsonUtil.java
diff --git a/rocketmq-prometheus-exporter/src/main/resources/application.properties b/src/main/resources/application.properties
similarity index 100%
rename from rocketmq-prometheus-exporter/src/main/resources/application.properties
rename to src/main/resources/application.properties
diff --git a/rocketmq-prometheus-exporter/src/main/resources/logback.xml b/src/main/resources/logback.xml
similarity index 100%
rename from rocketmq-prometheus-exporter/src/main/resources/logback.xml
rename to src/main/resources/logback.xml
diff --git a/rocketmq-prometheus-exporter/style/copyright/Apache.xml b/style/copyright/Apache.xml
similarity index 100%
rename from rocketmq-prometheus-exporter/style/copyright/Apache.xml
rename to style/copyright/Apache.xml
diff --git a/rocketmq-prometheus-exporter/style/copyright/profiles_settings.xml b/style/copyright/profiles_settings.xml
similarity index 100%
rename from rocketmq-prometheus-exporter/style/copyright/profiles_settings.xml
rename to style/copyright/profiles_settings.xml
diff --git a/rocketmq-prometheus-exporter/style/rmq_checkstyle.xml b/style/rmq_checkstyle.xml
similarity index 100%
rename from rocketmq-prometheus-exporter/style/rmq_checkstyle.xml
rename to style/rmq_checkstyle.xml
diff --git a/rocketmq-prometheus-exporter/style/rmq_codeStyle.xml b/style/rmq_codeStyle.xml
similarity index 100%
rename from rocketmq-prometheus-exporter/style/rmq_codeStyle.xml
rename to style/rmq_codeStyle.xml
[rocketmq-exporter] 39/43: add all metrics
Posted by wl...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
wlliqipeng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/rocketmq-exporter.git
commit f0e1125a8ca54e65edf299aab66c38e18564355b
Author: liwei5 <li...@vipkid.com.cn>
AuthorDate: Thu Dec 12 00:17:32 2019 +0800
add all metrics
---
pom.xml | 10 +-
.../exporter/aspect/admin/MQAdminAspect.java | 71 --
.../admin/annotation/MultiMQAdminCmdMethod.java | 30 -
.../exporter/collector/RMQMetricsCollector.java | 1178 ++++++++++++++++++--
.../rocketmq/exporter/config/RMQConfigure.java | 4 +-
.../rocketmq/exporter/config/ScheduleConfig.java | 26 +
.../exporter/controller/RMQMetricsController.java | 9 +-
.../exporter/exception/ServiceException.java | 31 -
.../exporter/model/BrokerRuntimeStats.java | 609 ++++++++++
.../exporter/model/metrics/BrokerMetric.java | 43 +-
.../model/metrics/ConsumerCountMetric.java | 58 +
.../exporter/model/metrics/ConsumerMetric.java | 45 +-
.../model/metrics/ConsumerQueueMetric.java | 93 --
.../model/metrics/ConsumerTopicDiffMetric.java | 61 +
.../model/metrics/DLQTopicOffsetMetric.java | 72 ++
.../exporter/model/metrics/ProducerMetric.java | 61 +-
.../exporter/model/metrics/TopicPutNumMetric.java | 83 ++
.../metrics/brokerruntime/BrokerRuntimeMetric.java | 91 ++
.../exporter/service/AbstractCommonService.java | 50 -
.../exporter/service/RMQMetricsService.java | 6 +-
.../exporter/service/client/MQAdminExtImpl.java | 279 ++---
.../exporter/service/client/MQAdminInstance.java | 118 +-
.../service/impl/RMQMetricsServiceImpl.java | 16 +-
.../rocketmq/exporter/task/MetricsCollectTask.java | 603 ++++++----
.../apache/rocketmq/exporter/util/JsonUtil.java | 39 +-
src/main/resources/application.properties | 14 -
src/main/resources/application.yml | 32 +
src/main/resources/logback.xml | 55 +-
28 files changed, 2849 insertions(+), 938 deletions(-)
diff --git a/pom.xml b/pom.xml
index cda549d..8a84c73 100644
--- a/pom.xml
+++ b/pom.xml
@@ -130,7 +130,15 @@
</resources>
</configuration>
</plugin>
- </plugins>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-compiler-plugin</artifactId>
+ <configuration>
+ <source>8</source>
+ <target>8</target>
+ </configuration>
+ </plugin>
+ </plugins>
</build>
diff --git a/src/main/java/org/apache/rocketmq/exporter/aspect/admin/MQAdminAspect.java b/src/main/java/org/apache/rocketmq/exporter/aspect/admin/MQAdminAspect.java
deleted file mode 100644
index 6b5435f..0000000
--- a/src/main/java/org/apache/rocketmq/exporter/aspect/admin/MQAdminAspect.java
+++ /dev/null
@@ -1,71 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-package org.apache.rocketmq.exporter.aspect.admin;
-
-import java.lang.reflect.Method;
-import org.apache.rocketmq.exporter.aspect.admin.annotation.MultiMQAdminCmdMethod;
-import org.apache.rocketmq.exporter.service.client.MQAdminInstance;
-import org.aspectj.lang.ProceedingJoinPoint;
-import org.aspectj.lang.annotation.Around;
-import org.aspectj.lang.annotation.Aspect;
-import org.aspectj.lang.annotation.Pointcut;
-import org.aspectj.lang.reflect.MethodSignature;
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
-import org.springframework.stereotype.Service;
-
-@Aspect
-@Service
-public class MQAdminAspect {
- private Logger logger = LoggerFactory.getLogger(MQAdminAspect.class);
-
- public MQAdminAspect() {
- }
-
- @Pointcut("execution(* org.apache.rocketmq.exporter.service.client.MQAdminExtImpl..*(..))")
- public void mQAdminMethodPointCut() {
-
- }
-
- @Pointcut("@annotation(org.apache.rocketmq.exporter.aspect.admin.annotation.MultiMQAdminCmdMethod)")
- public void multiMQAdminMethodPointCut() {
-
- }
-
- @Around(value = "mQAdminMethodPointCut() || multiMQAdminMethodPointCut()")
- public Object aroundMQAdminMethod(ProceedingJoinPoint joinPoint) throws Throwable {
- long start = System.currentTimeMillis();
- Object obj;
- try {
- MethodSignature signature = (MethodSignature)joinPoint.getSignature();
- Method method = signature.getMethod();
- MultiMQAdminCmdMethod multiMQAdminCmdMethod = method.getAnnotation(MultiMQAdminCmdMethod.class);
- if (multiMQAdminCmdMethod != null && multiMQAdminCmdMethod.timeoutMillis() > 0) {
- MQAdminInstance.initMQAdminInstance(multiMQAdminCmdMethod.timeoutMillis());
- }
- else {
- MQAdminInstance.initMQAdminInstance(0);
- }
- obj = joinPoint.proceed();
- }
- finally {
- MQAdminInstance.destroyMQAdminInstance();
- logger.debug("op=look method={} cost={}", joinPoint.getSignature().getName(), System.currentTimeMillis() - start);
- }
- return obj;
- }
-}
diff --git a/src/main/java/org/apache/rocketmq/exporter/aspect/admin/annotation/MultiMQAdminCmdMethod.java b/src/main/java/org/apache/rocketmq/exporter/aspect/admin/annotation/MultiMQAdminCmdMethod.java
deleted file mode 100644
index 7953fb3..0000000
--- a/src/main/java/org/apache/rocketmq/exporter/aspect/admin/annotation/MultiMQAdminCmdMethod.java
+++ /dev/null
@@ -1,30 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-package org.apache.rocketmq.exporter.aspect.admin.annotation;
-
-import java.lang.annotation.Documented;
-import java.lang.annotation.ElementType;
-import java.lang.annotation.Retention;
-import java.lang.annotation.RetentionPolicy;
-import java.lang.annotation.Target;
-
-@Target({ElementType.METHOD})
-@Retention(RetentionPolicy.RUNTIME)
-@Documented
-public @interface MultiMQAdminCmdMethod {
- long timeoutMillis() default 0;
-}
diff --git a/src/main/java/org/apache/rocketmq/exporter/collector/RMQMetricsCollector.java b/src/main/java/org/apache/rocketmq/exporter/collector/RMQMetricsCollector.java
index 2854712..253e9f9 100644
--- a/src/main/java/org/apache/rocketmq/exporter/collector/RMQMetricsCollector.java
+++ b/src/main/java/org/apache/rocketmq/exporter/collector/RMQMetricsCollector.java
@@ -17,12 +17,17 @@
package org.apache.rocketmq.exporter.collector;
import io.prometheus.client.Collector;
-import io.prometheus.client.CounterMetricFamily;
import io.prometheus.client.GaugeMetricFamily;
+import org.apache.rocketmq.common.MixAll;
+import org.apache.rocketmq.exporter.model.BrokerRuntimeStats;
import org.apache.rocketmq.exporter.model.metrics.BrokerMetric;
+import org.apache.rocketmq.exporter.model.metrics.ConsumerCountMetric;
import org.apache.rocketmq.exporter.model.metrics.ConsumerMetric;
-import org.apache.rocketmq.exporter.model.metrics.ConsumerQueueMetric;
+import org.apache.rocketmq.exporter.model.metrics.ConsumerTopicDiffMetric;
+import org.apache.rocketmq.exporter.model.metrics.DLQTopicOffsetMetric;
import org.apache.rocketmq.exporter.model.metrics.ProducerMetric;
+import org.apache.rocketmq.exporter.model.metrics.TopicPutNumMetric;
+import org.apache.rocketmq.exporter.model.metrics.brokerruntime.BrokerRuntimeMetric;
import java.util.ArrayList;
import java.util.Arrays;
@@ -31,160 +36,1159 @@ import java.util.Map;
import java.util.concurrent.ConcurrentHashMap;
public class RMQMetricsCollector extends Collector {
+ //max offset of normal consume queue
+ private ConcurrentHashMap<ProducerMetric, Double> topicOffset = new ConcurrentHashMap<>();
+ //max offset of retry topic consume queue
+ private ConcurrentHashMap<ProducerMetric, Double> topicRetryOffset = new ConcurrentHashMap<>();
+ //max offset of dlq consume queue
+ private ConcurrentHashMap<DLQTopicOffsetMetric, Double> topicDLQOffset = new ConcurrentHashMap<>();
- private ConcurrentHashMap<ProducerMetric, Double> topicPutNums = new ConcurrentHashMap<>();
- private ConcurrentHashMap<ProducerMetric, Double> topicPutSize = new ConcurrentHashMap<>();
+ //total put numbers for topics
+ private ConcurrentHashMap<TopicPutNumMetric, Double> topicPutNums = new ConcurrentHashMap<>();
+ //total get numbers for topics
+ private ConcurrentHashMap<TopicPutNumMetric, Double> topicPutSize = new ConcurrentHashMap<>();
- private ConcurrentHashMap<ProducerMetric, Double> topicOffset = new ConcurrentHashMap<>();
+ //diff for consumer group
+ private ConcurrentHashMap<ConsumerTopicDiffMetric, Long> consumerDiff = new ConcurrentHashMap<>();
+ //retry diff for consumer group
+ private ConcurrentHashMap<ConsumerTopicDiffMetric, Long> consumerRetryDiff = new ConcurrentHashMap<>();
+ //死信堆积 todo 检查是否存在这个数据 应该不存在
+ private ConcurrentHashMap<ConsumerTopicDiffMetric, Long> consumerDLQDiff = new ConcurrentHashMap<>();
+ //consumer count
+ private ConcurrentHashMap<ConsumerCountMetric, Integer> consumerCounts = new ConcurrentHashMap<>();
- private ConcurrentHashMap<BrokerMetric, Double> brokerPutNums = new ConcurrentHashMap<>();
- private ConcurrentHashMap<BrokerMetric, Double> brokerGetNums = new ConcurrentHashMap<>();
+ //broker offset for consumer-topic
+ private ConcurrentHashMap<ConsumerMetric, Long> groupBrokerTotalOffset = new ConcurrentHashMap<>();
+ //consumer offset for consumer-topic
+ private ConcurrentHashMap<ConsumerMetric, Long> groupConsumeTotalOffset = new ConcurrentHashMap<>();
+ //consume tps
+ private ConcurrentHashMap<ConsumerMetric, Double> groupConsumeTPS = new ConcurrentHashMap<>();
+ //consumed message count for consumer-topic
+ private ConcurrentHashMap<ConsumerMetric, Double> groupGetNums = new ConcurrentHashMap<>();
+ //consumed message size(byte) for consumer-topic
+ private ConcurrentHashMap<ConsumerMetric, Double> groupGetSize = new ConcurrentHashMap<>();
+ //re-consumed message count for consumer-topic
+ private ConcurrentHashMap<ConsumerMetric, Double> sendBackNums = new ConcurrentHashMap<>();
- private ConcurrentHashMap<ConsumerMetric, Double> groupGetNums = new ConcurrentHashMap<>();
- private ConcurrentHashMap<ConsumerMetric, Double> groupGetSize = new ConcurrentHashMap<>();
+ //total put message count for one broker
+ private ConcurrentHashMap<BrokerMetric, Double> brokerPutNums = new ConcurrentHashMap<>();
+ //total get message count for one broker
+ private ConcurrentHashMap<BrokerMetric, Double> brokerGetNums = new ConcurrentHashMap<>();
- private ConcurrentHashMap<ConsumerMetric, Double> sendBackNums = new ConcurrentHashMap<>();
- private ConcurrentHashMap<ConsumerMetric, Double> groupOffset = new ConcurrentHashMap<>();
+ private ConcurrentHashMap<BrokerRuntimeMetric, Long> brokerRuntimeMsgPutTotalTodayNow = new ConcurrentHashMap<>();
+ private ConcurrentHashMap<BrokerRuntimeMetric, Long> brokerRuntimeMsgGetTotalTodayNow = new ConcurrentHashMap<>();
- private ConcurrentHashMap<ConsumerQueueMetric, Double> groupGetLatency = new ConcurrentHashMap<>();
+ private ConcurrentHashMap<BrokerRuntimeMetric, Long> brokerRuntimeMsgGetTotalYesterdayMorning = new ConcurrentHashMap<>();
+ private ConcurrentHashMap<BrokerRuntimeMetric, Long> brokerRuntimeMsgPutTotalYesterdayMorning = new ConcurrentHashMap<>();
+ private ConcurrentHashMap<BrokerRuntimeMetric, Long> brokerRuntimeMsgGetTotalTodayMorning = new ConcurrentHashMap<>();
+ private ConcurrentHashMap<BrokerRuntimeMetric, Long> brokerRuntimeMsgPutTotalTodayMorning = new ConcurrentHashMap<>();
- private ConcurrentHashMap<ConsumerMetric, Double> groupGetLatencyByStoreTime = new ConcurrentHashMap<>();
+ private ConcurrentHashMap<BrokerRuntimeMetric, Long> brokerRuntimeDispatchBehindBytes = new ConcurrentHashMap<>();
+ private ConcurrentHashMap<BrokerRuntimeMetric, Long> brokerRuntimePutMessageSizeTotal = new ConcurrentHashMap<>();
+
+ private ConcurrentHashMap<BrokerRuntimeMetric, Double> brokerRuntimePutMessageAverageSize = new ConcurrentHashMap<>();
+ private ConcurrentHashMap<BrokerRuntimeMetric, Long> brokerRuntimeQueryThreadPoolQueueCapacity = new ConcurrentHashMap<>();
+ private ConcurrentHashMap<BrokerRuntimeMetric, Long> brokerRuntimeRemainTransientStoreBufferNumbs = new ConcurrentHashMap<>();
+ private ConcurrentHashMap<BrokerRuntimeMetric, Long> brokerRuntimeEarliestMessageTimeStamp = new ConcurrentHashMap<>();
+ private ConcurrentHashMap<BrokerRuntimeMetric, Long> brokerRuntimePutMessageEntireTimeMax = new ConcurrentHashMap<>();
+ private ConcurrentHashMap<BrokerRuntimeMetric, Long> brokerRuntimeStartAcceptSendRequestTimeStamp = new ConcurrentHashMap<>();
+ private ConcurrentHashMap<BrokerRuntimeMetric, Long> brokerRuntimeSendThreadPoolQueueSize = new ConcurrentHashMap<>();
+ private ConcurrentHashMap<BrokerRuntimeMetric, Long> brokerRuntimePutMessageTimesTotal = new ConcurrentHashMap<>();
+ private ConcurrentHashMap<BrokerRuntimeMetric, Long> brokerRuntimeGetMessageEntireTimeMax = new ConcurrentHashMap<>();
+ private ConcurrentHashMap<BrokerRuntimeMetric, Long> brokerRuntimePageCacheLockTimeMills = new ConcurrentHashMap<>();
+ private ConcurrentHashMap<BrokerRuntimeMetric, Double> brokerRuntimeCommitLogDiskRatio = new ConcurrentHashMap<>();
+ private ConcurrentHashMap<BrokerRuntimeMetric, Double> brokerRuntimeConsumeQueueDiskRatio = new ConcurrentHashMap<>();
+
+ private ConcurrentHashMap<BrokerRuntimeMetric, Double> brokerRuntimeGetFoundTps600 = new ConcurrentHashMap<>();
+ private ConcurrentHashMap<BrokerRuntimeMetric, Double> brokerRuntimeGetFoundTps60 = new ConcurrentHashMap<>();
+ private ConcurrentHashMap<BrokerRuntimeMetric, Double> brokerRuntimeGetFoundTps10 = new ConcurrentHashMap<>();
+ private ConcurrentHashMap<BrokerRuntimeMetric, Double> brokerRuntimeGetTotalTps600 = new ConcurrentHashMap<>();
+ private ConcurrentHashMap<BrokerRuntimeMetric, Double> brokerRuntimeGetTotalTps60 = new ConcurrentHashMap<>();
+ private ConcurrentHashMap<BrokerRuntimeMetric, Double> brokerRuntimeGetTotalTps10 = new ConcurrentHashMap<>();
+ private ConcurrentHashMap<BrokerRuntimeMetric, Double> brokerRuntimeGetTransferedTps600 = new ConcurrentHashMap<>();
+ private ConcurrentHashMap<BrokerRuntimeMetric, Double> brokerRuntimeGetTransferedTps60 = new ConcurrentHashMap<>();
+ private ConcurrentHashMap<BrokerRuntimeMetric, Double> brokerRuntimeGetTransferedTps10 = new ConcurrentHashMap<>();
+ private ConcurrentHashMap<BrokerRuntimeMetric, Double> brokerRuntimeGetMissTps600 = new ConcurrentHashMap<>();
+ private ConcurrentHashMap<BrokerRuntimeMetric, Double> brokerRuntimeGetMissTps60 = new ConcurrentHashMap<>();
+ private ConcurrentHashMap<BrokerRuntimeMetric, Double> brokerRuntimeGetMissTps10 = new ConcurrentHashMap<>();
+ private ConcurrentHashMap<BrokerRuntimeMetric, Double> brokerRuntimePutTps600 = new ConcurrentHashMap<>();
+ private ConcurrentHashMap<BrokerRuntimeMetric, Double> brokerRuntimePutTps60 = new ConcurrentHashMap<>();
+ private ConcurrentHashMap<BrokerRuntimeMetric, Double> brokerRuntimePutTps10 = new ConcurrentHashMap<>();
+
+ private ConcurrentHashMap<BrokerRuntimeMetric, Long> brokerRuntimeDispatchMaxBuffer = new ConcurrentHashMap<>();
+
+ private ConcurrentHashMap<BrokerRuntimeMetric, Integer> brokerRuntimePutMessageDistributeTimeMap10toMore = new ConcurrentHashMap<>();
+ private ConcurrentHashMap<BrokerRuntimeMetric, Integer> brokerRuntimePutMessageDistributeTimeMap5to10s = new ConcurrentHashMap<>();
+ private ConcurrentHashMap<BrokerRuntimeMetric, Integer> brokerRuntimePutMessageDistributeTimeMap4to5s = new ConcurrentHashMap<>();
+ private ConcurrentHashMap<BrokerRuntimeMetric, Integer> brokerRuntimePutMessageDistributeTimeMap3to4s = new ConcurrentHashMap<>();
+ private ConcurrentHashMap<BrokerRuntimeMetric, Integer> brokerRuntimePutMessageDistributeTimeMap2to3s = new ConcurrentHashMap<>();
+ private ConcurrentHashMap<BrokerRuntimeMetric, Integer> brokerRuntimePutMessageDistributeTimeMap1to2s = new ConcurrentHashMap<>();
+ private ConcurrentHashMap<BrokerRuntimeMetric, Integer> brokerRuntimePutMessageDistributeTimeMap500to1s = new ConcurrentHashMap<>();
+ private ConcurrentHashMap<BrokerRuntimeMetric, Integer> brokerRuntimePutMessageDistributeTimeMap200to500ms = new ConcurrentHashMap<>();
+ private ConcurrentHashMap<BrokerRuntimeMetric, Integer> brokerRuntimePutMessageDistributeTimeMap100to200ms = new ConcurrentHashMap<>();
+ private ConcurrentHashMap<BrokerRuntimeMetric, Integer> brokerRuntimePutMessageDistributeTimeMap50to100ms = new ConcurrentHashMap<>();
+ private ConcurrentHashMap<BrokerRuntimeMetric, Integer> brokerRuntimePutMessageDistributeTimeMap10to50ms = new ConcurrentHashMap<>();
+ private ConcurrentHashMap<BrokerRuntimeMetric, Integer> brokerRuntimePutMessageDistributeTimeMap0to10ms = new ConcurrentHashMap<>();
+ private ConcurrentHashMap<BrokerRuntimeMetric, Integer> brokerRuntimePutMessageDistributeTimeMap0ms = new ConcurrentHashMap<>();
+
+ private ConcurrentHashMap<BrokerRuntimeMetric, Long> brokerRuntimePullThreadPoolQueueCapacity = new ConcurrentHashMap<>();
+ private ConcurrentHashMap<BrokerRuntimeMetric, Long> brokerRuntimeSendThreadPoolQueueCapacity = new ConcurrentHashMap<>();
+ private ConcurrentHashMap<BrokerRuntimeMetric, Long> brokerRuntimePullThreadPoolQueueSize = new ConcurrentHashMap<>();
+ private ConcurrentHashMap<BrokerRuntimeMetric, Long> brokerRuntimeQueryThreadPoolQueueSize = new ConcurrentHashMap<>();
+
+ private ConcurrentHashMap<BrokerRuntimeMetric, Long> brokerRuntimePullThreadPoolQueueHeadWaitTimeMills = new ConcurrentHashMap<>();
+ private ConcurrentHashMap<BrokerRuntimeMetric, Long> brokerRuntimeQueryThreadPoolQueueHeadWaitTimeMills = new ConcurrentHashMap<>();
+ private ConcurrentHashMap<BrokerRuntimeMetric, Long> brokerRuntimeSendThreadPoolQueueHeadWaitTimeMills = new ConcurrentHashMap<>();
+
+ private ConcurrentHashMap<BrokerRuntimeMetric, Double> brokerRuntimeCommitLogDirCapacityFree = new ConcurrentHashMap<>();
+ private ConcurrentHashMap<BrokerRuntimeMetric, Double> brokerRuntimeCommitLogDirCapacityTotal = new ConcurrentHashMap<>();
+
+ private ConcurrentHashMap<BrokerRuntimeMetric, Long> brokerRuntimeCommitLogMaxOffset = new ConcurrentHashMap<>();
+ private ConcurrentHashMap<BrokerRuntimeMetric, Long> brokerRuntimeCommitLogMinOffset = new ConcurrentHashMap<>();
+ private ConcurrentHashMap<BrokerRuntimeMetric, Double> brokerRuntimeRemainHowManyDataToFlush = new ConcurrentHashMap<>();
+
+ private static List<String> GROUP_DIFF_LABEL_NAMES = Arrays.asList("group", "topic", "countOfOnlineConsumers");
+
+ private static <T extends Number> void loadGroupDiffMetric(GaugeMetricFamily family, Map.Entry<ConsumerTopicDiffMetric, T> entry) {
+ family.addMetric(
+ Arrays.asList(
+ entry.getKey().getGroup(),
+ entry.getKey().getTopic(),
+ entry.getKey().getCountOfOnlineConsumers()
+ ),
+ entry.getValue().doubleValue());
+ }
+
+ private static List<String> GROUP_COUNT_LABEL_NAMES = Arrays.asList("group", "caddr", "localaddr");
+
+ private void collectConsumerMetric(List<MetricFamilySamples> mfs) {
+ GaugeMetricFamily groupGetLatencyByConsumerDiff = new GaugeMetricFamily("rocketmq_group_diff", "GroupDiff", GROUP_DIFF_LABEL_NAMES);
+ for (Map.Entry<ConsumerTopicDiffMetric, Long> entry : consumerDiff.entrySet()) {
+ loadGroupDiffMetric(groupGetLatencyByConsumerDiff, entry);
+ }
+ mfs.add(groupGetLatencyByConsumerDiff);
+
+ GaugeMetricFamily groupGetLatencyByConsumerRetryDiff = new GaugeMetricFamily("rocketmq_group_retrydiff", "GroupRetryDiff", GROUP_DIFF_LABEL_NAMES);
+ for (Map.Entry<ConsumerTopicDiffMetric, Long> entry : consumerRetryDiff.entrySet()) {
+ loadGroupDiffMetric(groupGetLatencyByConsumerRetryDiff, entry);
+ }
+ mfs.add(groupGetLatencyByConsumerRetryDiff);
+
+ GaugeMetricFamily groupGetLatencyByConsumerDLQDiff = new GaugeMetricFamily("rocketmq_group_dlqdiff", "GroupDLQDiff", GROUP_DIFF_LABEL_NAMES);
+ for (Map.Entry<ConsumerTopicDiffMetric, Long> entry : consumerDLQDiff.entrySet()) {
+ loadGroupDiffMetric(groupGetLatencyByConsumerDLQDiff, entry);
+ }
+ mfs.add(groupGetLatencyByConsumerDLQDiff);
+
+ GaugeMetricFamily consumerCountsF = new GaugeMetricFamily("rocketmq_group_count", "GroupCount", GROUP_COUNT_LABEL_NAMES);
+ for (Map.Entry<ConsumerCountMetric, Integer> entry : consumerCounts.entrySet()) {
+ consumerCountsF.addMetric(
+ Arrays.asList(
+ entry.getKey().getGroup(),
+ entry.getKey().getCaddr(),
+ entry.getKey().getLocaladdr()
+ ),
+ entry.getValue().doubleValue());
+ }
+ mfs.add(consumerCountsF);
+
+ }
+
+
+ private static List<String> TOPIC_OFFSET_LABEL_NAMES = Arrays.asList(
+ "cluster", "brokerNames", "topic", "lastUpdateTimestamp"
+ );
+
+ private static List<String> DLQ_TOPIC_OFFSET_LABEL_NAMES = Arrays.asList(
+ "cluster", "brokerNames", "group", "lastUpdateTimestamp"
+ );
+
+ private void loadTopicOffsetMetric(GaugeMetricFamily family, Map.Entry<ProducerMetric, Double> entry) {
+ family.addMetric(
+ Arrays.asList(
+ entry.getKey().getClusterName(),
+ entry.getKey().getBrokerNames(),
+ entry.getKey().getTopicName(),
+ String.valueOf(entry.getKey().getLastUpdateTimestamp())
+ ),
+ entry.getValue());
+ }
+
+ private void collectTopicOffsetMetric(List<MetricFamilySamples> mfs) {
+ GaugeMetricFamily topicOffsetF = new GaugeMetricFamily("rocketmq_topic_offset", "TopicOffset", TOPIC_OFFSET_LABEL_NAMES);
+ for (Map.Entry<ProducerMetric, Double> entry : topicOffset.entrySet()) {
+ loadTopicOffsetMetric(topicOffsetF, entry);
+ }
+ mfs.add(topicOffsetF);
+
+ GaugeMetricFamily topicRetryOffsetF = new GaugeMetricFamily("rocketmq_topic_retry_offset", "TopicRetryOffset", TOPIC_OFFSET_LABEL_NAMES);
+ for (Map.Entry<ProducerMetric, Double> entry : topicRetryOffset.entrySet()) {
+ loadTopicOffsetMetric(topicRetryOffsetF, entry);
+ }
+ mfs.add(topicRetryOffsetF);
+
+ GaugeMetricFamily topicDLQOffsetF = new GaugeMetricFamily("rocketmq_topic_dlq_offset", "TopicRetryOffset", DLQ_TOPIC_OFFSET_LABEL_NAMES);
+ for (Map.Entry<DLQTopicOffsetMetric, Double> entry : topicDLQOffset.entrySet()) {
+ topicDLQOffsetF.addMetric(
+ Arrays.asList(
+ entry.getKey().getClusterName(),
+ entry.getKey().getBrokerNames(),
+ entry.getKey().getGroup(),
+ String.valueOf(entry.getKey().getLastUpdateTimestamp())
+ ),
+ entry.getValue());
+ }
+ mfs.add(topicDLQOffsetF);
+
+ }
@Override
public List<MetricFamilySamples> collect() {
List<MetricFamilySamples> mfs = new ArrayList<MetricFamilySamples>();
- GaugeMetricFamily topicPutNumsGauge = new GaugeMetricFamily("rocketmq_producer_tps", "TopicPutNums", Arrays.asList("cluster","broker","topic"));
- for (Map.Entry<ProducerMetric,Double> entry:topicPutNums.entrySet()) {
- topicPutNumsGauge.addMetric(Arrays.asList(entry.getKey().getClusterName(),entry.getKey().getBrokerName(),entry.getKey().getTopicName()), entry.getValue());
+ collectConsumerMetric(mfs);
+
+ collectTopicOffsetMetric(mfs);
+
+ collectTopicNums(mfs);
+
+ collectGroupNums(mfs);
+
+ collectBrokerNums(mfs);
+
+ collectBrokerRuntimeStats(mfs);
+
+ return mfs;
+ }
+
+ private static List<String> GROUP_PULL_LATENCY_LABEL_NAMES = Arrays.asList(
+ "cluster", "broker", "topic", "group", "queueid"
+ );
+ private static List<String> GROUP_LATENCY_BY_STORETIME_LABEL_NAMES = Arrays.asList(
+ "topic", "group"
+ );
+
+ private static List<String> BROKER_NUMS_LABEL_NAMES = Arrays.asList("cluster", "brokerIP", "brokerHost");
+
+ private static void loadBrokerNums(GaugeMetricFamily family, Map.Entry<BrokerMetric, Double> entry) {
+ family.addMetric(Arrays.asList(
+ entry.getKey().getClusterName(),
+ entry.getKey().getBrokerIP(),
+ entry.getKey().getBrokerHost()),
+ entry.getValue()
+ );
+ }
+
+ private void collectBrokerNums(List<MetricFamilySamples> mfs) {
+ GaugeMetricFamily brokerPutNumsGauge = new GaugeMetricFamily("rocketmq_broker_put_nums", "BrokerPutNums", BROKER_NUMS_LABEL_NAMES);
+ for (Map.Entry<BrokerMetric, Double> entry : brokerPutNums.entrySet()) {
+ loadBrokerNums(brokerPutNumsGauge, entry);
}
- mfs.add(topicPutNumsGauge);
+ mfs.add(brokerPutNumsGauge);
+ GaugeMetricFamily brokerGetNumsGauge = new GaugeMetricFamily("rocketmq_broker_get_nums", "BrokerGetNums", BROKER_NUMS_LABEL_NAMES);
+ for (Map.Entry<BrokerMetric, Double> entry : brokerGetNums.entrySet()) {
+ loadBrokerNums(brokerGetNumsGauge, entry);
+ }
+ mfs.add(brokerGetNumsGauge);
+ }
+
+
+ private static List<String> GROUP_NUMS_LABEL_NAMES = Arrays.asList(
+ "topic", "group"
+ );
- GaugeMetricFamily topicPutSizeGauge = new GaugeMetricFamily("rocketmq_producer_message_size", "TopicPutMessageSize", Arrays.asList("cluster","broker","topic"));
- for (Map.Entry<ProducerMetric, Double> entry: topicPutSize.entrySet()) {
- topicPutSizeGauge.addMetric(Arrays.asList(entry.getKey().getClusterName(),entry.getKey().getBrokerName(),entry.getKey().getTopicName()), entry.getValue());
+ private static <T extends Number> void loadGroupNumsMetric(GaugeMetricFamily family, Map.Entry<ConsumerMetric, T> entry) {
+ family.addMetric(Arrays.asList(
+ entry.getKey().getTopicName(),
+ entry.getKey().getConsumerGroupName()),
+ entry.getValue().doubleValue()
+ );
+ }
+
+ private void collectBrokerRuntimeStatsPutMessageDistributeTime(List<MetricFamilySamples> mfs) {
+ GaugeMetricFamily pmdt0 = new GaugeMetricFamily("rocketmq_brokeruntime_pmdt_0ms", "PutMessageDistributeTimeMap0ms", BROKER_RUNTIME_METRIC_LABEL_NAMES);
+ for (Map.Entry<BrokerRuntimeMetric, Integer> entry : brokerRuntimePutMessageDistributeTimeMap0ms.entrySet()) {
+ loadBrokerRuntimeStatsMetric(pmdt0, entry);
}
- mfs.add(topicPutSizeGauge);
+ mfs.add(pmdt0);
+ GaugeMetricFamily pmdt0to10ms = new GaugeMetricFamily("rocketmq_brokeruntime_pmdt_0to10ms", "PutMessageDistributeTimeMap0to10ms", BROKER_RUNTIME_METRIC_LABEL_NAMES);
+ for (Map.Entry<BrokerRuntimeMetric, Integer> entry : brokerRuntimePutMessageDistributeTimeMap0to10ms.entrySet()) {
+ loadBrokerRuntimeStatsMetric(pmdt0to10ms, entry);
+ }
+ mfs.add(pmdt0to10ms);
- CounterMetricFamily topicOffsetGauge = new CounterMetricFamily("rocketmq_producer_offset", "TopicOffset", Arrays.asList("cluster","broker","topic"));
- for (Map.Entry<ProducerMetric, Double> entry: topicOffset.entrySet()) {
- topicOffsetGauge.addMetric(Arrays.asList(entry.getKey().getClusterName(),entry.getKey().getBrokerName(),entry.getKey().getTopicName()), entry.getValue());
+ GaugeMetricFamily pmdt10to50ms = new GaugeMetricFamily("rocketmq_brokeruntime_pmdt_10to50ms", "PutMessageDistributeTimeMap10to50ms", BROKER_RUNTIME_METRIC_LABEL_NAMES);
+ for (Map.Entry<BrokerRuntimeMetric, Integer> entry : brokerRuntimePutMessageDistributeTimeMap10to50ms.entrySet()) {
+ loadBrokerRuntimeStatsMetric(pmdt10to50ms, entry);
}
- mfs.add(topicOffsetGauge);
+ mfs.add(pmdt10to50ms);
+ GaugeMetricFamily pmdt50to100ms = new GaugeMetricFamily("rocketmq_brokeruntime_pmdt_50to100ms", "PutMessageDistributeTimeMap50to100ms", BROKER_RUNTIME_METRIC_LABEL_NAMES);
+ for (Map.Entry<BrokerRuntimeMetric, Integer> entry : brokerRuntimePutMessageDistributeTimeMap50to100ms.entrySet()) {
+ loadBrokerRuntimeStatsMetric(pmdt50to100ms, entry);
+ }
+ mfs.add(pmdt50to100ms);
- GaugeMetricFamily brokerPutNumsGauge = new GaugeMetricFamily("rocketmq_broker_tps", "BrokerPutNums", Arrays.asList("cluster","broker"));
- for (Map.Entry<BrokerMetric, Double> entry: brokerPutNums.entrySet()) {
- brokerPutNumsGauge.addMetric(Arrays.asList(entry.getKey().getClusterName(),entry.getKey().getBrokerName()), entry.getValue());
+ GaugeMetricFamily pmdt100to200ms = new GaugeMetricFamily("rocketmq_brokeruntime_pmdt_100to200ms", "PutMessageDistributeTimeMap100to200ms", BROKER_RUNTIME_METRIC_LABEL_NAMES);
+ for (Map.Entry<BrokerRuntimeMetric, Integer> entry : brokerRuntimePutMessageDistributeTimeMap100to200ms.entrySet()) {
+ loadBrokerRuntimeStatsMetric(pmdt100to200ms, entry);
}
- mfs.add(brokerPutNumsGauge);
+ mfs.add(pmdt100to200ms);
+ GaugeMetricFamily pmdt200to500ms = new GaugeMetricFamily("rocketmq_brokeruntime_pmdt_200to500ms", "PutMessageDistributeTimeMap200to500ms", BROKER_RUNTIME_METRIC_LABEL_NAMES);
+ for (Map.Entry<BrokerRuntimeMetric, Integer> entry : brokerRuntimePutMessageDistributeTimeMap200to500ms.entrySet()) {
+ loadBrokerRuntimeStatsMetric(pmdt200to500ms, entry);
+ }
+ mfs.add(pmdt200to500ms);
- GaugeMetricFamily brokerGetNumsGauge = new GaugeMetricFamily("rocketmq_broker_qps", "BrokerGetNums", Arrays.asList("cluster","broker"));
- for (Map.Entry<BrokerMetric, Double> entry: brokerGetNums.entrySet()) {
- brokerGetNumsGauge.addMetric(Arrays.asList(entry.getKey().getClusterName(),entry.getKey().getBrokerName()), entry.getValue());
+ GaugeMetricFamily pmdt500to1s = new GaugeMetricFamily("rocketmq_brokeruntime_pmdt_500to1s", "PutMessageDistributeTimeMap500to1s", BROKER_RUNTIME_METRIC_LABEL_NAMES);
+ for (Map.Entry<BrokerRuntimeMetric, Integer> entry : brokerRuntimePutMessageDistributeTimeMap500to1s.entrySet()) {
+ loadBrokerRuntimeStatsMetric(pmdt500to1s, entry);
}
- mfs.add(brokerGetNumsGauge);
+ mfs.add(pmdt500to1s);
+ GaugeMetricFamily pmdt1to2s = new GaugeMetricFamily("rocketmq_brokeruntime_pmdt_1to2s", "PutMessageDistributeTimeMap1to2s", BROKER_RUNTIME_METRIC_LABEL_NAMES);
+ for (Map.Entry<BrokerRuntimeMetric, Integer> entry : brokerRuntimePutMessageDistributeTimeMap1to2s.entrySet()) {
+ loadBrokerRuntimeStatsMetric(pmdt1to2s, entry);
+ }
+ mfs.add(pmdt1to2s);
- GaugeMetricFamily groupGetNumsGauge = new GaugeMetricFamily("rocketmq_consumer_tps", "GroupGetNums", Arrays.asList("cluster","broker","topic","group"));
- for (Map.Entry<ConsumerMetric, Double> entry: groupGetNums.entrySet()) {
- groupGetNumsGauge.addMetric(Arrays.asList(entry.getKey().getClusterName(),entry.getKey().getBrokerName(),entry.getKey().getTopicName(),entry.getKey().getConsumerGroupName()), entry.getValue());
+ GaugeMetricFamily pmdt2to3s = new GaugeMetricFamily("rocketmq_brokeruntime_pmdt_2to3s", "PutMessageDistributeTimeMap2to3s", BROKER_RUNTIME_METRIC_LABEL_NAMES);
+ for (Map.Entry<BrokerRuntimeMetric, Integer> entry : brokerRuntimePutMessageDistributeTimeMap2to3s.entrySet()) {
+ loadBrokerRuntimeStatsMetric(pmdt2to3s, entry);
}
+ mfs.add(pmdt2to3s);
+ GaugeMetricFamily pmdt3to4s = new GaugeMetricFamily("rocketmq_brokeruntime_pmdt_3to4s", "PutMessageDistributeTimeMap3to4s", BROKER_RUNTIME_METRIC_LABEL_NAMES);
+ for (Map.Entry<BrokerRuntimeMetric, Integer> entry : brokerRuntimePutMessageDistributeTimeMap3to4s.entrySet()) {
+ loadBrokerRuntimeStatsMetric(pmdt3to4s, entry);
+ }
+ mfs.add(pmdt3to4s);
+
+ GaugeMetricFamily pmdt4to5s = new GaugeMetricFamily("rocketmq_brokeruntime_pmdt_4to5s", "PutMessageDistributeTimeMap4to5s", BROKER_RUNTIME_METRIC_LABEL_NAMES);
+ for (Map.Entry<BrokerRuntimeMetric, Integer> entry : brokerRuntimePutMessageDistributeTimeMap4to5s.entrySet()) {
+ loadBrokerRuntimeStatsMetric(pmdt4to5s, entry);
+ }
+ mfs.add(pmdt4to5s);
+
+ GaugeMetricFamily pmdt5to10s = new GaugeMetricFamily("rocketmq_brokeruntime_pmdt_5to10s", "PutMessageDistributeTimeMap5to10s", BROKER_RUNTIME_METRIC_LABEL_NAMES);
+ for (Map.Entry<BrokerRuntimeMetric, Integer> entry : brokerRuntimePutMessageDistributeTimeMap5to10s.entrySet()) {
+ loadBrokerRuntimeStatsMetric(pmdt5to10s, entry);
+ }
+ mfs.add(pmdt5to10s);
+
+ GaugeMetricFamily pmdt10stoMore = new GaugeMetricFamily("rocketmq_brokeruntime_pmdt_10stomore", "PutMessageDistributeTimeMap10toMore", BROKER_RUNTIME_METRIC_LABEL_NAMES);
+ for (Map.Entry<BrokerRuntimeMetric, Integer> entry : brokerRuntimePutMessageDistributeTimeMap10toMore.entrySet()) {
+ loadBrokerRuntimeStatsMetric(pmdt10stoMore, entry);
+ }
+ mfs.add(pmdt10stoMore);
+ }
+
+ private void collectGroupNums(List<MetricFamilySamples> mfs) {
+ GaugeMetricFamily groupGetNumsGauge = new GaugeMetricFamily("rocketmq_group_get_nums", "GroupGetNums", GROUP_NUMS_LABEL_NAMES);
+ for (Map.Entry<ConsumerMetric, Double> entry : groupGetNums.entrySet()) {
+ loadGroupNumsMetric(groupGetNumsGauge, entry);
+ }
mfs.add(groupGetNumsGauge);
+ GaugeMetricFamily groupConsumeTPSF = new GaugeMetricFamily("rocketmq_group_consume_tps", "GroupConsumeTPS", GROUP_NUMS_LABEL_NAMES);
+ for (Map.Entry<ConsumerMetric, Double> entry : groupConsumeTPS.entrySet()) {
+ loadGroupNumsMetric(groupConsumeTPSF, entry);
+ }
+ mfs.add(groupConsumeTPSF);
- GaugeMetricFamily groupGetSizeGauge = new GaugeMetricFamily("rocketmq_consumer_message_size", "GroupGetMessageSize", Arrays.asList("cluster","broker","topic","group"));
- for (Map.Entry<ConsumerMetric, Double> entry: groupGetSize.entrySet()) {
- groupGetSizeGauge.addMetric(Arrays.asList(entry.getKey().getClusterName(),entry.getKey().getBrokerName(),entry.getKey().getTopicName(),entry.getKey().getConsumerGroupName()), entry.getValue());
+ GaugeMetricFamily groupBrokerTotalOffsetF = new GaugeMetricFamily("rocketmq_group_broker_total_offset", "GroupBrokerTotalOffset", GROUP_NUMS_LABEL_NAMES);
+ for (Map.Entry<ConsumerMetric, Long> entry : groupBrokerTotalOffset.entrySet()) {
+ loadGroupNumsMetric(groupBrokerTotalOffsetF, entry);
}
- mfs.add(groupGetSizeGauge);
+ mfs.add(groupBrokerTotalOffsetF);
- CounterMetricFamily groupOffsetGauge = new CounterMetricFamily("rocketmq_consumer_offset", "GroupOffset", Arrays.asList("cluster","broker","topic","group"));
- for (Map.Entry<ConsumerMetric, Double> entry: groupOffset.entrySet()) {
- groupOffsetGauge.addMetric(Arrays.asList(entry.getKey().getClusterName(),entry.getKey().getBrokerName(),entry.getKey().getTopicName(),entry.getKey().getConsumerGroupName()), entry.getValue());
+ GaugeMetricFamily groupConsumeTotalOffsetF = new GaugeMetricFamily("rocketmq_group_consume_total_offset", "GroupConsumeTotalOffset", GROUP_NUMS_LABEL_NAMES);
+ for (Map.Entry<ConsumerMetric, Long> entry : groupBrokerTotalOffset.entrySet()) {
+ loadGroupNumsMetric(groupConsumeTotalOffsetF, entry);
}
- mfs.add(groupOffsetGauge);
+ mfs.add(groupConsumeTotalOffsetF);
+ GaugeMetricFamily groupGetSizeGauge = new GaugeMetricFamily("rocketmq_group_get_messagesize", "GroupGetMessageSize", GROUP_NUMS_LABEL_NAMES);
+ for (Map.Entry<ConsumerMetric, Double> entry : groupGetSize.entrySet()) {
+ loadGroupNumsMetric(groupGetSizeGauge, entry);
+ }
+ mfs.add(groupGetSizeGauge);
- GaugeMetricFamily sendBackNumsGauge = new GaugeMetricFamily("rocketmq_send_back_nums", "SendBackNums", Arrays.asList("cluster","broker","topic","group"));
- for (Map.Entry<ConsumerMetric, Double> entry: sendBackNums.entrySet()) {
- sendBackNumsGauge.addMetric(Arrays.asList(entry.getKey().getClusterName(),entry.getKey().getBrokerName(),entry.getKey().getTopicName(),entry.getKey().getConsumerGroupName()), entry.getValue());
+ GaugeMetricFamily sendBackNumsGauge = new GaugeMetricFamily("rocketmq_send_back_nums", "SendBackNums", GROUP_NUMS_LABEL_NAMES);
+ for (Map.Entry<ConsumerMetric, Double> entry : sendBackNums.entrySet()) {
+ loadGroupNumsMetric(sendBackNumsGauge, entry);
}
mfs.add(sendBackNumsGauge);
+ }
+ private void collectTopicNums(List<MetricFamilySamples> mfs) {
+ GaugeMetricFamily topicPutNumsGauge = new GaugeMetricFamily("rocketmq_topic_put_nums", "TopicPutNums", TOPIC_NUMS_LABEL_NAMES);
+ for (Map.Entry<TopicPutNumMetric, Double> entry : topicPutNums.entrySet()) {
+ loadTopicNumsMetric(topicPutNumsGauge, entry);
+ }
+ mfs.add(topicPutNumsGauge);
- GaugeMetricFamily groupGetLatencyGauge = new GaugeMetricFamily("rocketmq_group_get_latency", "GroupGetLatency", Arrays.asList("cluster","broker","topic","group","queueid"));
- for (Map.Entry<ConsumerQueueMetric, Double> entry: groupGetLatency.entrySet()) {
- groupGetLatencyGauge.addMetric(Arrays.asList(entry.getKey().getClusterName(),entry.getKey().getBrokerName(),entry.getKey().getTopicName(),entry.getKey().getConsumerGroupName(),entry.getKey().getQueueId()), entry.getValue());
+ GaugeMetricFamily topicPutSizeGauge = new GaugeMetricFamily("rocketmq_topic_put_messagesize", "TopicPutMessageSize", TOPIC_NUMS_LABEL_NAMES);
+ for (Map.Entry<TopicPutNumMetric, Double> entry : topicPutSize.entrySet()) {
+ loadTopicNumsMetric(topicPutSizeGauge, entry);
}
- mfs.add(groupGetLatencyGauge);
+ mfs.add(topicPutSizeGauge);
+ }
- GaugeMetricFamily groupGetLatencyByStoretimeGauge = new GaugeMetricFamily("rocketmq_group_get_latency_by_storetime", "GroupGetLatencyByStoreTime", Arrays.asList("cluster","broker","topic","group"));
- for (Map.Entry<ConsumerMetric, Double> entry: groupGetLatencyByStoreTime.entrySet()) {
- groupGetLatencyByStoretimeGauge.addMetric(Arrays.asList(entry.getKey().getClusterName(),entry.getKey().getBrokerName(),entry.getKey().getTopicName(),entry.getKey().getConsumerGroupName()), entry.getValue());
+ private static List<String> TOPIC_NUMS_LABEL_NAMES = Arrays.asList("cluster", "brokers", "topic");
+
+ private void loadTopicNumsMetric(GaugeMetricFamily family, Map.Entry<TopicPutNumMetric, Double> entry) {
+ family.addMetric(
+ Arrays.asList(
+ entry.getKey().getClusterName(),
+ entry.getKey().getBrokerNames(),
+ entry.getKey().getTopicName()
+ ),
+ entry.getValue()
+ );
+ }
+
+ public void addTopicOffsetMetric(String clusterName, String brokerNames, String topic, long lastUpdateTimestamp, double value) {
+ if (topic.startsWith(MixAll.RETRY_GROUP_TOPIC_PREFIX)) {
+ topicRetryOffset.put(new ProducerMetric(clusterName, brokerNames, topic, lastUpdateTimestamp), value);
+ } else if (topic.startsWith(MixAll.DLQ_GROUP_TOPIC_PREFIX)) {
+ topicDLQOffset.put(new DLQTopicOffsetMetric(clusterName, brokerNames, topic.replace(MixAll.DLQ_GROUP_TOPIC_PREFIX, ""), lastUpdateTimestamp), value);
+ } else {
+ topicOffset.put(new ProducerMetric(clusterName, brokerNames, topic, lastUpdateTimestamp), value);
}
- mfs.add(groupGetLatencyByStoretimeGauge);
+ }
- return mfs;
+ public void addGroupCountMetric(String group, String caddr, String localaddr, int count) {
+ this.consumerCounts.put(new ConsumerCountMetric(group, caddr, localaddr), count);
}
- public void AddTopicPutNumsMetric(String clusterName, String brokerName, String topic, double value)
- {
- topicPutNums.put(new ProducerMetric(clusterName,brokerName,topic),value);
+
+ public void addGroupDiffMetric(String countOfOnlineConsumers, String group, String topic, long value) {
+ if (topic.startsWith(MixAll.RETRY_GROUP_TOPIC_PREFIX)) {
+ this.consumerRetryDiff.put(new ConsumerTopicDiffMetric(group, topic, countOfOnlineConsumers), value);
+ } else if (topic.startsWith(MixAll.DLQ_GROUP_TOPIC_PREFIX)) {
+ this.consumerDLQDiff.put(new ConsumerTopicDiffMetric(group, topic, countOfOnlineConsumers), value);
+ } else {
+ this.consumerDiff.put(new ConsumerTopicDiffMetric(group, topic, countOfOnlineConsumers), value);
+ }
+ }
+
+ public void addTopicPutNumsMetric(String cluster, String brokerNames, String brokerIP, String brokerHost,
+ String topic, double value) {
+ topicPutNums.put(new TopicPutNumMetric(cluster, brokerNames, brokerIP, brokerHost, topic), value);
+ }
+
+ public void addTopicPutSizeMetric(String cluster, String brokerName, String brokerIP, String brokerHost,
+ String topic, double value) {
+ topicPutSize.put(new TopicPutNumMetric(cluster, brokerName, brokerIP, brokerHost, topic), value);
+ }
+
+ public void addGroupBrokerTotalOffsetMetric(String topic, String group, long value) {
+ groupBrokerTotalOffset.put(new ConsumerMetric(topic, group), value);
+ }
+
+ public void addGroupConsumerTotalOffsetMetric(String topic, String group, long value) {
+ groupConsumeTotalOffset.put(new ConsumerMetric(topic, group), value);
}
- public void AddTopicPutSizeMetric(String clusterName, String brokerName, String topic, double value)
- {
- topicPutSize.put(new ProducerMetric(clusterName,brokerName,topic),value);
+ public void addGroupConsumeTPSMetric(String topic, String group, double value) {
+ groupConsumeTPS.put(new ConsumerMetric(topic, group), value);
}
- public void AddTopicOffsetMetric(String clusterName, String brokerName, String topic, double value)
- {
- topicOffset.put(new ProducerMetric(clusterName,brokerName,topic),value);
+ public void addGroupGetNumsMetric(String topic, String group, double value) {
+ groupGetNums.put(new ConsumerMetric(topic, group), value);
}
- public void AddBrokerPutNumsMetric(String clusterName, String brokerName, double value)
- {
- brokerPutNums.put(new BrokerMetric(clusterName,brokerName),value);
+ public void addGroupGetSizeMetric(String topic, String group, double value) {
+ groupGetSize.put(new ConsumerMetric(topic, group), value);
}
- public void AddBrokerGetNumsMetric(String clusterName, String brokerName, double value)
- {
- brokerGetNums.put(new BrokerMetric(clusterName,brokerName),value);
+ public void addSendBackNumsMetric(String topic, String group, double value) {
+ sendBackNums.put(new ConsumerMetric(topic, group), value);
}
- public void AddGroupGetNumsMetric(String clusterName, String brokerName, String topic, String group, double value)
- {
- groupGetNums.put(new ConsumerMetric(clusterName,brokerName,topic,group),value);
+ public void addBrokerPutNumsMetric(String clusterName, String brokerIP, String brokerHost, double value) {
+ brokerPutNums.put(new BrokerMetric(clusterName, brokerIP, brokerHost), value);
}
- public void AddGroupGetSizeMetric(String clusterName, String brokerName, String topic, String group, double value)
- {
- groupGetSize.put(new ConsumerMetric(clusterName,brokerName,topic,group),value);
+ public void addBrokerGetNumsMetric(String clusterName, String brokerIP, String brokerHost, double value) {
+ brokerGetNums.put(new BrokerMetric(clusterName, brokerIP, brokerHost), value);
}
- public void AddGroupOffsetMetric(String clusterName, String brokerName, String topic, String group, double value)
- {
- groupOffset.put(new ConsumerMetric(clusterName,brokerName,topic,group),value);
+ public void addBrokerRuntimeStatsMetric(BrokerRuntimeStats stats, String clusterName, String brokerAddress, String brokerHost) {
+ addBrokerRuntimePutMessageDistributeTimeMap(
+ clusterName, brokerAddress, brokerHost,
+ stats.getBrokerVersionDesc(),
+ stats.getBootTimestamp(),
+ stats.getBrokerVersion(), stats);
+ addCommitLogDirCapacity(clusterName, brokerAddress, brokerHost, stats);
+ addAllKindOfTps(clusterName, brokerAddress, brokerHost, stats);
+
+ brokerRuntimeMsgPutTotalTodayNow.put(new BrokerRuntimeMetric(
+ clusterName, brokerAddress, brokerHost,
+ stats.getBrokerVersionDesc(),
+ stats.getBootTimestamp(),
+ stats.getBrokerVersion()), stats.getMsgPutTotalTodayNow());
+
+ brokerRuntimeMsgGetTotalTodayNow.put(new BrokerRuntimeMetric(
+ clusterName, brokerAddress, brokerHost,
+ stats.getBrokerVersionDesc(),
+ stats.getBootTimestamp(),
+ stats.getBrokerVersion()), stats.getMsgGetTotalTodayNow());
+
+ brokerRuntimeMsgPutTotalTodayMorning.put(new BrokerRuntimeMetric(
+ clusterName, brokerAddress, brokerHost,
+ stats.getBrokerVersionDesc(),
+ stats.getBootTimestamp(),
+ stats.getBrokerVersion()), stats.getMsgPutTotalTodayMorning());
+ brokerRuntimeMsgGetTotalTodayMorning.put(new BrokerRuntimeMetric(
+ clusterName, brokerAddress, brokerHost,
+ stats.getBrokerVersionDesc(),
+ stats.getBootTimestamp(),
+ stats.getBrokerVersion()), stats.getMsgGetTotalTodayMorning());
+ brokerRuntimeMsgPutTotalYesterdayMorning.put(new BrokerRuntimeMetric(
+ clusterName, brokerAddress, brokerHost,
+ stats.getBrokerVersionDesc(),
+ stats.getBootTimestamp(),
+ stats.getBrokerVersion()), stats.getMsgPutTotalYesterdayMorning());
+ brokerRuntimeMsgGetTotalYesterdayMorning.put(new BrokerRuntimeMetric(
+ clusterName, brokerAddress, brokerHost,
+ stats.getBrokerVersionDesc(),
+ stats.getBootTimestamp(),
+ stats.getBrokerVersion()), stats.getMsgGetTotalYesterdayMorning());
+ brokerRuntimeSendThreadPoolQueueHeadWaitTimeMills.put(new BrokerRuntimeMetric(
+ clusterName, brokerAddress, brokerHost,
+ stats.getBrokerVersionDesc(),
+ stats.getBootTimestamp(),
+ stats.getBrokerVersion()), stats.getSendThreadPoolQueueHeadWaitTimeMills());
+ brokerRuntimeQueryThreadPoolQueueHeadWaitTimeMills.put(new BrokerRuntimeMetric(
+ clusterName, brokerAddress, brokerHost,
+ stats.getBrokerVersionDesc(),
+ stats.getBootTimestamp(),
+ stats.getBrokerVersion()), stats.getQueryThreadPoolQueueHeadWaitTimeMills());
+ brokerRuntimePullThreadPoolQueueHeadWaitTimeMills.put(new BrokerRuntimeMetric(
+ clusterName, brokerAddress, brokerHost,
+ stats.getBrokerVersionDesc(),
+ stats.getBootTimestamp(),
+ stats.getBrokerVersion()), stats.getPullThreadPoolQueueHeadWaitTimeMills());
+ brokerRuntimeQueryThreadPoolQueueSize.put(new BrokerRuntimeMetric(
+ clusterName, brokerAddress, brokerHost,
+ stats.getBrokerVersionDesc(),
+ stats.getBootTimestamp(),
+ stats.getBrokerVersion()), stats.getQueryThreadPoolQueueSize());
+ brokerRuntimePullThreadPoolQueueSize.put(new BrokerRuntimeMetric(
+ clusterName, brokerAddress, brokerHost,
+ stats.getBrokerVersionDesc(),
+ stats.getBootTimestamp(),
+ stats.getBrokerVersion()), stats.getPullThreadPoolQueueSize());
+ brokerRuntimeSendThreadPoolQueueCapacity.put(new BrokerRuntimeMetric(
+ clusterName, brokerAddress, brokerHost,
+ stats.getBrokerVersionDesc(),
+ stats.getBootTimestamp(),
+ stats.getBrokerVersion()), stats.getSendThreadPoolQueueCapacity());
+ brokerRuntimePullThreadPoolQueueCapacity.put(new BrokerRuntimeMetric(
+ clusterName, brokerAddress, brokerHost,
+ stats.getBrokerVersionDesc(),
+ stats.getBootTimestamp(),
+ stats.getBrokerVersion()), stats.getPullThreadPoolQueueCapacity());
+
+ brokerRuntimeRemainHowManyDataToFlush.put(new BrokerRuntimeMetric(
+ clusterName, brokerAddress, brokerHost,
+ stats.getBrokerVersionDesc(),
+ stats.getBootTimestamp(),
+ stats.getBrokerVersion()), stats.getRemainHowManyDataToFlush());
+ brokerRuntimeCommitLogMinOffset.put(new BrokerRuntimeMetric(
+ clusterName, brokerAddress, brokerHost,
+ stats.getBrokerVersionDesc(),
+ stats.getBootTimestamp(),
+ stats.getBrokerVersion()), stats.getCommitLogMinOffset());
+ brokerRuntimeCommitLogMaxOffset.put(new BrokerRuntimeMetric(
+ clusterName, brokerAddress, brokerHost,
+ stats.getBrokerVersionDesc(),
+ stats.getBootTimestamp(),
+ stats.getBrokerVersion()), stats.getCommitLogMaxOffset());
+
+
+ brokerRuntimeDispatchMaxBuffer.put(new BrokerRuntimeMetric(
+ clusterName, brokerAddress, brokerHost,
+ stats.getBrokerVersionDesc(),
+ stats.getBootTimestamp(),
+ stats.getBrokerVersion()), stats.getDispatchMaxBuffer());
+ brokerRuntimeConsumeQueueDiskRatio.put(new BrokerRuntimeMetric(
+ clusterName, brokerAddress, brokerHost,
+ stats.getBrokerVersionDesc(),
+ stats.getBootTimestamp(),
+ stats.getBrokerVersion()), stats.getConsumeQueueDiskRatio());
+ brokerRuntimeCommitLogDiskRatio.put(new BrokerRuntimeMetric(
+ clusterName, brokerAddress, brokerHost,
+ stats.getBrokerVersionDesc(),
+ stats.getBootTimestamp(),
+ stats.getBrokerVersion()), stats.getCommitLogDiskRatio());
+ brokerRuntimePageCacheLockTimeMills.put(new BrokerRuntimeMetric(
+ clusterName, brokerAddress, brokerHost,
+ stats.getBrokerVersionDesc(),
+ stats.getBootTimestamp(),
+ stats.getBrokerVersion()), stats.getPageCacheLockTimeMills());
+ brokerRuntimeGetMessageEntireTimeMax.put(new BrokerRuntimeMetric(
+ clusterName, brokerAddress, brokerHost,
+ stats.getBrokerVersionDesc(),
+ stats.getBootTimestamp(),
+ stats.getBrokerVersion()), stats.getGetMessageEntireTimeMax());
+ brokerRuntimePutMessageTimesTotal.put(new BrokerRuntimeMetric(
+ clusterName, brokerAddress, brokerHost,
+ stats.getBrokerVersionDesc(),
+ stats.getBootTimestamp(),
+ stats.getBrokerVersion()), stats.getPutMessageTimesTotal());
+ brokerRuntimeSendThreadPoolQueueSize.put(new BrokerRuntimeMetric(
+ clusterName, brokerAddress, brokerHost,
+ stats.getBrokerVersionDesc(),
+ stats.getBootTimestamp(),
+ stats.getBrokerVersion()), stats.getSendThreadPoolQueueSize());
+ brokerRuntimeStartAcceptSendRequestTimeStamp.put(new BrokerRuntimeMetric(
+ clusterName, brokerAddress, brokerHost,
+ stats.getBrokerVersionDesc(),
+ stats.getBootTimestamp(),
+ stats.getBrokerVersion()), stats.getStartAcceptSendRequestTimeStamp());
+ brokerRuntimePutMessageEntireTimeMax.put(new BrokerRuntimeMetric(
+ clusterName, brokerAddress, brokerHost,
+ stats.getBrokerVersionDesc(),
+ stats.getBootTimestamp(),
+ stats.getBrokerVersion()), stats.getPutMessageEntireTimeMax());
+ brokerRuntimeEarliestMessageTimeStamp.put(new BrokerRuntimeMetric(
+ clusterName, brokerAddress, brokerHost,
+ stats.getBrokerVersionDesc(),
+ stats.getBootTimestamp(),
+ stats.getBrokerVersion()), stats.getEarliestMessageTimeStamp());
+ brokerRuntimeRemainTransientStoreBufferNumbs.put(new BrokerRuntimeMetric(
+ clusterName, brokerAddress, brokerHost,
+ stats.getBrokerVersionDesc(),
+ stats.getBootTimestamp(),
+ stats.getBrokerVersion()), stats.getRemainTransientStoreBufferNumbs());
+ brokerRuntimeQueryThreadPoolQueueCapacity.put(new BrokerRuntimeMetric(
+ clusterName, brokerAddress, brokerHost,
+ stats.getBrokerVersionDesc(),
+ stats.getBootTimestamp(),
+ stats.getBrokerVersion()), stats.getQueryThreadPoolQueueCapacity());
+ brokerRuntimePutMessageAverageSize.put(new BrokerRuntimeMetric(
+ clusterName, brokerAddress, brokerHost,
+ stats.getBrokerVersionDesc(),
+ stats.getBootTimestamp(),
+ stats.getBrokerVersion()), stats.getPutMessageAverageSize());
+ brokerRuntimePutMessageSizeTotal.put(new BrokerRuntimeMetric(
+ clusterName, brokerAddress, brokerHost,
+ stats.getBrokerVersionDesc(),
+ stats.getBootTimestamp(),
+ stats.getBrokerVersion()), stats.getQueryThreadPoolQueueCapacity());
+ brokerRuntimeDispatchBehindBytes.put(new BrokerRuntimeMetric(
+ clusterName, brokerAddress, brokerHost,
+ stats.getBrokerVersionDesc(),
+ stats.getBootTimestamp(),
+ stats.getBrokerVersion()), stats.getDispatchBehindBytes());
}
+ private void addAllKindOfTps(String brokerAddress, String clusterName, String brokerHost, BrokerRuntimeStats stats) {
+ brokerRuntimePutTps10.put(new BrokerRuntimeMetric(
+ clusterName,
+ brokerAddress, brokerHost,
+ stats.getBrokerVersionDesc(),
+ stats.getBootTimestamp(),
+ stats.getBrokerVersion()), stats.getPutTps().getTen());
+ brokerRuntimePutTps60.put(new BrokerRuntimeMetric(
+ clusterName,
+ brokerAddress, brokerHost,
+ stats.getBrokerVersionDesc(),
+ stats.getBootTimestamp(),
+ stats.getBrokerVersion()), stats.getPutTps().getSixty());
+ brokerRuntimePutTps600.put(new BrokerRuntimeMetric(
+ clusterName,
+ brokerAddress, brokerHost,
+ stats.getBrokerVersionDesc(),
+ stats.getBootTimestamp(),
+ stats.getBrokerVersion()), stats.getPutTps().getSixHundred());
+ brokerRuntimeGetMissTps10.put(new BrokerRuntimeMetric(
+ clusterName,
+ brokerAddress, brokerHost,
+ stats.getBrokerVersionDesc(),
+ stats.getBootTimestamp(),
+ stats.getBrokerVersion()), stats.getGetMissTps().getTen());
+ brokerRuntimeGetMissTps60.put(new BrokerRuntimeMetric(
+ clusterName,
+ brokerAddress, brokerHost,
+ stats.getBrokerVersionDesc(),
+ stats.getBootTimestamp(),
+ stats.getBrokerVersion()), stats.getGetMissTps().getSixty());
+ brokerRuntimeGetMissTps600.put(new BrokerRuntimeMetric(
+ clusterName,
+ brokerAddress, brokerHost,
+ stats.getBrokerVersionDesc(),
+ stats.getBootTimestamp(),
+ stats.getBrokerVersion()), stats.getGetMissTps().getSixHundred());
+ brokerRuntimeGetTransferedTps10.put(new BrokerRuntimeMetric(
+ clusterName,
+ brokerAddress, brokerHost,
+ stats.getBrokerVersionDesc(),
+ stats.getBootTimestamp(),
+ stats.getBrokerVersion()), stats.getGetTransferedTps().getTen());
+ brokerRuntimeGetTransferedTps60.put(new BrokerRuntimeMetric(
+ clusterName,
+ brokerAddress, brokerHost,
+ stats.getBrokerVersionDesc(),
+ stats.getBootTimestamp(),
+ stats.getBrokerVersion()), stats.getGetTransferedTps().getSixty());
+ brokerRuntimeGetTransferedTps600.put(new BrokerRuntimeMetric(
+ clusterName,
+ brokerAddress, brokerHost,
+ stats.getBrokerVersionDesc(),
+ stats.getBootTimestamp(),
+ stats.getBrokerVersion()), stats.getGetTransferedTps().getSixHundred());
+ brokerRuntimeGetTotalTps10.put(new BrokerRuntimeMetric(
+ clusterName,
+ brokerAddress, brokerHost,
+ stats.getBrokerVersionDesc(),
+ stats.getBootTimestamp(),
+ stats.getBrokerVersion()), stats.getGetTotalTps().getTen());
+ brokerRuntimeGetTotalTps60.put(new BrokerRuntimeMetric(
+ clusterName,
+ brokerAddress, brokerHost,
+ stats.getBrokerVersionDesc(),
+ stats.getBootTimestamp(),
+ stats.getBrokerVersion()), stats.getGetTotalTps().getSixty());
+ brokerRuntimeGetTotalTps600.put(new BrokerRuntimeMetric(
+ clusterName,
+ brokerAddress, brokerHost,
+ stats.getBrokerVersionDesc(),
+ stats.getBootTimestamp(),
+ stats.getBrokerVersion()), stats.getGetTotalTps().getSixHundred());
+ brokerRuntimeGetFoundTps10.put(new BrokerRuntimeMetric(
+ clusterName,
+ brokerAddress, brokerHost,
+ stats.getBrokerVersionDesc(),
+ stats.getBootTimestamp(),
+ stats.getBrokerVersion()), stats.getGetFoundTps().getTen());
+ brokerRuntimeGetFoundTps60.put(new BrokerRuntimeMetric(
+ clusterName,
+ brokerAddress, brokerHost,
+ stats.getBrokerVersionDesc(),
+ stats.getBootTimestamp(),
+ stats.getBrokerVersion()), stats.getGetFoundTps().getSixty());
+ brokerRuntimeGetFoundTps600.put(new BrokerRuntimeMetric(
+ clusterName,
+ brokerAddress, brokerHost,
+ stats.getBrokerVersionDesc(),
+ stats.getBootTimestamp(),
+ stats.getBrokerVersion()), stats.getGetFoundTps().getSixHundred());
+ brokerRuntimeGetFoundTps600.put(new BrokerRuntimeMetric(
+ clusterName,
+ brokerAddress, brokerHost,
+ stats.getBrokerVersionDesc(),
+ stats.getBootTimestamp(),
+ stats.getBrokerVersion()), stats.getGetFoundTps().getSixHundred());
+ }
- public void AddsendBackNumsMetric(String clusterName, String brokerName, String topic, String group, double value)
- {
- sendBackNums.put(new ConsumerMetric(clusterName,brokerName,topic,group),value);
+ private void addCommitLogDirCapacity(String clusterName, String brokerAddress, String brokerHost, BrokerRuntimeStats stats) {
+ brokerRuntimeCommitLogDirCapacityTotal.put(new BrokerRuntimeMetric(
+ clusterName,
+ brokerAddress, brokerHost,
+ stats.getBrokerVersionDesc(),
+ stats.getBootTimestamp(),
+ stats.getBrokerVersion()), stats.getCommitLogDirCapacityTotal());
+ brokerRuntimeCommitLogDirCapacityFree.put(new BrokerRuntimeMetric(
+ clusterName,
+ brokerAddress, brokerHost,
+ stats.getBrokerVersionDesc(),
+ stats.getBootTimestamp(),
+ stats.getBrokerVersion()), stats.getCommitLogDirCapacityFree());
}
- public void AddGroupGetLatencyMetric(String clusterName, String brokerName, String topic, String group, String queueId,double value) {
+ private void addBrokerRuntimePutMessageDistributeTimeMap(
+ String clusterName, String brokerAddress, String brokerHost,
+ String brokerDes, long bootTimestamp, int brokerVersion,
+ BrokerRuntimeStats stats) {
+ brokerRuntimePutMessageDistributeTimeMap0ms.put(new BrokerRuntimeMetric(
+ clusterName,
+ brokerAddress, brokerHost,
+ brokerDes,
+ bootTimestamp,
+ brokerVersion), stats.getPutMessageDistributeTimeMap().get("<=0ms"));
+ brokerRuntimePutMessageDistributeTimeMap0to10ms.put(new BrokerRuntimeMetric(
+ clusterName,
+ brokerAddress, brokerHost,
+ brokerDes,
+ bootTimestamp,
+ brokerVersion), stats.getPutMessageDistributeTimeMap().get("0~10ms"));
+ brokerRuntimePutMessageDistributeTimeMap10to50ms.put(new BrokerRuntimeMetric(
+ clusterName,
+ brokerAddress, brokerHost,
+ brokerDes,
+ bootTimestamp,
+ brokerVersion), stats.getPutMessageDistributeTimeMap().get("10~50ms"));
+ brokerRuntimePutMessageDistributeTimeMap50to100ms.put(new BrokerRuntimeMetric(
+ clusterName,
+ brokerAddress, brokerHost,
+ brokerDes,
+ bootTimestamp,
+ brokerVersion), stats.getPutMessageDistributeTimeMap().get("50~100ms"));
+ brokerRuntimePutMessageDistributeTimeMap100to200ms.put(new BrokerRuntimeMetric(
+ clusterName,
+ brokerAddress, brokerHost,
+ brokerDes,
+ bootTimestamp,
+ brokerVersion), stats.getPutMessageDistributeTimeMap().get("100~200ms"));
+ brokerRuntimePutMessageDistributeTimeMap200to500ms.put(new BrokerRuntimeMetric(
+ clusterName,
+ brokerAddress, brokerHost,
+ brokerDes,
+ bootTimestamp,
+ brokerVersion), stats.getPutMessageDistributeTimeMap().get("200~500ms"));
+ brokerRuntimePutMessageDistributeTimeMap500to1s.put(new BrokerRuntimeMetric(
+ clusterName,
+ brokerAddress, brokerHost,
+ brokerDes,
+ bootTimestamp,
+ brokerVersion), stats.getPutMessageDistributeTimeMap().get("500ms~1s"));
+ brokerRuntimePutMessageDistributeTimeMap1to2s.put(new BrokerRuntimeMetric(
+ clusterName,
+ brokerAddress, brokerHost,
+ brokerDes,
+ bootTimestamp,
+ brokerVersion), stats.getPutMessageDistributeTimeMap().get("1~2s"));
+ brokerRuntimePutMessageDistributeTimeMap2to3s.put(new BrokerRuntimeMetric(
+ clusterName,
+ brokerAddress, brokerHost,
+ brokerDes,
+ bootTimestamp,
+ brokerVersion), stats.getPutMessageDistributeTimeMap().get("2~3s"));
+ brokerRuntimePutMessageDistributeTimeMap3to4s.put(new BrokerRuntimeMetric(
+ clusterName,
+ brokerAddress, brokerHost,
+ brokerDes,
+ bootTimestamp,
+ brokerVersion), stats.getPutMessageDistributeTimeMap().get("3~4s"));
+ brokerRuntimePutMessageDistributeTimeMap4to5s.put(new BrokerRuntimeMetric(
+ clusterName,
+ brokerAddress, brokerHost,
+ brokerDes,
+ bootTimestamp,
+ brokerVersion), stats.getPutMessageDistributeTimeMap().get("4~5s"));
+ brokerRuntimePutMessageDistributeTimeMap5to10s.put(new BrokerRuntimeMetric(
+ clusterName,
+ brokerAddress, brokerHost,
+ brokerDes,
+ bootTimestamp,
+ brokerVersion), stats.getPutMessageDistributeTimeMap().get("5~10s"));
+ brokerRuntimePutMessageDistributeTimeMap10toMore.put(new BrokerRuntimeMetric(
+ clusterName,
+ brokerAddress, brokerHost,
+ brokerDes,
+ bootTimestamp,
+ brokerVersion), stats.getPutMessageDistributeTimeMap().get("10s~"));
+ }
- groupGetLatency.put(new ConsumerQueueMetric(clusterName,brokerName,topic,group,queueId),value);
+ private static <T extends Number> void loadBrokerRuntimeStatsMetric(GaugeMetricFamily family, Map.Entry<BrokerRuntimeMetric, T> entry) {
+ family.addMetric(Arrays.asList(
+ entry.getKey().getClusterName(),
+ entry.getKey().getBrokerAddress(),
+ entry.getKey().getBrokerHost(),
+ entry.getKey().getBrokerDes(),
+ String.valueOf(entry.getKey().getBootTimestamp()),
+ String.valueOf(entry.getKey().getBrokerVersion())
+ ), entry.getValue().doubleValue());
}
- public void AddGroupGetLatencyByStoreTimeMetric(String clusterName, String brokerName, String topic, String group,double value) {
+ private static List<String> BROKER_RUNTIME_METRIC_LABEL_NAMES = Arrays.asList("cluster", "brokerIP", "brokerHost", "des", "boottime", "broker_version");
+
+ private void collectBrokerRuntimeStats(List<MetricFamilySamples> mfs) {
+ collectBrokerRuntimeStatsPutMessageDistributeTime(mfs);
+
+ GaugeMetricFamily brokerRuntimeMsgPutTotalTodayNowF = new GaugeMetricFamily("rocketmq_brokeruntime_msg_put_total_today_now", "brokerRuntimeMsgPutTotalTodayNow", BROKER_RUNTIME_METRIC_LABEL_NAMES);
+ for (Map.Entry<BrokerRuntimeMetric, Long> entry : brokerRuntimeMsgPutTotalTodayNow.entrySet()) {
+ loadBrokerRuntimeStatsMetric(brokerRuntimeMsgPutTotalTodayNowF, entry);
+ }
+ mfs.add(brokerRuntimeMsgPutTotalTodayNowF);
+
+ GaugeMetricFamily brokerRuntimeMsgGetTotalTodayNowF = new GaugeMetricFamily("rocketmq_brokeruntime_msg_gettotal_today_now", "brokerRuntimeMsgGetTotalTodayNow", BROKER_RUNTIME_METRIC_LABEL_NAMES);
+ for (Map.Entry<BrokerRuntimeMetric, Long> entry : brokerRuntimeMsgGetTotalTodayNow.entrySet()) {
+ loadBrokerRuntimeStatsMetric(brokerRuntimeMsgGetTotalTodayNowF, entry);
+ }
+ mfs.add(brokerRuntimeMsgGetTotalTodayNowF);
+
+ GaugeMetricFamily brokerRuntimeDispatchBehindBytesF = new GaugeMetricFamily("rocketmq_brokeruntime_dispatch_behind_bytes", "brokerRuntimeDispatchBehindBytes", BROKER_RUNTIME_METRIC_LABEL_NAMES);
+ for (Map.Entry<BrokerRuntimeMetric, Long> entry : brokerRuntimeDispatchBehindBytes.entrySet()) {
+ loadBrokerRuntimeStatsMetric(brokerRuntimeDispatchBehindBytesF, entry);
+ }
+ mfs.add(brokerRuntimeDispatchBehindBytesF);
+
+ GaugeMetricFamily brokerRuntimePutMessageSizeTotalF = new GaugeMetricFamily("rocketmq_brokeruntime_put_message_size_total", "brokerRuntimePutMessageSizeTotal", BROKER_RUNTIME_METRIC_LABEL_NAMES);
+ for (Map.Entry<BrokerRuntimeMetric, Long> entry : brokerRuntimePutMessageSizeTotal.entrySet()) {
+ loadBrokerRuntimeStatsMetric(brokerRuntimePutMessageSizeTotalF, entry);
+ }
+ mfs.add(brokerRuntimePutMessageSizeTotalF);
+
+ GaugeMetricFamily brokerRuntimePutMessageAverageSizeF = new GaugeMetricFamily("rocketmq_brokeruntime_put_message_average_size", "brokerRuntimePutMessageAverageSize", BROKER_RUNTIME_METRIC_LABEL_NAMES);
+ for (Map.Entry<BrokerRuntimeMetric, Double> entry : brokerRuntimePutMessageAverageSize.entrySet()) {
+ loadBrokerRuntimeStatsMetric(brokerRuntimePutMessageAverageSizeF, entry);
+ }
+ mfs.add(brokerRuntimePutMessageAverageSizeF);
+
+ GaugeMetricFamily brokerRuntimeQueryThreadPoolQueueCapacityF = new GaugeMetricFamily("rocketmq_brokeruntime_query_threadpool_queue_capacity", "brokerRuntimeQueryThreadPoolQueueCapacity", BROKER_RUNTIME_METRIC_LABEL_NAMES);
+ for (Map.Entry<BrokerRuntimeMetric, Long> entry : brokerRuntimeQueryThreadPoolQueueCapacity.entrySet()) {
+ loadBrokerRuntimeStatsMetric(brokerRuntimeQueryThreadPoolQueueCapacityF, entry);
+ }
+ mfs.add(brokerRuntimeQueryThreadPoolQueueCapacityF);
+
+ GaugeMetricFamily brokerRuntimeRemainTransientStoreBufferNumbsF = new GaugeMetricFamily("rocketmq_brokeruntime_remain_transientstore_buffer_numbs", "brokerRuntimeRemainTransientStoreBufferNumbs", BROKER_RUNTIME_METRIC_LABEL_NAMES);
+ for (Map.Entry<BrokerRuntimeMetric, Long> entry : brokerRuntimeRemainTransientStoreBufferNumbs.entrySet()) {
+ loadBrokerRuntimeStatsMetric(brokerRuntimeRemainTransientStoreBufferNumbsF, entry);
+ }
+ mfs.add(brokerRuntimeRemainTransientStoreBufferNumbsF);
+
+ GaugeMetricFamily brokerRuntimeEarliestMessageTimeStampF = new GaugeMetricFamily("rocketmq_brokeruntime_earliest_message_timestamp", "brokerRuntimeEarliestMessageTimeStamp", BROKER_RUNTIME_METRIC_LABEL_NAMES);
+ for (Map.Entry<BrokerRuntimeMetric, Long> entry : brokerRuntimeEarliestMessageTimeStamp.entrySet()) {
+ loadBrokerRuntimeStatsMetric(brokerRuntimeEarliestMessageTimeStampF, entry);
+ }
+ mfs.add(brokerRuntimeEarliestMessageTimeStampF);
+
+ GaugeMetricFamily brokerRuntimePutMessageEntireTimeMaxF = new GaugeMetricFamily("rocketmq_brokeruntime_putmessage_entire_time_max", "brokerRuntimePutMessageEntireTimeMax", BROKER_RUNTIME_METRIC_LABEL_NAMES);
+ for (Map.Entry<BrokerRuntimeMetric, Long> entry : brokerRuntimePutMessageEntireTimeMax.entrySet()) {
+ loadBrokerRuntimeStatsMetric(brokerRuntimePutMessageEntireTimeMaxF, entry);
+ }
+ mfs.add(brokerRuntimePutMessageEntireTimeMaxF);
+
+ GaugeMetricFamily brokerRuntimeStartAcceptSendRequestTimeStampF = new GaugeMetricFamily("rocketmq_brokeruntime_start_accept_sendrequest_time", "brokerRuntimeStartAcceptSendRequestTimeStamp", BROKER_RUNTIME_METRIC_LABEL_NAMES);
+ for (Map.Entry<BrokerRuntimeMetric, Long> entry : brokerRuntimeStartAcceptSendRequestTimeStamp.entrySet()) {
+ loadBrokerRuntimeStatsMetric(brokerRuntimeStartAcceptSendRequestTimeStampF, entry);
+ }
+ mfs.add(brokerRuntimeStartAcceptSendRequestTimeStampF);
- groupGetLatencyByStoreTime.put(new ConsumerMetric(clusterName,brokerName,topic,group),value);
+ GaugeMetricFamily brokerRuntimeSendThreadPoolQueueSizeF = new GaugeMetricFamily("rocketmq_brokeruntime_send_threadpool_queue_size", "brokerRuntimeSendThreadPoolQueueSize", BROKER_RUNTIME_METRIC_LABEL_NAMES);
+ for (Map.Entry<BrokerRuntimeMetric, Long> entry : brokerRuntimeSendThreadPoolQueueSize.entrySet()) {
+ loadBrokerRuntimeStatsMetric(brokerRuntimeSendThreadPoolQueueSizeF, entry);
+ }
+ mfs.add(brokerRuntimeSendThreadPoolQueueSizeF);
+
+ GaugeMetricFamily brokerRuntimePutMessageTimesTotalF = new GaugeMetricFamily("rocketmq_brokeruntime_putmessage_times_total", "brokerRuntimePutMessageTimesTotal", BROKER_RUNTIME_METRIC_LABEL_NAMES);
+ for (Map.Entry<BrokerRuntimeMetric, Long> entry : brokerRuntimePutMessageTimesTotal.entrySet()) {
+ loadBrokerRuntimeStatsMetric(brokerRuntimePutMessageTimesTotalF, entry);
+ }
+ mfs.add(brokerRuntimePutMessageTimesTotalF);
+
+ GaugeMetricFamily brokerRuntimeGetMessageEntireTimeMaxF = new GaugeMetricFamily("rocketmq_brokeruntime_getmessage_entire_time_max", "brokerRuntimeGetMessageEntireTimeMax", BROKER_RUNTIME_METRIC_LABEL_NAMES);
+ for (Map.Entry<BrokerRuntimeMetric, Long> entry : brokerRuntimeGetMessageEntireTimeMax.entrySet()) {
+ loadBrokerRuntimeStatsMetric(brokerRuntimeGetMessageEntireTimeMaxF, entry);
+ }
+ mfs.add(brokerRuntimeGetMessageEntireTimeMaxF);
+
+ GaugeMetricFamily brokerRuntimePageCacheLockTimeMillsF = new GaugeMetricFamily("rocketmq_brokeruntime_pagecache_lock_time_mills", "brokerRuntimePageCacheLockTimeMills", BROKER_RUNTIME_METRIC_LABEL_NAMES);
+ for (Map.Entry<BrokerRuntimeMetric, Long> entry : brokerRuntimePageCacheLockTimeMills.entrySet()) {
+ loadBrokerRuntimeStatsMetric(brokerRuntimePageCacheLockTimeMillsF, entry);
+ }
+ mfs.add(brokerRuntimePageCacheLockTimeMillsF);
+
+ GaugeMetricFamily brokerRuntimeCommitLogDiskRatioF = new GaugeMetricFamily("rocketmq_brokeruntime_commitlog_disk_ratio", "brokerRuntimeCommitLogDiskRatio", BROKER_RUNTIME_METRIC_LABEL_NAMES);
+ for (Map.Entry<BrokerRuntimeMetric, Double> entry : brokerRuntimeCommitLogDiskRatio.entrySet()) {
+ loadBrokerRuntimeStatsMetric(brokerRuntimeCommitLogDiskRatioF, entry);
+ }
+ mfs.add(brokerRuntimeCommitLogDiskRatioF);
+
+ GaugeMetricFamily brokerRuntimeConsumeQueueDiskRatioF = new GaugeMetricFamily("rocketmq_brokeruntime_consumequeue_disk_ratio", "brokerRuntimeConsumeQueueDiskRatio", BROKER_RUNTIME_METRIC_LABEL_NAMES);
+ for (Map.Entry<BrokerRuntimeMetric, Double> entry : brokerRuntimeConsumeQueueDiskRatio.entrySet()) {
+ loadBrokerRuntimeStatsMetric(brokerRuntimeConsumeQueueDiskRatioF, entry);
+ }
+ mfs.add(brokerRuntimeConsumeQueueDiskRatioF);
+
+ GaugeMetricFamily brokerRuntimeGetFoundTps600F = new GaugeMetricFamily("rocketmq_brokeruntime_getfound_tps600", "brokerRuntimeGetFoundTps600", BROKER_RUNTIME_METRIC_LABEL_NAMES);
+ for (Map.Entry<BrokerRuntimeMetric, Double> entry : brokerRuntimeGetFoundTps600.entrySet()) {
+ loadBrokerRuntimeStatsMetric(brokerRuntimeGetFoundTps600F, entry);
+ }
+ mfs.add(brokerRuntimeGetFoundTps600F);
+
+ GaugeMetricFamily brokerRuntimeGetFoundTps60F = new GaugeMetricFamily("rocketmq_brokeruntime_getfound_tps60", "brokerRuntimeGetFoundTps60", BROKER_RUNTIME_METRIC_LABEL_NAMES);
+ for (Map.Entry<BrokerRuntimeMetric, Double> entry : brokerRuntimeGetFoundTps60.entrySet()) {
+ loadBrokerRuntimeStatsMetric(brokerRuntimeGetFoundTps60F, entry);
+ }
+ mfs.add(brokerRuntimeGetFoundTps60F);
+
+ GaugeMetricFamily brokerRuntimeGetFoundTps10F = new GaugeMetricFamily("rocketmq_brokeruntime_getfound_tps10", "brokerRuntimeGetFoundTps10", BROKER_RUNTIME_METRIC_LABEL_NAMES);
+ for (Map.Entry<BrokerRuntimeMetric, Double> entry : brokerRuntimeGetFoundTps10.entrySet()) {
+ loadBrokerRuntimeStatsMetric(brokerRuntimeGetFoundTps10F, entry);
+ }
+ mfs.add(brokerRuntimeGetFoundTps10F);
+
+ GaugeMetricFamily brokerRuntimeGetTotalTps600F = new GaugeMetricFamily("rocketmq_brokeruntime_gettotal_tps600", "brokerRuntimeGetTotalTps600", BROKER_RUNTIME_METRIC_LABEL_NAMES);
+ for (Map.Entry<BrokerRuntimeMetric, Double> entry : brokerRuntimeGetTotalTps600.entrySet()) {
+ loadBrokerRuntimeStatsMetric(brokerRuntimeGetTotalTps600F, entry);
+ }
+ mfs.add(brokerRuntimeGetTotalTps600F);
+
+ GaugeMetricFamily brokerRuntimeGetTotalTps60F = new GaugeMetricFamily("rocketmq_brokeruntime_gettotal_tps60", "brokerRuntimeGetTotalTps60", BROKER_RUNTIME_METRIC_LABEL_NAMES);
+ for (Map.Entry<BrokerRuntimeMetric, Double> entry : brokerRuntimeGetTotalTps60.entrySet()) {
+ loadBrokerRuntimeStatsMetric(brokerRuntimeGetTotalTps60F, entry);
+ }
+ mfs.add(brokerRuntimeGetTotalTps60F);
+
+ GaugeMetricFamily brokerRuntimeGetTotalTps10F = new GaugeMetricFamily("rocketmq_brokeruntime_gettotal_tps10", "brokerRuntimeGetTotalTps10", BROKER_RUNTIME_METRIC_LABEL_NAMES);
+ for (Map.Entry<BrokerRuntimeMetric, Double> entry : brokerRuntimeGetTotalTps10.entrySet()) {
+ loadBrokerRuntimeStatsMetric(brokerRuntimeGetTotalTps10F, entry);
+ }
+ mfs.add(brokerRuntimeGetTotalTps10F);
+
+ GaugeMetricFamily brokerRuntimeGetTransferedTps600F = new GaugeMetricFamily("rocketmq_brokeruntime_gettransfered_tps600", "brokerRuntimeGetTransferedTps600", BROKER_RUNTIME_METRIC_LABEL_NAMES);
+ for (Map.Entry<BrokerRuntimeMetric, Double> entry : brokerRuntimeGetTransferedTps600.entrySet()) {
+ loadBrokerRuntimeStatsMetric(brokerRuntimeGetTransferedTps600F, entry);
+ }
+ mfs.add(brokerRuntimeGetTransferedTps600F);
+
+ GaugeMetricFamily brokerRuntimeGetTransferedTps60F = new GaugeMetricFamily("rocketmq_brokeruntime_gettransfered_tps60", "brokerRuntimeGetTransferedTps60", BROKER_RUNTIME_METRIC_LABEL_NAMES);
+ for (Map.Entry<BrokerRuntimeMetric, Double> entry : brokerRuntimeGetTransferedTps60.entrySet()) {
+ loadBrokerRuntimeStatsMetric(brokerRuntimeGetTransferedTps60F, entry);
+ }
+ mfs.add(brokerRuntimeGetTransferedTps60F);
+
+ GaugeMetricFamily brokerRuntimeGetTransferedTps10F = new GaugeMetricFamily("rocketmq_brokeruntime_gettransfered_tps10", "brokerRuntimeGetTransferedTps10", BROKER_RUNTIME_METRIC_LABEL_NAMES);
+ for (Map.Entry<BrokerRuntimeMetric, Double> entry : brokerRuntimeGetTransferedTps10.entrySet()) {
+ loadBrokerRuntimeStatsMetric(brokerRuntimeGetTransferedTps10F, entry);
+ }
+ mfs.add(brokerRuntimeGetTransferedTps10F);
+
+ GaugeMetricFamily brokerRuntimeGetMissTps600F = new GaugeMetricFamily("rocketmq_brokeruntime_getmiss_tps600", "brokerRuntimeGetMissTps600", BROKER_RUNTIME_METRIC_LABEL_NAMES);
+ for (Map.Entry<BrokerRuntimeMetric, Double> entry : brokerRuntimeGetMissTps600.entrySet()) {
+ loadBrokerRuntimeStatsMetric(brokerRuntimeGetMissTps600F, entry);
+ }
+ mfs.add(brokerRuntimeGetMissTps600F);
+
+ GaugeMetricFamily brokerRuntimeGetMissTps60F = new GaugeMetricFamily("rocketmq_brokeruntime_getmiss_tps60", "brokerRuntimeGetMissTps60", BROKER_RUNTIME_METRIC_LABEL_NAMES);
+ for (Map.Entry<BrokerRuntimeMetric, Double> entry : brokerRuntimeGetMissTps60.entrySet()) {
+ loadBrokerRuntimeStatsMetric(brokerRuntimeGetMissTps60F, entry);
+ }
+ mfs.add(brokerRuntimeGetMissTps60F);
+
+ GaugeMetricFamily brokerRuntimeGetMissTps10F = new GaugeMetricFamily("rocketmq_brokeruntime_getmiss_tps10", "brokerRuntimeGetMissTps10", BROKER_RUNTIME_METRIC_LABEL_NAMES);
+ for (Map.Entry<BrokerRuntimeMetric, Double> entry : brokerRuntimeGetMissTps10.entrySet()) {
+ loadBrokerRuntimeStatsMetric(brokerRuntimeGetMissTps10F, entry);
+ }
+ mfs.add(brokerRuntimeGetMissTps10F);
+
+ GaugeMetricFamily brokerRuntimePutTps600F = new GaugeMetricFamily("rocketmq_brokeruntime_put_tps600", "brokerRuntimePutTps600", BROKER_RUNTIME_METRIC_LABEL_NAMES);
+ for (Map.Entry<BrokerRuntimeMetric, Double> entry : brokerRuntimePutTps600.entrySet()) {
+ loadBrokerRuntimeStatsMetric(brokerRuntimePutTps600F, entry);
+ }
+ mfs.add(brokerRuntimePutTps600F);
+
+ GaugeMetricFamily brokerRuntimePutTps60F = new GaugeMetricFamily("rocketmq_brokeruntime_put_tps60", "brokerRuntimePutTps60", BROKER_RUNTIME_METRIC_LABEL_NAMES);
+ for (Map.Entry<BrokerRuntimeMetric, Double> entry : brokerRuntimePutTps60.entrySet()) {
+ loadBrokerRuntimeStatsMetric(brokerRuntimePutTps60F, entry);
+ }
+ mfs.add(brokerRuntimePutTps60F);
+
+ GaugeMetricFamily brokerRuntimePutTps10F = new GaugeMetricFamily("rocketmq_brokeruntime_put_tps10", "brokerRuntimePutTps10", BROKER_RUNTIME_METRIC_LABEL_NAMES);
+ for (Map.Entry<BrokerRuntimeMetric, Double> entry : brokerRuntimePutTps10.entrySet()) {
+ loadBrokerRuntimeStatsMetric(brokerRuntimePutTps10F, entry);
+ }
+ mfs.add(brokerRuntimePutTps10F);
+
+ GaugeMetricFamily brokerRuntimeDispatchMaxBufferF = new GaugeMetricFamily("rocketmq_brokeruntime_dispatch_maxbuffer", "brokerRuntimeDispatchMaxBuffer", BROKER_RUNTIME_METRIC_LABEL_NAMES);
+ for (Map.Entry<BrokerRuntimeMetric, Long> entry : brokerRuntimeDispatchMaxBuffer.entrySet()) {
+ loadBrokerRuntimeStatsMetric(brokerRuntimeDispatchMaxBufferF, entry);
+ }
+ mfs.add(brokerRuntimeDispatchMaxBufferF);
+
+ GaugeMetricFamily brokerRuntimePullThreadPoolQueueCapacityF = new GaugeMetricFamily("rocketmq_brokeruntime_pull_threadpoolqueue_capacity", "brokerRuntimePullThreadPoolQueueCapacity", BROKER_RUNTIME_METRIC_LABEL_NAMES);
+ for (Map.Entry<BrokerRuntimeMetric, Long> entry : brokerRuntimePullThreadPoolQueueCapacity.entrySet()) {
+ loadBrokerRuntimeStatsMetric(brokerRuntimePullThreadPoolQueueCapacityF, entry);
+ }
+ mfs.add(brokerRuntimePullThreadPoolQueueCapacityF);
+
+ GaugeMetricFamily brokerRuntimeSendThreadPoolQueueCapacityF = new GaugeMetricFamily("rocketmq_brokeruntime_send_threadpoolqueue_capacity", "brokerRuntimeSendThreadPoolQueueCapacity", BROKER_RUNTIME_METRIC_LABEL_NAMES);
+ for (Map.Entry<BrokerRuntimeMetric, Long> entry : brokerRuntimeSendThreadPoolQueueCapacity.entrySet()) {
+ loadBrokerRuntimeStatsMetric(brokerRuntimeSendThreadPoolQueueCapacityF, entry);
+ }
+ mfs.add(brokerRuntimeSendThreadPoolQueueCapacityF);
+
+ GaugeMetricFamily brokerRuntimePullThreadPoolQueueSizeF = new GaugeMetricFamily("rocketmq_brokeruntime_pull_threadpoolqueue_size", "brokerRuntimePullThreadPoolQueueSizeF", BROKER_RUNTIME_METRIC_LABEL_NAMES);
+ for (Map.Entry<BrokerRuntimeMetric, Long> entry : brokerRuntimePullThreadPoolQueueSize.entrySet()) {
+ loadBrokerRuntimeStatsMetric(brokerRuntimePullThreadPoolQueueSizeF, entry);
+ }
+ mfs.add(brokerRuntimePullThreadPoolQueueSizeF);
+
+ GaugeMetricFamily brokerRuntimeQueryThreadPoolQueueSizeF = new GaugeMetricFamily("rocketmq_brokeruntime_query_threadpoolqueue_size", "brokerRuntimeQueryThreadPoolQueueSize", BROKER_RUNTIME_METRIC_LABEL_NAMES);
+ for (Map.Entry<BrokerRuntimeMetric, Long> entry : brokerRuntimeQueryThreadPoolQueueSize.entrySet()) {
+ loadBrokerRuntimeStatsMetric(brokerRuntimeQueryThreadPoolQueueSizeF, entry);
+ }
+ mfs.add(brokerRuntimeQueryThreadPoolQueueSizeF);
+
+ GaugeMetricFamily brokerRuntimePullThreadPoolQueueHeadWaitTimeMillsF = new GaugeMetricFamily("rocketmq_brokeruntime_pull_threadpoolqueue_headwait_timemills", "brokerRuntimePullThreadPoolQueueHeadWaitTimeMills", BROKER_RUNTIME_METRIC_LABEL_NAMES);
+ for (Map.Entry<BrokerRuntimeMetric, Long> entry : brokerRuntimePullThreadPoolQueueHeadWaitTimeMills.entrySet()) {
+ loadBrokerRuntimeStatsMetric(brokerRuntimePullThreadPoolQueueHeadWaitTimeMillsF, entry);
+ }
+ mfs.add(brokerRuntimePullThreadPoolQueueHeadWaitTimeMillsF);
+
+ GaugeMetricFamily brokerRuntimeQueryThreadPoolQueueHeadWaitTimeMillsF = new GaugeMetricFamily("rocketmq_brokeruntime_query_threadpoolqueue_headwait_timemills", "brokerRuntimeQueryThreadPoolQueueHeadWaitTimeMills", BROKER_RUNTIME_METRIC_LABEL_NAMES);
+ for (Map.Entry<BrokerRuntimeMetric, Long> entry : brokerRuntimeQueryThreadPoolQueueHeadWaitTimeMills.entrySet()) {
+ loadBrokerRuntimeStatsMetric(brokerRuntimeQueryThreadPoolQueueHeadWaitTimeMillsF, entry);
+ }
+ mfs.add(brokerRuntimeQueryThreadPoolQueueHeadWaitTimeMillsF);
+
+ GaugeMetricFamily brokerRuntimeSendThreadPoolQueueHeadWaitTimeMillsF = new GaugeMetricFamily("rocketmq_brokeruntime_send_threadpoolqueue_headwait_timemills", "brokerRuntimeSendThreadPoolQueueHeadWaitTimeMills", BROKER_RUNTIME_METRIC_LABEL_NAMES);
+ for (Map.Entry<BrokerRuntimeMetric, Long> entry : brokerRuntimeSendThreadPoolQueueHeadWaitTimeMills.entrySet()) {
+ loadBrokerRuntimeStatsMetric(brokerRuntimeSendThreadPoolQueueHeadWaitTimeMillsF, entry);
+ }
+ mfs.add(brokerRuntimeSendThreadPoolQueueHeadWaitTimeMillsF);
+
+ GaugeMetricFamily brokerRuntimeMsgGetTotalYesterdayMorningF = new GaugeMetricFamily("rocketmq_brokeruntime_msg_gettotal_yesterdaymorning", "brokerRuntimeMsgGetTotalYesterdayMorning", BROKER_RUNTIME_METRIC_LABEL_NAMES);
+ for (Map.Entry<BrokerRuntimeMetric, Long> entry : brokerRuntimeMsgGetTotalYesterdayMorning.entrySet()) {
+ loadBrokerRuntimeStatsMetric(brokerRuntimeMsgGetTotalYesterdayMorningF, entry);
+ }
+ mfs.add(brokerRuntimeMsgGetTotalYesterdayMorningF);
+
+ GaugeMetricFamily brokerRuntimeMsgPutTotalYesterdayMorningF = new GaugeMetricFamily("rocketmq_brokeruntime_msg_puttotal_yesterdaymorning", "brokerRuntimeMsgPutTotalYesterdayMorning", BROKER_RUNTIME_METRIC_LABEL_NAMES);
+ for (Map.Entry<BrokerRuntimeMetric, Long> entry : brokerRuntimeMsgPutTotalYesterdayMorning.entrySet()) {
+ loadBrokerRuntimeStatsMetric(brokerRuntimeMsgPutTotalYesterdayMorningF, entry);
+ }
+ mfs.add(brokerRuntimeMsgPutTotalYesterdayMorningF);
+
+ GaugeMetricFamily brokerRuntimeMsgGetTotalTodayMorningF = new GaugeMetricFamily("rocketmq_brokeruntime_msg_gettotal_todaymorning", "brokerRuntimeMsgGetTotalTodayMorning", BROKER_RUNTIME_METRIC_LABEL_NAMES);
+ for (Map.Entry<BrokerRuntimeMetric, Long> entry : brokerRuntimeMsgGetTotalTodayMorning.entrySet()) {
+ loadBrokerRuntimeStatsMetric(brokerRuntimeMsgGetTotalTodayMorningF, entry);
+ }
+ mfs.add(brokerRuntimeMsgGetTotalTodayMorningF);
+
+ GaugeMetricFamily brokerRuntimeMsgPutTotalTodayMorningF = new GaugeMetricFamily("rocketmq_brokeruntime_msg_puttotal_todaymorning", "brokerRuntimeMsgPutTotalTodayMorning", BROKER_RUNTIME_METRIC_LABEL_NAMES);
+ for (Map.Entry<BrokerRuntimeMetric, Long> entry : brokerRuntimeMsgPutTotalTodayMorning.entrySet()) {
+ loadBrokerRuntimeStatsMetric(brokerRuntimeMsgPutTotalTodayMorningF, entry);
+ }
+ mfs.add(brokerRuntimeMsgPutTotalTodayMorningF);
+
+ GaugeMetricFamily brokerRuntimeCommitLogDirCapacityFreeF = new GaugeMetricFamily("rocketmq_brokeruntime_commitlogdir_capacity_free", "brokerRuntimeCommitLogDirCapacityFree", BROKER_RUNTIME_METRIC_LABEL_NAMES);
+ for (Map.Entry<BrokerRuntimeMetric, Double> entry : brokerRuntimeCommitLogDirCapacityFree.entrySet()) {
+ loadBrokerRuntimeStatsMetric(brokerRuntimeCommitLogDirCapacityFreeF, entry);
+ }
+ mfs.add(brokerRuntimeCommitLogDirCapacityFreeF);
+
+ GaugeMetricFamily brokerRuntimeCommitLogDirCapacityTotalF = new GaugeMetricFamily("rocketmq_brokeruntime_commitlogdir_capacity_total", "brokerRuntimeCommitLogDirCapacityTotal", BROKER_RUNTIME_METRIC_LABEL_NAMES);
+ for (Map.Entry<BrokerRuntimeMetric, Double> entry : brokerRuntimeCommitLogDirCapacityTotal.entrySet()) {
+ loadBrokerRuntimeStatsMetric(brokerRuntimeCommitLogDirCapacityTotalF, entry);
+ }
+ mfs.add(brokerRuntimeCommitLogDirCapacityTotalF);
+
+ GaugeMetricFamily brokerRuntimeCommitLogMaxOffsetF = new GaugeMetricFamily("rocketmq_brokeruntime_commitlog_maxoffset", "brokerRuntimeCommitLogMaxOffset", BROKER_RUNTIME_METRIC_LABEL_NAMES);
+ for (Map.Entry<BrokerRuntimeMetric, Long> entry : brokerRuntimeCommitLogMaxOffset.entrySet()) {
+ loadBrokerRuntimeStatsMetric(brokerRuntimeCommitLogMaxOffsetF, entry);
+ }
+ mfs.add(brokerRuntimeCommitLogMaxOffsetF);
+
+ GaugeMetricFamily brokerRuntimeCommitLogMinOffsetF = new GaugeMetricFamily("rocketmq_brokeruntime_commitlog_minoffset", "brokerRuntimeCommitLogMinOffset", BROKER_RUNTIME_METRIC_LABEL_NAMES);
+ for (Map.Entry<BrokerRuntimeMetric, Long> entry : brokerRuntimeCommitLogMinOffset.entrySet()) {
+ loadBrokerRuntimeStatsMetric(brokerRuntimeCommitLogMinOffsetF, entry);
+ }
+ mfs.add(brokerRuntimeCommitLogMinOffsetF);
+
+ GaugeMetricFamily brokerRuntimeRemainHowManyDataToFlushF = new GaugeMetricFamily("rocketmq_brokeruntime_remain_howmanydata_toflush", "brokerRuntimeRemainHowManyDataToFlush", BROKER_RUNTIME_METRIC_LABEL_NAMES);
+ for (Map.Entry<BrokerRuntimeMetric, Double> entry : brokerRuntimeRemainHowManyDataToFlush.entrySet()) {
+ loadBrokerRuntimeStatsMetric(brokerRuntimeRemainHowManyDataToFlushF, entry);
+ }
+ mfs.add(brokerRuntimeRemainHowManyDataToFlushF);
}
}
\ No newline at end of file
diff --git a/src/main/java/org/apache/rocketmq/exporter/config/RMQConfigure.java b/src/main/java/org/apache/rocketmq/exporter/config/RMQConfigure.java
index 2b3956b..e090b68 100644
--- a/src/main/java/org/apache/rocketmq/exporter/config/RMQConfigure.java
+++ b/src/main/java/org/apache/rocketmq/exporter/config/RMQConfigure.java
@@ -23,8 +23,6 @@ import org.slf4j.LoggerFactory;
import org.springframework.boot.context.properties.ConfigurationProperties;
import org.springframework.context.annotation.Configuration;
-
-
import static org.apache.rocketmq.client.ClientConfig.SEND_MESSAGE_WITH_VIP_CHANNEL_PROPERTY;
@Configuration
@@ -69,9 +67,11 @@ public class RMQConfigure {
logger.info("setIsVIPChannel isVIPChannel={}", isVIPChannel);
}
}
+
public boolean isEnableCollect() {
return enableCollect;
}
+
public void setEnableCollect(boolean enableCollect) {
this.enableCollect = enableCollect;
}
diff --git a/src/main/java/org/apache/rocketmq/exporter/config/ScheduleConfig.java b/src/main/java/org/apache/rocketmq/exporter/config/ScheduleConfig.java
new file mode 100644
index 0000000..9422272
--- /dev/null
+++ b/src/main/java/org/apache/rocketmq/exporter/config/ScheduleConfig.java
@@ -0,0 +1,26 @@
+package org.apache.rocketmq.exporter.config;
+
+import org.springframework.beans.factory.annotation.Value;
+import org.springframework.context.annotation.Bean;
+import org.springframework.context.annotation.Configuration;
+import org.springframework.scheduling.annotation.SchedulingConfigurer;
+import org.springframework.scheduling.config.ScheduledTaskRegistrar;
+
+import java.util.concurrent.Executor;
+import java.util.concurrent.Executors;
+
+@Configuration
+public class ScheduleConfig implements SchedulingConfigurer {
+ @Override
+ public void configureTasks(ScheduledTaskRegistrar taskRegistrar) {
+ taskRegistrar.setScheduler(taskExecutor());
+ }
+
+ @Value("${task.count}")
+ private int taskCount;
+
+ @Bean(destroyMethod = "shutdown")
+ public Executor taskExecutor() {
+ return Executors.newScheduledThreadPool(this.taskCount);
+ }
+}
\ No newline at end of file
diff --git a/src/main/java/org/apache/rocketmq/exporter/controller/RMQMetricsController.java b/src/main/java/org/apache/rocketmq/exporter/controller/RMQMetricsController.java
index 4df69d1..e6fe31c 100644
--- a/src/main/java/org/apache/rocketmq/exporter/controller/RMQMetricsController.java
+++ b/src/main/java/org/apache/rocketmq/exporter/controller/RMQMetricsController.java
@@ -18,8 +18,6 @@ package org.apache.rocketmq.exporter.controller;
import org.apache.rocketmq.exporter.service.RMQMetricsService;
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
import org.springframework.boot.autoconfigure.EnableAutoConfiguration;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.ResponseBody;
@@ -33,19 +31,14 @@ import java.io.StringWriter;
@RestController
@EnableAutoConfiguration
public class RMQMetricsController {
-
- private final static Logger log = LoggerFactory.getLogger(RMQMetricsController.class);
-
@Resource
RMQMetricsService metricsService;
@RequestMapping(value = "${rocketmq.config.webTelemetryPath}")
@ResponseBody
private void metrics(HttpServletResponse response) throws IOException {
-
StringWriter writer = new StringWriter();
- metricsService.Metrics(writer);
-
+ metricsService.metrics(writer);
response.setHeader("Content-Type", "text/plain; version=0.0.4; charset=utf-8");
response.getOutputStream().print(writer.toString());
}
diff --git a/src/main/java/org/apache/rocketmq/exporter/exception/ServiceException.java b/src/main/java/org/apache/rocketmq/exporter/exception/ServiceException.java
deleted file mode 100644
index ccf3fdc..0000000
--- a/src/main/java/org/apache/rocketmq/exporter/exception/ServiceException.java
+++ /dev/null
@@ -1,31 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-package org.apache.rocketmq.exporter.exception;
-
-public class ServiceException extends RuntimeException {
- private static final long serialVersionUID = 9213584003139969215L;
- private int code;
-
- public ServiceException(int code, String message) {
- super(message);
- this.code = code;
- }
-
- public int getCode() {
- return code;
- }
-}
diff --git a/src/main/java/org/apache/rocketmq/exporter/model/BrokerRuntimeStats.java b/src/main/java/org/apache/rocketmq/exporter/model/BrokerRuntimeStats.java
new file mode 100644
index 0000000..b6fe3dc
--- /dev/null
+++ b/src/main/java/org/apache/rocketmq/exporter/model/BrokerRuntimeStats.java
@@ -0,0 +1,609 @@
+package org.apache.rocketmq.exporter.model;
+
+import org.apache.rocketmq.common.protocol.body.KVTable;
+
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+
+public class BrokerRuntimeStats {
+ //今天生产的消息总量
+ private long msgPutTotalTodayNow;
+ //今天消费的消息总量
+ private long msgGetTotalTodayNow;
+
+ //今天早上生产消息总量
+ private long msgPutTotalTodayMorning;
+ //今天早上消费消息总量
+ private long msgGetTotalTodayMorning;
+
+ //昨天早上生产的消息总量
+ private long msgPutTotalYesterdayMorning;
+ //昨天早上消费的消息总量
+ private long msgGetTotalYesterdayMorning;
+
+ //延迟消息位点
+ private List<ScheduleMessageOffsetTable> scheduleMessageOffsetTables = new ArrayList<>();
+
+ //发送线程最大等待时间
+ private long sendThreadPoolQueueHeadWaitTimeMills;
+ //拉取消息线程最大等待时间
+ private long queryThreadPoolQueueHeadWaitTimeMills;
+ //拉取线程最大等待时间
+ private long pullThreadPoolQueueHeadWaitTimeMills;
+
+ //查询线程任务个数
+ private long queryThreadPoolQueueSize;
+ //拉取线程任务个数
+ private long pullThreadPoolQueueSize;
+ //发送线程等待队列长度
+ private long sendThreadPoolQueueCapacity;
+ //拉取线程等待队列长度
+ private long pullThreadPoolQueueCapacity;
+
+ //刷pagecache时间统计
+ private Map<String, Integer> putMessageDistributeTimeMap = new HashMap<>();
+ //还有多少字节的数据没有刷盘
+ private double remainHowManyDataToFlush;
+
+ //commitlog 最小位点
+ private long commitLogMinOffset;
+ //commitlog 最大位点
+ private long commitLogMaxOffset;
+
+ //broker运行时间描述
+ private String runtime;
+ //broker 启动时间
+ private long bootTimestamp;
+ //broker 磁盘总量
+ private double commitLogDirCapacityTotal;
+ //broker 磁盘剩余
+ private double commitLogDirCapacityFree;
+ //broker 版本号
+ private int brokerVersion;
+ //
+ private long dispatchMaxBuffer;
+
+ private PutTps putTps = new PutTps();
+ private GetMissTps getMissTps = new GetMissTps();
+ private GetTransferedTps getTransferedTps = new GetTransferedTps();
+ private GetTotalTps getTotalTps = new GetTotalTps();
+ private GetFoundTps getFoundTps = new GetFoundTps();
+
+ private double consumeQueueDiskRatio;
+ private double commitLogDiskRatio;
+
+ //page cache锁定时间
+ private long pageCacheLockTimeMills;
+
+ private long getMessageEntireTimeMax;
+
+ private long putMessageTimesTotal;
+
+ private String brokerVersionDesc;
+ private long sendThreadPoolQueueSize;
+ private long startAcceptSendRequestTimeStamp;
+ private long putMessageEntireTimeMax;
+ private long earliestMessageTimeStamp;
+
+ private long remainTransientStoreBufferNumbs;
+ private long queryThreadPoolQueueCapacity;
+ //发送消息平均体积大小
+ private double putMessageAverageSize;
+ //全部发送消息数
+ private long putMessageSizeTotal;
+ private long dispatchBehindBytes;
+
+
+ public BrokerRuntimeStats(KVTable kvTable) {
+ this.msgPutTotalTodayNow = Long.parseLong(kvTable.getTable().get("msgPutTotalTodayNow"));
+
+ loadScheduleMessageOffsets(kvTable);
+ loadPutMessageDistributeTime(kvTable.getTable().get("putMessageDistributeTime"));
+
+ loadTps(this.putTps, kvTable.getTable().get("putTps"));
+ loadTps(this.getMissTps, kvTable.getTable().get("getMissTps"));
+ loadTps(this.getTransferedTps, kvTable.getTable().get("getTransferedTps"));
+ loadTps(this.getTotalTps, kvTable.getTable().get("getTotalTps"));
+ loadTps(this.getFoundTps, kvTable.getTable().get("getFoundTps"));
+
+ loadCommitLogDirCapacity(kvTable.getTable().get("commitLogDirCapacity"));
+
+ this.sendThreadPoolQueueHeadWaitTimeMills = Long.parseLong(kvTable.getTable().get("sendThreadPoolQueueHeadWaitTimeMills"));
+ this.queryThreadPoolQueueHeadWaitTimeMills = Long.parseLong(kvTable.getTable().get("queryThreadPoolQueueHeadWaitTimeMills"));
+
+ this.remainHowManyDataToFlush = Double.parseDouble(kvTable.getTable().get("remainHowManyDataToFlush").split(" ")[0]);//byte
+ this.msgGetTotalTodayNow = Long.parseLong(kvTable.getTable().get("msgGetTotalTodayNow"));
+ this.queryThreadPoolQueueSize = Long.parseLong(kvTable.getTable().get("queryThreadPoolQueueSize"));
+ this.bootTimestamp = Long.parseLong(kvTable.getTable().get("bootTimestamp"));
+ this.msgPutTotalYesterdayMorning = Long.parseLong(kvTable.getTable().get("msgPutTotalYesterdayMorning"));
+ this.msgGetTotalYesterdayMorning = Long.parseLong(kvTable.getTable().get("msgGetTotalYesterdayMorning"));
+ this.pullThreadPoolQueueSize = Long.parseLong(kvTable.getTable().get("pullThreadPoolQueueSize"));
+ this.commitLogMinOffset = Long.parseLong(kvTable.getTable().get("commitLogMinOffset"));
+ this.pullThreadPoolQueueHeadWaitTimeMills = Long.parseLong(kvTable.getTable().get("pullThreadPoolQueueHeadWaitTimeMills"));
+ this.runtime = kvTable.getTable().get("runtime");
+ this.dispatchMaxBuffer = Long.parseLong(kvTable.getTable().get("dispatchMaxBuffer"));
+ this.brokerVersion = Integer.parseInt(kvTable.getTable().get("brokerVersion"));
+ this.consumeQueueDiskRatio = Double.parseDouble(kvTable.getTable().get("consumeQueueDiskRatio"));
+ this.pageCacheLockTimeMills = Long.parseLong(kvTable.getTable().get("pageCacheLockTimeMills"));
+ this.commitLogDiskRatio = Double.parseDouble(kvTable.getTable().get("commitLogDiskRatio"));
+ this.commitLogMaxOffset = Long.parseLong(kvTable.getTable().get("commitLogMaxOffset"));
+ this.getMessageEntireTimeMax = Long.parseLong(kvTable.getTable().get("getMessageEntireTimeMax"));
+ this.msgPutTotalTodayMorning = Long.parseLong(kvTable.getTable().get("msgPutTotalTodayMorning"));
+ this.putMessageTimesTotal = Long.parseLong(kvTable.getTable().get("putMessageTimesTotal"));
+ this.msgGetTotalTodayMorning = Long.parseLong(kvTable.getTable().get("msgGetTotalTodayMorning"));
+ this.brokerVersionDesc = kvTable.getTable().get("brokerVersionDesc");
+ this.sendThreadPoolQueueSize = Long.parseLong(kvTable.getTable().get("sendThreadPoolQueueSize"));
+ this.startAcceptSendRequestTimeStamp = Long.parseLong(kvTable.getTable().get("startAcceptSendRequestTimeStamp"));
+ this.putMessageEntireTimeMax = Long.parseLong(kvTable.getTable().get("putMessageEntireTimeMax"));
+ this.earliestMessageTimeStamp = Long.parseLong(kvTable.getTable().get("earliestMessageTimeStamp"));
+ this.remainTransientStoreBufferNumbs = Long.parseLong(kvTable.getTable().get("remainTransientStoreBufferNumbs"));
+ this.queryThreadPoolQueueCapacity = Long.parseLong(kvTable.getTable().get("queryThreadPoolQueueCapacity"));
+ this.putMessageAverageSize = Double.parseDouble(kvTable.getTable().get("putMessageAverageSize"));
+ this.dispatchBehindBytes = Long.parseLong(kvTable.getTable().get("dispatchBehindBytes"));
+ this.putMessageSizeTotal = Long.parseLong(kvTable.getTable().get("putMessageSizeTotal"));
+ this.sendThreadPoolQueueCapacity = Long.parseLong(kvTable.getTable().get("sendThreadPoolQueueCapacity"));
+ this.pullThreadPoolQueueCapacity = Long.parseLong(kvTable.getTable().get("pullThreadPoolQueueCapacity"));
+
+ }
+
+ private void loadCommitLogDirCapacity(String commitLogDirCapacity) {
+ String[] arr = commitLogDirCapacity.split(" ");
+ double total = Double.parseDouble(arr[2]);
+ double free = Double.parseDouble(arr[6]);
+ this.commitLogDirCapacityTotal = total;
+ this.commitLogDirCapacityFree = free;
+ }
+
+ private void loadTps(PutTps putTps, String value) {
+ String[] arr = value.split(" ");
+ putTps.ten = Double.parseDouble(arr[0]);
+ putTps.sixty = Double.parseDouble(arr[1]);
+ putTps.sixHundred = Double.parseDouble(arr[2]);
+ }
+
+ private void loadPutMessageDistributeTime(String str) {
+ String[] arr = str.split(" ");
+ String key = "", value = "";
+ for (String ar : arr) {
+ String[] tarr = ar.split(":");
+ key = tarr[0].replace("[", "").replace("]", "");
+ value = tarr[1];
+ this.putMessageDistributeTimeMap.put(key, Integer.parseInt(value));
+ }
+ }
+
+ public void loadScheduleMessageOffsets(KVTable kvTable) {
+ for (String key : kvTable.getTable().keySet()) {
+ if (key.startsWith("scheduleMessageOffset")) {
+ String[] arr = kvTable.getTable().get(key).split(",");
+ ScheduleMessageOffsetTable table = new ScheduleMessageOffsetTable(
+ Long.parseLong(arr[0]),
+ Long.parseLong(arr[1])
+ );
+ this.scheduleMessageOffsetTables.add(table);
+ }
+ }
+ }
+
+ public static class ScheduleMessageOffsetTable {
+ private long delayOffset;
+ private long maxOffset;
+
+ public ScheduleMessageOffsetTable(long first, long second) {
+ this.delayOffset = first;
+ this.maxOffset = second;
+ }
+
+ public long getDelayOffset() {
+ return delayOffset;
+ }
+
+ public void setDelayOffset(long delayOffset) {
+ this.delayOffset = delayOffset;
+ }
+
+ public long getMaxOffset() {
+ return maxOffset;
+ }
+
+ public void setMaxOffset(long maxOffset) {
+ this.maxOffset = maxOffset;
+ }
+ }
+
+ public class PutTps {
+ private double ten;
+ private double sixty;
+ private double sixHundred;
+
+ public double getTen() {
+ return ten;
+ }
+
+ public void setTen(double ten) {
+ this.ten = ten;
+ }
+
+ public double getSixty() {
+ return sixty;
+ }
+
+ public void setSixty(double sixty) {
+ this.sixty = sixty;
+ }
+
+ public double getSixHundred() {
+ return sixHundred;
+ }
+
+ public void setSixHundred(double sixHundred) {
+ this.sixHundred = sixHundred;
+ }
+ }
+
+ public class GetMissTps extends PutTps {
+ }
+
+ public class GetTransferedTps extends PutTps {
+ }
+
+ public class GetTotalTps extends PutTps {
+ }
+
+ public class GetFoundTps extends PutTps {
+ }
+
+ public long getMsgPutTotalTodayNow() {
+ return msgPutTotalTodayNow;
+ }
+
+ public void setMsgPutTotalTodayNow(long msgPutTotalTodayNow) {
+ this.msgPutTotalTodayNow = msgPutTotalTodayNow;
+ }
+
+ public long getMsgGetTotalTodayNow() {
+ return msgGetTotalTodayNow;
+ }
+
+ public void setMsgGetTotalTodayNow(long msgGetTotalTodayNow) {
+ this.msgGetTotalTodayNow = msgGetTotalTodayNow;
+ }
+
+ public long getMsgPutTotalTodayMorning() {
+ return msgPutTotalTodayMorning;
+ }
+
+ public void setMsgPutTotalTodayMorning(long msgPutTotalTodayMorning) {
+ this.msgPutTotalTodayMorning = msgPutTotalTodayMorning;
+ }
+
+ public long getMsgGetTotalTodayMorning() {
+ return msgGetTotalTodayMorning;
+ }
+
+ public void setMsgGetTotalTodayMorning(long msgGetTotalTodayMorning) {
+ this.msgGetTotalTodayMorning = msgGetTotalTodayMorning;
+ }
+
+ public long getMsgPutTotalYesterdayMorning() {
+ return msgPutTotalYesterdayMorning;
+ }
+
+ public void setMsgPutTotalYesterdayMorning(long msgPutTotalYesterdayMorning) {
+ this.msgPutTotalYesterdayMorning = msgPutTotalYesterdayMorning;
+ }
+
+ public long getMsgGetTotalYesterdayMorning() {
+ return msgGetTotalYesterdayMorning;
+ }
+
+ public void setMsgGetTotalYesterdayMorning(long msgGetTotalYesterdayMorning) {
+ this.msgGetTotalYesterdayMorning = msgGetTotalYesterdayMorning;
+ }
+
+ public List<ScheduleMessageOffsetTable> getScheduleMessageOffsetTables() {
+ return scheduleMessageOffsetTables;
+ }
+
+ public void setScheduleMessageOffsetTables(List<ScheduleMessageOffsetTable> scheduleMessageOffsetTables) {
+ this.scheduleMessageOffsetTables = scheduleMessageOffsetTables;
+ }
+
+ public long getSendThreadPoolQueueHeadWaitTimeMills() {
+ return sendThreadPoolQueueHeadWaitTimeMills;
+ }
+
+ public void setSendThreadPoolQueueHeadWaitTimeMills(long sendThreadPoolQueueHeadWaitTimeMills) {
+ this.sendThreadPoolQueueHeadWaitTimeMills = sendThreadPoolQueueHeadWaitTimeMills;
+ }
+
+ public long getQueryThreadPoolQueueHeadWaitTimeMills() {
+ return queryThreadPoolQueueHeadWaitTimeMills;
+ }
+
+ public void setQueryThreadPoolQueueHeadWaitTimeMills(long queryThreadPoolQueueHeadWaitTimeMills) {
+ this.queryThreadPoolQueueHeadWaitTimeMills = queryThreadPoolQueueHeadWaitTimeMills;
+ }
+
+ public long getPullThreadPoolQueueHeadWaitTimeMills() {
+ return pullThreadPoolQueueHeadWaitTimeMills;
+ }
+
+ public void setPullThreadPoolQueueHeadWaitTimeMills(long pullThreadPoolQueueHeadWaitTimeMills) {
+ this.pullThreadPoolQueueHeadWaitTimeMills = pullThreadPoolQueueHeadWaitTimeMills;
+ }
+
+ public long getQueryThreadPoolQueueSize() {
+ return queryThreadPoolQueueSize;
+ }
+
+ public void setQueryThreadPoolQueueSize(long queryThreadPoolQueueSize) {
+ this.queryThreadPoolQueueSize = queryThreadPoolQueueSize;
+ }
+
+ public long getPullThreadPoolQueueSize() {
+ return pullThreadPoolQueueSize;
+ }
+
+ public void setPullThreadPoolQueueSize(long pullThreadPoolQueueSize) {
+ this.pullThreadPoolQueueSize = pullThreadPoolQueueSize;
+ }
+
+ public long getSendThreadPoolQueueCapacity() {
+ return sendThreadPoolQueueCapacity;
+ }
+
+ public void setSendThreadPoolQueueCapacity(long sendThreadPoolQueueCapacity) {
+ this.sendThreadPoolQueueCapacity = sendThreadPoolQueueCapacity;
+ }
+
+ public long getPullThreadPoolQueueCapacity() {
+ return pullThreadPoolQueueCapacity;
+ }
+
+ public void setPullThreadPoolQueueCapacity(long pullThreadPoolQueueCapacity) {
+ this.pullThreadPoolQueueCapacity = pullThreadPoolQueueCapacity;
+ }
+
+ public Map<String, Integer> getPutMessageDistributeTimeMap() {
+ return putMessageDistributeTimeMap;
+ }
+
+ public void setPutMessageDistributeTimeMap(Map<String, Integer> putMessageDistributeTimeMap) {
+ this.putMessageDistributeTimeMap = putMessageDistributeTimeMap;
+ }
+
+ public double getRemainHowManyDataToFlush() {
+ return remainHowManyDataToFlush;
+ }
+
+ public void setRemainHowManyDataToFlush(double remainHowManyDataToFlush) {
+ this.remainHowManyDataToFlush = remainHowManyDataToFlush;
+ }
+
+ public long getCommitLogMinOffset() {
+ return commitLogMinOffset;
+ }
+
+ public void setCommitLogMinOffset(long commitLogMinOffset) {
+ this.commitLogMinOffset = commitLogMinOffset;
+ }
+
+ public long getCommitLogMaxOffset() {
+ return commitLogMaxOffset;
+ }
+
+ public void setCommitLogMaxOffset(long commitLogMaxOffset) {
+ this.commitLogMaxOffset = commitLogMaxOffset;
+ }
+
+ public String getRuntime() {
+ return runtime;
+ }
+
+ public void setRuntime(String runtime) {
+ this.runtime = runtime;
+ }
+
+ public long getBootTimestamp() {
+ return bootTimestamp;
+ }
+
+ public void setBootTimestamp(long bootTimestamp) {
+ this.bootTimestamp = bootTimestamp;
+ }
+
+ public double getCommitLogDirCapacityTotal() {
+ return commitLogDirCapacityTotal;
+ }
+
+ public void setCommitLogDirCapacityTotal(double commitLogDirCapacityTotal) {
+ this.commitLogDirCapacityTotal = commitLogDirCapacityTotal;
+ }
+
+ public double getCommitLogDirCapacityFree() {
+ return commitLogDirCapacityFree;
+ }
+
+ public void setCommitLogDirCapacityFree(double commitLogDirCapacityFree) {
+ this.commitLogDirCapacityFree = commitLogDirCapacityFree;
+ }
+
+ public int getBrokerVersion() {
+ return brokerVersion;
+ }
+
+ public void setBrokerVersion(int brokerVersion) {
+ this.brokerVersion = brokerVersion;
+ }
+
+ public long getDispatchMaxBuffer() {
+ return dispatchMaxBuffer;
+ }
+
+ public void setDispatchMaxBuffer(long dispatchMaxBuffer) {
+ this.dispatchMaxBuffer = dispatchMaxBuffer;
+ }
+
+ public PutTps getPutTps() {
+ return putTps;
+ }
+
+ public void setPutTps(PutTps putTps) {
+ this.putTps = putTps;
+ }
+
+ public GetMissTps getGetMissTps() {
+ return getMissTps;
+ }
+
+ public void setGetMissTps(GetMissTps getMissTps) {
+ this.getMissTps = getMissTps;
+ }
+
+ public GetTransferedTps getGetTransferedTps() {
+ return getTransferedTps;
+ }
+
+ public void setGetTransferedTps(GetTransferedTps getTransferedTps) {
+ this.getTransferedTps = getTransferedTps;
+ }
+
+ public GetTotalTps getGetTotalTps() {
+ return getTotalTps;
+ }
+
+ public void setGetTotalTps(GetTotalTps getTotalTps) {
+ this.getTotalTps = getTotalTps;
+ }
+
+ public GetFoundTps getGetFoundTps() {
+ return getFoundTps;
+ }
+
+ public void setGetFoundTps(GetFoundTps getFoundTps) {
+ this.getFoundTps = getFoundTps;
+ }
+
+ public double getConsumeQueueDiskRatio() {
+ return consumeQueueDiskRatio;
+ }
+
+ public void setConsumeQueueDiskRatio(double consumeQueueDiskRatio) {
+ this.consumeQueueDiskRatio = consumeQueueDiskRatio;
+ }
+
+ public double getCommitLogDiskRatio() {
+ return commitLogDiskRatio;
+ }
+
+ public void setCommitLogDiskRatio(double commitLogDiskRatio) {
+ this.commitLogDiskRatio = commitLogDiskRatio;
+ }
+
+ public long getPageCacheLockTimeMills() {
+ return pageCacheLockTimeMills;
+ }
+
+ public void setPageCacheLockTimeMills(long pageCacheLockTimeMills) {
+ this.pageCacheLockTimeMills = pageCacheLockTimeMills;
+ }
+
+ public long getGetMessageEntireTimeMax() {
+ return getMessageEntireTimeMax;
+ }
+
+ public void setGetMessageEntireTimeMax(long getMessageEntireTimeMax) {
+ this.getMessageEntireTimeMax = getMessageEntireTimeMax;
+ }
+
+ public long getPutMessageTimesTotal() {
+ return putMessageTimesTotal;
+ }
+
+ public void setPutMessageTimesTotal(long putMessageTimesTotal) {
+ this.putMessageTimesTotal = putMessageTimesTotal;
+ }
+
+ public String getBrokerVersionDesc() {
+ return brokerVersionDesc;
+ }
+
+ public void setBrokerVersionDesc(String brokerVersionDesc) {
+ this.brokerVersionDesc = brokerVersionDesc;
+ }
+
+ public long getSendThreadPoolQueueSize() {
+ return sendThreadPoolQueueSize;
+ }
+
+ public void setSendThreadPoolQueueSize(long sendThreadPoolQueueSize) {
+ this.sendThreadPoolQueueSize = sendThreadPoolQueueSize;
+ }
+
+ public long getStartAcceptSendRequestTimeStamp() {
+ return startAcceptSendRequestTimeStamp;
+ }
+
+ public void setStartAcceptSendRequestTimeStamp(long startAcceptSendRequestTimeStamp) {
+ this.startAcceptSendRequestTimeStamp = startAcceptSendRequestTimeStamp;
+ }
+
+ public long getPutMessageEntireTimeMax() {
+ return putMessageEntireTimeMax;
+ }
+
+ public void setPutMessageEntireTimeMax(long putMessageEntireTimeMax) {
+ this.putMessageEntireTimeMax = putMessageEntireTimeMax;
+ }
+
+ public long getEarliestMessageTimeStamp() {
+ return earliestMessageTimeStamp;
+ }
+
+ public void setEarliestMessageTimeStamp(long earliestMessageTimeStamp) {
+ this.earliestMessageTimeStamp = earliestMessageTimeStamp;
+ }
+
+ public long getRemainTransientStoreBufferNumbs() {
+ return remainTransientStoreBufferNumbs;
+ }
+
+ public void setRemainTransientStoreBufferNumbs(long remainTransientStoreBufferNumbs) {
+ this.remainTransientStoreBufferNumbs = remainTransientStoreBufferNumbs;
+ }
+
+ public long getQueryThreadPoolQueueCapacity() {
+ return queryThreadPoolQueueCapacity;
+ }
+
+ public void setQueryThreadPoolQueueCapacity(long queryThreadPoolQueueCapacity) {
+ this.queryThreadPoolQueueCapacity = queryThreadPoolQueueCapacity;
+ }
+
+ public double getPutMessageAverageSize() {
+ return putMessageAverageSize;
+ }
+
+ public void setPutMessageAverageSize(double putMessageAverageSize) {
+ this.putMessageAverageSize = putMessageAverageSize;
+ }
+
+ public long getPutMessageSizeTotal() {
+ return putMessageSizeTotal;
+ }
+
+ public void setPutMessageSizeTotal(long putMessageSizeTotal) {
+ this.putMessageSizeTotal = putMessageSizeTotal;
+ }
+
+ public long getDispatchBehindBytes() {
+ return dispatchBehindBytes;
+ }
+
+ public void setDispatchBehindBytes(long dispatchBehindBytes) {
+ this.dispatchBehindBytes = dispatchBehindBytes;
+ }
+}
diff --git a/src/main/java/org/apache/rocketmq/exporter/model/metrics/BrokerMetric.java b/src/main/java/org/apache/rocketmq/exporter/model/metrics/BrokerMetric.java
index c7a0727..45caff2 100644
--- a/src/main/java/org/apache/rocketmq/exporter/model/metrics/BrokerMetric.java
+++ b/src/main/java/org/apache/rocketmq/exporter/model/metrics/BrokerMetric.java
@@ -17,32 +17,38 @@
package org.apache.rocketmq.exporter.model.metrics;
public class BrokerMetric {
+ private String clusterName;
+ private String brokerIP;
+ private String brokerHost;
- private String clusterName;
- private String brokerName;
-
-
- public void setClusterName(String cluster) {
-
- clusterName = cluster;
+ public BrokerMetric(String clusterName, String brokerIP, String brokerHost) {
+ this.clusterName = clusterName;
+ this.brokerIP = brokerIP;
+ this.brokerHost = brokerHost;
}
- public String getClusterName() {
+ public String getClusterName() {
return clusterName;
}
- void setBrokerName(String broker) {
- brokerName = broker;
+ public void setClusterName(String clusterName) {
+ this.clusterName = clusterName;
}
- public String getBrokerName() {
+ public String getBrokerIP() {
+ return brokerIP;
+ }
+
+ public void setBrokerIP(String brokerIP) {
+ this.brokerIP = brokerIP;
+ }
- return brokerName;
+ public String getBrokerHost() {
+ return brokerHost;
}
- public BrokerMetric(String cluster, String broker) {
- clusterName = cluster;
- brokerName = broker;
+ public void setBrokerHost(String brokerHost) {
+ this.brokerHost = brokerHost;
}
@Override
@@ -52,19 +58,20 @@ public class BrokerMetric {
}
BrokerMetric other = (BrokerMetric) obj;
- return other.clusterName.equals(clusterName) && other.brokerName.equals(brokerName);
+ return other.clusterName.equals(clusterName) &&
+ other.brokerIP.equals(brokerIP);
}
@Override
public int hashCode() {
int hash = 1;
hash = 37 * hash + clusterName.hashCode();
- hash = 37 * hash + brokerName.hashCode();
+ hash = 37 * hash + brokerIP.hashCode();
return hash;
}
@Override
public String toString() {
- return "ClusterName: " + clusterName + " BrokerName: " + brokerName;
+ return "ClusterName: " + clusterName + " brokerIP: " + brokerIP + " brokerHost: " + brokerHost;
}
}
diff --git a/src/main/java/org/apache/rocketmq/exporter/model/metrics/ConsumerCountMetric.java b/src/main/java/org/apache/rocketmq/exporter/model/metrics/ConsumerCountMetric.java
new file mode 100644
index 0000000..67d9f20
--- /dev/null
+++ b/src/main/java/org/apache/rocketmq/exporter/model/metrics/ConsumerCountMetric.java
@@ -0,0 +1,58 @@
+package org.apache.rocketmq.exporter.model.metrics;
+
+public class ConsumerCountMetric {
+ private String caddr;
+ private String localaddr;
+ private String group;
+
+ public ConsumerCountMetric(String group, String caddr, String localaddr) {
+ this.group = group;
+ this.caddr = caddr;
+ this.localaddr = localaddr;
+ }
+
+ public String getCaddr() {
+ return caddr;
+ }
+
+ public void setCaddr(String caddr) {
+ this.caddr = caddr;
+ }
+
+ public String getLocaladdr() {
+ return localaddr;
+ }
+
+ public void setLocaladdr(String localaddr) {
+ this.localaddr = localaddr;
+ }
+
+ public String getGroup() {
+ return group;
+ }
+
+ public void setGroup(String group) {
+ this.group = group;
+ }
+
+ @Override
+ public boolean equals(Object obj) {
+ if (!(obj instanceof ConsumerCountMetric)) {
+ return false;
+ }
+ ConsumerCountMetric other = (ConsumerCountMetric) obj;
+ return other.group.equals(group);
+ }
+
+ @Override
+ public int hashCode() {
+ int hash = 1;
+ hash = 37 * hash + group.hashCode();
+ return hash;
+ }
+
+ @Override
+ public String toString() {
+ return "group: " + group + " caddr: " + caddr + " localaddr: " + localaddr;
+ }
+}
diff --git a/src/main/java/org/apache/rocketmq/exporter/model/metrics/ConsumerMetric.java b/src/main/java/org/apache/rocketmq/exporter/model/metrics/ConsumerMetric.java
index 9530fff..a0036a2 100644
--- a/src/main/java/org/apache/rocketmq/exporter/model/metrics/ConsumerMetric.java
+++ b/src/main/java/org/apache/rocketmq/exporter/model/metrics/ConsumerMetric.java
@@ -17,32 +17,22 @@
package org.apache.rocketmq.exporter.model.metrics;
public class ConsumerMetric {
+ private String topicName;
+ private String consumerGroupName;
- private String clusterName;
- private String brokerName;
- private String topicName;
- private String consumerGroupName;
-
- public void setClusterName(String cluster) {
- clusterName = cluster;
- }
- public String getClusterName() {
- return clusterName;
- }
- void setBrokerName(String broker) {
- brokerName = broker;
- }
-
- public String getBrokerName() {
- return brokerName;
+ public ConsumerMetric(String topicName, String consumerGroupName) {
+ this.topicName = topicName;
+ this.consumerGroupName = consumerGroupName;
}
- public void setTopicName(String topic) {
- topicName = topic;
- }
public String getTopicName() {
return topicName;
}
+
+ public void setTopicName(String topicName) {
+ this.topicName = topicName;
+ }
+
public String getConsumerGroupName() {
return consumerGroupName;
}
@@ -51,13 +41,6 @@ public class ConsumerMetric {
this.consumerGroupName = consumerGroupName;
}
- public ConsumerMetric(String cluster, String broker, String topic,String consumerGroup) {
- clusterName = cluster;
- brokerName = broker;
- topicName = topic;
- consumerGroupName = consumerGroup;
- }
-
@Override
public boolean equals(Object obj) {
if (!(obj instanceof ConsumerMetric)) {
@@ -65,15 +48,13 @@ public class ConsumerMetric {
}
ConsumerMetric other = (ConsumerMetric) obj;
- return other.clusterName.equals(clusterName) && other.brokerName.equals(brokerName)
- && other.topicName.equals(topicName) && other.consumerGroupName.equals(consumerGroupName);
+ return other.topicName.equals(topicName) &&
+ other.consumerGroupName.equals(consumerGroupName);
}
@Override
public int hashCode() {
int hash = 1;
- hash = 37 * hash + clusterName.hashCode();
- hash = 37 * hash + brokerName.hashCode();
hash = 37 * hash + topicName.hashCode();
hash = 37 * hash + consumerGroupName.hashCode();
return hash;
@@ -81,6 +62,6 @@ public class ConsumerMetric {
@Override
public String toString() {
- return "ClusterName: " + clusterName + " BrokerName: " + brokerName + " topicName: " + topicName + " ConsumeGroupName: " + consumerGroupName;
+ return "topicName: " + topicName + " ConsumeGroupName: " + consumerGroupName;
}
}
diff --git a/src/main/java/org/apache/rocketmq/exporter/model/metrics/ConsumerQueueMetric.java b/src/main/java/org/apache/rocketmq/exporter/model/metrics/ConsumerQueueMetric.java
deleted file mode 100644
index a6453fc..0000000
--- a/src/main/java/org/apache/rocketmq/exporter/model/metrics/ConsumerQueueMetric.java
+++ /dev/null
@@ -1,93 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-package org.apache.rocketmq.exporter.model.metrics;
-
-public class ConsumerQueueMetric {
-
- private String clusterName;
- private String brokerName;
- private String topicName;
- private String consumerGroupName;
- private String queueId;
-
- public void setClusterName(String cluster) {
- clusterName = cluster;
- }
- public String getClusterName() {
- return clusterName;
- }
- void setBrokerName(String broker) {
- brokerName = broker;
- }
- public String getBrokerName() {
- return brokerName;
- }
- public void setTopicName(String topic) {
- topicName = topic;
- }
- public String getTopicName() {
- return topicName;
- }
- public String getConsumerGroupName() {
- return consumerGroupName;
- }
-
- public void setConsumerGroupName(String consumerGroupName) {
- this.consumerGroupName = consumerGroupName;
- }
- public String getQueueId() {
- return queueId;
- }
- public void setQueueId(String queueId) {
- this.queueId = queueId;
- }
- public ConsumerQueueMetric(String cluster, String broker, String topic, String consumerGroup,String queue) {
- clusterName = cluster;
- brokerName = broker;
- topicName = topic;
- consumerGroupName = consumerGroup;
- queueId = queue;
- }
-
- @Override
- public boolean equals(Object obj) {
- if (!(obj instanceof ConsumerQueueMetric)) {
- return false;
- }
- ConsumerQueueMetric other = (ConsumerQueueMetric) obj;
-
- return other.clusterName.equals(clusterName) && other.brokerName.equals(brokerName)
- && other.topicName.equals(topicName) && other.consumerGroupName.equals(consumerGroupName)
- && other.queueId.equals(queueId);
- }
-
- @Override
- public int hashCode() {
- int hash = 1;
- hash = 37 * hash + clusterName.hashCode();
- hash = 37 * hash + brokerName.hashCode();
- hash = 37 * hash + topicName.hashCode();
- hash = 37 * hash + consumerGroupName.hashCode();
- hash = 37 * hash + queueId.hashCode();
- return hash;
- }
-
- @Override
- public String toString() {
- return "ClusterName: " + clusterName + " BrokerName: " + brokerName + " topicName: " + topicName + " ConsumeGroupName: " + consumerGroupName + "queueId: " + queueId;
- }
-}
diff --git a/src/main/java/org/apache/rocketmq/exporter/model/metrics/ConsumerTopicDiffMetric.java b/src/main/java/org/apache/rocketmq/exporter/model/metrics/ConsumerTopicDiffMetric.java
new file mode 100644
index 0000000..84b15c1
--- /dev/null
+++ b/src/main/java/org/apache/rocketmq/exporter/model/metrics/ConsumerTopicDiffMetric.java
@@ -0,0 +1,61 @@
+package org.apache.rocketmq.exporter.model.metrics;
+
+public class ConsumerTopicDiffMetric {
+ public ConsumerTopicDiffMetric(String group, String topic, String countOfOnlineConsumers) {
+ this.group = group;
+ this.topic = topic;
+ this.countOfOnlineConsumers = countOfOnlineConsumers;
+ }
+
+ private String group;
+ private String topic;
+ private String countOfOnlineConsumers;
+
+ public String getGroup() {
+ return group;
+ }
+
+ public void setGroup(String group) {
+ this.group = group;
+ }
+
+ public String getTopic() {
+ return topic;
+ }
+
+ public void setTopic(String topic) {
+ this.topic = topic;
+ }
+
+ public String getCountOfOnlineConsumers() {
+ return countOfOnlineConsumers;
+ }
+
+ public void setCountOfOnlineConsumers(String countOfOnlineConsumers) {
+ this.countOfOnlineConsumers = countOfOnlineConsumers;
+ }
+
+ @Override
+ public boolean equals(Object obj) {
+ if (!(obj instanceof ConsumerTopicDiffMetric)) {
+ return false;
+ }
+ ConsumerTopicDiffMetric other = (ConsumerTopicDiffMetric) obj;
+
+ return other.group.equals(group) &&
+ other.topic.equals(topic);
+ }
+
+ @Override
+ public int hashCode() {
+ int hash = 1;
+ hash = 37 * hash + group.hashCode();
+ hash = 37 * hash + topic.hashCode();
+ return hash;
+ }
+
+ @Override
+ public String toString() {
+ return "ConsumerGroup: " + group + " Topic: " + topic;
+ }
+}
diff --git a/src/main/java/org/apache/rocketmq/exporter/model/metrics/DLQTopicOffsetMetric.java b/src/main/java/org/apache/rocketmq/exporter/model/metrics/DLQTopicOffsetMetric.java
new file mode 100644
index 0000000..da530de
--- /dev/null
+++ b/src/main/java/org/apache/rocketmq/exporter/model/metrics/DLQTopicOffsetMetric.java
@@ -0,0 +1,72 @@
+package org.apache.rocketmq.exporter.model.metrics;
+
+public class DLQTopicOffsetMetric {
+ private String clusterName;
+ private String brokerNames;
+ private String group;
+ private long lastUpdateTimestamp;
+
+ public DLQTopicOffsetMetric(String clusterName, String brokerNames, String group, long lastUpdateTimestamp) {
+ this.clusterName = clusterName;
+ this.brokerNames = brokerNames;
+ this.group = group;
+ this.lastUpdateTimestamp = lastUpdateTimestamp;
+ }
+
+ public String getClusterName() {
+ return clusterName;
+ }
+
+ public void setClusterName(String clusterName) {
+ this.clusterName = clusterName;
+ }
+
+ public String getBrokerNames() {
+ return brokerNames;
+ }
+
+ public void setBrokerNames(String brokerNames) {
+ this.brokerNames = brokerNames;
+ }
+
+ public String getGroup() {
+ return group;
+ }
+
+ public void setGroup(String group) {
+ this.group = group;
+ }
+
+ public long getLastUpdateTimestamp() {
+ return lastUpdateTimestamp;
+ }
+
+ public void setLastUpdateTimestamp(long lastUpdateTimestamp) {
+ this.lastUpdateTimestamp = lastUpdateTimestamp;
+ }
+
+ @Override
+ public boolean equals(Object obj) {
+ if (!(obj instanceof DLQTopicOffsetMetric)) {
+ return false;
+ }
+ DLQTopicOffsetMetric other = (DLQTopicOffsetMetric) obj;
+
+ return other.clusterName.equals(clusterName) &&
+ other.group.equals(group);
+
+ }
+
+ @Override
+ public int hashCode() {
+ int hash = 1;
+ hash = 37 * hash + clusterName.hashCode();
+ hash = 37 * hash + group.hashCode();
+ return hash;
+ }
+
+ @Override
+ public String toString() {
+ return "ClusterName: " + clusterName + " BrokerNames: " + brokerNames + " group: " + group;
+ }
+}
diff --git a/src/main/java/org/apache/rocketmq/exporter/model/metrics/ProducerMetric.java b/src/main/java/org/apache/rocketmq/exporter/model/metrics/ProducerMetric.java
index 72baa73..fb9c5a7 100644
--- a/src/main/java/org/apache/rocketmq/exporter/model/metrics/ProducerMetric.java
+++ b/src/main/java/org/apache/rocketmq/exporter/model/metrics/ProducerMetric.java
@@ -16,34 +16,50 @@
*/
package org.apache.rocketmq.exporter.model.metrics;
+//每个topic最大位点
public class ProducerMetric {
+ private String clusterName;
+ private String brokerNames;
+ private String topicName;
+ private long lastUpdateTimestamp;
- private String clusterName;
- private String brokerName;
- private String topicName;
-
- public void setClusterName(String cluster) {
- clusterName = cluster;
- }
- public String getClusterName() {
+ public String getClusterName() {
return clusterName;
}
- void setBrokerName(String broker) {
- brokerName = broker;
+
+ public void setClusterName(String clusterName) {
+ this.clusterName = clusterName;
}
- public String getBrokerName() {
- return brokerName;
+
+ public String getBrokerNames() {
+ return brokerNames;
}
- public void setTopicName(String topic) {
- topicName = topic;
+
+ public void setBrokerNames(String brokerNames) {
+ this.brokerNames = brokerNames;
}
- public String getTopicName() {
+
+ public String getTopicName() {
return topicName;
}
- public ProducerMetric(String cluster,String broker,String topic) {
- clusterName = cluster;
- brokerName = broker;
- topicName = topic;
+
+ public void setTopicName(String topicName) {
+ this.topicName = topicName;
+ }
+
+ public long getLastUpdateTimestamp() {
+ return lastUpdateTimestamp;
+ }
+
+ public void setLastUpdateTimestamp(long lastUpdateTimestamp) {
+ this.lastUpdateTimestamp = lastUpdateTimestamp;
+ }
+
+ public ProducerMetric(String clusterName, String brokerNames, String topicName, long lastUpdateTimestamp) {
+ this.clusterName = clusterName;
+ this.brokerNames = brokerNames;
+ this.topicName = topicName;
+ this.lastUpdateTimestamp = lastUpdateTimestamp;
}
@Override
@@ -53,21 +69,20 @@ public class ProducerMetric {
}
ProducerMetric other = (ProducerMetric) obj;
- return other.clusterName.equals(clusterName) && other.brokerName.equals(brokerName)
- && other.topicName.equals(topicName);
+ return other.clusterName.equals(clusterName) &&
+ other.topicName.equals(topicName);
}
@Override
public int hashCode() {
int hash = 1;
hash = 37 * hash + clusterName.hashCode();
- hash = 37 * hash + brokerName.hashCode();
hash = 37 * hash + topicName.hashCode();
return hash;
}
@Override
public String toString() {
- return "ClusterName: " + clusterName + " BrokerName: " + brokerName + " topicName: " + topicName;
+ return "ClusterName: " + clusterName + " BrokerNames: " + brokerNames + " topicName: " + topicName;
}
}
diff --git a/src/main/java/org/apache/rocketmq/exporter/model/metrics/TopicPutNumMetric.java b/src/main/java/org/apache/rocketmq/exporter/model/metrics/TopicPutNumMetric.java
new file mode 100644
index 0000000..37f4e86
--- /dev/null
+++ b/src/main/java/org/apache/rocketmq/exporter/model/metrics/TopicPutNumMetric.java
@@ -0,0 +1,83 @@
+package org.apache.rocketmq.exporter.model.metrics;
+
+public class TopicPutNumMetric {
+ private String clusterName;
+ private String brokerNames;
+ private String brokerIP;
+ private String brokerHost;
+ private String topicName;
+
+ public TopicPutNumMetric(String clusterName, String brokerNames, String brokerIP, String brokerHost, String topicName) {
+ this.clusterName = clusterName;
+ this.brokerNames = brokerNames;
+ this.brokerIP = brokerIP;
+ this.brokerHost = brokerHost;
+ this.topicName = topicName;
+ }
+
+ public String getClusterName() {
+ return clusterName;
+ }
+
+ public void setClusterName(String clusterName) {
+ this.clusterName = clusterName;
+ }
+
+ public String getBrokerNames() {
+ return brokerNames;
+ }
+
+ public void setBrokerNames(String brokerNames) {
+ this.brokerNames = brokerNames;
+ }
+
+ public String getTopicName() {
+ return topicName;
+ }
+
+ public void setTopicName(String topicName) {
+ this.topicName = topicName;
+ }
+
+ public String getBrokerIP() {
+ return brokerIP;
+ }
+
+ public void setBrokerIP(String brokerIP) {
+ this.brokerIP = brokerIP;
+ }
+
+ public String getBrokerHost() {
+ return brokerHost;
+ }
+
+ public void setBrokerHost(String brokerHost) {
+ this.brokerHost = brokerHost;
+ }
+
+ @Override
+ public boolean equals(Object obj) {
+ if (!(obj instanceof TopicPutNumMetric)) {
+ return false;
+ }
+ TopicPutNumMetric other = (TopicPutNumMetric) obj;
+
+ return other.clusterName.equals(clusterName) &&
+ other.brokerIP.equals(brokerIP) &&
+ other.topicName.equals(topicName);
+ }
+
+ @Override
+ public int hashCode() {
+ int hash = 1;
+ hash = 37 * hash + clusterName.hashCode();
+ hash = 37 * hash + topicName.hashCode();
+ hash = 37 * hash + brokerIP.hashCode();
+ return hash;
+ }
+
+ @Override
+ public String toString() {
+ return "ClusterName: " + clusterName + " brokerIP: " + brokerIP + " topicName: " + topicName;
+ }
+}
diff --git a/src/main/java/org/apache/rocketmq/exporter/model/metrics/brokerruntime/BrokerRuntimeMetric.java b/src/main/java/org/apache/rocketmq/exporter/model/metrics/brokerruntime/BrokerRuntimeMetric.java
new file mode 100644
index 0000000..beb583e
--- /dev/null
+++ b/src/main/java/org/apache/rocketmq/exporter/model/metrics/brokerruntime/BrokerRuntimeMetric.java
@@ -0,0 +1,91 @@
+package org.apache.rocketmq.exporter.model.metrics.brokerruntime;
+
+public class BrokerRuntimeMetric {
+ private String clusterName;
+ private String brokerAddress;
+ private String brokerHost;
+ private String brokerDes;
+ private long bootTimestamp;
+ private int brokerVersion;
+
+ public BrokerRuntimeMetric(String clusterName, String brokerAddress, String brokerHost, String brokerDes, long bootTimestamp, int brokerVersion) {
+ this.clusterName = clusterName;
+ this.brokerAddress = brokerAddress;
+ this.brokerHost = brokerHost;
+ this.brokerDes = brokerDes;
+ this.bootTimestamp = bootTimestamp;
+ this.brokerVersion = brokerVersion;
+ }
+
+ public String getClusterName() {
+ return clusterName;
+ }
+
+ public void setClusterName(String clusterName) {
+ this.clusterName = clusterName;
+ }
+
+ public String getBrokerAddress() {
+ return brokerAddress;
+ }
+
+ public void setBrokerAddress(String brokerAddress) {
+ this.brokerAddress = brokerAddress;
+ }
+
+ public String getBrokerHost() {
+ return brokerHost;
+ }
+
+ public void setBrokerHost(String brokerHost) {
+ this.brokerHost = brokerHost;
+ }
+
+ public String getBrokerDes() {
+ return brokerDes;
+ }
+
+ public void setBrokerDes(String brokerDes) {
+ this.brokerDes = brokerDes;
+ }
+
+ public long getBootTimestamp() {
+ return bootTimestamp;
+ }
+
+ public void setBootTimestamp(long bootTimestamp) {
+ this.bootTimestamp = bootTimestamp;
+ }
+
+ public int getBrokerVersion() {
+ return brokerVersion;
+ }
+
+ public void setBrokerVersion(int brokerVersion) {
+ this.brokerVersion = brokerVersion;
+ }
+
+ @Override
+ public boolean equals(Object obj) {
+ if (!(obj instanceof BrokerRuntimeMetric)) {
+ return false;
+ }
+ BrokerRuntimeMetric other = (BrokerRuntimeMetric) obj;
+
+ return other.clusterName.equals(clusterName) &&
+ other.brokerAddress.equals(brokerAddress);
+ }
+
+ @Override
+ public int hashCode() {
+ int hash = 1;
+ hash = 37 * hash + clusterName.hashCode();
+ hash = 37 * hash + brokerAddress.hashCode();
+ return hash;
+ }
+
+ @Override
+ public String toString() {
+ return "ClusterName: " + clusterName + " brokerAddress: " + brokerAddress + " brokerHost: " + brokerHost;
+ }
+}
diff --git a/src/main/java/org/apache/rocketmq/exporter/service/AbstractCommonService.java b/src/main/java/org/apache/rocketmq/exporter/service/AbstractCommonService.java
deleted file mode 100644
index 50f5b0a..0000000
--- a/src/main/java/org/apache/rocketmq/exporter/service/AbstractCommonService.java
+++ /dev/null
@@ -1,50 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-package org.apache.rocketmq.exporter.service;
-
-import com.google.common.base.Throwables;
-import com.google.common.collect.Sets;
-import org.apache.commons.collections.CollectionUtils;
-import org.apache.rocketmq.tools.admin.MQAdminExt;
-
-import javax.annotation.Resource;
-import java.util.HashMap;
-import java.util.List;
-import java.util.Set;
-
-public abstract class AbstractCommonService {
- @Resource
- protected MQAdminExt mqAdminExt;
- protected final Set<String> changeToBrokerNameSet(HashMap<String, Set<String>> clusterAddrTable,
- List<String> clusterNameList, List<String> brokerNameList) {
- Set<String> finalBrokerNameList = Sets.newHashSet();
- if (CollectionUtils.isNotEmpty(clusterNameList)) {
- try {
- for (String clusterName : clusterNameList) {
- finalBrokerNameList.addAll(clusterAddrTable.get(clusterName));
- }
- }
- catch (Exception e) {
- throw Throwables.propagate(e);
- }
- }
- if (CollectionUtils.isNotEmpty(brokerNameList)) {
- finalBrokerNameList.addAll(brokerNameList);
- }
- return finalBrokerNameList;
- }
-}
diff --git a/src/main/java/org/apache/rocketmq/exporter/service/RMQMetricsService.java b/src/main/java/org/apache/rocketmq/exporter/service/RMQMetricsService.java
index c1f8802..5e2c0d4 100644
--- a/src/main/java/org/apache/rocketmq/exporter/service/RMQMetricsService.java
+++ b/src/main/java/org/apache/rocketmq/exporter/service/RMQMetricsService.java
@@ -18,12 +18,12 @@ package org.apache.rocketmq.exporter.service;
import org.apache.rocketmq.exporter.collector.RMQMetricsCollector;
-
import java.io.IOException;
import java.io.StringWriter;
-public interface RMQMetricsService {
+public interface RMQMetricsService {
RMQMetricsCollector getCollector();
- void Metrics(StringWriter writer) throws IOException;
+
+ void metrics(StringWriter writer) throws IOException;
}
diff --git a/src/main/java/org/apache/rocketmq/exporter/service/client/MQAdminExtImpl.java b/src/main/java/org/apache/rocketmq/exporter/service/client/MQAdminExtImpl.java
index 17ec7d5..f24490c 100644
--- a/src/main/java/org/apache/rocketmq/exporter/service/client/MQAdminExtImpl.java
+++ b/src/main/java/org/apache/rocketmq/exporter/service/client/MQAdminExtImpl.java
@@ -18,10 +18,12 @@ package org.apache.rocketmq.exporter.service.client;
import com.google.common.base.Throwables;
import org.apache.rocketmq.client.QueryResult;
+import org.apache.rocketmq.client.consumer.DefaultMQPullConsumer;
import org.apache.rocketmq.client.consumer.PullResult;
import org.apache.rocketmq.client.exception.MQBrokerException;
import org.apache.rocketmq.client.exception.MQClientException;
import org.apache.rocketmq.client.impl.MQAdminImpl;
+import org.apache.rocketmq.client.impl.factory.MQClientInstance;
import org.apache.rocketmq.common.TopicConfig;
import org.apache.rocketmq.common.admin.ConsumeStats;
import org.apache.rocketmq.common.admin.RollbackStats;
@@ -55,11 +57,14 @@ import org.apache.rocketmq.remoting.exception.RemotingException;
import org.apache.rocketmq.remoting.exception.RemotingSendRequestException;
import org.apache.rocketmq.remoting.exception.RemotingTimeoutException;
import org.apache.rocketmq.remoting.protocol.RemotingCommand;
+import org.apache.rocketmq.tools.admin.DefaultMQAdminExt;
import org.apache.rocketmq.tools.admin.MQAdminExt;
import org.apache.rocketmq.tools.admin.api.MessageTrack;
import org.joor.Reflect;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
+import org.springframework.beans.factory.annotation.Autowired;
+import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.stereotype.Service;
import java.io.UnsupportedEncodingException;
@@ -70,8 +75,21 @@ import java.util.Set;
import static org.apache.rocketmq.remoting.protocol.RemotingSerializable.decode;
-@Service
+@Service("mqAdminExtImpl")
public class MQAdminExtImpl implements MQAdminExt {
+ @Autowired
+ @Qualifier("defaultMQAdminExt")
+ private DefaultMQAdminExt defaultMQAdminExt;
+
+ @Autowired
+ private DefaultMQPullConsumer pullConsumer;
+
+ @Autowired
+ private RemotingClient remotingClient;
+
+ @Autowired
+ private MQClientInstance mqClientInstance;
+
private Logger logger = LoggerFactory.getLogger(MQAdminExtImpl.class);
public MQAdminExtImpl() {
@@ -79,37 +97,35 @@ public class MQAdminExtImpl implements MQAdminExt {
public PullResult queryMsgByOffset(MessageQueue mq, long offset) throws Exception {
- return MQAdminInstance.threadLocalMQPullConsumer().pull(mq, "*", offset, 1);
+ return pullConsumer.pull(mq, "*", offset, 1);
}
@Override
public void updateBrokerConfig(String brokerAddr, Properties properties)
- throws RemotingConnectException, RemotingSendRequestException, RemotingTimeoutException,
- UnsupportedEncodingException, InterruptedException, MQBrokerException {
- MQAdminInstance.threadLocalMQAdminExt().updateBrokerConfig(brokerAddr, properties);
+ throws RemotingConnectException, RemotingSendRequestException, RemotingTimeoutException,
+ UnsupportedEncodingException, InterruptedException, MQBrokerException {
+ defaultMQAdminExt.updateBrokerConfig(brokerAddr, properties);
}
@Override
public void createAndUpdateTopicConfig(String addr, TopicConfig config)
- throws RemotingException, MQBrokerException, InterruptedException, MQClientException {
- MQAdminInstance.threadLocalMQAdminExt().createAndUpdateTopicConfig(addr, config);
+ throws RemotingException, MQBrokerException, InterruptedException, MQClientException {
+ defaultMQAdminExt.createAndUpdateTopicConfig(addr, config);
}
@Override
public void createAndUpdateSubscriptionGroupConfig(String addr, SubscriptionGroupConfig config)
- throws RemotingException, MQBrokerException, InterruptedException, MQClientException {
- MQAdminInstance.threadLocalMQAdminExt().createAndUpdateSubscriptionGroupConfig(addr, config);
+ throws RemotingException, MQBrokerException, InterruptedException, MQClientException {
+ defaultMQAdminExt.createAndUpdateSubscriptionGroupConfig(addr, config);
}
@Override
public SubscriptionGroupConfig examineSubscriptionGroupConfig(String addr, String group) {
- RemotingClient remotingClient = MQAdminInstance.threadLocalRemotingClient();
RemotingCommand request = RemotingCommand.createRequestCommand(RequestCode.GET_ALL_SUBSCRIPTIONGROUP_CONFIG, null);
RemotingCommand response = null;
try {
response = remotingClient.invokeSync(addr, request, 3000);
- }
- catch (Exception err) {
+ } catch (Exception err) {
throw Throwables.propagate(err);
}
assert response != null;
@@ -125,13 +141,11 @@ public class MQAdminExtImpl implements MQAdminExt {
@Override
public TopicConfig examineTopicConfig(String addr, String topic) {
- RemotingClient remotingClient = MQAdminInstance.threadLocalRemotingClient();
RemotingCommand request = RemotingCommand.createRequestCommand(RequestCode.GET_ALL_TOPIC_CONFIG, null);
RemotingCommand response = null;
try {
response = remotingClient.invokeSync(addr, request, 3000);
- }
- catch (Exception err) {
+ } catch (Exception err) {
throw Throwables.propagate(err);
}
switch (response.getCode()) {
@@ -146,238 +160,238 @@ public class MQAdminExtImpl implements MQAdminExt {
@Override
public TopicStatsTable examineTopicStats(String topic)
- throws RemotingException, MQClientException, InterruptedException, MQBrokerException {
- return MQAdminInstance.threadLocalMQAdminExt().examineTopicStats(topic);
+ throws RemotingException, MQClientException, InterruptedException, MQBrokerException {
+ return defaultMQAdminExt.examineTopicStats(topic);
}
@Override
public TopicList fetchAllTopicList() throws RemotingException, MQClientException, InterruptedException {
- TopicList topicList = MQAdminInstance.threadLocalMQAdminExt().fetchAllTopicList();
+ TopicList topicList = defaultMQAdminExt.fetchAllTopicList();
logger.debug("op=look={}", JsonUtil.obj2String(topicList.getTopicList()));
return topicList;
}
@Override
public KVTable fetchBrokerRuntimeStats(String brokerAddr)
- throws RemotingConnectException, RemotingSendRequestException, RemotingTimeoutException,
- InterruptedException, MQBrokerException {
- return MQAdminInstance.threadLocalMQAdminExt().fetchBrokerRuntimeStats(brokerAddr);
+ throws RemotingConnectException, RemotingSendRequestException, RemotingTimeoutException,
+ InterruptedException, MQBrokerException {
+ return defaultMQAdminExt.fetchBrokerRuntimeStats(brokerAddr);
}
@Override
public ConsumeStats examineConsumeStats(String consumerGroup)
- throws RemotingException, MQClientException, InterruptedException, MQBrokerException {
- return MQAdminInstance.threadLocalMQAdminExt().examineConsumeStats(consumerGroup);
+ throws RemotingException, MQClientException, InterruptedException, MQBrokerException {
+ return defaultMQAdminExt.examineConsumeStats(consumerGroup);
}
@Override
public ConsumeStats examineConsumeStats(String consumerGroup, String topic)
- throws RemotingException, MQClientException, InterruptedException, MQBrokerException {
- return MQAdminInstance.threadLocalMQAdminExt().examineConsumeStats(consumerGroup, topic);
+ throws RemotingException, MQClientException, InterruptedException, MQBrokerException {
+ return defaultMQAdminExt.examineConsumeStats(consumerGroup, topic);
}
@Override
public ClusterInfo examineBrokerClusterInfo()
- throws InterruptedException, MQBrokerException, RemotingTimeoutException, RemotingSendRequestException,
- RemotingConnectException {
- return MQAdminInstance.threadLocalMQAdminExt().examineBrokerClusterInfo();
+ throws InterruptedException, MQBrokerException, RemotingTimeoutException, RemotingSendRequestException,
+ RemotingConnectException {
+ return defaultMQAdminExt.examineBrokerClusterInfo();
}
@Override
public TopicRouteData examineTopicRouteInfo(String topic)
- throws RemotingException, MQClientException, InterruptedException {
- return MQAdminInstance.threadLocalMQAdminExt().examineTopicRouteInfo(topic);
+ throws RemotingException, MQClientException, InterruptedException {
+ return defaultMQAdminExt.examineTopicRouteInfo(topic);
}
@Override
public ConsumerConnection examineConsumerConnectionInfo(String consumerGroup)
- throws RemotingConnectException, RemotingSendRequestException, RemotingTimeoutException,
- InterruptedException, MQBrokerException, RemotingException, MQClientException {
- return MQAdminInstance.threadLocalMQAdminExt().examineConsumerConnectionInfo(consumerGroup);
+ throws
+ InterruptedException, MQBrokerException, RemotingException, MQClientException {
+ return defaultMQAdminExt.examineConsumerConnectionInfo(consumerGroup);
}
@Override
public ProducerConnection examineProducerConnectionInfo(String producerGroup, String topic)
- throws RemotingException, MQClientException, InterruptedException, MQBrokerException {
- return MQAdminInstance.threadLocalMQAdminExt().examineProducerConnectionInfo(producerGroup, topic);
+ throws RemotingException, MQClientException, InterruptedException, MQBrokerException {
+ return defaultMQAdminExt.examineProducerConnectionInfo(producerGroup, topic);
}
@Override
public List<String> getNameServerAddressList() {
- return MQAdminInstance.threadLocalMQAdminExt().getNameServerAddressList();
+ return defaultMQAdminExt.getNameServerAddressList();
}
@Override
public int wipeWritePermOfBroker(String namesrvAddr, String brokerName)
- throws RemotingCommandException, RemotingConnectException, RemotingSendRequestException,
- RemotingTimeoutException, InterruptedException, MQClientException {
- return MQAdminInstance.threadLocalMQAdminExt().wipeWritePermOfBroker(namesrvAddr, brokerName);
+ throws RemotingCommandException, RemotingConnectException, RemotingSendRequestException,
+ RemotingTimeoutException, InterruptedException, MQClientException {
+ return defaultMQAdminExt.wipeWritePermOfBroker(namesrvAddr, brokerName);
}
@Override
public void putKVConfig(String namespace, String key, String value) {
- MQAdminInstance.threadLocalMQAdminExt().putKVConfig(namespace, key, value);
+ defaultMQAdminExt.putKVConfig(namespace, key, value);
}
@Override
public String getKVConfig(String namespace, String key)
- throws RemotingException, MQClientException, InterruptedException {
- return MQAdminInstance.threadLocalMQAdminExt().getKVConfig(namespace, key);
+ throws RemotingException, MQClientException, InterruptedException {
+ return defaultMQAdminExt.getKVConfig(namespace, key);
}
@Override
public KVTable getKVListByNamespace(String namespace)
- throws RemotingException, MQClientException, InterruptedException {
- return MQAdminInstance.threadLocalMQAdminExt().getKVListByNamespace(namespace);
+ throws RemotingException, MQClientException, InterruptedException {
+ return defaultMQAdminExt.getKVListByNamespace(namespace);
}
@Override
public void deleteTopicInBroker(Set<String> addrs, String topic)
- throws RemotingException, MQBrokerException, InterruptedException, MQClientException {
+ throws RemotingException, MQBrokerException, InterruptedException, MQClientException {
logger.info("addrs={} topic={}", JsonUtil.obj2String(addrs), topic);
- MQAdminInstance.threadLocalMQAdminExt().deleteTopicInBroker(addrs, topic);
+ defaultMQAdminExt.deleteTopicInBroker(addrs, topic);
}
@Override
public void deleteTopicInNameServer(Set<String> addrs, String topic)
- throws RemotingException, MQBrokerException, InterruptedException, MQClientException {
- MQAdminInstance.threadLocalMQAdminExt().deleteTopicInNameServer(addrs, topic);
+ throws RemotingException, MQBrokerException, InterruptedException, MQClientException {
+ defaultMQAdminExt.deleteTopicInNameServer(addrs, topic);
}
@Override
public void deleteSubscriptionGroup(String addr, String groupName)
- throws RemotingException, MQBrokerException, InterruptedException, MQClientException {
- MQAdminInstance.threadLocalMQAdminExt().deleteSubscriptionGroup(addr, groupName);
+ throws RemotingException, MQBrokerException, InterruptedException, MQClientException {
+ defaultMQAdminExt.deleteSubscriptionGroup(addr, groupName);
}
@Override
public void createAndUpdateKvConfig(String namespace, String key, String value)
- throws RemotingException, MQBrokerException, InterruptedException, MQClientException {
- MQAdminInstance.threadLocalMQAdminExt().createAndUpdateKvConfig(namespace, key, value);
+ throws RemotingException, MQBrokerException, InterruptedException, MQClientException {
+ defaultMQAdminExt.createAndUpdateKvConfig(namespace, key, value);
}
@Override
public void deleteKvConfig(String namespace, String key)
- throws RemotingException, MQBrokerException, InterruptedException, MQClientException {
- MQAdminInstance.threadLocalMQAdminExt().deleteKvConfig(namespace, key);
+ throws RemotingException, MQBrokerException, InterruptedException, MQClientException {
+ defaultMQAdminExt.deleteKvConfig(namespace, key);
}
@Override
public List<RollbackStats> resetOffsetByTimestampOld(String consumerGroup, String topic, long timestamp,
- boolean force) throws RemotingException, MQBrokerException, InterruptedException, MQClientException {
- return MQAdminInstance.threadLocalMQAdminExt().resetOffsetByTimestampOld(consumerGroup, topic, timestamp, force);
+ boolean force) throws RemotingException, MQBrokerException, InterruptedException, MQClientException {
+ return defaultMQAdminExt.resetOffsetByTimestampOld(consumerGroup, topic, timestamp, force);
}
@Override
public Map<MessageQueue, Long> resetOffsetByTimestamp(String topic, String group, long timestamp,
- boolean isForce) throws RemotingException, MQBrokerException, InterruptedException, MQClientException {
- return MQAdminInstance.threadLocalMQAdminExt().resetOffsetByTimestamp(topic, group, timestamp, isForce);
+ boolean isForce) throws RemotingException, MQBrokerException, InterruptedException, MQClientException {
+ return defaultMQAdminExt.resetOffsetByTimestamp(topic, group, timestamp, isForce);
}
@Override
public void resetOffsetNew(String consumerGroup, String topic, long timestamp)
- throws RemotingException, MQBrokerException, InterruptedException, MQClientException {
- MQAdminInstance.threadLocalMQAdminExt().resetOffsetNew(consumerGroup, topic, timestamp);
+ throws RemotingException, MQBrokerException, InterruptedException, MQClientException {
+ defaultMQAdminExt.resetOffsetNew(consumerGroup, topic, timestamp);
}
@Override
public Map<String, Map<MessageQueue, Long>> getConsumeStatus(String topic, String group,
- String clientAddr) throws RemotingException, MQBrokerException, InterruptedException, MQClientException {
- return MQAdminInstance.threadLocalMQAdminExt().getConsumeStatus(topic, group, clientAddr);
+ String clientAddr) throws RemotingException, MQBrokerException, InterruptedException, MQClientException {
+ return defaultMQAdminExt.getConsumeStatus(topic, group, clientAddr);
}
@Override
public void createOrUpdateOrderConf(String key, String value, boolean isCluster)
- throws RemotingException, MQBrokerException, InterruptedException, MQClientException {
- MQAdminInstance.threadLocalMQAdminExt().createOrUpdateOrderConf(key, value, isCluster);
+ throws RemotingException, MQBrokerException, InterruptedException, MQClientException {
+ defaultMQAdminExt.createOrUpdateOrderConf(key, value, isCluster);
}
@Override
public GroupList queryTopicConsumeByWho(String topic)
- throws RemotingConnectException, RemotingSendRequestException, RemotingTimeoutException,
- InterruptedException, MQBrokerException, RemotingException, MQClientException {
- return MQAdminInstance.threadLocalMQAdminExt().queryTopicConsumeByWho(topic);
+ throws
+ InterruptedException, MQBrokerException, RemotingException, MQClientException {
+ return defaultMQAdminExt.queryTopicConsumeByWho(topic);
}
@Override
public boolean cleanExpiredConsumerQueue(String cluster)
- throws RemotingConnectException, RemotingSendRequestException, RemotingTimeoutException, MQClientException,
- InterruptedException {
- return MQAdminInstance.threadLocalMQAdminExt().cleanExpiredConsumerQueue(cluster);
+ throws RemotingConnectException, RemotingSendRequestException, RemotingTimeoutException, MQClientException,
+ InterruptedException {
+ return defaultMQAdminExt.cleanExpiredConsumerQueue(cluster);
}
@Override
public boolean cleanExpiredConsumerQueueByAddr(String addr)
- throws RemotingConnectException, RemotingSendRequestException, RemotingTimeoutException, MQClientException,
- InterruptedException {
- return MQAdminInstance.threadLocalMQAdminExt().cleanExpiredConsumerQueueByAddr(addr);
+ throws RemotingConnectException, RemotingSendRequestException, RemotingTimeoutException, MQClientException,
+ InterruptedException {
+ return defaultMQAdminExt.cleanExpiredConsumerQueueByAddr(addr);
}
@Override
public ConsumerRunningInfo getConsumerRunningInfo(String consumerGroup, String clientId, boolean jstack)
- throws RemotingException, MQClientException, InterruptedException {
- return MQAdminInstance.threadLocalMQAdminExt().getConsumerRunningInfo(consumerGroup, clientId, jstack);
+ throws RemotingException, MQClientException, InterruptedException {
+ return defaultMQAdminExt.getConsumerRunningInfo(consumerGroup, clientId, jstack);
}
@Override
public ConsumeMessageDirectlyResult consumeMessageDirectly(String consumerGroup, String clientId,
- String msgId) throws RemotingException, MQClientException, InterruptedException, MQBrokerException {
- return MQAdminInstance.threadLocalMQAdminExt().consumeMessageDirectly(consumerGroup, clientId, msgId);
+ String msgId) throws RemotingException, MQClientException, InterruptedException, MQBrokerException {
+ return defaultMQAdminExt.consumeMessageDirectly(consumerGroup, clientId, msgId);
}
@Override
public List<MessageTrack> messageTrackDetail(MessageExt msg)
- throws RemotingException, MQClientException, InterruptedException, MQBrokerException {
- return MQAdminInstance.threadLocalMQAdminExt().messageTrackDetail(msg);
+ throws RemotingException, MQClientException, InterruptedException, MQBrokerException {
+ return defaultMQAdminExt.messageTrackDetail(msg);
}
@Override
public void cloneGroupOffset(String srcGroup, String destGroup, String topic, boolean isOffline)
- throws RemotingException, MQClientException, InterruptedException, MQBrokerException {
- MQAdminInstance.threadLocalMQAdminExt().cloneGroupOffset(srcGroup, destGroup, topic, isOffline);
+ throws RemotingException, MQClientException, InterruptedException, MQBrokerException {
+ defaultMQAdminExt.cloneGroupOffset(srcGroup, destGroup, topic, isOffline);
}
@Override
public void createTopic(String key, String newTopic, int queueNum) throws MQClientException {
- MQAdminInstance.threadLocalMQAdminExt().createTopic(key, newTopic, queueNum);
+ defaultMQAdminExt.createTopic(key, newTopic, queueNum);
}
@Override
public void createTopic(String key, String newTopic, int queueNum, int topicSysFlag)
- throws MQClientException {
- MQAdminInstance.threadLocalMQAdminExt().createTopic(key, newTopic, queueNum, topicSysFlag);
+ throws MQClientException {
+ defaultMQAdminExt.createTopic(key, newTopic, queueNum, topicSysFlag);
}
@Override
public long searchOffset(MessageQueue mq, long timestamp) throws MQClientException {
- return MQAdminInstance.threadLocalMQAdminExt().searchOffset(mq, timestamp);
+ return defaultMQAdminExt.searchOffset(mq, timestamp);
}
@Override
public long maxOffset(MessageQueue mq) throws MQClientException {
- return MQAdminInstance.threadLocalMQAdminExt().maxOffset(mq);
+ return defaultMQAdminExt.maxOffset(mq);
}
@Override
public long minOffset(MessageQueue mq) throws MQClientException {
- return MQAdminInstance.threadLocalMQAdminExt().minOffset(mq);
+ return defaultMQAdminExt.minOffset(mq);
}
@Override
public long earliestMsgStoreTime(MessageQueue mq) throws MQClientException {
- return MQAdminInstance.threadLocalMQAdminExt().earliestMsgStoreTime(mq);
+ return defaultMQAdminExt.earliestMsgStoreTime(mq);
}
@Override
public MessageExt viewMessage(String msgId)
- throws RemotingException, MQBrokerException, InterruptedException, MQClientException {
- return MQAdminInstance.threadLocalMQAdminExt().viewMessage(msgId);
+ throws RemotingException, MQBrokerException, InterruptedException, MQClientException {
+ return defaultMQAdminExt.viewMessage(msgId);
}
@Override
public QueryResult queryMessage(String topic, String key, int maxNum, long begin, long end)
- throws MQClientException, InterruptedException {
- return MQAdminInstance.threadLocalMQAdminExt().queryMessage(topic, key, maxNum, begin, end);
+ throws MQClientException, InterruptedException {
+ return defaultMQAdminExt.queryMessage(topic, key, maxNum, begin, end);
}
@Override
@@ -396,8 +410,8 @@ public class MQAdminExtImpl implements MQAdminExt {
@Override
public List<QueueTimeSpan> queryConsumeTimeSpan(String topic,
- String group) throws InterruptedException, MQBrokerException, RemotingException, MQClientException {
- return MQAdminInstance.threadLocalMQAdminExt().queryConsumeTimeSpan(topic, group);
+ String group) throws InterruptedException, MQBrokerException, RemotingException, MQClientException {
+ return defaultMQAdminExt.queryConsumeTimeSpan(topic, group);
}
//MessageClientIDSetter.getNearlyTimeFromID has bug,so we subtract half a day
@@ -406,110 +420,111 @@ public class MQAdminExtImpl implements MQAdminExt {
//https://github.com/apache/incubator-rocketmq/pull/69
@Override
public MessageExt viewMessage(String topic,
- String msgId) throws RemotingException, MQBrokerException, InterruptedException, MQClientException {
+ String msgId) throws RemotingException, MQBrokerException, InterruptedException, MQClientException {
logger.info("MessageClientIDSetter.getNearlyTimeFromID(msgId)={} msgId={}", MessageClientIDSetter.getNearlyTimeFromID(msgId), msgId);
try {
return viewMessage(msgId);
+ } catch (Exception e) {
}
- catch (Exception e) {
- }
- MQAdminImpl mqAdminImpl = MQAdminInstance.threadLocalMqClientInstance().getMQAdminImpl();
+ MQAdminImpl mqAdminImpl = mqClientInstance.getMQAdminImpl();
QueryResult qr = Reflect.on(mqAdminImpl).call("queryMessage", topic, msgId, 32,
- MessageClientIDSetter.getNearlyTimeFromID(msgId).getTime() - 1000 * 60 * 60 * 13L, Long.MAX_VALUE, true).get();
+ MessageClientIDSetter.getNearlyTimeFromID(msgId).getTime() - 1000 * 60 * 60 * 13L, Long.MAX_VALUE, true).get();
if (qr != null && qr.getMessageList() != null && qr.getMessageList().size() > 0) {
return qr.getMessageList().get(0);
- }
- else {
+ } else {
return null;
}
}
@Override
public ConsumeMessageDirectlyResult consumeMessageDirectly(String consumerGroup, String clientId, String topic,
- String msgId) throws RemotingException, MQClientException, InterruptedException, MQBrokerException {
- return MQAdminInstance.threadLocalMQAdminExt().consumeMessageDirectly(consumerGroup, clientId, topic, msgId);
+ String msgId) throws RemotingException, MQClientException, InterruptedException, MQBrokerException {
+ return defaultMQAdminExt.consumeMessageDirectly(consumerGroup, clientId, topic, msgId);
}
@Override
public Properties getBrokerConfig(
- String brokerAddr) throws RemotingConnectException, RemotingSendRequestException, RemotingTimeoutException, UnsupportedEncodingException, InterruptedException, MQBrokerException {
- return MQAdminInstance.threadLocalMQAdminExt().getBrokerConfig(brokerAddr);
+ String brokerAddr) throws RemotingConnectException, RemotingSendRequestException, RemotingTimeoutException, UnsupportedEncodingException, InterruptedException, MQBrokerException {
+ return defaultMQAdminExt.getBrokerConfig(brokerAddr);
}
@Override
public TopicList fetchTopicsByCLuster(
- String clusterName) throws RemotingException, MQClientException, InterruptedException {
- return MQAdminInstance.threadLocalMQAdminExt().fetchTopicsByCLuster(clusterName);
+ String clusterName) throws RemotingException, MQClientException, InterruptedException {
+ return defaultMQAdminExt.fetchTopicsByCLuster(clusterName);
}
@Override
public boolean cleanUnusedTopic(
- String cluster) throws RemotingConnectException, RemotingSendRequestException, RemotingTimeoutException, MQClientException, InterruptedException {
- return MQAdminInstance.threadLocalMQAdminExt().cleanUnusedTopic(cluster);
+ String cluster) throws RemotingConnectException, RemotingSendRequestException, RemotingTimeoutException, MQClientException, InterruptedException {
+ return defaultMQAdminExt.cleanUnusedTopic(cluster);
}
@Override
public boolean cleanUnusedTopicByAddr(
- String addr) throws RemotingConnectException, RemotingSendRequestException, RemotingTimeoutException, MQClientException, InterruptedException {
- return MQAdminInstance.threadLocalMQAdminExt().cleanUnusedTopicByAddr(addr);
+ String addr) throws RemotingConnectException, RemotingSendRequestException, RemotingTimeoutException, MQClientException, InterruptedException {
+ return defaultMQAdminExt.cleanUnusedTopicByAddr(addr);
}
@Override
public BrokerStatsData viewBrokerStatsData(String brokerAddr, String statsName,
- String statsKey) throws RemotingConnectException, RemotingSendRequestException, RemotingTimeoutException, MQClientException, InterruptedException {
- return MQAdminInstance.threadLocalMQAdminExt().viewBrokerStatsData(brokerAddr, statsName, statsKey);
+ String statsKey) throws RemotingConnectException, RemotingSendRequestException, RemotingTimeoutException, MQClientException, InterruptedException {
+ return defaultMQAdminExt.viewBrokerStatsData(brokerAddr, statsName, statsKey);
}
@Override
public Set<String> getClusterList(
- String topic) throws RemotingConnectException, RemotingSendRequestException, RemotingTimeoutException, MQClientException, InterruptedException {
- return MQAdminInstance.threadLocalMQAdminExt().getClusterList(topic);
+ String topic) throws RemotingConnectException, RemotingSendRequestException, RemotingTimeoutException, MQClientException, InterruptedException {
+ return defaultMQAdminExt.getClusterList(topic);
}
@Override
public ConsumeStatsList fetchConsumeStatsInBroker(String brokerAddr, boolean isOrder,
- long timeoutMillis) throws RemotingConnectException, RemotingSendRequestException, RemotingTimeoutException, MQClientException, InterruptedException {
- return MQAdminInstance.threadLocalMQAdminExt().fetchConsumeStatsInBroker(brokerAddr, isOrder, timeoutMillis);
+ long timeoutMillis) throws RemotingConnectException, RemotingSendRequestException, RemotingTimeoutException, MQClientException, InterruptedException {
+ return defaultMQAdminExt.fetchConsumeStatsInBroker(brokerAddr, isOrder, timeoutMillis);
}
@Override
public Set<String> getTopicClusterList(
- String topic) throws InterruptedException, MQBrokerException, MQClientException, RemotingException {
- return MQAdminInstance.threadLocalMQAdminExt().getTopicClusterList(topic);
+ String topic) throws InterruptedException, MQBrokerException, MQClientException, RemotingException {
+ return defaultMQAdminExt.getTopicClusterList(topic);
}
@Override
public SubscriptionGroupWrapper getAllSubscriptionGroup(String brokerAddr,
- long timeoutMillis) throws InterruptedException, RemotingTimeoutException, RemotingSendRequestException, RemotingConnectException, MQBrokerException {
- return MQAdminInstance.threadLocalMQAdminExt().getAllSubscriptionGroup(brokerAddr, timeoutMillis);
+ long timeoutMillis) throws InterruptedException, RemotingTimeoutException, RemotingSendRequestException, RemotingConnectException, MQBrokerException {
+ return defaultMQAdminExt.getAllSubscriptionGroup(brokerAddr, timeoutMillis);
}
@Override
public TopicConfigSerializeWrapper getAllTopicGroup(String brokerAddr,
- long timeoutMillis) throws InterruptedException, RemotingTimeoutException, RemotingSendRequestException, RemotingConnectException, MQBrokerException {
- return MQAdminInstance.threadLocalMQAdminExt().getAllTopicGroup(brokerAddr, timeoutMillis);
+ long timeoutMillis) throws InterruptedException, RemotingTimeoutException, RemotingSendRequestException, RemotingConnectException, MQBrokerException {
+ return defaultMQAdminExt.getAllTopicGroup(brokerAddr, timeoutMillis);
}
@Override
public void updateConsumeOffset(String brokerAddr, String consumeGroup, MessageQueue mq,
- long offset) throws RemotingException, InterruptedException, MQBrokerException {
- MQAdminInstance.threadLocalMQAdminExt().updateConsumeOffset(brokerAddr, consumeGroup, mq, offset);
+ long offset) throws RemotingException, InterruptedException, MQBrokerException {
+ defaultMQAdminExt.updateConsumeOffset(brokerAddr, consumeGroup, mq, offset);
}
// 4.0.0 added
- @Override public void updateNameServerConfig(Properties properties,
- List<String> list) throws InterruptedException, RemotingConnectException, UnsupportedEncodingException, RemotingSendRequestException, RemotingTimeoutException, MQClientException, MQBrokerException {
+ @Override
+ public void updateNameServerConfig(Properties properties,
+ List<String> list) throws InterruptedException, RemotingConnectException, UnsupportedEncodingException, RemotingSendRequestException, RemotingTimeoutException, MQClientException, MQBrokerException {
}
- @Override public Map<String, Properties> getNameServerConfig(
- List<String> list) throws InterruptedException, RemotingTimeoutException, RemotingSendRequestException, RemotingConnectException, MQClientException, UnsupportedEncodingException {
+ @Override
+ public Map<String, Properties> getNameServerConfig(
+ List<String> list) throws InterruptedException, RemotingTimeoutException, RemotingSendRequestException, RemotingConnectException, MQClientException, UnsupportedEncodingException {
return null;
}
- @Override public QueryConsumeQueueResponseBody queryConsumeQueue(String brokerAddr, String topic,
- int queueId, long index, int count,
- String consumerGroup) throws InterruptedException, RemotingTimeoutException, RemotingSendRequestException, RemotingConnectException, MQClientException {
+ @Override
+ public QueryConsumeQueueResponseBody queryConsumeQueue(String brokerAddr, String topic,
+ int queueId, long index, int count,
+ String consumerGroup) throws InterruptedException, RemotingTimeoutException, RemotingSendRequestException, RemotingConnectException, MQClientException {
return null;
}
}
diff --git a/src/main/java/org/apache/rocketmq/exporter/service/client/MQAdminInstance.java b/src/main/java/org/apache/rocketmq/exporter/service/client/MQAdminInstance.java
index 6994c23..1b9b3df 100644
--- a/src/main/java/org/apache/rocketmq/exporter/service/client/MQAdminInstance.java
+++ b/src/main/java/org/apache/rocketmq/exporter/service/client/MQAdminInstance.java
@@ -16,104 +16,72 @@
*/
package org.apache.rocketmq.exporter.service.client;
+import org.apache.commons.lang3.StringUtils;
import org.apache.rocketmq.client.consumer.DefaultMQPullConsumer;
import org.apache.rocketmq.client.exception.MQClientException;
import org.apache.rocketmq.client.impl.MQClientAPIImpl;
import org.apache.rocketmq.client.impl.factory.MQClientInstance;
import org.apache.rocketmq.common.MixAll;
+import org.apache.rocketmq.exporter.config.RMQConfigure;
import org.apache.rocketmq.remoting.RemotingClient;
import org.apache.rocketmq.tools.admin.DefaultMQAdminExt;
import org.apache.rocketmq.tools.admin.DefaultMQAdminExtImpl;
-import org.apache.rocketmq.tools.admin.MQAdminExt;
import org.joor.Reflect;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import org.springframework.beans.factory.annotation.Autowired;
+import org.springframework.beans.factory.annotation.Qualifier;
+import org.springframework.context.annotation.Bean;
+import org.springframework.stereotype.Service;
import static org.apache.rocketmq.common.MixAll.TOOLS_CONSUMER_GROUP;
-
+@Service
public class MQAdminInstance {
-
- private static final ThreadLocal<DefaultMQAdminExt> MQ_ADMIN_EXT_THREAD_LOCAL = new ThreadLocal<DefaultMQAdminExt>();
-
- private static final ThreadLocal<DefaultMQPullConsumer> MQ_PULL_CONSUMER_THREAD_LOCAL = new ThreadLocal<DefaultMQPullConsumer>();
-
- private static final ThreadLocal<Integer> INIT_COUNTER = new ThreadLocal<Integer>();
-
- public static MQAdminExt threadLocalMQAdminExt() {
- DefaultMQAdminExt defaultMQAdminExt = MQ_ADMIN_EXT_THREAD_LOCAL.get();
- if (defaultMQAdminExt == null) {
- throw new IllegalStateException("defaultMQAdminExt should be init before you get this");
+ private final static Logger log = LoggerFactory.getLogger(MQAdminInstance.class);
+ @Autowired
+ private RMQConfigure configure;
+
+ @Bean(destroyMethod = "shutdown", name = "defaultMQAdminExt")
+ private DefaultMQAdminExt buildDefaultMQAdminExt() {
+ DefaultMQAdminExt defaultMQAdminExt = new DefaultMQAdminExt(5000L);
+ defaultMQAdminExt.setInstanceName("admin-" + System.currentTimeMillis());
+ try {
+ defaultMQAdminExt.start();
+ } catch (MQClientException ex) {
+ log.error(String.format("init default admin error, namesrv=%s", System.getProperty(MixAll.NAMESRV_ADDR_PROPERTY)), ex);
}
return defaultMQAdminExt;
}
-
- public static DefaultMQPullConsumer threadLocalMQPullConsumer() {
- DefaultMQPullConsumer pullConsumer = MQ_PULL_CONSUMER_THREAD_LOCAL.get();
- if (pullConsumer == null) {
- throw new IllegalStateException("pullConsumer should be init before you get this");
+ @Bean(destroyMethod = "shutdown")
+ private DefaultMQPullConsumer buildPullConsumer() throws Exception {
+ String namesrvAddress = configure.getNamesrvAddr();
+ if (StringUtils.isBlank(namesrvAddress)) {
+ log.error("init default pull consumer error, namesrv is null");
+ throw new Exception("init default pull consumer error, namesrv is null", null);
}
- return pullConsumer;
- }
-
-
- public static RemotingClient threadLocalRemotingClient() {
- MQClientInstance mqClientInstance = threadLocalMqClientInstance();
- MQClientAPIImpl mQClientAPIImpl = Reflect.on(mqClientInstance).get("mQClientAPIImpl");
- return Reflect.on(mQClientAPIImpl).get("remotingClient");
- }
-
- public static MQClientInstance threadLocalMqClientInstance() {
- DefaultMQAdminExtImpl defaultMQAdminExtImpl = Reflect.on(MQAdminInstance.threadLocalMQAdminExt()).get("defaultMQAdminExtImpl");
- return Reflect.on(defaultMQAdminExtImpl).get("mqClientInstance");
- }
-
- public static void initMQAdminInstance(long timeoutMillis) throws MQClientException {
- Integer nowCount = INIT_COUNTER.get();
- if (nowCount == null) {
- DefaultMQAdminExt defaultMQAdminExt;
- if (timeoutMillis > 0) {
- defaultMQAdminExt = new DefaultMQAdminExt(timeoutMillis);
- }
- else {
- defaultMQAdminExt = new DefaultMQAdminExt();
- }
- defaultMQAdminExt.setInstanceName("admin-" + Long.toString(System.currentTimeMillis()));
- defaultMQAdminExt.start();
- MQ_ADMIN_EXT_THREAD_LOCAL.set(defaultMQAdminExt);
-
-
- DefaultMQPullConsumer pullConsumer;
- pullConsumer = new DefaultMQPullConsumer(TOOLS_CONSUMER_GROUP,null);
- pullConsumer.setInstanceName("consumer-" + Long.toString(System.currentTimeMillis()));
- pullConsumer.setNamesrvAddr(System.getProperty(MixAll.NAMESRV_ADDR_PROPERTY, System.getenv(MixAll.NAMESRV_ADDR_ENV)));
+ DefaultMQPullConsumer pullConsumer = new DefaultMQPullConsumer(TOOLS_CONSUMER_GROUP, null);
+ pullConsumer.setInstanceName("consumer-" + System.currentTimeMillis());
+ pullConsumer.setNamesrvAddr(namesrvAddress);
+ try {
pullConsumer.start();
pullConsumer.getDefaultMQPullConsumerImpl().getPullAPIWrapper().setConnectBrokerByUser(true);
-
- MQ_PULL_CONSUMER_THREAD_LOCAL.set(pullConsumer);
- INIT_COUNTER.set(1);
- }
- else {
- INIT_COUNTER.set(nowCount + 1);
+ } catch (MQClientException ex) {
+ log.error(String.format("init default pull consumer error, namesrv=%s", System.getProperty(MixAll.NAMESRV_ADDR_PROPERTY)), ex);
}
+ return pullConsumer;
+ }
+ @Bean(destroyMethod = "shutdown")
+ private MQClientInstance buildInstance(@Qualifier("defaultMQAdminExt") DefaultMQAdminExt defaultMQAdminExt) {
+ DefaultMQAdminExtImpl defaultMQAdminExtImpl = Reflect.on(defaultMQAdminExt).get("defaultMQAdminExtImpl");
+ return Reflect.on(defaultMQAdminExtImpl).get("mqClientInstance");
}
- public static void destroyMQAdminInstance() {
- Integer nowCount = INIT_COUNTER.get() - 1;
- if (nowCount > 0) {
- INIT_COUNTER.set(nowCount);
- return;
- }
- MQAdminExt mqAdminExt = MQ_ADMIN_EXT_THREAD_LOCAL.get();
- if (mqAdminExt != null) {
- DefaultMQPullConsumer consumer = MQ_PULL_CONSUMER_THREAD_LOCAL.get();
- if (consumer != null) {
- consumer.shutdown();
- MQ_PULL_CONSUMER_THREAD_LOCAL.remove();
- }
- mqAdminExt.shutdown();
- MQ_ADMIN_EXT_THREAD_LOCAL.remove();
- INIT_COUNTER.remove();
- }
+ @Bean
+ private RemotingClient client(MQClientInstance instance) {
+ MQClientAPIImpl mQClientAPIImpl = Reflect.on(instance).get("mQClientAPIImpl");
+ return Reflect.on(mQClientAPIImpl).get("remotingClient");
}
}
diff --git a/src/main/java/org/apache/rocketmq/exporter/service/impl/RMQMetricsServiceImpl.java b/src/main/java/org/apache/rocketmq/exporter/service/impl/RMQMetricsServiceImpl.java
index 5dd008b..6cf977f 100644
--- a/src/main/java/org/apache/rocketmq/exporter/service/impl/RMQMetricsServiceImpl.java
+++ b/src/main/java/org/apache/rocketmq/exporter/service/impl/RMQMetricsServiceImpl.java
@@ -19,25 +19,17 @@ package org.apache.rocketmq.exporter.service.impl;
import io.prometheus.client.CollectorRegistry;
import io.prometheus.client.exporter.common.TextFormat;
import org.apache.rocketmq.exporter.collector.RMQMetricsCollector;
-import org.apache.rocketmq.exporter.service.AbstractCommonService;
import org.apache.rocketmq.exporter.service.RMQMetricsService;
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
import org.springframework.stereotype.Service;
import java.io.IOException;
import java.io.StringWriter;
@Service
-public class RMQMetricsServiceImpl extends AbstractCommonService implements RMQMetricsService {
-
- private Logger logger = LoggerFactory.getLogger(RMQMetricsServiceImpl.class);
-
- private CollectorRegistry registry = new CollectorRegistry();
-
+public class RMQMetricsServiceImpl implements RMQMetricsService {
+ private CollectorRegistry registry = new CollectorRegistry();
private final RMQMetricsCollector rmqMetricsCollector;
-
public RMQMetricsCollector getCollector() {
return rmqMetricsCollector;
}
@@ -46,8 +38,8 @@ public class RMQMetricsServiceImpl extends AbstractCommonService implements RMQM
rmqMetricsCollector = new RMQMetricsCollector();
rmqMetricsCollector.register(registry);
}
- public void Metrics(StringWriter writer) throws IOException {
+
+ public void metrics(StringWriter writer) throws IOException {
TextFormat.write004(writer, registry.metricFamilySamples());
- logger.info(writer.toString());
}
}
diff --git a/src/main/java/org/apache/rocketmq/exporter/task/MetricsCollectTask.java b/src/main/java/org/apache/rocketmq/exporter/task/MetricsCollectTask.java
index c6e6a5e..f1777ec 100644
--- a/src/main/java/org/apache/rocketmq/exporter/task/MetricsCollectTask.java
+++ b/src/main/java/org/apache/rocketmq/exporter/task/MetricsCollectTask.java
@@ -16,290 +16,483 @@
*/
package org.apache.rocketmq.exporter.task;
-import com.google.common.base.Throwables;
-import org.apache.rocketmq.client.consumer.PullResult;
-import org.apache.rocketmq.client.consumer.PullStatus;
+import com.alibaba.fastjson.JSON;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.rocketmq.client.exception.MQBrokerException;
+import org.apache.rocketmq.client.exception.MQClientException;
import org.apache.rocketmq.common.MixAll;
import org.apache.rocketmq.common.admin.ConsumeStats;
import org.apache.rocketmq.common.admin.OffsetWrapper;
import org.apache.rocketmq.common.admin.TopicOffset;
import org.apache.rocketmq.common.admin.TopicStatsTable;
import org.apache.rocketmq.common.message.MessageQueue;
+import org.apache.rocketmq.common.protocol.ResponseCode;
import org.apache.rocketmq.common.protocol.body.BrokerStatsData;
import org.apache.rocketmq.common.protocol.body.ClusterInfo;
+import org.apache.rocketmq.common.protocol.body.ConsumerConnection;
import org.apache.rocketmq.common.protocol.body.GroupList;
+import org.apache.rocketmq.common.protocol.body.KVTable;
import org.apache.rocketmq.common.protocol.body.TopicList;
import org.apache.rocketmq.common.protocol.route.BrokerData;
import org.apache.rocketmq.common.protocol.route.TopicRouteData;
-import org.apache.rocketmq.exporter.aspect.admin.annotation.MultiMQAdminCmdMethod;
import org.apache.rocketmq.exporter.config.RMQConfigure;
+import org.apache.rocketmq.exporter.model.BrokerRuntimeStats;
import org.apache.rocketmq.exporter.service.RMQMetricsService;
-import org.apache.rocketmq.exporter.service.client.MQAdminExtImpl;
import org.apache.rocketmq.exporter.util.Utils;
+import org.apache.rocketmq.remoting.exception.RemotingConnectException;
+import org.apache.rocketmq.remoting.exception.RemotingException;
+import org.apache.rocketmq.remoting.exception.RemotingSendRequestException;
+import org.apache.rocketmq.remoting.exception.RemotingTimeoutException;
import org.apache.rocketmq.store.stats.BrokerStatsManager;
import org.apache.rocketmq.tools.admin.MQAdminExt;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
+import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.scheduling.annotation.Scheduled;
import org.springframework.stereotype.Component;
+import javax.annotation.PostConstruct;
import javax.annotation.Resource;
-import java.util.Date;
-import java.util.HashMap;
import java.util.Map;
import java.util.Set;
@Component
public class MetricsCollectTask {
-
@Resource
+ @Qualifier("mqAdminExtImpl")
private MQAdminExt mqAdminExt;
@Resource
private RMQConfigure rmqConfigure;
-
@Resource
- private RMQMetricsService metricsService;
-
+ private RMQMetricsService metricsService;
private final static Logger log = LoggerFactory.getLogger(MetricsCollectTask.class);
- @Scheduled(cron = "15 0/1 * * * ?")
- @MultiMQAdminCmdMethod(timeoutMillis = 5000)
- public void collectOffset() {
+ @PostConstruct
+ public void init() throws InterruptedException, RemotingConnectException, RemotingTimeoutException, RemotingSendRequestException, MQBrokerException {
+ log.info("MetricsCollectTask init starting....");
+ long start = System.currentTimeMillis();
+ ClusterInfo clusterInfo = mqAdminExt.examineBrokerClusterInfo();
+ StringBuilder infoOut = new StringBuilder();
+ for (String clusterName : clusterInfo.getClusterAddrTable().keySet()) {
+ infoOut.append(String.format("cluster name= %s, broker name = %s%n", clusterName, clusterInfo.getClusterAddrTable().get(clusterName)));
+ }
+ for (String brokerName : clusterInfo.getBrokerAddrTable().keySet()) {
+ infoOut.append(String.format("broker name = %s，master broker address= %s%n", brokerName, clusterInfo.getBrokerAddrTable().get(brokerName).getBrokerAddrs().get(MixAll.MASTER_ID)));
+ }
+ log.info(infoOut.toString());
+ log.info(String.format("MetricsCollectTask init finished....cost:%d", System.currentTimeMillis() - start));
+ }
+
+ @Scheduled(cron = "${task.collectTopicOffset.cron}")
+ public void collectTopicOffset() {
if (!rmqConfigure.isEnableCollect()) {
return;
}
- Date date = new Date();
+ log.info("topic offset collection task starting....");
+ long start = System.currentTimeMillis();
+ TopicList topicList = null;
try {
- TopicList topicList = mqAdminExt.fetchAllTopicList();
- Set<String> topicSet = topicList.getTopicList();
- for (String topic : topicSet) {
- if (topic.startsWith(MixAll.RETRY_GROUP_TOPIC_PREFIX) || topic.startsWith(MixAll.DLQ_GROUP_TOPIC_PREFIX)) {
- continue;
+ topicList = mqAdminExt.fetchAllTopicList();
+ } catch (Exception ex) {
+ log.error(String.format("collectTopicOffset-exception comes getting topic list from namesrv, address is %s",
+ JSON.toJSONString(mqAdminExt.getNameServerAddressList())));
+ return;
+ }
+ Set<String> topicSet = topicList != null ? topicList.getTopicList() : null;
+ if (topicSet == null || topicSet.isEmpty()) {
+ log.error(String.format("collectTopicOffset-the topic list is empty. the namesrv address is %s",
+ JSON.toJSONString(mqAdminExt.getNameServerAddressList())));
+ return;
+ }
+ for (String topic : topicSet) {
+ TopicStatsTable topicStats = null;
+ try {
+ topicStats = mqAdminExt.examineTopicStats(topic);
+ } catch (Exception ex) {
+ log.error(String.format("collectTopicOffset-getting topic(%s) stats error. the namesrv address is %s",
+ topic,
+ JSON.toJSONString(mqAdminExt.getNameServerAddressList())));
+ continue;
+ }
+
+ Set<Map.Entry<MessageQueue, TopicOffset>> topicStatusEntries = topicStats.getOffsetTable().entrySet();
+
+ double totalMaxOffset = 0L;
+ long lastUpdateTimestamp = 0L;
+ StringBuilder sb = new StringBuilder();
+
+ for (Map.Entry<MessageQueue, TopicOffset> topicStatusEntry : topicStatusEntries) {
+ MessageQueue q = topicStatusEntry.getKey();
+ TopicOffset offset = topicStatusEntry.getValue();
+ totalMaxOffset += offset.getMaxOffset();
+ if (offset.getLastUpdateTimestamp() > lastUpdateTimestamp) {
+ lastUpdateTimestamp = offset.getLastUpdateTimestamp();
+ }
+ sb.append(q.getBrokerName()).append(" ");
+ }
+ metricsService.getCollector().addTopicOffsetMetric("", sb.toString(), topic, lastUpdateTimestamp, totalMaxOffset);
+ }
+ log.info("topic offset collection task finished...." + (System.currentTimeMillis() - start));
+ }
+
+ @Scheduled(cron = "${task.collectConsumerOffset.cron}")
+ public void collectConsumerOffset() {
+ if (!rmqConfigure.isEnableCollect()) {
+ return;
+ }
+ log.info("consumer offset collection task starting....");
+ long start = System.currentTimeMillis();
+ TopicList topicList = null;
+ try {
+ topicList = mqAdminExt.fetchAllTopicList();
+ } catch (Exception ex) {
+ log.error(String.format("collectConsumerOffset-fetch topic list from namesrv error, the address is %s",
+ JSON.toJSONString(mqAdminExt.getNameServerAddressList())), ex);
+ return;
+ }
+
+
+ Set<String> topicSet = topicList.getTopicList();
+ for (String topic : topicSet) {
+ GroupList groupList = null;
+
+ boolean isDLQTopic = topic.startsWith(MixAll.DLQ_GROUP_TOPIC_PREFIX);
+ if (isDLQTopic) {
+ continue;
+ }
+ try {
+ groupList = mqAdminExt.queryTopicConsumeByWho(topic);
+ } catch (Exception ex) {
+ log.warn(String.format("collectConsumerOffset-topic's consumer is empty, %s", topic));
+ continue;
+ }
+
+ if (groupList == null || groupList.getGroupList() == null || groupList.getGroupList().isEmpty()) {
+ log.warn(String.format("no any consumer for topic(%s), ignore this topic", topic));
+ continue;
+ }
+
+
+ for (String group : groupList.getGroupList()) {
+ ConsumeStats consumeStats = null;
+ ConsumerConnection onlineConsumers = null;
+ long diff = 0L, totalConsumerOffset = 0L, totalBrokerOffset = 0L;
+ int countOfOnlineConsumers = 0;
+
+ double consumeTPS = 0F;
+ try {
+ onlineConsumers = mqAdminExt.examineConsumerConnectionInfo(group);
+ } catch (InterruptedException | RemotingException ex) {
+ log.error(String.format("get topic's(%s) online consumers(%s) exception", topic, group), ex);
+ } catch (MQClientException ex) {
+ handleTopicNotExistException(ex.getResponseCode(), ex, topic, group);
+ } catch (MQBrokerException ex) {
+ handleTopicNotExistException(ex.getResponseCode(), ex, topic, group);
}
- String clusterName = null;
- ClusterInfo clusterInfo = mqAdminExt.examineBrokerClusterInfo();
- Set<Map.Entry<String, BrokerData>> clusterEntries = clusterInfo.getBrokerAddrTable().entrySet();
- for (Map.Entry<String, BrokerData> clusterEntry : clusterEntries) {
- clusterName = clusterEntry.getValue().getCluster();
- break;
+ if (onlineConsumers == null || onlineConsumers.getConnectionSet() == null || onlineConsumers.getConnectionSet().isEmpty()) {
+ log.warn(String.format("no any consumer online. topic=%s, consumer group=%s. ignore this", topic, group));
+ countOfOnlineConsumers = 0;
+ } else {
+ countOfOnlineConsumers = onlineConsumers.getConnectionSet().size();
}
- if (clusterName != null) {
- HashMap<String,Long> brokerOffsetMap = new HashMap<>();
- TopicStatsTable topicStatus = mqAdminExt.examineTopicStats(topic);
- Set<Map.Entry<MessageQueue, TopicOffset>> topicStatusEntries = topicStatus.getOffsetTable().entrySet();
- for (Map.Entry<MessageQueue, TopicOffset> topicStatusEntry : topicStatusEntries) {
- MessageQueue q = topicStatusEntry.getKey();
- TopicOffset offset = topicStatusEntry.getValue();
- if (brokerOffsetMap.containsKey(q.getBrokerName())) {
- brokerOffsetMap.put(q.getBrokerName(),brokerOffsetMap.get(q.getBrokerName()) + offset.getMaxOffset());
- }
- else {
- brokerOffsetMap.put(q.getBrokerName(),offset.getMaxOffset());
- }
- }
- Set<Map.Entry<String, Long>> brokerOffsetEntries = brokerOffsetMap.entrySet();
- for (Map.Entry<String, Long> brokerOffsetEntry : brokerOffsetEntries) {
- metricsService.getCollector().AddTopicOffsetMetric(clusterName,brokerOffsetEntry.getKey(), topic, brokerOffsetEntry.getValue());
- }
+ try {
+ consumeStats = mqAdminExt.examineConsumeStats(group, topic);
+ } catch (InterruptedException | RemotingException ex) {
+ log.error(String.format("get topic's(%s) consumer-stats(%s) exception", topic, group), ex);
+ } catch (MQClientException ex) {
+ handleTopicNotExistException(ex.getResponseCode(), ex, topic, group);
+ } catch (MQBrokerException ex) {
+ handleTopicNotExistException(ex.getResponseCode(), ex, topic, group);
+ }
+ if (consumeStats == null || consumeStats.getOffsetTable() == null || consumeStats.getOffsetTable().isEmpty()) {
+ log.warn(String.format("no any offset for consumer(%s), topic(%s), ignore this", group, topic));
+ continue;
+ }
+ {
+ diff = consumeStats.computeTotalDiff();
+ consumeTPS = consumeStats.getConsumeTps();
+ metricsService.getCollector().addGroupDiffMetric(String.valueOf(countOfOnlineConsumers), group, topic, diff);
+ metricsService.getCollector().addGroupConsumeTPSMetric(topic, group, consumeTPS);
}
+ Set<Map.Entry<MessageQueue, OffsetWrapper>> consumeStatusEntries = consumeStats.getOffsetTable().entrySet();
+ for (Map.Entry<MessageQueue, OffsetWrapper> consumeStatusEntry : consumeStatusEntries) {
+ MessageQueue q = consumeStatusEntry.getKey();
+ OffsetWrapper offset = consumeStatusEntry.getValue();
- HashMap<String,Long> consumeOffsetMap = new HashMap<>();
- GroupList groupList = mqAdminExt.queryTopicConsumeByWho(topic);
- if (groupList != null && !groupList.getGroupList().isEmpty()) {
- for (String group : groupList.getGroupList()) {
- try {
- ConsumeStats consumeStatus = mqAdminExt.examineConsumeStats(group, topic);
- Set<Map.Entry<MessageQueue, OffsetWrapper>> consumeStatusEntries = consumeStatus.getOffsetTable().entrySet();
- for (Map.Entry<MessageQueue, OffsetWrapper> consumeStatusEntry : consumeStatusEntries) {
- MessageQueue q = consumeStatusEntry.getKey();
- OffsetWrapper offset = consumeStatusEntry.getValue();
- if (consumeOffsetMap.containsKey(q.getBrokerName())) {
- consumeOffsetMap.put(q.getBrokerName(), consumeOffsetMap.get(q.getBrokerName()) + offset.getConsumerOffset());
- } else {
- consumeOffsetMap.put(q.getBrokerName(), offset.getConsumerOffset());
- }
- }
- } catch (Exception e) {
- log.info("ignore this consumer", e.getMessage());
- }
- Set<Map.Entry<String, Long>> consumeOffsetEntries = consumeOffsetMap.entrySet();
- for (Map.Entry<String, Long> consumeOffsetEntry : consumeOffsetEntries) {
- metricsService.getCollector().AddGroupOffsetMetric(clusterName,consumeOffsetEntry.getKey(), topic, group, consumeOffsetEntry.getValue());
- }
- consumeOffsetMap.clear();
- }
+ //topic + consumer group 生产offset
+ totalBrokerOffset += totalBrokerOffset + offset.getBrokerOffset();
+ //topic + consumer group 消费offset
+ totalConsumerOffset += offset.getConsumerOffset();
}
+ metricsService.getCollector().addGroupBrokerTotalOffsetMetric(topic, group, totalBrokerOffset);
+ metricsService.getCollector().addGroupConsumerTotalOffsetMetric(topic, group, totalBrokerOffset);
}
- } catch (Exception e) {
- log.info("error is " + e.getMessage());
}
+ log.info("consumer offset collection task finished...." + (System.currentTimeMillis() - start));
}
- @Scheduled(cron = "15 0/1 * * * ?")
- @MultiMQAdminCmdMethod(timeoutMillis = 5000)
- public void collectTopic() {
+ @Scheduled(cron = "${task.collectBrokerStatsTopic.cron}")
+ public void collectBrokerStatsTopic() {
if (!rmqConfigure.isEnableCollect()) {
return;
}
- Date date = new Date();
+ log.info("broker topic stats collection task starting....");
+ long start = System.currentTimeMillis();
+ Set<String> topicSet = null;
try {
TopicList topicList = mqAdminExt.fetchAllTopicList();
- Set<String> topicSet = topicList.getTopicList();
- for (String topic : topicSet) {
- if (topic.startsWith(MixAll.RETRY_GROUP_TOPIC_PREFIX) || topic.startsWith(MixAll.DLQ_GROUP_TOPIC_PREFIX)) {
- continue;
+ topicSet = topicList.getTopicList();
+ } catch (Exception ex) {
+ log.error(String.format("collectBrokerStatsTopic-fetch topic list from namesrv error, the address is %s",
+ JSON.toJSONString(mqAdminExt.getNameServerAddressList())), ex);
+ return;
+ }
+ if (topicSet == null || topicSet.isEmpty()) {
+ return;
+ }
+ ClusterInfo clusterInfo = null;
+ try {
+ clusterInfo = mqAdminExt.examineBrokerClusterInfo();
+ } catch (Exception ex) {
+ log.error(String.format("collectBrokerStatsTopic-fetch cluster info exception, the address is %s",
+ JSON.toJSONString(mqAdminExt.getNameServerAddressList())), ex);
+ return;
+ }
+
+ for (String topic : topicSet) {
+ if (topic.startsWith(MixAll.RETRY_GROUP_TOPIC_PREFIX) || topic.startsWith(MixAll.DLQ_GROUP_TOPIC_PREFIX)) {
+ continue;
+ }
+ TopicRouteData topicRouteData = null;
+
+ try {
+ topicRouteData = mqAdminExt.examineTopicRouteInfo(topic);
+ } catch (Exception ex) {
+ log.error(String.format("fetch topic route error. ignore %s", topic), ex);
+ continue;
+ }
+ for (BrokerData bd : topicRouteData.getBrokerDatas()) {
+ String masterAddr = bd.getBrokerAddrs().get(MixAll.MASTER_ID);
+ if (!StringUtils.isBlank(masterAddr)) {
+ BrokerStatsData bsd = null;
+ try {
+ //topic发了多少条消息
+ bsd = mqAdminExt.viewBrokerStatsData(masterAddr, BrokerStatsManager.TOPIC_PUT_NUMS, topic);
+ String brokerIP = clusterInfo.getBrokerAddrTable().get(bd.getBrokerName()).getBrokerAddrs().get(MixAll.MASTER_ID);
+ metricsService.getCollector().addTopicPutNumsMetric(
+ bd.getCluster(),
+ bd.getBrokerName(),
+ brokerIP,
+ "",
+ topic,
+ Utils.getFixedDouble(bsd.getStatsMinute().getTps())
+ );
+ } catch (MQClientException ex) {
+ if (ex.getResponseCode() == ResponseCode.SYSTEM_ERROR) {
+ log.error(String.format("TOPIC_PUT_NUMS-error, topic=%s, master broker=%s, %s", topic, masterAddr, ex.getErrorMessage()));
+ } else {
+ log.error(String.format("TOPIC_PUT_NUMS-error, topic=%s, master broker=%s", topic, masterAddr), ex);
+ }
+ } catch (RemotingTimeoutException | InterruptedException | RemotingSendRequestException | RemotingConnectException ex1) {
+ log.error(String.format("TOPIC_PUT_NUMS-error, topic=%s, master broker=%s", topic, masterAddr), ex1);
+ }
+ try {
+ //topic总共发了多少字节
+ bsd = mqAdminExt.viewBrokerStatsData(masterAddr, BrokerStatsManager.TOPIC_PUT_SIZE, topic);
+ String brokerIP = clusterInfo.getBrokerAddrTable().get(bd.getBrokerName()).getBrokerAddrs().get(MixAll.MASTER_ID);
+ metricsService.getCollector().addTopicPutSizeMetric(
+ bd.getCluster(),
+ bd.getBrokerName(),
+ brokerIP,
+ "",
+ topic,
+ Utils.getFixedDouble(bsd.getStatsMinute().getTps())
+ );
+ } catch (MQClientException ex) {
+ if (ex.getResponseCode() == ResponseCode.SYSTEM_ERROR) {
+ log.error(String.format("TOPIC_PUT_SIZE-error, topic=%s, master broker=%s, %s", topic, masterAddr, ex.getErrorMessage()));
+ } else {
+ log.error(String.format("TOPIC_PUT_SIZE-error, topic=%s, master broker=%s", topic, masterAddr), ex);
+ }
+ } catch (InterruptedException | RemotingConnectException | RemotingTimeoutException | RemotingSendRequestException ex) {
+ log.error(String.format("TOPIC_PUT_SIZE-error, topic=%s, master broker=%s", topic, masterAddr), ex);
+ }
}
- TopicRouteData topicRouteData = mqAdminExt.examineTopicRouteInfo(topic);
- GroupList groupList = mqAdminExt.queryTopicConsumeByWho(topic);
+ }
+ GroupList groupList = null;
+ try {
+ groupList = mqAdminExt.queryTopicConsumeByWho(topic);
+ } catch (Exception ex) {
+ log.error(String.format("collectBrokerStatsTopic-fetch consumers for topic(%s) error, ignore this topic", topic), ex);
+ return;
+ }
+ if (groupList.getGroupList() == null || groupList.getGroupList().isEmpty()) {
+ log.warn(String.format("collectBrokerStatsTopic-topic's consumer is empty, %s", topic));
+ return;
+ }
+ for (String group : groupList.getGroupList())
for (BrokerData bd : topicRouteData.getBrokerDatas()) {
String masterAddr = bd.getBrokerAddrs().get(MixAll.MASTER_ID);
if (masterAddr != null) {
+ String statsKey = String.format("%s@%s", topic, group);
+ BrokerStatsData bsd = null;
try {
- BrokerStatsData bsd = null;
- try {
- bsd = mqAdminExt.viewBrokerStatsData(masterAddr, BrokerStatsManager.TOPIC_PUT_NUMS, topic);
- metricsService.getCollector().AddTopicPutNumsMetric(bd.getCluster(), bd.getBrokerName(), topic, Utils.getFixedDouble(bsd.getStatsMinute().getTps()));
- }
- catch (Exception e) {
- log.info("error is " + e.getMessage());
- }
- try {
- bsd = mqAdminExt.viewBrokerStatsData(masterAddr, BrokerStatsManager.TOPIC_PUT_SIZE, topic);
- metricsService.getCollector().AddTopicPutSizeMetric(bd.getCluster(), bd.getBrokerName(), topic, Utils.getFixedDouble(bsd.getStatsMinute().getTps()));
+ //消费者消费了多少条消息
+ bsd = mqAdminExt.viewBrokerStatsData(masterAddr, BrokerStatsManager.GROUP_GET_NUMS, statsKey);
+ metricsService.getCollector().addGroupGetNumsMetric(
+ topic,
+ group,
+ Utils.getFixedDouble(bsd.getStatsMinute().getTps()));
+ } catch (MQClientException ex) {
+ if (ex.getResponseCode() == ResponseCode.SYSTEM_ERROR) {
+ log.error(String.format("GROUP_GET_NUMS-error, topic=%s, group=%s,master broker=%s, %s", topic, group, masterAddr, ex.getErrorMessage()));
+ } else {
+ log.error(String.format("GROUP_GET_NUMS-error, topic=%s, group=%s,master broker=%s", topic, group, masterAddr), ex);
}
- catch (Exception e) {
- log.info("error is " + e.getMessage());
+ } catch (InterruptedException | RemotingConnectException | RemotingTimeoutException | RemotingSendRequestException ex) {
+ log.error(String.format("GROUP_GET_NUMS-error, topic=%s, group=%s,master broker=%s", topic, group, masterAddr), ex);
+ }
+ try {
+ //消费者消费了多少字节
+ bsd = mqAdminExt.viewBrokerStatsData(masterAddr, BrokerStatsManager.GROUP_GET_SIZE, statsKey);
+ metricsService.getCollector().addGroupGetSizeMetric(
+ topic,
+ group,
+ Utils.getFixedDouble(bsd.getStatsMinute().getTps()));
+ } catch (MQClientException ex) {
+ if (ex.getResponseCode() == ResponseCode.SYSTEM_ERROR) {
+ log.error(String.format("GROUP_GET_SIZE-error, topic=%s, group=%s, master broker=%s, %s", topic, group, masterAddr, ex.getErrorMessage()));
+ } else {
+ log.error(String.format("GROUP_GET_SIZE-error, topic=%s, group=%s, master broker=%s", topic, group, masterAddr), ex);
}
- } catch (Exception e) {
- log.info("error is " + e.getMessage());
+ } catch (InterruptedException | RemotingConnectException | RemotingTimeoutException | RemotingSendRequestException ex) {
+ log.error(String.format("GROUP_GET_SIZE-error, topic=%s, group=%s, master broker=%s", topic, group, masterAddr), ex);
}
- }
- }
- if (groupList != null && !groupList.getGroupList().isEmpty()) {
- for (String group : groupList.getGroupList()) {
- for (BrokerData bd : topicRouteData.getBrokerDatas()) {
- String masterAddr = bd.getBrokerAddrs().get(MixAll.MASTER_ID);
- if (masterAddr != null) {
- try {
- String statsKey = String.format("%s@%s", topic, group);
- BrokerStatsData bsd = null;
- try {
- bsd = mqAdminExt.viewBrokerStatsData(masterAddr, BrokerStatsManager.GROUP_GET_NUMS, statsKey);
- metricsService.getCollector().AddGroupGetNumsMetric(bd.getCluster(), bd.getBrokerName(), topic, group, Utils.getFixedDouble(bsd.getStatsMinute().getTps()));
- } catch (Exception e) {
- log.info("error is " + e.getMessage());
- }
- try {
- bsd = mqAdminExt.viewBrokerStatsData(masterAddr, BrokerStatsManager.GROUP_GET_SIZE, statsKey);
- metricsService.getCollector().AddGroupGetSizeMetric(bd.getCluster(), bd.getBrokerName(), topic, group, Utils.getFixedDouble(bsd.getStatsMinute().getTps()));
- } catch (Exception e) {
- log.info("error is " + e.getMessage());
- }
- try {
-
- bsd = mqAdminExt.viewBrokerStatsData(masterAddr, BrokerStatsManager.SNDBCK_PUT_NUMS, statsKey);
- metricsService.getCollector().AddsendBackNumsMetric(bd.getCluster(), bd.getBrokerName(), topic, group, Utils.getFixedDouble(bsd.getStatsMinute().getTps()));
- } catch (Exception e) {
- log.info("error is " + e.getMessage());
- }
- try {
- collectLatencyMetrcisInner(topic, group, masterAddr, bd);
- } catch (Exception e) {
- log.info("error is " + e.getMessage());
- }
- } catch (Exception e) {
- log.info("error is " + e.getMessage());
- }
+ try {
+ //消费者重新消费topic的次数
+ bsd = mqAdminExt.viewBrokerStatsData(masterAddr, BrokerStatsManager.SNDBCK_PUT_NUMS, statsKey);
+ metricsService.getCollector().addSendBackNumsMetric(
+ topic,
+ group,
+ Utils.getFixedDouble(bsd.getStatsMinute().getTps()));
+ } catch (MQClientException ex) {
+ if (ex.getResponseCode() == ResponseCode.SYSTEM_ERROR) {
+ log.error(String.format("SNDBCK_PUT_NUMS-error, topic=%s, group=%s, master broker=%s, %s", topic, group, masterAddr, ex.getErrorMessage()));
+ } else {
+ log.error(String.format("SNDBCK_PUT_NUMS-error, topic=%s, group=%s, master broker=%s", topic, group, masterAddr), ex);
}
+ } catch (InterruptedException | RemotingConnectException | RemotingTimeoutException | RemotingSendRequestException ex) {
+ log.error(String.format("SNDBCK_PUT_NUMS-error, topic=%s, group=%s, master broker=%s", topic, group, masterAddr), ex);
}
}
}
- }
- }
- catch (Exception err) {
- throw Throwables.propagate(err);
}
+ log.info("broker topic stats collection task finished...." + (System.currentTimeMillis() - start));
}
- @Scheduled(cron = "15 0/1 * * * ?")
- @MultiMQAdminCmdMethod(timeoutMillis = 5000)
- public void collectBroker() {
+ @Scheduled(cron = "${task.collectBrokerStats.cron}")
+ public void collectBrokerStats() {
if (!rmqConfigure.isEnableCollect()) {
return;
}
+ log.info("broker stats collection task starting....");
+ long start = System.currentTimeMillis();
+ ClusterInfo clusterInfo = null;
try {
- Date date = new Date();
- ClusterInfo clusterInfo = mqAdminExt.examineBrokerClusterInfo();
- Set<Map.Entry<String, BrokerData>> clusterEntries = clusterInfo.getBrokerAddrTable().entrySet();
- for (Map.Entry<String, BrokerData> clusterEntry : clusterEntries) {
- String masterAddr = clusterEntry.getValue().getBrokerAddrs().get(MixAll.MASTER_ID);
- if (masterAddr != null) {
- try {
- BrokerStatsData bsd = null;
- try {
- bsd = mqAdminExt.viewBrokerStatsData(masterAddr, BrokerStatsManager.BROKER_PUT_NUMS,clusterEntry.getValue().getCluster());
- metricsService.getCollector().AddBrokerPutNumsMetric(clusterEntry.getValue().getCluster(), clusterEntry.getValue().getBrokerName(), Utils.getFixedDouble(bsd.getStatsMinute().getTps()));
- }
- catch (Exception e) {
- log.info("error is " + e.getMessage());
- }
- try {
- bsd = mqAdminExt.viewBrokerStatsData(masterAddr, BrokerStatsManager.BROKER_GET_NUMS, clusterEntry.getValue().getCluster());
- metricsService.getCollector().AddBrokerGetNumsMetric(clusterEntry.getValue().getCluster(), clusterEntry.getValue().getBrokerName(), Utils.getFixedDouble(bsd.getStatsMinute().getTps()));
- }
- catch (Exception e) {
- log.info("error is " + e.getMessage());
- }
- } catch (Exception e) {
- log.info("error is " + e.getMessage());
- }
- }
- }
- }
- catch (Exception err) {
- throw Throwables.propagate(err);
+ clusterInfo = mqAdminExt.examineBrokerClusterInfo();
+ } catch (Exception ex) {
+ log.error(String.format("collectBrokerStats-get cluster info from namesrv error. address is %s", JSON.toJSONString(mqAdminExt.getNameServerAddressList())), ex);
+ return;
}
- }
- private void collectLatencyMetrcisInner(String topic,String group,String masterAddr, BrokerData bd) throws Exception {
- long maxLagTime = 0;
- String statsKey;
- BrokerStatsData bsd = null;
- ConsumeStats consumeStatus = mqAdminExt.examineConsumeStats(group, topic);
- Set<Map.Entry<MessageQueue, OffsetWrapper>> consumeStatusEntries = consumeStatus.getOffsetTable().entrySet();
- for (Map.Entry<MessageQueue, OffsetWrapper> consumeStatusEntry : consumeStatusEntries) {
- MessageQueue q = consumeStatusEntry.getKey();
- OffsetWrapper offset = consumeStatusEntry.getValue();
- int queueId = q.getQueueId();
- statsKey = String.format("%d@%s@%s", queueId, topic, group);
+
+ Set<Map.Entry<String, BrokerData>> clusterEntries = clusterInfo.getBrokerAddrTable().entrySet();
+ for (Map.Entry<String, BrokerData> clusterEntry : clusterEntries) {
+ String masterAddr = clusterEntry.getValue().getBrokerAddrs().get(MixAll.MASTER_ID);
+ if (StringUtils.isBlank(masterAddr)) {
+ continue;
+ }
+ BrokerStatsData bsd = null;
try {
- bsd = mqAdminExt.viewBrokerStatsData(masterAddr, BrokerStatsManager.GROUP_GET_LATENCY, statsKey);
- metricsService.getCollector().AddGroupGetLatencyMetric(bd.getCluster(), bd.getBrokerName(), topic, group, String.format("%d", queueId), Utils.getFixedDouble(bsd.getStatsMinute().getTps()));
- } catch (Exception e) {
- log.info("error is " + e.getMessage());
+ bsd = mqAdminExt.viewBrokerStatsData(masterAddr, BrokerStatsManager.BROKER_PUT_NUMS, clusterEntry.getValue().getCluster());
+ String brokerIP = clusterEntry.getValue().getBrokerAddrs().get(MixAll.MASTER_ID);
+ metricsService.getCollector().addBrokerPutNumsMetric(
+ clusterEntry.getValue().getCluster(),
+ brokerIP,
+ "",
+ Utils.getFixedDouble(bsd.getStatsMinute().getTps()));
+ } catch (Exception ex) {
+ log.error(String.format("BROKER_PUT_NUMS-error, master broker=%s", masterAddr), ex);
}
- MQAdminExtImpl mqAdminImpl = (MQAdminExtImpl) mqAdminExt;
- PullResult consumePullResult = mqAdminImpl.queryMsgByOffset(q, offset.getConsumerOffset());
- long lagTime = 0;
- if (consumePullResult != null && consumePullResult.getPullStatus() == PullStatus.FOUND) {
- lagTime = System.currentTimeMillis() - consumePullResult.getMsgFoundList().get(0).getStoreTimestamp();
- if (offset.getBrokerOffset() == offset.getConsumerOffset()) {
- lagTime = 0;
- }
- } else if (consumePullResult.getPullStatus() == PullStatus.NO_MATCHED_MSG) {
- lagTime = 0;
- } else if (consumePullResult.getPullStatus() == PullStatus.OFFSET_ILLEGAL) {
- PullResult pullResult = mqAdminImpl.queryMsgByOffset(q, consumePullResult.getMinOffset());
- if (pullResult != null && pullResult.getPullStatus() == PullStatus.FOUND) {
- lagTime = System.currentTimeMillis() - consumePullResult.getMsgFoundList().get(0).getStoreTimestamp();
+ try {
+ bsd = mqAdminExt.viewBrokerStatsData(masterAddr, BrokerStatsManager.BROKER_GET_NUMS, clusterEntry.getValue().getCluster());
+ String brokerIP = clusterEntry.getValue().getBrokerAddrs().get(MixAll.MASTER_ID);
+ metricsService.getCollector().addBrokerGetNumsMetric(
+ clusterEntry.getValue().getCluster(),
+ brokerIP,
+ "",
+ Utils.getFixedDouble(bsd.getStatsMinute().getTps()));
+ } catch (Exception ex) {
+ log.error(String.format("BROKER_GET_NUMS-error, master broker=%s", masterAddr), ex);
+ }
+ }
+ log.info("broker stats collection task finished...." + (System.currentTimeMillis() - start));
+ }
+
+ @Scheduled(cron = "${task.collectBrokerRuntimeStats.cron}")
+ public void collectBrokerRuntimeStats() {
+ if (!rmqConfigure.isEnableCollect()) {
+ return;
+ }
+ log.info("broker runtime stats collection task starting....");
+ long start = System.currentTimeMillis();
+ ClusterInfo clusterInfo = null;
+ try {
+ clusterInfo = mqAdminExt.examineBrokerClusterInfo();
+ } catch (Exception ex) {
+ log.error(String.format("collectBrokerRuntimeStats-get cluster info from namesrv error. address is %s", JSON.toJSONString(mqAdminExt.getNameServerAddressList())), ex);
+ return;
+ }
+
+ Set<Map.Entry<String, BrokerData>> clusterEntries = clusterInfo.getBrokerAddrTable().entrySet();
+ for (Map.Entry<String, BrokerData> clusterEntry : clusterEntries) {
+ String masterAddr = clusterEntry.getValue().getBrokerAddrs().get(MixAll.MASTER_ID);
+ String clusterName = clusterEntry.getValue().getCluster();
+
+ KVTable kvTable = null;
+ if (!StringUtils.isBlank(masterAddr)) {
+ try {
+ kvTable = mqAdminExt.fetchBrokerRuntimeStats(masterAddr);
+ } catch (RemotingConnectException | RemotingSendRequestException | RemotingTimeoutException | InterruptedException ex) {
+ log.error(String.format("collectBrokerRuntimeStats-get fetch broker runtime stats error, address=%s", masterAddr), ex);
+ } catch (MQBrokerException ex) {
+ if (ex.getResponseCode() == ResponseCode.SYSTEM_ERROR) {
+ log.error(String.format("collectBrokerRuntimeStats-get fetch broker runtime stats error, address=%s, error=%s", masterAddr, ex.getErrorMessage()));
+ } else {
+ log.error(String.format("collectBrokerRuntimeStats-get fetch broker runtime stats error, address=%s", masterAddr), ex);
+ }
}
- } else {
- lagTime = 0;
}
- if (lagTime > maxLagTime) {
- maxLagTime = lagTime;
+ if (kvTable == null || kvTable.getTable() == null || kvTable.getTable().isEmpty()) {
+ continue;
}
+ try {
+ BrokerRuntimeStats brokerRuntimeStats = new BrokerRuntimeStats(kvTable);
+ metricsService.getCollector().addBrokerRuntimeStatsMetric(brokerRuntimeStats, clusterName, masterAddr, "");
+ } catch (Exception ex) {
+ log.error(String.format("collectBrokerRuntimeStats-parse or report broker runtime stats error, %s", JSON.toJSONString(kvTable)), ex);
+ }
+
+ }
+
+ log.info("broker runtime stats collection task finished...." + (System.currentTimeMillis() - start));
+ }
+
+ private void handleTopicNotExistException(int responseCode, Exception ex, String topic, String group) {
+ if (responseCode == ResponseCode.TOPIC_NOT_EXIST || responseCode == ResponseCode.CONSUMER_NOT_ONLINE) {
+ log.error(String.format("get topic's(%s) consumer-stats(%s) exception, detail: %s", topic, group, ex.getMessage()));
+ } else {
+ log.error(String.format("get topic's(%s) consumer-stats(%s) exception", topic, group), ex);
}
- metricsService.getCollector().AddGroupGetLatencyByStoreTimeMetric(bd.getCluster(), bd.getBrokerName(), topic, group, maxLagTime);
}
}
diff --git a/src/main/java/org/apache/rocketmq/exporter/util/JsonUtil.java b/src/main/java/org/apache/rocketmq/exporter/util/JsonUtil.java
index 29317f9..1cb147c 100644
--- a/src/main/java/org/apache/rocketmq/exporter/util/JsonUtil.java
+++ b/src/main/java/org/apache/rocketmq/exporter/util/JsonUtil.java
@@ -55,8 +55,7 @@ public class JsonUtil {
public static void writeValue(Writer writer, Object obj) {
try {
objectMapper.writeValue(writer, obj);
- }
- catch (IOException e) {
+ } catch (IOException e) {
Throwables.propagateIfPossible(e);
}
}
@@ -67,9 +66,8 @@ public class JsonUtil {
}
try {
- return src instanceof String ? (String)src : objectMapper.writeValueAsString(src);
- }
- catch (Exception e) {
+ return src instanceof String ? (String) src : objectMapper.writeValueAsString(src);
+ } catch (Exception e) {
logger.error("Parse Object to String error src=" + src, e);
return null;
}
@@ -81,9 +79,8 @@ public class JsonUtil {
}
try {
- return src instanceof byte[] ? (byte[])src : objectMapper.writeValueAsBytes(src);
- }
- catch (Exception e) {
+ return src instanceof byte[] ? (byte[]) src : objectMapper.writeValueAsBytes(src);
+ } catch (Exception e) {
logger.error("Parse Object to byte[] error", e);
return null;
}
@@ -95,9 +92,8 @@ public class JsonUtil {
}
str = escapesSpecialChar(str);
try {
- return clazz.equals(String.class) ? (T)str : objectMapper.readValue(str, clazz);
- }
- catch (Exception e) {
+ return clazz.equals(String.class) ? (T) str : objectMapper.readValue(str, clazz);
+ } catch (Exception e) {
logger.error("Parse String to Object error\nString: {}\nClass<T>: {}\nError: {}", str, clazz.getName(), e);
return null;
}
@@ -108,9 +104,8 @@ public class JsonUtil {
return null;
}
try {
- return clazz.equals(byte[].class) ? (T)bytes : objectMapper.readValue(bytes, clazz);
- }
- catch (Exception e) {
+ return clazz.equals(byte[].class) ? (T) bytes : objectMapper.readValue(bytes, clazz);
+ } catch (Exception e) {
logger.error("Parse byte[] to Object error\nbyte[]: {}\nClass<T>: {}\nError: {}", bytes, clazz.getName(), e);
return null;
}
@@ -122,11 +117,10 @@ public class JsonUtil {
}
str = escapesSpecialChar(str);
try {
- return (T)(typeReference.getType().equals(String.class) ? str : objectMapper.readValue(str, typeReference));
- }
- catch (Exception e) {
+ return (T) (typeReference.getType().equals(String.class) ? str : objectMapper.readValue(str, typeReference));
+ } catch (Exception e) {
logger.error("Parse String to Object error\nString: {}\nTypeReference<T>: {}\nError: {}", str,
- typeReference.getType(), e);
+ typeReference.getType(), e);
return null;
}
}
@@ -136,12 +130,11 @@ public class JsonUtil {
return null;
}
try {
- return (T)(typeReference.getType().equals(byte[].class) ? bytes : objectMapper.readValue(bytes,
- typeReference));
- }
- catch (Exception e) {
+ return (T) (typeReference.getType().equals(byte[].class) ? bytes : objectMapper.readValue(bytes,
+ typeReference));
+ } catch (Exception e) {
logger.error("Parse byte[] to Object error\nbyte[]: {}\nTypeReference<T>: {}\nError: {}", bytes,
- typeReference.getType(), e);
+ typeReference.getType(), e);
return null;
}
}
diff --git a/src/main/resources/application.properties b/src/main/resources/application.properties
deleted file mode 100644
index 083738e..0000000
--- a/src/main/resources/application.properties
+++ /dev/null
@@ -1,14 +0,0 @@
-server.port=5557
-
-spring.application.name=rocketmq-exporter
-spring.http.encoding.charset=UTF-8
-spring.http.encoding.enabled=true
-spring.http.encoding.force=true
-logging.config=classpath:logback.xml
-#if this value is empty,use env value rocketmq.config.namesrvAddr NAMESRV_ADDR
-rocketmq.config.namesrvAddr=127.0.0.1:9876
-
-
-rocketmq.config.enableCollect=true
-rocketmq.config.webTelemetryPath=/metrics
-rocketmq.config.rocketmqVersion=4_3_2
\ No newline at end of file
diff --git a/src/main/resources/application.yml b/src/main/resources/application.yml
new file mode 100644
index 0000000..8ff9e07
--- /dev/null
+++ b/src/main/resources/application.yml
@@ -0,0 +1,32 @@
+server:
+ port: 5557
+spring:
+ application:
+ name: rocketmq-exporter
+ http:
+ encoding:
+ charset: UTF-8
+ enabled: true
+ force: true
+logging:
+ config: classpath:logback.xml
+
+rocketmq:
+ config:
+ webTelemetryPath: /metrics
+ rocketmqVersion: 4_2_0
+ namesrvAddr: 127.0.0.1:9876 #
+ enableCollect: true
+
+task:
+ count: 5
+ collectTopicOffset:
+ cron: 15 0/1 * * * ?
+ collectConsumerOffset:
+ cron: 15 0/1 * * * ?
+ collectBrokerStatsTopic:
+ cron: 15 0/1 * * * ?
+ collectBrokerStats:
+ cron: 15 0/1 * * * ?
+ collectBrokerRuntimeStats:
+ cron: 15 0/1 * * * ?
diff --git a/src/main/resources/logback.xml b/src/main/resources/logback.xml
index b5291d3..8032516 100644
--- a/src/main/resources/logback.xml
+++ b/src/main/resources/logback.xml
@@ -1,33 +1,32 @@
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
- <appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
- <encoder charset="UTF-8">
- <pattern>[%d{yyyy-MM-dd HH:mm:ss.SSS}] %5p %m%n</pattern>
- </encoder>
- </appender>
+ <appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
+ <encoder charset="UTF-8">
+ <pattern>[%d{yyyy-MM-dd HH:mm:ss.SSS}] %5p %m%n</pattern>
+ </encoder>
+ </appender>
- <appender name="FILE"
- class="ch.qos.logback.core.rolling.RollingFileAppender">
- <file>${user.home}/logs/exporterlogs/rocketmq-exporter.log</file>
- <append>true</append>
- <rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
- <fileNamePattern>${user.home}/logs/exporterlogs/rocketmq-exporter-%d{yyyy-MM-dd}.%i.log
- </fileNamePattern>
- <timeBasedFileNamingAndTriggeringPolicy
- class="ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP">
- <maxFileSize>104857600</maxFileSize>
- </timeBasedFileNamingAndTriggeringPolicy>
- <MaxHistory>10</MaxHistory>
- </rollingPolicy>
- <encoder>
- <pattern>[%d{yyyy-MM-dd HH:mm:ss.SSS}] %5p %m%n</pattern>
- <charset class="java.nio.charset.Charset">UTF-8</charset>
- </encoder>
- </appender>
+ <appender name="FILE"
+ class="ch.qos.logback.core.rolling.RollingFileAppender">
+ <file>./logs/exporterlogs/rocketmq-exporter.log</file>
+ <append>true</append>
+ <rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
+ <fileNamePattern>./logs/exporterlogs/rocketmq-exporter-%d{yyyy-MM-dd}.%i.log
+ </fileNamePattern>
+ <timeBasedFileNamingAndTriggeringPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP">
+ <maxFileSize>104857600</maxFileSize>
+ </timeBasedFileNamingAndTriggeringPolicy>
+ <totalSizeCap>20gb</totalSizeCap>
+ </rollingPolicy>
+ <encoder>
+ <pattern>[%d{yyyy-MM-dd HH:mm:ss.SSS}] %5p %m%n</pattern>
+ <charset class="java.nio.charset.Charset">UTF-8</charset>
+ </encoder>
+ </appender>
- <root level="INFO">
- <appender-ref ref="STDOUT" />
- <appender-ref ref="FILE" />
- </root>
+ <root level="INFO">
+ <appender-ref ref="STDOUT"/>
+ <appender-ref ref="FILE"/>
+ </root>
-</configuration>
\ No newline at end of file
+</configuration>
[rocketmq-exporter] 31/43: Update README.md
Posted by wl...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
wlliqipeng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/rocketmq-exporter.git
commit 025b973f464c944caa5263ce3621cb0eea793760
Author: von gosling <vo...@apache.org>
AuthorDate: Mon Jul 22 17:47:36 2019 +0800
Update README.md
[rocketmq-exporter] 10/43: amend the wold style
Posted by wl...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
wlliqipeng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/rocketmq-exporter.git
commit 9f7265f37ae34c463d61424eca0cd3a86b98e138
Author: fengqing <fe...@sunlands.com>
AuthorDate: Fri Jun 21 16:42:10 2019 +0800
amend the wold style
---
README.md | 418 +++++++++++++++++++++++++++++++-------------------------------
1 file changed, 209 insertions(+), 209 deletions(-)
diff --git a/README.md b/README.md
index c9d3c42..d1eda8f 100644
--- a/README.md
+++ b/README.md
@@ -1,210 +1,210 @@
-RocketMQ_exporter
-==============
-
-RocketMQ exporter for Prometheus.
-
-Table of Contents
------------------
-- [Compatibility](#compatibility)
-- [Dependency](#dependency)
-- [Download](#download)
-- [Compile](#compile)
- - [Build Binary](#build-binary)
- - [Build Docker Image](#build-docker-image)
-- [Run](#run)
- - [Run Binary](#run-binary)
- - [Run Docker Image](#run-docker-image)
-- [Flags](#flags)
-- [Metrics](#metrics)
- - [Brokers](#brokers)
- - [Topics](#topics)
- - [Consumer Groups](#consumer-groups)
-- [Grafana Dashboard](#Grafana Dashboard)
-- [Contribute](#contribute)
-
-Compatibility
--------------
-
-Support [Apache RocketMQ](https://rocketmq.apache.org) version 4.3.2 (and later).
-
-Dependency
-----------
-
-- [Prometheus](https://prometheus.io)
-
-Compile
--------
-
-### Build Binary
-
-```shell
-mvn clean install
-```
-
-### Build Docker Image
-
-```shell
-mvn package -Dmaven.test.skip=true docker:build
-```
-
-Run
----
-
-### Run Binary
-
-```shell
-java -jar rocketmq-exporter-0.0.1-SNAPSHOT.jar [--rocketmq.config.namesrvAddr="127.0.0.1:9876" ...]
-```
-
-### Run Docker Image
-
-```
-docker container run -itd --rm -p 5557:5557 breezecoolyang/rocketmq-exporter [--rocketmq.config.namesrvAddr="127.0.0.1:9876" ...]
-```
-
-Flags
----
-
-This image is configurable using different flags
-
-|Flag name | Default | Description |
-| -----------------------------------|--------------------|----------------------------------------------------|
-| `rocketmq.config.namesrvAddr` | 127.0.0.1:9876 |name server address for broker cluster |
-| `rocketmq.config.webTelemetryPath` | /metrics |Path under which to expose metrics |
-| `server.port` | 5557 |Address to listen on for web interface and telemetry|
-| `rocketmq.config.rocketmqVersion` | V4_3_2 |rocketmq broker version |
-
-Metrics
--------
-
-Documents about exposed Prometheus metrics.
-
-### Broker
-
-**Metrics details**
-
-| Name | Exposed information |
-| ------------ | ---------------------------------------------------- |
-| `rocketmq_broker_tps` | total put message numbers per second for this broker |
-| `rocketmq_broker_qps` | total get message numbers per second for this broker |
-
-**Metrics output example**
-
-```txt
-# HELP rocketmq_broker_tps BrokerPutNums
-# TYPE rocketmq_broker_tps gauge
-rocketmq_broker_tps{cluster="MQCluster",broker="broker-a",} 7.933333333333334
-rocketmq_broker_tps{cluster="MQCluster",broker="broker-b",} 7.916666666666667
-# HELP rocketmq_broker_qps BrokerGetNums
-# TYPE rocketmq_broker_qps gauge
-rocketmq_broker_qps{cluster="MQCluster",broker="broker-a",} 8.2
-rocketmq_broker_qps{cluster="MQCluster",broker="broker-b",} 8.15
-```
-
-### Topics
-
-**Metrics details**
-
-| Name | Exposed information |
-| ------------------- | -------------------------------------------------- |
-| `rocketmq_producer_tps` | sending messages number per second for this topic |
-| `rocketmq_producer_put_size` | sending messages size per second for this topic |
-| `rocketmq_producer_offset` | Current Offset of a Broker for this topic |
-
-**Metrics output example**
-
-```txt
-# HELP rocketmq_producer_tps TopicPutNums
-# TYPE rocketmq_producer_tps gauge
-rocketmq_producer_tps{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",} 7.933333333333334
-rocketmq_producer_tps{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",} 7.916666666666667
-# HELP rocketmq_producer_put_size TopicPutSize
-# TYPE rocketmq_producer_put_size gauge
-rocketmq_producer_put_size{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",} 1642.2
-rocketmq_producer_put_size{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",} 1638.75
-# HELP rocketmq_producer_offset TopicOffset
-# TYPE rocketmq_producer_offset counter
-rocketmq_producer_offset{cluster="MQCluster",broker="broker-a",topic="TBW102",} 0.0
-rocketmq_producer_offset{cluster="MQCluster",broker="broker-b",topic="DEV_TID_tfq",} 1878633.0
-rocketmq_producer_offset{cluster="MQCluster",broker="broker-a",topic="DEV_TID_tfq",} 3843787.0
-rocketmq_producer_offset{cluster="MQCluster",broker="broker-b",topic="DEV_TID_20190304",} 0.0
-rocketmq_producer_offset{cluster="MQCluster",broker="broker-a",topic="BenchmarkTest",} 0.0
-rocketmq_producer_offset{cluster="MQCluster",broker="broker-b",topic="DEV_TID_20190305",} 0.0
-rocketmq_producer_offset{cluster="MQCluster",broker="broker-b",topic="MQCluster",} 0.0
-rocketmq_producer_offset{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",} 2798195.0
-rocketmq_producer_offset{cluster="MQCluster",broker="broker-b",topic="BenchmarkTest",} 0.0
-rocketmq_producer_offset{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",} 1459666.0
-rocketmq_producer_offset{cluster="MQCluster",broker="broker-a",topic="MQCluster",} 0.0
-rocketmq_producer_offset{cluster="MQCluster",broker="broker-a",topic="SELF_TEST_TOPIC",} 0.0
-rocketmq_producer_offset{cluster="MQCluster",broker="broker-a",topic="OFFSET_MOVED_EVENT",} 0.0
-rocketmq_producer_offset{cluster="MQCluster",broker="broker-b",topic="broker-b",} 0.0
-rocketmq_producer_offset{cluster="MQCluster",broker="broker-a",topic="broker-a",} 0.0
-rocketmq_producer_offset{cluster="MQCluster",broker="broker-b",topic="SELF_TEST_TOPIC",} 0.0
-rocketmq_producer_offset{cluster="MQCluster",broker="broker-b",topic="RMQ_SYS_TRANS_HALF_TOPIC",} 0.0
-rocketmq_producer_offset{cluster="MQCluster",broker="broker-a",topic="DEV_TID_20190305",} 0.0
-rocketmq_producer_offset{cluster="MQCluster",broker="broker-b",topic="OFFSET_MOVED_EVENT",} 0.0
-rocketmq_producer_offset{cluster="MQCluster",broker="broker-a",topic="RMQ_SYS_TRANS_HALF_TOPIC",} 0.0
-rocketmq_producer_offset{cluster="MQCluster",broker="broker-b",topic="TBW102",} 0.0
-rocketmq_producer_offset{cluster="MQCluster",broker="broker-a",topic="DEV_TID_20190304",} 0.0
-
-```
-
-### Consumer Groups
-
-**Metrics details**
-
-| Name | Exposed information |
-| --------------------------------- | ------------------------------------------------------------ |
-| `rocketmq_consumer_tps` | consumer message numbers per second for this Topic |
-| `rocketmq_consumer_get_size` | consumer message size per second for this Topic |
-| `rocketmq_consumer_offset` | consumer offset for this topic |
-| `rocketmq_group_get_latency` | consumer latency on some topic for one queue |
-| `rocketmq_group_get_latency_by_storetime ` | consumer latency between message consume time and message store time on some topic |
-
-**Metrics output example**
-
-```txt
-# HELP rocketmq_consumer_tps GroupGetNums
-# TYPE rocketmq_consumer_tps gauge
-rocketmq_consumer_tps{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",} 7.916666666666667
-rocketmq_consumer_tps{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",} 7.933333333333334
-# HELP rocketmq_consumer_get_size GroupGetSize
-# TYPE rocketmq_consumer_get_size gauge
-rocketmq_consumer_get_size{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",} 1638.75
-rocketmq_consumer_get_size{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",} 1642.2
-# HELP rocketmq_consumer_offset GroupOffset
-# TYPE rocketmq_consumer_offset counter
-rocketmq_consumer_offset{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",} 1462030.0
-rocketmq_consumer_offset{cluster="MQCluster",broker="broker-a",topic="DEV_TID_tfq",group="DEV_CID_cfq",} 3843787.0
-rocketmq_consumer_offset{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",} 2800569.0
-rocketmq_consumer_offset{cluster="MQCluster",broker="broker-b",topic="DEV_TID_tfq",group="DEV_CID_cfq",} 1878633.0
-# HELP rocketmq_group_get_latency GroupGetLatency
-# TYPE rocketmq_group_get_latency gauge
-rocketmq_group_get_latency{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="0",} 0.05
-rocketmq_group_get_latency{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="1",} 0.0
-rocketmq_group_get_latency{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="7",} 0.05
-rocketmq_group_get_latency{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="6",} 0.016666666666666666
-rocketmq_group_get_latency{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="3",} 0.0
-rocketmq_group_get_latency{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="7",} 0.03333333333333333
-rocketmq_group_get_latency{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="4",} 0.0
-rocketmq_group_get_latency{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="5",} 0.03333333333333333
-rocketmq_group_get_latency{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="6",} 0.016666666666666666
-rocketmq_group_get_latency{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="2",} 0.0
-rocketmq_group_get_latency{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="3",} 0.0
-rocketmq_group_get_latency{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="0",} 0.0
-rocketmq_group_get_latency{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="4",} 0.0
-rocketmq_group_get_latency{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="1",} 0.03333333333333333
-rocketmq_group_get_latency{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="5",} 0.0
-rocketmq_group_get_latency{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="2",} 0.0
-# HELP rocketmq_group_get_latency_by_storetime GroupGetLatencyByStoreTime
-# TYPE rocketmq_group_get_latency_by_storetime gauge
-rocketmq_group_get_latency_by_storetime{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",} 3215.0
-rocketmq_group_get_latency_by_storetime{cluster="MQCluster",broker="broker-a",topic="DEV_TID_tfq",group="DEV_CID_cfq",} 0.0
-rocketmq_group_get_latency_by_storetime{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",} 3232.0
-rocketmq_group_get_latency_by_storetime{cluster="MQCluster",broker="broker-b",topic="DEV_TID_tfq",group="DEV_CID_cfq",} 0.0
-```
-
-Grafana Dashboard
--------
-Grafana Dashboard ID: 10405, name: RocketMQ Exporter Overview.
+RocketMQ_exporter
+==============
+
+RocketMQ exporter for Prometheus.
+
+Table of Contents
+-----------------
+- [Compatibility](#compatibility)
+- [Dependency](#dependency)
+- [Download](#download)
+- [Compile](#compile)
+ - [Build Binary](#build-binary)
+ - [Build Docker Image](#build-docker-image)
+- [Run](#run)
+ - [Run Binary](#run-binary)
+ - [Run Docker Image](#run-docker-image)
+- [Flags](#flags)
+- [Metrics](#metrics)
+ - [Brokers](#brokers)
+ - [Topics](#topics)
+ - [Consumer Groups](#consumer-groups)
+- [Grafana Dashboard](#Grafana-Dashboard)
+- [Contribute](#contribute)
+
+Compatibility
+-------------
+
+Support [Apache RocketMQ](https://rocketmq.apache.org) version 4.3.2 (and later).
+
+Dependency
+----------
+
+- [Prometheus](https://prometheus.io)
+
+Compile
+-------
+
+### Build Binary
+
+```shell
+mvn clean install
+```
+
+### Build Docker Image
+
+```shell
+mvn package -Dmaven.test.skip=true docker:build
+```
+
+Run
+---
+
+### Run Binary
+
+```shell
+java -jar rocketmq-exporter-0.0.1-SNAPSHOT.jar [--rocketmq.config.namesrvAddr="127.0.0.1:9876" ...]
+```
+
+### Run Docker Image
+
+```
+docker container run -itd --rm -p 5557:5557 breezecoolyang/rocketmq-exporter [--rocketmq.config.namesrvAddr="127.0.0.1:9876" ...]
+```
+
+Flags
+---
+
+This image is configurable using different flags
+
+|Flag name | Default | Description |
+| -----------------------------------|--------------------|----------------------------------------------------|
+| `rocketmq.config.namesrvAddr` | 127.0.0.1:9876 |name server address for broker cluster |
+| `rocketmq.config.webTelemetryPath` | /metrics |Path under which to expose metrics |
+| `server.port` | 5557 |Address to listen on for web interface and telemetry|
+| `rocketmq.config.rocketmqVersion` | V4_3_2 |rocketmq broker version |
+
+Metrics
+-------
+
+Documents about exposed Prometheus metrics.
+
+### Broker
+
+**Metrics details**
+
+| Name | Exposed information |
+| ------------ | ---------------------------------------------------- |
+| `rocketmq_broker_tps` | total put message numbers per second for this broker |
+| `rocketmq_broker_qps` | total get message numbers per second for this broker |
+
+**Metrics output example**
+
+```txt
+# HELP rocketmq_broker_tps BrokerPutNums
+# TYPE rocketmq_broker_tps gauge
+rocketmq_broker_tps{cluster="MQCluster",broker="broker-a",} 7.933333333333334
+rocketmq_broker_tps{cluster="MQCluster",broker="broker-b",} 7.916666666666667
+# HELP rocketmq_broker_qps BrokerGetNums
+# TYPE rocketmq_broker_qps gauge
+rocketmq_broker_qps{cluster="MQCluster",broker="broker-a",} 8.2
+rocketmq_broker_qps{cluster="MQCluster",broker="broker-b",} 8.15
+```
+
+### Topics
+
+**Metrics details**
+
+| Name | Exposed information |
+| ------------------- | -------------------------------------------------- |
+| `rocketmq_producer_tps` | sending messages number per second for this topic |
+| `rocketmq_producer_put_size` | sending messages size per second for this topic |
+| `rocketmq_producer_offset` | Current Offset of a Broker for this topic |
+
+**Metrics output example**
+
+```txt
+# HELP rocketmq_producer_tps TopicPutNums
+# TYPE rocketmq_producer_tps gauge
+rocketmq_producer_tps{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",} 7.933333333333334
+rocketmq_producer_tps{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",} 7.916666666666667
+# HELP rocketmq_producer_put_size TopicPutSize
+# TYPE rocketmq_producer_put_size gauge
+rocketmq_producer_put_size{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",} 1642.2
+rocketmq_producer_put_size{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",} 1638.75
+# HELP rocketmq_producer_offset TopicOffset
+# TYPE rocketmq_producer_offset counter
+rocketmq_producer_offset{cluster="MQCluster",broker="broker-a",topic="TBW102",} 0.0
+rocketmq_producer_offset{cluster="MQCluster",broker="broker-b",topic="DEV_TID_tfq",} 1878633.0
+rocketmq_producer_offset{cluster="MQCluster",broker="broker-a",topic="DEV_TID_tfq",} 3843787.0
+rocketmq_producer_offset{cluster="MQCluster",broker="broker-b",topic="DEV_TID_20190304",} 0.0
+rocketmq_producer_offset{cluster="MQCluster",broker="broker-a",topic="BenchmarkTest",} 0.0
+rocketmq_producer_offset{cluster="MQCluster",broker="broker-b",topic="DEV_TID_20190305",} 0.0
+rocketmq_producer_offset{cluster="MQCluster",broker="broker-b",topic="MQCluster",} 0.0
+rocketmq_producer_offset{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",} 2798195.0
+rocketmq_producer_offset{cluster="MQCluster",broker="broker-b",topic="BenchmarkTest",} 0.0
+rocketmq_producer_offset{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",} 1459666.0
+rocketmq_producer_offset{cluster="MQCluster",broker="broker-a",topic="MQCluster",} 0.0
+rocketmq_producer_offset{cluster="MQCluster",broker="broker-a",topic="SELF_TEST_TOPIC",} 0.0
+rocketmq_producer_offset{cluster="MQCluster",broker="broker-a",topic="OFFSET_MOVED_EVENT",} 0.0
+rocketmq_producer_offset{cluster="MQCluster",broker="broker-b",topic="broker-b",} 0.0
+rocketmq_producer_offset{cluster="MQCluster",broker="broker-a",topic="broker-a",} 0.0
+rocketmq_producer_offset{cluster="MQCluster",broker="broker-b",topic="SELF_TEST_TOPIC",} 0.0
+rocketmq_producer_offset{cluster="MQCluster",broker="broker-b",topic="RMQ_SYS_TRANS_HALF_TOPIC",} 0.0
+rocketmq_producer_offset{cluster="MQCluster",broker="broker-a",topic="DEV_TID_20190305",} 0.0
+rocketmq_producer_offset{cluster="MQCluster",broker="broker-b",topic="OFFSET_MOVED_EVENT",} 0.0
+rocketmq_producer_offset{cluster="MQCluster",broker="broker-a",topic="RMQ_SYS_TRANS_HALF_TOPIC",} 0.0
+rocketmq_producer_offset{cluster="MQCluster",broker="broker-b",topic="TBW102",} 0.0
+rocketmq_producer_offset{cluster="MQCluster",broker="broker-a",topic="DEV_TID_20190304",} 0.0
+
+```
+
+### Consumer Groups
+
+**Metrics details**
+
+| Name | Exposed information |
+| --------------------------------- | ------------------------------------------------------------ |
+| `rocketmq_consumer_tps` | consumer message numbers per second for this Topic |
+| `rocketmq_consumer_get_size` | consumer message size per second for this Topic |
+| `rocketmq_consumer_offset` | consumer offset for this topic |
+| `rocketmq_group_get_latency` | consumer latency on some topic for one queue |
+| `rocketmq_group_get_latency_by_storetime ` | consumer latency between message consume time and message store time on some topic |
+
+**Metrics output example**
+
+```txt
+# HELP rocketmq_consumer_tps GroupGetNums
+# TYPE rocketmq_consumer_tps gauge
+rocketmq_consumer_tps{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",} 7.916666666666667
+rocketmq_consumer_tps{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",} 7.933333333333334
+# HELP rocketmq_consumer_get_size GroupGetSize
+# TYPE rocketmq_consumer_get_size gauge
+rocketmq_consumer_get_size{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",} 1638.75
+rocketmq_consumer_get_size{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",} 1642.2
+# HELP rocketmq_consumer_offset GroupOffset
+# TYPE rocketmq_consumer_offset counter
+rocketmq_consumer_offset{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",} 1462030.0
+rocketmq_consumer_offset{cluster="MQCluster",broker="broker-a",topic="DEV_TID_tfq",group="DEV_CID_cfq",} 3843787.0
+rocketmq_consumer_offset{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",} 2800569.0
+rocketmq_consumer_offset{cluster="MQCluster",broker="broker-b",topic="DEV_TID_tfq",group="DEV_CID_cfq",} 1878633.0
+# HELP rocketmq_group_get_latency GroupGetLatency
+# TYPE rocketmq_group_get_latency gauge
+rocketmq_group_get_latency{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="0",} 0.05
+rocketmq_group_get_latency{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="1",} 0.0
+rocketmq_group_get_latency{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="7",} 0.05
+rocketmq_group_get_latency{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="6",} 0.016666666666666666
+rocketmq_group_get_latency{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="3",} 0.0
+rocketmq_group_get_latency{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="7",} 0.03333333333333333
+rocketmq_group_get_latency{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="4",} 0.0
+rocketmq_group_get_latency{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="5",} 0.03333333333333333
+rocketmq_group_get_latency{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="6",} 0.016666666666666666
+rocketmq_group_get_latency{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="2",} 0.0
+rocketmq_group_get_latency{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="3",} 0.0
+rocketmq_group_get_latency{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="0",} 0.0
+rocketmq_group_get_latency{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="4",} 0.0
+rocketmq_group_get_latency{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="1",} 0.03333333333333333
+rocketmq_group_get_latency{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="5",} 0.0
+rocketmq_group_get_latency{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",queueid="2",} 0.0
+# HELP rocketmq_group_get_latency_by_storetime GroupGetLatencyByStoreTime
+# TYPE rocketmq_group_get_latency_by_storetime gauge
+rocketmq_group_get_latency_by_storetime{cluster="MQCluster",broker="broker-b",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",} 3215.0
+rocketmq_group_get_latency_by_storetime{cluster="MQCluster",broker="broker-a",topic="DEV_TID_tfq",group="DEV_CID_cfq",} 0.0
+rocketmq_group_get_latency_by_storetime{cluster="MQCluster",broker="broker-a",topic="DEV_TID_topic_tfq",group="DEV_CID_consumer_cfq",} 3232.0
+rocketmq_group_get_latency_by_storetime{cluster="MQCluster",broker="broker-b",topic="DEV_TID_tfq",group="DEV_CID_cfq",} 0.0
+```
+
+Grafana Dashboard
+-------
+Grafana Dashboard ID: 10405, name: RocketMQ Exporter Overview.
For details of the dashboard please see [RocketMQ Exporter Overview](https://grafana.com/dashboards/10405).
\ No newline at end of file
[rocketmq-exporter] 02/43: Add rocketmq-prometheus exporter
Posted by wl...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
wlliqipeng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/rocketmq-exporter.git
commit d3baba8fbd201c2d71e4fe9ba50b0420aca24c21
Author: duhenglucky <du...@gmail.com>
AuthorDate: Fri Mar 29 13:31:46 2019 +0800
Add rocketmq-prometheus exporter
---
rocketmq-prometheus-exporter/README.md | 5 +++++
1 file changed, 5 insertions(+)
diff --git a/rocketmq-prometheus-exporter/README.md b/rocketmq-prometheus-exporter/README.md
new file mode 100644
index 0000000..a9dc2f8
--- /dev/null
+++ b/rocketmq-prometheus-exporter/README.md
@@ -0,0 +1,5 @@
+# RocketMQ-prometheus-exporter
+
+## Overview
+
+
[rocketmq-exporter] 25/43: add use example for read me file
Posted by wl...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
wlliqipeng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/rocketmq-exporter.git
commit 316dfe6408ec4dc7416fd84a617cc083425c6dff
Author: fengqing <fe...@sunlands.com>
AuthorDate: Mon Jul 22 15:58:51 2019 +0800
add use example for read me file
---
README.md | 8 +++--
rocketmq_exporter_use_example.md | 68 ++++++++++++++++++++++++++++++++++++++--
2 files changed, 71 insertions(+), 5 deletions(-)
diff --git a/README.md b/README.md
index 531878e..f97a09b 100644
--- a/README.md
+++ b/README.md
@@ -20,7 +20,7 @@ Table of Contents
- [Topics](#topics)
- [Consumer Groups](#consumer-groups)
- [Grafana Dashboard](#Grafana-Dashboard)
-- [Contribute](#contribute)
+- [Use Example](#Use-Example)
Compatibility
-------------
@@ -208,4 +208,8 @@ rocketmq_group_get_latency_by_storetime{cluster="MQCluster",broker="broker-b",to
Grafana Dashboard
-------
Grafana Dashboard ID: 10477, name: RocketMQ Exporter Overview.
-For details of the dashboard please see [RocketMQ Exporter Overview](https://grafana.com/dashboards/10477).
\ No newline at end of file
+For details of the dashboard please see [RocketMQ Exporter Overview](https://grafana.com/dashboards/10477).
+
+Use Example
+-------------
+For details of the use example please refer to [use example](./rocketmq_exporter_use_example.md)
\ No newline at end of file
diff --git a/rocketmq_exporter_use_example.md b/rocketmq_exporter_use_example.md
index 38077d5..17892d9 100644
--- a/rocketmq_exporter_use_example.md
+++ b/rocketmq_exporter_use_example.md
@@ -36,9 +36,36 @@ cd prometheus-2.7.0-rc.1.linux-amd64/
./prometheus --config.file=prometheus.yml --web.listen-address=:5555
```
-The default listening port number of Prometheus is 9090. In order not conflicts with other processes on the system, we reset the listening port number to 5555 in the startup parameters. Then go to website http://<server IP address>:5555 through browser and users can verify whether the Prometheus has been successfully installed. Since the RocketMQ-Exporter process has been started, the data of RocketMQ-Exporter can be retrieved by Prometheus at this time. At this time, users only need t [...]
+The default listening port number of Prometheus is 9090. In order not conflicts with other processes on the system, we reset the listening port number to 5555 in the startup parameters. Then go to website http://<server IP address>:5555 through browser and users can verify whether the Prometheus has been successfully installed. Since the RocketMQ-Exporter process has been started, the data of RocketMQ-Exporter can be retrieved by Prometheus at this time. At this time, users only need t [...]
+
+```
+# my global config
+global:
+ scrape_interval: 15s # Set the scrape interval to every 15 seconds. Default is every 1 minute.
+ evaluation_interval: 15s # Evaluate rules every 15 seconds. The default is every 1 minute.
+ # scrape_timeout is set to the global default (10s).
+
+
+ # Load rules once and periodically evaluate them according to the global 'evaluation_interval'.
+ rule_files:
+ # - "first_rules.yml"
+ # - "second_rules.yml"
+
+
+ scrape_configs:
+ - job_name: 'prometheus'
+ static_configs:
+ - targets: ['localhost:5555']
+
+
+ - job_name: 'exporter'
+ static_configs:
+ - targets: ['localhost:5557']
+```
+
-## 5 Creating Grafana dashboard for RocketMQ ##
+
+## 5 Create Grafana dashboard for RocketMQ ##
Prometheus' own metric display platform is not as good as Grafana. In order to better show RocketMQ's metrics, Grafana can be used to show the metrics that Prometheus gets. Firstly go to the official website https://grafana.com/grafana/download to download installation file. Here is a an example for binary file installation.
@@ -53,5 +80,40 @@ Similarly, in order not to conflict with the ports of other processes, users can
./bin/grafana-server web
```
-Then, by accessing http://<server IP address>:55555 through the browser, users can verify whether the Grafana has been successfully installed. The system default username and password are admin/admin. The first time users log in to the system, users will be asked to change the password. In addition, users need to set Grafana's data source to Prometheus. For the convenience of users, RocketMQ's dashboard configuration file has been uploaded to Grafana's official website https://grafana.c [...]
+Then, by accessing http://<server IP address>:55555 through the browser, users can verify whether the Grafana has been successfully installed. The system default username and password are admin/admin. The first time users log in to the system, users will be asked to change the password. In addition, users need to set Grafana's data source to Prometheus. If user have start up Prometheus like above, now the data source address will be http://<server IP address>:5555. For the convenience o [...]
+
+## 6 Configure alarms in Prometheus
+
+If users want to configure alarms,there are two things users should do.
+Firstly, modify the Prometheus configuration file prometheus.yml and add the following configuration:
+
+```
+rule_files:
+ - /etc/prometheus/rules/*.rules
+```
+
+Secondly, create an alert file in the directory /etc/prometheus/rules/. The content will be like as follows. For more details, please refer to the file [example.rules](./example.rules)
+
+```
+groups:
+- name: GaleraAlerts
+ rules:
+ - alert: RocketMQClusterProduceHigh
+ expr: sum(rocketmq_producer_tps) by (cluster) >= 10
+ for: 3m
+ labels:
+ severity: warning
+ annotations:
+ description: '{{$labels.cluster}} Sending tps too high.'
+ summary: cluster send tps too high
+ - alert: RocketMQClusterProduceLow
+ expr: sum(rocketmq_producer_tps) by (cluster) < 1
+ for: 3m
+ labels:
+ severity: warning
+ annotations:
+ description: '{{$labels.cluster}} Sending tps too low.'
+ summary: cluster send tps too low
+```
+
[rocketmq-exporter] 38/43: Fix wrong link
Posted by wl...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
wlliqipeng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/rocketmq-exporter.git
commit 476df337024bd15a80522b064a851822bbd4f9e5
Author: von gosling <vo...@apache.org>
AuthorDate: Mon Sep 16 14:51:50 2019 +0800
Fix wrong link
---
README.md | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/README.md b/README.md
index 2b87f1e..afc45e2 100644
--- a/README.md
+++ b/README.md
@@ -14,7 +14,7 @@ Table of Contents
- [Run Binary](#run-binary)
- [Run Docker Image](#run-docker-image)
- [Metrics](#metrics)
- - [Brokers](#brokers)
+ - [Broker](#broker)
- [Topics](#topics)
- [Consumer Groups](#consumer-groups)
- [Grafana Dashboard](#grafana-dashboard)