You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@inlong.apache.org by GitBox <gi...@apache.org> on 2021/12/01 02:52:33 UTC
[GitHub] [incubator-inlong] shink opened a new pull request #1866: [INLONG-1712][Feature][Agent] Add agent metric statistics
shink opened a new pull request #1866:
URL: https://github.com/apache/incubator-inlong/pull/1866
Fixes: #1712
### Motivation
Expose agent metrics by prometheus. See #1712.
### Modifications
- inlong-agent/agent-core
- inlong-agent/agent-common
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-inlong] dockerzhang commented on a change in pull request #1866: [INLONG-1712][Feature][Agent] Add agent Prometheus metric statistics
Posted by GitBox <gi...@apache.org>.
dockerzhang commented on a change in pull request #1866:
URL: https://github.com/apache/incubator-inlong/pull/1866#discussion_r764537440
##########
File path: inlong-agent/agent-core/src/main/java/org/apache/inlong/agent/core/AgentMain.java
##########
@@ -111,11 +126,23 @@ public static void main(String[] args) throws Exception {
try {
manager.start();
stopManagerIfKilled(manager);
+
+ boolean enablePrometheus = AgentConfiguration.getAgentConf()
+ .getBoolean(PROMETHEUS_ENABLE, DEFAULT_PROMETHEUS_ENABLE);
+ if (enablePrometheus) {
+ // starting metrics server
+ int metricsServerPort = AgentConfiguration.getAgentConf()
+ .getInt(PROMETHEUS_EXPORTER_PORT, DEFAULT_PROMETHEUS_EXPORTER_PORT);
+ LOGGER.info("Starting prometheus metrics server on port {}", metricsServerPort);
+ metricsServer = new HTTPServer(metricsServerPort);
+ }
+
manager.join();
} catch (Exception ex) {
LOGGER.error("exception caught", ex);
} finally {
manager.stop();
+ metricsServer.stop();
Review comment:
if `enablePrometheus` is `false`, here may cause an exception.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-inlong] dockerzhang commented on a change in pull request #1866: [INLONG-1712][Feature][Agent] Add agent metric statistics
Posted by GitBox <gi...@apache.org>.
dockerzhang commented on a change in pull request #1866:
URL: https://github.com/apache/incubator-inlong/pull/1866#discussion_r759919905
##########
File path: inlong-agent/agent-common/src/main/java/org/apache/inlong/agent/constants/AgentConstants.java
##########
@@ -160,4 +160,7 @@
public static final String AGENT_LOCAL_IP = "agent.local.ip";
+ public static final String PROMETHEUS_EXPORTER_PORT = "agent.prometheus.exporter.port";
+ public static final int DEFAULT_PROMETHEUS_EXPORTER_PORT = 8080;
Review comment:
@healchow the default port for Prometheus is 8080
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-inlong] healchow commented on a change in pull request #1866: [INLONG-1712][Feature][Agent] Add agent metric statistics
Posted by GitBox <gi...@apache.org>.
healchow commented on a change in pull request #1866:
URL: https://github.com/apache/incubator-inlong/pull/1866#discussion_r759889482
##########
File path: inlong-agent/conf/agent.properties
##########
@@ -101,3 +101,10 @@ channel.memory.capacity=5000
agent.manager.vip.http.host=
agent.manager.vip.http.port=
+
+
+############################
+# prometheus config
+############################
+# prometheus exporter server default port
+agent.prometheus.exporter.port=8080
Review comment:
ditto.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-inlong] codecov-commenter commented on pull request #1866: [INLONG-1712][Feature][Agent] Add agent metric statistics
Posted by GitBox <gi...@apache.org>.
codecov-commenter commented on pull request #1866:
URL: https://github.com/apache/incubator-inlong/pull/1866#issuecomment-983237826
# [Codecov](https://codecov.io/gh/apache/incubator-inlong/pull/1866?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#1866](https://codecov.io/gh/apache/incubator-inlong/pull/1866?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (1a3e40e) into [master](https://codecov.io/gh/apache/incubator-inlong/commit/8b0bc7ae8cddc29fb85d05aea00ece629809c260?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (8b0bc7a) will **increase** coverage by `0.01%`.
> The diff coverage is `n/a`.
[](https://codecov.io/gh/apache/incubator-inlong/pull/1866?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #1866 +/- ##
============================================
+ Coverage 11.52% 11.53% +0.01%
+ Complexity 1051 1050 -1
============================================
Files 400 400
Lines 34789 34789
Branches 5504 5504
============================================
+ Hits 4009 4014 +5
+ Misses 30013 30009 -4
+ Partials 767 766 -1
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-inlong/pull/1866?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [.../inlong/tubemq/corebase/policies/FlowCtrlItem.java](https://codecov.io/gh/apache/incubator-inlong/pull/1866/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-aW5sb25nLXR1YmVtcS90dWJlbXEtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvaW5sb25nL3R1YmVtcS9jb3JlYmFzZS9wb2xpY2llcy9GbG93Q3RybEl0ZW0uamF2YQ==) | `38.88% <0.00%> (-1.12%)` | :arrow_down: |
| [.../tubemq/corebase/policies/FlowCtrlRuleHandler.java](https://codecov.io/gh/apache/incubator-inlong/pull/1866/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-aW5sb25nLXR1YmVtcS90dWJlbXEtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvaW5sb25nL3R1YmVtcS9jb3JlYmFzZS9wb2xpY2llcy9GbG93Q3RybFJ1bGVIYW5kbGVyLmphdmE=) | `34.07% <0.00%> (-0.45%)` | :arrow_down: |
| [.../producer/qltystats/DefaultBrokerRcvQltyStats.java](https://codecov.io/gh/apache/incubator-inlong/pull/1866/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-aW5sb25nLXR1YmVtcS90dWJlbXEtY2xpZW50L3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9pbmxvbmcvdHViZW1xL2NsaWVudC9wcm9kdWNlci9xbHR5c3RhdHMvRGVmYXVsdEJyb2tlclJjdlFsdHlTdGF0cy5qYXZh) | `45.70% <0.00%> (+0.39%)` | :arrow_up: |
| [.../java/org/apache/flume/sink/tubemq/TubemqSink.java](https://codecov.io/gh/apache/incubator-inlong/pull/1866/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-aW5sb25nLXR1YmVtcS90dWJlbXEtY29ubmVjdG9ycy90dWJlbXEtY29ubmVjdG9yLWZsdW1lL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9mbHVtZS9zaW5rL3R1YmVtcS9UdWJlbXFTaW5rLmphdmE=) | `55.42% <0.00%> (+4.00%)` | :arrow_up: |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-inlong/pull/1866?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/incubator-inlong/pull/1866?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [8b0bc7a...1a3e40e](https://codecov.io/gh/apache/incubator-inlong/pull/1866?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-inlong] dockerzhang commented on a change in pull request #1866: [INLONG-1712][Feature][Agent] Add agent metric statistics
Posted by GitBox <gi...@apache.org>.
dockerzhang commented on a change in pull request #1866:
URL: https://github.com/apache/incubator-inlong/pull/1866#discussion_r759921647
##########
File path: inlong-agent/conf/agent.properties
##########
@@ -101,3 +101,10 @@ channel.memory.capacity=5000
agent.manager.vip.http.host=
agent.manager.vip.http.port=
+
+
+############################
+# prometheus config
+############################
+# prometheus exporter server default port
+agent.prometheus.exporter.port=8080
Review comment:
@shink it's better to add a config to enable or disable expose Prometheus HTTP server.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-inlong] gosonzhang commented on pull request #1866: [INLONG-1712][Feature][Agent] Add agent Prometheus metric statistics
Posted by GitBox <gi...@apache.org>.
gosonzhang commented on pull request #1866:
URL: https://github.com/apache/incubator-inlong/pull/1866#issuecomment-988003602
Are you still paying attention to this issue?
Someone has suggested some amendments, please confirm them, thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-inlong] healchow commented on a change in pull request #1866: [INLONG-1712][Feature][Agent] Add agent metric statistics
Posted by GitBox <gi...@apache.org>.
healchow commented on a change in pull request #1866:
URL: https://github.com/apache/incubator-inlong/pull/1866#discussion_r759889482
##########
File path: inlong-agent/conf/agent.properties
##########
@@ -101,3 +101,10 @@ channel.memory.capacity=5000
agent.manager.vip.http.host=
agent.manager.vip.http.port=
+
+
+############################
+# prometheus config
+############################
+# prometheus exporter server default port
+agent.prometheus.exporter.port=8080
Review comment:
It is recommended to use some infrequently used ports.
##########
File path: inlong-agent/agent-common/src/main/java/org/apache/inlong/agent/constants/AgentConstants.java
##########
@@ -160,4 +160,7 @@
public static final String AGENT_LOCAL_IP = "agent.local.ip";
+ public static final String PROMETHEUS_EXPORTER_PORT = "agent.prometheus.exporter.port";
+ public static final int DEFAULT_PROMETHEUS_EXPORTER_PORT = 8080;
Review comment:
It is recommended to use some infrequently used ports.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-inlong] EMsnap removed a comment on pull request #1866: [INLONG-1712][Feature][Agent] Add agent Prometheus metric statistics
Posted by GitBox <gi...@apache.org>.
EMsnap removed a comment on pull request #1866:
URL: https://github.com/apache/incubator-inlong/pull/1866#issuecomment-984270609
Pls add some uts
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-inlong] dockerzhang commented on a change in pull request #1866: [INLONG-1712][Feature][Agent] Add agent Prometheus metric statistics
Posted by GitBox <gi...@apache.org>.
dockerzhang commented on a change in pull request #1866:
URL: https://github.com/apache/incubator-inlong/pull/1866#discussion_r764499964
##########
File path: inlong-agent/agent-common/src/main/java/org/apache/inlong/agent/constants/AgentConstants.java
##########
@@ -160,4 +160,10 @@
public static final String AGENT_LOCAL_IP = "agent.local.ip";
+ public static final String PROMETHEUS_ENABLE = "agent.prometheus.enable";
+ public static final boolean DEFAULT_PROMETHEUS_ENABLE = false;
+
Review comment:
what's difference about `PROMETHEUS_ENABLE` and `DEFAULT_PROMETHEUS_ENABLE`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-inlong] gosonzhang edited a comment on pull request #1866: [INLONG-1712][Feature][Agent] Add agent Prometheus metric statistics
Posted by GitBox <gi...@apache.org>.
gosonzhang edited a comment on pull request #1866:
URL: https://github.com/apache/incubator-inlong/pull/1866#issuecomment-988003602
@shink, are you still paying attention to this issue?
Someone has suggested some amendments, please confirm them, thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-inlong] gosonzhang merged pull request #1866: [INLONG-1712][Feature][Agent] Add agent Prometheus metric statistics
Posted by GitBox <gi...@apache.org>.
gosonzhang merged pull request #1866:
URL: https://github.com/apache/incubator-inlong/pull/1866
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-inlong] EMsnap commented on pull request #1866: [INLONG-1712][Feature][Agent] Add agent Prometheus metric statistics
Posted by GitBox <gi...@apache.org>.
EMsnap commented on pull request #1866:
URL: https://github.com/apache/incubator-inlong/pull/1866#issuecomment-984270609
Pls add some uts
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-inlong] dockerzhang commented on pull request #1866: [INLONG-1712][Feature][Agent] Add agent Prometheus metric statistics
Posted by GitBox <gi...@apache.org>.
dockerzhang commented on pull request #1866:
URL: https://github.com/apache/incubator-inlong/pull/1866#issuecomment-988494259
the agent has exposed many JMX metrics, it's better to reuse them.
https://inlong.apache.org/docs/next/modules/agent/overview#monitor-metrics-configuration
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org