You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by GitBox <gi...@apache.org> on 2021/03/23 09:12:14 UTC
[GitHub] [ozone] hgruck opened a new pull request #2074: HDDS-5010 Fixing Prometheus duplicated TYPE definition
hgruck opened a new pull request #2074:
URL: https://github.com/apache/ozone/pull/2074
## What changes were proposed in this pull request?
Currently, PrometheusMetricsSink adds **# TYPE** definition to each metrics line even though the metrics are repeatedly appearing in the endpoint due to differing in labels, e.g.
```
# TYPE hdds_dispatcher_counter counter
hdds_dispatcher_counter{type="PutBlock",hostname="gruck-ozone-1.gruck-ozone.root.hwx.site"} 0
# TYPE hdds_dispatcher_counter counter
hdds_dispatcher_counter{type="ReadContainer",hostname="gruck-ozone-1.gruck-ozone.root.hwx.site"} 0
```
Expected format is:
```
# TYPE hdds_dispatcher_counter counter
hdds_dispatcher_counter{type="PutBlock",hostname="gruck-ozone-1.gruck-ozone.root.hwx.site"} 0
hdds_dispatcher_counter{type="ReadContainer",hostname="gruck-ozone-1.gruck-ozone.root.hwx.site"} 0
```
## What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-5010
## How was this patch tested?
Unit test added + manually.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] elek commented on a change in pull request #2074: HDDS-5010 Fixing Prometheus duplicated TYPE definition
Posted by GitBox <gi...@apache.org>.
elek commented on a change in pull request #2074:
URL: https://github.com/apache/ozone/pull/2074#discussion_r599611346
##########
File path: hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/server/http/PrometheusMetricsSink.java
##########
@@ -64,43 +66,48 @@ public void putMetrics(MetricsRecord metricsRecord) {
String key = prometheusName(
metricsRecord.name(), metrics.name());
- StringBuilder builder = new StringBuilder();
- builder.append("# TYPE ")
- .append(key)
- .append(" ")
- .append(metrics.type().toString().toLowerCase())
- .append("\n");
-
- StringBuilder prometheusMetricKey = new StringBuilder();
- prometheusMetricKey.append(key)
- .append("{");
- String sep = "";
-
- //add tags
- for (MetricsTag tag : metricsRecord.tags()) {
- String tagName = tag.name().toLowerCase();
-
- //ignore specific tag which includes sub-hierarchy
- if (!tagName.equals("numopenconnectionsperuser")) {
- prometheusMetricKey.append(sep)
- .append(tagName)
- .append("=\"")
- .append(tag.value())
- .append("\"");
- sep = ",";
- }
+ String prometheusMetricKeyAsString =
+ getPrometheusMetricKeyAsString(metricsRecord, key);
+
+ String metricKey = "# TYPE "
+ + key
+ + " "
+ + metrics.type().toString().toLowerCase();
+
+ if (!metricLines.containsKey(metricKey)){
+ metricLines.put(metricKey, new TreeMap<>());
Review comment:
```suggestion
metricLines.put(metricKey, Collections.synchronizedSortedMap(new TreeMap<>()));
```
Fix me if I am wrong, but I think the sub maps should also be synchronized.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] elek merged pull request #2074: HDDS-5010 Fixing Prometheus duplicated TYPE definition
Posted by GitBox <gi...@apache.org>.
elek merged pull request #2074:
URL: https://github.com/apache/ozone/pull/2074
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] hgruck commented on a change in pull request #2074: HDDS-5010 Fixing Prometheus duplicated TYPE definition
Posted by GitBox <gi...@apache.org>.
hgruck commented on a change in pull request #2074:
URL: https://github.com/apache/ozone/pull/2074#discussion_r600418521
##########
File path: hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/server/http/PrometheusMetricsSink.java
##########
@@ -64,43 +66,48 @@ public void putMetrics(MetricsRecord metricsRecord) {
String key = prometheusName(
metricsRecord.name(), metrics.name());
- StringBuilder builder = new StringBuilder();
- builder.append("# TYPE ")
- .append(key)
- .append(" ")
- .append(metrics.type().toString().toLowerCase())
- .append("\n");
-
- StringBuilder prometheusMetricKey = new StringBuilder();
- prometheusMetricKey.append(key)
- .append("{");
- String sep = "";
-
- //add tags
- for (MetricsTag tag : metricsRecord.tags()) {
- String tagName = tag.name().toLowerCase();
-
- //ignore specific tag which includes sub-hierarchy
- if (!tagName.equals("numopenconnectionsperuser")) {
- prometheusMetricKey.append(sep)
- .append(tagName)
- .append("=\"")
- .append(tag.value())
- .append("\"");
- sep = ",";
- }
+ String prometheusMetricKeyAsString =
+ getPrometheusMetricKeyAsString(metricsRecord, key);
+
+ String metricKey = "# TYPE "
+ + key
+ + " "
+ + metrics.type().toString().toLowerCase();
+
+ if (!metricLines.containsKey(metricKey)){
+ metricLines.put(metricKey, new TreeMap<>());
Review comment:
Thanks a lot for the suggestion, I've made the changes based on @adoroszlai idea in a534cc2.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] elek commented on pull request #2074: HDDS-5010 Fixing Prometheus duplicated TYPE definition
Posted by GitBox <gi...@apache.org>.
elek commented on pull request #2074:
URL: https://github.com/apache/ozone/pull/2074#issuecomment-805998977
Merging it now, thanks the contribution @hgruck and the review @adoroszlai
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] hgruck commented on a change in pull request #2074: HDDS-5010 Fixing Prometheus duplicated TYPE definition
Posted by GitBox <gi...@apache.org>.
hgruck commented on a change in pull request #2074:
URL: https://github.com/apache/ozone/pull/2074#discussion_r600418739
##########
File path: hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/server/http/PrometheusMetricsSink.java
##########
@@ -64,43 +66,48 @@ public void putMetrics(MetricsRecord metricsRecord) {
String key = prometheusName(
metricsRecord.name(), metrics.name());
- StringBuilder builder = new StringBuilder();
- builder.append("# TYPE ")
- .append(key)
- .append(" ")
- .append(metrics.type().toString().toLowerCase())
- .append("\n");
-
- StringBuilder prometheusMetricKey = new StringBuilder();
- prometheusMetricKey.append(key)
- .append("{");
- String sep = "";
-
- //add tags
- for (MetricsTag tag : metricsRecord.tags()) {
- String tagName = tag.name().toLowerCase();
-
- //ignore specific tag which includes sub-hierarchy
- if (!tagName.equals("numopenconnectionsperuser")) {
- prometheusMetricKey.append(sep)
- .append(tagName)
- .append("=\"")
- .append(tag.value())
- .append("\"");
- sep = ",";
- }
+ String prometheusMetricKeyAsString =
+ getPrometheusMetricKeyAsString(metricsRecord, key);
+
+ String metricKey = "# TYPE "
+ + key
+ + " "
+ + metrics.type().toString().toLowerCase();
+
+ if (!metricLines.containsKey(metricKey)){
+ metricLines.put(metricKey, new TreeMap<>());
}
- prometheusMetricKey.append("}");
- String prometheusMetricKeyAsString = prometheusMetricKey.toString();
- builder.append(prometheusMetricKeyAsString);
- builder.append(" ");
- builder.append(metrics.value());
- builder.append("\n");
- metricLines.put(prometheusMetricKeyAsString, builder.toString());
+ metricLines.get(metricKey).put(prometheusMetricKeyAsString,
Review comment:
Good idea, I've added the changes in a534cc2.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] adoroszlai commented on a change in pull request #2074: HDDS-5010 Fixing Prometheus duplicated TYPE definition
Posted by GitBox <gi...@apache.org>.
adoroszlai commented on a change in pull request #2074:
URL: https://github.com/apache/ozone/pull/2074#discussion_r599628973
##########
File path: hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/server/http/PrometheusMetricsSink.java
##########
@@ -64,43 +66,48 @@ public void putMetrics(MetricsRecord metricsRecord) {
String key = prometheusName(
metricsRecord.name(), metrics.name());
- StringBuilder builder = new StringBuilder();
- builder.append("# TYPE ")
- .append(key)
- .append(" ")
- .append(metrics.type().toString().toLowerCase())
- .append("\n");
-
- StringBuilder prometheusMetricKey = new StringBuilder();
- prometheusMetricKey.append(key)
- .append("{");
- String sep = "";
-
- //add tags
- for (MetricsTag tag : metricsRecord.tags()) {
- String tagName = tag.name().toLowerCase();
-
- //ignore specific tag which includes sub-hierarchy
- if (!tagName.equals("numopenconnectionsperuser")) {
- prometheusMetricKey.append(sep)
- .append(tagName)
- .append("=\"")
- .append(tag.value())
- .append("\"");
- sep = ",";
- }
+ String prometheusMetricKeyAsString =
+ getPrometheusMetricKeyAsString(metricsRecord, key);
+
+ String metricKey = "# TYPE "
+ + key
+ + " "
+ + metrics.type().toString().toLowerCase();
+
+ if (!metricLines.containsKey(metricKey)){
+ metricLines.put(metricKey, new TreeMap<>());
}
- prometheusMetricKey.append("}");
- String prometheusMetricKeyAsString = prometheusMetricKey.toString();
- builder.append(prometheusMetricKeyAsString);
- builder.append(" ");
- builder.append(metrics.value());
- builder.append("\n");
- metricLines.put(prometheusMetricKeyAsString, builder.toString());
+ metricLines.get(metricKey).put(prometheusMetricKeyAsString,
Review comment:
Please use `computeIfAbsent` instead of `containsKey`, `put` and `get`. Only individual calls are synchronized, so the sequence may lead to unexpected results.
```java
metricLines.computeIfAbsent(metricKey, any -> synchronizedSortedMap(new TreeMap<>()))
.put(prometheusMetricKeyAsString, builder.toString());
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org