You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "xichen01 (via GitHub)" <gi...@apache.org> on 2023/11/08 11:17:55 UTC
Re: [PR] HDDS-9522. Add percentile for ProtocolMessageMetrics [ozone]
xichen01 commented on code in PR #5479:
URL: https://github.com/apache/ozone/pull/5479#discussion_r1386451752
##########
hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/ProtocolMessageMetrics.java:
##########
@@ -89,6 +126,7 @@ public void unregister() {
@Override
public void getMetrics(MetricsCollector collector, boolean all) {
+ registry.snapshot(collector.addRecord(registry.info()), all);
Review Comment:
It is better to put the type of operation in the tag rather than the name of the metric, as with other metrics. This makes it easier to write SQL for monitoring such as Grafana.
```
om_client_protocol_message_metrics_time15s50th_percentile_latency{hostname="xxx", type="CreateKey"} 0
```
##########
hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/ProtocolMessageMetrics.java:
##########
@@ -40,32 +44,59 @@ public class ProtocolMessageMetrics<KEY> implements MetricsSource {
private final String description;
+ private final MetricsRegistry registry;
+
+ private final boolean quantileEnable;
+
private final Map<KEY, AtomicLong> counters =
new ConcurrentHashMap<>();
private final Map<KEY, AtomicLong> elapsedTimes =
new ConcurrentHashMap<>();
+ private final Map<KEY, MutableQuantiles[]> quantiles =
+ new ConcurrentHashMap<>();
+
private final AtomicInteger concurrency = new AtomicInteger(0);
public static <KEY> ProtocolMessageMetrics<KEY> create(String name,
- String description, KEY[] types) {
- return new ProtocolMessageMetrics<KEY>(name, description, types);
+ String description, KEY[] types, ConfigurationSource conf) {
+ return new ProtocolMessageMetrics<KEY>(name, description, types, conf);
}
public ProtocolMessageMetrics(String name, String description,
- KEY[] values) {
+ KEY[] values, ConfigurationSource conf) {
this.name = name;
this.description = description;
+ registry = new MetricsRegistry(name + "MessageMetrics");
+ int[] intervals = conf.getInts(
+ OzoneConfigKeys.OZONE_PROTOCOL_MESSAGE_METRICS_PERCENTILES_INTERVALS);
+ quantileEnable = (intervals.length > 0);
for (KEY value : values) {
counters.put(value, new AtomicLong(0));
elapsedTimes.put(value, new AtomicLong(0));
+ if (quantileEnable) {
+ MutableQuantiles[] mutableQuantiles =
+ new MutableQuantiles[intervals.length];
+ quantiles.put(value, mutableQuantiles);
+ for (int i = 0; i < intervals.length; i++) {
+ mutableQuantiles[i] = registry.newQuantiles(
+ value.toString() + "RpcTime" + intervals[i] + "s",
+ value.toString() + "rpc time in milli second",
+ "ops", "latency", intervals[i]);
Review Comment:
It would be better to clarify the unit of time in the name of the Metrics, such as Use `latencyMS`
##########
hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/ProtocolMessageMetrics.java:
##########
@@ -40,32 +44,59 @@ public class ProtocolMessageMetrics<KEY> implements MetricsSource {
private final String description;
+ private final MetricsRegistry registry;
+
+ private final boolean quantileEnable;
+
private final Map<KEY, AtomicLong> counters =
new ConcurrentHashMap<>();
private final Map<KEY, AtomicLong> elapsedTimes =
new ConcurrentHashMap<>();
+ private final Map<KEY, MutableQuantiles[]> quantiles =
+ new ConcurrentHashMap<>();
+
private final AtomicInteger concurrency = new AtomicInteger(0);
public static <KEY> ProtocolMessageMetrics<KEY> create(String name,
- String description, KEY[] types) {
- return new ProtocolMessageMetrics<KEY>(name, description, types);
+ String description, KEY[] types, ConfigurationSource conf) {
+ return new ProtocolMessageMetrics<KEY>(name, description, types, conf);
}
public ProtocolMessageMetrics(String name, String description,
Review Comment:
The `ProtocolMessageMetrics` constructor can be made private, and we have a `create` for creating `ProtocolMessageMetrics`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org