You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "aierate (via GitHub)" <gi...@apache.org> on 2024/01/23 04:07:32 UTC
[PR] HDDS-10182. Add P99 quantiles and Min/Max Metrics for Xceiver Client Performance Metrics for S3G and Client [ozone]
aierate opened a new pull request, #6070:
URL: https://github.com/apache/ozone/pull/6070
## What changes were proposed in this pull request?
- Use the AggregatedMetrics replica the MutableRate XceiverClient Metrics of XceiverClientMetrics for Client and S3G.
- The AggregatedMetrics including MutableQuantiles and MutableMinMax and MutableStat Metrics, the MutableStat is same with original MutableRate Metrics.
- Replacing MutableRate with AggregatedMetrics ensures that the original MutableRate remains unchanged, and adds configurable MutableQuantiles and MutableMinMax metrics.
- Reference from the following linkļ¼
https://github.com/apache/ozone/pull/5627
### Before
http://ozone:9878/jmx?qry=Hadoop:service=S3Gateway,name=XceiverClientMetrics
```
{
"name" : "Hadoop:service=S3Gateway,name=XceiverClientMetrics",
"modelerType" : "XceiverClientMetrics",
"tag.Context" : "dfs",
"tag.Hostname" : "conway-hadoop3",
"numPendingCreateContainer" : 0,
"opCountCreateContainer" : 0,
"CreateContainerLatencyNumOps" : 0,
"CreateContainerLatencyAvgTime" : 0.0,
.
.
.
"opCountWriteChunk" : 0,
"WriteChunkLatencyNumOps" : 0,
"WriteChunkLatencyAvgTime" : 0,
.
.
.
"numPendingStreamWrite" : 0,
"opCountStreamWrite" : 0,
"StreamWriteLatencyNumOps" : 0,
"StreamWriteLatencyAvgTime" : 0.0,
"EcReconstructionFailsTotal" : 0,
"EcReconstructionTotal" : 0,
"PendingOps" : 0,
"TotalOps" : 0
},
```
### After
http://ozone:9878/jmx?qry=Hadoop:service=S3Gateway,name=XceiverClientMetrics
Default (Add MutableMinMax Metrics):
```
{
"name" : "Hadoop:service=S3Gateway,name=XceiverClientMetrics",
"modelerType" : "XceiverClientMetrics",
"tag.Context" : "dfs",
"tag.Hostname" : "conway-hadoop3",
"numPendingCreateContainer" : 0,
"opCountCreateContainer" : 0,
"CreateContainerLatencyNumOps" : 0,
"CreateContainerLatencyAvgTime" : 0.0,
"CreateContainerLatencyIMinTime" : 0,
"CreateContainerLatencyIMaxTime" : 0,
.
.
.
"opCountWriteChunk" : 1,
"WriteChunkLatencyNumOps" : 1,
"WriteChunkLatencyAvgTime" : 701.0,
"WriteChunkLatencyIMinTime" : 0,
"WriteChunkLatencyIMaxTime" : 0,
.
.
.
"numPendingStreamWrite" : 0,
"opCountStreamWrite" : 0,
"StreamWriteLatencyNumOps" : 0,
"StreamWriteLatencyAvgTime" : 0.0,
"StreamWriteLatencyIMinTime" : 0,
"StreamWriteLatencyIMaxTime" : 0,
"EcReconstructionFailsTotal" : 0,
"EcReconstructionTotal" : 0,
"PendingOps" : 0,
"TotalOps" : 2
},
```
Enable quantiles Metrics:
```
<property>
<name>ozone.xceiver.metrics.percentiles.intervals.seconds</name>
<value>60,300</value>
</property>
```
http://ozone:9878/jmx?qry=Hadoop:service=S3Gateway,name=XceiverClientMetrics
```
{
"name" : "Hadoop:service=S3Gateway,name=XceiverClientMetrics",
"modelerType" : "XceiverClientMetrics",
"tag.Context" : "dfs",
"tag.Hostname" : "conway-hadoop3",
"numPendingCreateContainer" : 0,
"opCountCreateContainer" : 0,
"CreateContainerLatencyNumOps" : 0,
"CreateContainerLatencyAvgTime" : 0.0,
"CreateContainerLatency60sNumOps" : 0,
"CreateContainerLatency60s50thPercentileTime" : 0,
"CreateContainerLatency60s75thPercentileTime" : 0,
"CreateContainerLatency60s90thPercentileTime" : 0,
"CreateContainerLatency60s95thPercentileTime" : 0,
"CreateContainerLatency60s99thPercentileTime" : 0,
"CreateContainerLatency300sNumOps" : 0,
"CreateContainerLatency300s50thPercentileTime" : 0,
"CreateContainerLatency300s75thPercentileTime" : 0,
"CreateContainerLatency300s90thPercentileTime" : 0,
"CreateContainerLatency300s95thPercentileTime" : 0,
"CreateContainerLatency300s99thPercentileTime" : 0,
.
.
.
"numPendingWriteChunk" : 0,
"opCountWriteChunk" : 1,
"WriteChunkLatencyNumOps" : 1,
"WriteChunkLatencyAvgTime" : 722.0,
"WriteChunkLatency60sNumOps" : 1,
"WriteChunkLatency60s50thPercentileTime" : 722,
"WriteChunkLatency60s75thPercentileTime" : 722,
"WriteChunkLatency60s90thPercentileTime" : 722,
"WriteChunkLatency60s95thPercentileTime" : 722,
"WriteChunkLatency60s99thPercentileTime" : 722,
"WriteChunkLatency300sNumOps" : 0,
"WriteChunkLatency300s50thPercentileTime" : 0,
"WriteChunkLatency300s75thPercentileTime" : 0,
"WriteChunkLatency300s90thPercentileTime" : 0,
"WriteChunkLatency300s95thPercentileTime" : 0,
"WriteChunkLatency300s99thPercentileTime" : 0,
"WriteChunkLatencyIMinTime" : 0,
"WriteChunkLatencyIMaxTime" : 0,
.
.
.
"numPendingStreamWrite" : 0,
"opCountStreamWrite" : 0,
"StreamWriteLatencyNumOps" : 0,
"StreamWriteLatencyAvgTime" : 0.0,
"StreamWriteLatency60sNumOps" : 0,
"StreamWriteLatency60s50thPercentileTime" : 0,
"StreamWriteLatency60s75thPercentileTime" : 0,
"StreamWriteLatency60s90thPercentileTime" : 0,
"StreamWriteLatency60s95thPercentileTime" : 0,
"StreamWriteLatency60s99thPercentileTime" : 0,
"StreamWriteLatency300sNumOps" : 0,
"StreamWriteLatency300s50thPercentileTime" : 0,
"StreamWriteLatency300s75thPercentileTime" : 0,
"StreamWriteLatency300s90thPercentileTime" : 0,
"StreamWriteLatency300s95thPercentileTime" : 0,
"StreamWriteLatency300s99thPercentileTime" : 0,
"StreamWriteLatencyIMinTime" : 0,
"StreamWriteLatencyIMaxTime" : 0,
"EcReconstructionFailsTotal" : 0,
"EcReconstructionTotal" : 0,
"PendingOps" : 0,
"TotalOps" : 2
}
```
## What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-10182
## How was this patch tested?
Manually Test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-10182. Add P99 quantiles and Min/Max Metrics for Xceiver Client Performance Metrics [ozone]
Posted by "xichen01 (via GitHub)" <gi...@apache.org>.
xichen01 merged PR #6070:
URL: https://github.com/apache/ozone/pull/6070
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-10182. Add P99 quantiles and Min/Max Metrics for Xceiver Client Performance Metrics [ozone]
Posted by "aierate (via GitHub)" <gi...@apache.org>.
aierate commented on PR #6070:
URL: https://github.com/apache/ozone/pull/6070#issuecomment-1916434440
@xBis7 Can you help me review the code? Thank you.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-10182. Add P99 quantiles and Min/Max Metrics for Xceiver Client Performance Metrics [ozone]
Posted by "xichen01 (via GitHub)" <gi...@apache.org>.
xichen01 commented on PR #6070:
URL: https://github.com/apache/ozone/pull/6070#issuecomment-2006119482
Thanks @aierate for the patch, @kerneltime @xichen01 for the review.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-10182. Add P99 quantiles and Min/Max Metrics for Xceiver Client Performance Metrics [ozone]
Posted by "adoroszlai (via GitHub)" <gi...@apache.org>.
adoroszlai commented on PR #6070:
URL: https://github.com/apache/ozone/pull/6070#issuecomment-1921351191
@duongkame @kerneltime @tanvipenumudy please take a look
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-10182. Add P99 quantiles and Min/Max Metrics for Xceiver Client Performance Metrics [ozone]
Posted by "adoroszlai (via GitHub)" <gi...@apache.org>.
adoroszlai commented on code in PR #6070:
URL: https://github.com/apache/ozone/pull/6070#discussion_r1464849577
##########
hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/OzoneConfigKeys.java:
##########
@@ -675,6 +675,8 @@ public final class OzoneConfigKeys {
public static final String HDDS_SCM_CLIENT_FAILOVER_MAX_RETRY =
"hdds.scmclient.failover.max.retry";
+ public static final String OZONE_XCEIVER_METRICS_PERCENTILES_INTERVALS_SECONDS_KEY =
+ "ozone.xceiver.metrics.percentiles.intervals.seconds";
Review Comment:
`xceiver` is an "internal" name, it may be very obscure for admins. No other configuration key mentions it.
`XceiverClientMetrics` is described as `"Storage Container Client Metrics"`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-10182. Add P99 quantiles and Min/Max Metrics for Xceiver Client Performance Metrics [ozone]
Posted by "xichen01 (via GitHub)" <gi...@apache.org>.
xichen01 commented on code in PR #6070:
URL: https://github.com/apache/ozone/pull/6070#discussion_r1474063266
##########
hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/OzoneConfigKeys.java:
##########
@@ -675,6 +675,8 @@ public final class OzoneConfigKeys {
public static final String HDDS_SCM_CLIENT_FAILOVER_MAX_RETRY =
"hdds.scmclient.failover.max.retry";
+ public static final String OZONE_XCEIVER_METRICS_PERCENTILES_INTERVALS_SECONDS_KEY =
+ "ozone.xceiver.metrics.percentiles.intervals.seconds";
Review Comment:
@aierate suggest: `ozone.xceiver.client.metrics.percentiles.intervals.seconds`, how do you think?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-10182. Add P99 quantiles and Min/Max Metrics for Xceiver Client Performance Metrics [ozone]
Posted by "aierate (via GitHub)" <gi...@apache.org>.
aierate commented on PR #6070:
URL: https://github.com/apache/ozone/pull/6070#issuecomment-1909319571
> @tanvipenumudy Thanks for you Patch.
>
> You need to have `XceiverClientMetrics` implement `MetricsSource` and override `getMetrics`. just like:
>
> ```java
> public final class XceiverClientMetrics implements MetricsSource {
> @Override
> public void (MetricsCollector collector, boolean all) {
> //....
> }
> }
> ```
>
> This is because Hadoop Metrics only supports the automatic retrieval of values from Hadoop's built-in classes. `MutableMinMax` is a custom class, so if `getMetrics` is not overridden, then `MutableMinMax` will not be updated. You can test that the `Min/Max` metric will always be 0 in the current implementation.
Thanks for your suggestion very much. I am fixed this bug, please take a look.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-10182. Add P99 quantiles and Min/Max Metrics for Xceiver Client Performance Metrics [ozone]
Posted by "xichen01 (via GitHub)" <gi...@apache.org>.
xichen01 commented on PR #6070:
URL: https://github.com/apache/ozone/pull/6070#issuecomment-1908287090
@tanvipenumudy Thanks for you Patch.
You need to have `XceiverClientMetrics` implement `MetricsSource` and override `getMetrics`.
just like:
```java
public final class XceiverClientMetrics implements MetricsSource {
@Override
public void (MetricsCollector collector, boolean all) {
//....
}
}
```
This is because Hadoop Metrics only supports the automatic retrieval of values from Hadoop's built-in classes. `MutableMinMax` is a custom class, so if `getMetrics` is not overridden, then `MutableMinMax` will not be updated.
You can test that the `Min/Max` metric will always be 0 in the current implementation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-10182. Add P99 quantiles and Min/Max Metrics for Xceiver Client Performance Metrics [ozone]
Posted by "xichen01 (via GitHub)" <gi...@apache.org>.
xichen01 commented on code in PR #6070:
URL: https://github.com/apache/ozone/pull/6070#discussion_r1473078016
##########
hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/OzoneConfigKeys.java:
##########
@@ -675,6 +675,8 @@ public final class OzoneConfigKeys {
public static final String HDDS_SCM_CLIENT_FAILOVER_MAX_RETRY =
"hdds.scmclient.failover.max.retry";
+ public static final String OZONE_XCEIVER_METRICS_PERCENTILES_INTERVALS_SECONDS_KEY =
+ "ozone.xceiver.metrics.percentiles.intervals.seconds";
Review Comment:
@adoroszlai. Unfortunately the metric name is already `XceiverClientMetrics`
https://github.com/apache/ozone/blob/e89fee375efb1aaeb59978c1c708b6808e38b267/hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/XceiverClientMetrics.java#L37
So `ozone.xceiver.metrics.xxx` seems a more direct name. How do you think?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-10182. Add P99 quantiles and Min/Max Metrics for Xceiver Client Performance Metrics [ozone]
Posted by "xichen01 (via GitHub)" <gi...@apache.org>.
xichen01 commented on PR #6070:
URL: https://github.com/apache/ozone/pull/6070#issuecomment-1921067424
LGTM
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-10182. Add P99 quantiles and Min/Max Metrics for Xceiver Client Performance Metrics [ozone]
Posted by "adoroszlai (via GitHub)" <gi...@apache.org>.
adoroszlai commented on PR #6070:
URL: https://github.com/apache/ozone/pull/6070#issuecomment-1908048030
@tanvipenumudy please take a look
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-10182. Add P99 quantiles and Min/Max Metrics for Xceiver Client Performance Metrics [ozone]
Posted by "adoroszlai (via GitHub)" <gi...@apache.org>.
adoroszlai commented on code in PR #6070:
URL: https://github.com/apache/ozone/pull/6070#discussion_r1473303508
##########
hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/OzoneConfigKeys.java:
##########
@@ -675,6 +675,8 @@ public final class OzoneConfigKeys {
public static final String HDDS_SCM_CLIENT_FAILOVER_MAX_RETRY =
"hdds.scmclient.failover.max.retry";
+ public static final String OZONE_XCEIVER_METRICS_PERCENTILES_INTERVALS_SECONDS_KEY =
+ "ozone.xceiver.metrics.percentiles.intervals.seconds";
Review Comment:
Thanks @xichen01 for the info. If this appears as part of metric names, then I guess the proposed config key is fine.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-10182. Add P99 quantiles and Min/Max Metrics for Xceiver Client Performance Metrics [ozone]
Posted by "aierate (via GitHub)" <gi...@apache.org>.
aierate commented on PR #6070:
URL: https://github.com/apache/ozone/pull/6070#issuecomment-1905840208
@xichen01 Can you help me review the code? Thank you.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-10182. Add P99 quantiles and Min/Max Metrics for Xceiver Client Performance Metrics [ozone]
Posted by "adoroszlai (via GitHub)" <gi...@apache.org>.
adoroszlai commented on code in PR #6070:
URL: https://github.com/apache/ozone/pull/6070#discussion_r1470922730
##########
hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/OzoneConfigKeys.java:
##########
@@ -675,6 +675,8 @@ public final class OzoneConfigKeys {
public static final String HDDS_SCM_CLIENT_FAILOVER_MAX_RETRY =
"hdds.scmclient.failover.max.retry";
+ public static final String OZONE_XCEIVER_METRICS_PERCENTILES_INTERVALS_SECONDS_KEY =
+ "ozone.xceiver.metrics.percentiles.intervals.seconds";
Review Comment:
@aierate I still think this config key should be renamed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-10182. Add P99 quantiles and Min/Max Metrics for Xceiver Client Performance Metrics [ozone]
Posted by "aierate (via GitHub)" <gi...@apache.org>.
aierate commented on PR #6070:
URL: https://github.com/apache/ozone/pull/6070#issuecomment-1909342492
> > @tanvipenumudy Thanks for you Patch.
> > You need to have `XceiverClientMetrics` implement `MetricsSource` and override `getMetrics`. just like:
> > ```java
> > public final class XceiverClientMetrics implements MetricsSource {
> > @Override
> > public void (MetricsCollector collector, boolean all) {
> > //....
> > }
> > }
> > ```
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > This is because Hadoop Metrics only supports the automatic retrieval of values from Hadoop's built-in classes. `MutableMinMax` is a custom class, so if `getMetrics` is not overridden, then `MutableMinMax` will not be updated. You can test that the `Min/Max` metric will always be 0 in the current implementation.
>
> @xichen01 Thanks for your suggestion very much. I am fixed this bug, please take a look.
Here is the jmx after fixed this bug.
http://ozone:9878/jmx?qry=Hadoop:service=S3Gateway,name=XceiverClientMetrics
```
{
"name" : "Hadoop:service=S3Gateway,name=XceiverClientMetrics",
"modelerType" : "XceiverClientMetrics",
"tag.Context" : "dfs",
"tag.Hostname" : "conway-hadoop3",
"PendingOps" : 0,
"TotalOps" : 4,
"EcReconstructionTotal" : 0,
"EcReconstructionFailsTotal" : 0,
"numPendingCreateContainer" : 0,
"opCountCreateContainer" : 0,
"CreateContainerLatencyNumOps" : 0,
"CreateContainerLatencyAvgTime" : 0.0,
"CreateContainerLatency60sNumOps" : 0,
"CreateContainerLatency60s50thPercentileTime" : 0,
"CreateContainerLatency60s75thPercentileTime" : 0,
"CreateContainerLatency60s90thPercentileTime" : 0,
"CreateContainerLatency60s95thPercentileTime" : 0,
"CreateContainerLatency60s99thPercentileTime" : 0,
"CreateContainerLatency300sNumOps" : 0,
"CreateContainerLatency300s50thPercentileTime" : 0,
"CreateContainerLatency300s75thPercentileTime" : 0,
"CreateContainerLatency300s90thPercentileTime" : 0,
"CreateContainerLatency300s95thPercentileTime" : 0,
"CreateContainerLatency300s99thPercentileTime" : 0,
"CreateContainerLatencyIMinTime" : 3.4028234663852886E38,
"CreateContainerLatencyIMaxTime" : 1.401298464324817E-45,
.
.
.
"numPendingPutBlock" : 0,
"opCountPutBlock" : 2,
"PutBlockLatencyNumOps" : 2,
"PutBlockLatencyAvgTime" : 30.0,
"PutBlockLatency60sNumOps" : 1,
"PutBlockLatency60s50thPercentileTime" : 30,
"PutBlockLatency60s75thPercentileTime" : 30,
"PutBlockLatency60s90thPercentileTime" : 30,
"PutBlockLatency60s95thPercentileTime" : 30,
"PutBlockLatency60s99thPercentileTime" : 30,
"PutBlockLatency300sNumOps" : 0,
"PutBlockLatency300s50thPercentileTime" : 0,
"PutBlockLatency300s75thPercentileTime" : 0,
"PutBlockLatency300s90thPercentileTime" : 0,
"PutBlockLatency300s95thPercentileTime" : 0,
"PutBlockLatency300s99thPercentileTime" : 0,
"PutBlockLatencyIMinTime" : 30.0,
"PutBlockLatencyIMaxTime" : 30.0,
.
.
.
"numPendingStreamWrite" : 0,
"opCountStreamWrite" : 0,
"StreamWriteLatencyNumOps" : 0,
"StreamWriteLatencyAvgTime" : 0.0,
"StreamWriteLatency60sNumOps" : 0,
"StreamWriteLatency60s50thPercentileTime" : 0,
"StreamWriteLatency60s75thPercentileTime" : 0,
"StreamWriteLatency60s90thPercentileTime" : 0,
"StreamWriteLatency60s95thPercentileTime" : 0,
"StreamWriteLatency60s99thPercentileTime" : 0,
"StreamWriteLatency300sNumOps" : 0,
"StreamWriteLatency300s50thPercentileTime" : 0,
"StreamWriteLatency300s75thPercentileTime" : 0,
"StreamWriteLatency300s90thPercentileTime" : 0,
"StreamWriteLatency300s95thPercentileTime" : 0,
"StreamWriteLatency300s99thPercentileTime" : 0,
"StreamWriteLatencyIMinTime" : 3.4028234663852886E38,
"StreamWriteLatencyIMaxTime" : 1.401298464324817E-45
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-10182. Add P99 quantiles and Min/Max Metrics for Xceiver Client Performance Metrics [ozone]
Posted by "aierate (via GitHub)" <gi...@apache.org>.
aierate commented on PR #6070:
URL: https://github.com/apache/ozone/pull/6070#issuecomment-1909938984
All unit tests have passed. https://github.com/aierate/ozone/actions
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-10182. Add P99 quantiles and Min/Max Metrics for Xceiver Client Performance Metrics [ozone]
Posted by "aierate (via GitHub)" <gi...@apache.org>.
aierate commented on code in PR #6070:
URL: https://github.com/apache/ozone/pull/6070#discussion_r1470972880
##########
hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/OzoneConfigKeys.java:
##########
@@ -675,6 +675,8 @@ public final class OzoneConfigKeys {
public static final String HDDS_SCM_CLIENT_FAILOVER_MAX_RETRY =
"hdds.scmclient.failover.max.retry";
+ public static final String OZONE_XCEIVER_METRICS_PERCENTILES_INTERVALS_SECONDS_KEY =
+ "ozone.xceiver.metrics.percentiles.intervals.seconds";
Review Comment:
Thanks @adoroszlai , I think it can be rename to OZONE_STORANG_CONTAINER_CLIENT_METRICS_PERCENTILES_INTERVALS_SECONDS_KEY, what do you think?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-10182. Add P99 quantiles and Min/Max Metrics for Xceiver Client Performance Metrics [ozone]
Posted by "adoroszlai (via GitHub)" <gi...@apache.org>.
adoroszlai commented on code in PR #6070:
URL: https://github.com/apache/ozone/pull/6070#discussion_r1464849577
##########
hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/OzoneConfigKeys.java:
##########
@@ -675,6 +675,8 @@ public final class OzoneConfigKeys {
public static final String HDDS_SCM_CLIENT_FAILOVER_MAX_RETRY =
"hdds.scmclient.failover.max.retry";
+ public static final String OZONE_XCEIVER_METRICS_PERCENTILES_INTERVALS_SECONDS_KEY =
+ "ozone.xceiver.metrics.percentiles.intervals.seconds";
Review Comment:
`xceiver` is an "internal" name, it may be very obscure for admins. No other configuration key mentions it.
`XceiverClientMetrics` is described as `"Storage Container Client Metrics"`.
I don't really know what would be a good name for this config key. Please feel free to discuss with other reviewers.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org