[jira] [Updated] (HADOOP-13782) Make MutableRates metrics thread-local write, aggregate-on-read

["Erik Krogen (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HADOOP-13782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Erik Krogen updated HADOOP-13782:
---------------------------------
    Attachment: HADOOP-13782.000.patch

Attaching v000 patch.

We have decided to create a new class, {{MutableRatesWithAggregation}}, rather than modify the existing {{MutableRates}} metrics class since it is an {{Evolving}} interface which can only be changed at a minor version bump, to be able to backport to older versions. I have aded a comment on {{MutableRates}} noting that its performance in high contention situations will be lacking in comparison.

I have updated {{RpcDetailedMetrics}}, the only consumer of {{MutableRates}}, to use {{MutableRatesWithAggregation}} instead since it is a point of potentially very high contention. 



> Make MutableRates metrics thread-local write, aggregate-on-read
> ---------------------------------------------------------------
>
>                 Key: HADOOP-13782
>                 URL: https://issues.apache.org/jira/browse/HADOOP-13782
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: metrics
>            Reporter: Erik Krogen
>            Assignee: Erik Krogen
>         Attachments: HADOOP-13782.000.patch
>
>
> Currently the {{MutableRates}} metrics class serializes all writes to metrics it contains because of its use of {{MetricsRegistry.add()}} (i.e., even two increments of unrelated metrics contained within the same {{MutableRates}} object will serialize w.r.t. each other). This class is used by {{RpcDetailedMetrics}}, which may have many hundreds of threads contending to modify these metrics. Instead we should allow updates to unrelated metrics objects to happen concurrently. To do so we can let each thread locally collect metrics, and on a {{snapshot}}, aggregate the metrics from all of the threads. 
> I have collected some benchmark performance numbers in HADOOP-13747 (https://issues.apache.org/jira/secure/attachment/12835043/benchmark_results) which indicate that this can bring significantly higher performance in high contention situations. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org