[jira] [Commented] (HADOOP-13747) Use LongAdder for more efficient metrics tracking

["Erik Krogen (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HADOOP-13747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15665789#comment-15665789 ] 

Erik Krogen commented on HADOOP-13747:
--------------------------------------

Since HADOOP-13782 created the new {{MutableRatesWithAggregation}} class, which provides high throughput for a collection of {{MutableRate}}-style metrics under contention through the use of thread-local metric collection with aggregation, {{RpcDetailedMetrics}} and the {{FSNamesystemLock}} metrics (introduced in HDFS-10872) are no longer affected by this change.

The {{LongAdder}} change proposed here could still be useful in situations where a single {{MutableRate}} or {{MutableCounter}} is accessed by many threads, e.g. many of the metrics within {{RpcMetrics}}.

> Use LongAdder for more efficient metrics tracking
> -------------------------------------------------
>
>                 Key: HADOOP-13747
>                 URL: https://issues.apache.org/jira/browse/HADOOP-13747
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: metrics
>            Reporter: Zhe Zhang
>            Assignee: Erik Krogen
>         Attachments: HADOOP-13747.patch, benchmark_results
>
>
> Currently many metrics, including {{RpcMetrics}} and {{RpcDetailedMetrics}}, use a synchronized counter to be updated by all handler threads (multiple hundreds in large production clusters). As [~andrew.wang] suggested, it'd be more efficient to use the [LongAdder | http://gee.cs.oswego.edu/cgi-bin/viewcvs.cgi/jsr166/src/jsr166e/LongAdder.java?view=co] library which dynamically create intermediate-result variables.
> Assigning to [~xkrogen] who has already done some investigation on this.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org