You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Igor Dvorzhak (JIRA)" <ji...@apache.org> on 2017/12/22 02:30:00 UTC
[jira] [Comment Edited] (HADOOP-15124) Slow FileSystem.Statistics
counters implementation
[ https://issues.apache.org/jira/browse/HADOOP-15124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16300770#comment-16300770 ]
Igor Dvorzhak edited comment on HADOOP-15124 at 12/22/17 2:29 AM:
------------------------------------------------------------------
I have added a draft implementation of a property that disables per-thread statistics to the PR.
Here is performance profiling results (Statistics code paths) of latest changes:
|| || || ||TeraGen 1TB|| || || ||
|| || ||GCS|| || ||HDFS|| ||
| |No changes|LongAdder + Per-Thread|LongAdder|No changes|LongAdder + Per-Thread|LongAdder|
|CPU, ms|1,430|1,880|3.86|805.85|349.64|172.6|
|CPU, %|23.4|28.4|0.057|10.9|4.61|2.29|
|Wall, ms|11,420|1,900|3.79|11,270|503.7|242.83|
|Wall, %|6.45|1.11|0.002|6.34|0.303|0.146|
Legend:
* "No changes" - stock Hadoop
* "LongAdder + Per-Thread" - FS.Statistics with LongAdder backend and with per-thread statistic enabled
* "LongAdder" - FS.Statistics with LongAdder backend and with per-thread statistic disabled
Performance profiling results summary: this change improves performance in all tested cases except slight regression in CPU time for GCS with enabled per-thread statistics.
While change is not complete yet (needs tests, javadocs and proper property wiring), main functipnaity looks ready for review already.
What should I do to move this change forward? (I think it will be better to try to get consensus on it, before investing more time into finishing it, in case it will be rejected).
Also, I didn't figure out yet how to pass configuration/property value to FileSystem.Statistics class. Maybe you can advise here?
was (Author: medb):
I have added a draft implementation of a property that disables per-thread statistics to the PR.
Here is performance profiling results of latest changes:
|| || || ||TeraGen 1TB|| || || ||
|| || ||GCS|| || ||HDFS|| ||
| |No changes|LongAdder + Per-Thread|LongAdder|No changes|LongAdder + Per-Thread|LongAdder|
|CPU, ms|1,430|1,880|3.86|805.85|349.64|172.6|
|CPU, %|23.4|28.4|0.057|10.9|4.61|2.29|
|Wall, ms|11,420|1,900|3.79|11,270|503.7|242.83|
|Wall, %|6.45|1.11|0.002|6.34|0.303|0.146|
Legend:
* "No changes" - stock Hadoop
* "LongAdder + Per-Thread" - FS.Statistics with LongAdder backend and with per-thread statistic enabled
* "LongAdder" - FS.Statistics with LongAdder backend and with per-thread statistic disabled
Performance profiling results summary: this change improves performance in all tested cases except slight regression in CPU time for GCS with enabled per-thread statistics.
While change is not complete yet (needs tests, javadocs and proper property wiring), main functipnaity looks ready for review already.
What should I do to move this change forward? (I think it will be better to try to get consensus on it, before investing more time into finishing it, in case it will be rejected).
Also, I didn't figure out yet how to pass configuration/property value to FileSystem.Statistics class. Maybe you can advise here?
> Slow FileSystem.Statistics counters implementation
> --------------------------------------------------
>
> Key: HADOOP-15124
> URL: https://issues.apache.org/jira/browse/HADOOP-15124
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: common
> Affects Versions: 2.9.0, 2.8.3, 2.7.5, 3.0.0
> Reporter: Igor Dvorzhak
> Assignee: Igor Dvorzhak
> Labels: common, filesystem, statistics
>
> While profiling 1TB TeraGen job on Hadoop 2.8.2 cluster (Google Dataproc, 2 workers, GCS connector) I saw that FileSystem.Statistics code paths Wall time is 5.58% and CPU time is 26.5% of total execution time.
> After switching FileSystem.Statistics implementation to LongAdder, consumed Wall time decreased to 0.006% and CPU time to 0.104% of total execution time.
> Total job runtime decreased from 66 mins to 61 mins.
> These results are not conclusive, because I didn't benchmark multiple times to average results, but regardless of performance gains switching to LongAdder simplifies code and reduces its complexity.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org