You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Wei-Chiu Chuang (JIRA)" <ji...@apache.org> on 2018/06/21 17:04:00 UTC
[jira] [Comment Edited] (HADOOP-15549) Upgrade to
commons-configuration 2.1 regresses task CPU consumption
[ https://issues.apache.org/jira/browse/HADOOP-15549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16519579#comment-16519579 ]
Wei-Chiu Chuang edited comment on HADOOP-15549 at 6/21/18 5:03 PM:
-------------------------------------------------------------------
+1 I reviewed HADOOP-13660 so I take responsibility for that change :)
Verified manually properties are loaded properly, and also verified the same usage is limited to this file only. This change is more similar to pre-HADOOP-13660 so I feel safe about it.
was (Author: jojochuang):
+1 I reviewed HADOOP-13660 so I take responsibility for that change :)
Verified manually properties are loaded properly. This change is more similar to pre-HADOOP-13660 so I feel safe about it.
> Upgrade to commons-configuration 2.1 regresses task CPU consumption
> -------------------------------------------------------------------
>
> Key: HADOOP-15549
> URL: https://issues.apache.org/jira/browse/HADOOP-15549
> Project: Hadoop Common
> Issue Type: Bug
> Components: metrics
> Affects Versions: 3.0.2
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Priority: Major
> Attachments: hadoop-15549.txt
>
>
> HADOOP-13660 upgraded from commons-configuration 1.x to 2.x. commons-configuration is used when parsing the metrics configuration properties file. The new builder API used in the new version apparently makes use of a bunch of very bloated reflection and classloading nonsense to achieve the same goal, and this results in a regression of >100ms of CPU time as measured by a program which simply initializes DefaultMetricsSystem.
> This isn't a big deal for long-running daemons, but for MR tasks which might only run a few seconds on poorly-tuned jobs, this can be noticeable.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org