You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Daryn Sharp (JIRA)" <ji...@apache.org> on 2019/06/11 16:54:00 UTC
[jira] [Commented] (HADOOP-9570) Configuration.addResource() should
only parse the new resource
[ https://issues.apache.org/jira/browse/HADOOP-9570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16861213#comment-16861213 ]
Daryn Sharp commented on HADOOP-9570:
-------------------------------------
This is verbosely spewing warnings about final params. Would appreciate some help from a configuration guru.
> Configuration.addResource() should only parse the new resource
> --------------------------------------------------------------
>
> Key: HADOOP-9570
> URL: https://issues.apache.org/jira/browse/HADOOP-9570
> Project: Hadoop Common
> Issue Type: Improvement
> Components: conf
> Environment: Ubuntu LXC
> Reporter: Gopal V
> Assignee: Daryn Sharp
> Priority: Minor
> Attachments: HADOOP-9570.branch-2.8.patch, HADOOP-9750.patch
>
>
> Hadoop configuration parsing re-parses all configuration files when a new resource is added to the configuration object.
> This is wasteful and runs through every deprecation check unnecessarily.
> {code}
> JobConf clone = new JobConf(job);
> clone.addResource(...);
> {code}
> will reparse every file including core-site, hdfs-site etc.
> Resource addition can be taken care of cleanly, if the addResourceObject() call applies the new resource, followed by applying the overlay, without re-parsing any of the older files already loaded into the hashtable.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org