You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Szilard Nemeth (JIRA)" <ji...@apache.org> on 2018/08/31 08:44:00 UTC
[jira] [Updated] (HADOOP-15708) Reading values from Configuration
before adding deprecations make it impossible to read value with deprecated
key
[ https://issues.apache.org/jira/browse/HADOOP-15708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Szilard Nemeth updated HADOOP-15708:
------------------------------------
Attachment: HADOOP-15708-testcase.patch
> Reading values from Configuration before adding deprecations make it impossible to read value with deprecated key
> -----------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-15708
> URL: https://issues.apache.org/jira/browse/HADOOP-15708
> Project: Hadoop Common
> Issue Type: Bug
> Reporter: Szilard Nemeth
> Assignee: Szilard Nemeth
> Priority: Major
> Attachments: HADOOP-15708-testcase.patch
>
>
> Hadoop Common contains a widely used Configuration class.
> This class can handle deprecations of properties, e.g. if property 'A' gets deprecated with an alternative property key 'B', users can access property values with keys 'A' and 'B'.
> Unfortunately, this does not work in one case.
> When a config file is specified (for instance, XML) and a property is read with the config.get() method, the config is loaded from the file at this time.
> If the deprecation mapping is not yet specified by the time any config value is retrieved and the XML config refers to a deprecated key, then the deprecation mapping specified, the config value cannot be retrieved neither with the deprecated nor with the new key.
> The attached patch contains a testcase that reproduces this wrong behavior.
> Here are the steps outlined what the testcase does:
> 1. Creates an XML config file with a deprecated property
> 2. Adds the config to the Configuration object
> 3. Retrieves the config with its deprecated key (it does not really matter which property the user gets, could be any)
> 4. Specifies the deprecation rules including the one defined in the config
> 5. Prints and asserts the property retrieved from the config with both the deprecated and the new property keys.
> For reference, here is the log of one execution that actually shows what the issue is:
> {noformat}
> Loaded items: 1
> Looked up property value with name hadoop.zk.address: null
> Looked up property value with name yarn.resourcemanager.zk-address: dummyZkAddress
> Contents of config file: [<?xml version="1.0"?>, <configuration>, <property><name>yarn.resourcemanager.zk-address</name><value>dummyZkAddress</value></property>, </configuration>]
> Looked up property value with name hadoop.zk.address: null
> 2018-08-31 10:10:06,484 INFO Configuration.deprecation (Configuration.java:logDeprecation(1397)) - yarn.resourcemanager.zk-address is deprecated. Instead, use hadoop.zk.address
> Looked up property value with name hadoop.zk.address: null
> Looked up property value with name hadoop.zk.address: null
> java.lang.AssertionError:
> Expected :dummyZkAddress
> Actual :null
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org