[jira] [Work started] (HIVE-22160) Job Configuration Properties Unable to Override hive-site.xml Properties in Master Node

["Haotian Zhang (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HIVE-22160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Work on HIVE-22160 started by Haotian Zhang.
--------------------------------------------
> Job Configuration Properties Unable to Override hive-site.xml Properties in Master Node
> ---------------------------------------------------------------------------------------
>
>                 Key: HIVE-22160
>                 URL: https://issues.apache.org/jira/browse/HIVE-22160
>             Project: Hive
>          Issue Type: Bug
>          Components: Configuration
>    Affects Versions: 3.1.2
>            Reporter: Haotian Zhang
>            Assignee: Haotian Zhang
>            Priority: Major
>              Labels: pull-request-available
>          Time Spent: 10m
>  Remaining Estimate: 0h
>
> When running on the master node, these two ways of creating HiveConf can result in different Hive configurations:
> {code:java}
> HiveConf hiveConf = HCatUtil.getHiveConf(job.getConfiguration());
> {code}
> the above one referred as Style1, and
> {code:java}
> HiveConf hiveConf = new HiveConf();
> hiveConf.addResource(job.getConfiguration());
> {code}
> the above one referred as Style2. If there are properties exist both in hiveSite and jobConf. For example, in hive-site.xml, we have property 
> {noformat}
> hive.metastore.uris = AAA{noformat}
> And if at the same time in jobConf, such as action configurations in Oozie's workflow.xml, we have 
> {noformat}
> hive.metastore.uris = BBB{noformat}
> Then in Style1, the "hive.metastore.uris" property in hiveConf is "AAA" and in Style2, the property in hiveConf is "BBB".
> Here is another example to illustrate this. Suppose in hiveSite, "hive.metastore.uris" is AAA and in jobConf it is BBB, when executing the following code:
> {code:java}
> HiveConf hiveConf = HCatUtil.getHiveConf(job.getConfiguration());
> log.info("the hive metastore uri in hiveconf first is: " + hiveConf.get("hive.metastore.uris"));
> hiveConf = HCatUtil.getHiveConf(job.getConfiguration());
> log.info("the hive metastore uri in hiveconf second is: " + hiveConf.get("hive.metastore.uris"));
> {code}
> And the log output is:
> {noformat}
> the hive metastore uri in hiveconf first is: AAA
> the hive metastore uri in hiveconf first is: BBB{noformat}
> In all, the jobConf does not got set to the configuration as expected when there's already such a property existed in hive site. This happen only on the master node but not on task nodes.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)