[jira] [Created] (OOZIE-3007) Ability to override default Map Reduce properties via default action config file

["Attila Sasvari (JIRA)" <ji...@apache.org>]

Attila Sasvari created OOZIE-3007:
-------------------------------------

             Summary: Ability to override default Map Reduce properties via default action config file
                 Key: OOZIE-3007
                 URL: https://issues.apache.org/jira/browse/OOZIE-3007
             Project: Oozie
          Issue Type: Bug
            Reporter: Attila Sasvari


According to https://oozie.apache.org/docs/4.2.0/AG_ActionConfiguration.html , it is possible to specify for default configuration parameters for Oozie actions by the means of action configuration files. 

However, this mechanism does not work for Map Reduce actions.

For example, assume we created {{/var/lib/oozie/action-conf/map-reduce.xml}} (readable by Oozie) with the following content:

{code}
 cat map-reduce.xml
<configuration>
  <property>
    <name>oozie.launcher.mapred.fairscheduler.pool</name>
    <value>root.support.coe</value>
  </property>
  <property>
    <name>oozie.launcher.mapred.job.queue.name</name>
    <value>oozie_queue</value>
  </property>
  <property>
    <name>mapreduce.job.queuename</name>
    <value>oozie_queue</value>
  </property>
  <property>
    <name>mapred.queue.name</name>
    <value>oozie_queue</value>
    </property>
</configuration>
{code}

and added the following to {{oozie-site.xml}}
{code}
<property>
  <name>oozie.service.HadoopAccessorService.action.configurations</name>
  <value>*=/var/lib/oozie/action-conf</value>
</property>
{code}

the mapper task started by Oozie will be put into the {{default}} queue instead of {{oozie-launcher}}.

h5. Possible reason:
HadoopAccessorService's {{loadHadoopConfigs()}} loads default hadoop configuration settings from hadoop config files. 

{{JavaActionExecutor}} loads default action configuration from given files with [createActionDefaultConf|https://github.com/apache/oozie/blob/84eb493642d35b8bb1e7527ef51b6ecfa40dd592/core/src/main/java/org/apache/oozie/service/HadoopAccessorService.java#L458] via HadoopAccessorService in [setupActionConf|https://github.com/apache/oozie/blob/84eb493642d35b8bb1e7527ef51b6ecfa40dd592/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java#L389]. 
{code}
XConfiguration actionDefaults = has.createActionDefaultConf(conf.get(HADOOP_YARN_RM), getType());
XConfiguration.injectDefaults(actionDefaults, actionConf);
{code}

However, {{actionConf}} already contains default value for "{{mapreduce.job.queuename}}" so [injectDefaults|https://github.com/apache/oozie/blob/84eb493642d35b8bb1e7527ef51b6ecfa40dd592/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java#L389] wont add user specified "{{mapreduce.job.queuename}}" default queue (i.e. {{oozie_queue}}).

I tested this behaviour with a 4.2.0 based Oozie, but the existing code did not change here by OYA.

  



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)