[jira] [Created] (OOZIE-2343) Shell Action should take Oozie Action config and setup HADOOP_CONF_DIR

["Robert Kanter (JIRA)" <ji...@apache.org>]

Robert Kanter created OOZIE-2343:
------------------------------------

             Summary: Shell Action should take Oozie Action config and setup HADOOP_CONF_DIR
                 Key: OOZIE-2343
                 URL: https://issues.apache.org/jira/browse/OOZIE-2343
             Project: Oozie
          Issue Type: Improvement
          Components: action
    Affects Versions: trunk
            Reporter: Robert Kanter
            Assignee: Robert Kanter


Despite being not recommended, many users still run Hadoop ecosystem programs from the Shell Action (e.g. hive, beeline, sqoop, etc), even though we have Actions for those.  One of the common problems we've been seeing here lately is that {{HADOOP_CONF_DIR}} is inherited from the NM, and is pointing to the NM's copy of the Hadoop *-site.xml files, which is often not well suited for client programs.  This can lead to all kinds of weird problems, such as permissions errors, missing logs, etc.  

As a workaround, we've been telling users to set {{HADOOP_CONF_DIR}} to {{/etc/hadoop/conf}}, however, this has some disadvantages.  Ideally, these client programs should pick up {{OOZIE_ACTION_CONF_XML}} instead.  Given that {{OOZIE_ACTION_CONF_XML}} is an amalgamation of all of the Hadoop *-site.xml plus things Oozie and the user set, we can simply have the Shell Action copy-paste it to the correctly named *-site.xml files, and point {{HADOOP_CONF_DIR}} to there.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)