[jira] [Commented] (OOZIE-2187) Add a way to specify a default JT/RM and NN

["Jeffrey Olchovy (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/OOZIE-2187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15099135#comment-15099135 ] 

Jeffrey Olchovy commented on OOZIE-2187:
----------------------------------------

>> The global section parsing code used to add the <name-node>, <job-tracker>, <job-xml>, and <configuration> from <global> to every action, even if they didn't want it (e.g. email action). While not necessarily harmful, this could be problematic in the future and results in larger database usage. It now only adds a <name-node> and <job-tracker> if requiresNameNodeJobTracker() is true and only adds a <job-xml> and <configuration> if a new supportsConfigurationJobXML() is true.

This caused a regression when we upgraded our cluster from CDH 5.4.x to CDH 5.5.x. The {{fs}} action does not require NN/JT yet the documentation states that:
"As of schema 0.4, if a name-node element is specified, then it is not necessary for any of the paths to start with the file system URI as it is taken from the name-node element. This is also true if the name-node is specified in the global section (see Global Configurations)"

We were relying on global configuration to infer the scheme for paths that we were passed to the {{fs}} action. After the upgrade, the global configuration is no longer applicable to the {{fs}} action resulting in errors such as the following:
{noformat}
FS001: Missing scheme in path [/com/tapad/devicegraph/v4/graphs/2016/01/08/device-graph]
{noformat}

> Add a way to specify a default JT/RM and NN
> -------------------------------------------
>
>                 Key: OOZIE-2187
>                 URL: https://issues.apache.org/jira/browse/OOZIE-2187
>             Project: Oozie
>          Issue Type: New Feature
>          Components: core
>            Reporter: Robert Kanter
>            Assignee: Robert Kanter
>             Fix For: trunk
>
>         Attachments: OOZIE-2187.002.patch, OOZIE-2187.003.patch, OOZIE-2187.patch, OOZIE-2187.patch, OOZIE-2187.patch, OOZIE-2187.patch
>
>
> Oozie is cluster agnostic, which is why we require an RM/JT and NN per action in your workflow (or once via the <global> section).  In practice, many users use one Oozie server per cluster, so it's an extra burden for them to have to specify this all the time.  It would be convenient if we added configuration properties to oozie-site that would let you specify a default RM/JT and NN to use.  
> This way, these users could completely omit the {{<job-tracker>}} and {{<name-node>}} fields from their workflows; as an added benefit, they can easily update these values if they ever rename/move their RM/JT or NN.  We'd of course still allow specifying  {{<job-tracker>}} and {{<name-node>}} in each action and {{<global>}} to allow individual workflows or actions to override the default.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)