You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@oozie.apache.org by "Peter Cseh (JIRA)" <ji...@apache.org> on 2018/03/20 21:06:00 UTC
[jira] [Comment Edited] (OOZIE-3056) Implement new mechanism to
specify ShareLibs for workflow actions
[ https://issues.apache.org/jira/browse/OOZIE-3056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16407023#comment-16407023 ]
Peter Cseh edited comment on OOZIE-3056 at 3/20/18 9:05 PM:
------------------------------------------------------------
Attaching first version. I went with Alternative 1), we're overwriting sharelib values with the one that has bigger priority.
was (Author: gezapeti):
Attaching first version.
> Implement new mechanism to specify ShareLibs for workflow actions
> -----------------------------------------------------------------
>
> Key: OOZIE-3056
> URL: https://issues.apache.org/jira/browse/OOZIE-3056
> Project: Oozie
> Issue Type: New Feature
> Components: core
> Affects Versions: 5.0.0
> Reporter: Attila Sasvari
> Assignee: Peter Cseh
> Priority: Major
> Fix For: 5.0.0
>
> Attachments: OOZIE-3056.001.patch
>
>
> OOZIE-2687 introduces the {{launcher}} element for workflows:
> {code}
> <launcher>
> <memory>1024</memory>
> <vcores>1</vcores>
> <java-opts>-Dsome.property=true -XX:+RandomJVMSwitch</java-opts>
> <env>key=value</env>
> <queue>root.oozie</queue>
> <sharelib>spark,hive</sharelib>
> </launcher>
> {code}
> The purpose of this ticket is to discuss and implement new mechanism for handling ShareLib.
> {{addActionShareLib}} in {{JavaActionExecutor}} should adjusted. Regarding "precedence order":
> if global and an action level {{launcher}} and {{configuration}} (e.g. {{oozie.action.sharelib.for.#ACTIONTYPE#}}) tries to override the sharelib the following should apply:
> {quote}
> config properties defined in an action's <configuration> have priority over an action's <job-xml>, which has priority over the global section's <configuration> and <job-xml>, which has priority over the action defaults in oozie-site, and so on.
> {quote}
> Here we have multiple choices how to handle sharelib:
> - Alternative 1: override sharelib in a way that is consistent with current way of handling Oozie configuration settings.
> - Alternative 2: make sharelib additive
> -- For example, if there is a global {{launcher}} with {{sharelib}} element in a workflow that includes multiple ShareLibs (e.g. A,B), and {{oozie.action.sharelib.for.#ACTIONTYPE#}} is also specified for an action's configuration (e.g. C,D), then we take the union of the specified entities (A,B,C,D would be included).
> -- It's inconsistent with everything else
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)