You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@oozie.apache.org by "Alejandro Abdelnur (JIRA)" <ji...@apache.org> on 2012/05/08 08:00:21 UTC
[jira] [Created] (OOZIE-830) add support for multiple/configurable
sharelibs for each action type
Alejandro Abdelnur created OOZIE-830:
----------------------------------------
Summary: add support for multiple/configurable sharelibs for each action type
Key: OOZIE-830
URL: https://issues.apache.org/jira/browse/OOZIE-830
Project: Oozie
Issue Type: New Feature
Components: action
Affects Versions: trunk
Reporter: Alejandro Abdelnur
Assignee: Alejandro Abdelnur
Fix For: trunk
Currently there is a fixed sharelib per action type. I.e.:
{code}
/share/lib/mapreduce-streaming/
pig/
hive/
sqoop/
{code}
It many situations it would be desirable to support multiple versions of sharelib per component, have a system default, and allow users to override the default for a specific version. I.e.:
{code}
/share/lib/mapreduce-streaming/
pig-0_8/ (default)
pig-0_9/
sqoop/
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (OOZIE-830) add support for
multiple/configurable sharelibs for each action type
Posted by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OOZIE-830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13271650#comment-13271650 ]
jiraposter@reviews.apache.org commented on OOZIE-830:
-----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/5063/#review7739
-----------------------------------------------------------
Ship it!
+1 looks good after virag's comments
- Mayank
On 2012-05-08 07:06:23, Alejandro Abdelnur wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/5063/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2012-05-08 07:06:23)
bq.
bq.
bq. Review request for oozie.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. The sharelibrary override is supported at Oozie server level and at workflow job level.
bq.
bq. The share library name will be resolved using the following precedence order:
bq.
bq. action.sharelib.for.#ACTIONTYPE# in the action configuration (at action level)
bq. action.sharelib.for.#ACTIONTYPE# in the job configuration (at job level)
bq. action.sharelib.for.#ACTIONTYPE# in the oozie server configuration (at oozie server level)
bq. action's ActionExecutor getDefaultShareLibName() method (default)
bq.
bq.
bq. This addresses bug OOZIE-830.
bq. https://issues.apache.org/jira/browse/OOZIE-830
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. trunk/core/src/main/java/org/apache/oozie/action/hadoop/DistcpActionExecutor.java 1335346
bq. trunk/core/src/main/java/org/apache/oozie/action/hadoop/HiveActionExecutor.java 1335346
bq. trunk/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 1335346
bq. trunk/core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java 1335346
bq. trunk/core/src/main/java/org/apache/oozie/action/hadoop/PigActionExecutor.java 1335346
bq. trunk/core/src/main/java/org/apache/oozie/action/hadoop/SqoopActionExecutor.java 1335346
bq. trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java 1335346
bq. trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java 1335346
bq. trunk/docs/src/site/twiki/WorkflowFunctionalSpec.twiki 1335346
bq.
bq. Diff: https://reviews.apache.org/r/5063/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. besides testcases, tested in a deployed setup
bq.
bq.
bq. Thanks,
bq.
bq. Alejandro
bq.
bq.
> add support for multiple/configurable sharelibs for each action type
> --------------------------------------------------------------------
>
> Key: OOZIE-830
> URL: https://issues.apache.org/jira/browse/OOZIE-830
> Project: Oozie
> Issue Type: New Feature
> Components: action
> Affects Versions: trunk
> Reporter: Alejandro Abdelnur
> Assignee: Alejandro Abdelnur
> Fix For: trunk
>
>
> Currently there is a fixed sharelib per action type. I.e.:
> {code}
> /share/lib/mapreduce-streaming/
> pig/
> hive/
> sqoop/
> {code}
> It many situations it would be desirable to support multiple versions of sharelib per component, have a system default, and allow users to override the default for a specific version. I.e.:
> {code}
> /share/lib/mapreduce-streaming/
> pig-0_8/ (default)
> pig-0_9/
> sqoop/
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (OOZIE-830) add support for
multiple/configurable sharelibs for each action type
Posted by "Alejandro Abdelnur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OOZIE-830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13270234#comment-13270234 ]
Alejandro Abdelnur commented on OOZIE-830:
------------------------------------------
The sharelibrary override would supported at Oozie server level and at workflow job level.
The share library name would be resolved using the following precedence order:
* action.sharelib.for.#ACTIONTYPE# in the action configuration (at action level)
* action.sharelib.for.#ACTIONTYPE# in the job configuration (at job level)
* action.sharelib.for.#ACTIONTYPE# in the oozie server configuration (at oozie server level)
* action's ActionExecutor getDefaultShareLibName() method (default)
> add support for multiple/configurable sharelibs for each action type
> --------------------------------------------------------------------
>
> Key: OOZIE-830
> URL: https://issues.apache.org/jira/browse/OOZIE-830
> Project: Oozie
> Issue Type: New Feature
> Components: action
> Affects Versions: trunk
> Reporter: Alejandro Abdelnur
> Assignee: Alejandro Abdelnur
> Fix For: trunk
>
>
> Currently there is a fixed sharelib per action type. I.e.:
> {code}
> /share/lib/mapreduce-streaming/
> pig/
> hive/
> sqoop/
> {code}
> It many situations it would be desirable to support multiple versions of sharelib per component, have a system default, and allow users to override the default for a specific version. I.e.:
> {code}
> /share/lib/mapreduce-streaming/
> pig-0_8/ (default)
> pig-0_9/
> sqoop/
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (OOZIE-830) add support for
multiple/configurable sharelibs for each action type
Posted by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OOZIE-830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13270276#comment-13270276 ]
jiraposter@reviews.apache.org commented on OOZIE-830:
-----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/5063/
-----------------------------------------------------------
(Updated 2012-05-08 07:06:23.568711)
Review request for oozie.
Summary
-------
The sharelibrary override is supported at Oozie server level and at workflow job level.
The share library name will be resolved using the following precedence order:
action.sharelib.for.#ACTIONTYPE# in the action configuration (at action level)
action.sharelib.for.#ACTIONTYPE# in the job configuration (at job level)
action.sharelib.for.#ACTIONTYPE# in the oozie server configuration (at oozie server level)
action's ActionExecutor getDefaultShareLibName() method (default)
This addresses bug OOZIE-830.
https://issues.apache.org/jira/browse/OOZIE-830
Diffs
-----
trunk/core/src/main/java/org/apache/oozie/action/hadoop/DistcpActionExecutor.java 1335346
trunk/core/src/main/java/org/apache/oozie/action/hadoop/HiveActionExecutor.java 1335346
trunk/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 1335346
trunk/core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java 1335346
trunk/core/src/main/java/org/apache/oozie/action/hadoop/PigActionExecutor.java 1335346
trunk/core/src/main/java/org/apache/oozie/action/hadoop/SqoopActionExecutor.java 1335346
trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java 1335346
trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java 1335346
trunk/docs/src/site/twiki/WorkflowFunctionalSpec.twiki 1335346
Diff: https://reviews.apache.org/r/5063/diff
Testing (updated)
-------
besides testcases, tested in a deployed setup
Thanks,
Alejandro
> add support for multiple/configurable sharelibs for each action type
> --------------------------------------------------------------------
>
> Key: OOZIE-830
> URL: https://issues.apache.org/jira/browse/OOZIE-830
> Project: Oozie
> Issue Type: New Feature
> Components: action
> Affects Versions: trunk
> Reporter: Alejandro Abdelnur
> Assignee: Alejandro Abdelnur
> Fix For: trunk
>
>
> Currently there is a fixed sharelib per action type. I.e.:
> {code}
> /share/lib/mapreduce-streaming/
> pig/
> hive/
> sqoop/
> {code}
> It many situations it would be desirable to support multiple versions of sharelib per component, have a system default, and allow users to override the default for a specific version. I.e.:
> {code}
> /share/lib/mapreduce-streaming/
> pig-0_8/ (default)
> pig-0_9/
> sqoop/
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (OOZIE-830) add support for
multiple/configurable sharelibs for each action type
Posted by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OOZIE-830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13271907#comment-13271907 ]
jiraposter@reviews.apache.org commented on OOZIE-830:
-----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/5063/#review7749
-----------------------------------------------------------
i'll prepare a patch addressing Virag's corrections.
trunk/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java
<https://reviews.apache.org/r/5063/#comment17052>
here is injected into the launcherconf, in submitLauncher is injected into the actionconf.
- Alejandro
On 2012-05-08 07:06:23, Alejandro Abdelnur wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/5063/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2012-05-08 07:06:23)
bq.
bq.
bq. Review request for oozie.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. The sharelibrary override is supported at Oozie server level and at workflow job level.
bq.
bq. The share library name will be resolved using the following precedence order:
bq.
bq. action.sharelib.for.#ACTIONTYPE# in the action configuration (at action level)
bq. action.sharelib.for.#ACTIONTYPE# in the job configuration (at job level)
bq. action.sharelib.for.#ACTIONTYPE# in the oozie server configuration (at oozie server level)
bq. action's ActionExecutor getDefaultShareLibName() method (default)
bq.
bq.
bq. This addresses bug OOZIE-830.
bq. https://issues.apache.org/jira/browse/OOZIE-830
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. trunk/core/src/main/java/org/apache/oozie/action/hadoop/DistcpActionExecutor.java 1335346
bq. trunk/core/src/main/java/org/apache/oozie/action/hadoop/HiveActionExecutor.java 1335346
bq. trunk/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 1335346
bq. trunk/core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java 1335346
bq. trunk/core/src/main/java/org/apache/oozie/action/hadoop/PigActionExecutor.java 1335346
bq. trunk/core/src/main/java/org/apache/oozie/action/hadoop/SqoopActionExecutor.java 1335346
bq. trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java 1335346
bq. trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java 1335346
bq. trunk/docs/src/site/twiki/WorkflowFunctionalSpec.twiki 1335346
bq.
bq. Diff: https://reviews.apache.org/r/5063/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. besides testcases, tested in a deployed setup
bq.
bq.
bq. Thanks,
bq.
bq. Alejandro
bq.
bq.
> add support for multiple/configurable sharelibs for each action type
> --------------------------------------------------------------------
>
> Key: OOZIE-830
> URL: https://issues.apache.org/jira/browse/OOZIE-830
> Project: Oozie
> Issue Type: New Feature
> Components: action
> Affects Versions: trunk
> Reporter: Alejandro Abdelnur
> Assignee: Alejandro Abdelnur
> Fix For: trunk
>
>
> Currently there is a fixed sharelib per action type. I.e.:
> {code}
> /share/lib/mapreduce-streaming/
> pig/
> hive/
> sqoop/
> {code}
> It many situations it would be desirable to support multiple versions of sharelib per component, have a system default, and allow users to override the default for a specific version. I.e.:
> {code}
> /share/lib/mapreduce-streaming/
> pig-0_8/ (default)
> pig-0_9/
> sqoop/
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (OOZIE-830) add support for multiple/configurable
sharelibs for each action type
Posted by "Alejandro Abdelnur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OOZIE-830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alejandro Abdelnur resolved OOZIE-830.
--------------------------------------
Resolution: Fixed
committed to trunk
> add support for multiple/configurable sharelibs for each action type
> --------------------------------------------------------------------
>
> Key: OOZIE-830
> URL: https://issues.apache.org/jira/browse/OOZIE-830
> Project: Oozie
> Issue Type: New Feature
> Components: action
> Affects Versions: trunk
> Reporter: Alejandro Abdelnur
> Assignee: Alejandro Abdelnur
> Fix For: trunk
>
>
> Currently there is a fixed sharelib per action type. I.e.:
> {code}
> /share/lib/mapreduce-streaming/
> pig/
> hive/
> sqoop/
> {code}
> It many situations it would be desirable to support multiple versions of sharelib per component, have a system default, and allow users to override the default for a specific version. I.e.:
> {code}
> /share/lib/mapreduce-streaming/
> pig-0_8/ (default)
> pig-0_9/
> sqoop/
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (OOZIE-830) add support for
multiple/configurable sharelibs for each action type
Posted by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OOZIE-830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13270254#comment-13270254 ]
jiraposter@reviews.apache.org commented on OOZIE-830:
-----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/5063/
-----------------------------------------------------------
Review request for oozie.
Summary
-------
The sharelibrary override is supported at Oozie server level and at workflow job level.
The share library name will be resolved using the following precedence order:
action.sharelib.for.#ACTIONTYPE# in the action configuration (at action level)
action.sharelib.for.#ACTIONTYPE# in the job configuration (at job level)
action.sharelib.for.#ACTIONTYPE# in the oozie server configuration (at oozie server level)
action's ActionExecutor getDefaultShareLibName() method (default)
This addresses bug OOZIE-830.
https://issues.apache.org/jira/browse/OOZIE-830
Diffs
-----
trunk/core/src/main/java/org/apache/oozie/action/hadoop/DistcpActionExecutor.java 1335346
trunk/core/src/main/java/org/apache/oozie/action/hadoop/HiveActionExecutor.java 1335346
trunk/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 1335346
trunk/core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java 1335346
trunk/core/src/main/java/org/apache/oozie/action/hadoop/PigActionExecutor.java 1335346
trunk/core/src/main/java/org/apache/oozie/action/hadoop/SqoopActionExecutor.java 1335346
trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java 1335346
trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java 1335346
trunk/docs/src/site/twiki/WorkflowFunctionalSpec.twiki 1335346
Diff: https://reviews.apache.org/r/5063/diff
Testing
-------
Thanks,
Alejandro
> add support for multiple/configurable sharelibs for each action type
> --------------------------------------------------------------------
>
> Key: OOZIE-830
> URL: https://issues.apache.org/jira/browse/OOZIE-830
> Project: Oozie
> Issue Type: New Feature
> Components: action
> Affects Versions: trunk
> Reporter: Alejandro Abdelnur
> Assignee: Alejandro Abdelnur
> Fix For: trunk
>
>
> Currently there is a fixed sharelib per action type. I.e.:
> {code}
> /share/lib/mapreduce-streaming/
> pig/
> hive/
> sqoop/
> {code}
> It many situations it would be desirable to support multiple versions of sharelib per component, have a system default, and allow users to override the default for a specific version. I.e.:
> {code}
> /share/lib/mapreduce-streaming/
> pig-0_8/ (default)
> pig-0_9/
> sqoop/
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (OOZIE-830) add support for
multiple/configurable sharelibs for each action type
Posted by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OOZIE-830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13270956#comment-13270956 ]
jiraposter@reviews.apache.org commented on OOZIE-830:
-----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/5063/#review7707
-----------------------------------------------------------
Looks good...some comments mainly related to javadoc
trunk/core/src/main/java/org/apache/oozie/action/hadoop/DistcpActionExecutor.java
<https://reviews.apache.org/r/5063/#comment16977>
javadoc missing
trunk/core/src/main/java/org/apache/oozie/action/hadoop/HiveActionExecutor.java
<https://reviews.apache.org/r/5063/#comment16979>
Does this sentence need modification as the function name has changed?.. Same comment at other places
trunk/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java
<https://reviews.apache.org/r/5063/#comment16983>
do you want to remove 'postfix' from method signature here too?
trunk/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java
<https://reviews.apache.org/r/5063/#comment16980>
why is this function called twice? here and in submitLauncher
trunk/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java
<https://reviews.apache.org/r/5063/#comment16981>
missing javadoc
trunk/docs/src/site/twiki/WorkflowFunctionalSpec.twiki
<https://reviews.apache.org/r/5063/#comment16982>
'.' after distcp actions
- Virag
On 2012-05-08 07:06:23, Alejandro Abdelnur wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/5063/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2012-05-08 07:06:23)
bq.
bq.
bq. Review request for oozie.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. The sharelibrary override is supported at Oozie server level and at workflow job level.
bq.
bq. The share library name will be resolved using the following precedence order:
bq.
bq. action.sharelib.for.#ACTIONTYPE# in the action configuration (at action level)
bq. action.sharelib.for.#ACTIONTYPE# in the job configuration (at job level)
bq. action.sharelib.for.#ACTIONTYPE# in the oozie server configuration (at oozie server level)
bq. action's ActionExecutor getDefaultShareLibName() method (default)
bq.
bq.
bq. This addresses bug OOZIE-830.
bq. https://issues.apache.org/jira/browse/OOZIE-830
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. trunk/core/src/main/java/org/apache/oozie/action/hadoop/DistcpActionExecutor.java 1335346
bq. trunk/core/src/main/java/org/apache/oozie/action/hadoop/HiveActionExecutor.java 1335346
bq. trunk/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 1335346
bq. trunk/core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java 1335346
bq. trunk/core/src/main/java/org/apache/oozie/action/hadoop/PigActionExecutor.java 1335346
bq. trunk/core/src/main/java/org/apache/oozie/action/hadoop/SqoopActionExecutor.java 1335346
bq. trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java 1335346
bq. trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java 1335346
bq. trunk/docs/src/site/twiki/WorkflowFunctionalSpec.twiki 1335346
bq.
bq. Diff: https://reviews.apache.org/r/5063/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. besides testcases, tested in a deployed setup
bq.
bq.
bq. Thanks,
bq.
bq. Alejandro
bq.
bq.
> add support for multiple/configurable sharelibs for each action type
> --------------------------------------------------------------------
>
> Key: OOZIE-830
> URL: https://issues.apache.org/jira/browse/OOZIE-830
> Project: Oozie
> Issue Type: New Feature
> Components: action
> Affects Versions: trunk
> Reporter: Alejandro Abdelnur
> Assignee: Alejandro Abdelnur
> Fix For: trunk
>
>
> Currently there is a fixed sharelib per action type. I.e.:
> {code}
> /share/lib/mapreduce-streaming/
> pig/
> hive/
> sqoop/
> {code}
> It many situations it would be desirable to support multiple versions of sharelib per component, have a system default, and allow users to override the default for a specific version. I.e.:
> {code}
> /share/lib/mapreduce-streaming/
> pig-0_8/ (default)
> pig-0_9/
> sqoop/
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (OOZIE-830) add support for
multiple/configurable sharelibs for each action type
Posted by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OOZIE-830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13271912#comment-13271912 ]
jiraposter@reviews.apache.org commented on OOZIE-830:
-----------------------------------------------------
bq. On 2012-05-09 20:35:13, Mohammad Islam wrote:
bq. > How will a user know what are the system supported versions? There could be a utility for that.
bq. >
bq. > Another point, if a user gives a wrong or not supported version, how will that be communicated to the end-user.
bq. >
listing the contents of the HDFS /user/oozie/share/lib directory will tell the supported versions.
a wrong version will make the action to fail with JARs not found, if the dir is not avail in HDFS the action won't have the JARs files in its classpath
bq. On 2012-05-09 20:35:13, Mohammad Islam wrote:
bq. > trunk/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java, line 1116
bq. > <https://reviews.apache.org/r/5063/diff/3/?file=107695#file107695line1116>
bq. >
bq. > If name is 'null' after that, what will happen?
bq. > Should we throw an error?
bq. >
bq. >
no, if null it means the action does not have a sharelib (ie mapreduce or java)
- Alejandro
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/5063/#review7741
-----------------------------------------------------------
On 2012-05-08 07:06:23, Alejandro Abdelnur wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/5063/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2012-05-08 07:06:23)
bq.
bq.
bq. Review request for oozie.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. The sharelibrary override is supported at Oozie server level and at workflow job level.
bq.
bq. The share library name will be resolved using the following precedence order:
bq.
bq. action.sharelib.for.#ACTIONTYPE# in the action configuration (at action level)
bq. action.sharelib.for.#ACTIONTYPE# in the job configuration (at job level)
bq. action.sharelib.for.#ACTIONTYPE# in the oozie server configuration (at oozie server level)
bq. action's ActionExecutor getDefaultShareLibName() method (default)
bq.
bq.
bq. This addresses bug OOZIE-830.
bq. https://issues.apache.org/jira/browse/OOZIE-830
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. trunk/core/src/main/java/org/apache/oozie/action/hadoop/DistcpActionExecutor.java 1335346
bq. trunk/core/src/main/java/org/apache/oozie/action/hadoop/HiveActionExecutor.java 1335346
bq. trunk/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 1335346
bq. trunk/core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java 1335346
bq. trunk/core/src/main/java/org/apache/oozie/action/hadoop/PigActionExecutor.java 1335346
bq. trunk/core/src/main/java/org/apache/oozie/action/hadoop/SqoopActionExecutor.java 1335346
bq. trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java 1335346
bq. trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java 1335346
bq. trunk/docs/src/site/twiki/WorkflowFunctionalSpec.twiki 1335346
bq.
bq. Diff: https://reviews.apache.org/r/5063/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. besides testcases, tested in a deployed setup
bq.
bq.
bq. Thanks,
bq.
bq. Alejandro
bq.
bq.
> add support for multiple/configurable sharelibs for each action type
> --------------------------------------------------------------------
>
> Key: OOZIE-830
> URL: https://issues.apache.org/jira/browse/OOZIE-830
> Project: Oozie
> Issue Type: New Feature
> Components: action
> Affects Versions: trunk
> Reporter: Alejandro Abdelnur
> Assignee: Alejandro Abdelnur
> Fix For: trunk
>
>
> Currently there is a fixed sharelib per action type. I.e.:
> {code}
> /share/lib/mapreduce-streaming/
> pig/
> hive/
> sqoop/
> {code}
> It many situations it would be desirable to support multiple versions of sharelib per component, have a system default, and allow users to override the default for a specific version. I.e.:
> {code}
> /share/lib/mapreduce-streaming/
> pig-0_8/ (default)
> pig-0_9/
> sqoop/
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (OOZIE-830) add support for
multiple/configurable sharelibs for each action type
Posted by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OOZIE-830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13271777#comment-13271777 ]
jiraposter@reviews.apache.org commented on OOZIE-830:
-----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/5063/#review7741
-----------------------------------------------------------
How will a user know what are the system supported versions? There could be a utility for that.
Another point, if a user gives a wrong or not supported version, how will that be communicated to the end-user.
trunk/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java
<https://reviews.apache.org/r/5063/#comment17041>
If name is 'null' after that, what will happen?
Should we throw an error?
- Mohammad
On 2012-05-08 07:06:23, Alejandro Abdelnur wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/5063/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2012-05-08 07:06:23)
bq.
bq.
bq. Review request for oozie.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. The sharelibrary override is supported at Oozie server level and at workflow job level.
bq.
bq. The share library name will be resolved using the following precedence order:
bq.
bq. action.sharelib.for.#ACTIONTYPE# in the action configuration (at action level)
bq. action.sharelib.for.#ACTIONTYPE# in the job configuration (at job level)
bq. action.sharelib.for.#ACTIONTYPE# in the oozie server configuration (at oozie server level)
bq. action's ActionExecutor getDefaultShareLibName() method (default)
bq.
bq.
bq. This addresses bug OOZIE-830.
bq. https://issues.apache.org/jira/browse/OOZIE-830
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. trunk/core/src/main/java/org/apache/oozie/action/hadoop/DistcpActionExecutor.java 1335346
bq. trunk/core/src/main/java/org/apache/oozie/action/hadoop/HiveActionExecutor.java 1335346
bq. trunk/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 1335346
bq. trunk/core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java 1335346
bq. trunk/core/src/main/java/org/apache/oozie/action/hadoop/PigActionExecutor.java 1335346
bq. trunk/core/src/main/java/org/apache/oozie/action/hadoop/SqoopActionExecutor.java 1335346
bq. trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java 1335346
bq. trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java 1335346
bq. trunk/docs/src/site/twiki/WorkflowFunctionalSpec.twiki 1335346
bq.
bq. Diff: https://reviews.apache.org/r/5063/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. besides testcases, tested in a deployed setup
bq.
bq.
bq. Thanks,
bq.
bq. Alejandro
bq.
bq.
> add support for multiple/configurable sharelibs for each action type
> --------------------------------------------------------------------
>
> Key: OOZIE-830
> URL: https://issues.apache.org/jira/browse/OOZIE-830
> Project: Oozie
> Issue Type: New Feature
> Components: action
> Affects Versions: trunk
> Reporter: Alejandro Abdelnur
> Assignee: Alejandro Abdelnur
> Fix For: trunk
>
>
> Currently there is a fixed sharelib per action type. I.e.:
> {code}
> /share/lib/mapreduce-streaming/
> pig/
> hive/
> sqoop/
> {code}
> It many situations it would be desirable to support multiple versions of sharelib per component, have a system default, and allow users to override the default for a specific version. I.e.:
> {code}
> /share/lib/mapreduce-streaming/
> pig-0_8/ (default)
> pig-0_9/
> sqoop/
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (OOZIE-830) add support for
multiple/configurable sharelibs for each action type
Posted by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OOZIE-830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13270255#comment-13270255 ]
jiraposter@reviews.apache.org commented on OOZIE-830:
-----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/5063/
-----------------------------------------------------------
(Updated 2012-05-08 06:29:52.868583)
Review request for oozie.
Changes
-------
removing trailing spaces
Summary
-------
The sharelibrary override is supported at Oozie server level and at workflow job level.
The share library name will be resolved using the following precedence order:
action.sharelib.for.#ACTIONTYPE# in the action configuration (at action level)
action.sharelib.for.#ACTIONTYPE# in the job configuration (at job level)
action.sharelib.for.#ACTIONTYPE# in the oozie server configuration (at oozie server level)
action's ActionExecutor getDefaultShareLibName() method (default)
This addresses bug OOZIE-830.
https://issues.apache.org/jira/browse/OOZIE-830
Diffs (updated)
-----
trunk/core/src/main/java/org/apache/oozie/action/hadoop/DistcpActionExecutor.java 1335346
trunk/core/src/main/java/org/apache/oozie/action/hadoop/HiveActionExecutor.java 1335346
trunk/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 1335346
trunk/core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java 1335346
trunk/core/src/main/java/org/apache/oozie/action/hadoop/PigActionExecutor.java 1335346
trunk/core/src/main/java/org/apache/oozie/action/hadoop/SqoopActionExecutor.java 1335346
trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java 1335346
trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java 1335346
trunk/docs/src/site/twiki/WorkflowFunctionalSpec.twiki 1335346
Diff: https://reviews.apache.org/r/5063/diff
Testing
-------
Thanks,
Alejandro
> add support for multiple/configurable sharelibs for each action type
> --------------------------------------------------------------------
>
> Key: OOZIE-830
> URL: https://issues.apache.org/jira/browse/OOZIE-830
> Project: Oozie
> Issue Type: New Feature
> Components: action
> Affects Versions: trunk
> Reporter: Alejandro Abdelnur
> Assignee: Alejandro Abdelnur
> Fix For: trunk
>
>
> Currently there is a fixed sharelib per action type. I.e.:
> {code}
> /share/lib/mapreduce-streaming/
> pig/
> hive/
> sqoop/
> {code}
> It many situations it would be desirable to support multiple versions of sharelib per component, have a system default, and allow users to override the default for a specific version. I.e.:
> {code}
> /share/lib/mapreduce-streaming/
> pig-0_8/ (default)
> pig-0_9/
> sqoop/
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira