You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@oozie.apache.org by Peter Cseh via Review Board <no...@reviews.apache.org> on 2017/11/15 13:34:02 UTC
Review Request 63836: OOZIE-2869 Ability to add/remove MapReduce jars
from action classpath.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63836/
-----------------------------------------------------------
Review request for oozie.
Bugs: OOZIE-2869
https://issues.apache.org/jira/browse/OOZIE-2869
Repository: oozie-git
Description
-------
I've introduced a property to provide the possibility to add/remove jars from the classpath for each action type using the "oozie.action.mapreduce.needed.for" prefix.
Diffs
-----
core/src/main/java/org/apache/oozie/action/hadoop/DistcpActionExecutor.java 961994ef14c5ef86be6609cde68bd98b39af8a2f
core/src/main/java/org/apache/oozie/action/hadoop/HiveActionExecutor.java d8c613bbbf9ed0d6b741a83861aeb82009084e3c
core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 8cb76cf086264ac09d8c9bbff22989d6e5eac69f
core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java 233dce7838b8d0ff0a18317d00768e87a0e5d7c8
core/src/main/java/org/apache/oozie/action/hadoop/PigActionExecutor.java 6541d52e35689a3776fa2ab3e6eb420c14c16a12
core/src/main/java/org/apache/oozie/action/hadoop/SparkActionExecutor.java 80d64ec80e683ea571748c2df976cf6bd22063cd
core/src/main/java/org/apache/oozie/action/hadoop/SqoopActionExecutor.java a0dfd31775507aeafaf09f1a316da0f52eac0a22
core/src/main/resources/oozie-default.xml 91d281125b236a9641faff54a69947b4440254d5
sharelib/spark/pom.xml b40a3528137cedf58c0591cf9d010c38eec75d44
Diff: https://reviews.apache.org/r/63836/diff/1/
Testing
-------
Thanks,
Peter Cseh
Re: Review Request 63836: OOZIE-2869 Ability to add/remove MapReduce
jars from action classpath.
Posted by András Piros via Review Board <no...@reviews.apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63836/#review191193
-----------------------------------------------------------
Ship it!
Ship It!
- András Piros
On Nov. 15, 2017, 1:34 p.m., Peter Cseh wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/63836/
> -----------------------------------------------------------
>
> (Updated Nov. 15, 2017, 1:34 p.m.)
>
>
> Review request for oozie.
>
>
> Bugs: OOZIE-2869
> https://issues.apache.org/jira/browse/OOZIE-2869
>
>
> Repository: oozie-git
>
>
> Description
> -------
>
> I've introduced a property to provide the possibility to add/remove jars from the classpath for each action type using the "oozie.action.mapreduce.needed.for" prefix.
>
>
> Diffs
> -----
>
> core/src/main/java/org/apache/oozie/action/hadoop/DistcpActionExecutor.java 961994ef14c5ef86be6609cde68bd98b39af8a2f
> core/src/main/java/org/apache/oozie/action/hadoop/HiveActionExecutor.java d8c613bbbf9ed0d6b741a83861aeb82009084e3c
> core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 8cb76cf086264ac09d8c9bbff22989d6e5eac69f
> core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java 233dce7838b8d0ff0a18317d00768e87a0e5d7c8
> core/src/main/java/org/apache/oozie/action/hadoop/PigActionExecutor.java 6541d52e35689a3776fa2ab3e6eb420c14c16a12
> core/src/main/java/org/apache/oozie/action/hadoop/SparkActionExecutor.java 80d64ec80e683ea571748c2df976cf6bd22063cd
> core/src/main/java/org/apache/oozie/action/hadoop/SqoopActionExecutor.java a0dfd31775507aeafaf09f1a316da0f52eac0a22
> core/src/main/resources/oozie-default.xml 91d281125b236a9641faff54a69947b4440254d5
> sharelib/spark/pom.xml b40a3528137cedf58c0591cf9d010c38eec75d44
>
>
> Diff: https://reviews.apache.org/r/63836/diff/1/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Peter Cseh
>
>
Re: Review Request 63836: OOZIE-2869 Ability to add/remove MapReduce
jars from action classpath.
Posted by Peter Cseh via Review Board <no...@reviews.apache.org>.
> On Nov. 15, 2017, 2:11 p.m., András Piros wrote:
> > Would be advisable to have a test case covering different `needed.for` scenarios, that is, the new functionality.
>
> Peter Cseh wrote:
> Yeah, I agree. I'm not sure I can test this properly. I'm opening up a Jira to create an AppSubmissionContextFactory or something similar so we can write tests against that part.
OOZIE-3131
- Peter
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63836/#review191060
-----------------------------------------------------------
On Nov. 15, 2017, 1:34 p.m., Peter Cseh wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/63836/
> -----------------------------------------------------------
>
> (Updated Nov. 15, 2017, 1:34 p.m.)
>
>
> Review request for oozie.
>
>
> Bugs: OOZIE-2869
> https://issues.apache.org/jira/browse/OOZIE-2869
>
>
> Repository: oozie-git
>
>
> Description
> -------
>
> I've introduced a property to provide the possibility to add/remove jars from the classpath for each action type using the "oozie.action.mapreduce.needed.for" prefix.
>
>
> Diffs
> -----
>
> core/src/main/java/org/apache/oozie/action/hadoop/DistcpActionExecutor.java 961994ef14c5ef86be6609cde68bd98b39af8a2f
> core/src/main/java/org/apache/oozie/action/hadoop/HiveActionExecutor.java d8c613bbbf9ed0d6b741a83861aeb82009084e3c
> core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 8cb76cf086264ac09d8c9bbff22989d6e5eac69f
> core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java 233dce7838b8d0ff0a18317d00768e87a0e5d7c8
> core/src/main/java/org/apache/oozie/action/hadoop/PigActionExecutor.java 6541d52e35689a3776fa2ab3e6eb420c14c16a12
> core/src/main/java/org/apache/oozie/action/hadoop/SparkActionExecutor.java 80d64ec80e683ea571748c2df976cf6bd22063cd
> core/src/main/java/org/apache/oozie/action/hadoop/SqoopActionExecutor.java a0dfd31775507aeafaf09f1a316da0f52eac0a22
> core/src/main/resources/oozie-default.xml 91d281125b236a9641faff54a69947b4440254d5
> sharelib/spark/pom.xml b40a3528137cedf58c0591cf9d010c38eec75d44
>
>
> Diff: https://reviews.apache.org/r/63836/diff/1/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Peter Cseh
>
>
Re: Review Request 63836: OOZIE-2869 Ability to add/remove MapReduce
jars from action classpath.
Posted by Peter Cseh via Review Board <no...@reviews.apache.org>.
> On Nov. 15, 2017, 2:11 p.m., András Piros wrote:
> > Would be advisable to have a test case covering different `needed.for` scenarios, that is, the new functionality.
Yeah, I agree. I'm not sure I can test this properly. I'm opening up a Jira to create an AppSubmissionContextFactory or something similar so we can write tests against that part.
- Peter
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63836/#review191060
-----------------------------------------------------------
On Nov. 15, 2017, 1:34 p.m., Peter Cseh wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/63836/
> -----------------------------------------------------------
>
> (Updated Nov. 15, 2017, 1:34 p.m.)
>
>
> Review request for oozie.
>
>
> Bugs: OOZIE-2869
> https://issues.apache.org/jira/browse/OOZIE-2869
>
>
> Repository: oozie-git
>
>
> Description
> -------
>
> I've introduced a property to provide the possibility to add/remove jars from the classpath for each action type using the "oozie.action.mapreduce.needed.for" prefix.
>
>
> Diffs
> -----
>
> core/src/main/java/org/apache/oozie/action/hadoop/DistcpActionExecutor.java 961994ef14c5ef86be6609cde68bd98b39af8a2f
> core/src/main/java/org/apache/oozie/action/hadoop/HiveActionExecutor.java d8c613bbbf9ed0d6b741a83861aeb82009084e3c
> core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 8cb76cf086264ac09d8c9bbff22989d6e5eac69f
> core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java 233dce7838b8d0ff0a18317d00768e87a0e5d7c8
> core/src/main/java/org/apache/oozie/action/hadoop/PigActionExecutor.java 6541d52e35689a3776fa2ab3e6eb420c14c16a12
> core/src/main/java/org/apache/oozie/action/hadoop/SparkActionExecutor.java 80d64ec80e683ea571748c2df976cf6bd22063cd
> core/src/main/java/org/apache/oozie/action/hadoop/SqoopActionExecutor.java a0dfd31775507aeafaf09f1a316da0f52eac0a22
> core/src/main/resources/oozie-default.xml 91d281125b236a9641faff54a69947b4440254d5
> sharelib/spark/pom.xml b40a3528137cedf58c0591cf9d010c38eec75d44
>
>
> Diff: https://reviews.apache.org/r/63836/diff/1/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Peter Cseh
>
>
Re: Review Request 63836: OOZIE-2869 Ability to add/remove MapReduce
jars from action classpath.
Posted by András Piros via Review Board <no...@reviews.apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63836/#review191060
-----------------------------------------------------------
Would be advisable to have a test case covering different `needed.for` scenarios, that is, the new functionality.
- András Piros
On Nov. 15, 2017, 1:34 p.m., Peter Cseh wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/63836/
> -----------------------------------------------------------
>
> (Updated Nov. 15, 2017, 1:34 p.m.)
>
>
> Review request for oozie.
>
>
> Bugs: OOZIE-2869
> https://issues.apache.org/jira/browse/OOZIE-2869
>
>
> Repository: oozie-git
>
>
> Description
> -------
>
> I've introduced a property to provide the possibility to add/remove jars from the classpath for each action type using the "oozie.action.mapreduce.needed.for" prefix.
>
>
> Diffs
> -----
>
> core/src/main/java/org/apache/oozie/action/hadoop/DistcpActionExecutor.java 961994ef14c5ef86be6609cde68bd98b39af8a2f
> core/src/main/java/org/apache/oozie/action/hadoop/HiveActionExecutor.java d8c613bbbf9ed0d6b741a83861aeb82009084e3c
> core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 8cb76cf086264ac09d8c9bbff22989d6e5eac69f
> core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java 233dce7838b8d0ff0a18317d00768e87a0e5d7c8
> core/src/main/java/org/apache/oozie/action/hadoop/PigActionExecutor.java 6541d52e35689a3776fa2ab3e6eb420c14c16a12
> core/src/main/java/org/apache/oozie/action/hadoop/SparkActionExecutor.java 80d64ec80e683ea571748c2df976cf6bd22063cd
> core/src/main/java/org/apache/oozie/action/hadoop/SqoopActionExecutor.java a0dfd31775507aeafaf09f1a316da0f52eac0a22
> core/src/main/resources/oozie-default.xml 91d281125b236a9641faff54a69947b4440254d5
> sharelib/spark/pom.xml b40a3528137cedf58c0591cf9d010c38eec75d44
>
>
> Diff: https://reviews.apache.org/r/63836/diff/1/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Peter Cseh
>
>