Re: Review Request 37452: OOZIE-2277: Honor oozie.action.sharelib.for.spark in Spark jobs

[Rohini Palaniswamy <ro...@gmail.com>]

> On Aug. 27, 2015, 10:03 p.m., Rohini Palaniswamy wrote:
> > sharelib/spark/src/main/java/org.apache.oozie.action.hadoop/SparkMain.java, line 78
> > <https://reviews.apache.org/r/37452/diff/1/?file=1039664#file1039664line78>
> >
> >     If local mode will be ever used in Oozie, then all the new code can go into a if (yarnClusterMode || yarnClientMode) block to be done only for non-local mode.
> 
> Robert Kanter wrote:
>     local mode still requires setting --jars, so I'd have to duplicate that part of the new code in an else statement which I think might be harder to follow/maintain.  Can we leave this as is?

Sure.


- Rohini


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37452/#review95987
-----------------------------------------------------------


On Aug. 28, 2015, 5:03 a.m., Robert Kanter wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/37452/
> -----------------------------------------------------------
> 
> (Updated Aug. 28, 2015, 5:03 a.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-2277
>     https://issues.apache.org/jira/browse/OOZIE-2277
> 
> 
> Repository: oozie-git
> 
> 
> Description
> -------
> 
> https://issues.apache.org/jira/browse/OOZIE-2277
> 
> 
> Diffs
> -----
> 
>   core/src/main/java/org/apache/oozie/service/SparkConfigurationService.java 1b7cf4a 
>   core/src/main/resources/oozie-default.xml 32a1df0 
>   core/src/test/java/org/apache/oozie/service/TestSparkConfigurationService.java b2c499d 
>   docs/src/site/twiki/DG_SparkActionExtension.twiki 32ebe12 
>   sharelib/spark/pom.xml 6f7e74a 
>   sharelib/spark/src/main/java/org.apache.oozie.action.hadoop/SparkMain.java b18a0b9 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestSparkActionExecutor.java f271abc 
> 
> Diff: https://reviews.apache.org/r/37452/diff/
> 
> 
> Testing
> -------
> 
> - Ran unit tests with Hadoop 1 and Hadoop 2
> - Ran in a Hadoop 2 cluster with local, yarn-client, and yarn-cluster modes
> 
> 
> Thanks,
> 
> Robert Kanter
> 
>