You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@oozie.apache.org by Serega Sheypak <se...@gmail.com> on 2016/04/26 10:57:39 UTC
oozie.action.conf.xml in java action, is it possible to pass this
conf to spawned MR job?
Hi, I'm using oozie java action to start scalding/spark/mr job. Oozie java
action launcher has correct classpath here:
oozie.action.conf.xml
actionConf.addResource(new Path("file:///",
System.getProperty("oozie.action.conf.xml")));
Is there any way to pass that conf to spawned job? Right now started
job doesn't inherit classpath from oozie launcher and looses all
dependent jar, since only oozie action sees them.
Re: oozie.action.conf.xml in java action, is it possible to pass this
conf to spawned MR job?
Posted by Serega Sheypak <se...@gmail.com>.
Thanks, I did it, it works. I looked for a way to solve this problem using
configuration, without coding.
2016-04-26 16:18 GMT+02:00 satish saley <sa...@gmail.com>:
> You need to add oozie.action.conf.xml to job configuration
> of scalding/spark/mr job.
>
> The action configuration is written by Oozie to a file location mentioned
> in
>
> the system property "oozie.action.conf.xml"
>
> https://cwiki.apache.org/confluence/display/OOZIE/Java%20Cookbook
>
> JobConf jobConf = new JobConf(true);
> String actionXml = System.getProperty("oozie.action.conf.xml");
> jobConf.addResource(new Path("file:///", actionXml));
>
> If you use the above configuration for the job you are launching it should
> carry over the mapreduce.cache.files setting which puts the jars in the
> classpath.
>
> For other actions (pig, hive, mapreduce, etc) the conf is passed by Oozie.
> Since java action only has a main method, you will have to load the action
> config yourself.
>
>
> Sent from Yahoo Mail for iPhone <https://yho.com/footer0>
>
>
> On Tuesday, April 26, 2016, 1:58 AM, Serega Sheypak <
> serega.sheypak@gmail.com> wrote:
>
> Hi, I'm using oozie java action to start scalding/spark/mr job. Oozie java
> action launcher has correct classpath here:
> oozie.action.conf.xml
>
> actionConf.addResource(new Path("file:///",
> System.getProperty("oozie.action.conf.xml")));
>
> Is there any way to pass that conf to spawned job? Right now started
> job doesn't inherit classpath from oozie launcher and looses all
> dependent jar, since only oozie action sees them.
>
>
Re: oozie.action.conf.xml in java action, is it possible to pass
this conf to spawned MR job?
Posted by satish saley <sa...@gmail.com>.
blockquote, div.yahoo_quoted { margin-left: 0 !important; border-left:1px #715FFA solid !important; padding-left:1ex !important; background-color:white !important; } You need to add oozie.action.conf.xml to job configuration of scalding/spark/mr job.
The action configuration is written by Oozie to a file location mentioned in
the system property "oozie.action.conf.xml"
https://cwiki.apache.org/confluence/display/OOZIE/Java%20Cookbook
JobConf jobConf = new JobConf(true);
String actionXml = System.getProperty("oozie.action.conf.xml");
jobConf.addResource(new Path("file:///", actionXml));
If you use the above configuration for the job you are launching it should
carry over the mapreduce.cache.files setting which puts the jars in the
classpath.
For other actions (pig, hive, mapreduce, etc) the conf is passed by Oozie.
Since java action only has a main method, you will have to load the action
config yourself.
Sent from Yahoo Mail for iPhone
On Tuesday, April 26, 2016, 1:58 AM, Serega Sheypak <se...@gmail.com> wrote:
Hi, I'm using oozie java action to start scalding/spark/mr job. Oozie java
action launcher has correct classpath here:
oozie.action.conf.xml
actionConf.addResource(new Path("file:///",
System.getProperty("oozie.action.conf.xml")));
Is there any way to pass that conf to spawned job? Right now started
job doesn't inherit classpath from oozie launcher and looses all
dependent jar, since only oozie action sees them.