Oozie - java action - how to include third party libraries

[BHASKER ALLENE <qr...@virginmedia.com>]

Hi,
As per the oozie Map reduce cook book "libjars" option is not supported by
Oozie.
How do we include third party libraries in Java action.
One option I am thinking is include my hadoop command in a shell script
and use shell script option instead of Java action.

Regards,
Bhasker

Re: Oozie - java action - how to include third party libraries

[Harish Krishnan <ha...@gmail.com>]

Hi,

There are multiple ways to add the jars to the lib path.
I would suggest you try one of the following

1) in your job.properties, add the path to your jar files (comma separated)
to *oozie.libpath *property
2) create a *lib* directory under your application's directory and put your
jars under lib so oozie can pick them up.

Let me know if these work.

Thanks & Regards,
Harish.T.K


On Thu, Oct 18, 2012 at 2:46 AM, BHASKER ALLENE <qr...@virginmedia.com>wrote:

> Hi,
> As per the oozie Map reduce cook book "libjars" option is not supported by
> Oozie.
> How do we include third party libraries in Java action.
> One option I am thinking is include my hadoop command in a shell script
> and use shell script option instead of Java action.
>
> Regards,
> Bhasker
>

Re: Oozie - java action - how to include third party libraries

[Harsh J <ha...@cloudera.com>]

Hi Bhasker,

In your WF app, if you place your dependent jars into the "lib"
sub-directory, they should become automatically available for your
Java action class to use (by being on the classpath).

Does this not work for you? Something like the following structure is
that I mean.

myapp/
myapp/workflow.xml
myapp/lib/
myapp/lib/main.jar
myapp/lib/dependency.jar

Or does your Java application actually submit a MR job and/or invoke
ToolRunner.run?

On Thu, Oct 18, 2012 at 3:16 PM, BHASKER ALLENE <qr...@virginmedia.com> wrote:
> Hi,
> As per the oozie Map reduce cook book "libjars" option is not supported by
> Oozie.
> How do we include third party libraries in Java action.
> One option I am thinking is include my hadoop command in a shell script
> and use shell script option instead of Java action.
>
> Regards,
> Bhasker



-- 
Harsh J

Re: Oozie - java action - how to include third party libraries

[Jarek Jarcec Cecho <ja...@apache.org>]

There is multiple ways how to provide extra libraries to your workflow. It seems that you're looking for particular workflow, so in that case you should add your additional jars to lib/ directory of your workflow folder on hdfs. E.g.:

hdfs://cool-cluster/some/path/workflow.xml
hdfs://cool-cluster/some/path/lib/my-cool-library.jar

Oozie should automatically propagate all libraries in lib/ directory to classpath of all actions.

Jarcec

On Thu, Oct 18, 2012 at 10:46:17AM +0100, BHASKER ALLENE wrote:
> Hi,
> As per the oozie Map reduce cook book "libjars" option is not supported by
> Oozie.
> How do we include third party libraries in Java action.
> One option I am thinking is include my hadoop command in a shell script
> and use shell script option instead of Java action.
> 
> Regards,
> Bhasker