You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@oozie.apache.org by Morgrim Murdargent <mo...@gmail.com> on 2016/03/24 09:57:56 UTC
About java action and sharelibs
Hello everyone.
I send you this mail because I have a question related to the java actions
in oozie and to the sharelibs.
Here is my Use Case :
1. I have a simple workflow xml with a java action
2. I have a job properties with the property "oozie.use.system.libpath=true"
3. I have the following oozie sharelibs :
###
# hdfs dfs -ls /user/oozie/share/lib/
Found 1 items
drwxr-xr-x - oozie hdfs 0 2016-03-14 18:30
/user/oozie/share/lib/lib_20151124100459
# hdfs dfs -ls /user/oozie/share/lib/*
Found 11 items
drwxr-xr-x - oozie hdfs 0 2015-11-24 10:05
/user/oozie/share/lib/lib_20151124100459/distcp
drwxr-xr-x - oozie hdfs 0 2016-03-14 18:32
/user/oozie/share/lib/lib_20151124100459/hbase
drwxr-xr-x - oozie hdfs 0 2015-11-24 10:05
/user/oozie/share/lib/lib_20151124100459/hcatalog
drwxr-xr-x - oozie hdfs 0 2015-11-24 10:05
/user/oozie/share/lib/lib_20151124100459/hive
drwxr-xr-x - oozie hdfs 0 2015-11-24 10:05
/user/oozie/share/lib/lib_20151124100459/hive2
drwxr-xr-x - oozie hdfs 0 2015-11-24 10:05
/user/oozie/share/lib/lib_20151124100459/mapreduce-streaming
drwxr-xr-x - oozie hdfs 0 2015-11-24 10:05
/user/oozie/share/lib/lib_20151124100459/oozie
drwxr-xr-x - oozie hdfs 0 2015-11-24 10:05
/user/oozie/share/lib/lib_20151124100459/pig
-rwxr-xr-x 3 oozie hdfs 1396 2015-11-24 10:05
/user/oozie/share/lib/lib_20151124100459/sharelib.properties
drwxr-xr-x - oozie hdfs 0 2015-11-24 10:05
/user/oozie/share/lib/lib_20151124100459/spark
drwxr-xr-x - oozie hdfs 0 2015-11-24 10:05
/user/oozie/share/lib/lib_20151124100459/sqoop
###
My questions are :
Q1 - When I execute my oozie workflow containing my java action, which
libraries from the sharelibs are available for my java project ? All ?
Nothing ?
Q2 - Do I have to add in my workflow.xml the property
"oozie.action.sharelib.for.java" with specific values like
hcatalog,hive,pig,hbase ?
Thank you in advance for your help !
Best regards.
Morgrim
Re: About java action and sharelibs
Posted by Morgrim Murdargent <mo...@gmail.com>.
Re.
Thank you guys for your answer.
About the point 1, so by default, without specifying anything, only the
libs contained in /user/oozie/share/lib/lib_20151124100459/oozie will be
added to my classpath.
About the point 2, Abhishek, you say that generally, I don't need to add
the property. But when I have to interact with any component of my cluster
I need to use it no ? I mean, imagine I want to interact with HiveServer2,
I need the libs from the folder
/user/oozie/share/lib/lib_20151124100459/hive and/or
/user/oozie/share/lib/lib_20151124100459/hive2 no ?
Best regards.
Morgrim.
On Mon, Apr 4, 2016 at 2:56 PM, Abhishek Bafna <ba...@gmail.com> wrote:
> Hi,
>
> Answer-1: It will bring *.* from the oozie directory into your class path.
>
> Answer-2: In a general sense, you need not add the property
> "oozie.action.sharelib.for.java=xxx,xxx" for your java action, unless you
> want to add some more jars (libraries) in your class path. The "xxx" value,
> represents the directory name under your
> "/user/oozie/share/lib/lib_*****/xxx" path.
>
> Thanks,
> Abhishek
>
> On Mon, Apr 4, 2016 at 5:16 PM, 권병창 <n_...@naver.com> wrote:
>
> > Hi.
> >
> > Q1. oozie
> > Q2. You should add oozie.action.sharelib.for.java=xxx,xxx to
> > job.properties not workflow.xml
> >
> >
> > 2016. 4. 4., 오후 8:06, Morgrim Murdargent <mo...@gmail.com> 작성:
> >
> > Hey everyone.
> >
> > Has anyone any idea about this ?
> >
> > Best regards.
> >
> > Morgrim.
> >
> > On Thu, Mar 24, 2016 at 9:57 AM, Morgrim Murdargent <
> > morgrim.oozie@gmail.com
> > > wrote:
> >
> > > Hello everyone.
> > >
> > > I send you this mail because I have a question related to the java
> > actions
> > > in oozie and to the sharelibs.
> > >
> > > Here is my Use Case :
> > > 1. I have a simple workflow xml with a java action
> > > 2. I have a job properties with the property
> > > "oozie.use.system.libpath=true"
> > > 3. I have the following oozie sharelibs :
> > > ###
> > > # hdfs dfs -ls /user/oozie/share/lib/
> > > Found 1 items
> > > drwxr-xr-x - oozie hdfs 0 2016-03-14 18:30
> > > /user/oozie/share/lib/lib_20151124100459
> > >
> > > # hdfs dfs -ls /user/oozie/share/lib/*
> > > Found 11 items
> > > drwxr-xr-x - oozie hdfs 0 2015-11-24 10:05
> > > /user/oozie/share/lib/lib_20151124100459/distcp
> > > drwxr-xr-x - oozie hdfs 0 2016-03-14 18:32
> > > /user/oozie/share/lib/lib_20151124100459/hbase
> > > drwxr-xr-x - oozie hdfs 0 2015-11-24 10:05
> > > /user/oozie/share/lib/lib_20151124100459/hcatalog
> > > drwxr-xr-x - oozie hdfs 0 2015-11-24 10:05
> > > /user/oozie/share/lib/lib_20151124100459/hive
> > > drwxr-xr-x - oozie hdfs 0 2015-11-24 10:05
> > > /user/oozie/share/lib/lib_20151124100459/hive2
> > > drwxr-xr-x - oozie hdfs 0 2015-11-24 10:05
> > > /user/oozie/share/lib/lib_20151124100459/mapreduce-streaming
> > > drwxr-xr-x - oozie hdfs 0 2015-11-24 10:05
> > > /user/oozie/share/lib/lib_20151124100459/oozie
> > > drwxr-xr-x - oozie hdfs 0 2015-11-24 10:05
> > > /user/oozie/share/lib/lib_20151124100459/pig
> > > -rwxr-xr-x 3 oozie hdfs 1396 2015-11-24 10:05
> > > /user/oozie/share/lib/lib_20151124100459/sharelib.properties
> > > drwxr-xr-x - oozie hdfs 0 2015-11-24 10:05
> > > /user/oozie/share/lib/lib_20151124100459/spark
> > > drwxr-xr-x - oozie hdfs 0 2015-11-24 10:05
> > > /user/oozie/share/lib/lib_20151124100459/sqoop
> > > ###
> > >
> > > My questions are :
> > > Q1 - When I execute my oozie workflow containing my java action, which
> > > libraries from the sharelibs are available for my java project ? All ?
> > > Nothing ?
> > >
> > > Q2 - Do I have to add in my workflow.xml the property
> > > "oozie.action.sharelib.for.java" with specific values like
> > > hcatalog,hive,pig,hbase ?
> > >
> > > Thank you in advance for your help !
> > >
> > > Best regards.
> > >
> > > Morgrim
> > >
> >
> >
> >
>
Re: About java action and sharelibs
Posted by Abhishek Bafna <ba...@gmail.com>.
Hi,
Answer-1: It will bring *.* from the oozie directory into your class path.
Answer-2: In a general sense, you need not add the property
"oozie.action.sharelib.for.java=xxx,xxx" for your java action, unless you
want to add some more jars (libraries) in your class path. The "xxx" value,
represents the directory name under your
"/user/oozie/share/lib/lib_*****/xxx" path.
Thanks,
Abhishek
On Mon, Apr 4, 2016 at 5:16 PM, 권병창 <n_...@naver.com> wrote:
> Hi.
>
> Q1. oozie
> Q2. You should add oozie.action.sharelib.for.java=xxx,xxx to
> job.properties not workflow.xml
>
>
> 2016. 4. 4., 오후 8:06, Morgrim Murdargent <mo...@gmail.com> 작성:
>
> Hey everyone.
>
> Has anyone any idea about this ?
>
> Best regards.
>
> Morgrim.
>
> On Thu, Mar 24, 2016 at 9:57 AM, Morgrim Murdargent <
> morgrim.oozie@gmail.com
> > wrote:
>
> > Hello everyone.
> >
> > I send you this mail because I have a question related to the java
> actions
> > in oozie and to the sharelibs.
> >
> > Here is my Use Case :
> > 1. I have a simple workflow xml with a java action
> > 2. I have a job properties with the property
> > "oozie.use.system.libpath=true"
> > 3. I have the following oozie sharelibs :
> > ###
> > # hdfs dfs -ls /user/oozie/share/lib/
> > Found 1 items
> > drwxr-xr-x - oozie hdfs 0 2016-03-14 18:30
> > /user/oozie/share/lib/lib_20151124100459
> >
> > # hdfs dfs -ls /user/oozie/share/lib/*
> > Found 11 items
> > drwxr-xr-x - oozie hdfs 0 2015-11-24 10:05
> > /user/oozie/share/lib/lib_20151124100459/distcp
> > drwxr-xr-x - oozie hdfs 0 2016-03-14 18:32
> > /user/oozie/share/lib/lib_20151124100459/hbase
> > drwxr-xr-x - oozie hdfs 0 2015-11-24 10:05
> > /user/oozie/share/lib/lib_20151124100459/hcatalog
> > drwxr-xr-x - oozie hdfs 0 2015-11-24 10:05
> > /user/oozie/share/lib/lib_20151124100459/hive
> > drwxr-xr-x - oozie hdfs 0 2015-11-24 10:05
> > /user/oozie/share/lib/lib_20151124100459/hive2
> > drwxr-xr-x - oozie hdfs 0 2015-11-24 10:05
> > /user/oozie/share/lib/lib_20151124100459/mapreduce-streaming
> > drwxr-xr-x - oozie hdfs 0 2015-11-24 10:05
> > /user/oozie/share/lib/lib_20151124100459/oozie
> > drwxr-xr-x - oozie hdfs 0 2015-11-24 10:05
> > /user/oozie/share/lib/lib_20151124100459/pig
> > -rwxr-xr-x 3 oozie hdfs 1396 2015-11-24 10:05
> > /user/oozie/share/lib/lib_20151124100459/sharelib.properties
> > drwxr-xr-x - oozie hdfs 0 2015-11-24 10:05
> > /user/oozie/share/lib/lib_20151124100459/spark
> > drwxr-xr-x - oozie hdfs 0 2015-11-24 10:05
> > /user/oozie/share/lib/lib_20151124100459/sqoop
> > ###
> >
> > My questions are :
> > Q1 - When I execute my oozie workflow containing my java action, which
> > libraries from the sharelibs are available for my java project ? All ?
> > Nothing ?
> >
> > Q2 - Do I have to add in my workflow.xml the property
> > "oozie.action.sharelib.for.java" with specific values like
> > hcatalog,hive,pig,hbase ?
> >
> > Thank you in advance for your help !
> >
> > Best regards.
> >
> > Morgrim
> >
>
>
>
Re: About java action and sharelibs
Posted by 권병창 <n_...@naver.com>.
Hi.
Q1. oozie
Q2. You should add oozie.action.sharelib.for.java=xxx,xxx to job.properties not workflow.xml
2016. 4. 4., 오후 8:06, Morgrim Murdargent <mo...@gmail.com> 작성:
Hey everyone.
Has anyone any idea about this ?
Best regards.
Morgrim.
On Thu, Mar 24, 2016 at 9:57 AM, Morgrim Murdargent <morgrim.oozie@gmail.com
> wrote:
> Hello everyone.
>
> I send you this mail because I have a question related to the java actions
> in oozie and to the sharelibs.
>
> Here is my Use Case :
> 1. I have a simple workflow xml with a java action
> 2. I have a job properties with the property
> "oozie.use.system.libpath=true"
> 3. I have the following oozie sharelibs :
> ###
> # hdfs dfs -ls /user/oozie/share/lib/
> Found 1 items
> drwxr-xr-x - oozie hdfs 0 2016-03-14 18:30
> /user/oozie/share/lib/lib_20151124100459
>
> # hdfs dfs -ls /user/oozie/share/lib/*
> Found 11 items
> drwxr-xr-x - oozie hdfs 0 2015-11-24 10:05
> /user/oozie/share/lib/lib_20151124100459/distcp
> drwxr-xr-x - oozie hdfs 0 2016-03-14 18:32
> /user/oozie/share/lib/lib_20151124100459/hbase
> drwxr-xr-x - oozie hdfs 0 2015-11-24 10:05
> /user/oozie/share/lib/lib_20151124100459/hcatalog
> drwxr-xr-x - oozie hdfs 0 2015-11-24 10:05
> /user/oozie/share/lib/lib_20151124100459/hive
> drwxr-xr-x - oozie hdfs 0 2015-11-24 10:05
> /user/oozie/share/lib/lib_20151124100459/hive2
> drwxr-xr-x - oozie hdfs 0 2015-11-24 10:05
> /user/oozie/share/lib/lib_20151124100459/mapreduce-streaming
> drwxr-xr-x - oozie hdfs 0 2015-11-24 10:05
> /user/oozie/share/lib/lib_20151124100459/oozie
> drwxr-xr-x - oozie hdfs 0 2015-11-24 10:05
> /user/oozie/share/lib/lib_20151124100459/pig
> -rwxr-xr-x 3 oozie hdfs 1396 2015-11-24 10:05
> /user/oozie/share/lib/lib_20151124100459/sharelib.properties
> drwxr-xr-x - oozie hdfs 0 2015-11-24 10:05
> /user/oozie/share/lib/lib_20151124100459/spark
> drwxr-xr-x - oozie hdfs 0 2015-11-24 10:05
> /user/oozie/share/lib/lib_20151124100459/sqoop
> ###
>
> My questions are :
> Q1 - When I execute my oozie workflow containing my java action, which
> libraries from the sharelibs are available for my java project ? All ?
> Nothing ?
>
> Q2 - Do I have to add in my workflow.xml the property
> "oozie.action.sharelib.for.java" with specific values like
> hcatalog,hive,pig,hbase ?
>
> Thank you in advance for your help !
>
> Best regards.
>
> Morgrim
>
Re: About java action and sharelibs
Posted by Morgrim Murdargent <mo...@gmail.com>.
Hey everyone.
Has anyone any idea about this ?
Best regards.
Morgrim.
On Thu, Mar 24, 2016 at 9:57 AM, Morgrim Murdargent <morgrim.oozie@gmail.com
> wrote:
> Hello everyone.
>
> I send you this mail because I have a question related to the java actions
> in oozie and to the sharelibs.
>
> Here is my Use Case :
> 1. I have a simple workflow xml with a java action
> 2. I have a job properties with the property
> "oozie.use.system.libpath=true"
> 3. I have the following oozie sharelibs :
> ###
> # hdfs dfs -ls /user/oozie/share/lib/
> Found 1 items
> drwxr-xr-x - oozie hdfs 0 2016-03-14 18:30
> /user/oozie/share/lib/lib_20151124100459
>
> # hdfs dfs -ls /user/oozie/share/lib/*
> Found 11 items
> drwxr-xr-x - oozie hdfs 0 2015-11-24 10:05
> /user/oozie/share/lib/lib_20151124100459/distcp
> drwxr-xr-x - oozie hdfs 0 2016-03-14 18:32
> /user/oozie/share/lib/lib_20151124100459/hbase
> drwxr-xr-x - oozie hdfs 0 2015-11-24 10:05
> /user/oozie/share/lib/lib_20151124100459/hcatalog
> drwxr-xr-x - oozie hdfs 0 2015-11-24 10:05
> /user/oozie/share/lib/lib_20151124100459/hive
> drwxr-xr-x - oozie hdfs 0 2015-11-24 10:05
> /user/oozie/share/lib/lib_20151124100459/hive2
> drwxr-xr-x - oozie hdfs 0 2015-11-24 10:05
> /user/oozie/share/lib/lib_20151124100459/mapreduce-streaming
> drwxr-xr-x - oozie hdfs 0 2015-11-24 10:05
> /user/oozie/share/lib/lib_20151124100459/oozie
> drwxr-xr-x - oozie hdfs 0 2015-11-24 10:05
> /user/oozie/share/lib/lib_20151124100459/pig
> -rwxr-xr-x 3 oozie hdfs 1396 2015-11-24 10:05
> /user/oozie/share/lib/lib_20151124100459/sharelib.properties
> drwxr-xr-x - oozie hdfs 0 2015-11-24 10:05
> /user/oozie/share/lib/lib_20151124100459/spark
> drwxr-xr-x - oozie hdfs 0 2015-11-24 10:05
> /user/oozie/share/lib/lib_20151124100459/sqoop
> ###
>
> My questions are :
> Q1 - When I execute my oozie workflow containing my java action, which
> libraries from the sharelibs are available for my java project ? All ?
> Nothing ?
>
> Q2 - Do I have to add in my workflow.xml the property
> "oozie.action.sharelib.for.java" with specific values like
> hcatalog,hive,pig,hbase ?
>
> Thank you in advance for your help !
>
> Best regards.
>
> Morgrim
>