You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@oozie.apache.org by Filix Suo <sy...@gmail.com> on 2014/08/14 04:49:42 UTC

share lib does not exist

hi all,

I build and install oozie, then I try to run the examples in oozie. 
It throw a org.apache.oozie.action.ActionExecutorException: File /user/filix/share/lib does not exist. 

2014-08-14 10:44:23,891  INFO ActionStartXCommand:541 - SERVER[Filix-MacBook-Pro.local] USER[filix] GROUP[-] TOKEN[] APP[shell-wf] JOB[0000000-140814104417138-oozie-fili-W] ACTION[0000000-140814104417138-oozie-fili-W@:start:] Start action [0000000-140814104417138-oozie-fili-W@:start:] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10]
2014-08-14 10:44:23,892  INFO ActionStartXCommand:541 - SERVER[Filix-MacBook-Pro.local] USER[filix] GROUP[-] TOKEN[] APP[shell-wf] JOB[0000000-140814104417138-oozie-fili-W] ACTION[0000000-140814104417138-oozie-fili-W@:start:] [***0000000-140814104417138-oozie-fili-W@:start:***]Action status=DONE
2014-08-14 10:44:23,892  INFO ActionStartXCommand:541 - SERVER[Filix-MacBook-Pro.local] USER[filix] GROUP[-] TOKEN[] APP[shell-wf] JOB[0000000-140814104417138-oozie-fili-W] ACTION[0000000-140814104417138-oozie-fili-W@:start:] [***0000000-140814104417138-oozie-fili-W@:start:***]Action updated in DB!
2014-08-14 10:44:24,136  INFO ActionStartXCommand:541 - SERVER[Filix-MacBook-Pro.local] USER[filix] GROUP[-] TOKEN[] APP[shell-wf] JOB[0000000-140814104417138-oozie-fili-W] ACTION[0000000-140814104417138-oozie-fili-W@shell-node] Start action [0000000-140814104417138-oozie-fili-W@shell-node] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10]
2014-08-14 10:44:24,220  WARN ActionStartXCommand:544 - SERVER[Filix-MacBook-Pro.local] USER[filix] GROUP[-] TOKEN[] APP[shell-wf] JOB[0000000-140814104417138-oozie-fili-W] ACTION[0000000-140814104417138-oozie-fili-W@shell-node] Error starting action [shell-node]. ErrorType [FAILED], ErrorCode [It should never happen], Message [File /user/filix/share/lib/lib_20140814100844 does not exist]
org.apache.oozie.action.ActionExecutorException: File /user/filix/share/lib/lib_20140814100844 does not exist
	at org.apache.oozie.action.hadoop.JavaActionExecutor.addSystemShareLibForAction(JavaActionExecutor.java:620)
	at org.apache.oozie.action.hadoop.JavaActionExecutor.addAllShareLibs(JavaActionExecutor.java:698)
	at org.apache.oozie.action.hadoop.JavaActionExecutor.setLibFilesArchives(JavaActionExecutor.java:689)
	at org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:884)
	at org.apache.oozie.action.hadoop.JavaActionExecutor.start(JavaActionExecutor.java:1134)
	at org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:228)
	at org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:63)
	at org.apache.oozie.command.XCommand.call(XCommand.java:281)
	at org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:323)
	at org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:252)
	at org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:174)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:744)
2014-08-14 10:44:24,222  WARN ActionStartXCommand:544 - SERVER[Filix-MacBook-Pro.local] USER[filix] GROUP[-] TOKEN[] APP[shell-wf] JOB[0000000-140814104417138-oozie-fili-W] ACTION[0000000-140814104417138-oozie-fili-W@shell-node] Failing Job due to failed action [shell-node]
2014-08-14 10:44:24,224  WARN LiteWorkflowInstance:544 - SERVER[Filix-MacBook-Pro.local] USER[filix] GROUP[-] TOKEN[] APP[shell-wf] JOB[0000000-140814104417138-oozie-fili-W] ACTION[0000000-140814104417138-oozie-fili-W@shell-node] Workflow Failed. Failing node [shell-node]
2014-08-14 10:44:24,370  INFO KillXCommand:541 - SERVER[Filix-MacBook-Pro.local] USER[filix] GROUP[-] TOKEN[] APP[shell-wf] JOB[0000000-140814104417138-oozie-fili-W] ACTION[-] STARTED WorkflowKillXCommand for jobId=0000000-140814104417138-oozie-fili-W
2014-08-14 10:44:24,394  INFO KillXCommand:541 - SERVER[Filix-MacBook-Pro.local] USER[filix] GROUP[-] TOKEN[] APP[shell-wf] JOB[0000000-140814104417138-oozie-fili-W] ACTION[-] ENDED WorkflowKillXCommand for jobId=0000000-140814104417138-oozie-fili-W


However I have uploaded the share lib to hdfs:

$ hadoop fs -ls /user/filix/share/lib/lib_20140814100844
 Found 8 items
drwxr-xr-x   - filix supergroup          0 2014-08-14 10:08 /user/filix/share/lib/lib_20140814100844/distcp
drwxr-xr-x   - filix supergroup          0 2014-08-14 10:08 /user/filix/share/lib/lib_20140814100844/hcatalog
drwxr-xr-x   - filix supergroup          0 2014-08-14 10:08 /user/filix/share/lib/lib_20140814100844/hive
drwxr-xr-x   - filix supergroup          0 2014-08-14 10:08 /user/filix/share/lib/lib_20140814100844/mapreduce-streaming
drwxr-xr-x   - filix supergroup          0 2014-08-14 10:08 /user/filix/share/lib/lib_20140814100844/oozie
drwxr-xr-x   - filix supergroup          0 2014-08-14 10:08 /user/filix/share/lib/lib_20140814100844/pig
-rw-r--r--   3 filix supergroup       1384 2014-08-14 10:08 /user/filix/share/lib/lib_20140814100844/sharelib.properties
drwxr-xr-x   - filix supergroup          0 2014-08-14 10:08 /user/filix/share/lib/lib_20140814100844/sqoop

here is my conf/oozie-site.xml:
<property>
    <name>oozie.service.WorkflowAppService.system.libpath</name>
    <value>/user/${user.name}/share/lib/lib_20140814100844</value>
    <description>
        System library path to use for workflow applications.
        This path is added to workflow application if their job properties sets
        the property 'oozie.use.system.libpath' to true.
    </description>
</property>


I set the  oozie.use.system.libpath=false in job.properties, but it seems does not work(still throw the exception).

My hadoop is v2.2.0 and oozie is4.2.0-SNAPSHOT.

Anyone help?

Best regards!

Re: share lib does not exist

Posted by Filix Suo <sy...@gmail.com>.
Hi Purshotam,

Thanks for your reply.
I have setter the oozie.service.WorkflowAppService.system.libpath to /user/${user.name}/share/lib.
And this is the error log:
 
2014-08-17 10:37:22,452  INFO ActionStartXCommand:541 - SERVER[Filix-MacBook-Pro.local] USER[filix] GROUP[-] TOKEN[] APP[shell-wf] JOB[0000000-140817103542240-oozie-fili-W] ACTION[0000000-140817103542240-oozie-fili-W@:start:] Start action [0000000-140817103542240-oozie-fili-W@:start:] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10]
2014-08-17 10:37:22,452  INFO ActionStartXCommand:541 - SERVER[Filix-MacBook-Pro.local] USER[filix] GROUP[-] TOKEN[] APP[shell-wf] JOB[0000000-140817103542240-oozie-fili-W] ACTION[0000000-140817103542240-oozie-fili-W@:start:] [***0000000-140817103542240-oozie-fili-W@:start:***]Action status=DONE
2014-08-17 10:37:22,453  INFO ActionStartXCommand:541 - SERVER[Filix-MacBook-Pro.local] USER[filix] GROUP[-] TOKEN[] APP[shell-wf] JOB[0000000-140817103542240-oozie-fili-W] ACTION[0000000-140817103542240-oozie-fili-W@:start:] [***0000000-140817103542240-oozie-fili-W@:start:***]Action updated in DB!
2014-08-17 10:37:22,674  INFO ActionStartXCommand:541 - SERVER[Filix-MacBook-Pro.local] USER[filix] GROUP[-] TOKEN[] APP[shell-wf] JOB[0000000-140817103542240-oozie-fili-W] ACTION[0000000-140817103542240-oozie-fili-W@shell-node] Start action [0000000-140817103542240-oozie-fili-W@shell-node] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10]
2014-08-17 10:37:22,751  WARN ActionStartXCommand:544 - SERVER[Filix-MacBook-Pro.local] USER[filix] GROUP[-] TOKEN[] APP[shell-wf] JOB[0000000-140817103542240-oozie-fili-W] ACTION[0000000-140817103542240-oozie-fili-W@shell-node] Error starting action [shell-node]. ErrorType [FAILED], ErrorCode [It should never happen], Message [File /user/filix/share/lib does not exist]
org.apache.oozie.action.ActionExecutorException: File /user/filix/share/lib does not exist
	at org.apache.oozie.action.hadoop.JavaActionExecutor.addSystemShareLibForAction(JavaActionExecutor.java:620)
	at org.apache.oozie.action.hadoop.JavaActionExecutor.addAllShareLibs(JavaActionExecutor.java:698)
	at org.apache.oozie.action.hadoop.JavaActionExecutor.setLibFilesArchives(JavaActionExecutor.java:689)
	at org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:884)
	at org.apache.oozie.action.hadoop.JavaActionExecutor.start(JavaActionExecutor.java:1134)
	at org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:228)
	at org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:63)
	at org.apache.oozie.command.XCommand.call(XCommand.java:281)
	at org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:323)
	at org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:252)
	at org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:174)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:744)
2014-08-17 10:37:22,752  WARN ActionStartXCommand:544 - SERVER[Filix-MacBook-Pro.local] USER[filix] GROUP[-] TOKEN[] APP[shell-wf] JOB[0000000-140817103542240-oozie-fili-W] ACTION[0000000-140817103542240-oozie-fili-W@shell-node] Failing Job due to failed action [shell-node]
2014-08-17 10:37:22,754  WARN LiteWorkflowInstance:544 - SERVER[Filix-MacBook-Pro.local] USER[filix] GROUP[-] TOKEN[] APP[shell-wf] JOB[0000000-140817103542240-oozie-fili-W] ACTION[0000000-140817103542240-oozie-fili-W@shell-node] Workflow Failed. Failing node [shell-node]
2014-08-17 10:37:22,858  INFO KillXCommand:541 - SERVER[Filix-MacBook-Pro.local] USER[filix] GROUP[-] TOKEN[] APP[shell-wf] JOB[0000000-140817103542240-oozie-fili-W] ACTION[-] STARTED WorkflowKillXCommand for jobId=0000000-140817103542240-oozie-fili-W
2014-08-17 10:37:22,876  INFO KillXCommand:541 - SERVER[Filix-MacBook-Pro.local] USER[filix] GROUP[-] TOKEN[] APP[shell-wf] JOB[0000000-140817103542240-oozie-fili-W] ACTION[-] ENDED WorkflowKillXCommand for jobId=0000000-140817103542240-oozie-fili-W


在 2014年8月15日,上午10:21,Purshotam Shah <pu...@yahoo-inc.com.INVALID> 写道:

> Please ignore the previous mail. Your sharelib setup looks correct.
> Somehow I didn’t see the last part of email.
> 
> Please share the error message after setting below property.
> <property>
> 	<name>oozie.service.WorkflowAppService.system.libpath</name>
> 	<value>/user/${user.name}/share/lib</value>
> </property>
> 
> 
> Puru.
> 
> 
> On 8/14/14, 7:11 PM, "Purshotam Shah" <pu...@yahoo-inc.com.INVALID>
> wrote:
> 
>> Filix,
>> Oozie doesn’t create sharelib dir(parent dir), it expect that it will be
>> there with proper permission.
>> Can you try creating "/user/filix/share/lib”?
>> 
>> How are you deploying sharelib?
>> 
>> Please refer below Doc.
>> https://oozie.apache.org/docs/4.0.1/AG_Install.html#Oozie_Share_Lib
>> 
>> https://oozie.apache.org/docs/4.0.1/AG_Install.html#Oozie_Server_Setup
>> 
>> 
>> -----
>> sharelib create -fs FS_URI [-locallib SHARED_LIBRARY] (create sharelib for
>> oozie,"
>> 				FS_URI is the fs.default.name"
>> 				for hdfs uri; SHARED_LIBRARY, path to the"
>> 				Oozie sharelib to install, it can be a tarball"
>> 				or an expanded version of it. If ommited,"
>> 				the Oozie sharelib tarball from the Oozie"
>> 				installation directory will be used)"
>> 				(action failes if sharelib is already installed"
>> 				in HDFS)”
>> 
>> ----
>> Let us know it still fails.
>> 
>> Puru.
>> 
>> 
>> 
>> 
>> 
>> On 8/14/14, 6:53 PM, "Filix Suo" <sy...@gmail.com> wrote:
>> 
>>> Hi Robert,
>>> 
>>> Thanks for your answer.
>>> I tried as you said, but it still throw the error.
>>> 
>>> 
>>> 在 2014年8月15日,上午1:36,Robert Kanter <rk...@cloudera.com> 写道:
>>> 
>>>> Hi Filix,
>>>> 
>>>> You should not include the lib_<timestamp> part of the path in the
>>>> oozie-site property.  Oozie knows to look for the latest
>>>> lib_<timestamp>
>>>> dir under that location.
>>>> 
>>>> e.g
>>>> <property>
>>>>   <name>oozie.service.WorkflowAppService.system.libpath</name>
>>>>   <value>/user/${user.name}/share/lib</value>
>>>>   <description>
>>>>       System library path to use for workflow applications.
>>>>       This path is added to workflow application if their job
>>>> properties
>>>> sets
>>>>       the property 'oozie.use.system.libpath' to true.
>>>>   </description>
>>>> </property>
>>>> 
>>>> 
>>>> 
>>>> 
>>>> 
>>>> On Wed, Aug 13, 2014 at 7:49 PM, Filix Suo <sy...@gmail.com> wrote:
>>>> 
>>>>> hi all,
>>>>> 
>>>>> I build and install oozie, then I try to run the examples in oozie.
>>>>> It throw a org.apache.oozie.action.ActionExecutorException: File
>>>>> /user/filix/share/lib does not exist.
>>>>> 
>>>>> 2014-08-14 10:44:23,891  INFO ActionStartXCommand:541 -
>>>>> SERVER[Filix-MacBook-Pro.local] USER[filix] GROUP[-] TOKEN[]
>>>>> APP[shell-wf]
>>>>> JOB[0000000-140814104417138-oozie-fili-W]
>>>>> ACTION[0000000-140814104417138-oozie-fili-W@:start:] Start action
>>>>> [0000000-140814104417138-oozie-fili-W@:start:] with user-retry state :
>>>>> userRetryCount [0], userRetryMax [0], userRetryInterval [10]
>>>>> 2014-08-14 10:44:23,892  INFO ActionStartXCommand:541 -
>>>>> SERVER[Filix-MacBook-Pro.local] USER[filix] GROUP[-] TOKEN[]
>>>>> APP[shell-wf]
>>>>> JOB[0000000-140814104417138-oozie-fili-W]
>>>>> ACTION[0000000-140814104417138-oozie-fili-W@:start:]
>>>>> [***0000000-140814104417138-oozie-fili-W@:start:***]Action status=DONE
>>>>> 2014-08-14 10:44:23,892  INFO ActionStartXCommand:541 -
>>>>> SERVER[Filix-MacBook-Pro.local] USER[filix] GROUP[-] TOKEN[]
>>>>> APP[shell-wf]
>>>>> JOB[0000000-140814104417138-oozie-fili-W]
>>>>> ACTION[0000000-140814104417138-oozie-fili-W@:start:]
>>>>> [***0000000-140814104417138-oozie-fili-W@:start:***]Action updated in
>>>>> DB!
>>>>> 2014-08-14 10:44:24,136  INFO ActionStartXCommand:541 -
>>>>> SERVER[Filix-MacBook-Pro.local] USER[filix] GROUP[-] TOKEN[]
>>>>> APP[shell-wf]
>>>>> JOB[0000000-140814104417138-oozie-fili-W]
>>>>> ACTION[0000000-140814104417138-oozie-fili-W@shell-node] Start action
>>>>> [0000000-140814104417138-oozie-fili-W@shell-node] with user-retry
>>>>> state :
>>>>> userRetryCount [0], userRetryMax [0], userRetryInterval [10]
>>>>> 2014-08-14 10:44:24,220  WARN ActionStartXCommand:544 -
>>>>> SERVER[Filix-MacBook-Pro.local] USER[filix] GROUP[-] TOKEN[]
>>>>> APP[shell-wf]
>>>>> JOB[0000000-140814104417138-oozie-fili-W]
>>>>> ACTION[0000000-140814104417138-oozie-fili-W@shell-node] Error starting
>>>>> action [shell-node]. ErrorType [FAILED], ErrorCode [It should never
>>>>> happen], Message [File /user/filix/share/lib/lib_20140814100844 does
>>>>> not
>>>>> exist]
>>>>> org.apache.oozie.action.ActionExecutorException: File
>>>>> /user/filix/share/lib/lib_20140814100844 does not exist
>>>>>       at
>>>>> 
>>>>> org.apache.oozie.action.hadoop.JavaActionExecutor.addSystemShareLibForA
>>>>> c
>>>>> tion(JavaActionExecutor.java:620)
>>>>>       at
>>>>> 
>>>>> org.apache.oozie.action.hadoop.JavaActionExecutor.addAllShareLibs(JavaA
>>>>> c
>>>>> tionExecutor.java:698)
>>>>>       at
>>>>> 
>>>>> org.apache.oozie.action.hadoop.JavaActionExecutor.setLibFilesArchives(J
>>>>> a
>>>>> vaActionExecutor.java:689)
>>>>>       at
>>>>> 
>>>>> org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaAc
>>>>> t
>>>>> ionExecutor.java:884)
>>>>>       at
>>>>> 
>>>>> org.apache.oozie.action.hadoop.JavaActionExecutor.start(JavaActionExecu
>>>>> t
>>>>> or.java:1134)
>>>>>       at
>>>>> 
>>>>> org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCom
>>>>> m
>>>>> and.java:228)
>>>>>       at
>>>>> 
>>>>> org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCom
>>>>> m
>>>>> and.java:63)
>>>>>       at org.apache.oozie.command.XCommand.call(XCommand.java:281)
>>>>>       at
>>>>> 
>>>>> org.apache.oozie.service.CallableQueueService$CompositeCallable.call(Ca
>>>>> l
>>>>> lableQueueService.java:323)
>>>>>       at
>>>>> 
>>>>> org.apache.oozie.service.CallableQueueService$CompositeCallable.call(Ca
>>>>> l
>>>>> lableQueueService.java:252)
>>>>>       at
>>>>> 
>>>>> org.apache.oozie.service.CallableQueueService$CallableWrapper.run(Calla
>>>>> b
>>>>> leQueueService.java:174)
>>>>>       at
>>>>> 
>>>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.ja
>>>>> v
>>>>> a:1145)
>>>>>       at
>>>>> 
>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.j
>>>>> a
>>>>> va:615)
>>>>>       at java.lang.Thread.run(Thread.java:744)
>>>>> 2014-08-14 10:44:24,222  WARN ActionStartXCommand:544 -
>>>>> SERVER[Filix-MacBook-Pro.local] USER[filix] GROUP[-] TOKEN[]
>>>>> APP[shell-wf]
>>>>> JOB[0000000-140814104417138-oozie-fili-W]
>>>>> ACTION[0000000-140814104417138-oozie-fili-W@shell-node] Failing Job
>>>>> due
>>>>> to failed action [shell-node]
>>>>> 2014-08-14 10:44:24,224  WARN LiteWorkflowInstance:544 -
>>>>> SERVER[Filix-MacBook-Pro.local] USER[filix] GROUP[-] TOKEN[]
>>>>> APP[shell-wf]
>>>>> JOB[0000000-140814104417138-oozie-fili-W]
>>>>> ACTION[0000000-140814104417138-oozie-fili-W@shell-node] Workflow
>>>>> Failed.
>>>>> Failing node [shell-node]
>>>>> 2014-08-14 10:44:24,370  INFO KillXCommand:541 -
>>>>> SERVER[Filix-MacBook-Pro.local] USER[filix] GROUP[-] TOKEN[]
>>>>> APP[shell-wf]
>>>>> JOB[0000000-140814104417138-oozie-fili-W] ACTION[-] STARTED
>>>>> WorkflowKillXCommand for jobId=0000000-140814104417138-oozie-fili-W
>>>>> 2014-08-14 10:44:24,394  INFO KillXCommand:541 -
>>>>> SERVER[Filix-MacBook-Pro.local] USER[filix] GROUP[-] TOKEN[]
>>>>> APP[shell-wf]
>>>>> JOB[0000000-140814104417138-oozie-fili-W] ACTION[-] ENDED
>>>>> WorkflowKillXCommand for jobId=0000000-140814104417138-oozie-fili-W
>>>>> 
>>>>> 
>>>>> However I have uploaded the share lib to hdfs:
>>>>> 
>>>>> $ hadoop fs -ls /user/filix/share/lib/lib_20140814100844
>>>>> Found 8 items
>>>>> drwxr-xr-x   - filix supergroup          0 2014-08-14 10:08
>>>>> /user/filix/share/lib/lib_20140814100844/distcp
>>>>> drwxr-xr-x   - filix supergroup          0 2014-08-14 10:08
>>>>> /user/filix/share/lib/lib_20140814100844/hcatalog
>>>>> drwxr-xr-x   - filix supergroup          0 2014-08-14 10:08
>>>>> /user/filix/share/lib/lib_20140814100844/hive
>>>>> drwxr-xr-x   - filix supergroup          0 2014-08-14 10:08
>>>>> /user/filix/share/lib/lib_20140814100844/mapreduce-streaming
>>>>> drwxr-xr-x   - filix supergroup          0 2014-08-14 10:08
>>>>> /user/filix/share/lib/lib_20140814100844/oozie
>>>>> drwxr-xr-x   - filix supergroup          0 2014-08-14 10:08
>>>>> /user/filix/share/lib/lib_20140814100844/pig
>>>>> -rw-r--r--   3 filix supergroup       1384 2014-08-14 10:08
>>>>> /user/filix/share/lib/lib_20140814100844/sharelib.properties
>>>>> drwxr-xr-x   - filix supergroup          0 2014-08-14 10:08
>>>>> /user/filix/share/lib/lib_20140814100844/sqoop
>>>>> 
>>>>> here is my conf/oozie-site.xml:
>>>>> <property>
>>>>>   <name>oozie.service.WorkflowAppService.system.libpath</name>
>>>>>   <value>/user/${user.name}/share/lib/lib_20140814100844</value>
>>>>>   <description>
>>>>>       System library path to use for workflow applications.
>>>>>       This path is added to workflow application if their job
>>>>> properties
>>>>> sets
>>>>>       the property 'oozie.use.system.libpath' to true.
>>>>>   </description>
>>>>> </property>
>>>>> 
>>>>> 
>>>>> I set the  oozie.use.system.libpath=false in job.properties, but it
>>>>> seems
>>>>> does not work(still throw the exception).
>>>>> 
>>>>> My hadoop is v2.2.0 and oozie is4.2.0-SNAPSHOT.
>>>>> 
>>>>> Anyone help?
>>>>> 
>>>>> Best regards!
>>>>> 
>>> 
>> 
> 


Re: share lib does not exist

Posted by Purshotam Shah <pu...@yahoo-inc.com.INVALID>.
Please ignore the previous mail. Your sharelib setup looks correct.
Somehow I didn’t see the last part of email.

Please share the error message after setting below property.
<property>
	<name>oozie.service.WorkflowAppService.system.libpath</name>
	<value>/user/${user.name}/share/lib</value>
</property>


Puru.


On 8/14/14, 7:11 PM, "Purshotam Shah" <pu...@yahoo-inc.com.INVALID>
wrote:

>Filix,
>  Oozie doesn’t create sharelib dir(parent dir), it expect that it will be
>there with proper permission.
>Can you try creating "/user/filix/share/lib”?
>
>How are you deploying sharelib?
>
>Please refer below Doc.
>https://oozie.apache.org/docs/4.0.1/AG_Install.html#Oozie_Share_Lib
>
>https://oozie.apache.org/docs/4.0.1/AG_Install.html#Oozie_Server_Setup
>
>
>-----
>sharelib create -fs FS_URI [-locallib SHARED_LIBRARY] (create sharelib for
>oozie,"
>				FS_URI is the fs.default.name"
>				for hdfs uri; SHARED_LIBRARY, path to the"
>				Oozie sharelib to install, it can be a tarball"
>				or an expanded version of it. If ommited,"
>				the Oozie sharelib tarball from the Oozie"
>				installation directory will be used)"
>				(action failes if sharelib is already installed"
>				in HDFS)”
>
>----
>Let us know it still fails.
>
>Puru.
>
>
>
>
>
>On 8/14/14, 6:53 PM, "Filix Suo" <sy...@gmail.com> wrote:
>
>>Hi Robert,
>>
>>Thanks for your answer.
>>I tried as you said, but it still throw the error.
>>
>>
>>在 2014年8月15日,上午1:36,Robert Kanter <rk...@cloudera.com> 写道:
>>
>>> Hi Filix,
>>> 
>>> You should not include the lib_<timestamp> part of the path in the
>>> oozie-site property.  Oozie knows to look for the latest
>>>lib_<timestamp>
>>> dir under that location.
>>> 
>>> e.g
>>> <property>
>>>    <name>oozie.service.WorkflowAppService.system.libpath</name>
>>>    <value>/user/${user.name}/share/lib</value>
>>>    <description>
>>>        System library path to use for workflow applications.
>>>        This path is added to workflow application if their job
>>>properties
>>> sets
>>>        the property 'oozie.use.system.libpath' to true.
>>>    </description>
>>> </property>
>>> 
>>> 
>>> 
>>> 
>>> 
>>> On Wed, Aug 13, 2014 at 7:49 PM, Filix Suo <sy...@gmail.com> wrote:
>>> 
>>>> hi all,
>>>> 
>>>> I build and install oozie, then I try to run the examples in oozie.
>>>> It throw a org.apache.oozie.action.ActionExecutorException: File
>>>> /user/filix/share/lib does not exist.
>>>> 
>>>> 2014-08-14 10:44:23,891  INFO ActionStartXCommand:541 -
>>>> SERVER[Filix-MacBook-Pro.local] USER[filix] GROUP[-] TOKEN[]
>>>>APP[shell-wf]
>>>> JOB[0000000-140814104417138-oozie-fili-W]
>>>> ACTION[0000000-140814104417138-oozie-fili-W@:start:] Start action
>>>> [0000000-140814104417138-oozie-fili-W@:start:] with user-retry state :
>>>> userRetryCount [0], userRetryMax [0], userRetryInterval [10]
>>>> 2014-08-14 10:44:23,892  INFO ActionStartXCommand:541 -
>>>> SERVER[Filix-MacBook-Pro.local] USER[filix] GROUP[-] TOKEN[]
>>>>APP[shell-wf]
>>>> JOB[0000000-140814104417138-oozie-fili-W]
>>>> ACTION[0000000-140814104417138-oozie-fili-W@:start:]
>>>> [***0000000-140814104417138-oozie-fili-W@:start:***]Action status=DONE
>>>> 2014-08-14 10:44:23,892  INFO ActionStartXCommand:541 -
>>>> SERVER[Filix-MacBook-Pro.local] USER[filix] GROUP[-] TOKEN[]
>>>>APP[shell-wf]
>>>> JOB[0000000-140814104417138-oozie-fili-W]
>>>> ACTION[0000000-140814104417138-oozie-fili-W@:start:]
>>>> [***0000000-140814104417138-oozie-fili-W@:start:***]Action updated in
>>>>DB!
>>>> 2014-08-14 10:44:24,136  INFO ActionStartXCommand:541 -
>>>> SERVER[Filix-MacBook-Pro.local] USER[filix] GROUP[-] TOKEN[]
>>>>APP[shell-wf]
>>>> JOB[0000000-140814104417138-oozie-fili-W]
>>>> ACTION[0000000-140814104417138-oozie-fili-W@shell-node] Start action
>>>> [0000000-140814104417138-oozie-fili-W@shell-node] with user-retry
>>>>state :
>>>> userRetryCount [0], userRetryMax [0], userRetryInterval [10]
>>>> 2014-08-14 10:44:24,220  WARN ActionStartXCommand:544 -
>>>> SERVER[Filix-MacBook-Pro.local] USER[filix] GROUP[-] TOKEN[]
>>>>APP[shell-wf]
>>>> JOB[0000000-140814104417138-oozie-fili-W]
>>>> ACTION[0000000-140814104417138-oozie-fili-W@shell-node] Error starting
>>>> action [shell-node]. ErrorType [FAILED], ErrorCode [It should never
>>>> happen], Message [File /user/filix/share/lib/lib_20140814100844 does
>>>>not
>>>> exist]
>>>> org.apache.oozie.action.ActionExecutorException: File
>>>> /user/filix/share/lib/lib_20140814100844 does not exist
>>>>        at
>>>> 
>>>>org.apache.oozie.action.hadoop.JavaActionExecutor.addSystemShareLibForA
>>>>c
>>>>tion(JavaActionExecutor.java:620)
>>>>        at
>>>> 
>>>>org.apache.oozie.action.hadoop.JavaActionExecutor.addAllShareLibs(JavaA
>>>>c
>>>>tionExecutor.java:698)
>>>>        at
>>>> 
>>>>org.apache.oozie.action.hadoop.JavaActionExecutor.setLibFilesArchives(J
>>>>a
>>>>vaActionExecutor.java:689)
>>>>        at
>>>> 
>>>>org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaAc
>>>>t
>>>>ionExecutor.java:884)
>>>>        at
>>>> 
>>>>org.apache.oozie.action.hadoop.JavaActionExecutor.start(JavaActionExecu
>>>>t
>>>>or.java:1134)
>>>>        at
>>>> 
>>>>org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCom
>>>>m
>>>>and.java:228)
>>>>        at
>>>> 
>>>>org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCom
>>>>m
>>>>and.java:63)
>>>>        at org.apache.oozie.command.XCommand.call(XCommand.java:281)
>>>>        at
>>>> 
>>>>org.apache.oozie.service.CallableQueueService$CompositeCallable.call(Ca
>>>>l
>>>>lableQueueService.java:323)
>>>>        at
>>>> 
>>>>org.apache.oozie.service.CallableQueueService$CompositeCallable.call(Ca
>>>>l
>>>>lableQueueService.java:252)
>>>>        at
>>>> 
>>>>org.apache.oozie.service.CallableQueueService$CallableWrapper.run(Calla
>>>>b
>>>>leQueueService.java:174)
>>>>        at
>>>> 
>>>>java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.ja
>>>>v
>>>>a:1145)
>>>>        at
>>>> 
>>>>java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.j
>>>>a
>>>>va:615)
>>>>        at java.lang.Thread.run(Thread.java:744)
>>>> 2014-08-14 10:44:24,222  WARN ActionStartXCommand:544 -
>>>> SERVER[Filix-MacBook-Pro.local] USER[filix] GROUP[-] TOKEN[]
>>>>APP[shell-wf]
>>>> JOB[0000000-140814104417138-oozie-fili-W]
>>>> ACTION[0000000-140814104417138-oozie-fili-W@shell-node] Failing Job
>>>>due
>>>> to failed action [shell-node]
>>>> 2014-08-14 10:44:24,224  WARN LiteWorkflowInstance:544 -
>>>> SERVER[Filix-MacBook-Pro.local] USER[filix] GROUP[-] TOKEN[]
>>>>APP[shell-wf]
>>>> JOB[0000000-140814104417138-oozie-fili-W]
>>>> ACTION[0000000-140814104417138-oozie-fili-W@shell-node] Workflow
>>>>Failed.
>>>> Failing node [shell-node]
>>>> 2014-08-14 10:44:24,370  INFO KillXCommand:541 -
>>>> SERVER[Filix-MacBook-Pro.local] USER[filix] GROUP[-] TOKEN[]
>>>>APP[shell-wf]
>>>> JOB[0000000-140814104417138-oozie-fili-W] ACTION[-] STARTED
>>>> WorkflowKillXCommand for jobId=0000000-140814104417138-oozie-fili-W
>>>> 2014-08-14 10:44:24,394  INFO KillXCommand:541 -
>>>> SERVER[Filix-MacBook-Pro.local] USER[filix] GROUP[-] TOKEN[]
>>>>APP[shell-wf]
>>>> JOB[0000000-140814104417138-oozie-fili-W] ACTION[-] ENDED
>>>> WorkflowKillXCommand for jobId=0000000-140814104417138-oozie-fili-W
>>>> 
>>>> 
>>>> However I have uploaded the share lib to hdfs:
>>>> 
>>>> $ hadoop fs -ls /user/filix/share/lib/lib_20140814100844
>>>> Found 8 items
>>>> drwxr-xr-x   - filix supergroup          0 2014-08-14 10:08
>>>> /user/filix/share/lib/lib_20140814100844/distcp
>>>> drwxr-xr-x   - filix supergroup          0 2014-08-14 10:08
>>>> /user/filix/share/lib/lib_20140814100844/hcatalog
>>>> drwxr-xr-x   - filix supergroup          0 2014-08-14 10:08
>>>> /user/filix/share/lib/lib_20140814100844/hive
>>>> drwxr-xr-x   - filix supergroup          0 2014-08-14 10:08
>>>> /user/filix/share/lib/lib_20140814100844/mapreduce-streaming
>>>> drwxr-xr-x   - filix supergroup          0 2014-08-14 10:08
>>>> /user/filix/share/lib/lib_20140814100844/oozie
>>>> drwxr-xr-x   - filix supergroup          0 2014-08-14 10:08
>>>> /user/filix/share/lib/lib_20140814100844/pig
>>>> -rw-r--r--   3 filix supergroup       1384 2014-08-14 10:08
>>>> /user/filix/share/lib/lib_20140814100844/sharelib.properties
>>>> drwxr-xr-x   - filix supergroup          0 2014-08-14 10:08
>>>> /user/filix/share/lib/lib_20140814100844/sqoop
>>>> 
>>>> here is my conf/oozie-site.xml:
>>>> <property>
>>>>    <name>oozie.service.WorkflowAppService.system.libpath</name>
>>>>    <value>/user/${user.name}/share/lib/lib_20140814100844</value>
>>>>    <description>
>>>>        System library path to use for workflow applications.
>>>>        This path is added to workflow application if their job
>>>>properties
>>>> sets
>>>>        the property 'oozie.use.system.libpath' to true.
>>>>    </description>
>>>> </property>
>>>> 
>>>> 
>>>> I set the  oozie.use.system.libpath=false in job.properties, but it
>>>>seems
>>>> does not work(still throw the exception).
>>>> 
>>>> My hadoop is v2.2.0 and oozie is4.2.0-SNAPSHOT.
>>>> 
>>>> Anyone help?
>>>> 
>>>> Best regards!
>>>> 
>>
>


Re: share lib does not exist

Posted by Purshotam Shah <pu...@yahoo-inc.com.INVALID>.
Filix,
  Oozie doesn’t create sharelib dir(parent dir), it expect that it will be
there with proper permission.
Can you try creating "/user/filix/share/lib”?

How are you deploying sharelib?

Please refer below Doc.
https://oozie.apache.org/docs/4.0.1/AG_Install.html#Oozie_Share_Lib

https://oozie.apache.org/docs/4.0.1/AG_Install.html#Oozie_Server_Setup


-----
sharelib create -fs FS_URI [-locallib SHARED_LIBRARY] (create sharelib for
oozie,"
				FS_URI is the fs.default.name"
				for hdfs uri; SHARED_LIBRARY, path to the"
				Oozie sharelib to install, it can be a tarball"
				or an expanded version of it. If ommited,"
				the Oozie sharelib tarball from the Oozie"
				installation directory will be used)"
				(action failes if sharelib is already installed"
				in HDFS)”

----
Let us know it still fails.

Puru.





On 8/14/14, 6:53 PM, "Filix Suo" <sy...@gmail.com> wrote:

>Hi Robert,
>
>Thanks for your answer.
>I tried as you said, but it still throw the error.
>
>
>在 2014年8月15日,上午1:36,Robert Kanter <rk...@cloudera.com> 写道:
>
>> Hi Filix,
>> 
>> You should not include the lib_<timestamp> part of the path in the
>> oozie-site property.  Oozie knows to look for the latest lib_<timestamp>
>> dir under that location.
>> 
>> e.g
>> <property>
>>    <name>oozie.service.WorkflowAppService.system.libpath</name>
>>    <value>/user/${user.name}/share/lib</value>
>>    <description>
>>        System library path to use for workflow applications.
>>        This path is added to workflow application if their job
>>properties
>> sets
>>        the property 'oozie.use.system.libpath' to true.
>>    </description>
>> </property>
>> 
>> 
>> 
>> 
>> 
>> On Wed, Aug 13, 2014 at 7:49 PM, Filix Suo <sy...@gmail.com> wrote:
>> 
>>> hi all,
>>> 
>>> I build and install oozie, then I try to run the examples in oozie.
>>> It throw a org.apache.oozie.action.ActionExecutorException: File
>>> /user/filix/share/lib does not exist.
>>> 
>>> 2014-08-14 10:44:23,891  INFO ActionStartXCommand:541 -
>>> SERVER[Filix-MacBook-Pro.local] USER[filix] GROUP[-] TOKEN[]
>>>APP[shell-wf]
>>> JOB[0000000-140814104417138-oozie-fili-W]
>>> ACTION[0000000-140814104417138-oozie-fili-W@:start:] Start action
>>> [0000000-140814104417138-oozie-fili-W@:start:] with user-retry state :
>>> userRetryCount [0], userRetryMax [0], userRetryInterval [10]
>>> 2014-08-14 10:44:23,892  INFO ActionStartXCommand:541 -
>>> SERVER[Filix-MacBook-Pro.local] USER[filix] GROUP[-] TOKEN[]
>>>APP[shell-wf]
>>> JOB[0000000-140814104417138-oozie-fili-W]
>>> ACTION[0000000-140814104417138-oozie-fili-W@:start:]
>>> [***0000000-140814104417138-oozie-fili-W@:start:***]Action status=DONE
>>> 2014-08-14 10:44:23,892  INFO ActionStartXCommand:541 -
>>> SERVER[Filix-MacBook-Pro.local] USER[filix] GROUP[-] TOKEN[]
>>>APP[shell-wf]
>>> JOB[0000000-140814104417138-oozie-fili-W]
>>> ACTION[0000000-140814104417138-oozie-fili-W@:start:]
>>> [***0000000-140814104417138-oozie-fili-W@:start:***]Action updated in
>>>DB!
>>> 2014-08-14 10:44:24,136  INFO ActionStartXCommand:541 -
>>> SERVER[Filix-MacBook-Pro.local] USER[filix] GROUP[-] TOKEN[]
>>>APP[shell-wf]
>>> JOB[0000000-140814104417138-oozie-fili-W]
>>> ACTION[0000000-140814104417138-oozie-fili-W@shell-node] Start action
>>> [0000000-140814104417138-oozie-fili-W@shell-node] with user-retry
>>>state :
>>> userRetryCount [0], userRetryMax [0], userRetryInterval [10]
>>> 2014-08-14 10:44:24,220  WARN ActionStartXCommand:544 -
>>> SERVER[Filix-MacBook-Pro.local] USER[filix] GROUP[-] TOKEN[]
>>>APP[shell-wf]
>>> JOB[0000000-140814104417138-oozie-fili-W]
>>> ACTION[0000000-140814104417138-oozie-fili-W@shell-node] Error starting
>>> action [shell-node]. ErrorType [FAILED], ErrorCode [It should never
>>> happen], Message [File /user/filix/share/lib/lib_20140814100844 does
>>>not
>>> exist]
>>> org.apache.oozie.action.ActionExecutorException: File
>>> /user/filix/share/lib/lib_20140814100844 does not exist
>>>        at
>>> 
>>>org.apache.oozie.action.hadoop.JavaActionExecutor.addSystemShareLibForAc
>>>tion(JavaActionExecutor.java:620)
>>>        at
>>> 
>>>org.apache.oozie.action.hadoop.JavaActionExecutor.addAllShareLibs(JavaAc
>>>tionExecutor.java:698)
>>>        at
>>> 
>>>org.apache.oozie.action.hadoop.JavaActionExecutor.setLibFilesArchives(Ja
>>>vaActionExecutor.java:689)
>>>        at
>>> 
>>>org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaAct
>>>ionExecutor.java:884)
>>>        at
>>> 
>>>org.apache.oozie.action.hadoop.JavaActionExecutor.start(JavaActionExecut
>>>or.java:1134)
>>>        at
>>> 
>>>org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXComm
>>>and.java:228)
>>>        at
>>> 
>>>org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXComm
>>>and.java:63)
>>>        at org.apache.oozie.command.XCommand.call(XCommand.java:281)
>>>        at
>>> 
>>>org.apache.oozie.service.CallableQueueService$CompositeCallable.call(Cal
>>>lableQueueService.java:323)
>>>        at
>>> 
>>>org.apache.oozie.service.CallableQueueService$CompositeCallable.call(Cal
>>>lableQueueService.java:252)
>>>        at
>>> 
>>>org.apache.oozie.service.CallableQueueService$CallableWrapper.run(Callab
>>>leQueueService.java:174)
>>>        at
>>> 
>>>java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.jav
>>>a:1145)
>>>        at
>>> 
>>>java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.ja
>>>va:615)
>>>        at java.lang.Thread.run(Thread.java:744)
>>> 2014-08-14 10:44:24,222  WARN ActionStartXCommand:544 -
>>> SERVER[Filix-MacBook-Pro.local] USER[filix] GROUP[-] TOKEN[]
>>>APP[shell-wf]
>>> JOB[0000000-140814104417138-oozie-fili-W]
>>> ACTION[0000000-140814104417138-oozie-fili-W@shell-node] Failing Job due
>>> to failed action [shell-node]
>>> 2014-08-14 10:44:24,224  WARN LiteWorkflowInstance:544 -
>>> SERVER[Filix-MacBook-Pro.local] USER[filix] GROUP[-] TOKEN[]
>>>APP[shell-wf]
>>> JOB[0000000-140814104417138-oozie-fili-W]
>>> ACTION[0000000-140814104417138-oozie-fili-W@shell-node] Workflow
>>>Failed.
>>> Failing node [shell-node]
>>> 2014-08-14 10:44:24,370  INFO KillXCommand:541 -
>>> SERVER[Filix-MacBook-Pro.local] USER[filix] GROUP[-] TOKEN[]
>>>APP[shell-wf]
>>> JOB[0000000-140814104417138-oozie-fili-W] ACTION[-] STARTED
>>> WorkflowKillXCommand for jobId=0000000-140814104417138-oozie-fili-W
>>> 2014-08-14 10:44:24,394  INFO KillXCommand:541 -
>>> SERVER[Filix-MacBook-Pro.local] USER[filix] GROUP[-] TOKEN[]
>>>APP[shell-wf]
>>> JOB[0000000-140814104417138-oozie-fili-W] ACTION[-] ENDED
>>> WorkflowKillXCommand for jobId=0000000-140814104417138-oozie-fili-W
>>> 
>>> 
>>> However I have uploaded the share lib to hdfs:
>>> 
>>> $ hadoop fs -ls /user/filix/share/lib/lib_20140814100844
>>> Found 8 items
>>> drwxr-xr-x   - filix supergroup          0 2014-08-14 10:08
>>> /user/filix/share/lib/lib_20140814100844/distcp
>>> drwxr-xr-x   - filix supergroup          0 2014-08-14 10:08
>>> /user/filix/share/lib/lib_20140814100844/hcatalog
>>> drwxr-xr-x   - filix supergroup          0 2014-08-14 10:08
>>> /user/filix/share/lib/lib_20140814100844/hive
>>> drwxr-xr-x   - filix supergroup          0 2014-08-14 10:08
>>> /user/filix/share/lib/lib_20140814100844/mapreduce-streaming
>>> drwxr-xr-x   - filix supergroup          0 2014-08-14 10:08
>>> /user/filix/share/lib/lib_20140814100844/oozie
>>> drwxr-xr-x   - filix supergroup          0 2014-08-14 10:08
>>> /user/filix/share/lib/lib_20140814100844/pig
>>> -rw-r--r--   3 filix supergroup       1384 2014-08-14 10:08
>>> /user/filix/share/lib/lib_20140814100844/sharelib.properties
>>> drwxr-xr-x   - filix supergroup          0 2014-08-14 10:08
>>> /user/filix/share/lib/lib_20140814100844/sqoop
>>> 
>>> here is my conf/oozie-site.xml:
>>> <property>
>>>    <name>oozie.service.WorkflowAppService.system.libpath</name>
>>>    <value>/user/${user.name}/share/lib/lib_20140814100844</value>
>>>    <description>
>>>        System library path to use for workflow applications.
>>>        This path is added to workflow application if their job
>>>properties
>>> sets
>>>        the property 'oozie.use.system.libpath' to true.
>>>    </description>
>>> </property>
>>> 
>>> 
>>> I set the  oozie.use.system.libpath=false in job.properties, but it
>>>seems
>>> does not work(still throw the exception).
>>> 
>>> My hadoop is v2.2.0 and oozie is4.2.0-SNAPSHOT.
>>> 
>>> Anyone help?
>>> 
>>> Best regards!
>>> 
>


Re: share lib does not exist

Posted by Filix Suo <sy...@gmail.com>.
Hi Robert,

Thanks for your answer.
I tried as you said, but it still throw the error.


在 2014年8月15日,上午1:36,Robert Kanter <rk...@cloudera.com> 写道:

> Hi Filix,
> 
> You should not include the lib_<timestamp> part of the path in the
> oozie-site property.  Oozie knows to look for the latest lib_<timestamp>
> dir under that location.
> 
> e.g
> <property>
>    <name>oozie.service.WorkflowAppService.system.libpath</name>
>    <value>/user/${user.name}/share/lib</value>
>    <description>
>        System library path to use for workflow applications.
>        This path is added to workflow application if their job properties
> sets
>        the property 'oozie.use.system.libpath' to true.
>    </description>
> </property>
> 
> 
> 
> 
> 
> On Wed, Aug 13, 2014 at 7:49 PM, Filix Suo <sy...@gmail.com> wrote:
> 
>> hi all,
>> 
>> I build and install oozie, then I try to run the examples in oozie.
>> It throw a org.apache.oozie.action.ActionExecutorException: File
>> /user/filix/share/lib does not exist.
>> 
>> 2014-08-14 10:44:23,891  INFO ActionStartXCommand:541 -
>> SERVER[Filix-MacBook-Pro.local] USER[filix] GROUP[-] TOKEN[] APP[shell-wf]
>> JOB[0000000-140814104417138-oozie-fili-W]
>> ACTION[0000000-140814104417138-oozie-fili-W@:start:] Start action
>> [0000000-140814104417138-oozie-fili-W@:start:] with user-retry state :
>> userRetryCount [0], userRetryMax [0], userRetryInterval [10]
>> 2014-08-14 10:44:23,892  INFO ActionStartXCommand:541 -
>> SERVER[Filix-MacBook-Pro.local] USER[filix] GROUP[-] TOKEN[] APP[shell-wf]
>> JOB[0000000-140814104417138-oozie-fili-W]
>> ACTION[0000000-140814104417138-oozie-fili-W@:start:]
>> [***0000000-140814104417138-oozie-fili-W@:start:***]Action status=DONE
>> 2014-08-14 10:44:23,892  INFO ActionStartXCommand:541 -
>> SERVER[Filix-MacBook-Pro.local] USER[filix] GROUP[-] TOKEN[] APP[shell-wf]
>> JOB[0000000-140814104417138-oozie-fili-W]
>> ACTION[0000000-140814104417138-oozie-fili-W@:start:]
>> [***0000000-140814104417138-oozie-fili-W@:start:***]Action updated in DB!
>> 2014-08-14 10:44:24,136  INFO ActionStartXCommand:541 -
>> SERVER[Filix-MacBook-Pro.local] USER[filix] GROUP[-] TOKEN[] APP[shell-wf]
>> JOB[0000000-140814104417138-oozie-fili-W]
>> ACTION[0000000-140814104417138-oozie-fili-W@shell-node] Start action
>> [0000000-140814104417138-oozie-fili-W@shell-node] with user-retry state :
>> userRetryCount [0], userRetryMax [0], userRetryInterval [10]
>> 2014-08-14 10:44:24,220  WARN ActionStartXCommand:544 -
>> SERVER[Filix-MacBook-Pro.local] USER[filix] GROUP[-] TOKEN[] APP[shell-wf]
>> JOB[0000000-140814104417138-oozie-fili-W]
>> ACTION[0000000-140814104417138-oozie-fili-W@shell-node] Error starting
>> action [shell-node]. ErrorType [FAILED], ErrorCode [It should never
>> happen], Message [File /user/filix/share/lib/lib_20140814100844 does not
>> exist]
>> org.apache.oozie.action.ActionExecutorException: File
>> /user/filix/share/lib/lib_20140814100844 does not exist
>>        at
>> org.apache.oozie.action.hadoop.JavaActionExecutor.addSystemShareLibForAction(JavaActionExecutor.java:620)
>>        at
>> org.apache.oozie.action.hadoop.JavaActionExecutor.addAllShareLibs(JavaActionExecutor.java:698)
>>        at
>> org.apache.oozie.action.hadoop.JavaActionExecutor.setLibFilesArchives(JavaActionExecutor.java:689)
>>        at
>> org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:884)
>>        at
>> org.apache.oozie.action.hadoop.JavaActionExecutor.start(JavaActionExecutor.java:1134)
>>        at
>> org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:228)
>>        at
>> org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:63)
>>        at org.apache.oozie.command.XCommand.call(XCommand.java:281)
>>        at
>> org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:323)
>>        at
>> org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:252)
>>        at
>> org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:174)
>>        at
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>>        at
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>>        at java.lang.Thread.run(Thread.java:744)
>> 2014-08-14 10:44:24,222  WARN ActionStartXCommand:544 -
>> SERVER[Filix-MacBook-Pro.local] USER[filix] GROUP[-] TOKEN[] APP[shell-wf]
>> JOB[0000000-140814104417138-oozie-fili-W]
>> ACTION[0000000-140814104417138-oozie-fili-W@shell-node] Failing Job due
>> to failed action [shell-node]
>> 2014-08-14 10:44:24,224  WARN LiteWorkflowInstance:544 -
>> SERVER[Filix-MacBook-Pro.local] USER[filix] GROUP[-] TOKEN[] APP[shell-wf]
>> JOB[0000000-140814104417138-oozie-fili-W]
>> ACTION[0000000-140814104417138-oozie-fili-W@shell-node] Workflow Failed.
>> Failing node [shell-node]
>> 2014-08-14 10:44:24,370  INFO KillXCommand:541 -
>> SERVER[Filix-MacBook-Pro.local] USER[filix] GROUP[-] TOKEN[] APP[shell-wf]
>> JOB[0000000-140814104417138-oozie-fili-W] ACTION[-] STARTED
>> WorkflowKillXCommand for jobId=0000000-140814104417138-oozie-fili-W
>> 2014-08-14 10:44:24,394  INFO KillXCommand:541 -
>> SERVER[Filix-MacBook-Pro.local] USER[filix] GROUP[-] TOKEN[] APP[shell-wf]
>> JOB[0000000-140814104417138-oozie-fili-W] ACTION[-] ENDED
>> WorkflowKillXCommand for jobId=0000000-140814104417138-oozie-fili-W
>> 
>> 
>> However I have uploaded the share lib to hdfs:
>> 
>> $ hadoop fs -ls /user/filix/share/lib/lib_20140814100844
>> Found 8 items
>> drwxr-xr-x   - filix supergroup          0 2014-08-14 10:08
>> /user/filix/share/lib/lib_20140814100844/distcp
>> drwxr-xr-x   - filix supergroup          0 2014-08-14 10:08
>> /user/filix/share/lib/lib_20140814100844/hcatalog
>> drwxr-xr-x   - filix supergroup          0 2014-08-14 10:08
>> /user/filix/share/lib/lib_20140814100844/hive
>> drwxr-xr-x   - filix supergroup          0 2014-08-14 10:08
>> /user/filix/share/lib/lib_20140814100844/mapreduce-streaming
>> drwxr-xr-x   - filix supergroup          0 2014-08-14 10:08
>> /user/filix/share/lib/lib_20140814100844/oozie
>> drwxr-xr-x   - filix supergroup          0 2014-08-14 10:08
>> /user/filix/share/lib/lib_20140814100844/pig
>> -rw-r--r--   3 filix supergroup       1384 2014-08-14 10:08
>> /user/filix/share/lib/lib_20140814100844/sharelib.properties
>> drwxr-xr-x   - filix supergroup          0 2014-08-14 10:08
>> /user/filix/share/lib/lib_20140814100844/sqoop
>> 
>> here is my conf/oozie-site.xml:
>> <property>
>>    <name>oozie.service.WorkflowAppService.system.libpath</name>
>>    <value>/user/${user.name}/share/lib/lib_20140814100844</value>
>>    <description>
>>        System library path to use for workflow applications.
>>        This path is added to workflow application if their job properties
>> sets
>>        the property 'oozie.use.system.libpath' to true.
>>    </description>
>> </property>
>> 
>> 
>> I set the  oozie.use.system.libpath=false in job.properties, but it seems
>> does not work(still throw the exception).
>> 
>> My hadoop is v2.2.0 and oozie is4.2.0-SNAPSHOT.
>> 
>> Anyone help?
>> 
>> Best regards!
>> 


Re: share lib does not exist

Posted by Robert Kanter <rk...@cloudera.com>.
Hi Filix,

You should not include the lib_<timestamp> part of the path in the
oozie-site property.  Oozie knows to look for the latest lib_<timestamp>
dir under that location.

e.g
<property>
    <name>oozie.service.WorkflowAppService.system.libpath</name>
    <value>/user/${user.name}/share/lib</value>
    <description>
        System library path to use for workflow applications.
        This path is added to workflow application if their job properties
sets
        the property 'oozie.use.system.libpath' to true.
    </description>
</property>





On Wed, Aug 13, 2014 at 7:49 PM, Filix Suo <sy...@gmail.com> wrote:

> hi all,
>
> I build and install oozie, then I try to run the examples in oozie.
> It throw a org.apache.oozie.action.ActionExecutorException: File
> /user/filix/share/lib does not exist.
>
> 2014-08-14 10:44:23,891  INFO ActionStartXCommand:541 -
> SERVER[Filix-MacBook-Pro.local] USER[filix] GROUP[-] TOKEN[] APP[shell-wf]
> JOB[0000000-140814104417138-oozie-fili-W]
> ACTION[0000000-140814104417138-oozie-fili-W@:start:] Start action
> [0000000-140814104417138-oozie-fili-W@:start:] with user-retry state :
> userRetryCount [0], userRetryMax [0], userRetryInterval [10]
> 2014-08-14 10:44:23,892  INFO ActionStartXCommand:541 -
> SERVER[Filix-MacBook-Pro.local] USER[filix] GROUP[-] TOKEN[] APP[shell-wf]
> JOB[0000000-140814104417138-oozie-fili-W]
> ACTION[0000000-140814104417138-oozie-fili-W@:start:]
> [***0000000-140814104417138-oozie-fili-W@:start:***]Action status=DONE
> 2014-08-14 10:44:23,892  INFO ActionStartXCommand:541 -
> SERVER[Filix-MacBook-Pro.local] USER[filix] GROUP[-] TOKEN[] APP[shell-wf]
> JOB[0000000-140814104417138-oozie-fili-W]
> ACTION[0000000-140814104417138-oozie-fili-W@:start:]
> [***0000000-140814104417138-oozie-fili-W@:start:***]Action updated in DB!
> 2014-08-14 10:44:24,136  INFO ActionStartXCommand:541 -
> SERVER[Filix-MacBook-Pro.local] USER[filix] GROUP[-] TOKEN[] APP[shell-wf]
> JOB[0000000-140814104417138-oozie-fili-W]
> ACTION[0000000-140814104417138-oozie-fili-W@shell-node] Start action
> [0000000-140814104417138-oozie-fili-W@shell-node] with user-retry state :
> userRetryCount [0], userRetryMax [0], userRetryInterval [10]
> 2014-08-14 10:44:24,220  WARN ActionStartXCommand:544 -
> SERVER[Filix-MacBook-Pro.local] USER[filix] GROUP[-] TOKEN[] APP[shell-wf]
> JOB[0000000-140814104417138-oozie-fili-W]
> ACTION[0000000-140814104417138-oozie-fili-W@shell-node] Error starting
> action [shell-node]. ErrorType [FAILED], ErrorCode [It should never
> happen], Message [File /user/filix/share/lib/lib_20140814100844 does not
> exist]
> org.apache.oozie.action.ActionExecutorException: File
> /user/filix/share/lib/lib_20140814100844 does not exist
>         at
> org.apache.oozie.action.hadoop.JavaActionExecutor.addSystemShareLibForAction(JavaActionExecutor.java:620)
>         at
> org.apache.oozie.action.hadoop.JavaActionExecutor.addAllShareLibs(JavaActionExecutor.java:698)
>         at
> org.apache.oozie.action.hadoop.JavaActionExecutor.setLibFilesArchives(JavaActionExecutor.java:689)
>         at
> org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:884)
>         at
> org.apache.oozie.action.hadoop.JavaActionExecutor.start(JavaActionExecutor.java:1134)
>         at
> org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:228)
>         at
> org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:63)
>         at org.apache.oozie.command.XCommand.call(XCommand.java:281)
>         at
> org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:323)
>         at
> org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:252)
>         at
> org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:174)
>         at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>         at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>         at java.lang.Thread.run(Thread.java:744)
> 2014-08-14 10:44:24,222  WARN ActionStartXCommand:544 -
> SERVER[Filix-MacBook-Pro.local] USER[filix] GROUP[-] TOKEN[] APP[shell-wf]
> JOB[0000000-140814104417138-oozie-fili-W]
> ACTION[0000000-140814104417138-oozie-fili-W@shell-node] Failing Job due
> to failed action [shell-node]
> 2014-08-14 10:44:24,224  WARN LiteWorkflowInstance:544 -
> SERVER[Filix-MacBook-Pro.local] USER[filix] GROUP[-] TOKEN[] APP[shell-wf]
> JOB[0000000-140814104417138-oozie-fili-W]
> ACTION[0000000-140814104417138-oozie-fili-W@shell-node] Workflow Failed.
> Failing node [shell-node]
> 2014-08-14 10:44:24,370  INFO KillXCommand:541 -
> SERVER[Filix-MacBook-Pro.local] USER[filix] GROUP[-] TOKEN[] APP[shell-wf]
> JOB[0000000-140814104417138-oozie-fili-W] ACTION[-] STARTED
> WorkflowKillXCommand for jobId=0000000-140814104417138-oozie-fili-W
> 2014-08-14 10:44:24,394  INFO KillXCommand:541 -
> SERVER[Filix-MacBook-Pro.local] USER[filix] GROUP[-] TOKEN[] APP[shell-wf]
> JOB[0000000-140814104417138-oozie-fili-W] ACTION[-] ENDED
> WorkflowKillXCommand for jobId=0000000-140814104417138-oozie-fili-W
>
>
> However I have uploaded the share lib to hdfs:
>
> $ hadoop fs -ls /user/filix/share/lib/lib_20140814100844
>  Found 8 items
> drwxr-xr-x   - filix supergroup          0 2014-08-14 10:08
> /user/filix/share/lib/lib_20140814100844/distcp
> drwxr-xr-x   - filix supergroup          0 2014-08-14 10:08
> /user/filix/share/lib/lib_20140814100844/hcatalog
> drwxr-xr-x   - filix supergroup          0 2014-08-14 10:08
> /user/filix/share/lib/lib_20140814100844/hive
> drwxr-xr-x   - filix supergroup          0 2014-08-14 10:08
> /user/filix/share/lib/lib_20140814100844/mapreduce-streaming
> drwxr-xr-x   - filix supergroup          0 2014-08-14 10:08
> /user/filix/share/lib/lib_20140814100844/oozie
> drwxr-xr-x   - filix supergroup          0 2014-08-14 10:08
> /user/filix/share/lib/lib_20140814100844/pig
> -rw-r--r--   3 filix supergroup       1384 2014-08-14 10:08
> /user/filix/share/lib/lib_20140814100844/sharelib.properties
> drwxr-xr-x   - filix supergroup          0 2014-08-14 10:08
> /user/filix/share/lib/lib_20140814100844/sqoop
>
> here is my conf/oozie-site.xml:
> <property>
>     <name>oozie.service.WorkflowAppService.system.libpath</name>
>     <value>/user/${user.name}/share/lib/lib_20140814100844</value>
>     <description>
>         System library path to use for workflow applications.
>         This path is added to workflow application if their job properties
> sets
>         the property 'oozie.use.system.libpath' to true.
>     </description>
> </property>
>
>
> I set the  oozie.use.system.libpath=false in job.properties, but it seems
> does not work(still throw the exception).
>
> My hadoop is v2.2.0 and oozie is4.2.0-SNAPSHOT.
>
> Anyone help?
>
> Best regards!
>