You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@oozie.apache.org by Tim Chan <ti...@chan.net> on 2012/07/09 11:51:16 UTC
problems with shell action, can not find executable
I'm providing:
<env-var>PATH=/usr/local/nz/bin</env-var>
which is where my executable is located.
I get an error saying:
Cannot run program "nzsql" (in directory
"/hdata02/hadoop/mapred/local/taskTracker/oozie/jobcache/job_201206011141_7196/attempt_201206011141_7196_m_000000_0/work"):
java.io.IOException: error=2, No such file or directory
The nzsql executable is located in that path I provided.
Re: problems with shell action, can not find executable
Posted by Mohammad Islam <mi...@yahoo.com>.
Hi Tim,
In your case, did you try by giving the absolute path as exec ?
However, there should be a way of passing the environment. The env-var is passed to the environment of your application. It is "not" to the
environment from where the application is launched.
To pass that, we can set the following properties of oozie launcher.
If I remember correctly I set the following properties to set the cwd into PATH and LD_LIBRARY_PATH through wf.xml.
<property>
<name>oozie.launcher.mapred.child.env</name>=
<value>PATH=.:$PATH,LD_LIBRARY_PATH=$LD_LIBRARY_PATH:.</value></property>
Since you already have a setup, could you pls try this out?
Regards,
Mohammad
________________________________
From: Tim Chan <ti...@chan.net>
To: oozie-users@incubator.apache.org; Mohammad Islam <mi...@yahoo.com>
Sent: Wednesday, July 11, 2012 12:20 AM
Subject: Re: problems with shell action, can not find executable
Hi Mohammad,
Here is my workflow.xml snippet of the action:
<action name="truncate-table">
<shell xmlns="uri:oozie:shell-action:0.1">
<job-tracker>${jobTracker}</job-tracker>
<name-node>${nameNode}</name-node>
<configuration>
<property>
<name>mapred.job.queue.name</name>
<value>${queueName}</value>
</property>
</configuration>
<exec>nzsql</exec>
...
<env-var>PATH=/usr/local/nz/bin</env-var>
<capture-output/>
</shell>
<ok to="process-cross-ref-tsv-and-hdfs-put"/>
<error to="fail"/>
</action>
I've checked multiple times and the executable indeed can be found on
the data node with in the path /usr/local/nz/bin
Thanks,
Tim
On Wed, Jul 11, 2012 at 12:02 AM, Mohammad Islam <mi...@yahoo.com> wrote:
> Hi Tim,
> According to the documents, it should be supported in a standard way as you mentioned.=
>
> I think you already followed the instructions provided at
> http://incubator.apache.org/oozie/docs/3.2.0-=0Aincubating/docs/DG_ShellActionExtension.html#Shell_Action
> "The env-var element, if present, contains the environemnt to be passed to the Shell command. env-var
> should contain only one pair of environment variable and value. If the pair contains the variable such as $PATH, it should follow the Unix convention such as PATH=3D$PATH:mypath"
>
> How did you invoke the executable?
> Can you pls include your related wf.xml section?
>
>
> Regards,
> Mohammad
>
>
> ________________________________
> From: Tim Chan <ti...@chan.net>
> To: oozie-users@incubator.apache.org
> Sent: Monday, July 9, 2012 11:44 AM
> Subject: Re: problems with shell action, can not find executable
>
> Thanks Mona. I was able to get it to work this way, though it is less
> than ideal.
>
> On Mon, Jul 9, 2012 at 11:08 AM, Mona Chitnis <ch...@yahoo-inc.com> wrote:
>> Hello Tim,
>>
>> The executable will always be looked for in the application's localized
>> current working directory(in this case -
>> /hdata02/hadoop/mapred/local/taskTracker/oozie/jobcache/job_201206011141_71
>> 96/attempt_201206011141_7196_m_000000_0/work)
>>
>> Hence you should put your executable in your workflow directory. Then use
>> the <file> tag to refer to it with a symbolic name if required, as shown
>> here.
>>
>> <file>${EXEC}#${EXEC}</file> <!--Copies the executable to compute node's
>> current working directory -->
>>
>>
>> EXEC refers to your executable's path in hdfs.
>> --
>> Mona Chitnis
>>
>>
>>
>>
>> On 7/9/12 2:51 AM, "Tim Chan" <ti...@chan.net> wrote:
>>
>>>I'm providing:
>>>
>>> <env-var>PATH=/usr/local/nz/bin</env-var>
>>>
>>>which is where my executable is located.
>>>
>>>I get an error saying:
>>>
>>>Cannot run program "nzsql" (in directory
>>>"/hdata02/hadoop/mapred/local/taskTracker/oozie/jobcache/job_201206011141_
>>>7196/attempt_201206011141_7196_m_000000_0/work"):
>>>java.io.IOException: error=2, No such file or directory
>>>
>>>The nzsql executable is located in that path I provided.
>>
>
>
>
> --
> Tim Chan // tim@chan.net // 213.784.2523
--
Tim Chan // tim@chan.net // 213.784.2523
Re: problems with shell action, can not find executable
Posted by Tim Chan <ti...@chan.net>.
Hi Mohammad,
Here is my workflow.xml snippet of the action:
<action name="truncate-table">
<shell xmlns="uri:oozie:shell-action:0.1">
<job-tracker>${jobTracker}</job-tracker>
<name-node>${nameNode}</name-node>
<configuration>
<property>
<name>mapred.job.queue.name</name>
<value>${queueName}</value>
</property>
</configuration>
<exec>nzsql</exec>
...
<env-var>PATH=/usr/local/nz/bin</env-var>
<capture-output/>
</shell>
<ok to="process-cross-ref-tsv-and-hdfs-put"/>
<error to="fail"/>
</action>
I've checked multiple times and the executable indeed can be found on
the data node with in the path /usr/local/nz/bin
Thanks,
Tim
On Wed, Jul 11, 2012 at 12:02 AM, Mohammad Islam <mi...@yahoo.com> wrote:
> Hi Tim,
> According to the documents, it should be supported in a standard way as you mentioned.=
>
> I think you already followed the instructions provided at
> http://incubator.apache.org/oozie/docs/3.2.0-=0Aincubating/docs/DG_ShellActionExtension.html#Shell_Action
> "The env-var element, if present, contains the environemnt to be passed to the Shell command. env-var
> should contain only one pair of environment variable and value. If the pair contains the variable such as $PATH, it should follow the Unix convention such as PATH=3D$PATH:mypath"
>
> How did you invoke the executable?
> Can you pls include your related wf.xml section?
>
>
> Regards,
> Mohammad
>
>
> ________________________________
> From: Tim Chan <ti...@chan.net>
> To: oozie-users@incubator.apache.org
> Sent: Monday, July 9, 2012 11:44 AM
> Subject: Re: problems with shell action, can not find executable
>
> Thanks Mona. I was able to get it to work this way, though it is less
> than ideal.
>
> On Mon, Jul 9, 2012 at 11:08 AM, Mona Chitnis <ch...@yahoo-inc.com> wrote:
>> Hello Tim,
>>
>> The executable will always be looked for in the application's localized
>> current working directory(in this case -
>> /hdata02/hadoop/mapred/local/taskTracker/oozie/jobcache/job_201206011141_71
>> 96/attempt_201206011141_7196_m_000000_0/work)
>>
>> Hence you should put your executable in your workflow directory. Then use
>> the <file> tag to refer to it with a symbolic name if required, as shown
>> here.
>>
>> <file>${EXEC}#${EXEC}</file> <!--Copies the executable to compute node's
>> current working directory -->
>>
>>
>> EXEC refers to your executable's path in hdfs.
>> --
>> Mona Chitnis
>>
>>
>>
>>
>> On 7/9/12 2:51 AM, "Tim Chan" <ti...@chan.net> wrote:
>>
>>>I'm providing:
>>>
>>> <env-var>PATH=/usr/local/nz/bin</env-var>
>>>
>>>which is where my executable is located.
>>>
>>>I get an error saying:
>>>
>>>Cannot run program "nzsql" (in directory
>>>"/hdata02/hadoop/mapred/local/taskTracker/oozie/jobcache/job_201206011141_
>>>7196/attempt_201206011141_7196_m_000000_0/work"):
>>>java.io.IOException: error=2, No such file or directory
>>>
>>>The nzsql executable is located in that path I provided.
>>
>
>
>
> --
> Tim Chan // tim@chan.net // 213.784.2523
--
Tim Chan // tim@chan.net // 213.784.2523
Re: problems with shell action, can not find executable
Posted by Mohammad Islam <mi...@yahoo.com>.
Hi Tim,
According to the documents, it should be supported in a standard way as you mentioned.=
I think you already followed the instructions provided at
http://incubator.apache.org/oozie/docs/3.2.0-=0Aincubating/docs/DG_ShellActionExtension.html#Shell_Action
"The env-var element, if present, contains the environemnt to be passed to the Shell command. env-var
should contain only one pair of environment variable and value. If the pair contains the variable such as $PATH, it should follow the Unix convention such as PATH=3D$PATH:mypath"
How did you invoke the executable?
Can you pls include your related wf.xml section?
Regards,
Mohammad
________________________________
From: Tim Chan <ti...@chan.net>
To: oozie-users@incubator.apache.org
Sent: Monday, July 9, 2012 11:44 AM
Subject: Re: problems with shell action, can not find executable
Thanks Mona. I was able to get it to work this way, though it is less
than ideal.
On Mon, Jul 9, 2012 at 11:08 AM, Mona Chitnis <ch...@yahoo-inc.com> wrote:
> Hello Tim,
>
> The executable will always be looked for in the application's localized
> current working directory(in this case -
> /hdata02/hadoop/mapred/local/taskTracker/oozie/jobcache/job_201206011141_71
> 96/attempt_201206011141_7196_m_000000_0/work)
>
> Hence you should put your executable in your workflow directory. Then use
> the <file> tag to refer to it with a symbolic name if required, as shown
> here.
>
> <file>${EXEC}#${EXEC}</file> <!--Copies the executable to compute node's
> current working directory -->
>
>
> EXEC refers to your executable's path in hdfs.
> --
> Mona Chitnis
>
>
>
>
> On 7/9/12 2:51 AM, "Tim Chan" <ti...@chan.net> wrote:
>
>>I'm providing:
>>
>> <env-var>PATH=/usr/local/nz/bin</env-var>
>>
>>which is where my executable is located.
>>
>>I get an error saying:
>>
>>Cannot run program "nzsql" (in directory
>>"/hdata02/hadoop/mapred/local/taskTracker/oozie/jobcache/job_201206011141_
>>7196/attempt_201206011141_7196_m_000000_0/work"):
>>java.io.IOException: error=2, No such file or directory
>>
>>The nzsql executable is located in that path I provided.
>
--
Tim Chan // tim@chan.net // 213.784.2523
Re: problems with shell action, can not find executable
Posted by Tim Chan <ti...@chan.net>.
Thanks Mona. I was able to get it to work this way, though it is less
than ideal.
On Mon, Jul 9, 2012 at 11:08 AM, Mona Chitnis <ch...@yahoo-inc.com> wrote:
> Hello Tim,
>
> The executable will always be looked for in the application's localized
> current working directory(in this case -
> /hdata02/hadoop/mapred/local/taskTracker/oozie/jobcache/job_201206011141_71
> 96/attempt_201206011141_7196_m_000000_0/work)
>
> Hence you should put your executable in your workflow directory. Then use
> the <file> tag to refer to it with a symbolic name if required, as shown
> here.
>
> <file>${EXEC}#${EXEC}</file> <!--Copies the executable to compute node's
> current working directory -->
>
>
> EXEC refers to your executable's path in hdfs.
> --
> Mona Chitnis
>
>
>
>
> On 7/9/12 2:51 AM, "Tim Chan" <ti...@chan.net> wrote:
>
>>I'm providing:
>>
>> <env-var>PATH=/usr/local/nz/bin</env-var>
>>
>>which is where my executable is located.
>>
>>I get an error saying:
>>
>>Cannot run program "nzsql" (in directory
>>"/hdata02/hadoop/mapred/local/taskTracker/oozie/jobcache/job_201206011141_
>>7196/attempt_201206011141_7196_m_000000_0/work"):
>>java.io.IOException: error=2, No such file or directory
>>
>>The nzsql executable is located in that path I provided.
>
--
Tim Chan // tim@chan.net // 213.784.2523
Re: problems with shell action, can not find executable
Posted by Mona Chitnis <ch...@yahoo-inc.com>.
Hello Tim,
The executable will always be looked for in the application's localized
current working directory(in this case -
/hdata02/hadoop/mapred/local/taskTracker/oozie/jobcache/job_201206011141_71
96/attempt_201206011141_7196_m_000000_0/work)
Hence you should put your executable in your workflow directory. Then use
the <file> tag to refer to it with a symbolic name if required, as shown
here.
<file>${EXEC}#${EXEC}</file> <!--Copies the executable to compute node's
current working directory -->
EXEC refers to your executable's path in hdfs.
--
Mona Chitnis
On 7/9/12 2:51 AM, "Tim Chan" <ti...@chan.net> wrote:
>I'm providing:
>
> <env-var>PATH=/usr/local/nz/bin</env-var>
>
>which is where my executable is located.
>
>I get an error saying:
>
>Cannot run program "nzsql" (in directory
>"/hdata02/hadoop/mapred/local/taskTracker/oozie/jobcache/job_201206011141_
>7196/attempt_201206011141_7196_m_000000_0/work"):
>java.io.IOException: error=2, No such file or directory
>
>The nzsql executable is located in that path I provided.