You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@oozie.apache.org by Gäde, Sebastian <s1...@hft-leipzig.de> on 2014/06/20 14:12:57 UTC

shell action failing

Hi all,

I'm trying to run a Perl script using the shell action. But it is always failing to
java.io.IOException: Cannot run program "result_import.pl" (in directory 
/home/hduser/hadoop/mapred/nodemanager/usercache/hduser/appcache/application_1403241782872_0041/container_1403241782872_0041_01_000001"): error=2, Datei oder Verzeichnis nicht gefunden [=File or Directory not found]

This is the shell part of my workflow.xml:
        <shell xmlns="uri:oozie:shell-action:0.1">
                <job-tracker>${jobTracker}</job-tracker>
                <name-node>${nameNode}</name-node>
                <configuration>
                        <property>
                        <name>mapreduce.job.queue.name</name>
                        <value>${queueName}</value>
                        </property>
                </configuration>
                <exec>${script}</exec>
                <file>${scriptPath}#${script}</file>
        </shell>

and my job.properties:
oozieProjectRoot=${nameNode}/apps/cdrreader
oozie.wf.application.path=${oozieProjectRoot}
script=result_import.pl
scriptPath=${oozieProjectRoot}/result_import.pl

I am, however, able to exec 'ls' with argument '-la' leaving all other options as shown above:
Stdoutput lrwxrwxrwx 1 hduser hadoop    68 Jun 20 13:04 result_import.pl -> /home/hduser/hadoop/mapred/nodemanager/filecache/36/result_import.pl

This confuses me as there seems to be a link named result_import.pl, so why can't I exec it?

Thanks!
Seb.

AW: shell action failing

Posted by Gäde, Sebastian <s1...@hft-leipzig.de>.

Hi Mohammad,

Thanks for your reply.

Neither ./result_import.pl nor echo $PATH worked (same error as before). <exec>echo</exec> with <argument>$PATH</argument> gives '$PATH' as output...

However, I found that Oozie logs a 'List of passing environment' where PATH was set to
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:

I tried <env-var>PATH=$PATH:.</env-var> without success.

I finally used <exec>perl</exec> with <argument>result_import.pl</argument> which seems to be working fine. :-)

Cheers
Seb.
________________________________________
Von: Mohammad Islam [mislam77@yahoo.com.INVALID]
Gesendet: Samstag, 21. Juni 2014 03:55
An: user@oozie.apache.org
Betreff: Re: shell action failing

Hi Seb,
Not sure what is the cause.
Can you test by replacing this
<exec>${script}</exec> with  <exec>./result_import.pl</exec>

It could be possible "." is not in PATH. You can also debug by running "echo $PATH".

Regards,
Mohammad



On Friday, June 20, 2014 5:13 AM, "Gäde, Sebastian" <s1...@hft-leipzig.de> wrote:



Hi all,

I'm trying to run a Perl script using the shell action. But it is always failing to
java.io.IOException: Cannot run program "result_import.pl" (in directory
/home/hduser/hadoop/mapred/nodemanager/usercache/hduser/appcache/application_1403241782872_0041/container_1403241782872_0041_01_000001"): error=2, Datei oder Verzeichnis nicht gefunden [=File or Directory not found]

This is the shell part of my workflow.xml:
        <shell xmlns="uri:oozie:shell-action:0.1">
                <job-tracker>${jobTracker}</job-tracker>
                <name-node>${nameNode}</name-node>
                <configuration>
                        <property>
                        <name>mapreduce.job.queue.name</name>
                        <value>${queueName}</value>
                        </property>
                </configuration>
                <exec>${script}</exec>
                <file>${scriptPath}#${script}</file>
        </shell>

and my job.properties:
oozieProjectRoot=${nameNode}/apps/cdrreader
oozie.wf.application.path=${oozieProjectRoot}
script=result_import.pl
scriptPath=${oozieProjectRoot}/result_import.pl

I am, however, able to exec 'ls' with argument '-la' leaving all other options as shown above:
Stdoutput lrwxrwxrwx 1 hduser hadoop    68 Jun 20 13:04 result_import.pl -> /home/hduser/hadoop/mapred/nodemanager/filecache/36/result_import.pl

This confuses me as there seems to be a link named result_import.pl, so why can't I exec it?

Thanks!
Seb.

AW: shell action failing

Posted by Gäde, Sebastian <s1...@hft-leipzig.de>.

Hi Mohammad,

Thanks for your reply.

Neither ./result_import.pl nor echo $PATH worked (same error as before). <exec>echo</exec> with <argument>$PATH</argument> gives '$PATH' as output...

However, I found that Oozie logs a 'List of passing environment' where PATH was set to
________________________________________
Von: Mohammad Islam [mislam77@yahoo.com.INVALID]
Gesendet: Samstag, 21. Juni 2014 03:55
An: user@oozie.apache.org
Betreff: Re: shell action failing

Hi Seb,
Not sure what is the cause.
Can you test by replacing this
<exec>${script}</exec> with  <exec>./result_import.pl</exec>

It could be possible "." is not in PATH. You can also debug by running "echo $PATH".

Regards,
Mohammad



On Friday, June 20, 2014 5:13 AM, "Gäde, Sebastian" <s1...@hft-leipzig.de> wrote:



Hi all,

I'm trying to run a Perl script using the shell action. But it is always failing to
java.io.IOException: Cannot run program "result_import.pl" (in directory
/home/hduser/hadoop/mapred/nodemanager/usercache/hduser/appcache/application_1403241782872_0041/container_1403241782872_0041_01_000001"): error=2, Datei oder Verzeichnis nicht gefunden [=File or Directory not found]

This is the shell part of my workflow.xml:
        <shell xmlns="uri:oozie:shell-action:0.1">
                <job-tracker>${jobTracker}</job-tracker>
                <name-node>${nameNode}</name-node>
                <configuration>
                        <property>
                        <name>mapreduce.job.queue.name</name>
                        <value>${queueName}</value>
                        </property>
                </configuration>
                <exec>${script}</exec>
                <file>${scriptPath}#${script}</file>
        </shell>

and my job.properties:
oozieProjectRoot=${nameNode}/apps/cdrreader
oozie.wf.application.path=${oozieProjectRoot}
script=result_import.pl
scriptPath=${oozieProjectRoot}/result_import.pl

I am, however, able to exec 'ls' with argument '-la' leaving all other options as shown above:
Stdoutput lrwxrwxrwx 1 hduser hadoop    68 Jun 20 13:04 result_import.pl -> /home/hduser/hadoop/mapred/nodemanager/filecache/36/result_import.pl

This confuses me as there seems to be a link named result_import.pl, so why can't I exec it?

Thanks!
Seb.

Re: shell action failing

Posted by Mohammad Islam <mi...@yahoo.com.INVALID>.

Hi Seb,
Not sure what is the cause.
Can you test by replacing this 
<exec>${script}</exec> with  <exec>./result_import.pl</exec>

It could be possible "." is not in PATH. You can also debug by running "echo $PATH".

Regards,
Mohammad



On Friday, June 20, 2014 5:13 AM, "Gäde, Sebastian" <s1...@hft-leipzig.de> wrote:
 


Hi all,

I'm trying to run a Perl script using the shell action. But it is always failing to
java.io.IOException: Cannot run program "result_import.pl" (in directory 
/home/hduser/hadoop/mapred/nodemanager/usercache/hduser/appcache/application_1403241782872_0041/container_1403241782872_0041_01_000001"): error=2, Datei oder Verzeichnis nicht gefunden [=File or Directory not found]

This is the shell part of my workflow.xml:
        <shell xmlns="uri:oozie:shell-action:0.1">
                <job-tracker>${jobTracker}</job-tracker>
                <name-node>${nameNode}</name-node>
                <configuration>
                        <property>
                        <name>mapreduce.job.queue.name</name>
                        <value>${queueName}</value>
                        </property>
                </configuration>
                <exec>${script}</exec>
                <file>${scriptPath}#${script}</file>
        </shell>

and my job.properties:
oozieProjectRoot=${nameNode}/apps/cdrreader
oozie.wf.application.path=${oozieProjectRoot}
script=result_import.pl
scriptPath=${oozieProjectRoot}/result_import.pl

I am, however, able to exec 'ls' with argument '-la' leaving all other options as shown above:
Stdoutput lrwxrwxrwx 1 hduser hadoop    68 Jun 20 13:04 result_import.pl -> /home/hduser/hadoop/mapred/nodemanager/filecache/36/result_import.pl

This confuses me as there seems to be a link named result_import.pl, so why can't I exec it?

Thanks!
Seb.