running jar file in shell script

[Savitha Devi Inbasekaran <Sa...@smartek21.com>]

I am trying to run a shell script with jar file but I am getting an exception "Can't open /home/hduser/wdetl/weatherETL.sh
Failing Oozie Launcher, Main class [org.apache.oozie.action.hadoop.ShellMain], exit code [1]". The shell script and oozie workflow are given below.


------------ weatherETL.sh ---------------
#!/bin/bash
cd /home/hduser/wdetl
java -jar dist/WeatherData.jar
cd ~
------------oozie workflow----------
<workflow-app xmlns="uri:oozie:workflow:0.2" name="weather-etl">

     <start to="shell-etlforweather"/>

     <action name="shell-etlforweather">
         <shell xmlns="uri:oozie:shell-action:0.1">
             <job-tracker>${jobTracker}</job-tracker>
             <name-node>${nameNode}</name-node>
             <configuration>
                 <property>
                     <name>mapred.job.queue.name</name>
                     <value>${queueName}</value>
                 </property>
             </configuration>
             <exec>sh</exec>
<argument>/home/hduser/wdetl/weatherETL.sh</argument>

         </shell>
                <ok to="end"/>
         <error to="fail"/>
     </action>
<kill name="fail">
         <message>Node failed, error
message[${wf:errorMessage(wf:lastErrorNode())}]</message>
</kill>
<end name="end"/>
</workflow-app>
Thanks,
Savitha

Re: running jar file in shell script

[Mohammad Islam <mi...@yahoo.com>]

At first, the file "/home/hduser/wdetl/weatherETL.sh" needs to be in HDFS. Then you can use <file> tag to make it available to run on compute node.
If you need more specific example, please let me know.





________________________________
 From: Savitha Devi Inbasekaran <Sa...@smartek21.com>
To: "user@oozie.apache.org" <us...@oozie.apache.org> 
Sent: Tuesday, April 23, 2013 3:19 AM
Subject: running jar file in shell script
 

I am trying to run a shell script with jar file but I am getting an exception "Can't open /home/hduser/wdetl/weatherETL.sh
Failing Oozie Launcher, Main class [org.apache.oozie.action.hadoop.ShellMain], exit code [1]". The shell script and oozie workflow are given below.


------------ weatherETL.sh ---------------
#!/bin/bash
cd /home/hduser/wdetl
java -jar dist/WeatherData.jar
cd ~
------------oozie workflow----------
<workflow-app xmlns="uri:oozie:workflow:0.2" name="weather-etl">

     <start to="shell-etlforweather"/>

     <action name="shell-etlforweather">
         <shell xmlns="uri:oozie:shell-action:0.1">
             <job-tracker>${jobTracker}</job-tracker>
             <name-node>${nameNode}</name-node>
             <configuration>
                 <property>
                     <name>mapred.job.queue.name</name>
                     <value>${queueName}</value>
                 </property>
             </configuration>
             <exec>sh</exec>
<argument>/home/hduser/wdetl/weatherETL.sh</argument>

         </shell>
                <ok to="end"/>
         <error to="fail"/>
     </action>
<kill name="fail">
         <message>Node failed, error
message[${wf:errorMessage(wf:lastErrorNode())}]</message>
</kill>
<end name="end"/>
</workflow-app>
Thanks,
Savitha

Re: running jar file in shell script

[Robert Kanter <rk...@cloudera.com>]

HI Savitha,

You should be able to find a more specific error message in the log for the
launcher job from the JT Web UI.  Can you look there and paste the error
here?

Though my guess is that the problem is with your shell script.  Long story
short, the shell action is executed on an arbitrary node in your cluster,
so unless /home/hduser/wdetl/weatherETL.sh, /home/hduser/wdetl,
and /home/hduser/wdetl/dist/WeatherData.jar exist on all nodes in your
cluster, its not going to work reliably.  You should instead put your shell
script and any necessary files (e.g. WeatherData.jar) in HDFS with your
workflow and include them with the <file> element so Oozie will take care
of localizing them for you.

Is there a reason your using a shell script to run a jar file instead of
simply using the Java action to execute the jar directly?  (though its
still executed on an arbitrary node too, so what I said in the previous
paragraph still applies).

- Robert





On Tue, Apr 23, 2013 at 3:19 AM, Savitha Devi Inbasekaran <
Savithadi@smartek21.com> wrote:

> I am trying to run a shell script with jar file but I am getting an
> exception "Can't open /home/hduser/wdetl/weatherETL.sh
> Failing Oozie Launcher, Main class
> [org.apache.oozie.action.hadoop.ShellMain], exit code [1]". The shell
> script and oozie workflow are given below.
>
>
> ------------ weatherETL.sh ---------------
> #!/bin/bash
> cd /home/hduser/wdetl
> java -jar dist/WeatherData.jar
> cd ~
> ------------oozie workflow----------
> <workflow-app xmlns="uri:oozie:workflow:0.2" name="weather-etl">
>
>      <start to="shell-etlforweather"/>
>
>      <action name="shell-etlforweather">
>          <shell xmlns="uri:oozie:shell-action:0.1">
>              <job-tracker>${jobTracker}</job-tracker>
>              <name-node>${nameNode}</name-node>
>              <configuration>
>                  <property>
>                      <name>mapred.job.queue.name</name>
>                      <value>${queueName}</value>
>                  </property>
>              </configuration>
>              <exec>sh</exec>
> <argument>/home/hduser/wdetl/weatherETL.sh</argument>
>
>          </shell>
>                 <ok to="end"/>
>          <error to="fail"/>
>      </action>
> <kill name="fail">
>          <message>Node failed, error
> message[${wf:errorMessage(wf:lastErrorNode())}]</message>
> </kill>
> <end name="end"/>
> </workflow-app>
> Thanks,
> Savitha
>
>