Re: Could be a config issue: Execute an unix command thru oozie java action.

[dejace sr <de...@gmail.com>]

Thanks Mohammad.

Sorry to miss replying you as I logged-in to this account after a while as
(re) started working on oozie.
Pls find the answers below:


On Thu, Oct 6, 2011 at 1:13 AM, Mohammad Islam <mi...@yahoo.com> wrote:

> Hi DJ,
> Would you please try with this in your java action definition?
> In general, the default is 512MB for the launcher mapper.
>
> <property> <name>oozie.launcher.mapred.child.java.opts</name>
> <value>-server -Xmx1G -Djava.net.preferIPv4Stack=true</value>
> <description>setting memory usage to 1024MB</description>
> </property>
> Is it that any hadoop command (e.g. hadoop version, hadoop fs -ls) is
> failing?
>
> Btw, are you running in secure hadoop?
> If you don't have any confidential logic in your CommandExecutor.java,
> would you mind to include that here.
>
> DJ --> I don't have any confidential logic in it. Below is an article
which explains about the apache commons Exec library with an example.
http://blog.sanaulla.info/2010/09/07/execute-external-process-from-within-jvm-using-apache-commons-exec-library/

All I am trying to do in the CommandExecutor is to run any kind of unix
command that could be possibly run on the unix command prompt.
My use case is to run some external perl scripts or shell scripts that is
needed for my workflow/pipeline.

Later Mayank helped similar to the one also discussed here:
http://tech.groups.yahoo.com/group/Oozie-users/message/582

Thanks both of you.

Its quite likely that I might be coming back with some more implementation
issues. So, here is a background of what I am trying to achieve:
I am trying to adopt oozie for managing workflows of 'indexing pipeline for
search engine'. The indexing pipeline is written in java hadoop/hbase.


Regards,
> Mohammad
>
>
>
>
> ________________________________
> From: dejace sr <de...@gmail.com>
> To: oozie-users@incubator.apache.org
> Sent: Wednesday, October 5, 2011 4:28 PM
> Subject: Could be a config issue: Execute an unix command thru oozie java
> action.
>
> Below could be a configuration issue in the compute or hadoop cluster.
> Not sure what should be changed to avoid this problem.
>
> Pls let me know your thoughts.
>
> Thanks,
> DJ
>
> What I am trying to do?
> Execute an unix command thru oozie java action.
>
> Below are the errors I get:
>
> 1. >>> Invoking Main class now >>>
>
> Command being executed = hadoop fs -lsr /user/dr/pipeline
> Error occurred during initialization of VM
> Could not reserve enough space for object heap
> 1
> Execution failed
>
>
> 2. >>> Invoking Main class now >>>
>
> Command being executed = ls -ltr
> total 20
> drwxr-xr-x 2 mapred mapred  4096 Oct  5 15:09 tmp
> -rwxrwxrwx 1 mapred mapred 15248 Oct  5 15:09 action.xml
> 0
> Execution Successful
>
>
> oozie workflow.xml
>
> <action name='test3'>
> <java>
> <job-tracker>${jobTracker}</job-tracker>
> <name-node>${nameNode}</name-node>
> <configuration>
> <property>
> <name>mapred.job.queue.name</name>
> <value>default</value>
> </property>
> </configuration>
>
> <main-class>CommandExecutor</main-class>
> <arg>hadoop</arg>
> <arg>fs</arg>
> <arg>-lsr</arg>
> <arg>/user/dr/pipeline</arg>
> </java>
>
> <ok to='end' />
> <error to='fail' />
> </action>
>
>
> similar for "ls -ltr"
>
> NOTE:
> The java program CommandExecutor does not have any issue when ran on unix
> command prompt
> I am using apache's org.apache.commons.exec.Executor to execute external
> programs/commands.
>