You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@oozie.apache.org by Asanka Sanjaya Herath <an...@gmail.com> on 2016/01/13 05:27:49 UTC
How to submit an Oozie job using another Oozie job
Hi,
I'm writing a simple Java Oozie job. That job is supposed to spawn new
Oozie jobs or directly submit new jobs to Hadoop. Is there a way to achieve
this using Oozie API?
--
Thanks,
Regards,
ASH
Re: How to submit an Oozie job using another Oozie job
Posted by Hanmay Udgiri <ha...@gmail.com>.
Cool
On 14-Jan-2016 13:06, "Asanka Sanjaya Herath" <an...@gmail.com> wrote:
> Hi,
>
> I think I figure out the problem. It is a permission issue. When I look the
> yarn application log, I see following error.
>
> E0507: Could not access to
> [hdfs://xx.xxx.xx.xx8020/user/hdfs/email-dispatcher-tenant-manager-1.0/],
> Permission denied: user=yarn, access=EXECUTE,
> inode="/user/hdfs/email-dispatcher-tenant-manager-1.0":hdfs:hdfs:drwx------
> at
>
> org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:319)
> at
>
> org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkTraverse(FSPermissionChecker.java:259)
> at
>
> org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:205)
> at
>
> org.apache.ranger.authorization.hadoop.RangerHdfsAuthorizer$RangerAccessControlEnforcer.checkPermission(RangerHdfsAuthorizer.java:300)
> at
>
> org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:190)
> at
>
> org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:1771)
> at
>
> org.apache.hadoop.hdfs.server.namenode.FSDirStatAndListingOp.getFileInfo(FSDirStatAndListingOp.java:108)
> at
>
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getFileInfo(FSNamesystem.java:3857)
> at
>
> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getFileInfo(NameNodeRpcServer.java:1012)
> at
>
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getFileInfo(ClientNamenodeProtocolServerSideTranslatorPB.java:843)
> at
>
> org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
> at
>
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:616)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:969) at
> org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2137) at
> org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2133) at
> java.security.AccessController.doPrivileged(Native Method) at
> javax.security.auth.Subject.doAs(Subject.java:415) at
>
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2131)
> at
> org.apache.oozie.client.OozieClient.handleError(OozieClient.java:612)
> at
> org.apache.oozie.client.OozieClient$JobSubmit.call(OozieClient.java:695)
> at
> org.apache.oozie.client.OozieClient$JobSubmit.call(OozieClient.java:665)
> at
>
> org.apache.oozie.client.OozieClient$ClientCallable.call(OozieClient.java:560)
> at org.apache.oozie.client.OozieClient.run(OozieClient.java:850)
> at
>
> com.sonasoft.sonacloud.email.dispatcher.manager.Utils.submitDispatcherJobForTenantToOozie(Utils.java:155)
> at
>
> com.sonasoft.sonacloud.email.dispatcher.manager.Utils.spawnEmailDispatcherJobsPerTenant(Utils.java:43)
> at
>
> com.sonasoft.sonacloud.email.dispatcher.manager.MainClass.main(MainClass.java:27)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
>
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at org.apache.oozie.action.hadoop.JavaMain.run(JavaMain.java:56)
> at
> org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:47)
> at org.apache.oozie.action.hadoop.JavaMain.main(JavaMain.java:35)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
>
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at
> org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:236)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:453)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
>
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
> End of LogType:stderr
>
>
>
>
> On Wed, Jan 13, 2016 at 7:05 PM, Asanka Sanjaya Herath <angal.sh@gmail.com
> >
> wrote:
>
> > No. Only the child job runs using java API. The manager job is submitted
> > to Oozie manually using the command line. The above provided workflow.xml
> > is for the child job.
> >
> > On Wed, Jan 13, 2016 at 7:00 PM, Hanmay Udgiri <
> hanmayya.udgiri@gmail.com>
> > wrote:
> >
> >> I guess these two jobs run using the java API. What about workflow.xml
> of
> >> child job?
> >> On 13-Jan-2016 6:26 pm, "Asanka Sanjaya Herath" <an...@gmail.com>
> >> wrote:
> >>
> >> > There are basically two jobs.
> >> >
> >> > 1. manager job
> >> > 2. child job
> >> >
> >> >
> >> > Manager job is running on Oozie and responsible for creating new
> >> instances
> >> > of child jobs in Oozie when needed.
> >> >
> >> > I can see the manager job run in Oozie without giving any error. When
> >> the
> >> > submitDispatcherJobForTenantToOozie(String tenantId) method invoked in
> >> > manager job, it should submit a new child job to Oozie. The method is
> >> > invoking properly. But I cannot see any child job in Oozie console.
> >> >
> >> > But When I run the manager job directly in hadoop(Without using
> Oozie),
> >> it
> >> > spawns new child jobs in Oozie properly.
> >> >
> >> >
> >> >
> >> > On Wed, Jan 13, 2016 at 6:20 PM, Hanmay Udgiri <
> >> hanmayya.udgiri@gmail.com>
> >> > wrote:
> >> >
> >> > > So are u able to see the job running in HUE?.U can also view job in
> >> the
> >> > > oozie web console ? What about the workflow.xml of tenantapp?Is
> >> > > The MainClass job able to call tenantapp workflow?
> >> >
> >> > On 13-Jan-2016 6:09 pm, "Asanka Sanjaya Herath" <an...@gmail.com>
> >> > wrote:
> >> > >
> >> > > > Java method to create a job in Oozie.
> >> > > >
> >> > > > private void submitDispatcherJobForTenantToOozie(String tenantId)
> >> > > > throws OozieClientException {
> >> > > >
> >> > > > // get a OozieClient for local Oozie
> >> > > > OozieClient wc = new
> >> > > >
> >> > >
> >> >
> >>
> OozieClient(PropertyValuesProvider.getInstance().getPropValue("oozieURL"));
> >> > > >
> >> > > > // create a workflow job configuration and set the workflow
> >> > > > application path
> >> > > > Properties conf = wc.createConfiguration();
> >> > > > conf.setProperty(OozieClient.APP_PATH,
> >> > > > PropertyValuesProvider.getInstance().
> >> > > > getPropValue("dispatcherTenantManagerAppPath"));
> >> > > >
> >> > > > // setting workflow parameters
> >> > > > conf.setProperty("jobTracker", "sandbox.hortonworks.com:8050
> ");
> >> > > > conf.setProperty("nameNode", "hdfs://
> >> sandbox.hortonworks.com:8020
> >> > ");
> >> > > > conf.setProperty("queueName", "default");
> >> > > > conf.setProperty("appRoot",
> >> "email-dispatcher-tenant-manager-1.0");
> >> > > >
> >> > > > //set tenantId and retention period
> >> > > > conf.setProperty("tenantId", tenantId);
> >> > > >
> >> > > > // submit and start the workflow job
> >> > > > String jobId = wc.run(conf);
> >> > > > }
> >> > > >
> >> > > >
> >> > > > Workfolw.xml of child job:
> >> > > >
> >> > > >
> >> > > > <workflow-app xmlns="uri:oozie:workflow:0.2" name="java-main-wf">
> >> > > > <start to="java-node"/>
> >> > > > <action name="java-node">
> >> > > > <java>
> >> > > > <job-tracker>${jobTracker}</job-tracker>
> >> > > > <name-node>${nameNode}</name-node>
> >> > > > <configuration>
> >> > > > <property>
> >> > > > <name>mapred.job.queue.name</name>
> >> > > > <value>${queueName}</value>
> >> > > > </property>
> >> > > > </configuration>
> >> > > > <main-class>com.MainClass</main-class>
> >> > > > </java>
> >> > > > <ok to="end"/>
> >> > > > <error to="fail"/>
> >> > > > </action>
> >> > > > <kill name="fail">
> >> > > > <message>Java failed, error
> >> > > > message[${wf:errorMessage(wf:lastErrorNode())}]</message>
> >> > > > </kill>
> >> > > > <end name="end"/>
> >> > > > </workflow-app>
> >> > > >
> >> > > >
> >> > > >
> >> > > > On Wed, Jan 13, 2016 at 6:00 PM, Hanmay Udgiri <
> >> > > hanmayya.udgiri@gmail.com>
> >> > > > wrote:
> >> > > >
> >> > > > > Can u share the workflow.xml or the java code?
> >> > > > > On 13-Jan-2016 17:34, "Asanka Sanjaya Herath" <
> angal.sh@gmail.com
> >> >
> >> > > > wrote:
> >> > > > >
> >> > > > > > Hi Hanmayya,
> >> > > > > >
> >> > > > > > I followed the given link but it failed to spawn new Oozie
> jobs.
> >> > But
> >> > > > > when I
> >> > > > > > run the same program directly using yarn jar command It spawns
> >> the
> >> > > jobs
> >> > > > > in
> >> > > > > > Oozie. The yarn log is not giving any errors too.
> >> > > > > >
> >> > > > > > On Wed, Jan 13, 2016 at 11:42 AM, Asanka Sanjaya Herath <
> >> > > > > > angal.sh@gmail.com>
> >> > > > > > wrote:
> >> > > > > >
> >> > > > > > > Hi Hanmayya,
> >> > > > > > >
> >> > > > > > > Thank you for the response. I'll follow the given link and
> >> update
> >> > > > > status.
> >> > > > > > >
> >> > > > > > >
> >> > > > > > >> Do you want to execute multiple Oozie workflows from the
> Java
> >> > > > class?
> >> > > > > > >
> >> > > > > > >
> >> > > > > > > No I want to execute set of instances of one Oozie workflow.
> >> > > > > > >
> >> > > > > > >
> >> > > > > > > On Wed, Jan 13, 2016 at 10:43 AM, Hanmay Udgiri <
> >> > > > > > hanmayya.udgiri@gmail.com
> >> > > > > > > > wrote:
> >> > > > > > >
> >> > > > > > >> You can use OozieClient Java class.
> >> > > > > > >>
> >> > > > > > >>
> >> > > > > > >>
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> http://www.hadooped.blogspot.in/2013/06/apache-oozie-part-8-java-api-for.html
> >> > > > > > >>
> >> > > > > > >> Do you want to execute multiple Oozie workflows from the
> Java
> >> > > class?
> >> > > > > > >>
> >> > > > > > >> Thanks
> >> > > > > > >> Hanmayya
> >> > > > > > >>
> >> > > > > > >> On Wed, Jan 13, 2016 at 9:57 AM, Asanka Sanjaya Herath <
> >> > > > > > >> angal.sh@gmail.com>
> >> > > > > > >> wrote:
> >> > > > > > >>
> >> > > > > > >> > Hi,
> >> > > > > > >> >
> >> > > > > > >> > I'm writing a simple Java Oozie job. That job is supposed
> >> to
> >> > > spawn
> >> > > > > new
> >> > > > > > >> > Oozie jobs or directly submit new jobs to Hadoop. Is
> there
> >> a
> >> > way
> >> > > > to
> >> > > > > > >> achieve
> >> > > > > > >> > this using Oozie API?
> >> > > > > > >> >
> >> > > > > > >> > --
> >> > > > > > >> > Thanks,
> >> > > > > > >> > Regards,
> >> > > > > > >> > ASH
> >> > > > > > >> >
> >> > > > > > >>
> >> > > > > > >>
> >> > > > > > >>
> >> > > > > > >> --
> >> > > > > > >> Thanks and Regards
> >> > > > > > >> Hanmayya Udgiri
> >> > > > > > >>
> >> > > > > > >
> >> > > > > > >
> >> > > > > > >
> >> > > > > > > --
> >> > > > > > > Thanks,
> >> > > > > > > Regards,
> >> > > > > > > ASH
> >> > > > > > >
> >> > > > > >
> >> > > > > >
> >> > > > > >
> >> > > > > > --
> >> > > > > > Thanks,
> >> > > > > > Regards,
> >> > > > > > ASH
> >> > > > > >
> >> > > > >
> >> > > >
> >> > > >
> >> > > >
> >> > > > --
> >> > > > Thanks,
> >> > > > Regards,
> >> > > > ASH
> >> > > >
> >> > >
> >> >
> >> >
> >> >
> >> > --
> >> > Thanks,
> >> > Regards,
> >> > ASH
> >> >
> >>
> >
> >
> >
> > --
> > Thanks,
> > Regards,
> > ASH
> >
>
>
>
> --
> Thanks,
> Regards,
> ASH
>
Re: How to submit an Oozie job using another Oozie job
Posted by Asanka Sanjaya Herath <an...@gmail.com>.
Hi,
I think I figure out the problem. It is a permission issue. When I look the
yarn application log, I see following error.
E0507: Could not access to
[hdfs://xx.xxx.xx.xx8020/user/hdfs/email-dispatcher-tenant-manager-1.0/],
Permission denied: user=yarn, access=EXECUTE,
inode="/user/hdfs/email-dispatcher-tenant-manager-1.0":hdfs:hdfs:drwx------
at
org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:319)
at
org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkTraverse(FSPermissionChecker.java:259)
at
org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:205)
at
org.apache.ranger.authorization.hadoop.RangerHdfsAuthorizer$RangerAccessControlEnforcer.checkPermission(RangerHdfsAuthorizer.java:300)
at
org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:190)
at
org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:1771)
at
org.apache.hadoop.hdfs.server.namenode.FSDirStatAndListingOp.getFileInfo(FSDirStatAndListingOp.java:108)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getFileInfo(FSNamesystem.java:3857)
at
org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getFileInfo(NameNodeRpcServer.java:1012)
at
org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getFileInfo(ClientNamenodeProtocolServerSideTranslatorPB.java:843)
at
org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
at
org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:616)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:969) at
org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2137) at
org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2133) at
java.security.AccessController.doPrivileged(Native Method) at
javax.security.auth.Subject.doAs(Subject.java:415) at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2131)
at
org.apache.oozie.client.OozieClient.handleError(OozieClient.java:612)
at
org.apache.oozie.client.OozieClient$JobSubmit.call(OozieClient.java:695)
at
org.apache.oozie.client.OozieClient$JobSubmit.call(OozieClient.java:665)
at
org.apache.oozie.client.OozieClient$ClientCallable.call(OozieClient.java:560)
at org.apache.oozie.client.OozieClient.run(OozieClient.java:850)
at
com.sonasoft.sonacloud.email.dispatcher.manager.Utils.submitDispatcherJobForTenantToOozie(Utils.java:155)
at
com.sonasoft.sonacloud.email.dispatcher.manager.Utils.spawnEmailDispatcherJobsPerTenant(Utils.java:43)
at
com.sonasoft.sonacloud.email.dispatcher.manager.MainClass.main(MainClass.java:27)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.oozie.action.hadoop.JavaMain.run(JavaMain.java:56)
at
org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:47)
at org.apache.oozie.action.hadoop.JavaMain.main(JavaMain.java:35)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:236)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:453)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
End of LogType:stderr
On Wed, Jan 13, 2016 at 7:05 PM, Asanka Sanjaya Herath <an...@gmail.com>
wrote:
> No. Only the child job runs using java API. The manager job is submitted
> to Oozie manually using the command line. The above provided workflow.xml
> is for the child job.
>
> On Wed, Jan 13, 2016 at 7:00 PM, Hanmay Udgiri <ha...@gmail.com>
> wrote:
>
>> I guess these two jobs run using the java API. What about workflow.xml of
>> child job?
>> On 13-Jan-2016 6:26 pm, "Asanka Sanjaya Herath" <an...@gmail.com>
>> wrote:
>>
>> > There are basically two jobs.
>> >
>> > 1. manager job
>> > 2. child job
>> >
>> >
>> > Manager job is running on Oozie and responsible for creating new
>> instances
>> > of child jobs in Oozie when needed.
>> >
>> > I can see the manager job run in Oozie without giving any error. When
>> the
>> > submitDispatcherJobForTenantToOozie(String tenantId) method invoked in
>> > manager job, it should submit a new child job to Oozie. The method is
>> > invoking properly. But I cannot see any child job in Oozie console.
>> >
>> > But When I run the manager job directly in hadoop(Without using Oozie),
>> it
>> > spawns new child jobs in Oozie properly.
>> >
>> >
>> >
>> > On Wed, Jan 13, 2016 at 6:20 PM, Hanmay Udgiri <
>> hanmayya.udgiri@gmail.com>
>> > wrote:
>> >
>> > > So are u able to see the job running in HUE?.U can also view job in
>> the
>> > > oozie web console ? What about the workflow.xml of tenantapp?Is
>> > > The MainClass job able to call tenantapp workflow?
>> >
>> > On 13-Jan-2016 6:09 pm, "Asanka Sanjaya Herath" <an...@gmail.com>
>> > wrote:
>> > >
>> > > > Java method to create a job in Oozie.
>> > > >
>> > > > private void submitDispatcherJobForTenantToOozie(String tenantId)
>> > > > throws OozieClientException {
>> > > >
>> > > > // get a OozieClient for local Oozie
>> > > > OozieClient wc = new
>> > > >
>> > >
>> >
>> OozieClient(PropertyValuesProvider.getInstance().getPropValue("oozieURL"));
>> > > >
>> > > > // create a workflow job configuration and set the workflow
>> > > > application path
>> > > > Properties conf = wc.createConfiguration();
>> > > > conf.setProperty(OozieClient.APP_PATH,
>> > > > PropertyValuesProvider.getInstance().
>> > > > getPropValue("dispatcherTenantManagerAppPath"));
>> > > >
>> > > > // setting workflow parameters
>> > > > conf.setProperty("jobTracker", "sandbox.hortonworks.com:8050");
>> > > > conf.setProperty("nameNode", "hdfs://
>> sandbox.hortonworks.com:8020
>> > ");
>> > > > conf.setProperty("queueName", "default");
>> > > > conf.setProperty("appRoot",
>> "email-dispatcher-tenant-manager-1.0");
>> > > >
>> > > > //set tenantId and retention period
>> > > > conf.setProperty("tenantId", tenantId);
>> > > >
>> > > > // submit and start the workflow job
>> > > > String jobId = wc.run(conf);
>> > > > }
>> > > >
>> > > >
>> > > > Workfolw.xml of child job:
>> > > >
>> > > >
>> > > > <workflow-app xmlns="uri:oozie:workflow:0.2" name="java-main-wf">
>> > > > <start to="java-node"/>
>> > > > <action name="java-node">
>> > > > <java>
>> > > > <job-tracker>${jobTracker}</job-tracker>
>> > > > <name-node>${nameNode}</name-node>
>> > > > <configuration>
>> > > > <property>
>> > > > <name>mapred.job.queue.name</name>
>> > > > <value>${queueName}</value>
>> > > > </property>
>> > > > </configuration>
>> > > > <main-class>com.MainClass</main-class>
>> > > > </java>
>> > > > <ok to="end"/>
>> > > > <error to="fail"/>
>> > > > </action>
>> > > > <kill name="fail">
>> > > > <message>Java failed, error
>> > > > message[${wf:errorMessage(wf:lastErrorNode())}]</message>
>> > > > </kill>
>> > > > <end name="end"/>
>> > > > </workflow-app>
>> > > >
>> > > >
>> > > >
>> > > > On Wed, Jan 13, 2016 at 6:00 PM, Hanmay Udgiri <
>> > > hanmayya.udgiri@gmail.com>
>> > > > wrote:
>> > > >
>> > > > > Can u share the workflow.xml or the java code?
>> > > > > On 13-Jan-2016 17:34, "Asanka Sanjaya Herath" <angal.sh@gmail.com
>> >
>> > > > wrote:
>> > > > >
>> > > > > > Hi Hanmayya,
>> > > > > >
>> > > > > > I followed the given link but it failed to spawn new Oozie jobs.
>> > But
>> > > > > when I
>> > > > > > run the same program directly using yarn jar command It spawns
>> the
>> > > jobs
>> > > > > in
>> > > > > > Oozie. The yarn log is not giving any errors too.
>> > > > > >
>> > > > > > On Wed, Jan 13, 2016 at 11:42 AM, Asanka Sanjaya Herath <
>> > > > > > angal.sh@gmail.com>
>> > > > > > wrote:
>> > > > > >
>> > > > > > > Hi Hanmayya,
>> > > > > > >
>> > > > > > > Thank you for the response. I'll follow the given link and
>> update
>> > > > > status.
>> > > > > > >
>> > > > > > >
>> > > > > > >> Do you want to execute multiple Oozie workflows from the Java
>> > > > class?
>> > > > > > >
>> > > > > > >
>> > > > > > > No I want to execute set of instances of one Oozie workflow.
>> > > > > > >
>> > > > > > >
>> > > > > > > On Wed, Jan 13, 2016 at 10:43 AM, Hanmay Udgiri <
>> > > > > > hanmayya.udgiri@gmail.com
>> > > > > > > > wrote:
>> > > > > > >
>> > > > > > >> You can use OozieClient Java class.
>> > > > > > >>
>> > > > > > >>
>> > > > > > >>
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> http://www.hadooped.blogspot.in/2013/06/apache-oozie-part-8-java-api-for.html
>> > > > > > >>
>> > > > > > >> Do you want to execute multiple Oozie workflows from the Java
>> > > class?
>> > > > > > >>
>> > > > > > >> Thanks
>> > > > > > >> Hanmayya
>> > > > > > >>
>> > > > > > >> On Wed, Jan 13, 2016 at 9:57 AM, Asanka Sanjaya Herath <
>> > > > > > >> angal.sh@gmail.com>
>> > > > > > >> wrote:
>> > > > > > >>
>> > > > > > >> > Hi,
>> > > > > > >> >
>> > > > > > >> > I'm writing a simple Java Oozie job. That job is supposed
>> to
>> > > spawn
>> > > > > new
>> > > > > > >> > Oozie jobs or directly submit new jobs to Hadoop. Is there
>> a
>> > way
>> > > > to
>> > > > > > >> achieve
>> > > > > > >> > this using Oozie API?
>> > > > > > >> >
>> > > > > > >> > --
>> > > > > > >> > Thanks,
>> > > > > > >> > Regards,
>> > > > > > >> > ASH
>> > > > > > >> >
>> > > > > > >>
>> > > > > > >>
>> > > > > > >>
>> > > > > > >> --
>> > > > > > >> Thanks and Regards
>> > > > > > >> Hanmayya Udgiri
>> > > > > > >>
>> > > > > > >
>> > > > > > >
>> > > > > > >
>> > > > > > > --
>> > > > > > > Thanks,
>> > > > > > > Regards,
>> > > > > > > ASH
>> > > > > > >
>> > > > > >
>> > > > > >
>> > > > > >
>> > > > > > --
>> > > > > > Thanks,
>> > > > > > Regards,
>> > > > > > ASH
>> > > > > >
>> > > > >
>> > > >
>> > > >
>> > > >
>> > > > --
>> > > > Thanks,
>> > > > Regards,
>> > > > ASH
>> > > >
>> > >
>> >
>> >
>> >
>> > --
>> > Thanks,
>> > Regards,
>> > ASH
>> >
>>
>
>
>
> --
> Thanks,
> Regards,
> ASH
>
--
Thanks,
Regards,
ASH
Re: How to submit an Oozie job using another Oozie job
Posted by Asanka Sanjaya Herath <an...@gmail.com>.
No. Only the child job runs using java API. The manager job is submitted to
Oozie manually using the command line. The above provided workflow.xml is
for the child job.
On Wed, Jan 13, 2016 at 7:00 PM, Hanmay Udgiri <ha...@gmail.com>
wrote:
> I guess these two jobs run using the java API. What about workflow.xml of
> child job?
> On 13-Jan-2016 6:26 pm, "Asanka Sanjaya Herath" <an...@gmail.com>
> wrote:
>
> > There are basically two jobs.
> >
> > 1. manager job
> > 2. child job
> >
> >
> > Manager job is running on Oozie and responsible for creating new
> instances
> > of child jobs in Oozie when needed.
> >
> > I can see the manager job run in Oozie without giving any error. When the
> > submitDispatcherJobForTenantToOozie(String tenantId) method invoked in
> > manager job, it should submit a new child job to Oozie. The method is
> > invoking properly. But I cannot see any child job in Oozie console.
> >
> > But When I run the manager job directly in hadoop(Without using Oozie),
> it
> > spawns new child jobs in Oozie properly.
> >
> >
> >
> > On Wed, Jan 13, 2016 at 6:20 PM, Hanmay Udgiri <
> hanmayya.udgiri@gmail.com>
> > wrote:
> >
> > > So are u able to see the job running in HUE?.U can also view job in the
> > > oozie web console ? What about the workflow.xml of tenantapp?Is
> > > The MainClass job able to call tenantapp workflow?
> >
> > On 13-Jan-2016 6:09 pm, "Asanka Sanjaya Herath" <an...@gmail.com>
> > wrote:
> > >
> > > > Java method to create a job in Oozie.
> > > >
> > > > private void submitDispatcherJobForTenantToOozie(String tenantId)
> > > > throws OozieClientException {
> > > >
> > > > // get a OozieClient for local Oozie
> > > > OozieClient wc = new
> > > >
> > >
> >
> OozieClient(PropertyValuesProvider.getInstance().getPropValue("oozieURL"));
> > > >
> > > > // create a workflow job configuration and set the workflow
> > > > application path
> > > > Properties conf = wc.createConfiguration();
> > > > conf.setProperty(OozieClient.APP_PATH,
> > > > PropertyValuesProvider.getInstance().
> > > > getPropValue("dispatcherTenantManagerAppPath"));
> > > >
> > > > // setting workflow parameters
> > > > conf.setProperty("jobTracker", "sandbox.hortonworks.com:8050");
> > > > conf.setProperty("nameNode", "hdfs://
> sandbox.hortonworks.com:8020
> > ");
> > > > conf.setProperty("queueName", "default");
> > > > conf.setProperty("appRoot",
> "email-dispatcher-tenant-manager-1.0");
> > > >
> > > > //set tenantId and retention period
> > > > conf.setProperty("tenantId", tenantId);
> > > >
> > > > // submit and start the workflow job
> > > > String jobId = wc.run(conf);
> > > > }
> > > >
> > > >
> > > > Workfolw.xml of child job:
> > > >
> > > >
> > > > <workflow-app xmlns="uri:oozie:workflow:0.2" name="java-main-wf">
> > > > <start to="java-node"/>
> > > > <action name="java-node">
> > > > <java>
> > > > <job-tracker>${jobTracker}</job-tracker>
> > > > <name-node>${nameNode}</name-node>
> > > > <configuration>
> > > > <property>
> > > > <name>mapred.job.queue.name</name>
> > > > <value>${queueName}</value>
> > > > </property>
> > > > </configuration>
> > > > <main-class>com.MainClass</main-class>
> > > > </java>
> > > > <ok to="end"/>
> > > > <error to="fail"/>
> > > > </action>
> > > > <kill name="fail">
> > > > <message>Java failed, error
> > > > message[${wf:errorMessage(wf:lastErrorNode())}]</message>
> > > > </kill>
> > > > <end name="end"/>
> > > > </workflow-app>
> > > >
> > > >
> > > >
> > > > On Wed, Jan 13, 2016 at 6:00 PM, Hanmay Udgiri <
> > > hanmayya.udgiri@gmail.com>
> > > > wrote:
> > > >
> > > > > Can u share the workflow.xml or the java code?
> > > > > On 13-Jan-2016 17:34, "Asanka Sanjaya Herath" <an...@gmail.com>
> > > > wrote:
> > > > >
> > > > > > Hi Hanmayya,
> > > > > >
> > > > > > I followed the given link but it failed to spawn new Oozie jobs.
> > But
> > > > > when I
> > > > > > run the same program directly using yarn jar command It spawns
> the
> > > jobs
> > > > > in
> > > > > > Oozie. The yarn log is not giving any errors too.
> > > > > >
> > > > > > On Wed, Jan 13, 2016 at 11:42 AM, Asanka Sanjaya Herath <
> > > > > > angal.sh@gmail.com>
> > > > > > wrote:
> > > > > >
> > > > > > > Hi Hanmayya,
> > > > > > >
> > > > > > > Thank you for the response. I'll follow the given link and
> update
> > > > > status.
> > > > > > >
> > > > > > >
> > > > > > >> Do you want to execute multiple Oozie workflows from the Java
> > > > class?
> > > > > > >
> > > > > > >
> > > > > > > No I want to execute set of instances of one Oozie workflow.
> > > > > > >
> > > > > > >
> > > > > > > On Wed, Jan 13, 2016 at 10:43 AM, Hanmay Udgiri <
> > > > > > hanmayya.udgiri@gmail.com
> > > > > > > > wrote:
> > > > > > >
> > > > > > >> You can use OozieClient Java class.
> > > > > > >>
> > > > > > >>
> > > > > > >>
> > > > > >
> > > > >
> > > >
> > >
> >
> http://www.hadooped.blogspot.in/2013/06/apache-oozie-part-8-java-api-for.html
> > > > > > >>
> > > > > > >> Do you want to execute multiple Oozie workflows from the Java
> > > class?
> > > > > > >>
> > > > > > >> Thanks
> > > > > > >> Hanmayya
> > > > > > >>
> > > > > > >> On Wed, Jan 13, 2016 at 9:57 AM, Asanka Sanjaya Herath <
> > > > > > >> angal.sh@gmail.com>
> > > > > > >> wrote:
> > > > > > >>
> > > > > > >> > Hi,
> > > > > > >> >
> > > > > > >> > I'm writing a simple Java Oozie job. That job is supposed to
> > > spawn
> > > > > new
> > > > > > >> > Oozie jobs or directly submit new jobs to Hadoop. Is there a
> > way
> > > > to
> > > > > > >> achieve
> > > > > > >> > this using Oozie API?
> > > > > > >> >
> > > > > > >> > --
> > > > > > >> > Thanks,
> > > > > > >> > Regards,
> > > > > > >> > ASH
> > > > > > >> >
> > > > > > >>
> > > > > > >>
> > > > > > >>
> > > > > > >> --
> > > > > > >> Thanks and Regards
> > > > > > >> Hanmayya Udgiri
> > > > > > >>
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > Thanks,
> > > > > > > Regards,
> > > > > > > ASH
> > > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Thanks,
> > > > > > Regards,
> > > > > > ASH
> > > > > >
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Thanks,
> > > > Regards,
> > > > ASH
> > > >
> > >
> >
> >
> >
> > --
> > Thanks,
> > Regards,
> > ASH
> >
>
--
Thanks,
Regards,
ASH
Re: How to submit an Oozie job using another Oozie job
Posted by Hanmay Udgiri <ha...@gmail.com>.
I guess these two jobs run using the java API. What about workflow.xml of
child job?
On 13-Jan-2016 6:26 pm, "Asanka Sanjaya Herath" <an...@gmail.com> wrote:
> There are basically two jobs.
>
> 1. manager job
> 2. child job
>
>
> Manager job is running on Oozie and responsible for creating new instances
> of child jobs in Oozie when needed.
>
> I can see the manager job run in Oozie without giving any error. When the
> submitDispatcherJobForTenantToOozie(String tenantId) method invoked in
> manager job, it should submit a new child job to Oozie. The method is
> invoking properly. But I cannot see any child job in Oozie console.
>
> But When I run the manager job directly in hadoop(Without using Oozie), it
> spawns new child jobs in Oozie properly.
>
>
>
> On Wed, Jan 13, 2016 at 6:20 PM, Hanmay Udgiri <ha...@gmail.com>
> wrote:
>
> > So are u able to see the job running in HUE?.U can also view job in the
> > oozie web console ? What about the workflow.xml of tenantapp?Is
> > The MainClass job able to call tenantapp workflow?
>
> On 13-Jan-2016 6:09 pm, "Asanka Sanjaya Herath" <an...@gmail.com>
> wrote:
> >
> > > Java method to create a job in Oozie.
> > >
> > > private void submitDispatcherJobForTenantToOozie(String tenantId)
> > > throws OozieClientException {
> > >
> > > // get a OozieClient for local Oozie
> > > OozieClient wc = new
> > >
> >
> OozieClient(PropertyValuesProvider.getInstance().getPropValue("oozieURL"));
> > >
> > > // create a workflow job configuration and set the workflow
> > > application path
> > > Properties conf = wc.createConfiguration();
> > > conf.setProperty(OozieClient.APP_PATH,
> > > PropertyValuesProvider.getInstance().
> > > getPropValue("dispatcherTenantManagerAppPath"));
> > >
> > > // setting workflow parameters
> > > conf.setProperty("jobTracker", "sandbox.hortonworks.com:8050");
> > > conf.setProperty("nameNode", "hdfs://sandbox.hortonworks.com:8020
> ");
> > > conf.setProperty("queueName", "default");
> > > conf.setProperty("appRoot", "email-dispatcher-tenant-manager-1.0");
> > >
> > > //set tenantId and retention period
> > > conf.setProperty("tenantId", tenantId);
> > >
> > > // submit and start the workflow job
> > > String jobId = wc.run(conf);
> > > }
> > >
> > >
> > > Workfolw.xml of child job:
> > >
> > >
> > > <workflow-app xmlns="uri:oozie:workflow:0.2" name="java-main-wf">
> > > <start to="java-node"/>
> > > <action name="java-node">
> > > <java>
> > > <job-tracker>${jobTracker}</job-tracker>
> > > <name-node>${nameNode}</name-node>
> > > <configuration>
> > > <property>
> > > <name>mapred.job.queue.name</name>
> > > <value>${queueName}</value>
> > > </property>
> > > </configuration>
> > > <main-class>com.MainClass</main-class>
> > > </java>
> > > <ok to="end"/>
> > > <error to="fail"/>
> > > </action>
> > > <kill name="fail">
> > > <message>Java failed, error
> > > message[${wf:errorMessage(wf:lastErrorNode())}]</message>
> > > </kill>
> > > <end name="end"/>
> > > </workflow-app>
> > >
> > >
> > >
> > > On Wed, Jan 13, 2016 at 6:00 PM, Hanmay Udgiri <
> > hanmayya.udgiri@gmail.com>
> > > wrote:
> > >
> > > > Can u share the workflow.xml or the java code?
> > > > On 13-Jan-2016 17:34, "Asanka Sanjaya Herath" <an...@gmail.com>
> > > wrote:
> > > >
> > > > > Hi Hanmayya,
> > > > >
> > > > > I followed the given link but it failed to spawn new Oozie jobs.
> But
> > > > when I
> > > > > run the same program directly using yarn jar command It spawns the
> > jobs
> > > > in
> > > > > Oozie. The yarn log is not giving any errors too.
> > > > >
> > > > > On Wed, Jan 13, 2016 at 11:42 AM, Asanka Sanjaya Herath <
> > > > > angal.sh@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > Hi Hanmayya,
> > > > > >
> > > > > > Thank you for the response. I'll follow the given link and update
> > > > status.
> > > > > >
> > > > > >
> > > > > >> Do you want to execute multiple Oozie workflows from the Java
> > > class?
> > > > > >
> > > > > >
> > > > > > No I want to execute set of instances of one Oozie workflow.
> > > > > >
> > > > > >
> > > > > > On Wed, Jan 13, 2016 at 10:43 AM, Hanmay Udgiri <
> > > > > hanmayya.udgiri@gmail.com
> > > > > > > wrote:
> > > > > >
> > > > > >> You can use OozieClient Java class.
> > > > > >>
> > > > > >>
> > > > > >>
> > > > >
> > > >
> > >
> >
> http://www.hadooped.blogspot.in/2013/06/apache-oozie-part-8-java-api-for.html
> > > > > >>
> > > > > >> Do you want to execute multiple Oozie workflows from the Java
> > class?
> > > > > >>
> > > > > >> Thanks
> > > > > >> Hanmayya
> > > > > >>
> > > > > >> On Wed, Jan 13, 2016 at 9:57 AM, Asanka Sanjaya Herath <
> > > > > >> angal.sh@gmail.com>
> > > > > >> wrote:
> > > > > >>
> > > > > >> > Hi,
> > > > > >> >
> > > > > >> > I'm writing a simple Java Oozie job. That job is supposed to
> > spawn
> > > > new
> > > > > >> > Oozie jobs or directly submit new jobs to Hadoop. Is there a
> way
> > > to
> > > > > >> achieve
> > > > > >> > this using Oozie API?
> > > > > >> >
> > > > > >> > --
> > > > > >> > Thanks,
> > > > > >> > Regards,
> > > > > >> > ASH
> > > > > >> >
> > > > > >>
> > > > > >>
> > > > > >>
> > > > > >> --
> > > > > >> Thanks and Regards
> > > > > >> Hanmayya Udgiri
> > > > > >>
> > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Thanks,
> > > > > > Regards,
> > > > > > ASH
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Thanks,
> > > > > Regards,
> > > > > ASH
> > > > >
> > > >
> > >
> > >
> > >
> > > --
> > > Thanks,
> > > Regards,
> > > ASH
> > >
> >
>
>
>
> --
> Thanks,
> Regards,
> ASH
>
Re: How to submit an Oozie job using another Oozie job
Posted by Asanka Sanjaya Herath <an...@gmail.com>.
There are basically two jobs.
1. manager job
2. child job
Manager job is running on Oozie and responsible for creating new instances
of child jobs in Oozie when needed.
I can see the manager job run in Oozie without giving any error. When the
submitDispatcherJobForTenantToOozie(String tenantId) method invoked in
manager job, it should submit a new child job to Oozie. The method is
invoking properly. But I cannot see any child job in Oozie console.
But When I run the manager job directly in hadoop(Without using Oozie), it
spawns new child jobs in Oozie properly.
On Wed, Jan 13, 2016 at 6:20 PM, Hanmay Udgiri <ha...@gmail.com>
wrote:
> So are u able to see the job running in HUE?.U can also view job in the
> oozie web console ? What about the workflow.xml of tenantapp?Is
> The MainClass job able to call tenantapp workflow?
On 13-Jan-2016 6:09 pm, "Asanka Sanjaya Herath" <an...@gmail.com> wrote:
>
> > Java method to create a job in Oozie.
> >
> > private void submitDispatcherJobForTenantToOozie(String tenantId)
> > throws OozieClientException {
> >
> > // get a OozieClient for local Oozie
> > OozieClient wc = new
> >
> OozieClient(PropertyValuesProvider.getInstance().getPropValue("oozieURL"));
> >
> > // create a workflow job configuration and set the workflow
> > application path
> > Properties conf = wc.createConfiguration();
> > conf.setProperty(OozieClient.APP_PATH,
> > PropertyValuesProvider.getInstance().
> > getPropValue("dispatcherTenantManagerAppPath"));
> >
> > // setting workflow parameters
> > conf.setProperty("jobTracker", "sandbox.hortonworks.com:8050");
> > conf.setProperty("nameNode", "hdfs://sandbox.hortonworks.com:8020");
> > conf.setProperty("queueName", "default");
> > conf.setProperty("appRoot", "email-dispatcher-tenant-manager-1.0");
> >
> > //set tenantId and retention period
> > conf.setProperty("tenantId", tenantId);
> >
> > // submit and start the workflow job
> > String jobId = wc.run(conf);
> > }
> >
> >
> > Workfolw.xml of child job:
> >
> >
> > <workflow-app xmlns="uri:oozie:workflow:0.2" name="java-main-wf">
> > <start to="java-node"/>
> > <action name="java-node">
> > <java>
> > <job-tracker>${jobTracker}</job-tracker>
> > <name-node>${nameNode}</name-node>
> > <configuration>
> > <property>
> > <name>mapred.job.queue.name</name>
> > <value>${queueName}</value>
> > </property>
> > </configuration>
> > <main-class>com.MainClass</main-class>
> > </java>
> > <ok to="end"/>
> > <error to="fail"/>
> > </action>
> > <kill name="fail">
> > <message>Java failed, error
> > message[${wf:errorMessage(wf:lastErrorNode())}]</message>
> > </kill>
> > <end name="end"/>
> > </workflow-app>
> >
> >
> >
> > On Wed, Jan 13, 2016 at 6:00 PM, Hanmay Udgiri <
> hanmayya.udgiri@gmail.com>
> > wrote:
> >
> > > Can u share the workflow.xml or the java code?
> > > On 13-Jan-2016 17:34, "Asanka Sanjaya Herath" <an...@gmail.com>
> > wrote:
> > >
> > > > Hi Hanmayya,
> > > >
> > > > I followed the given link but it failed to spawn new Oozie jobs. But
> > > when I
> > > > run the same program directly using yarn jar command It spawns the
> jobs
> > > in
> > > > Oozie. The yarn log is not giving any errors too.
> > > >
> > > > On Wed, Jan 13, 2016 at 11:42 AM, Asanka Sanjaya Herath <
> > > > angal.sh@gmail.com>
> > > > wrote:
> > > >
> > > > > Hi Hanmayya,
> > > > >
> > > > > Thank you for the response. I'll follow the given link and update
> > > status.
> > > > >
> > > > >
> > > > >> Do you want to execute multiple Oozie workflows from the Java
> > class?
> > > > >
> > > > >
> > > > > No I want to execute set of instances of one Oozie workflow.
> > > > >
> > > > >
> > > > > On Wed, Jan 13, 2016 at 10:43 AM, Hanmay Udgiri <
> > > > hanmayya.udgiri@gmail.com
> > > > > > wrote:
> > > > >
> > > > >> You can use OozieClient Java class.
> > > > >>
> > > > >>
> > > > >>
> > > >
> > >
> >
> http://www.hadooped.blogspot.in/2013/06/apache-oozie-part-8-java-api-for.html
> > > > >>
> > > > >> Do you want to execute multiple Oozie workflows from the Java
> class?
> > > > >>
> > > > >> Thanks
> > > > >> Hanmayya
> > > > >>
> > > > >> On Wed, Jan 13, 2016 at 9:57 AM, Asanka Sanjaya Herath <
> > > > >> angal.sh@gmail.com>
> > > > >> wrote:
> > > > >>
> > > > >> > Hi,
> > > > >> >
> > > > >> > I'm writing a simple Java Oozie job. That job is supposed to
> spawn
> > > new
> > > > >> > Oozie jobs or directly submit new jobs to Hadoop. Is there a way
> > to
> > > > >> achieve
> > > > >> > this using Oozie API?
> > > > >> >
> > > > >> > --
> > > > >> > Thanks,
> > > > >> > Regards,
> > > > >> > ASH
> > > > >> >
> > > > >>
> > > > >>
> > > > >>
> > > > >> --
> > > > >> Thanks and Regards
> > > > >> Hanmayya Udgiri
> > > > >>
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Thanks,
> > > > > Regards,
> > > > > ASH
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Thanks,
> > > > Regards,
> > > > ASH
> > > >
> > >
> >
> >
> >
> > --
> > Thanks,
> > Regards,
> > ASH
> >
>
--
Thanks,
Regards,
ASH
Re: How to submit an Oozie job using another Oozie job
Posted by Hanmay Udgiri <ha...@gmail.com>.
So are u able to see the job running in HUE?.U can also view job in the
oozie web console ? What about the workflow.xml of tenantapp?Is
The MainClass job able to call tenantapp workflow?
On 13-Jan-2016 6:09 pm, "Asanka Sanjaya Herath" <an...@gmail.com> wrote:
> Java method to create a job in Oozie.
>
> private void submitDispatcherJobForTenantToOozie(String tenantId)
> throws OozieClientException {
>
> // get a OozieClient for local Oozie
> OozieClient wc = new
> OozieClient(PropertyValuesProvider.getInstance().getPropValue("oozieURL"));
>
> // create a workflow job configuration and set the workflow
> application path
> Properties conf = wc.createConfiguration();
> conf.setProperty(OozieClient.APP_PATH,
> PropertyValuesProvider.getInstance().
> getPropValue("dispatcherTenantManagerAppPath"));
>
> // setting workflow parameters
> conf.setProperty("jobTracker", "sandbox.hortonworks.com:8050");
> conf.setProperty("nameNode", "hdfs://sandbox.hortonworks.com:8020");
> conf.setProperty("queueName", "default");
> conf.setProperty("appRoot", "email-dispatcher-tenant-manager-1.0");
>
> //set tenantId and retention period
> conf.setProperty("tenantId", tenantId);
>
> // submit and start the workflow job
> String jobId = wc.run(conf);
> }
>
>
> Workfolw.xml of child job:
>
>
> <workflow-app xmlns="uri:oozie:workflow:0.2" name="java-main-wf">
> <start to="java-node"/>
> <action name="java-node">
> <java>
> <job-tracker>${jobTracker}</job-tracker>
> <name-node>${nameNode}</name-node>
> <configuration>
> <property>
> <name>mapred.job.queue.name</name>
> <value>${queueName}</value>
> </property>
> </configuration>
> <main-class>com.MainClass</main-class>
> </java>
> <ok to="end"/>
> <error to="fail"/>
> </action>
> <kill name="fail">
> <message>Java failed, error
> message[${wf:errorMessage(wf:lastErrorNode())}]</message>
> </kill>
> <end name="end"/>
> </workflow-app>
>
>
>
> On Wed, Jan 13, 2016 at 6:00 PM, Hanmay Udgiri <ha...@gmail.com>
> wrote:
>
> > Can u share the workflow.xml or the java code?
> > On 13-Jan-2016 17:34, "Asanka Sanjaya Herath" <an...@gmail.com>
> wrote:
> >
> > > Hi Hanmayya,
> > >
> > > I followed the given link but it failed to spawn new Oozie jobs. But
> > when I
> > > run the same program directly using yarn jar command It spawns the jobs
> > in
> > > Oozie. The yarn log is not giving any errors too.
> > >
> > > On Wed, Jan 13, 2016 at 11:42 AM, Asanka Sanjaya Herath <
> > > angal.sh@gmail.com>
> > > wrote:
> > >
> > > > Hi Hanmayya,
> > > >
> > > > Thank you for the response. I'll follow the given link and update
> > status.
> > > >
> > > >
> > > >> Do you want to execute multiple Oozie workflows from the Java
> class?
> > > >
> > > >
> > > > No I want to execute set of instances of one Oozie workflow.
> > > >
> > > >
> > > > On Wed, Jan 13, 2016 at 10:43 AM, Hanmay Udgiri <
> > > hanmayya.udgiri@gmail.com
> > > > > wrote:
> > > >
> > > >> You can use OozieClient Java class.
> > > >>
> > > >>
> > > >>
> > >
> >
> http://www.hadooped.blogspot.in/2013/06/apache-oozie-part-8-java-api-for.html
> > > >>
> > > >> Do you want to execute multiple Oozie workflows from the Java class?
> > > >>
> > > >> Thanks
> > > >> Hanmayya
> > > >>
> > > >> On Wed, Jan 13, 2016 at 9:57 AM, Asanka Sanjaya Herath <
> > > >> angal.sh@gmail.com>
> > > >> wrote:
> > > >>
> > > >> > Hi,
> > > >> >
> > > >> > I'm writing a simple Java Oozie job. That job is supposed to spawn
> > new
> > > >> > Oozie jobs or directly submit new jobs to Hadoop. Is there a way
> to
> > > >> achieve
> > > >> > this using Oozie API?
> > > >> >
> > > >> > --
> > > >> > Thanks,
> > > >> > Regards,
> > > >> > ASH
> > > >> >
> > > >>
> > > >>
> > > >>
> > > >> --
> > > >> Thanks and Regards
> > > >> Hanmayya Udgiri
> > > >>
> > > >
> > > >
> > > >
> > > > --
> > > > Thanks,
> > > > Regards,
> > > > ASH
> > > >
> > >
> > >
> > >
> > > --
> > > Thanks,
> > > Regards,
> > > ASH
> > >
> >
>
>
>
> --
> Thanks,
> Regards,
> ASH
>
Re: How to submit an Oozie job using another Oozie job
Posted by Asanka Sanjaya Herath <an...@gmail.com>.
Java method to create a job in Oozie.
private void submitDispatcherJobForTenantToOozie(String tenantId)
throws OozieClientException {
// get a OozieClient for local Oozie
OozieClient wc = new
OozieClient(PropertyValuesProvider.getInstance().getPropValue("oozieURL"));
// create a workflow job configuration and set the workflow application path
Properties conf = wc.createConfiguration();
conf.setProperty(OozieClient.APP_PATH, PropertyValuesProvider.getInstance().
getPropValue("dispatcherTenantManagerAppPath"));
// setting workflow parameters
conf.setProperty("jobTracker", "sandbox.hortonworks.com:8050");
conf.setProperty("nameNode", "hdfs://sandbox.hortonworks.com:8020");
conf.setProperty("queueName", "default");
conf.setProperty("appRoot", "email-dispatcher-tenant-manager-1.0");
//set tenantId and retention period
conf.setProperty("tenantId", tenantId);
// submit and start the workflow job
String jobId = wc.run(conf);
}
Workfolw.xml of child job:
<workflow-app xmlns="uri:oozie:workflow:0.2" name="java-main-wf">
<start to="java-node"/>
<action name="java-node">
<java>
<job-tracker>${jobTracker}</job-tracker>
<name-node>${nameNode}</name-node>
<configuration>
<property>
<name>mapred.job.queue.name</name>
<value>${queueName}</value>
</property>
</configuration>
<main-class>com.MainClass</main-class>
</java>
<ok to="end"/>
<error to="fail"/>
</action>
<kill name="fail">
<message>Java failed, error
message[${wf:errorMessage(wf:lastErrorNode())}]</message>
</kill>
<end name="end"/>
</workflow-app>
On Wed, Jan 13, 2016 at 6:00 PM, Hanmay Udgiri <ha...@gmail.com>
wrote:
> Can u share the workflow.xml or the java code?
> On 13-Jan-2016 17:34, "Asanka Sanjaya Herath" <an...@gmail.com> wrote:
>
> > Hi Hanmayya,
> >
> > I followed the given link but it failed to spawn new Oozie jobs. But
> when I
> > run the same program directly using yarn jar command It spawns the jobs
> in
> > Oozie. The yarn log is not giving any errors too.
> >
> > On Wed, Jan 13, 2016 at 11:42 AM, Asanka Sanjaya Herath <
> > angal.sh@gmail.com>
> > wrote:
> >
> > > Hi Hanmayya,
> > >
> > > Thank you for the response. I'll follow the given link and update
> status.
> > >
> > >
> > >> Do you want to execute multiple Oozie workflows from the Java class?
> > >
> > >
> > > No I want to execute set of instances of one Oozie workflow.
> > >
> > >
> > > On Wed, Jan 13, 2016 at 10:43 AM, Hanmay Udgiri <
> > hanmayya.udgiri@gmail.com
> > > > wrote:
> > >
> > >> You can use OozieClient Java class.
> > >>
> > >>
> > >>
> >
> http://www.hadooped.blogspot.in/2013/06/apache-oozie-part-8-java-api-for.html
> > >>
> > >> Do you want to execute multiple Oozie workflows from the Java class?
> > >>
> > >> Thanks
> > >> Hanmayya
> > >>
> > >> On Wed, Jan 13, 2016 at 9:57 AM, Asanka Sanjaya Herath <
> > >> angal.sh@gmail.com>
> > >> wrote:
> > >>
> > >> > Hi,
> > >> >
> > >> > I'm writing a simple Java Oozie job. That job is supposed to spawn
> new
> > >> > Oozie jobs or directly submit new jobs to Hadoop. Is there a way to
> > >> achieve
> > >> > this using Oozie API?
> > >> >
> > >> > --
> > >> > Thanks,
> > >> > Regards,
> > >> > ASH
> > >> >
> > >>
> > >>
> > >>
> > >> --
> > >> Thanks and Regards
> > >> Hanmayya Udgiri
> > >>
> > >
> > >
> > >
> > > --
> > > Thanks,
> > > Regards,
> > > ASH
> > >
> >
> >
> >
> > --
> > Thanks,
> > Regards,
> > ASH
> >
>
--
Thanks,
Regards,
ASH
Re: How to submit an Oozie job using another Oozie job
Posted by Hanmay Udgiri <ha...@gmail.com>.
Can u share the workflow.xml or the java code?
On 13-Jan-2016 17:34, "Asanka Sanjaya Herath" <an...@gmail.com> wrote:
> Hi Hanmayya,
>
> I followed the given link but it failed to spawn new Oozie jobs. But when I
> run the same program directly using yarn jar command It spawns the jobs in
> Oozie. The yarn log is not giving any errors too.
>
> On Wed, Jan 13, 2016 at 11:42 AM, Asanka Sanjaya Herath <
> angal.sh@gmail.com>
> wrote:
>
> > Hi Hanmayya,
> >
> > Thank you for the response. I'll follow the given link and update status.
> >
> >
> >> Do you want to execute multiple Oozie workflows from the Java class?
> >
> >
> > No I want to execute set of instances of one Oozie workflow.
> >
> >
> > On Wed, Jan 13, 2016 at 10:43 AM, Hanmay Udgiri <
> hanmayya.udgiri@gmail.com
> > > wrote:
> >
> >> You can use OozieClient Java class.
> >>
> >>
> >>
> http://www.hadooped.blogspot.in/2013/06/apache-oozie-part-8-java-api-for.html
> >>
> >> Do you want to execute multiple Oozie workflows from the Java class?
> >>
> >> Thanks
> >> Hanmayya
> >>
> >> On Wed, Jan 13, 2016 at 9:57 AM, Asanka Sanjaya Herath <
> >> angal.sh@gmail.com>
> >> wrote:
> >>
> >> > Hi,
> >> >
> >> > I'm writing a simple Java Oozie job. That job is supposed to spawn new
> >> > Oozie jobs or directly submit new jobs to Hadoop. Is there a way to
> >> achieve
> >> > this using Oozie API?
> >> >
> >> > --
> >> > Thanks,
> >> > Regards,
> >> > ASH
> >> >
> >>
> >>
> >>
> >> --
> >> Thanks and Regards
> >> Hanmayya Udgiri
> >>
> >
> >
> >
> > --
> > Thanks,
> > Regards,
> > ASH
> >
>
>
>
> --
> Thanks,
> Regards,
> ASH
>
Re: How to submit an Oozie job using another Oozie job
Posted by Asanka Sanjaya Herath <an...@gmail.com>.
Hi Hanmayya,
I followed the given link but it failed to spawn new Oozie jobs. But when I
run the same program directly using yarn jar command It spawns the jobs in
Oozie. The yarn log is not giving any errors too.
On Wed, Jan 13, 2016 at 11:42 AM, Asanka Sanjaya Herath <an...@gmail.com>
wrote:
> Hi Hanmayya,
>
> Thank you for the response. I'll follow the given link and update status.
>
>
>> Do you want to execute multiple Oozie workflows from the Java class?
>
>
> No I want to execute set of instances of one Oozie workflow.
>
>
> On Wed, Jan 13, 2016 at 10:43 AM, Hanmay Udgiri <hanmayya.udgiri@gmail.com
> > wrote:
>
>> You can use OozieClient Java class.
>>
>>
>> http://www.hadooped.blogspot.in/2013/06/apache-oozie-part-8-java-api-for.html
>>
>> Do you want to execute multiple Oozie workflows from the Java class?
>>
>> Thanks
>> Hanmayya
>>
>> On Wed, Jan 13, 2016 at 9:57 AM, Asanka Sanjaya Herath <
>> angal.sh@gmail.com>
>> wrote:
>>
>> > Hi,
>> >
>> > I'm writing a simple Java Oozie job. That job is supposed to spawn new
>> > Oozie jobs or directly submit new jobs to Hadoop. Is there a way to
>> achieve
>> > this using Oozie API?
>> >
>> > --
>> > Thanks,
>> > Regards,
>> > ASH
>> >
>>
>>
>>
>> --
>> Thanks and Regards
>> Hanmayya Udgiri
>>
>
>
>
> --
> Thanks,
> Regards,
> ASH
>
--
Thanks,
Regards,
ASH
Re: How to submit an Oozie job using another Oozie job
Posted by Asanka Sanjaya Herath <an...@gmail.com>.
Hi Hanmayya,
Thank you for the response. I'll follow the given link and update status.
> Do you want to execute multiple Oozie workflows from the Java class?
No I want to execute set of instances of one Oozie workflow.
On Wed, Jan 13, 2016 at 10:43 AM, Hanmay Udgiri <ha...@gmail.com>
wrote:
> You can use OozieClient Java class.
>
>
> http://www.hadooped.blogspot.in/2013/06/apache-oozie-part-8-java-api-for.html
>
> Do you want to execute multiple Oozie workflows from the Java class?
>
> Thanks
> Hanmayya
>
> On Wed, Jan 13, 2016 at 9:57 AM, Asanka Sanjaya Herath <angal.sh@gmail.com
> >
> wrote:
>
> > Hi,
> >
> > I'm writing a simple Java Oozie job. That job is supposed to spawn new
> > Oozie jobs or directly submit new jobs to Hadoop. Is there a way to
> achieve
> > this using Oozie API?
> >
> > --
> > Thanks,
> > Regards,
> > ASH
> >
>
>
>
> --
> Thanks and Regards
> Hanmayya Udgiri
>
--
Thanks,
Regards,
ASH
Re: How to submit an Oozie job using another Oozie job
Posted by Hanmay Udgiri <ha...@gmail.com>.
You can use OozieClient Java class.
http://www.hadooped.blogspot.in/2013/06/apache-oozie-part-8-java-api-for.html
Do you want to execute multiple Oozie workflows from the Java class?
Thanks
Hanmayya
On Wed, Jan 13, 2016 at 9:57 AM, Asanka Sanjaya Herath <an...@gmail.com>
wrote:
> Hi,
>
> I'm writing a simple Java Oozie job. That job is supposed to spawn new
> Oozie jobs or directly submit new jobs to Hadoop. Is there a way to achieve
> this using Oozie API?
>
> --
> Thanks,
> Regards,
> ASH
>
--
Thanks and Regards
Hanmayya Udgiri