You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@oozie.apache.org by Adrian Liu <ad...@yahoo-inc.com> on 2012/03/26 09:43:12 UTC
Question about oozie java action
Hi,
I wrote an oozie workflow that calls a java action. The java action calls System.out.println("Hello world");, but I didn't see that phrase anywhere. I also make the java program call "mkdir Hello", again I didn't find this directory created anywhere.
Could anyone please help on this? Thanks.
Adrian Liu
adrianl@yahoo-inc.com
Re: Question about oozie java action
Posted by Alejandro Abdelnur <tu...@cloudera.com>.
Adrian,
The SSH action is back, as an extension action.
Thanks.
Alejandro
On Mon, Mar 26, 2012 at 11:12 AM, Adrian Liu <ad...@yahoo-inc.com> wrote:
> My oozie version is 3.1.2, and I set xmlns="uri:oozie:workflow:0.2" in the
> workflow.xml.
>
> According to calling scripts in java as the link you provided, is my
> understanding correct? I think the script will be copied to the HDFS, and
> be executed on some nodes in their local file system. So if the script
> has some commands that are not supported in the nodes, it cannot be run
> successfully.
>
> Yes I thought using SSH action is very suitable, but then I read from the
> document that the SSH action is removed in oozie schema 0.2, so is it
> suggested to use that feature? I'm now just trying to work around by the
> java action.
>
> On Mar 26, 2012, at 4:52 PM, Mohammad Islam wrote:
>
> >
> >
> > Which version of Oozie are you using?
> > In 3.2,running shell command/script will be much easier. 3.2 is under
> QE certification and expected to be done in couple of weeks.
> >
> > However, in earlier versions, you could also achieve the same using the
> help from this link:
> > https://github.com/yahoo/oozie/wiki/Oozie-WF-use-cases
> >
> > Look for "Java-Main Action with Script support".
> >
> > One question is not clear to me " This script is originally not located
> on the cluster". Do you want to execute your script out side the cluster?
> If yes, answer is "SSH action.".
> >
> > Please elaborate.
> >
> > Regards,
> > Mohammad Islam
> >
> >
> >
> >
> >
> > ----- Original Message -----
> > From: Adrian Liu <ad...@yahoo-inc.com>
> > To: "oozie-users@incubator.apache.org" <oo...@incubator.apache.org>;
> Mohammad Islam <mi...@yahoo.com>
> > Cc:
> > Sent: Monday, March 26, 2012 1:42 AM
> > Subject: Re: Question about oozie java action
> >
> > Hi Mohammad,
> >
> > Thanks for your quick reply!
> >
> > I've found my print output following your steps.
> >
> > I want to call "mkdir Hello" command to simulate calling shell scripts
> within the java program.
> >
> > My actual task will be calling a script that did a lot of prepare work
> and then commit a mapreduce job to the cluster. I want to simply exec this
> script in the java program to put this script's work into the oozie flow.
> This script is originally not located on the cluster. That was what I
> want to do.
> >
> > On Mar 26, 2012, at 4:13 PM, Mohammad Islam wrote:
> >
> >> Hi Adrian,
> >> The output should be written into Launcher Mapper stdout file.
> >> How to find the file?
> >> From Oozie UI, click to WF job id which will bring a new window. Then
> click to the java action, that will pop up another window.
> >> Click on the right of "console URL" that will take you to Hadoop
> Launcher mapper. Then you need to click couple of times to go to the
> appropriate stdout of attempt log. More details could be found :
> >> http://incubator.apache.org/oozie/map-reduce-cookbook.html
> >>
> >> Look for "CASE-8: HOW TO USE THE OOZIE WEB-CONSOLE"
> >>
> >> "mkdir Hello" will create the directory in the current working
> directory of the task tracker. In general, it has a very limited usage.
> What are you trying to achieve with this?
> >>
> >>
> >> In short, Oozie runs java/MR/pig/hive action from launcher mapper.
> Please see here for details:
> >> http://incubator.apache.org/oozie/overview.html.
> >>
> >>
> >> Please feel free to ask further question.
> >>
> >> Regards,
> >> Mohammad
> >>
> >>
> >>
> >>
> >> ----- Original Message -----
> >> From: Adrian Liu <ad...@yahoo-inc.com>
> >> To: "oozie-users@incubator.apache.org" <
> oozie-users@incubator.apache.org>
> >> Cc:
> >> Sent: Monday, March 26, 2012 12:43 AM
> >> Subject: Question about oozie java action
> >>
> >> Hi,
> >>
> >> I wrote an oozie workflow that calls a java action. The java action
> calls System.out.println("Hello world");, but I didn't see that phrase
> anywhere. I also make the java program call "mkdir Hello", again I didn't
> find this directory created anywhere.
> >>
> >> Could anyone please help on this? Thanks.
> >>
> >> Adrian Liu
> >> adrianl@yahoo-inc.com
> >
> > Adrian Liu
> > adrianl@yahoo-inc.com
>
> Adrian Liu
> adrianl@yahoo-inc.com
>
>
>
>
Re: Question about oozie java action
Posted by Adrian Liu <ad...@yahoo-inc.com>.
My oozie version is 3.1.2, and I set xmlns="uri:oozie:workflow:0.2" in the workflow.xml.
According to calling scripts in java as the link you provided, is my understanding correct? I think the script will be copied to the HDFS, and be executed on some nodes in their local file system. So if the script has some commands that are not supported in the nodes, it cannot be run successfully.
Yes I thought using SSH action is very suitable, but then I read from the document that the SSH action is removed in oozie schema 0.2, so is it suggested to use that feature? I'm now just trying to work around by the java action.
On Mar 26, 2012, at 4:52 PM, Mohammad Islam wrote:
>
>
> Which version of Oozie are you using?
> In 3.2,running shell command/script will be much easier. 3.2 is under QE certification and expected to be done in couple of weeks.
>
> However, in earlier versions, you could also achieve the same using the help from this link:
> https://github.com/yahoo/oozie/wiki/Oozie-WF-use-cases
>
> Look for "Java-Main Action with Script support".
>
> One question is not clear to me " This script is originally not located on the cluster". Do you want to execute your script out side the cluster? If yes, answer is "SSH action.".
>
> Please elaborate.
>
> Regards,
> Mohammad Islam
>
>
>
>
>
> ----- Original Message -----
> From: Adrian Liu <ad...@yahoo-inc.com>
> To: "oozie-users@incubator.apache.org" <oo...@incubator.apache.org>; Mohammad Islam <mi...@yahoo.com>
> Cc:
> Sent: Monday, March 26, 2012 1:42 AM
> Subject: Re: Question about oozie java action
>
> Hi Mohammad,
>
> Thanks for your quick reply!
>
> I've found my print output following your steps.
>
> I want to call "mkdir Hello" command to simulate calling shell scripts within the java program.
>
> My actual task will be calling a script that did a lot of prepare work and then commit a mapreduce job to the cluster. I want to simply exec this script in the java program to put this script's work into the oozie flow. This script is originally not located on the cluster. That was what I want to do.
>
> On Mar 26, 2012, at 4:13 PM, Mohammad Islam wrote:
>
>> Hi Adrian,
>> The output should be written into Launcher Mapper stdout file.
>> How to find the file?
>> From Oozie UI, click to WF job id which will bring a new window. Then click to the java action, that will pop up another window.
>> Click on the right of "console URL" that will take you to Hadoop Launcher mapper. Then you need to click couple of times to go to the appropriate stdout of attempt log. More details could be found :
>> http://incubator.apache.org/oozie/map-reduce-cookbook.html
>>
>> Look for "CASE-8: HOW TO USE THE OOZIE WEB-CONSOLE"
>>
>> "mkdir Hello" will create the directory in the current working directory of the task tracker. In general, it has a very limited usage. What are you trying to achieve with this?
>>
>>
>> In short, Oozie runs java/MR/pig/hive action from launcher mapper. Please see here for details:
>> http://incubator.apache.org/oozie/overview.html.
>>
>>
>> Please feel free to ask further question.
>>
>> Regards,
>> Mohammad
>>
>>
>>
>>
>> ----- Original Message -----
>> From: Adrian Liu <ad...@yahoo-inc.com>
>> To: "oozie-users@incubator.apache.org" <oo...@incubator.apache.org>
>> Cc:
>> Sent: Monday, March 26, 2012 12:43 AM
>> Subject: Question about oozie java action
>>
>> Hi,
>>
>> I wrote an oozie workflow that calls a java action. The java action calls System.out.println("Hello world");, but I didn't see that phrase anywhere. I also make the java program call "mkdir Hello", again I didn't find this directory created anywhere.
>>
>> Could anyone please help on this? Thanks.
>>
>> Adrian Liu
>> adrianl@yahoo-inc.com
>
> Adrian Liu
> adrianl@yahoo-inc.com
Adrian Liu
adrianl@yahoo-inc.com
Re: Question about oozie java action
Posted by Mohammad Islam <mi...@yahoo.com>.
Which version of Oozie are you using?
In 3.2,running shell command/script will be much easier. 3.2 is under QE certification and expected to be done in couple of weeks.
However, in earlier versions, you could also achieve the same using the help from this link:
https://github.com/yahoo/oozie/wiki/Oozie-WF-use-cases
Look for "Java-Main Action with Script support".
One question is not clear to me " This script is originally not located on the cluster". Do you want to execute your script out side the cluster? If yes, answer is "SSH action.".
Please elaborate.
Regards,
Mohammad Islam
----- Original Message -----
From: Adrian Liu <ad...@yahoo-inc.com>
To: "oozie-users@incubator.apache.org" <oo...@incubator.apache.org>; Mohammad Islam <mi...@yahoo.com>
Cc:
Sent: Monday, March 26, 2012 1:42 AM
Subject: Re: Question about oozie java action
Hi Mohammad,
Thanks for your quick reply!
I've found my print output following your steps.
I want to call "mkdir Hello" command to simulate calling shell scripts within the java program.
My actual task will be calling a script that did a lot of prepare work and then commit a mapreduce job to the cluster. I want to simply exec this script in the java program to put this script's work into the oozie flow. This script is originally not located on the cluster. That was what I want to do.
On Mar 26, 2012, at 4:13 PM, Mohammad Islam wrote:
> Hi Adrian,
> The output should be written into Launcher Mapper stdout file.
> How to find the file?
> From Oozie UI, click to WF job id which will bring a new window. Then click to the java action, that will pop up another window.
> Click on the right of "console URL" that will take you to Hadoop Launcher mapper. Then you need to click couple of times to go to the appropriate stdout of attempt log. More details could be found :
> http://incubator.apache.org/oozie/map-reduce-cookbook.html
>
> Look for "CASE-8: HOW TO USE THE OOZIE WEB-CONSOLE"
>
> "mkdir Hello" will create the directory in the current working directory of the task tracker. In general, it has a very limited usage. What are you trying to achieve with this?
>
>
> In short, Oozie runs java/MR/pig/hive action from launcher mapper. Please see here for details:
> http://incubator.apache.org/oozie/overview.html.
>
>
> Please feel free to ask further question.
>
> Regards,
> Mohammad
>
>
>
>
> ----- Original Message -----
> From: Adrian Liu <ad...@yahoo-inc.com>
> To: "oozie-users@incubator.apache.org" <oo...@incubator.apache.org>
> Cc:
> Sent: Monday, March 26, 2012 12:43 AM
> Subject: Question about oozie java action
>
> Hi,
>
> I wrote an oozie workflow that calls a java action. The java action calls System.out.println("Hello world");, but I didn't see that phrase anywhere. I also make the java program call "mkdir Hello", again I didn't find this directory created anywhere.
>
> Could anyone please help on this? Thanks.
>
> Adrian Liu
> adrianl@yahoo-inc.com
Adrian Liu
adrianl@yahoo-inc.com
Re: Question about oozie java action
Posted by Adrian Liu <ad...@yahoo-inc.com>.
Hi Mohammad,
Thanks for your quick reply!
I've found my print output following your steps.
I want to call "mkdir Hello" command to simulate calling shell scripts within the java program.
My actual task will be calling a script that did a lot of prepare work and then commit a mapreduce job to the cluster. I want to simply exec this script in the java program to put this script's work into the oozie flow. This script is originally not located on the cluster. That was what I want to do.
On Mar 26, 2012, at 4:13 PM, Mohammad Islam wrote:
> Hi Adrian,
> The output should be written into Launcher Mapper stdout file.
> How to find the file?
> From Oozie UI, click to WF job id which will bring a new window. Then click to the java action, that will pop up another window.
> Click on the right of "console URL" that will take you to Hadoop Launcher mapper. Then you need to click couple of times to go to the appropriate stdout of attempt log. More details could be found :
> http://incubator.apache.org/oozie/map-reduce-cookbook.html
>
> Look for "CASE-8: HOW TO USE THE OOZIE WEB-CONSOLE"
>
> "mkdir Hello" will create the directory in the current working directory of the task tracker. In general, it has a very limited usage. What are you trying to achieve with this?
>
>
> In short, Oozie runs java/MR/pig/hive action from launcher mapper. Please see here for details:
> http://incubator.apache.org/oozie/overview.html.
>
>
> Please feel free to ask further question.
>
> Regards,
> Mohammad
>
>
>
>
> ----- Original Message -----
> From: Adrian Liu <ad...@yahoo-inc.com>
> To: "oozie-users@incubator.apache.org" <oo...@incubator.apache.org>
> Cc:
> Sent: Monday, March 26, 2012 12:43 AM
> Subject: Question about oozie java action
>
> Hi,
>
> I wrote an oozie workflow that calls a java action. The java action calls System.out.println("Hello world");, but I didn't see that phrase anywhere. I also make the java program call "mkdir Hello", again I didn't find this directory created anywhere.
>
> Could anyone please help on this? Thanks.
>
> Adrian Liu
> adrianl@yahoo-inc.com
Adrian Liu
adrianl@yahoo-inc.com
Re: Question about oozie java action
Posted by Mohammad Islam <mi...@yahoo.com>.
Hi Adrian,
The output should be written into Launcher Mapper stdout file.
How to find the file?
From Oozie UI, click to WF job id which will bring a new window. Then click to the java action, that will pop up another window.
Click on the right of "console URL" that will take you to Hadoop Launcher mapper. Then you need to click couple of times to go to the appropriate stdout of attempt log. More details could be found :
http://incubator.apache.org/oozie/map-reduce-cookbook.html
Look for "CASE-8: HOW TO USE THE OOZIE WEB-CONSOLE"
"mkdir Hello" will create the directory in the current working directory of the task tracker. In general, it has a very limited usage. What are you trying to achieve with this?
In short, Oozie runs java/MR/pig/hive action from launcher mapper. Please see here for details:
http://incubator.apache.org/oozie/overview.html.
Please feel free to ask further question.
Regards,
Mohammad
----- Original Message -----
From: Adrian Liu <ad...@yahoo-inc.com>
To: "oozie-users@incubator.apache.org" <oo...@incubator.apache.org>
Cc:
Sent: Monday, March 26, 2012 12:43 AM
Subject: Question about oozie java action
Hi,
I wrote an oozie workflow that calls a java action. The java action calls System.out.println("Hello world");, but I didn't see that phrase anywhere. I also make the java program call "mkdir Hello", again I didn't find this directory created anywhere.
Could anyone please help on this? Thanks.
Adrian Liu
adrianl@yahoo-inc.com