You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@oozie.apache.org by Mohammad Islam <mi...@yahoo.com> on 2011/12/04 09:05:24 UTC
Review Request: OOZIE-578 Support shell action in Oozie WF
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3000/
-----------------------------------------------------------
Review request for oozie.
Summary
-------
More context at JIRA OOZIE-578.
This addresses bug OOZIE-578.
https://issues.apache.org/jira/browse/OOZIE-578
Diffs
-----
http://svn.apache.org/repos/asf/incubator/oozie/trunk/client/src/main/java/org/apache/oozie/cli/OozieCLI.java 1209346
http://svn.apache.org/repos/asf/incubator/oozie/trunk/client/src/main/resources/shell-action-0.1.xsd PRE-CREATION
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 1209346
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java PRE-CREATION
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java PRE-CREATION
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/resources/oozie-default.xml 1209346
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/test/java/org/apache/oozie/action/hadoop/ShellTestCase.java PRE-CREATION
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestShellActionExecutor.java PRE-CREATION
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestShellMain.java PRE-CREATION
http://svn.apache.org/repos/asf/incubator/oozie/trunk/docs/src/site/twiki/DG_ShellActionExtension.twiki PRE-CREATION
http://svn.apache.org/repos/asf/incubator/oozie/trunk/docs/src/site/twiki/index.twiki 1209346
Diff: https://reviews.apache.org/r/3000/diff
Testing
-------
Thanks,
Mohammad
Re: Review Request: OOZIE-578 Support shell action in Oozie WF
Posted by Mohammad Islam <mi...@yahoo.com>.
> On 2011-12-05 18:40:43, Alejandro Abdelnur wrote:
> > http://svn.apache.org/repos/asf/incubator/oozie/trunk/client/src/main/resources/shell-action-0.1.xsd, line 34
> > <https://reviews.apache.org/r/3000/diff/2/?file=61824#file61824line34>
> >
> > env vars setting is missing, this is quite important when using shells.
> >
> > we should be able to specify multiple ENV vars, with one of the two syntaxes:
> >
> > <env-var>VAR=VALUE</env-var>
> >
> > or
> >
> > <environment>
> > <variable>
> > <name>VALUE</name>
> > <value>VALUE</value>
> > </variable>
> > </environment>
> >
>
> Mohammad Islam wrote:
> I considered that also an option. So user can specify it here and oozie will add it to oozie.launcher.mapred.child.env.
> I will add that.
> In addition, currently, User could still add the env directly through oozie.launcher.mapred.child.env.
>
> Alejandro Abdelnur wrote:
> I don't think these ENV settings should affect the launcher ENV, these ENV settings are exclusively for the SHELL it will be executed by the launcher.
>
> If the user wants to alter the launcher ENV for some reason, they always can do it with the oozie.launcher.mapred.child.env, but by definition, anything that is oozie.launcher.* is not propagated to the action itself.
>
> Mohammad Islam wrote:
> My use case is as follows:
> 1. <file>myscr.sh</file>
> 2. <exec>myscr.sh<exec>
>
> In this case, if Oozie LM executes the command, it will not find the executable "myscr.sh" because PATH variable doesn't contain CWD(.).
> However, if it is like <exec>./myscr.sh<exec>, it will work. That means user has to give the full path of the executable. How can a user configure this through PATH variable.
>
>
> Alejandro Abdelnur wrote:
> well adding current dir to launcher process path PATH would do. just that.
>
> but still i'd argue that unix users are use to do the ./SCRIPT thingy as current dir is commonly not in the PATH.
>
> Else, we could in the ShellMain to look at the script name and if does not start with './' prepend it
>
> Mohammad Islam wrote:
> It is not always current dir in the path. It could be <archive> that could expand to a different directory. So in that case, the exec will be <MY_TAR>/script.sh. So pre-appending might not work.
>
>
> Another thing, in addition to PATH, do we need to add LD_LIBRARY_PATH (for C++ exec), java.library.path(for java exec). I didn't test those use-cases.
>
> So I will add **only** PATH to child.env. Others, I will check.
>
> Otherwise all env should be pass through <environment> section.
> For implementation, what is the best way of passing these env/variables. Can think of the followings:
> Option 1: Use the same concept of args. For example args.0=var1, args.1=value1, etc...
> Option 2: Pass the whole environment section as value and parse it in the ShellMain. (Not a good approach)
>
>
>
> Alejandro Abdelnur wrote:
> if yo are using archive, the path is not the TAR name but the the directory in the expanded TAR, as the expansion happens in CWD, the './' should work. What am I missing here?
>
> For ENV passing, it should be one element per VAR, you don't want to get into parsing there.
My all scripts/.so are in one directory under archive (say ./MYARC_DIR/prod/bin/myscr.sh). User wants to use myscr.sh. Moreover, he wants to specify the path in PATH environment. as PATH=./MYARC_DIR/prod/bin/:$PATH. Pre-appending "./" might not work in this case.
For ENV passing, you prefer first option where we will pass the var key and value as we use the technique for arguments/parameter(for PIG) passing.
- Mohammad
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3000/#review3624
-----------------------------------------------------------
On 2011-12-04 08:05:24, Mohammad Islam wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/3000/
> -----------------------------------------------------------
>
> (Updated 2011-12-04 08:05:24)
>
>
> Review request for oozie.
>
>
> Summary
> -------
>
> More context at JIRA OOZIE-578.
>
>
> This addresses bug OOZIE-578.
> https://issues.apache.org/jira/browse/OOZIE-578
>
>
> Diffs
> -----
>
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/client/src/main/java/org/apache/oozie/cli/OozieCLI.java 1209346
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/client/src/main/resources/shell-action-0.1.xsd PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 1209346
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/resources/oozie-default.xml 1209346
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/test/java/org/apache/oozie/action/hadoop/ShellTestCase.java PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestShellActionExecutor.java PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestShellMain.java PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/docs/src/site/twiki/DG_ShellActionExtension.twiki PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/docs/src/site/twiki/index.twiki 1209346
>
> Diff: https://reviews.apache.org/r/3000/diff
>
>
> Testing
> -------
>
>
> Thanks,
>
> Mohammad
>
>
Re: Review Request: OOZIE-578 Support shell action in Oozie WF
Posted by Alejandro Abdelnur <tu...@cloudera.com>.
> On 2011-12-05 18:40:43, Alejandro Abdelnur wrote:
> > http://svn.apache.org/repos/asf/incubator/oozie/trunk/client/src/main/resources/shell-action-0.1.xsd, line 34
> > <https://reviews.apache.org/r/3000/diff/2/?file=61824#file61824line34>
> >
> > env vars setting is missing, this is quite important when using shells.
> >
> > we should be able to specify multiple ENV vars, with one of the two syntaxes:
> >
> > <env-var>VAR=VALUE</env-var>
> >
> > or
> >
> > <environment>
> > <variable>
> > <name>VALUE</name>
> > <value>VALUE</value>
> > </variable>
> > </environment>
> >
>
> Mohammad Islam wrote:
> I considered that also an option. So user can specify it here and oozie will add it to oozie.launcher.mapred.child.env.
> I will add that.
> In addition, currently, User could still add the env directly through oozie.launcher.mapred.child.env.
>
> Alejandro Abdelnur wrote:
> I don't think these ENV settings should affect the launcher ENV, these ENV settings are exclusively for the SHELL it will be executed by the launcher.
>
> If the user wants to alter the launcher ENV for some reason, they always can do it with the oozie.launcher.mapred.child.env, but by definition, anything that is oozie.launcher.* is not propagated to the action itself.
>
> Mohammad Islam wrote:
> My use case is as follows:
> 1. <file>myscr.sh</file>
> 2. <exec>myscr.sh<exec>
>
> In this case, if Oozie LM executes the command, it will not find the executable "myscr.sh" because PATH variable doesn't contain CWD(.).
> However, if it is like <exec>./myscr.sh<exec>, it will work. That means user has to give the full path of the executable. How can a user configure this through PATH variable.
>
well adding current dir to launcher process path PATH would do. just that.
but still i'd argue that unix users are use to do the ./SCRIPT thingy as current dir is commonly not in the PATH.
Else, we could in the ShellMain to look at the script name and if does not start with './' prepend it
> On 2011-12-05 18:40:43, Alejandro Abdelnur wrote:
> > http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java, line 106
> > <https://reviews.apache.org/r/3000/diff/2/?file=61826#file61826line106>
> >
> > as the shellmain is forking a new process, these changes are not required for the launcher mapper process, but for the process the shellmain launches.
>
> Mohammad Islam wrote:
> I think we need it here. I spend a lot of time on this.
> For example, my script is in CWD of LM. If I don't set it up in *.child.env. It will never get the script if I do it RunTime.exec(args, envp).
> Other environment passed in RunTime.exec(args, envp), doesn't have any impact on finding the executables own path. However, it impacts on within the spawned script itself.
> That's why I do it in *.child.env that will be available to LM to find the shell executable itself as well as to the spawned script.
>
>
> Alejandro Abdelnur wrote:
> If using the ProcessBuilder class, you can easily set the workding directory with the directory(File) method.
>
> Mohammad Islam wrote:
> Does it add the CWD in the classpath?
ProcessBuilder is to launch Shells, nothing specific to classpath handling
> On 2011-12-05 18:40:43, Alejandro Abdelnur wrote:
> > http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java, line 29
> > <https://reviews.apache.org/r/3000/diff/2/?file=61827#file61827line29>
> >
> > trailing spaces (through out the patch)
>
> Mohammad Islam wrote:
> I never add it. The oozie-foramtting.xml did it for me!! This is there for long time.
> Do you have any new oozie-formatting xml?
>
>
> Alejandro Abdelnur wrote:
> Don't use Eclipse, use IntelliJ
sorry, I meant I DON'T USE ECLIPSE...
> On 2011-12-05 18:40:43, Alejandro Abdelnur wrote:
> > http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java, line 132
> > <https://reviews.apache.org/r/3000/diff/2/?file=61827#file61827line132>
> >
> > consuming the stdout/stderr of the forked process must be done in different threads, else you may be be blocking processing/overflowing buffers.
> >
> >
>
> Mohammad Islam wrote:
> Are you asking to spawn a new Thread to do it?
>
> Alejandro Abdelnur wrote:
> actually, 2 new threads, one to consume STDOUT and other STDERR, they should be daemon threads and exit the loop on -1 read as well.
>
> both outputs should be redirected to to the launcher process STDOUT/STDERR (as then they end up in the launcher logs).
>
> the STDOUT should also be captured (a la tee) so it can be used for <capture-output> functionality.
>
> Mohammad Islam wrote:
> Not sure about the capture-output. If user writes a lot of output for its own debugging and we consider that as capture-output, it might not work or go out-of-max-size error very often.
> What we could do, by default, capture-output will be false.
> If a user needs capture output, we should write it to the capture-output file.
>
we should preserve the same functionality we have in SSH and Java action, if <capture-output/> is set, the STDOUT of the action is assumed to be in Properties file format and it becomes the action outputdata.
Any debugging the user wants to do it should go to STDERR.
This should be clearly documented.
- Alejandro
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3000/#review3624
-----------------------------------------------------------
On 2011-12-04 08:05:24, Mohammad Islam wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/3000/
> -----------------------------------------------------------
>
> (Updated 2011-12-04 08:05:24)
>
>
> Review request for oozie.
>
>
> Summary
> -------
>
> More context at JIRA OOZIE-578.
>
>
> This addresses bug OOZIE-578.
> https://issues.apache.org/jira/browse/OOZIE-578
>
>
> Diffs
> -----
>
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/client/src/main/java/org/apache/oozie/cli/OozieCLI.java 1209346
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/client/src/main/resources/shell-action-0.1.xsd PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 1209346
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/resources/oozie-default.xml 1209346
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/test/java/org/apache/oozie/action/hadoop/ShellTestCase.java PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestShellActionExecutor.java PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestShellMain.java PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/docs/src/site/twiki/DG_ShellActionExtension.twiki PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/docs/src/site/twiki/index.twiki 1209346
>
> Diff: https://reviews.apache.org/r/3000/diff
>
>
> Testing
> -------
>
>
> Thanks,
>
> Mohammad
>
>
Re: Review Request: OOZIE-578 Support shell action in Oozie WF
Posted by Mohammad Islam <mi...@yahoo.com>.
> On 2011-12-05 18:40:43, Alejandro Abdelnur wrote:
> > http://svn.apache.org/repos/asf/incubator/oozie/trunk/client/src/main/resources/shell-action-0.1.xsd, line 34
> > <https://reviews.apache.org/r/3000/diff/2/?file=61824#file61824line34>
> >
> > env vars setting is missing, this is quite important when using shells.
> >
> > we should be able to specify multiple ENV vars, with one of the two syntaxes:
> >
> > <env-var>VAR=VALUE</env-var>
> >
> > or
> >
> > <environment>
> > <variable>
> > <name>VALUE</name>
> > <value>VALUE</value>
> > </variable>
> > </environment>
> >
>
> Mohammad Islam wrote:
> I considered that also an option. So user can specify it here and oozie will add it to oozie.launcher.mapred.child.env.
> I will add that.
> In addition, currently, User could still add the env directly through oozie.launcher.mapred.child.env.
>
> Alejandro Abdelnur wrote:
> I don't think these ENV settings should affect the launcher ENV, these ENV settings are exclusively for the SHELL it will be executed by the launcher.
>
> If the user wants to alter the launcher ENV for some reason, they always can do it with the oozie.launcher.mapred.child.env, but by definition, anything that is oozie.launcher.* is not propagated to the action itself.
My use case is as follows:
1. <file>myscr.sh</file>
2. <exec>myscr.sh<exec>
In this case, if Oozie LM executes the command, it will not find the executable "myscr.sh" because PATH variable doesn't contain CWD(.).
However, if it is like <exec>./myscr.sh<exec>, it will work. That means user has to give the full path of the executable. How can a user configure this through PATH variable.
> On 2011-12-05 18:40:43, Alejandro Abdelnur wrote:
> > http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java, line 106
> > <https://reviews.apache.org/r/3000/diff/2/?file=61826#file61826line106>
> >
> > as the shellmain is forking a new process, these changes are not required for the launcher mapper process, but for the process the shellmain launches.
>
> Mohammad Islam wrote:
> I think we need it here. I spend a lot of time on this.
> For example, my script is in CWD of LM. If I don't set it up in *.child.env. It will never get the script if I do it RunTime.exec(args, envp).
> Other environment passed in RunTime.exec(args, envp), doesn't have any impact on finding the executables own path. However, it impacts on within the spawned script itself.
> That's why I do it in *.child.env that will be available to LM to find the shell executable itself as well as to the spawned script.
>
>
> Alejandro Abdelnur wrote:
> If using the ProcessBuilder class, you can easily set the workding directory with the directory(File) method.
Does it add the CWD in the classpath?
> On 2011-12-05 18:40:43, Alejandro Abdelnur wrote:
> > http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java, line 132
> > <https://reviews.apache.org/r/3000/diff/2/?file=61827#file61827line132>
> >
> > consuming the stdout/stderr of the forked process must be done in different threads, else you may be be blocking processing/overflowing buffers.
> >
> >
>
> Mohammad Islam wrote:
> Are you asking to spawn a new Thread to do it?
>
> Alejandro Abdelnur wrote:
> actually, 2 new threads, one to consume STDOUT and other STDERR, they should be daemon threads and exit the loop on -1 read as well.
>
> both outputs should be redirected to to the launcher process STDOUT/STDERR (as then they end up in the launcher logs).
>
> the STDOUT should also be captured (a la tee) so it can be used for <capture-output> functionality.
Not sure about the capture-output. If user writes a lot of output for its own debugging and we consider that as capture-output, it might not work or go out-of-max-size error very often.
What we could do, by default, capture-output will be false.
If a user needs capture output, we should write it to the capture-output file.
- Mohammad
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3000/#review3624
-----------------------------------------------------------
On 2011-12-04 08:05:24, Mohammad Islam wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/3000/
> -----------------------------------------------------------
>
> (Updated 2011-12-04 08:05:24)
>
>
> Review request for oozie.
>
>
> Summary
> -------
>
> More context at JIRA OOZIE-578.
>
>
> This addresses bug OOZIE-578.
> https://issues.apache.org/jira/browse/OOZIE-578
>
>
> Diffs
> -----
>
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/client/src/main/java/org/apache/oozie/cli/OozieCLI.java 1209346
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/client/src/main/resources/shell-action-0.1.xsd PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 1209346
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/resources/oozie-default.xml 1209346
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/test/java/org/apache/oozie/action/hadoop/ShellTestCase.java PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestShellActionExecutor.java PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestShellMain.java PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/docs/src/site/twiki/DG_ShellActionExtension.twiki PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/docs/src/site/twiki/index.twiki 1209346
>
> Diff: https://reviews.apache.org/r/3000/diff
>
>
> Testing
> -------
>
>
> Thanks,
>
> Mohammad
>
>
Re: Review Request: OOZIE-578 Support shell action in Oozie WF
Posted by Mohammad Islam <mi...@yahoo.com>.
> On 2011-12-05 18:40:43, Alejandro Abdelnur wrote:
> > http://svn.apache.org/repos/asf/incubator/oozie/trunk/client/src/main/resources/shell-action-0.1.xsd, line 34
> > <https://reviews.apache.org/r/3000/diff/2/?file=61824#file61824line34>
> >
> > env vars setting is missing, this is quite important when using shells.
> >
> > we should be able to specify multiple ENV vars, with one of the two syntaxes:
> >
> > <env-var>VAR=VALUE</env-var>
> >
> > or
> >
> > <environment>
> > <variable>
> > <name>VALUE</name>
> > <value>VALUE</value>
> > </variable>
> > </environment>
> >
>
> Mohammad Islam wrote:
> I considered that also an option. So user can specify it here and oozie will add it to oozie.launcher.mapred.child.env.
> I will add that.
> In addition, currently, User could still add the env directly through oozie.launcher.mapred.child.env.
>
> Alejandro Abdelnur wrote:
> I don't think these ENV settings should affect the launcher ENV, these ENV settings are exclusively for the SHELL it will be executed by the launcher.
>
> If the user wants to alter the launcher ENV for some reason, they always can do it with the oozie.launcher.mapred.child.env, but by definition, anything that is oozie.launcher.* is not propagated to the action itself.
>
> Mohammad Islam wrote:
> My use case is as follows:
> 1. <file>myscr.sh</file>
> 2. <exec>myscr.sh<exec>
>
> In this case, if Oozie LM executes the command, it will not find the executable "myscr.sh" because PATH variable doesn't contain CWD(.).
> However, if it is like <exec>./myscr.sh<exec>, it will work. That means user has to give the full path of the executable. How can a user configure this through PATH variable.
>
>
> Alejandro Abdelnur wrote:
> well adding current dir to launcher process path PATH would do. just that.
>
> but still i'd argue that unix users are use to do the ./SCRIPT thingy as current dir is commonly not in the PATH.
>
> Else, we could in the ShellMain to look at the script name and if does not start with './' prepend it
It is not always current dir in the path. It could be <archive> that could expand to a different directory. So in that case, the exec will be <MY_TAR>/script.sh. So pre-appending might not work.
Another thing, in addition to PATH, do we need to add LD_LIBRARY_PATH (for C++ exec), java.library.path(for java exec). I didn't test those use-cases.
So I will add **only** PATH to child.env. Others, I will check.
Otherwise all env should be pass through <environment> section.
For implementation, what is the best way of passing these env/variables. Can think of the followings:
Option 1: Use the same concept of args. For example args.0=var1, args.1=value1, etc...
Option 2: Pass the whole environment section as value and parse it in the ShellMain. (Not a good approach)
- Mohammad
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3000/#review3624
-----------------------------------------------------------
On 2011-12-04 08:05:24, Mohammad Islam wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/3000/
> -----------------------------------------------------------
>
> (Updated 2011-12-04 08:05:24)
>
>
> Review request for oozie.
>
>
> Summary
> -------
>
> More context at JIRA OOZIE-578.
>
>
> This addresses bug OOZIE-578.
> https://issues.apache.org/jira/browse/OOZIE-578
>
>
> Diffs
> -----
>
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/client/src/main/java/org/apache/oozie/cli/OozieCLI.java 1209346
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/client/src/main/resources/shell-action-0.1.xsd PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 1209346
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/resources/oozie-default.xml 1209346
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/test/java/org/apache/oozie/action/hadoop/ShellTestCase.java PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestShellActionExecutor.java PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestShellMain.java PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/docs/src/site/twiki/DG_ShellActionExtension.twiki PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/docs/src/site/twiki/index.twiki 1209346
>
> Diff: https://reviews.apache.org/r/3000/diff
>
>
> Testing
> -------
>
>
> Thanks,
>
> Mohammad
>
>
Re: Review Request: OOZIE-578 Support shell action in Oozie WF
Posted by Alejandro Abdelnur <tu...@cloudera.com>.
> On 2011-12-05 18:40:43, Alejandro Abdelnur wrote:
> > http://svn.apache.org/repos/asf/incubator/oozie/trunk/client/src/main/resources/shell-action-0.1.xsd, line 34
> > <https://reviews.apache.org/r/3000/diff/2/?file=61824#file61824line34>
> >
> > env vars setting is missing, this is quite important when using shells.
> >
> > we should be able to specify multiple ENV vars, with one of the two syntaxes:
> >
> > <env-var>VAR=VALUE</env-var>
> >
> > or
> >
> > <environment>
> > <variable>
> > <name>VALUE</name>
> > <value>VALUE</value>
> > </variable>
> > </environment>
> >
>
> Mohammad Islam wrote:
> I considered that also an option. So user can specify it here and oozie will add it to oozie.launcher.mapred.child.env.
> I will add that.
> In addition, currently, User could still add the env directly through oozie.launcher.mapred.child.env.
I don't think these ENV settings should affect the launcher ENV, these ENV settings are exclusively for the SHELL it will be executed by the launcher.
If the user wants to alter the launcher ENV for some reason, they always can do it with the oozie.launcher.mapred.child.env, but by definition, anything that is oozie.launcher.* is not propagated to the action itself.
> On 2011-12-05 18:40:43, Alejandro Abdelnur wrote:
> > http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java, line 106
> > <https://reviews.apache.org/r/3000/diff/2/?file=61826#file61826line106>
> >
> > as the shellmain is forking a new process, these changes are not required for the launcher mapper process, but for the process the shellmain launches.
>
> Mohammad Islam wrote:
> I think we need it here. I spend a lot of time on this.
> For example, my script is in CWD of LM. If I don't set it up in *.child.env. It will never get the script if I do it RunTime.exec(args, envp).
> Other environment passed in RunTime.exec(args, envp), doesn't have any impact on finding the executables own path. However, it impacts on within the spawned script itself.
> That's why I do it in *.child.env that will be available to LM to find the shell executable itself as well as to the spawned script.
>
If using the ProcessBuilder class, you can easily set the workding directory with the directory(File) method.
> On 2011-12-05 18:40:43, Alejandro Abdelnur wrote:
> > http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java, line 29
> > <https://reviews.apache.org/r/3000/diff/2/?file=61827#file61827line29>
> >
> > trailing spaces (through out the patch)
>
> Mohammad Islam wrote:
> I never add it. The oozie-foramtting.xml did it for me!! This is there for long time.
> Do you have any new oozie-formatting xml?
>
Don't use Eclipse, use IntelliJ
> On 2011-12-05 18:40:43, Alejandro Abdelnur wrote:
> > http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java, line 74
> > <https://reviews.apache.org/r/3000/diff/2/?file=61827#file61827line74>
> >
> > We should use ProcessBuilder instead Runtime.
> >
> > ProcessBuilder allows to easily set the environment.
> >
> >
>
> Mohammad Islam wrote:
> I will check the ProcessBuilder. I thought both are same.
ProcessBuilder has some functionality that makes easier to deal with subprocesses, see prev comment on setting directory.
> On 2011-12-05 18:40:43, Alejandro Abdelnur wrote:
> > http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java, line 132
> > <https://reviews.apache.org/r/3000/diff/2/?file=61827#file61827line132>
> >
> > consuming the stdout/stderr of the forked process must be done in different threads, else you may be be blocking processing/overflowing buffers.
> >
> >
>
> Mohammad Islam wrote:
> Are you asking to spawn a new Thread to do it?
actually, 2 new threads, one to consume STDOUT and other STDERR, they should be daemon threads and exit the loop on -1 read as well.
both outputs should be redirected to to the launcher process STDOUT/STDERR (as then they end up in the launcher logs).
the STDOUT should also be captured (a la tee) so it can be used for <capture-output> functionality.
> On 2011-12-05 18:40:43, Alejandro Abdelnur wrote:
> > http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/resources/oozie-default.xml, line 1052
> > <https://reviews.apache.org/r/3000/diff/2/?file=61828#file61828line1052>
> >
> > this should go in the wf.schemas property, no there.
>
> Mohammad Islam wrote:
> ok.
by bad, this is not there. We should open a JIRA for that.
- Alejandro
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3000/#review3624
-----------------------------------------------------------
On 2011-12-04 08:05:24, Mohammad Islam wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/3000/
> -----------------------------------------------------------
>
> (Updated 2011-12-04 08:05:24)
>
>
> Review request for oozie.
>
>
> Summary
> -------
>
> More context at JIRA OOZIE-578.
>
>
> This addresses bug OOZIE-578.
> https://issues.apache.org/jira/browse/OOZIE-578
>
>
> Diffs
> -----
>
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/client/src/main/java/org/apache/oozie/cli/OozieCLI.java 1209346
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/client/src/main/resources/shell-action-0.1.xsd PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 1209346
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/resources/oozie-default.xml 1209346
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/test/java/org/apache/oozie/action/hadoop/ShellTestCase.java PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestShellActionExecutor.java PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestShellMain.java PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/docs/src/site/twiki/DG_ShellActionExtension.twiki PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/docs/src/site/twiki/index.twiki 1209346
>
> Diff: https://reviews.apache.org/r/3000/diff
>
>
> Testing
> -------
>
>
> Thanks,
>
> Mohammad
>
>
Re: Review Request: OOZIE-578 Support shell action in Oozie WF
Posted by Mohammad Islam <mi...@yahoo.com>.
> On 2011-12-05 18:40:43, Alejandro Abdelnur wrote:
> > http://svn.apache.org/repos/asf/incubator/oozie/trunk/client/src/main/resources/shell-action-0.1.xsd, line 33
> > <https://reviews.apache.org/r/3000/diff/2/?file=61824#file61824line33>
> >
> > main-class is overridable via the oozie.launcher.main.class, no need for an element in the schema for it.
ok
> On 2011-12-05 18:40:43, Alejandro Abdelnur wrote:
> > http://svn.apache.org/repos/asf/incubator/oozie/trunk/client/src/main/resources/shell-action-0.1.xsd, line 34
> > <https://reviews.apache.org/r/3000/diff/2/?file=61824#file61824line34>
> >
> > env vars setting is missing, this is quite important when using shells.
> >
> > we should be able to specify multiple ENV vars, with one of the two syntaxes:
> >
> > <env-var>VAR=VALUE</env-var>
> >
> > or
> >
> > <environment>
> > <variable>
> > <name>VALUE</name>
> > <value>VALUE</value>
> > </variable>
> > </environment>
> >
I considered that also an option. So user can specify it here and oozie will add it to oozie.launcher.mapred.child.env.
I will add that.
In addition, currently, User could still add the env directly through oozie.launcher.mapred.child.env.
> On 2011-12-05 18:40:43, Alejandro Abdelnur wrote:
> > http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java, line 1
> > <https://reviews.apache.org/r/3000/diff/2/?file=61826#file61826line1>
> >
> > missing license header
ok.
> On 2011-12-05 18:40:43, Alejandro Abdelnur wrote:
> > http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java, line 22
> > <https://reviews.apache.org/r/3000/diff/2/?file=61826#file61826line22>
> >
> > remove automatically generated javadocs, they are noise. (this applies to all such javadocs in this patch)
ok.
> On 2011-12-05 18:40:43, Alejandro Abdelnur wrote:
> > http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java, line 106
> > <https://reviews.apache.org/r/3000/diff/2/?file=61826#file61826line106>
> >
> > as the shellmain is forking a new process, these changes are not required for the launcher mapper process, but for the process the shellmain launches.
I think we need it here. I spend a lot of time on this.
For example, my script is in CWD of LM. If I don't set it up in *.child.env. It will never get the script if I do it RunTime.exec(args, envp).
Other environment passed in RunTime.exec(args, envp), doesn't have any impact on finding the executables own path. However, it impacts on within the spawned script itself.
That's why I do it in *.child.env that will be available to LM to find the shell executable itself as well as to the spawned script.
> On 2011-12-05 18:40:43, Alejandro Abdelnur wrote:
> > http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java, line 29
> > <https://reviews.apache.org/r/3000/diff/2/?file=61827#file61827line29>
> >
> > trailing spaces (through out the patch)
I never add it. The oozie-foramtting.xml did it for me!! This is there for long time.
Do you have any new oozie-formatting xml?
> On 2011-12-05 18:40:43, Alejandro Abdelnur wrote:
> > http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java, line 74
> > <https://reviews.apache.org/r/3000/diff/2/?file=61827#file61827line74>
> >
> > We should use ProcessBuilder instead Runtime.
> >
> > ProcessBuilder allows to easily set the environment.
> >
> >
I will check the ProcessBuilder. I thought both are same.
> On 2011-12-05 18:40:43, Alejandro Abdelnur wrote:
> > http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java, line 132
> > <https://reviews.apache.org/r/3000/diff/2/?file=61827#file61827line132>
> >
> > consuming the stdout/stderr of the forked process must be done in different threads, else you may be be blocking processing/overflowing buffers.
> >
> >
Are you asking to spawn a new Thread to do it?
> On 2011-12-05 18:40:43, Alejandro Abdelnur wrote:
> > http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/resources/oozie-default.xml, line 1052
> > <https://reviews.apache.org/r/3000/diff/2/?file=61828#file61828line1052>
> >
> > this should go in the wf.schemas property, no there.
ok.
- Mohammad
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3000/#review3624
-----------------------------------------------------------
On 2011-12-04 08:05:24, Mohammad Islam wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/3000/
> -----------------------------------------------------------
>
> (Updated 2011-12-04 08:05:24)
>
>
> Review request for oozie.
>
>
> Summary
> -------
>
> More context at JIRA OOZIE-578.
>
>
> This addresses bug OOZIE-578.
> https://issues.apache.org/jira/browse/OOZIE-578
>
>
> Diffs
> -----
>
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/client/src/main/java/org/apache/oozie/cli/OozieCLI.java 1209346
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/client/src/main/resources/shell-action-0.1.xsd PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 1209346
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/resources/oozie-default.xml 1209346
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/test/java/org/apache/oozie/action/hadoop/ShellTestCase.java PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestShellActionExecutor.java PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestShellMain.java PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/docs/src/site/twiki/DG_ShellActionExtension.twiki PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/docs/src/site/twiki/index.twiki 1209346
>
> Diff: https://reviews.apache.org/r/3000/diff
>
>
> Testing
> -------
>
>
> Thanks,
>
> Mohammad
>
>
Re: Review Request: OOZIE-578 Support shell action in Oozie WF
Posted by Alejandro Abdelnur <tu...@cloudera.com>.
> On 2011-12-05 18:40:43, Alejandro Abdelnur wrote:
> > http://svn.apache.org/repos/asf/incubator/oozie/trunk/client/src/main/resources/shell-action-0.1.xsd, line 34
> > <https://reviews.apache.org/r/3000/diff/2/?file=61824#file61824line34>
> >
> > env vars setting is missing, this is quite important when using shells.
> >
> > we should be able to specify multiple ENV vars, with one of the two syntaxes:
> >
> > <env-var>VAR=VALUE</env-var>
> >
> > or
> >
> > <environment>
> > <variable>
> > <name>VALUE</name>
> > <value>VALUE</value>
> > </variable>
> > </environment>
> >
>
> Mohammad Islam wrote:
> I considered that also an option. So user can specify it here and oozie will add it to oozie.launcher.mapred.child.env.
> I will add that.
> In addition, currently, User could still add the env directly through oozie.launcher.mapred.child.env.
>
> Alejandro Abdelnur wrote:
> I don't think these ENV settings should affect the launcher ENV, these ENV settings are exclusively for the SHELL it will be executed by the launcher.
>
> If the user wants to alter the launcher ENV for some reason, they always can do it with the oozie.launcher.mapred.child.env, but by definition, anything that is oozie.launcher.* is not propagated to the action itself.
>
> Mohammad Islam wrote:
> My use case is as follows:
> 1. <file>myscr.sh</file>
> 2. <exec>myscr.sh<exec>
>
> In this case, if Oozie LM executes the command, it will not find the executable "myscr.sh" because PATH variable doesn't contain CWD(.).
> However, if it is like <exec>./myscr.sh<exec>, it will work. That means user has to give the full path of the executable. How can a user configure this through PATH variable.
>
>
> Alejandro Abdelnur wrote:
> well adding current dir to launcher process path PATH would do. just that.
>
> but still i'd argue that unix users are use to do the ./SCRIPT thingy as current dir is commonly not in the PATH.
>
> Else, we could in the ShellMain to look at the script name and if does not start with './' prepend it
>
> Mohammad Islam wrote:
> It is not always current dir in the path. It could be <archive> that could expand to a different directory. So in that case, the exec will be <MY_TAR>/script.sh. So pre-appending might not work.
>
>
> Another thing, in addition to PATH, do we need to add LD_LIBRARY_PATH (for C++ exec), java.library.path(for java exec). I didn't test those use-cases.
>
> So I will add **only** PATH to child.env. Others, I will check.
>
> Otherwise all env should be pass through <environment> section.
> For implementation, what is the best way of passing these env/variables. Can think of the followings:
> Option 1: Use the same concept of args. For example args.0=var1, args.1=value1, etc...
> Option 2: Pass the whole environment section as value and parse it in the ShellMain. (Not a good approach)
>
>
if yo are using archive, the path is not the TAR name but the the directory in the expanded TAR, as the expansion happens in CWD, the './' should work. What am I missing here?
For ENV passing, it should be one element per VAR, you don't want to get into parsing there.
- Alejandro
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3000/#review3624
-----------------------------------------------------------
On 2011-12-04 08:05:24, Mohammad Islam wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/3000/
> -----------------------------------------------------------
>
> (Updated 2011-12-04 08:05:24)
>
>
> Review request for oozie.
>
>
> Summary
> -------
>
> More context at JIRA OOZIE-578.
>
>
> This addresses bug OOZIE-578.
> https://issues.apache.org/jira/browse/OOZIE-578
>
>
> Diffs
> -----
>
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/client/src/main/java/org/apache/oozie/cli/OozieCLI.java 1209346
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/client/src/main/resources/shell-action-0.1.xsd PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 1209346
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/resources/oozie-default.xml 1209346
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/test/java/org/apache/oozie/action/hadoop/ShellTestCase.java PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestShellActionExecutor.java PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestShellMain.java PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/docs/src/site/twiki/DG_ShellActionExtension.twiki PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/docs/src/site/twiki/index.twiki 1209346
>
> Diff: https://reviews.apache.org/r/3000/diff
>
>
> Testing
> -------
>
>
> Thanks,
>
> Mohammad
>
>
Re: Review Request: OOZIE-578 Support shell action in Oozie WF
Posted by Alejandro Abdelnur <tu...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3000/#review3624
-----------------------------------------------------------
http://svn.apache.org/repos/asf/incubator/oozie/trunk/client/src/main/resources/shell-action-0.1.xsd
<https://reviews.apache.org/r/3000/#comment8080>
main-class is overridable via the oozie.launcher.main.class, no need for an element in the schema for it.
http://svn.apache.org/repos/asf/incubator/oozie/trunk/client/src/main/resources/shell-action-0.1.xsd
<https://reviews.apache.org/r/3000/#comment8083>
env vars setting is missing, this is quite important when using shells.
we should be able to specify multiple ENV vars, with one of the two syntaxes:
<env-var>VAR=VALUE</env-var>
or
<environment>
<variable>
<name>VALUE</name>
<value>VALUE</value>
</variable>
</environment>
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java
<https://reviews.apache.org/r/3000/#comment8084>
missing license header
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java
<https://reviews.apache.org/r/3000/#comment8085>
remove automatically generated javadocs, they are noise. (this applies to all such javadocs in this patch)
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java
<https://reviews.apache.org/r/3000/#comment8087>
as the shellmain is forking a new process, these changes are not required for the launcher mapper process, but for the process the shellmain launches.
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java
<https://reviews.apache.org/r/3000/#comment8086>
trailing spaces (through out the patch)
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java
<https://reviews.apache.org/r/3000/#comment8088>
We should use ProcessBuilder instead Runtime.
ProcessBuilder allows to easily set the environment.
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java
<https://reviews.apache.org/r/3000/#comment8089>
consuming the stdout/stderr of the forked process must be done in different threads, else you may be be blocking processing/overflowing buffers.
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/resources/oozie-default.xml
<https://reviews.apache.org/r/3000/#comment8090>
this should go in the wf.schemas property, no there.
- Alejandro
On 2011-12-04 08:05:24, Mohammad Islam wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/3000/
> -----------------------------------------------------------
>
> (Updated 2011-12-04 08:05:24)
>
>
> Review request for oozie.
>
>
> Summary
> -------
>
> More context at JIRA OOZIE-578.
>
>
> This addresses bug OOZIE-578.
> https://issues.apache.org/jira/browse/OOZIE-578
>
>
> Diffs
> -----
>
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/client/src/main/java/org/apache/oozie/cli/OozieCLI.java 1209346
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/client/src/main/resources/shell-action-0.1.xsd PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 1209346
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/resources/oozie-default.xml 1209346
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/test/java/org/apache/oozie/action/hadoop/ShellTestCase.java PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestShellActionExecutor.java PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestShellMain.java PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/docs/src/site/twiki/DG_ShellActionExtension.twiki PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/docs/src/site/twiki/index.twiki 1209346
>
> Diff: https://reviews.apache.org/r/3000/diff
>
>
> Testing
> -------
>
>
> Thanks,
>
> Mohammad
>
>
Re: Review Request: OOZIE-578 Support shell action in Oozie WF
Posted by Alejandro Abdelnur <tu...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3000/#review3693
-----------------------------------------------------------
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java
<https://reviews.apache.org/r/3000/#comment8223>
the argument and env-var lists creation logic could be done in a private method and reused for both.
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java
<https://reviews.apache.org/r/3000/#comment8224>
these lines would be simpler by doing:
actionConf.setBoolean(ShellMain.CONF_OOZIE_SHELL_CAPTURE_OUTPUT,
actionXml.getChild("capture-output", ns) != null);
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java
<https://reviews.apache.org/r/3000/#comment8225>
why do we have to add the child env to the launcher, it only should be added to the env of the shell we'll start from the ShellMain, no?
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java
<https://reviews.apache.org/r/3000/#comment8226>
these comments seem incorrect
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java
<https://reviews.apache.org/r/3000/#comment8227>
the exec and argumens should be moved to the execute() method, then a single param is required to it, the actionConf, and it makes things more consistent (all actionConf handling happens in one place)
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java
<https://reviews.apache.org/r/3000/#comment8228>
remove commented code
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java
<https://reviews.apache.org/r/3000/#comment8229>
Here would be where the CWD is added to the PATH, instead doing in actionexecutor for the launcher
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java
<https://reviews.apache.org/r/3000/#comment8230>
echoShellOutput should be named 'handleShellOutput', and it could return a Thread[], then the threads vars for handling stdout and stderr are local vars instead of instance.
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java
<https://reviews.apache.org/r/3000/#comment8231>
no need for checking for != null, they never are
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java
<https://reviews.apache.org/r/3000/#comment8232>
tab/trailing-space
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java
<https://reviews.apache.org/r/3000/#comment8233>
this method does not seem to be used.
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java
<https://reviews.apache.org/r/3000/#comment8235>
why special handling for HADOOP_TOKEN_FILE_LOCATION, it will be inherited by the child process from the current ENV
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java
<https://reviews.apache.org/r/3000/#comment8234>
is this an expected ENV VAR? by whom? using DOTs in it seems dod.
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java
<https://reviews.apache.org/r/3000/#comment8236>
remove commented code
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java
<https://reviews.apache.org/r/3000/#comment8237>
the shellactionexecutor should check that all env-var values have an '=', and fail if not, then here is always expected an '=' and no need for an if else
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java
<https://reviews.apache.org/r/3000/#comment8238>
this method does not seem used.
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java
<https://reviews.apache.org/r/3000/#comment8239>
this method does not seem used
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java
<https://reviews.apache.org/r/3000/#comment8240>
needCapture is not being set to the local var
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java
<https://reviews.apache.org/r/3000/#comment8241>
this is not correct, the file for capture output should be opened outside of the while loop that reads the STDOUT (here is being opened on every line)
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java
<https://reviews.apache.org/r/3000/#comment8242>
this method is not being used, remove.
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/resources/oozie-default.xml
<https://reviews.apache.org/r/3000/#comment8243>
as with the hive, sqoop, and discp extensions, this should be defined in the default oozie-site.xml. otherwise it will be in indadvertedly overridden.
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/resources/oozie-default.xml
<https://reviews.apache.org/r/3000/#comment8244>
this is an action extension, it must go in the ext property in the default oozie-site.xml (commented out together with sqoop, hive, distcp)
- Alejandro
On 2011-12-07 00:51:10, Mohammad Islam wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/3000/
> -----------------------------------------------------------
>
> (Updated 2011-12-07 00:51:10)
>
>
> Review request for oozie.
>
>
> Summary
> -------
>
> More context at JIRA OOZIE-578.
>
>
> This addresses bug OOZIE-578.
> https://issues.apache.org/jira/browse/OOZIE-578
>
>
> Diffs
> -----
>
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/docs/src/site/twiki/index.twiki 1209346
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/docs/src/site/twiki/DG_ShellActionExtension.twiki PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestShellMain.java PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/resources/oozie-default.xml 1209346
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/test/java/org/apache/oozie/action/hadoop/ShellTestCase.java PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestShellActionExecutor.java PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/client/src/main/java/org/apache/oozie/cli/OozieCLI.java 1209346
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/client/src/main/resources/shell-action-0.1.xsd PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 1209346
>
> Diff: https://reviews.apache.org/r/3000/diff
>
>
> Testing
> -------
>
>
> Thanks,
>
> Mohammad
>
>
Re: Review Request: OOZIE-578 Support shell action in Oozie WF
Posted by Mohammad Islam <mi...@yahoo.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3000/#review3694
-----------------------------------------------------------
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java
<https://reviews.apache.org/r/3000/#comment8245>
good idea. will do that.
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java
<https://reviews.apache.org/r/3000/#comment8246>
ok
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java
<https://reviews.apache.org/r/3000/#comment8265>
This is very tricky thing.
First the executable path is determined by the environment of the launching process.
The environment that we pass to exec command is passed as an environment of the spawned process. Not used to find the process itself. LM's environment's path is used to the executable path.
I tested it.
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java
<https://reviews.apache.org/r/3000/#comment8247>
will remove.
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java
<https://reviews.apache.org/r/3000/#comment8248>
ok
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java
<https://reviews.apache.org/r/3000/#comment8249>
will remove all commented code
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java
<https://reviews.apache.org/r/3000/#comment8251>
ok
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java
<https://reviews.apache.org/r/3000/#comment8252>
if it is returned from the method, then no need to test.
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java
<https://reviews.apache.org/r/3000/#comment8250>
ok
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java
<https://reviews.apache.org/r/3000/#comment8253>
will remove the duplicate methods
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java
<https://reviews.apache.org/r/3000/#comment8254>
ok
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java
<https://reviews.apache.org/r/3000/#comment8256>
so parsing will be done twice.
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java
<https://reviews.apache.org/r/3000/#comment8257>
This one is used in line 81
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java
<https://reviews.apache.org/r/3000/#comment8258>
good catch
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java
<https://reviews.apache.org/r/3000/#comment8259>
correct
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/resources/oozie-default.xml
<https://reviews.apache.org/r/3000/#comment8260>
I want shell , email action to be top level by default.
The user will need to configure so many things, if not using our oozie-site.xml
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/resources/oozie-default.xml
<https://reviews.apache.org/r/3000/#comment8261>
The same is true here. So many extra configurations those are candidate for default settings. We need to change our default stuffs.
- Mohammad
On 2011-12-07 00:51:10, Mohammad Islam wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/3000/
> -----------------------------------------------------------
>
> (Updated 2011-12-07 00:51:10)
>
>
> Review request for oozie.
>
>
> Summary
> -------
>
> More context at JIRA OOZIE-578.
>
>
> This addresses bug OOZIE-578.
> https://issues.apache.org/jira/browse/OOZIE-578
>
>
> Diffs
> -----
>
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/docs/src/site/twiki/index.twiki 1209346
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/docs/src/site/twiki/DG_ShellActionExtension.twiki PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestShellMain.java PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/resources/oozie-default.xml 1209346
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/test/java/org/apache/oozie/action/hadoop/ShellTestCase.java PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestShellActionExecutor.java PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/client/src/main/java/org/apache/oozie/cli/OozieCLI.java 1209346
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/client/src/main/resources/shell-action-0.1.xsd PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 1209346
>
> Diff: https://reviews.apache.org/r/3000/diff
>
>
> Testing
> -------
>
>
> Thanks,
>
> Mohammad
>
>
Re: Review Request: OOZIE-578 Support shell action in Oozie WF
Posted by Mohammad Islam <mi...@yahoo.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3000/#review3707
-----------------------------------------------------------
http://svn.apache.org/repos/asf/incubator/oozie/trunk/client/src/main/resources/shell-action-0.1.xsd
<https://reviews.apache.org/r/3000/#comment8290>
ok.
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java
<https://reviews.apache.org/r/3000/#comment8291>
Want to remove these 3 files.
The same thing is already being done in Launcher Mapper
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java
<https://reviews.apache.org/r/3000/#comment8292>
ok
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java
<https://reviews.apache.org/r/3000/#comment8293>
Good catch.
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java
<https://reviews.apache.org/r/3000/#comment8294>
yes
http://svn.apache.org/repos/asf/incubator/oozie/trunk/docs/src/site/twiki/DG_ShellActionExtension.twiki
<https://reviews.apache.org/r/3000/#comment8295>
ok.
http://svn.apache.org/repos/asf/incubator/oozie/trunk/docs/src/site/twiki/DG_ShellActionExtension.twiki
<https://reviews.apache.org/r/3000/#comment8296>
ok.
http://svn.apache.org/repos/asf/incubator/oozie/trunk/docs/src/site/twiki/DG_ShellActionExtension.twiki
<https://reviews.apache.org/r/3000/#comment8297>
Writing an extra Java code (Java Action) to run another Java code is not that convenient.
People could use in different ways.
I will write something on $VAR and ${VAR}
http://svn.apache.org/repos/asf/incubator/oozie/trunk/docs/src/site/twiki/DG_ShellActionExtension.twiki
<https://reviews.apache.org/r/3000/#comment8298>
ok
- Mohammad
On 2011-12-07 08:27:05, Mohammad Islam wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/3000/
> -----------------------------------------------------------
>
> (Updated 2011-12-07 08:27:05)
>
>
> Review request for oozie.
>
>
> Summary
> -------
>
> More context at JIRA OOZIE-578.
>
>
> This addresses bug OOZIE-578.
> https://issues.apache.org/jira/browse/OOZIE-578
>
>
> Diffs
> -----
>
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/docs/src/site/twiki/index.twiki 1209346
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/docs/src/site/twiki/DG_ShellActionExtension.twiki PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestShellMain.java PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/test/java/org/apache/oozie/action/hadoop/ShellTestCase.java PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestShellActionExecutor.java PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/client/src/main/java/org/apache/oozie/cli/OozieCLI.java 1209346
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/client/src/main/resources/shell-action-0.1.xsd PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/conf/oozie-site.xml 1209346
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 1209346
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/3000/diff
>
>
> Testing
> -------
>
>
> Thanks,
>
> Mohammad
>
>
Re: Review Request: OOZIE-578 Support shell action in Oozie WF
Posted by Alejandro Abdelnur <tu...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3000/#review3727
-----------------------------------------------------------
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java
<https://reviews.apache.org/r/3000/#comment8375>
This does not work, params values cannot be modified by a method as Java does pass parameters by value.
The input/error streams are already closed by the run() method of their threads. they are not needed in this method.
http://svn.apache.org/repos/asf/incubator/oozie/trunk/docs/src/site/twiki/DG_ShellActionExtension.twiki
<https://reviews.apache.org/r/3000/#comment8378>
There is no mention of capture-output in the docs.
http://svn.apache.org/repos/asf/incubator/oozie/trunk/docs/src/site/twiki/DG_ShellActionExtension.twiki
<https://reviews.apache.org/r/3000/#comment8376>
why java & classpath in exec and argument???
http://svn.apache.org/repos/asf/incubator/oozie/trunk/docs/src/site/twiki/DG_ShellActionExtension.twiki
<https://reviews.apache.org/r/3000/#comment8377>
what is this?
- Alejandro
On 2011-12-07 19:32:12, Mohammad Islam wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/3000/
> -----------------------------------------------------------
>
> (Updated 2011-12-07 19:32:12)
>
>
> Review request for oozie.
>
>
> Summary
> -------
>
> More context at JIRA OOZIE-578.
>
>
> This addresses bug OOZIE-578.
> https://issues.apache.org/jira/browse/OOZIE-578
>
>
> Diffs
> -----
>
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/docs/src/site/twiki/DG_ShellActionExtension.twiki PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestShellMain.java PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestShellActionExecutor.java PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/test/java/org/apache/oozie/action/hadoop/ShellTestCase.java PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/client/src/main/java/org/apache/oozie/cli/OozieCLI.java 1209346
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/client/src/main/resources/shell-action-0.1.xsd PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/conf/oozie-site.xml 1209346
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 1209346
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/docs/src/site/twiki/index.twiki 1209346
>
> Diff: https://reviews.apache.org/r/3000/diff
>
>
> Testing
> -------
>
>
> Thanks,
>
> Mohammad
>
>
Re: Review Request: OOZIE-578 Support shell action in Oozie WF
Posted by Mohammad Islam <mi...@yahoo.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3000/#review3731
-----------------------------------------------------------
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java
<https://reviews.apache.org/r/3000/#comment8383>
No they are not closed in the run().
http://svn.apache.org/repos/asf/incubator/oozie/trunk/docs/src/site/twiki/DG_ShellActionExtension.twiki
<https://reviews.apache.org/r/3000/#comment8387>
will add one.
http://svn.apache.org/repos/asf/incubator/oozie/trunk/docs/src/site/twiki/DG_ShellActionExtension.twiki
<https://reviews.apache.org/r/3000/#comment8384>
I don't get the concern.
if i run my java cade as follows:
java -classpath <CLASSPATH> MyClass
First one is the executable and rest are args.
http://svn.apache.org/repos/asf/incubator/oozie/trunk/docs/src/site/twiki/DG_ShellActionExtension.twiki
<https://reviews.apache.org/r/3000/#comment8386>
this should be moved to the previous section. cut-paste issue.
- Mohammad
On 2011-12-07 19:32:12, Mohammad Islam wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/3000/
> -----------------------------------------------------------
>
> (Updated 2011-12-07 19:32:12)
>
>
> Review request for oozie.
>
>
> Summary
> -------
>
> More context at JIRA OOZIE-578.
>
>
> This addresses bug OOZIE-578.
> https://issues.apache.org/jira/browse/OOZIE-578
>
>
> Diffs
> -----
>
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/docs/src/site/twiki/DG_ShellActionExtension.twiki PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestShellMain.java PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestShellActionExecutor.java PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/test/java/org/apache/oozie/action/hadoop/ShellTestCase.java PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/client/src/main/java/org/apache/oozie/cli/OozieCLI.java 1209346
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/client/src/main/resources/shell-action-0.1.xsd PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/conf/oozie-site.xml 1209346
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 1209346
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/docs/src/site/twiki/index.twiki 1209346
>
> Diff: https://reviews.apache.org/r/3000/diff
>
>
> Testing
> -------
>
>
> Thanks,
>
> Mohammad
>
>
Re: Review Request: OOZIE-578 Support shell action in Oozie WF
Posted by Alejandro Abdelnur <tu...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3000/#review3758
-----------------------------------------------------------
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java
<https://reviews.apache.org/r/3000/#comment8494>
there are many javadocs through out the patch with:
* incorrect/non-existing params
* no description for the params/return/exception
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java
<https://reviews.apache.org/r/3000/#comment8496>
The action.xml file patch (which comes as a system property to the main class) should be set as an ENV variable to invoked shell
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java
<https://reviews.apache.org/r/3000/#comment8491>
typo 'environment'
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java
<https://reviews.apache.org/r/3000/#comment8492>
The print should be KEY=VALUE instead of :KEY::VALUE:
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java
<https://reviews.apache.org/r/3000/#comment8493>
why do we print the commands here, they are printed in the previous method 'printCommand;
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestShellActionExecutor.java
<https://reviews.apache.org/r/3000/#comment8495>
capture output is not being tested
http://svn.apache.org/repos/asf/incubator/oozie/trunk/docs/src/site/twiki/DG_ShellActionExtension.twiki
<https://reviews.apache.org/r/3000/#comment8497>
the example is correct, it would just seem more appropriate to use a perl/sh invocation.
- Alejandro
On 2011-12-08 10:08:48, Mohammad Islam wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/3000/
> -----------------------------------------------------------
>
> (Updated 2011-12-08 10:08:48)
>
>
> Review request for oozie.
>
>
> Summary
> -------
>
> More context at JIRA OOZIE-578.
>
>
> This addresses bug OOZIE-578.
> https://issues.apache.org/jira/browse/OOZIE-578
>
>
> Diffs
> -----
>
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/client/src/main/java/org/apache/oozie/cli/OozieCLI.java 1209346
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/client/src/main/resources/shell-action-0.1.xsd PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/conf/oozie-site.xml 1209346
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 1209346
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/test/java/org/apache/oozie/action/hadoop/ShellTestCase.java PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestShellActionExecutor.java PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestShellMain.java PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/docs/src/site/twiki/DG_ShellActionExtension.twiki PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/docs/src/site/twiki/index.twiki 1209346
>
> Diff: https://reviews.apache.org/r/3000/diff
>
>
> Testing
> -------
>
>
> Thanks,
>
> Mohammad
>
>
Re: Review Request: OOZIE-578 Support shell action in Oozie WF
Posted by Mohammad Islam <mi...@yahoo.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3000/#review3760
-----------------------------------------------------------
http://svn.apache.org/repos/asf/incubator/oozie/trunk/docs/src/site/twiki/DG_ShellActionExtension.twiki
<https://reviews.apache.org/r/3000/#comment8499>
in the previous example, i covered shell, perl, C etc by changing the property. This is different example.
- Mohammad
On 2011-12-08 10:08:48, Mohammad Islam wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/3000/
> -----------------------------------------------------------
>
> (Updated 2011-12-08 10:08:48)
>
>
> Review request for oozie.
>
>
> Summary
> -------
>
> More context at JIRA OOZIE-578.
>
>
> This addresses bug OOZIE-578.
> https://issues.apache.org/jira/browse/OOZIE-578
>
>
> Diffs
> -----
>
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/client/src/main/java/org/apache/oozie/cli/OozieCLI.java 1209346
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/client/src/main/resources/shell-action-0.1.xsd PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/conf/oozie-site.xml 1209346
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 1209346
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/test/java/org/apache/oozie/action/hadoop/ShellTestCase.java PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestShellActionExecutor.java PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestShellMain.java PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/docs/src/site/twiki/DG_ShellActionExtension.twiki PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/docs/src/site/twiki/index.twiki 1209346
>
> Diff: https://reviews.apache.org/r/3000/diff
>
>
> Testing
> -------
>
>
> Thanks,
>
> Mohammad
>
>
Re: Review Request: OOZIE-578 Support shell action in Oozie WF
Posted by Alejandro Abdelnur <tu...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3000/#review3774
-----------------------------------------------------------
Ship it!
After the comment regarding the name of the ENV var for the action.xml
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java
<https://reviews.apache.org/r/3000/#comment8547>
the name of the ENV variable should in capital letters, ie OOZIE_ACTION_CONF_XML.
And it should be mentioned in the documentation twiki
- Alejandro
On 2011-12-09 03:58:13, Mohammad Islam wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/3000/
> -----------------------------------------------------------
>
> (Updated 2011-12-09 03:58:13)
>
>
> Review request for oozie.
>
>
> Summary
> -------
>
> More context at JIRA OOZIE-578.
>
>
> This addresses bug OOZIE-578.
> https://issues.apache.org/jira/browse/OOZIE-578
>
>
> Diffs
> -----
>
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/client/src/main/java/org/apache/oozie/cli/OozieCLI.java 1209346
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/client/src/main/resources/shell-action-0.1.xsd PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/conf/oozie-site.xml 1209346
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 1209346
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/test/java/org/apache/oozie/action/hadoop/ShellTestCase.java PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestShellActionExecutor.java PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestShellMain.java PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/docs/src/site/twiki/DG_ShellActionExtension.twiki PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/docs/src/site/twiki/index.twiki 1209346
>
> Diff: https://reviews.apache.org/r/3000/diff
>
>
> Testing
> -------
>
>
> Thanks,
>
> Mohammad
>
>
Re: Review Request: OOZIE-578 Support shell action in Oozie WF
Posted by Mohammad Islam <mi...@yahoo.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3000/
-----------------------------------------------------------
(Updated 2011-12-09 03:58:13.368460)
Review request for oozie.
Changes
-------
Including Angelo's comments
Summary
-------
More context at JIRA OOZIE-578.
This addresses bug OOZIE-578.
https://issues.apache.org/jira/browse/OOZIE-578
Diffs (updated)
-----
http://svn.apache.org/repos/asf/incubator/oozie/trunk/client/src/main/java/org/apache/oozie/cli/OozieCLI.java 1209346
http://svn.apache.org/repos/asf/incubator/oozie/trunk/client/src/main/resources/shell-action-0.1.xsd PRE-CREATION
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/conf/oozie-site.xml 1209346
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 1209346
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java PRE-CREATION
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java PRE-CREATION
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/test/java/org/apache/oozie/action/hadoop/ShellTestCase.java PRE-CREATION
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestShellActionExecutor.java PRE-CREATION
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestShellMain.java PRE-CREATION
http://svn.apache.org/repos/asf/incubator/oozie/trunk/docs/src/site/twiki/DG_ShellActionExtension.twiki PRE-CREATION
http://svn.apache.org/repos/asf/incubator/oozie/trunk/docs/src/site/twiki/index.twiki 1209346
Diff: https://reviews.apache.org/r/3000/diff
Testing
-------
Thanks,
Mohammad
Re: Review Request: OOZIE-578 Support shell action in Oozie WF
Posted by "Angelo K. Huang" <an...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3000/#review3771
-----------------------------------------------------------
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java
<https://reviews.apache.org/r/3000/#comment8543>
theArray = handleShellOutput(actionConf, input, error);
- Angelo K.
On 2011-12-09 03:08:45, Mohammad Islam wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/3000/
> -----------------------------------------------------------
>
> (Updated 2011-12-09 03:08:45)
>
>
> Review request for oozie.
>
>
> Summary
> -------
>
> More context at JIRA OOZIE-578.
>
>
> This addresses bug OOZIE-578.
> https://issues.apache.org/jira/browse/OOZIE-578
>
>
> Diffs
> -----
>
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/client/src/main/java/org/apache/oozie/cli/OozieCLI.java 1209346
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/client/src/main/resources/shell-action-0.1.xsd PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/conf/oozie-site.xml 1209346
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 1209346
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/test/java/org/apache/oozie/action/hadoop/ShellTestCase.java PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestShellActionExecutor.java PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestShellMain.java PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/docs/src/site/twiki/DG_ShellActionExtension.twiki PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/docs/src/site/twiki/index.twiki 1209346
>
> Diff: https://reviews.apache.org/r/3000/diff
>
>
> Testing
> -------
>
>
> Thanks,
>
> Mohammad
>
>
Re: Review Request: OOZIE-578 Support shell action in Oozie WF
Posted by Mohammad Islam <mi...@yahoo.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3000/#review3773
-----------------------------------------------------------
java doc for public methods were done.
updated some non-public methods too based on the comments,
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java
<https://reviews.apache.org/r/3000/#comment8546>
correct
- Mohammad
On 2011-12-09 03:08:45, Mohammad Islam wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/3000/
> -----------------------------------------------------------
>
> (Updated 2011-12-09 03:08:45)
>
>
> Review request for oozie.
>
>
> Summary
> -------
>
> More context at JIRA OOZIE-578.
>
>
> This addresses bug OOZIE-578.
> https://issues.apache.org/jira/browse/OOZIE-578
>
>
> Diffs
> -----
>
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/client/src/main/java/org/apache/oozie/cli/OozieCLI.java 1209346
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/client/src/main/resources/shell-action-0.1.xsd PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/conf/oozie-site.xml 1209346
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 1209346
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/test/java/org/apache/oozie/action/hadoop/ShellTestCase.java PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestShellActionExecutor.java PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestShellMain.java PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/docs/src/site/twiki/DG_ShellActionExtension.twiki PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/docs/src/site/twiki/index.twiki 1209346
>
> Diff: https://reviews.apache.org/r/3000/diff
>
>
> Testing
> -------
>
>
> Thanks,
>
> Mohammad
>
>
Re: Review Request: OOZIE-578 Support shell action in Oozie WF
Posted by "Angelo K. Huang" <an...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3000/#review3770
-----------------------------------------------------------
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java
<https://reviews.apache.org/r/3000/#comment8538>
return ....
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java
<https://reviews.apache.org/r/3000/#comment8539>
return....
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java
<https://reviews.apache.org/r/3000/#comment8540>
your mean LM STDOUT?
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java
<https://reviews.apache.org/r/3000/#comment8541>
return ....
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java
<https://reviews.apache.org/r/3000/#comment8542>
return ...
- Angelo K.
On 2011-12-09 03:08:45, Mohammad Islam wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/3000/
> -----------------------------------------------------------
>
> (Updated 2011-12-09 03:08:45)
>
>
> Review request for oozie.
>
>
> Summary
> -------
>
> More context at JIRA OOZIE-578.
>
>
> This addresses bug OOZIE-578.
> https://issues.apache.org/jira/browse/OOZIE-578
>
>
> Diffs
> -----
>
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/client/src/main/java/org/apache/oozie/cli/OozieCLI.java 1209346
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/client/src/main/resources/shell-action-0.1.xsd PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/conf/oozie-site.xml 1209346
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 1209346
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/test/java/org/apache/oozie/action/hadoop/ShellTestCase.java PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestShellActionExecutor.java PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestShellMain.java PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/docs/src/site/twiki/DG_ShellActionExtension.twiki PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/docs/src/site/twiki/index.twiki 1209346
>
> Diff: https://reviews.apache.org/r/3000/diff
>
>
> Testing
> -------
>
>
> Thanks,
>
> Mohammad
>
>
Re: Review Request: OOZIE-578 Support shell action in Oozie WF
Posted by Mohammad Islam <mi...@yahoo.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3000/
-----------------------------------------------------------
(Updated 2011-12-09 03:08:45.019786)
Review request for oozie.
Changes
-------
Adding the review comments.
Summary
-------
More context at JIRA OOZIE-578.
This addresses bug OOZIE-578.
https://issues.apache.org/jira/browse/OOZIE-578
Diffs (updated)
-----
http://svn.apache.org/repos/asf/incubator/oozie/trunk/client/src/main/java/org/apache/oozie/cli/OozieCLI.java 1209346
http://svn.apache.org/repos/asf/incubator/oozie/trunk/client/src/main/resources/shell-action-0.1.xsd PRE-CREATION
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/conf/oozie-site.xml 1209346
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 1209346
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java PRE-CREATION
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java PRE-CREATION
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/test/java/org/apache/oozie/action/hadoop/ShellTestCase.java PRE-CREATION
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestShellActionExecutor.java PRE-CREATION
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestShellMain.java PRE-CREATION
http://svn.apache.org/repos/asf/incubator/oozie/trunk/docs/src/site/twiki/DG_ShellActionExtension.twiki PRE-CREATION
http://svn.apache.org/repos/asf/incubator/oozie/trunk/docs/src/site/twiki/index.twiki 1209346
Diff: https://reviews.apache.org/r/3000/diff
Testing
-------
Thanks,
Mohammad
Re: Review Request: OOZIE-578 Support shell action in Oozie WF
Posted by Mohammad Islam <mi...@yahoo.com>.
> On 2011-12-09 01:30:13, Alejandro Abdelnur wrote:
> > Don't see an answer to my suggestion that the action.xml file path should be exposed in an ENV var to the shell being invoked.
> >
> > Other than that and the comment about capture-output not being verified, things look good.
action.xml is already added in the patch
> On 2011-12-09 01:30:13, Alejandro Abdelnur wrote:
> > http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestShellActionExecutor.java, line 176
> > <https://reviews.apache.org/r/3000/diff/9-10/?file=63168#file63168line176>
> >
> > i don't find the test where the capture-data is verified
i didn't find that in any other action(shell and Java etc). Do you have any such? Because it is already being used.
- Mohammad
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3000/#review3761
-----------------------------------------------------------
On 2011-12-09 00:09:18, Mohammad Islam wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/3000/
> -----------------------------------------------------------
>
> (Updated 2011-12-09 00:09:18)
>
>
> Review request for oozie.
>
>
> Summary
> -------
>
> More context at JIRA OOZIE-578.
>
>
> This addresses bug OOZIE-578.
> https://issues.apache.org/jira/browse/OOZIE-578
>
>
> Diffs
> -----
>
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/client/src/main/java/org/apache/oozie/cli/OozieCLI.java 1209346
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/client/src/main/resources/shell-action-0.1.xsd PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/conf/oozie-site.xml 1209346
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 1209346
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/test/java/org/apache/oozie/action/hadoop/ShellTestCase.java PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestShellActionExecutor.java PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestShellMain.java PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/docs/src/site/twiki/DG_ShellActionExtension.twiki PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/docs/src/site/twiki/index.twiki 1209346
>
> Diff: https://reviews.apache.org/r/3000/diff
>
>
> Testing
> -------
>
>
> Thanks,
>
> Mohammad
>
>
Re: Review Request: OOZIE-578 Support shell action in Oozie WF
Posted by Alejandro Abdelnur <tu...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3000/#review3761
-----------------------------------------------------------
Don't see an answer to my suggestion that the action.xml file path should be exposed in an ENV var to the shell being invoked.
Other than that and the comment about capture-output not being verified, things look good.
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestShellActionExecutor.java
<https://reviews.apache.org/r/3000/#comment8500>
i don't find the test where the capture-data is verified
- Alejandro
On 2011-12-09 00:09:18, Mohammad Islam wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/3000/
> -----------------------------------------------------------
>
> (Updated 2011-12-09 00:09:18)
>
>
> Review request for oozie.
>
>
> Summary
> -------
>
> More context at JIRA OOZIE-578.
>
>
> This addresses bug OOZIE-578.
> https://issues.apache.org/jira/browse/OOZIE-578
>
>
> Diffs
> -----
>
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/client/src/main/java/org/apache/oozie/cli/OozieCLI.java 1209346
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/client/src/main/resources/shell-action-0.1.xsd PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/conf/oozie-site.xml 1209346
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 1209346
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/test/java/org/apache/oozie/action/hadoop/ShellTestCase.java PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestShellActionExecutor.java PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestShellMain.java PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/docs/src/site/twiki/DG_ShellActionExtension.twiki PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/docs/src/site/twiki/index.twiki 1209346
>
> Diff: https://reviews.apache.org/r/3000/diff
>
>
> Testing
> -------
>
>
> Thanks,
>
> Mohammad
>
>
Re: Review Request: OOZIE-578 Support shell action in Oozie WF
Posted by Mohammad Islam <mi...@yahoo.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3000/
-----------------------------------------------------------
(Updated 2011-12-09 00:09:18.613693)
Review request for oozie.
Changes
-------
public methods are provided with full java docs.
Summary
-------
More context at JIRA OOZIE-578.
This addresses bug OOZIE-578.
https://issues.apache.org/jira/browse/OOZIE-578
Diffs (updated)
-----
http://svn.apache.org/repos/asf/incubator/oozie/trunk/client/src/main/java/org/apache/oozie/cli/OozieCLI.java 1209346
http://svn.apache.org/repos/asf/incubator/oozie/trunk/client/src/main/resources/shell-action-0.1.xsd PRE-CREATION
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/conf/oozie-site.xml 1209346
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 1209346
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java PRE-CREATION
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java PRE-CREATION
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/test/java/org/apache/oozie/action/hadoop/ShellTestCase.java PRE-CREATION
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestShellActionExecutor.java PRE-CREATION
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestShellMain.java PRE-CREATION
http://svn.apache.org/repos/asf/incubator/oozie/trunk/docs/src/site/twiki/DG_ShellActionExtension.twiki PRE-CREATION
http://svn.apache.org/repos/asf/incubator/oozie/trunk/docs/src/site/twiki/index.twiki 1209346
Diff: https://reviews.apache.org/r/3000/diff
Testing
-------
Thanks,
Mohammad
Re: Review Request: OOZIE-578 Support shell action in Oozie WF
Posted by Mohammad Islam <mi...@yahoo.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3000/
-----------------------------------------------------------
(Updated 2011-12-08 10:08:48.683398)
Review request for oozie.
Summary
-------
More context at JIRA OOZIE-578.
This addresses bug OOZIE-578.
https://issues.apache.org/jira/browse/OOZIE-578
Diffs (updated)
-----
http://svn.apache.org/repos/asf/incubator/oozie/trunk/client/src/main/java/org/apache/oozie/cli/OozieCLI.java 1209346
http://svn.apache.org/repos/asf/incubator/oozie/trunk/client/src/main/resources/shell-action-0.1.xsd PRE-CREATION
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/conf/oozie-site.xml 1209346
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 1209346
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java PRE-CREATION
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java PRE-CREATION
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/test/java/org/apache/oozie/action/hadoop/ShellTestCase.java PRE-CREATION
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestShellActionExecutor.java PRE-CREATION
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestShellMain.java PRE-CREATION
http://svn.apache.org/repos/asf/incubator/oozie/trunk/docs/src/site/twiki/DG_ShellActionExtension.twiki PRE-CREATION
http://svn.apache.org/repos/asf/incubator/oozie/trunk/docs/src/site/twiki/index.twiki 1209346
Diff: https://reviews.apache.org/r/3000/diff
Testing
-------
Thanks,
Mohammad
Re: Review Request: OOZIE-578 Support shell action in Oozie WF
Posted by Mohammad Islam <mi...@yahoo.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3000/
-----------------------------------------------------------
(Updated 2011-12-08 02:59:07.030835)
Review request for oozie.
Changes
-------
Update doc.
Summary
-------
More context at JIRA OOZIE-578.
This addresses bug OOZIE-578.
https://issues.apache.org/jira/browse/OOZIE-578
Diffs (updated)
-----
http://svn.apache.org/repos/asf/incubator/oozie/trunk/client/src/main/java/org/apache/oozie/cli/OozieCLI.java 1209346
http://svn.apache.org/repos/asf/incubator/oozie/trunk/client/src/main/resources/shell-action-0.1.xsd PRE-CREATION
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/conf/oozie-site.xml 1209346
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 1209346
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java PRE-CREATION
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java PRE-CREATION
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/test/java/org/apache/oozie/action/hadoop/ShellTestCase.java PRE-CREATION
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestShellActionExecutor.java PRE-CREATION
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestShellMain.java PRE-CREATION
http://svn.apache.org/repos/asf/incubator/oozie/trunk/docs/src/site/twiki/DG_ShellActionExtension.twiki PRE-CREATION
http://svn.apache.org/repos/asf/incubator/oozie/trunk/docs/src/site/twiki/index.twiki 1209346
Diff: https://reviews.apache.org/r/3000/diff
Testing
-------
Thanks,
Mohammad
Re: Review Request: OOZIE-578 Support shell action in Oozie WF
Posted by Mohammad Islam <mi...@yahoo.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3000/
-----------------------------------------------------------
(Updated 2011-12-07 19:32:12.656881)
Review request for oozie.
Changes
-------
Updating the documentation text.
Summary
-------
More context at JIRA OOZIE-578.
This addresses bug OOZIE-578.
https://issues.apache.org/jira/browse/OOZIE-578
Diffs (updated)
-----
http://svn.apache.org/repos/asf/incubator/oozie/trunk/docs/src/site/twiki/DG_ShellActionExtension.twiki PRE-CREATION
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestShellMain.java PRE-CREATION
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestShellActionExecutor.java PRE-CREATION
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/test/java/org/apache/oozie/action/hadoop/ShellTestCase.java PRE-CREATION
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java PRE-CREATION
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java PRE-CREATION
http://svn.apache.org/repos/asf/incubator/oozie/trunk/client/src/main/java/org/apache/oozie/cli/OozieCLI.java 1209346
http://svn.apache.org/repos/asf/incubator/oozie/trunk/client/src/main/resources/shell-action-0.1.xsd PRE-CREATION
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/conf/oozie-site.xml 1209346
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 1209346
http://svn.apache.org/repos/asf/incubator/oozie/trunk/docs/src/site/twiki/index.twiki 1209346
Diff: https://reviews.apache.org/r/3000/diff
Testing
-------
Thanks,
Mohammad
Re: Review Request: OOZIE-578 Support shell action in Oozie WF
Posted by Mohammad Islam <mi...@yahoo.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3000/
-----------------------------------------------------------
(Updated 2011-12-07 19:17:34.193890)
Review request for oozie.
Changes
-------
Including comments
Summary
-------
More context at JIRA OOZIE-578.
This addresses bug OOZIE-578.
https://issues.apache.org/jira/browse/OOZIE-578
Diffs (updated)
-----
http://svn.apache.org/repos/asf/incubator/oozie/trunk/client/src/main/java/org/apache/oozie/cli/OozieCLI.java 1209346
http://svn.apache.org/repos/asf/incubator/oozie/trunk/client/src/main/resources/shell-action-0.1.xsd PRE-CREATION
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/conf/oozie-site.xml 1209346
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 1209346
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java PRE-CREATION
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java PRE-CREATION
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/test/java/org/apache/oozie/action/hadoop/ShellTestCase.java PRE-CREATION
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestShellActionExecutor.java PRE-CREATION
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestShellMain.java PRE-CREATION
http://svn.apache.org/repos/asf/incubator/oozie/trunk/docs/src/site/twiki/DG_ShellActionExtension.twiki PRE-CREATION
http://svn.apache.org/repos/asf/incubator/oozie/trunk/docs/src/site/twiki/index.twiki 1209346
Diff: https://reviews.apache.org/r/3000/diff
Testing
-------
Thanks,
Mohammad
Re: Review Request: OOZIE-578 Support shell action in Oozie WF
Posted by Alejandro Abdelnur <tu...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3000/#review3703
-----------------------------------------------------------
http://svn.apache.org/repos/asf/incubator/oozie/trunk/client/src/main/resources/shell-action-0.1.xsd
<https://reviews.apache.org/r/3000/#comment8284>
we shouldn't have java-opts in the schema. users can always use the oozie.laucher.hadoop.child.... property if they need to tweak something for the launcher.
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java
<https://reviews.apache.org/r/3000/#comment8285>
there are trailing space through out the patch
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java
<https://reviews.apache.org/r/3000/#comment8271>
here the files in the current dir should be printed to STDOUT.
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java
<https://reviews.apache.org/r/3000/#comment8272>
remove commented code
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java
<https://reviews.apache.org/r/3000/#comment8273>
this println is not needed,
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java
<https://reviews.apache.org/r/3000/#comment8274>
remove this println
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java
<https://reviews.apache.org/r/3000/#comment8275>
remove this println
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java
<https://reviews.apache.org/r/3000/#comment8277>
STDOUT and STDERR streams should be closed in the thread run() method, after the while read exits. Not here as the Thread.start() is not blocking
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java
<https://reviews.apache.org/r/3000/#comment8278>
here there should the finally block closing the streams
http://svn.apache.org/repos/asf/incubator/oozie/trunk/docs/src/site/twiki/DG_ShellActionExtension.twiki
<https://reviews.apache.org/r/3000/#comment8280>
It would be more appropriate to state that Shell launcher configuration can be specified ...., correct?
Currently the configuration (actionconf.xml) file is not made avail to the shell.
Which makes me think that we should expose the the actionconf.xml file to the shell by setting a OOZIE_ACTION_CONF env var pointing to the file (it the ShellMain we would set that ENV to the value of the system property that has the path)
http://svn.apache.org/repos/asf/incubator/oozie/trunk/docs/src/site/twiki/DG_ShellActionExtension.twiki
<https://reviews.apache.org/r/3000/#comment8282>
capture-output is not in the syntax, nor explained
http://svn.apache.org/repos/asf/incubator/oozie/trunk/docs/src/site/twiki/DG_ShellActionExtension.twiki
<https://reviews.apache.org/r/3000/#comment8281>
while this example is 100% correct, I would use instead an example that is calling a shell instead of calling Java.
For Java user should use the Java action.
Another thing to mention is that when setting ENV variables, if a user want sto reference other ENV variables, he/she should use $VAR (never use ${VAR} as that would be processed as an EL)
http://svn.apache.org/repos/asf/incubator/oozie/trunk/docs/src/site/twiki/DG_ShellActionExtension.twiki
<https://reviews.apache.org/r/3000/#comment8283>
we shouldn't have java-opts in the schema. users can always use the oozie.laucher.hadoop.child.... property if they need to tweak something for the launcher.
- Alejandro
On 2011-12-07 08:27:05, Mohammad Islam wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/3000/
> -----------------------------------------------------------
>
> (Updated 2011-12-07 08:27:05)
>
>
> Review request for oozie.
>
>
> Summary
> -------
>
> More context at JIRA OOZIE-578.
>
>
> This addresses bug OOZIE-578.
> https://issues.apache.org/jira/browse/OOZIE-578
>
>
> Diffs
> -----
>
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/docs/src/site/twiki/index.twiki 1209346
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/docs/src/site/twiki/DG_ShellActionExtension.twiki PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestShellMain.java PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/test/java/org/apache/oozie/action/hadoop/ShellTestCase.java PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestShellActionExecutor.java PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/client/src/main/java/org/apache/oozie/cli/OozieCLI.java 1209346
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/client/src/main/resources/shell-action-0.1.xsd PRE-CREATION
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/conf/oozie-site.xml 1209346
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 1209346
> http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/3000/diff
>
>
> Testing
> -------
>
>
> Thanks,
>
> Mohammad
>
>
Re: Review Request: OOZIE-578 Support shell action in Oozie WF
Posted by Mohammad Islam <mi...@yahoo.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3000/
-----------------------------------------------------------
(Updated 2011-12-07 08:27:05.720289)
Review request for oozie.
Changes
-------
Addressing the new comments.
Summary
-------
More context at JIRA OOZIE-578.
This addresses bug OOZIE-578.
https://issues.apache.org/jira/browse/OOZIE-578
Diffs (updated)
-----
http://svn.apache.org/repos/asf/incubator/oozie/trunk/docs/src/site/twiki/index.twiki 1209346
http://svn.apache.org/repos/asf/incubator/oozie/trunk/docs/src/site/twiki/DG_ShellActionExtension.twiki PRE-CREATION
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestShellMain.java PRE-CREATION
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java PRE-CREATION
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/test/java/org/apache/oozie/action/hadoop/ShellTestCase.java PRE-CREATION
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestShellActionExecutor.java PRE-CREATION
http://svn.apache.org/repos/asf/incubator/oozie/trunk/client/src/main/java/org/apache/oozie/cli/OozieCLI.java 1209346
http://svn.apache.org/repos/asf/incubator/oozie/trunk/client/src/main/resources/shell-action-0.1.xsd PRE-CREATION
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/conf/oozie-site.xml 1209346
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 1209346
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java PRE-CREATION
Diff: https://reviews.apache.org/r/3000/diff
Testing
-------
Thanks,
Mohammad
Re: Review Request: OOZIE-578 Support shell action in Oozie WF
Posted by Mohammad Islam <mi...@yahoo.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3000/
-----------------------------------------------------------
(Updated 2011-12-07 00:51:10.830336)
Review request for oozie.
Changes
-------
Changing the thread blocking using join..
Summary
-------
More context at JIRA OOZIE-578.
This addresses bug OOZIE-578.
https://issues.apache.org/jira/browse/OOZIE-578
Diffs (updated)
-----
http://svn.apache.org/repos/asf/incubator/oozie/trunk/docs/src/site/twiki/index.twiki 1209346
http://svn.apache.org/repos/asf/incubator/oozie/trunk/docs/src/site/twiki/DG_ShellActionExtension.twiki PRE-CREATION
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestShellMain.java PRE-CREATION
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/resources/oozie-default.xml 1209346
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/test/java/org/apache/oozie/action/hadoop/ShellTestCase.java PRE-CREATION
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestShellActionExecutor.java PRE-CREATION
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java PRE-CREATION
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java PRE-CREATION
http://svn.apache.org/repos/asf/incubator/oozie/trunk/client/src/main/java/org/apache/oozie/cli/OozieCLI.java 1209346
http://svn.apache.org/repos/asf/incubator/oozie/trunk/client/src/main/resources/shell-action-0.1.xsd PRE-CREATION
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 1209346
Diff: https://reviews.apache.org/r/3000/diff
Testing
-------
Thanks,
Mohammad
Re: Review Request: OOZIE-578 Support shell action in Oozie WF
Posted by Mohammad Islam <mi...@yahoo.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3000/
-----------------------------------------------------------
(Updated 2011-12-06 20:36:38.447841)
Review request for oozie.
Changes
-------
Applying the comments.
Will create a JIRA for *.wf.schemas.
Will remove the space from the comments manually before final commits.
Summary
-------
More context at JIRA OOZIE-578.
This addresses bug OOZIE-578.
https://issues.apache.org/jira/browse/OOZIE-578
Diffs (updated)
-----
http://svn.apache.org/repos/asf/incubator/oozie/trunk/client/src/main/java/org/apache/oozie/cli/OozieCLI.java 1209346
http://svn.apache.org/repos/asf/incubator/oozie/trunk/client/src/main/resources/shell-action-0.1.xsd PRE-CREATION
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 1209346
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java PRE-CREATION
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java PRE-CREATION
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/main/resources/oozie-default.xml 1209346
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/test/java/org/apache/oozie/action/hadoop/ShellTestCase.java PRE-CREATION
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestShellActionExecutor.java PRE-CREATION
http://svn.apache.org/repos/asf/incubator/oozie/trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestShellMain.java PRE-CREATION
http://svn.apache.org/repos/asf/incubator/oozie/trunk/docs/src/site/twiki/DG_ShellActionExtension.twiki PRE-CREATION
http://svn.apache.org/repos/asf/incubator/oozie/trunk/docs/src/site/twiki/index.twiki 1209346
Diff: https://reviews.apache.org/r/3000/diff
Testing
-------
Thanks,
Mohammad