Re: Review Request: Added new config 'shell' for Exec source

[Roshan Naik <ro...@hortonworks.com>]

> On Jan. 28, 2013, 7:27 p.m., Brock Noland wrote:
> > flume-ng-core/src/main/java/org/apache/flume/source/ExecSource.java, line 248
> > <https://reviews.apache.org/r/7748/diff/2/?file=186015#file186015line248>
> >
> >     nit: I understand the other member variables are not final, but since we are adding a new one, can we make this one final?

This cannot be final..  its value is set by the configure method.


> On Jan. 28, 2013, 7:27 p.m., Brock Noland wrote:
> > flume-ng-core/src/test/java/org/apache/flume/source/TestExecSource.java, line 58
> > <https://reviews.apache.org/r/7748/diff/2/?file=186017#file186017line58>
> >
> >     +1 for moving this to shared methods. However, I cannot think of any other tests except the HBase tests which use static stuff. I think it makes sense if the static stuff is heavy weight but these objects are light weight. Let's stick with Before and After here.


>  'it makes sense if the static stuff is heavy weight but these objects are light weight'
I am not clear what heavy/light weight has to do with this being static. 

> Let's stick with Before and After here.
I assume you are suggesting moving things from @BeforeClass to  @Before. I dont see why we would want to de-optimize things by doing things multiple times when there is no need for it. 


> On Jan. 28, 2013, 7:27 p.m., Brock Noland wrote:
> > flume-ng-core/src/test/resources/test_command.txt, line 3
> > <https://reviews.apache.org/r/7748/diff/2/?file=186018#file186018line3>
> >
> >     Let's add tests to test shell arithmetic and also the more modern $( ) proces substitution.

Brock these are arithmetic and $( ) features are very specific to certain shells. Although i dont mind adding more tests where needed .. it seems futile to do these shell specific feature testing. 

The specified shell will process the whole command/script.  If either $( ) or  shell arithmetic does not work.. we have a case for filing a bug against that particular shell or maybe upgrading bash to a newer version.

On a side note I have validated that this mechanism works on Windows with powershell too.


- Roshan


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/7748/#review15760
-----------------------------------------------------------


On Nov. 6, 2012, 9:52 p.m., Roshan Naik wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/7748/
> -----------------------------------------------------------
> 
> (Updated Nov. 6, 2012, 9:52 p.m.)
> 
> 
> Review request for Flume.
> 
> 
> Description
> -------
> 
> Added new optional config directive 'shell' for Exec Source. One can specify how to invoke a shell to run the command (e.g. /bin/sh -c) . This is only needed for commands that use features like wildcards, backticks, pipes, etc that are supported by the shell. 
> 
> 
> This addresses bug https://issues.apache.org/jira/browse/FLUME-1661.
>     https://issues.apache.org/jira/browse/https://issues.apache.org/jira/browse/FLUME-1661
> 
> 
> Diffs
> -----
> 
>   flume-ng-core/src/main/java/org/apache/flume/source/ExecSource.java 46f672f 
>   flume-ng-core/src/main/java/org/apache/flume/source/ExecSourceConfigurationConstants.java 0ba0508 
>   flume-ng-core/src/test/java/org/apache/flume/source/TestExecSource.java 8bcf320 
>   flume-ng-core/src/test/resources/test_command.txt PRE-CREATION 
>   flume-ng-doc/sphinx/FlumeUserGuide.rst 3af400e 
>   pom.xml 363c0e4 
> 
> Diff: https://reviews.apache.org/r/7748/diff/
> 
> 
> Testing
> -------
> 
> Have added a few test cases for both simple and more complex commands.
> 
> 
> Thanks,
> 
> Roshan Naik
> 
>