[jira] [Commented] (MAPREDUCE-4847) Command Parsing in Hadoop Streaming

["Mariappan Asokan (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/MAPREDUCE-4847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13510236#comment-13510236 ] 

Mariappan Asokan commented on MAPREDUCE-4847:
---------------------------------------------

Hi Peng Lei,
  By invoking "sh" in Hadoop streaming code, you may be creating a dependency on UNIX(or some ugly branching based on OS.)  On Windows, there is no "sh" unless you install additional software like cygwin.  Why can't you put your complex command in a file, give execute permission to the file, and then specify the file name as mapper/reducer command?  You can create this file on the fly if you want.  This way, the streaming command will work in its current form on UNIX and Windows and there is no change needed.

-- Asokan

                
> Command Parsing in Hadoop Streaming
> -----------------------------------
>
>                 Key: MAPREDUCE-4847
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4847
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>          Components: contrib/streaming
>            Reporter: Peng Lei
>              Labels: features
>   Original Estimate: 4h
>  Remaining Estimate: 4h
>
> Hadoop streaming parse the mapper and reducer commands by itself, this is not a good choice, when I write a complex mapper/reducer script inline, such as 'perl -ne ...', it don't work.
> An alternative way is to send the command to the shell, simply create new process(sh -c "command_and_args"), this not also simplize the streaming code, but also improve its capability!

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira