[jira] [Resolved] (PIG-3113) Shell command execution hangs job

["Rohini Palaniswamy (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/PIG-3113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Rohini Palaniswamy resolved PIG-3113.
-------------------------------------
    Resolution: Duplicate

> Shell command execution hangs job
> ---------------------------------
>
>                 Key: PIG-3113
>                 URL: https://issues.apache.org/jira/browse/PIG-3113
>             Project: Pig
>          Issue Type: Bug
>          Components: impl
>    Affects Versions: 0.8.1
>            Reporter: James
>
> Executing a shell command inside a Pig script has the potential to deadlock the job. For example, the following statement will block when somebigfile.txt is sufficiently large:
> {code}
> %declare input `cat /path/to/somebigfile.txt`
> {code}
> This happens because PreprocessorContext.executeShellCommand(String) incorrectly uses Runtime.exec().  The sub-process's stderr and stdout streams should be read in a separate thread to prevent p.waitFor() from hanging when the sub-process's output is larger than the output buffer.
> Per the Java Process class javadoc: "Because some native platforms only provide limited buffer size for standard input and output streams, failure to promptly write the input stream or read the output stream of the subprocess may cause the subprocess to block, and even deadlock".
> See http://www.javaworld.com/jw-12-2000/jw-1229-traps.html for a correct solution.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)