[jira] [Commented] (PIG-4976) streaming job with store clause stuck if the script fail

["Daniel Dai (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/PIG-4976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15454501#comment-15454501 ] 

Daniel Dai commented on PIG-4976:
---------------------------------

The reason is, when the process exit, the input/output stream closed. Pig continue write to/flush the process input stream and raise exception. The exception is not handled properly so that the script process is killed, but Pig is still waiting for binaryInputQueue.

If the streaming command does not have store clause, Pig will use ProcessOutputThread to handle the process output, which can detect process exit and send signal to Pig's data pipeline. If there is store clause, Pig will not use ProcessOutputThread.

> streaming job with store clause stuck if the script fail
> --------------------------------------------------------
>
>                 Key: PIG-4976
>                 URL: https://issues.apache.org/jira/browse/PIG-4976
>             Project: Pig
>          Issue Type: Bug
>          Components: impl
>            Reporter: Daniel Dai
>            Assignee: Daniel Dai
>             Fix For: 0.17.0
>
>
> When investigating PIG-4972, I also notice Pig job stuck when the perl script have syntax error. This happens if we have output clause in stream specification (means use a file as staging). The bug exist in both Tez and MR, and it is not a regression.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)