[jira] Commented: (HIVE-869) Allow ScriptOperator to consume not all input data

["Zheng Shao (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HIVE-869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12763859#action_12763859 ] 

Zheng Shao commented on HIVE-869:
---------------------------------

@HIVE-869.1.patch:Overall looks great!

3 minor issues:

1. We might want to change the query:
"SELECT TRANSFORM(*) USING 'true' AS a, b, c FROM src LIMIT 1"
to:
"SELECT TRANSFORM(*) USING 'true' AS a, b, c FROM (SELECT * FROM src LIMIT 1) tmp"
This makes sure there is a single input line to "true" and very possibly it will die in "close".
This applies to the query both in positive and negative tests.

2. 
+        LOG.error("Got broken pipe: ignoring exception and setting operator to done");
Let's write this:
+        LOG.warn("Got broken pipe: ignoring exception and setting operator to done");

3.
Let's add a line in the error message when writing out this and the error is broken pipe:
+        LOG.error("Error in writing to script: " + e.getMessage() + ");
if (error is brokenpipe) {
LOG.info("The script did not consume all input data. This is considered as an error.");
LOG.info("set " + HiveConf.IGNOREBROKENPIPE.getName() + "=true; to ignore it.");
}


> Allow ScriptOperator to consume not all input data
> --------------------------------------------------
>
>                 Key: HIVE-869
>                 URL: https://issues.apache.org/jira/browse/HIVE-869
>             Project: Hadoop Hive
>          Issue Type: New Feature
>    Affects Versions: 0.5.0
>            Reporter: Zheng Shao
>            Assignee: Paul Yang
>             Fix For: 0.5.0
>
>         Attachments: HIVE-869.1.patch
>
>
> The ScriptOperator (SELECT TRANSFORM(a, b, c) USING 'myscript' AS (d, e, f) ...) has a problem:
> If the user script exits without consuming all data from standard input, then we will report an error even if the exit code from the user script is 0.
> We want to have an option, when enabled, ScriptOperator will return successfully in that case.
> If the option is not enabled, then we should stick to the current behavior.
> The option can be called: "hive.script.ignore.broken.pipe.exception".

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.