You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Raghotham Murthy (JIRA)" <ji...@apache.org> on 2009/02/03 03:21:59 UTC
[jira] Commented: (HIVE-251) Failures in Transform don't stop the
job
[ https://issues.apache.org/jira/browse/HIVE-251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12669847#action_12669847 ]
Raghotham Murthy commented on HIVE-251:
---------------------------------------
more information about this error based on Venky's tests.
For the case where the job succeeds even though the script fails, the following logs appear:
--
2009-02-02 17:49:26,251 INFO org.apache.hadoop.hive.ql.exec.ScriptOperator: StreamThread ErrorProcessor done
*2009-02-02 17:49:26,251 ERROR org.apache.hadoop.hive.ql.exec.ScriptOperator: Script failed with code 1*
2009-02-02 17:49:26,251 INFO org.apache.hadoop.hive.ql.exec.ScriptOperator: StreamThread OutputProcessor done
2009-02-02 17:49:26,251 INFO org.apache.hadoop.hive.ql.exec.ScriptOperator: DESERIALIZE_ERRORS:0
2009-02-02 17:49:26,252 INFO org.apache.hadoop.hive.ql.exec.ScriptOperator: SERIALIZE_ERRORS:0
2009-02-02 17:49:26,296 INFO org.apache.hadoop.mapred.TaskRunner: Task 'task_200901301729_7391_m_000000_0' done.
--
For the case where the job fails:
--
2009-02-02 17:59:03,398 INFO org.apache.hadoop.hive.ql.exec.ScriptOperator: StreamThread ErrorProcessor done
*2009-02-02 17:59:03,398 ERROR org.apache.hadoop.hive.ql.exec.ScriptOperator: Error in writing to script: Broken pipe*
2009-02-02 17:59:03,398 INFO org.apache.hadoop.hive.ql.exec.ScriptOperator: StreamThread OutputProcessor done
2009-02-02 17:59:03,399 INFO org.apache.hadoop.hive.ql.exec.MapOperator: DESERIALIZE_ERRORS:0
2009-02-02 17:59:03,399 INFO org.apache.hadoop.hive.ql.exec.ScriptOperator: SERIALIZE_ERRORS:0
2009-02-02 17:59:03,399 INFO org.apache.hadoop.hive.ql.exec.ScriptOperator: DESERIALIZE_ERRORS:0
2009-02-02 17:59:03,429 WARN org.apache.hadoop.mapred.TaskTracker: Error running child
java.lang.RuntimeException: java.io.IOException: Broken pipe
--
Looks like exceptions thrown in ScriptOperator.close are being ignored.
> Failures in Transform don't stop the job
> ----------------------------------------
>
> Key: HIVE-251
> URL: https://issues.apache.org/jira/browse/HIVE-251
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Serializers/Deserializers
> Reporter: S. Alex Smith
>
> If the program executed via a SELECT TRANSFORM() USING 'foo' exits with a non-zero exit status, Hive proceeds as if nothing bad happened. The main way that the user knows something bad has happened is if the user checks the logs (probably because he got no output). This is doubly bad if the program only fails part of the time (say, on certain inputs) since the job will still produce output and thus the problem will likely go undetected.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.