[jira] [Reopened] (TEZ-248) 0 number of tasks causes Arithmetic exception

["Gunther Hagleitner (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/TEZ-248?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Gunther Hagleitner reopened TEZ-248:
------------------------------------


Re-opening this. This is difficult to handle on the hive side. Now as we're moving split generation into the AM it becomes virtually impossible for hive to handle.

The use case again is this:

Hive tables can be empty folders. That simply means that there are currently 0 rows for that table. These tables can be used in group by/aggregate or subqueries - basically anywhere in the tree. Splits generated in the AM will be empty and the parallelism will be 0. Tez should in this case simply consider the 0 parallelism task to be completed and continue the downstream processing with an empty dataset. 

> 0 number of tasks causes Arithmetic exception
> ---------------------------------------------
>
>                 Key: TEZ-248
>                 URL: https://issues.apache.org/jira/browse/TEZ-248
>             Project: Apache Tez
>          Issue Type: Sub-task
>            Reporter: Gunther Hagleitner
>            Assignee: Hitesh Shah
>
> When hive queries an empty table it will generate a map vertex with "0 tasks." That's because the splitinfo decides that's the right number. When submitting this here's what Tez sez:
> 2013-06-16 15:42:33,040 DEBUG [AsyncDispatcher event handler] org.apache.tez.dag.app.dag.impl.TaskAttemptImpl: Diagnostics update for attempt_1371421728954_7_1_000001_000000_3: Error: java.lang.ArithmeticException: / by zero
> 	at org.apache.tez.engine.common.shuffle.impl.Shuffle.run(Shuffle.java:122)
> 	at org.apache.tez.engine.lib.input.ShuffledMergedInput.initialize(ShuffledMergedInput.java:67)
> 	at org.apache.tez.mapreduce.processor.reduce.ReduceProcessor.process(ReduceProcessor.java:119)
> 	at org.apache.tez.engine.task.RuntimeTask.run(RuntimeTask.java:79)
> 	at org.apache.tez.mapreduce.task.MRRuntimeTask.run(MRRuntimeTask.java:144)
> 	at org.apache.hadoop.mapred.YarnTezDagChild.runTezTask(YarnTezDagChild.java:334)
> 	at org.apache.hadoop.mapred.YarnTezDagChild.access$000(YarnTezDagChild.java:76)
> 	at org.apache.hadoop.mapred.YarnTezDagChild$2.run(YarnTezDagChild.java:178)
> 	at java.security.AccessController.doPrivileged(Native Method)
> 	at javax.security.auth.Subject.doAs(Subject.java:396)
> 	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1477)
> 	at org.apache.hadoop.mapred.YarnTezDagChild.main(YarnTezDagChild.java:175)



--
This message was sent by Atlassian JIRA
(v6.1#6144)