[jira] [Commented] (TEZ-1163) Tez Auto Reducer-parallelism throws Divide-by-Zero

["Gopal V (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/TEZ-1163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14049611#comment-14049611 ] 

Gopal V commented on TEZ-1163:
------------------------------

[~rajesh.balamohan]: Seems like a simple enough safety net, +1.

> Tez Auto Reducer-parallelism throws Divide-by-Zero
> --------------------------------------------------
>
>                 Key: TEZ-1163
>                 URL: https://issues.apache.org/jira/browse/TEZ-1163
>             Project: Apache Tez
>          Issue Type: Bug
>    Affects Versions: 0.5.0
>            Reporter: Gopal V
>            Assignee: Rajesh Balamohan
>            Priority: Minor
>         Attachments: TEZ-1163.1.patch
>
>
> When the per-reducer estimate is set to 256Mb and over-all data output is ~200Mb, gets a divide-by-zero.
> {code}
> 2014-05-31 16:17:29,125 FATAL [AsyncDispatcher event handler] org.apache.hadoop.yarn.event.AsyncDispatcher: Error in dispatcher thread
> java.lang.ArithmeticException: / by zero
> 	at org.apache.tez.dag.library.vertexmanager.ShuffleVertexManager.determineParallelismAndApply(ShuffleVertexManager.java:392)
> 	at org.apache.tez.dag.library.vertexmanager.ShuffleVertexManager.schedulePendingTasks(ShuffleVertexManager.java:446)
> {code}
> Because of incorrect value set for minTasks
> {code}
> vertexmanager.ShuffleVertexManager: Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:true desiredTaskIput:256000000 minTasks:0
> {code}



--
This message was sent by Atlassian JIRA
(v6.2#6252)