[jira] [Commented] (TEZ-1400) Reducers stuck when enabling auto-reduce parallelism (MRR case)

["Bikas Saha (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/TEZ-1400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14091978#comment-14091978 ] 

Bikas Saha commented on TEZ-1400:
---------------------------------

Reducer 4 should not start until reducer 3 has started some tasks. Unless Reducer 4 has min threshold set to 0. IMO, min threshold should not be set to 0 but to some small number such that at least 1 source task is needed to trigger start.

> Reducers stuck when enabling auto-reduce parallelism (MRR case)
> ---------------------------------------------------------------
>
>                 Key: TEZ-1400
>                 URL: https://issues.apache.org/jira/browse/TEZ-1400
>             Project: Apache Tez
>          Issue Type: Bug
>    Affects Versions: 0.5.0
>            Reporter: Rajesh Balamohan
>            Assignee: Rajesh Balamohan
>              Labels: performance
>         Attachments: dag.dot
>
>
> In M -> R1 -> R2 case, if R1 is optimized by auto-parallelism R2 gets stuck waiting for events.
> e.g
> Map 1: 0/1      Map 2: -/-      Map 5: 0/1      Map 6: 0/1      Map 7: 0/1      Reducer 3: 0/23 Reducer 4: 0/1
> ...
> ...
> Map 1: 1/1      Map 2: 148(+13)/161     Map 5: 1/1      Map 6: 1/1      Map 7: 1/1      Reducer 3: 0(+3)/3      Reducer 4: 0(+1)/1  <== Auto reduce parallelism kicks in
> ..
> Map 1: 1/1      Map 2: 161/161  Map 5: 1/1      Map 6: 1/1      Map 7: 1/1      Reducer 3: 3/3  Reducer 4: 0(+1)/1
> Job is stuck waiting for events in Reducer 4.
>  [fetcher [Reducer_3] #23] org.apache.tez.runtime.library.common.shuffle.impl.ShuffleScheduler: copy(3 of 23 at 0.02 MB/s) <=== *Waiting for 20 more partitions, even though Reducer3 has been optimized to use 3 reducers



--
This message was sent by Atlassian JIRA
(v6.2#6252)