[jira] [Created] (TEZ-2940) Invalid shuffle max slow start setting causes vertex to hang indefinitely

["Jonathan Eagles (JIRA)" <ji...@apache.org>]

Jonathan Eagles created TEZ-2940:
------------------------------------

             Summary: Invalid shuffle max slow start setting causes vertex to hang indefinitely
                 Key: TEZ-2940
                 URL: https://issues.apache.org/jira/browse/TEZ-2940
             Project: Apache Tez
          Issue Type: Bug
            Reporter: Jonathan Eagles
            Assignee: Jonathan Eagles


If invalid shuffle vertex manager max slow start setting is used, vertex can defer indefinitely if output completed tasks output size is less than the desired input task size.

{noformat}
2015-11-12 14:54:48,378 [INFO] [Dispatcher thread {Central}] |vertexmanager.ShuffleVertexManager|: Shuffle Vertex Manager: settings minFrac:0.9 maxFrac:95.0 auto:true desiredTaskIput:134217728 minTasks:1
{noformat}

{noformat}
2015-11-12 14:55:31,432 [INFO] [App Shared Pool - #6] |vertexmanager.ShuffleVertexManager|: Defer scheduling tasks; vertex=scope-218, totalNumBipartiteSourceTasks=165, completedSourceTasksOutputSize=977373325, numVertexManagerEventsReceived=165, numBipartiteSourceTasksCompleted=165, maxThreshold=15675.0
{noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)