You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@tez.apache.org by "Jonathan Eagles (JIRA)" <ji...@apache.org> on 2015/11/12 18:32:10 UTC
[jira] [Updated] (TEZ-2940) Invalid shuffle max slow start setting
causes vertex to hang indefinitely
[ https://issues.apache.org/jira/browse/TEZ-2940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Eagles updated TEZ-2940:
---------------------------------
Attachment: TEZ-2940.1.patch
> Invalid shuffle max slow start setting causes vertex to hang indefinitely
> -------------------------------------------------------------------------
>
> Key: TEZ-2940
> URL: https://issues.apache.org/jira/browse/TEZ-2940
> Project: Apache Tez
> Issue Type: Bug
> Reporter: Jonathan Eagles
> Assignee: Jonathan Eagles
> Attachments: TEZ-2940.1.patch
>
>
> If invalid shuffle vertex manager max slow start setting is used, vertex can defer indefinitely if output completed tasks output size is less than the desired input task size.
> {noformat}
> 2015-11-12 14:54:48,378 [INFO] [Dispatcher thread {Central}] |vertexmanager.ShuffleVertexManager|: Shuffle Vertex Manager: settings minFrac:0.9 maxFrac:95.0 auto:true desiredTaskIput:134217728 minTasks:1
> {noformat}
> {noformat}
> 2015-11-12 14:55:31,432 [INFO] [App Shared Pool - #6] |vertexmanager.ShuffleVertexManager|: Defer scheduling tasks; vertex=scope-218, totalNumBipartiteSourceTasks=165, completedSourceTasksOutputSize=977373325, numVertexManagerEventsReceived=165, numBipartiteSourceTasksCompleted=165, maxThreshold=15675.0
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)