[jira] [Updated] (TEZ-3535) YarnTaskScheduler can hold onto low priority containers until they expire

["Jason Lowe (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/TEZ-3535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jason Lowe updated TEZ-3535:
----------------------------
    Attachment: TEZ-3535.001.patch

Here's a prototype of the idea I pitched in TEZ-3491.  This changes the scheduler to assign new containers to tasks just like we do when container reuse is disabled.  In other words, we only do something different for used containers when container reuse is enabled.


> YarnTaskScheduler can hold onto low priority containers until they expire
> -------------------------------------------------------------------------
>
>                 Key: TEZ-3535
>                 URL: https://issues.apache.org/jira/browse/TEZ-3535
>             Project: Apache Tez
>          Issue Type: Bug
>    Affects Versions: 0.7.1, 0.8.4
>            Reporter: Jason Lowe
>            Assignee: Jason Lowe
>         Attachments: TEZ-3535.001.patch
>
>
> With container reuse enabled, YarnTaskScheduler will retain but not schedule any container allocations that are lower priority than the highest priority task requests.  This can lead to poor performance as these lower priority containers clog up resources needed for high priority allocations.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)