[jira] [Updated] (TEZ-585) Unify code path for reused and new containers

["Bikas Saha (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/TEZ-585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Bikas Saha updated TEZ-585:
---------------------------

    Attachment: TEZ-585.1.patch

Adds new containers to delay queue with nextScheduleTime greater than max seen schedule time. Thus reused containers will be preferred over new containers.
Frees up unused new containers when they cannot be matched at all or when they are unused and no pending requests are present.

> Unify code path for reused and new containers
> ---------------------------------------------
>
>                 Key: TEZ-585
>                 URL: https://issues.apache.org/jira/browse/TEZ-585
>             Project: Apache Tez
>          Issue Type: Bug
>            Reporter: Bikas Saha
>         Attachments: TEZ-585.1.patch
>
>
> Without this there is a race between new and reused containers for task assignments. It is not correct to assign a task to a rack-local new container if there is a rack-local reused container that can also be matched. Unifying the assignment logic can help prevent this systematically. 
> This also sets us up to trade-off rack-local reused container vs node-local new container if we want to do that.



--
This message was sent by Atlassian JIRA
(v6.1#6144)