You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@tez.apache.org by "Bikas Saha (JIRA)" <ji...@apache.org> on 2014/07/11 04:20:04 UTC

[jira] [Comment Edited] (TEZ-1269) TaskScheduler prematurely releases containers

    [ https://issues.apache.org/jira/browse/TEZ-1269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14058237#comment-14058237 ] 

Bikas Saha edited comment on TEZ-1269 at 7/11/14 2:18 AM:
----------------------------------------------------------

1) Removes the session specific check to release an idle container.
2) Creates a min held containers that maintains a minimum held pool only in session mode.
3) Randomizes the idle container expire time between a min and max value to prevent a cliff of de-allocations.

Existing tests broke for 1 and 2. So fixing them gives some coverage. This also fixes TestMRRJobsDAGApi.testBroadcastAndOneToOneExample() so it now can run in non-session mode and get expected container reuse for the 1-1 edge.

Will think of some way to test that min-max randomness if folks think thats necessary. [~gopalv] [~hitesh] Please review.


was (Author: bikassaha):
1) Removes the session specific check to release an idle container.
2) Creates a min held containers that maintains a minimum held pool only in session mode.
3) Randomizes the idle container expire time between a min and max value to prevent a cliff of de-allocations.

Existing tests broke for 1 and 2. So fixing them gives some coverage. Will think of some way to test that min-max randomness if folks think thats necessary. [~gopalv] [~hitesh] Please review.

> TaskScheduler prematurely releases containers
> ---------------------------------------------
>
>                 Key: TEZ-1269
>                 URL: https://issues.apache.org/jira/browse/TEZ-1269
>             Project: Apache Tez
>          Issue Type: Bug
>            Reporter: Bikas Saha
>            Assignee: Bikas Saha
>         Attachments: TEZ-1269.1.patch
>
>
> It checks for session mode and if not true, and if there are no outstanding requests, then it releases the containers before the container timeout has expired. If the state machine is on its way to scheduling new tasks during this time then they will not be able to reuse these containers.



--
This message was sent by Atlassian JIRA
(v6.2#6252)