[jira] [Commented] (MAPREDUCE-5381) Support graceful decommission of tasktracker

["Luke Lu (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/MAPREDUCE-5381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13791002#comment-13791002 ] 

Luke Lu commented on MAPREDUCE-5381:
------------------------------------

Instead of adding a new decommission API and more code in JT, another more compatible way would be doing this all in the client using MAPREDUCE-4900 as a sub-command/option ({{\-\-graceful}} and optionally {{\-\-timeout}}) of {{--refreshNodes}}, such that the admin client sets the target TTs' slots to 0 and wait for running tasks/jobs metrics to become zero or timeout and then call the {{refreshNodes}} API.

> Support graceful decommission of tasktracker
> --------------------------------------------
>
>                 Key: MAPREDUCE-5381
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5381
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>          Components: mrv1
>    Affects Versions: 1.2.0
>            Reporter: Luke Lu
>            Assignee: Binglin Chang
>         Attachments: MAPREDUCE-5381-graceful-decomm.v1.patch
>
>
> When TTs are decommissioned for non-fault reasons (capacity change etc.), it's desirable to minimize the impact to running jobs.
> Currently if a TT is decommissioned, all running tasks on the TT need to be rescheduled on other TTs. Further more, for finished map tasks, if their map output are not fetched by the reducers of the job, these map tasks will need to be rerun as well.
> We propose to introduce a mechanism to optionally gracefully decommission a tasktracker.



--
This message was sent by Atlassian JIRA
(v6.1#6144)