[jira] [Resolved] (MAPREDUCE-452) tasktracker checkpointing capability

["Allen Wittenauer (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/MAPREDUCE-452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Allen Wittenauer resolved MAPREDUCE-452.
----------------------------------------

    Resolution: Fixed

Marking this as fixed since YARN provides this capability.

> tasktracker checkpointing capability
> ------------------------------------
>
>                 Key: MAPREDUCE-452
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-452
>             Project: Hadoop Map/Reduce
>          Issue Type: New Feature
>            Reporter: Pete Wyckoff
>            Priority: Minor
>
> This relates to allowing a resource manager (e.g., hadoop on demand) to grow and (rarely) shrink jobs on the fly.
> Growing is already supported. Shrinking could be done in 2 ways - (1) consider the machine dead and allow speculative execution to take care of it or (2) moving the existing map outputs from that machine somewhere else (another machine, dfs) - "task tracker checkpointing" 
> In the case of IO only intensive jobs,  checkpointing the tasktracker doesn't do much for you.  But, in the case of CPU or other scarce resource (e.g., a DB or Webpage cache...), the checkpointing could be very useful.  The question is how often is this the case and how useful?



--
This message was sent by Atlassian JIRA
(v6.2#6252)