[jira] Commented: (HADOOP-3687) Ability to pause/resume tasks

["Vivek Ratan (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HADOOP-3687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12637264#action_12637264 ] 

Vivek Ratan commented on HADOOP-3687:
-------------------------------------

I'm wondering if you really need this feature, now that we have pluggable schedulers (HADOOP-3412, HADOOP-3444, HADOOP-3746). Schedulers decide what is the best task to run, given priorities and resources, and other constraints. If you're concerned about low-priority jobs blocking resources for higher-priority jobs that are submitted later, you may want to look at one of these schedulers, or write your own that deals with the situation. Granted that you still won't be able to pause tasks, but you may not need to do that if your scheduler works correctly. It may start running a task from a low-running job, and then assign a task from the higher-priority job to the next available TT. Yes, you can reach a situation where tasks from the lower-priority job are consuming all resources and all long-running, so the higher-priority job is waiting,but there are ways to counter that. HADOOP-3444 provides capacities and user limits, along with preemption, to prevent one user or one job from taking over the system. 

I think supporting pausing tasks is non-trivial so you want to make sure there is a real need for it. 

> Ability to pause/resume tasks
> -----------------------------
>
>                 Key: HADOOP-3687
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3687
>             Project: Hadoop Core
>          Issue Type: New Feature
>          Components: mapred
>            Reporter: Chris Smith
>            Assignee: Chris Smith
>            Priority: Minor
>         Attachments: hadoop-pausing.8.trunk.patch
>
>
> It would be nice to be able to pause (and subsequently resume) tasks that are currently running, in order to allow tasks from higher priority jobs to execute. At present it is quite easy for long-running tasks from low priority jobs to block a task from a newer high priority job, and there is no way to force the execution of the high priority task without killing the low priority jobs.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.