[jira] Commented: (HADOOP-4100) Scheduler.assignTasks should not be dealing with cleanupTask

["Hemanth Yamijala (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HADOOP-4100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12628838#action_12628838 ] 

Hemanth Yamijala commented on HADOOP-4100:
------------------------------------------

I am assuming that since this JIRA is filed, further discussions on the approach and implementation of this issue will happen here.

In the discussion thread mentioned in the description, it is suggested that the new cleanupTask introduced b HADOOP-3150 is scheduled from the framework directly, as opposed to having it scheduled from schedulers. In general, this seems the right approach. So, +1. 

One question I have (and possibly this was indirectly brought up by Matei as well), should the running of the cleanupTask count as slots used by the Job - as this could impact fairness and capacity limits ? From what I've seen, it appears that each Job has atmost 1 cleanup task - correct ? In that case, it might not be *that* critical to count these for quotas etc. Comments ?

> Scheduler.assignTasks should not be dealing with cleanupTask
> ------------------------------------------------------------
>
>                 Key: HADOOP-4100
>                 URL: https://issues.apache.org/jira/browse/HADOOP-4100
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: mapred
>    Affects Versions: 0.19.0
>            Reporter: Devaraj Das
>            Assignee: Amareshwari Sriramadasu
>             Fix For: 0.19.0
>
>
> HADOOP-3150 introduced the notion of a cleanupTask. The implementation of the scheduling of the cleanupTask is such that the schedulers have to be aware of the cleanupTask. It would be very nice if schedulers didn't have to be aware of this special task. There is a discussion thread - https://issues.apache.org/jira/browse/HADOOP-3150?focusedCommentId=12628635#action_12628635

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.