[jira] [Commented] (YARN-2) Enhance CS to schedule accounting for both memory and cpu cores

["Junping Du (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/YARN-2?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13433917#comment-13433917 ] 

Junping Du commented on YARN-2:
-------------------------------

For two @Ignore issues in TestFairScheduler, I found it reveals two tiny bugs:
testChoiceOfPreemptedContainers: There are something wrong with Resource.equal() that it is possible to compare two object: one is ResourcePBImpl (a normal resource object) and the other is Resource (Resource.NONE). We should check the compared obj is instance of Resource rather than forcing them the same class (that is the general way of implementing equal() but not suitable here).
testPreemptionDecision: That's something wrong with Resources.createResource(memory), if memory == 0, then we should put core = 0 there as the caller tries to create a none resource. Thoughts?
Base on two findings, I update the code a bit and attached as YARN-2-help.patch which pass local unit test (with applying YARN-2 patch). Arun, please feel free to merge into your patch if you think it is helpful. 

                
> Enhance CS to schedule accounting for both memory and cpu cores
> ---------------------------------------------------------------
>
>                 Key: YARN-2
>                 URL: https://issues.apache.org/jira/browse/YARN-2
>             Project: Hadoop YARN
>          Issue Type: New Feature
>          Components: capacityscheduler, scheduler
>            Reporter: Arun C Murthy
>            Assignee: Arun C Murthy
>             Fix For: 2.1.0-alpha
>
>         Attachments: MAPREDUCE-4327.patch, MAPREDUCE-4327.patch, MAPREDUCE-4327.patch, MAPREDUCE-4327-v2.patch, MAPREDUCE-4327-v3.patch, MAPREDUCE-4327-v4.patch, MAPREDUCE-4327-v5.patch, YARN-2-help.patch, YARN-2.patch
>
>
> With YARN being a general purpose system, it would be useful for several applications (MPI et al) to specify not just memory but also CPU (cores) for their resource requirements. Thus, it would be useful to the CapacityScheduler to account for both.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira