You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Andrew Ferguson (JIRA)" <ji...@apache.org> on 2012/07/07 00:45:35 UTC
[jira] [Updated] (MAPREDUCE-4327) Enhance CS to schedule accounting
for both memory and cpu cores
[ https://issues.apache.org/jira/browse/MAPREDUCE-4327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andrew Ferguson updated MAPREDUCE-4327:
---------------------------------------
Attachment: MAPREDUCE-4327-v5.patch
This extended and updated version now includes tests, and support for CPU cores information throughout the resource manager.
It also incorporates the feedback from Robert above.
Although this page is very large, there are bulk of the code is either 1) new and updated tests, or 2) updates to the RM and NM webapps, queue metrics, etc. which all need to be updated to display CPU cores as well.
While obviously it would be easier to read this patch if it were split into pieces, the new tests for CPU as a scheduable resource require the updated queue metrics and accounting, creating an inter-dependency. I am certainly open to suggestions from anyone who sees how to split this patch into chunks! :-)
I have tested this patch locally, and it appears to pass the YARN and MapReduce test suites.
your comments and patience appreciated.
thanks,
Andrew
> Enhance CS to schedule accounting for both memory and cpu cores
> ---------------------------------------------------------------
>
> Key: MAPREDUCE-4327
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4327
> Project: Hadoop Map/Reduce
> Issue Type: New Feature
> Components: mrv2, resourcemanager, scheduler
> Affects Versions: 2.0.0-alpha
> Reporter: Arun C Murthy
> Assignee: Arun C Murthy
> Attachments: MAPREDUCE-4327-v2.patch, MAPREDUCE-4327-v3.patch, MAPREDUCE-4327-v4.patch, MAPREDUCE-4327-v5.patch, MAPREDUCE-4327.patch
>
>
> With YARN being a general purpose system, it would be useful for several applications (MPI et al) to specify not just memory but also CPU (cores) for their resource requirements. Thus, it would be useful to the CapacityScheduler to account for both.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira