You are viewing a plain text version of this content. The canonical link for it is here.

Posted to mapreduce-user@hadoop.apache.org by Oleg Sukhoroslov <ol...@gmail.com> on 2014/12/05 11:06:35 UTC

Scheduler questions

Hi,

We have a Hadoop cluster which is used by many users (students) 
competing for resources.
We would like to prioritize jobs based on resource usage history per user.
Such functionality is common among classic HPC schedulers, e.g.
http://docs.adaptivecomputing.com/suite/8-0-1/enterprise/help.htm#topics/moabWorkloadManager/topics/fairness/6.3fairshare.html

AFAIK, the FairShareScheduler takes into account only current resource 
usage.
Are there any plans or ongoing efforts to implement fair share 
scheduling based on resource usage history?

A similar question regarding limiting job resource usage in a similar 
fashion as in classical HPC clusters.
Are there any plans to support specifying and enforcing limits on total 
cpu time per job or task?

Thanks a lot

-- Oleg