[jira] [Created] (MAPREDUCE-4265) Support Queue type exclusive or shared

["Dheeren Beborrtha (JIRA)" <ji...@apache.org>]

Dheeren Beborrtha created MAPREDUCE-4265:
--------------------------------------------

             Summary: Support Queue type exclusive or shared
                 Key: MAPREDUCE-4265
                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4265
             Project: Hadoop Map/Reduce
          Issue Type: Improvement
          Components: contrib/capacity-sched
    Affects Versions: 0.20.205.0
            Reporter: Dheeren Beborrtha


In the current implementation Capacity, MaxCapacity are the two main parameters that can be used for sharing resources on the cluster.  But there is no feature to protect the resource of a queue. In the absence of pre-emption as a feature, if someone decides to reserve its capacity for its exclusive use,  while rest of the queues are set for sharing, it is not simply possible. 
If we choose one queue to be set for exclusive use by setting Capacity=MaxCapacity, this queue gets penalized, if others choose to set for sharing with their MaxCapacity more than capacity. This is an unintended consequence. On the other hand ,  if every queue is  set to  its Capacity=MaxCapacity  the cluster is being partitioned into distinct virtual clusters.
So, the suggestion is, to add an additional queue parameter  <queue_type> with value <exclusive> or <shared>. By default a queue is always shared type. If queue_type is exclusive, the capacity of this queue is non sharable and should be taken out from  total capacity while scheduling jobs in other queues.


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira