[jira] [Commented] (MAPREDUCE-4256) Improve resource scheduling

["Radim Kolar (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/MAPREDUCE-4256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13480731#comment-13480731 ] 

Radim Kolar commented on MAPREDUCE-4256:
----------------------------------------

About optional resources like GPU for computation. Lets task to specify weight in application resource requests.

for example request like "gpu:1:.8" means requesting 1 of gpu resources with weight 0.8 -> give node 80% bonus if gpu resource is available

it could handle data locality as well, because for some compute tasks data locality its not needed because you do not move much data over net. In that case request will be "data:1:0.0"
                
> Improve resource scheduling
> ---------------------------
>
>                 Key: MAPREDUCE-4256
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4256
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>          Components: resourcemanager
>            Reporter: Radim Kolar
>
> Currently resource manager supports only Memory resource during container allocation.
> I propose following improvements:
> 1. add support for CPU utilization. Node CPU used information can be obtained by ResourceCalculatorPlugin.
> 2. add support for custom resources. In node configuration will be something like:
> name=node.resource.GPU, value=1 (node has 1 GPU).
> If job will need to use GPU for computation, it will add "GPU=1" requirement to its job config and Resource Manager will allocate container on node with GPU available.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira