[jira] [Commented] (YARN-291) Dynamic resource configuration on NM

["Luke Lu (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/YARN-291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13618522#comment-13618522 ] 

Luke Lu commented on YARN-291:
------------------------------

Resource scheduling is fundamentally centralized at RM. The global resource view is currently bootstrapped via the node registration process, which is more of a historical artifact based on convenience, since the resource view can also be constructed directly on RM via an inventory database. It's a round about, inconvenient and inefficient way to (re)construct the resource view by modifying per node config explicitly and propagate partial views to RM, if you already have an inventory database.

For a practical example: if you have OLTP workload (say HBase) sharing the same hardware with YARN and there is a load surge on HBase, we need to stop scheduling tasks/containers immediately on relevant (potentially all) nodes. The current patch (JMX is just used as a portable protocol for external management client to communicate with RM) can take effect immediately most efficiently. If we explicitly modify each nodemanager config and let the NM-RM protocol to propagate the change to RM, it would waste resource (CPU and network bandwidth) to contact (potentially) all the nodemanagers and cause unnecessary scheduling delays if the propagation is via regular heartbeat and/or DDoS the RM if (potentially) all the NMs need to re-register out-of-band.

This is not about JMX based tricks. This is about changing global resource view directly where the scheduler is vs the Rube Goldbergish way of changing NM config individually and propagate changes to RM to reconstruct the resource view. IMO, the direct way is better because NM doesn't really care about what resource it really has.
                
> Dynamic resource configuration on NM
> ------------------------------------
>
>                 Key: YARN-291
>                 URL: https://issues.apache.org/jira/browse/YARN-291
>             Project: Hadoop YARN
>          Issue Type: New Feature
>          Components: nodemanager, scheduler
>            Reporter: Junping Du
>            Assignee: Junping Du
>              Labels: features
>         Attachments: Elastic Resources for YARN-v0.2.pdf, YARN-291-AddClientRMProtocolToSetNodeResource-03.patch, YARN-291-all-v1.patch, YARN-291-core-HeartBeatAndScheduler-01.patch, YARN-291-JMXInterfaceOnNM-02.patch, YARN-291-OnlyUpdateWhenResourceChange-01-fix.patch, YARN-291-YARNClientCommandline-04.patch
>
>
> The current Hadoop YARN resource management logic assumes per node resource is static during the lifetime of the NM process. Allowing run-time configuration on per node resource will give us finer granularity of resource elasticity. This allows Hadoop workloads to coexist with other workloads on the same hardware efficiently, whether or not the environment is virtualized. About more background and design details, please refer: HADOOP-9165.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira