You are viewing a plain text version of this content. The canonical link for it is here.

Posted to yarn-dev@hadoop.apache.org by "Tsuyoshi OZAWA (JIRA)" <ji...@apache.org> on 2013/12/27 06:57:50 UTC

[jira] [Created] (YARN-1543) Stateless multiple resource manager

Tsuyoshi OZAWA created YARN-1543:
------------------------------------

             Summary: Stateless multiple resource manager
                 Key: YARN-1543
                 URL: https://issues.apache.org/jira/browse/YARN-1543
             Project: Hadoop YARN
          Issue Type: Improvement
            Reporter: Tsuyoshi OZAWA


Currently, YARN is implemented based on statefull single resource manager architecture. This approach can raise operation costs compare to stateless multi resource manager architecture, because we need to care which resource manager is active to ensure availability. 

We can avoid this problem by saving all states into updatable and transactional RMStateStore such as ZooKeeper or RDBMS. If we take this approach, multiple resource managers can be launched at the same time, we can solve this problem. Operation cost can be reduced, because we don't need to maintain specific(current master) resource manager.

This idea is based on a paper: “Omega: ﬂexible, scalable schedulers for large compute clusters”[1].

[1] http://eurosys2013.tudos.org/wp-content/uploads/2013/paper/Schwarzkopf.pdf




--
This message was sent by Atlassian JIRA
(v6.1.5#6160)