You are viewing a plain text version of this content. The canonical link for it is here.

Posted to yarn-issues@hadoop.apache.org by "Kishore Chaliparambil (JIRA)" <ji...@apache.org> on 2015/07/23 03:13:06 UTC

[jira] [Updated] (YARN-2884) Proxying all AM-RM communications

     [ https://issues.apache.org/jira/browse/YARN-2884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Kishore Chaliparambil updated YARN-2884:
----------------------------------------
    Attachment: YARN-2884-V1.patch

Added a new service in NodeManager called AMRMProxyService. This is by default disabled and can be enabled using "yarn.nodemanager.amrmproxy.enable". This service can be used for proxying calls from the Application Master to the Resource Manager. 

This feature is needed for:
1) Federation - to transparently scale the application across multiple sub-clusters (YARN-2915)
2) Mercury – to queue containers directly in NMs for distributed scheduling (YARN-2877)
3) Security/Throttling etc. – prevent DDoS attacks on RM




> Proxying all AM-RM communications
> ---------------------------------
>
>                 Key: YARN-2884
>                 URL: https://issues.apache.org/jira/browse/YARN-2884
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: nodemanager, resourcemanager
>            Reporter: Carlo Curino
>            Assignee: Kishore Chaliparambil
>         Attachments: YARN-2884-V1.patch
>
>
> We introduce the notion of an RMProxy, running on each node (or once per rack). Upon start the AM is forced (via tokens and configuration) to direct all its requests to a new services running on the NM that provide a proxy to the central RM. 
> This give us a place to:
> 1) perform distributed scheduling decisions
> 2) throttling mis-behaving AMs
> 3) mask the access to a federation of RMs



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)