You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-issues@hadoop.apache.org by "Aroop Maliakkal (JIRA)" <ji...@apache.org> on 2017/03/01 04:59:45 UTC
[jira] [Commented] (YARN-6254) Provide a mechanism to whitelist the
RM REST API clients
[ https://issues.apache.org/jira/browse/YARN-6254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15889500#comment-15889500 ]
Aroop Maliakkal commented on YARN-6254:
---------------------------------------
Here is the symptom of this issue what we are seeing :-
We have some customers who crawls the resourcemanager REST APIs for monitoring and workflow purposes . Given that these resourcemanager responses are pretty heavy (in our case, /v1/cluster/apps is throwing around 300M of size_download data), we feel it is critical to have some kind of governance with platform team to control who all can access these REST APIs.
{quote}hadoop-hadoop-resourcemanager.log.2017-02-24-18.gz:2017-02-24 18:08:01,484 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Size of event-queue is 312000
hadoop-hadoop-resourcemanager.log.2017-02-24-18.gz:2017-02-24 18:08:01,484 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Size of event-queue is 313000
hadoop-hadoop-resourcemanager.log.2017-02-24-18.gz:2017-02-24 18:08:01,485 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Size of event-queue is 314000
hadoop-hadoop-resourcemanager.log.2017-02-24-18.gz:2017-02-24 18:08:01,485 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Size of event-queue is 315000
hadoop-hadoop-resourcemanager.log.2017-02-24-18.gz:2017-02-24 18:08:01,485 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Size of event-queue is 316000
hadoop-hadoop-resourcemanager.log.2017-02-24-18.gz:2017-02-24 18:08:01,485 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Size of event-queue is 317000
hadoop-hadoop-resourcemanager.log.2017-02-24-18.gz:2017-02-24 18:08:01,485 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Size of event-queue is 318000
hadoop-hadoop-resourcemanager.log.2017-02-24-18.gz:2017-02-24 18:08:01,485 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Size of event-queue is 319000
hadoop-hadoop-resourcemanager.log.2017-02-24-18.gz:2017-02-24 18:08:01,485 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Size of event-queue is 320000
hadoop-hadoop-resourcemanager.log.2017-02-24-18.gz:2017-02-24 18:08:01,485 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Size of event-queue is 320000
hadoop-hadoop-resourcemanager.log.2017-02-24-18.gz:2017-02-24 18:08:01,485 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Size of event-queue is 321000
hadoop-hadoop-resourcemanager.log.2017-02-24-18.gz:2017-02-24 18:08:01,485 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Size of event-queue is 322000
hadoop-hadoop-resourcemanager.log.2017-02-24-18.gz:2017-02-24 18:08:01,486 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Size of event-queue is 323000
hadoop-hadoop-resourcemanager.log.2017-02-24-18.gz:2017-02-24 18:08:01,487 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Size of event-queue is 324000
hadoop-hadoop-resourcemanager.log.2017-02-24-18.gz:2017-02-24 18:08:01,487 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Size of event-queue is 324000
hadoop-hadoop-resourcemanager.log.2017-02-24-18.gz:2017-02-24 18:08:01,487 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Size of event-queue is 325000
hadoop-hadoop-resourcemanager.log.2017-02-24-18.gz:2017-02-24 18:08:01,487 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Size of event-queue is 326000
hadoop-hadoop-resourcemanager.log.2017-02-24-18.gz:2017-02-24 18:08:01,487 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Size of event-queue is 327000
hadoop-hadoop-resourcemanager.log.2017-02-24-18.gz:2017-02-24 18:08:01,487 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Size of event-queue is 328000
hadoop-hadoop-resourcemanager.log.2017-02-24-18.gz:2017-02-24 18:08:01,487 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Size of event-queue is 329000
hadoop-hadoop-resourcemanager.log.2017-02-24-18.gz:2017-02-24 18:08:01,487 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Size of event-queue is 330000
hadoop-hadoop-resourcemanager.log.2017-02-24-18.gz:2017-02-24 18:08:01,487 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Size of event-queue is 331000
hadoop-hadoop-resourcemanager.log.2017-02-24-18.gz:2017-02-24 18:08:01,487 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Size of event-queue is 332000
{quote}
> Provide a mechanism to whitelist the RM REST API clients
> --------------------------------------------------------
>
> Key: YARN-6254
> URL: https://issues.apache.org/jira/browse/YARN-6254
> Project: Hadoop YARN
> Issue Type: New Feature
> Components: resourcemanager
> Affects Versions: 2.7.1
> Reporter: Aroop Maliakkal
>
> Currently RM REST APIs are open to everyone. Can we provide a whitelist feature so that we can control what frequency and what hosts can hit the RM REST APIs ?
> Thanks,
> /Aroop
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org