You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-issues@hadoop.apache.org by "Prabhu Joseph (Jira)" <ji...@apache.org> on 2019/10/18 18:34:00 UTC
[jira] [Updated] (YARN-9920) YarnAuthorizationProvider
AccessRequest has Null RemoteAddress in case of FairScheduler
[ https://issues.apache.org/jira/browse/YARN-9920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Prabhu Joseph updated YARN-9920:
--------------------------------
Description:
YarnAuthorizationProvider AccessRequest has Null RemoteAddress in case of FairScheduler. FSQueue#hasAccess uses Server.getRemoteAddress() which will be Null when the call is from RMWebServices and EventDispatcher. It works fine when called by IPC Server Handler.
FSQueue#hasAccess is called at three places where (2) and (3) returns NULL.
*1. IPC Server -> RMAppManager#createAndPopulateNewRMApp -> FSQueue#hasAccess -> Server.getRemoteAddress returns correct Remote IP.*
*2. IPC Server -> RMAppManager#createAndPopulateNewRMApp -> AppAddedSchedulerEvent*
*EventDispatcher -> FairScheduler#addApplication -> FSQueue.hasAccess -> Server.getRemoteAddress returns NULL*
{code:java}
org.apache.hadoop.yarn.security.ConfiguredYarnAuthorizer.checkPermission(ConfiguredYarnAuthorizer.java:101)
at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSQueue.hasAccess(FSQueue.java:316)
at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.addApplication(FairScheduler.java:509)
at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.handle(FairScheduler.java:1268)
at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.handle(FairScheduler.java:133)
at org.apache.hadoop.yarn.event.EventDispatcher$EventProcessor.run(EventDispatcher.java:66)
{code}
*3. RMWebServices -> QueueACLsManager#checkAccess -> FSQueue.hasAccess -> Server.getRemoteAddress returns NULL.*
{code:java}
org.apache.hadoop.yarn.security.ConfiguredYarnAuthorizer.checkPermission(ConfiguredYarnAuthorizer.java:101)
at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSQueue.hasAccess(FSQueue.java:316)
at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.checkAccess(FairScheduler.java:1610)
at org.apache.hadoop.yarn.server.resourcemanager.security.QueueACLsManager.checkAccess(QueueACLsManager.java:84)
at org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices.hasAccess(RMWebServices.java:270)
at org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices.getApps(RMWebServices.java:553)
{code}
Have verified with CapacityScheduler and it works fine.
was:
YarnAuthorizationProvider AccessRequest has Null RemoteAddress in case of FairScheduler. FSQueue#hasAccess uses Server.getRemoteAddress() which will be Null when the call is from RMWebServices and EventDispatcher. It works fine only when called by IPC Server Handler.
FSQueue#hasAccess is called at three places where (2) and (3) returns NULL.
1. IPC Server -> RMAppManager#createAndPopulateNewRMApp -> FSQueue#hasAccess -> Server.getRemoteAddress returns correct Remote IP.
2. IPC Server -> RMAppManager#createAndPopulateNewRMApp -> AppAddedSchedulerEvent
EventDispatcher -> FairScheduler#addApplication -> FSQueue.hasAccess -> Server.getRemoteAddress returns NULL.
{code}
org.apache.hadoop.yarn.security.ConfiguredYarnAuthorizer.checkPermission(ConfiguredYarnAuthorizer.java:101)
at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSQueue.hasAccess(FSQueue.java:316)
at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.addApplication(FairScheduler.java:509)
at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.handle(FairScheduler.java:1268)
at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.handle(FairScheduler.java:133)
at org.apache.hadoop.yarn.event.EventDispatcher$EventProcessor.run(EventDispatcher.java:66)
{code}
3. RMWebServices -> QueueACLsManager#checkAccess -> FSQueue.hasAccess -> Server.getRemoteAddress returns NULL.
{code}
org.apache.hadoop.yarn.security.ConfiguredYarnAuthorizer.checkPermission(ConfiguredYarnAuthorizer.java:101)
at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSQueue.hasAccess(FSQueue.java:316)
at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.checkAccess(FairScheduler.java:1610)
at org.apache.hadoop.yarn.server.resourcemanager.security.QueueACLsManager.checkAccess(QueueACLsManager.java:84)
at org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices.hasAccess(RMWebServices.java:270)
at org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices.getApps(RMWebServices.java:553)
{code}
Have verified with CapacityScheduler and it works fine.
> YarnAuthorizationProvider AccessRequest has Null RemoteAddress in case of FairScheduler
> ---------------------------------------------------------------------------------------
>
> Key: YARN-9920
> URL: https://issues.apache.org/jira/browse/YARN-9920
> Project: Hadoop YARN
> Issue Type: Bug
> Affects Versions: 3.3.0
> Reporter: Prabhu Joseph
> Assignee: Prabhu Joseph
> Priority: Major
>
> YarnAuthorizationProvider AccessRequest has Null RemoteAddress in case of FairScheduler. FSQueue#hasAccess uses Server.getRemoteAddress() which will be Null when the call is from RMWebServices and EventDispatcher. It works fine when called by IPC Server Handler.
> FSQueue#hasAccess is called at three places where (2) and (3) returns NULL.
> *1. IPC Server -> RMAppManager#createAndPopulateNewRMApp -> FSQueue#hasAccess -> Server.getRemoteAddress returns correct Remote IP.*
>
> *2. IPC Server -> RMAppManager#createAndPopulateNewRMApp -> AppAddedSchedulerEvent*
> *EventDispatcher -> FairScheduler#addApplication -> FSQueue.hasAccess -> Server.getRemoteAddress returns NULL*
>
> {code:java}
> org.apache.hadoop.yarn.security.ConfiguredYarnAuthorizer.checkPermission(ConfiguredYarnAuthorizer.java:101)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSQueue.hasAccess(FSQueue.java:316)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.addApplication(FairScheduler.java:509)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.handle(FairScheduler.java:1268)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.handle(FairScheduler.java:133)
> at org.apache.hadoop.yarn.event.EventDispatcher$EventProcessor.run(EventDispatcher.java:66)
> {code}
>
> *3. RMWebServices -> QueueACLsManager#checkAccess -> FSQueue.hasAccess -> Server.getRemoteAddress returns NULL.*
> {code:java}
> org.apache.hadoop.yarn.security.ConfiguredYarnAuthorizer.checkPermission(ConfiguredYarnAuthorizer.java:101)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSQueue.hasAccess(FSQueue.java:316)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.checkAccess(FairScheduler.java:1610)
> at org.apache.hadoop.yarn.server.resourcemanager.security.QueueACLsManager.checkAccess(QueueACLsManager.java:84)
> at org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices.hasAccess(RMWebServices.java:270)
> at org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices.getApps(RMWebServices.java:553)
> {code}
>
> Have verified with CapacityScheduler and it works fine.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org