[jira] [Commented] (YARN-3137) CapacityScheduler.checkAccess unnecessarily grabs the scheduler lock

["Jason Lowe (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/YARN-3137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16006467#comment-16006467 ] 

Jason Lowe commented on YARN-3137:
----------------------------------

Seeing this lock contention quite a bit in 2.8, and I guess I'm still a little confused why we need a full CapacityScheduler lock to do this.  Even if we have a lock, we're either checking just before or just after some kind of hierarchy change.  Queue lookup is not by path but simply a concurrent hashmap lookup, so that's not an issue.   Either the queue is there or it's not.  Then we can invoke checkAccess on the queue itself.  The queue knows its path in the hierarchy, and again either we're checking before the queue is moved or after.  As long as we lock the specific queue when trying to check the ACL we should be fine, but maybe I'm missing something.

Can someone elaborate the scenario where moving the checkAccess lock from the highly-contended CapacityScheduler lock to a queue-specific lock causes problems when the hierarchy changes?

> CapacityScheduler.checkAccess unnecessarily grabs the scheduler lock
> --------------------------------------------------------------------
>
>                 Key: YARN-3137
>                 URL: https://issues.apache.org/jira/browse/YARN-3137
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>    Affects Versions: 2.5.0
>            Reporter: Jason Lowe
>            Assignee: Varun Saxena
>
> The queues are stored in a ConcurrentHashMap and the code already checks for a null queue.  At best we need to lock individual queues when processing the access check, but I don't see why we need to grab the highly-contested scheduler lock to lookup which queue to use for the hasAccess call.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org