[jira] [Commented] (YARN-3091) [Umbrella] Improve and fix locks of RM scheduler

["zhengchenyu (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/YARN-3091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15760613#comment-15760613 ] 

zhengchenyu commented on YARN-3091:
-----------------------------------

Sorry, although this issue is completed, I still wanna ask for advice.
I think the improvement of performance is not enough, if used read-write lock. Though read-write lock is not better than synchronized, and solved the problem of dead lock, but it's coarse-grained.
For example, FairScheduler.update() hold a write block, but it still block the FairScheduler.checkAccess(), we know most code of FairScheduler.update() doesn't matter FairScheduler.checkAccess().
I think we should associate the lock with resources instead of function. For exmaple, demand is resource of the queue, we could operate it by CAS.
So can we optimize with other way?

> [Umbrella] Improve and fix locks of RM scheduler
> ------------------------------------------------
>
>                 Key: YARN-3091
>                 URL: https://issues.apache.org/jira/browse/YARN-3091
>             Project: Hadoop YARN
>          Issue Type: Task
>          Components: capacityscheduler, fairscheduler, resourcemanager, scheduler
>            Reporter: Wangda Tan
>
> In existing YARN RM scheduler, there're some issues of using locks. For example:
> - Many unnecessary synchronized locks, we have seen several cases recently that too frequent access of scheduler makes scheduler hang. Which could be addressed by using read/write lock. Components include scheduler, CS queues, apps
> - Some fields not properly locked (Like clusterResource)
> We can address them together in this ticket.
> (More details see comments below)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org