[jira] Commented: (HADOOP-4984) Code to create the UI display string for queues in the Capacity Scheduler needs to be synchronized, and needs to better update its information

["Vivek Ratan (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HADOOP-4984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12661050#action_12661050 ] 

Vivek Ratan commented on HADOOP-4984:
-------------------------------------

The refactoring work in HADOOP-4980 fixes this problem. While  I realize that we don't usually want to clump together more than one fix in a patch, the work in HADOOP-4980 went a long long way in simplifying the fix for this patch, so I didn't create a separate patch here. By making the _SchedulingInfo_ class be functionally 'outside' the scheduler class, and thus unaware of the latter's data structures, and by moving the generation of the display strings to the concerned _QueueSchedulingInfo_ and _TaskSchedulingInfo_ objects, the synchronization problem is easily addressed. we also don't update the QSI objects, preferring to show potentially slightly stale information, but without a performance penalty. 

> Code to create the UI display string for queues in the Capacity Scheduler needs to be synchronized, and needs to better update its information
> ----------------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: HADOOP-4984
>                 URL: https://issues.apache.org/jira/browse/HADOOP-4984
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: contrib/capacity-sched
>            Reporter: Vivek Ratan
>
> There are a couple of problems with _SchedulingInfo.toString()_, the code which creates the UI display string for a queue: 
> * it needs synchronized access to the _QueueSchedulingInfo_ object, as this same object can be updated by the reclaim-capacity thread, and during a heartbeat.
> * the code directly updates its count of running map/reduce tasks. this should be done in a better way, perhaps by calling updateQSIObjects(), rather than walking through the data structures directly. It's also not clear that we want to pay the performance penalty of updating the structures. it maybe OK to provide slightly stale info (the 'staleness' is tiny, in a steady-state and large system, where heartbeats are coming in frequently). 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.