[jira] [Commented] (YARN-473) Capacity Scheduler webpage and REST API not showing correct number of pending applications

["Raju Bairishetti (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/YARN-473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14659695#comment-14659695 ] 

Raju Bairishetti commented on YARN-473:
---------------------------------------

REST API is showing wrong number of pending applications as it was populating the value from *pendingApplications set* (i.e. size of pendingApplications set). IMO, it should get the value from Queue metrics instead of getting from in-memory data structures.

LeafQueue : 
{noformat}
1) *pendingApplications* : all submitted applications will be added to this set first. 

2) *activeApplications* :  applications added to this set from pendingApplications set if the number of active applications are lesser than maximum number of active applications. 
{noformat}

Jmx metrics showing appsPending metric properly. AppsPending will be incremented when the application is submitted to Queue and decremented only when the application is actually launched(i.e. allocated some resources to it). 

IMO, Rest call for queue info should also use queue metrics instead of depending on other data structures. Applications from pendingApplications set will be removed before the launching of application.

pendingApplciations set contains the applications which are not ready for schedulable at this moment.

One more interesting fact, CapacitySchedulerPage(in UI) has "Num Non-Schedulable Applications" and  getting the value form pendingApplications set. Rest API call is correct in this case.

 I am thinking couple of approaches to fix this issue:
{noformat}
1) Rename the pendingApplications to nonSchedulableApplications in CapacitySchedulerLeafQueueInfo class and introduce new fields (pendingApplications) in the CapacitySchedulerLeafQueueInfo and get this value from QueueMetrics.
  
     refactor activeApplications to schedulableApplications and introducing a new field(activeApplications) for the same which tells exactly how many AMs are there.

2. Refactor "Num Non-Schedulable Applications:" to "Num of Pending Applications"  and  refactor "Num Schedulable Applications" to activeApplications. Update properly through QueueMetrics instead of depending on in-memory data structures in the LeafQueue to get the values.
{noformat}


> Capacity Scheduler webpage and REST API not showing correct number of pending applications
> ------------------------------------------------------------------------------------------
>
>                 Key: YARN-473
>                 URL: https://issues.apache.org/jira/browse/YARN-473
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: capacityscheduler
>    Affects Versions: 0.23.6
>            Reporter: Kendall Thrapp
>            Assignee: Mit Desai
>              Labels: usability
>
> The Capacity Scheduler REST API (http://hadoop.apache.org/docs/r0.23.6/hadoop-yarn/hadoop-yarn-site/ResourceManagerRest.html#Cluster_Scheduler_API) is not returning the correct number of pending applications.  numPendingApplications is almost always zero, even if there are dozens of pending apps.
> In investigating this, I discovered that the Resource Manager's Scheduler webpage is also showing an incorrect but different number of pending applications.  For example, the cluster I'm looking at right now currently has 15 applications in the ACCEPTED state, but the Cluster Metrics table near the top of the page says there are only 2 pending apps.  The REST API says there are zero pending apps.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)