You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@yunikorn.apache.org by "Peter Bacsko (Jira)" <ji...@apache.org> on 2025/11/03 12:11:00 UTC

[jira] [Commented] (YUNIKORN-3148) Incorrect headroom calculation when collecting outstanding requests

    [ https://issues.apache.org/jira/browse/YUNIKORN-3148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18035035#comment-18035035 ] 

Peter Bacsko commented on YUNIKORN-3148:
----------------------------------------

Based on my analysis so far, what happens to {{userHeadroom}} is not a concern, at least when it comes to modifications. We got a copy from {{Manager.Headroom()}} anyway and this is new for each application.

However, we do need to think about whether this is correct or not. Calling code:
{noformat}
func (sq *Queue) GetQueueOutstandingRequests(total *[]*Allocation) {
    if sq.IsLeafQueue() {
       headRoom := sq.getMaxHeadRoom()
       // while calculating outstanding requests, we calculate all the requests that can fit into the queue's headroom,
       // all these requests are qualified to trigger the up scaling.
       for _, app := range sq.sortApplications(false) {
          // calculate the users' headroom
          userHeadroom := ugm.GetUserManager().Headroom(app.queuePath, app.ApplicationID, app.user)  <-- always new object, even if user is the same
          app.getOutstandingRequests(headRoom, userHeadroom, total)
       }{noformat}
I do think we need to save the {{userHeadroom}} object for every user in a map and then re-use it when needed:
{noformat}
        userHeadrooms := make(map[string]*resources.Resource)
        for _, app := range sq.sortApplications(false) {
          userHeadroom := userHeadrooms[app.user]
          if userHeadroom  == nil {
            uhr := ugm.GetUserManager().Headroom(app.queuePath, app.ApplicationID, app.user)
            userHeadrooms[app.user] = uhr
            userHeadroom = uhr
          }

          app.getOutstandingRequests(headRoom, userHeadroom, total)
        }
 {noformat}

> Incorrect headroom calculation when collecting outstanding requests
> -------------------------------------------------------------------
>
>                 Key: YUNIKORN-3148
>                 URL: https://issues.apache.org/jira/browse/YUNIKORN-3148
>             Project: Apache YuniKorn
>          Issue Type: Improvement
>          Components: core - scheduler
>            Reporter: Peter Bacsko
>            Assignee: Peter Bacsko
>            Priority: Critical
>
>  YUNIKORN-2794 introduced a bug in the Application code. We're no longer mutating the {{headRoom}} object in-place in {{{}Application.getOutstandingRequests(){}}}, instead, we re-assign its value to a newly created object.
> Before:
> {noformat}
> headRoom.SubOnlyExisting(request.GetAllocatedResource())
> userHeadRoom.SubOnlyExisting(request.GetAllocatedResource()) {noformat}
> After:
> {noformat}
> headRoom = resources.SubOnlyExisting(headRoom, request.GetAllocatedResource())
> userHeadRoom = resources.SubOnlyExisting(userHeadRoom, request.GetAllocatedResource())  {noformat}
> Problem is, this does not change the object pointed to outside the function, so every iteration in {{Queue.GetQueueOutstandingRequests()}} starts with the original {{headRoom}} value for every application. This leads to undesired behavior, because we'll end up collecting more asks than needed, which in turn triggers unnecessary cluster upscale.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@yunikorn.apache.org
For additional commands, e-mail: issues-help@yunikorn.apache.org