You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@yunikorn.apache.org by "Peter Bacsko (Jira)" <ji...@apache.org> on 2023/11/28 16:12:00 UTC

[jira] [Commented] (YUNIKORN-2201) Evaluate the performance impact of Headroom() and CanRunApp()

    [ https://issues.apache.org/jira/browse/YUNIKORN-2201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17790642#comment-17790642 ] 

Peter Bacsko commented on YUNIKORN-2201:
----------------------------------------

The following config was used for the MockScheculer perf test case:
{noformat}
partitions:
  - name: default
    queues:
      - name: root
        submitacl: "*"
        queues:
          - name: a
            limits:
              - limit:
                users:
                - nobody
                maxapplications: 100
                maxresources: {memory: 10T, vcore: 100000}
          - name: b
          - name: c
          - name: d
          - name: e
{noformat}

Three scenarios were checked: no limit hit, maxapplications hit, maxresources hit.

1. No limit hit
Very low impact of Headroom() and CanRunApp(). Throughput is basically unaffected.
 !benchmark_nolimit_hit.png! 

2. "maxapplications" set to 20
Queue "root.a" is expected to run 80 apps. In this case, CanRunApp() failed more often, but the actual cost is still very low.
 !benchmark_maxapps_hit.png! 

3. "maxresources" set to 1000M
Requests for "root.a" sums up to 10000M, so only 10% of it is allowed.
In this case, Headroom() is still not expensive, however the headroom check ({{!userHeadroom.FitInMaxUndef(request.GetAllocatedResource())}}) takes significant amount of CPU time, because every single ask is checked against the user headroom.
 !benchmark_resourcequota_hit.png! 

Both headroom check (user & queue) could be improved:
{noformat}
               if !userHeadroom.FitInMaxUndef(request.GetAllocatedResource()) {
			continue
		}

		// resource must fit in headroom otherwise skip the request (unless preemption could help)
		if !headRoom.FitInMaxUndef(request.GetAllocatedResource()) {
{noformat}


> Evaluate the performance impact of Headroom() and CanRunApp()
> -------------------------------------------------------------
>
>                 Key: YUNIKORN-2201
>                 URL: https://issues.apache.org/jira/browse/YUNIKORN-2201
>             Project: Apache YuniKorn
>          Issue Type: Sub-task
>          Components: core - scheduler
>            Reporter: Peter Bacsko
>            Assignee: Peter Bacsko
>            Priority: Major
>         Attachments: benchmark_maxapps_hit.png, benchmark_nolimit_hit.png, benchmark_resourcequota_hit.png
>
>
> {{Manager.CanRunApp()}} and {{Manager.Headroom()}} are constantly called from the scheduling cycle.
> We need to see how this affects the overall performance of Yunikorn.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@yunikorn.apache.org
For additional commands, e-mail: issues-help@yunikorn.apache.org