[jira] [Commented] (YUNIKORN-1715) Yunikorn performance improvements

["Arun (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/YUNIKORN-1715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17773635#comment-17773635 ] 

Arun commented on YUNIKORN-1715:
--------------------------------

Hi,
I am interested in helping, let me know if there is anything I can pickup.

> Yunikorn performance improvements
> ---------------------------------
>
>                 Key: YUNIKORN-1715
>                 URL: https://issues.apache.org/jira/browse/YUNIKORN-1715
>             Project: Apache YuniKorn
>          Issue Type: Improvement
>          Components: core - scheduler
>            Reporter: Peter Bacsko
>            Assignee: Peter Bacsko
>            Priority: Major
>              Labels: release-notes
>
> There are some methods/functions in Yunikorn which are called frequently and often unnecessarily. On a large, busy cluster, eliminating these calls can result in faster scheduling cycle, therefore better throughput.
> In the listed cases below, we can re-use a previously computed value and the expensive copy/sort phase can be eliminated completely.
> {*}Retrieving node iterators{*}: in {{{}baseNodeCollection.getNodeIteratorInternal(){}}}, we always clone the tree of sorted nodes, then we build a slice. The node tree is only modified when a node gets a new score (plus node add/removal). By reusing the sorted list, we avoid cloning an {{*btree.BTree}} structure and creating {{[]*Node}} slices.
> {*}Queue sorting{*}: only need sorting if the following occurred:
>  * Allocated resource changed in one of the child queues (most common)
>  * Pending resource changed from 0 to "n", or from "n" to 0 (affects filtering)
>  * Child queue got stopped (affects filtering)
>  * Child queue structure changed on config update
> {*}Application sorting{*}: in {{Queue.TryAllocate()}} and {{{}Queue.TryPlaceholderAllocate(){}}}, {{sortApplications()}} always runs. In every iteration, it calls {{Queue.GetCopyOfApps()}} then proceeds to sort the apps. It only has to run if something relevant happens from the sorting POV:
>  * Application added/removed
>  * Ask added to an application
>  * Ask max priority changed in at least one application
>  * Allocated resource changed in at least one application
> {*}Request sorting{*}: request (ask) sorting is only necessary when the following occurs:
>  * Ask added
>  * pendingAskRepeat gets 0 in an ask
> {*}Misc{*}: we can have a bunch of other stuff that helps performance.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@yunikorn.apache.org
For additional commands, e-mail: issues-help@yunikorn.apache.org