You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@yunikorn.apache.org by "Wilfred Spiegelenburg (Jira)" <ji...@apache.org> on 2023/12/04 22:22:00 UTC
[jira] [Resolved] (YUNIKORN-1715) Yunikorn performance improvements
[ https://issues.apache.org/jira/browse/YUNIKORN-1715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Wilfred Spiegelenburg resolved YUNIKORN-1715.
---------------------------------------------
Fix Version/s: 1.4.0
1.3.0
Resolution: Fixed
Closing this jira. Subtask have already been spreak over multiple releases which makes tracking much more difficult. New issues should not be linked into this Jira. If more needs to be done that requires an umbrella Jira we should log a new one.
Single performance enhancements should just be tracked as single improvement jiras.
> Yunikorn performance improvements
> ---------------------------------
>
> Key: YUNIKORN-1715
> URL: https://issues.apache.org/jira/browse/YUNIKORN-1715
> Project: Apache YuniKorn
> Issue Type: Improvement
> Components: core - scheduler
> Reporter: Peter Bacsko
> Assignee: Peter Bacsko
> Priority: Major
> Labels: release-notes
> Fix For: 1.4.0, 1.3.0
>
>
> There are some methods/functions in Yunikorn which are called frequently and often unnecessarily. On a large, busy cluster, eliminating these calls can result in faster scheduling cycle, therefore better throughput.
> In the listed cases below, we can re-use a previously computed value and the expensive copy/sort phase can be eliminated completely.
> {*}Retrieving node iterators{*}: in {{{}baseNodeCollection.getNodeIteratorInternal(){}}}, we always clone the tree of sorted nodes, then we build a slice. The node tree is only modified when a node gets a new score (plus node add/removal). By reusing the sorted list, we avoid cloning an {{*btree.BTree}} structure and creating {{[]*Node}} slices.
> {*}Queue sorting{*}: only need sorting if the following occurred:
> * Allocated resource changed in one of the child queues (most common)
> * Pending resource changed from 0 to "n", or from "n" to 0 (affects filtering)
> * Child queue got stopped (affects filtering)
> * Child queue structure changed on config update
> {*}Application sorting{*}: in {{Queue.TryAllocate()}} and {{{}Queue.TryPlaceholderAllocate(){}}}, {{sortApplications()}} always runs. In every iteration, it calls {{Queue.GetCopyOfApps()}} then proceeds to sort the apps. It only has to run if something relevant happens from the sorting POV:
> * Application added/removed
> * Ask added to an application
> * Ask max priority changed in at least one application
> * Allocated resource changed in at least one application
> {*}Request sorting{*}: request (ask) sorting is only necessary when the following occurs:
> * Ask added
> * pendingAskRepeat gets 0 in an ask
> {*}Misc{*}: we can have a bunch of other stuff that helps performance.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@yunikorn.apache.org
For additional commands, e-mail: dev-help@yunikorn.apache.org