You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@yunikorn.apache.org by "Weiwei Yang (Jira)" <ji...@apache.org> on 2021/09/01 04:30:00 UTC
[jira] [Commented] (YUNIKORN-831) node sorting should check other
resources of nodes instead of comparing node id directly
[ https://issues.apache.org/jira/browse/YUNIKORN-831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17407807#comment-17407807 ]
Weiwei Yang commented on YUNIKORN-831:
--------------------------------------
Oh. this is a good catch, thank you [~chia7712]
When you say the largest usage on your cluster is storage, what is that specifically? I guess you are not referring to memory.
For the fix here, ideally, we should calculate a node score as a float based on the consideration of all resource types. The fix here seems to calculate "multiple" scores per resource type and use that for comparison. that can solve the problem, but does not seem to be the best solution. It still does not solve the issue I mentioned here: https://github.com/apache/incubator-yunikorn-core/pull/307#discussion_r693313432. this must be fixed for the 1.0 release otherwise our sorting result will be sub-optimal.
cc [~wilfreds], [~sunil.govind@gmail.com] on this, we have some previous discussions, want to make sure we consider all people's opinion for this one.
> node sorting should check other resources of nodes instead of comparing node id directly
> ----------------------------------------------------------------------------------------
>
> Key: YUNIKORN-831
> URL: https://issues.apache.org/jira/browse/YUNIKORN-831
> Project: Apache YuniKorn
> Issue Type: Bug
> Reporter: Chia-Ping Tsai
> Assignee: Chia-Ping Tsai
> Priority: Major
> Labels: pull-request-available
>
> from comment:
> https://issues.apache.org/jira/browse/YUNIKORN-807?focusedCommentId=17407164&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17407164
>
> It seems to me a small behavior change is introduced by this patch. This patch scores node according to `resources.LargestUsageShare`. However, the largest usage in our cluster is always storage (and all nodes have same storage capacity). It results in the order is always based on `node id`. Before this patch, the order will compare remaining "usage" (vcore, memory, etc) when largest usage is equal.
> There are two possible solutions.
> # the nodeScore of nodeRef should keep all usages ([]float64) and the implementation of `Less` function should reuse `resources.CompareShares`
> # add an new config to enable users to control scope. for example, the following config means we should compare vcore instead of largest resource
> {code:java}
> nodesortpolicy:
> type: binpacking
> usage: vcore{code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@yunikorn.apache.org
For additional commands, e-mail: issues-help@yunikorn.apache.org