[jira] [Updated] (YUNIKORN-21) Revisit node sorting algorithm for fairness

["Tao Yang (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/YUNIKORN-21?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Tao Yang updated YUNIKORN-21:
-----------------------------
    Attachment: Improve node sorting algorithm v1.pdf

> Revisit node sorting algorithm for fairness
> -------------------------------------------
>
>                 Key: YUNIKORN-21
>                 URL: https://issues.apache.org/jira/browse/YUNIKORN-21
>             Project: Apache YuniKorn
>          Issue Type: Improvement
>          Components: core - scheduler
>            Reporter: Wangda Tan
>            Priority: Major
>         Attachments: Improve node sorting algorithm v1.pdf
>
>
> Currently, we're using DominantRatio for the node sorting algorithm
> {code:java}
> func CompUsageShares(left, right *Resource) int {
>  lshares := getShares(left,nil) rshares := getShares(right,nil)
>  return compareShares(lshares, rshares) 
> }{code}
> Which is not good, two reasons:
>  # Dominate resource compare is about 8X more expensive than single float compares for two resource types.
>  # Dominate resource is not stable when we have scarce resource types like GPU. A node with 192GB mem, 32 vcores, and 1 GPU available, compared to 168GB mem, 64 vcore and 8 GPU available; the prior one can go first because of the following logic:
> {code:java}
> if total == nil || total.Resources[k] == 0 {
>  // negative share is logged
>  if v < 0 {
>   log.Logger().Debug("usage is negative no total, share is also negative", zap.Int64("resource quantity", int64(v))) 
>  }
>  shares[idx] = float64(v) idx++ continue
> }{code}
> I think we should discard dominate resource compare for node resource. Instead, we just use one resource type (like vcores) to compare available resource.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@yunikorn.apache.org
For additional commands, e-mail: issues-help@yunikorn.apache.org