You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@yunikorn.apache.org by "Weiwei Yang (Jira)" <ji...@apache.org> on 2020/11/26 07:33:00 UTC

[jira] [Created] (YUNIKORN-471) Heuristic cluster auto-scaler backed by YuniKorn

Weiwei Yang created YUNIKORN-471:
------------------------------------

             Summary: Heuristic cluster auto-scaler backed by YuniKorn
                 Key: YUNIKORN-471
                 URL: https://issues.apache.org/jira/browse/YUNIKORN-471
             Project: Apache YuniKorn
          Issue Type: New Feature
          Components: core - scheduler
            Reporter: Weiwei Yang


Currently, the cluster auto-scaler has some limitations. It looks at the pending pods and tries to scale up to satisfy the outstanding requests. This usually causes less efficiency and could significantly increase job level latency.

We had a meeting with [~taoyang], [~Weihao Zheng], [~Zhou Kang], and majorly discussed an optimized solution to handle auto-scaling, a more "heuristic" approach. The rough idea is to create a new auto-scaler component, that leverages the metrics from the YuniKorn scheduler in order to make more efficient upscaling/downscaling decisions.

The idea is to make this component generic enough to plug with different cloud vendors. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@yunikorn.apache.org
For additional commands, e-mail: dev-help@yunikorn.apache.org