You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@myriad.apache.org by "Juan Pedro (JIRA)" <ji...@apache.org> on 2018/04/13 10:15:00 UTC

[jira] [Commented] (MYRIAD-12) Implement auto scaling

    [ https://issues.apache.org/jira/browse/MYRIAD-12?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16437126#comment-16437126 ] 

Juan Pedro commented on MYRIAD-12:
----------------------------------

Hi. Other ideas:

- Option to limit the scaling according to the time (window for batch jobs).
- Scale by zones, according to constraints, where long-lived processes may not need resources (night time).
- Scale on certain nodes, scaling nodes (according to constraints), where the need for resources varies depending on the time or workload.

> Implement auto scaling
> ----------------------
>
>                 Key: MYRIAD-12
>                 URL: https://issues.apache.org/jira/browse/MYRIAD-12
>             Project: Myriad
>          Issue Type: Improvement
>            Reporter: Santosh Marella
>            Assignee: Santosh Marella
>
> - Pluggable criteria to detect "YARN starvation".. some ideas:
>   - if all NMs are full and more than X number of resource requests waiting in the queue
>   - if all NMs are full and more than Y number of apps are waiting in the queue    
> - How to scale? i.e. "how many" NM instances are required to bring down the starvation
>   -  "detect the extent of starvation" by querying the resource requests in the pipieline and scale up accordingly



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)