[jira] [Assigned] (YARN-9656) Plugin to avoid scheduling jobs on node which are not in "schedulable" state, but are healthy otherwise.

["Prashant Golash (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/YARN-9656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Prashant Golash reassigned YARN-9656:
-------------------------------------

    Assignee: Prashant Golash

> Plugin to avoid scheduling jobs on node which are not in "schedulable" state, but are healthy otherwise.
> --------------------------------------------------------------------------------------------------------
>
>                 Key: YARN-9656
>                 URL: https://issues.apache.org/jira/browse/YARN-9656
>             Project: Hadoop YARN
>          Issue Type: Improvement
>          Components: nodemanager, resourcemanager
>    Affects Versions: 2.9.1, 3.1.2
>            Reporter: Prashant Golash
>            Assignee: Prashant Golash
>            Priority: Major
>
> Creating this Jira to get idea from the community if this is something helpful which can be done in YARN. Some times the nodes go in a bad state for e.g. (H/W problem: I/O is bad; Fan problem). In some other scenarios, if CGroup is not enabled, nodes may be running very high on CPU and the jobs scheduled on them will suffer.
>  
> The idea is three-fold:
>  # Gather relevant metrics from node-managers and put in some form (for e.g. exclude file).
>  # RM loads the files and put the nodes as part of the blacklist.
>  # Once the node becomes good, they can again be put in the whitelist.
> Various optimizations can be done here, but I would like to understand if this is something which could be helpful as an upstream feature in YARN.
>  
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org