You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-issues@hadoop.apache.org by "Bibin A Chundatt (JIRA)" <ji...@apache.org> on 2019/06/11 13:15:00 UTC

[jira] [Updated] (YARN-9618) NodeListManager event improvement

     [ https://issues.apache.org/jira/browse/YARN-9618?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Bibin A Chundatt updated YARN-9618:
-----------------------------------
    Description: 
Current implementation nodelistmanager event blocks async dispacher and can cause RM crash and slowing down event processing.


# Cluster restart with 1K running apps . Each usable event will create 1K events over all events could be 5k*1k events for 5K cluster
# Event processing is blocked till new events are added to queue.

Solution :

# Add another async Event handler similar to scheduler.
# Instead of adding events to dispatcher directly call RMApp event handler.



  was:
Current implementation nodelistmanager event blocks async dispacher and can cause RM crash and slowing down event processing.


# Cluster restart with 1K running apps . Each usable event will create 1K events over all events could be 1k*1k events
# Event processing is blocked till new events are added to queue.

Solution :

# Add another async Event handler similar to scheduler.
# Instead of adding events to dispatcher directly call RMApp event handler.




> NodeListManager event improvement
> ---------------------------------
>
>                 Key: YARN-9618
>                 URL: https://issues.apache.org/jira/browse/YARN-9618
>             Project: Hadoop YARN
>          Issue Type: Improvement
>            Reporter: Bibin A Chundatt
>            Priority: Critical
>
> Current implementation nodelistmanager event blocks async dispacher and can cause RM crash and slowing down event processing.
> # Cluster restart with 1K running apps . Each usable event will create 1K events over all events could be 5k*1k events for 5K cluster
> # Event processing is blocked till new events are added to queue.
> Solution :
> # Add another async Event handler similar to scheduler.
> # Instead of adding events to dispatcher directly call RMApp event handler.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org