[jira] [Assigned] (YARN-9916) Improving Async Dispatcher

["Prashant Golash (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/YARN-9916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Prashant Golash reassigned YARN-9916:
-------------------------------------

    Assignee: Prashant Golash

> Improving Async Dispatcher
> --------------------------
>
>                 Key: YARN-9916
>                 URL: https://issues.apache.org/jira/browse/YARN-9916
>             Project: Hadoop YARN
>          Issue Type: Improvement
>            Reporter: Prashant Golash
>            Assignee: Prashant Golash
>            Priority: Major
>
> Currently, async dispatcher works in the single-threaded model.
>  
> There is another queue for the scheduler handler, but not all handlers are non-blocking. In our cluster, this queue can go sometimes to 16M events, which takes time to drain.
>  
> We should think of improving it:
>  
>  # Either make multi-threads in the dispatcher which will pick queue events, but this would require careful evaluation of the order of events.
>  # Or Make all downstream handlers similar to scheduler queue (this also needs careful evaluation of out of order events).
> Any other ideas are also welcome.
>  
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org