[jira] [Commented] (YARN-9916) Improving Async Dispatcher

["Adam Antal (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/YARN-9916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16956952#comment-16956952 ] 

Adam Antal commented on YARN-9916:
----------------------------------

I think this is related (if not the dupe) to YARN-9927.

> Improving Async Dispatcher
> --------------------------
>
>                 Key: YARN-9916
>                 URL: https://issues.apache.org/jira/browse/YARN-9916
>             Project: Hadoop YARN
>          Issue Type: Improvement
>            Reporter: Prashant Golash
>            Assignee: Prashant Golash
>            Priority: Major
>
> Currently, async dispatcher works in the single-threaded model.
>  
> There is another queue for the scheduler handler, but not all handlers are non-blocking. In our cluster, this queue can go sometimes to 16M events, which takes time to drain.
>  
> We should think of improving it:
>  
>  # Either make multi-threads in the dispatcher which will pick queue events, but this would require careful evaluation of the order of events.
>  # Or Make all downstream handlers similar to scheduler queue (this also needs careful evaluation of out of order events).
> Any other ideas are also welcome.
>  
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org