[jira] [Created] (YARN-9916) Improving Async Dispatcher

["Prashant Golash (Jira)" <ji...@apache.org>]

Prashant Golash created YARN-9916:
-------------------------------------

             Summary: Improving Async Dispatcher
                 Key: YARN-9916
                 URL: https://issues.apache.org/jira/browse/YARN-9916
             Project: Hadoop YARN
          Issue Type: Improvement
            Reporter: Prashant Golash


Currently, async dispatcher works in the single-threaded model.

 

There is another queue for the scheduler handler, but not all handlers are non-blocking. In our cluster, this queue can go sometimes to 16M events, which takes time to drain.

 

We should think of improving it:

 
 # Either make multi-threads in the dispatcher which will pick queue events, but this would require careful evaluation of the order of events.
 # Or Make all downstream handlers similar to scheduler queue (this also needs careful evaluation of out of order events).

Any other ideas are also welcome.

 

 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org