[jira] [Updated] (MAPREDUCE-5124) AM lacks flow control for task events

["Tsuyoshi OZAWA (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/MAPREDUCE-5124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Tsuyoshi OZAWA updated MAPREDUCE-5124:
--------------------------------------

    Attachment: MAPREDUCE-5124-prototype.txt

I've designed and prototyped to deal with this problem as follows:

1. AsyncDispatcher should have the capacity of blocking queue to limit memory usage.
2. AsyncDispatcher should have the capacity of handling events per sec to limit CPU usage.

This prototype can only handle 1 in AsyncDispatcher.
If this design is acceptable, I'll create umbrella tickets and continue to implement. Any idea?
                
> AM lacks flow control for task events
> -------------------------------------
>
>                 Key: MAPREDUCE-5124
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5124
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: mr-am
>    Affects Versions: 2.0.3-alpha, 0.23.5
>            Reporter: Jason Lowe
>         Attachments: MAPREDUCE-5124-prototype.txt
>
>
> The AM does not have any flow control to limit the incoming rate of events from tasks.  If the AM is unable to keep pace with the rate of incoming events for a sufficient period of time then it will eventually exhaust the heap and crash.  MAPREDUCE-5043 addressed a major bottleneck for event processing, but the AM could still get behind if it's starved for CPU and/or handling a very large job with tens of thousands of active tasks.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira