You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-issues@hadoop.apache.org by "Hadoop QA (Jira)" <ji...@apache.org> on 2020/02/26 06:15:00 UTC
[jira] [Commented] (YARN-8789) Add BoundedQueue to AsyncDispatcher
[ https://issues.apache.org/jira/browse/YARN-8789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17045180#comment-17045180 ]
Hadoop QA commented on YARN-8789:
---------------------------------
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 9s{color} | {color:red} YARN-8789 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | YARN-8789 |
| JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12941839/YARN-8789.14.patch |
| Console output | https://builds.apache.org/job/PreCommit-YARN-Build/25577/console |
| Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This message was automatically generated.
> Add BoundedQueue to AsyncDispatcher
> -----------------------------------
>
> Key: YARN-8789
> URL: https://issues.apache.org/jira/browse/YARN-8789
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: applications
> Affects Versions: 3.2.0
> Reporter: David Mollitor
> Assignee: David Mollitor
> Priority: Major
> Attachments: YARN-8789.1.patch, YARN-8789.10.patch, YARN-8789.12.patch, YARN-8789.14.patch, YARN-8789.2.patch, YARN-8789.3.patch, YARN-8789.4.patch, YARN-8789.5.patch, YARN-8789.6.patch, YARN-8789.7.patch, YARN-8789.7.patch, YARN-8789.8.patch, YARN-8789.9.patch
>
>
> I recently came across a scenario where an MR ApplicationMaster was failing with an OOM exception. It had many thousands of Mappers and thousands of Reducers. It was noted that in the logging that the event-queue of {{AsyncDispatcher}} had a very large number of item in it and was seemingly never decreasing.
> I started looking at the code and thought it could use some clean up, simplification, and the ability to specify a bounded queue so that any incoming events are throttled until they can be processed. This will protect the ApplicationMaster from a flood of events.
> Logging Message:
> Size of event-queue is xxx
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org