You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-issues@hadoop.apache.org by "Arun Suresh (JIRA)" <ji...@apache.org> on 2017/11/24 06:53:00 UTC
[jira] [Commented] (YARN-5276) print more info when event queue is
blocked
[ https://issues.apache.org/jira/browse/YARN-5276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16264987#comment-16264987 ]
Arun Suresh commented on YARN-5276:
-----------------------------------
Thanks [~sandflee] for raising this and for the patch.
Yup, it is definitely useful to find out exactly which events are backed up in the queue.
In our production clusters, we do have a patch this is similar to what you have put up. One suggestion though: instead of iterating an creating a new map everytime (not to mention the performance cost of iterating thru the queue everytime), what we do is to keep a ConcurrentHashMap<EventType, Counter> as a persistent field in the Dispatcher. We then update the counter (or insert/remove an entry if required) everytime an event is inserted or removed from the queue. And we just dump the map if the total size of the queue is greater than some value.
Additionally, it would also be useful to dump the last event that was dispatched - this can give a hint of which handler potentially caused the backup
> print more info when event queue is blocked
> -------------------------------------------
>
> Key: YARN-5276
> URL: https://issues.apache.org/jira/browse/YARN-5276
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: nodemanager, resourcemanager
> Reporter: sandflee
> Assignee: sandflee
> Labels: oct16-easy
> Attachments: YARN-5276.01.patch, YARN-5276.02.patch, YARN-5276.03.patch, YARN-5276.04.patch
>
>
> we now see logs like "Size of event-queue is 498000, Size of event-queue is 499000" and difficult to know which event flood the queue.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org