You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-issues@hadoop.apache.org by "Wangda Tan (JIRA)" <ji...@apache.org> on 2016/09/08 19:55:20 UTC
[jira] [Comment Edited] (YARN-5296) NMs going OutOfMemory because
ContainerMetrics leak in ContainerMonitorImpl
[ https://issues.apache.org/jira/browse/YARN-5296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15474832#comment-15474832 ]
Wangda Tan edited comment on YARN-5296 at 9/8/16 7:54 PM:
----------------------------------------------------------
[~djp]/[~jianhe],
I'm trying to understand why minimal of YARN-1643 is required for branch-2.7:
When the STOP_MONITORING_CONTAINER is called, container will be added to containersToBeRemoved, and in the running thread, all containers in the containersToBeRemoved will be called:
{code}
ContainerMetrics.forContainer(
containerId, containerMetricsPeriodMs,
containerMetricsUnregisterDelayMs).finished();
{code}
It seems to me there's no issue here, please comment if you think different.
Thanks,
was (Author: leftnoteasy):
[~djp]/[~jianhe],
I'm trying to understand why minimal of YARN-1643 is required:
When the STOP_MONITORING_CONTAINER is called, container will be added to containersToBeRemoved, and in the running thread, all containers in the containersToBeRemoved will be called:
{code}
ContainerMetrics.forContainer(
containerId, containerMetricsPeriodMs,
containerMetricsUnregisterDelayMs).finished();
{code}
It seems to me there's no issue here, please comment if you think different.
Thanks,
> NMs going OutOfMemory because ContainerMetrics leak in ContainerMonitorImpl
> ---------------------------------------------------------------------------
>
> Key: YARN-5296
> URL: https://issues.apache.org/jira/browse/YARN-5296
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
> Affects Versions: 2.9.0
> Reporter: Karam Singh
> Assignee: Junping Du
> Fix For: 2.9.0, 3.0.0-alpha1
>
> Attachments: YARN-5296-v2.1.patch, YARN-5296-v2.patch, YARN-5296.patch, after v2 fix.png, before v2 fix.png
>
>
> Ran tests in following manner,
> 1. Run GridMix of 768 sequestionally around 17 times to execute about 12.9K apps.
> 2. After 4-5hrs take Check NM Heap using Memory Analyser. It report around 96% Heap is being used my ContainerMetrics
> 3. Run 7 more GridMix run for have around 18.2apps ran in total. Again check NM heap using Memory Analyser again 96% heap is being used by ContainerMetrics.
> 4. Start one more grimdmix run, while run going on , NMs started going down with OOM, around running 18.7K+, On analysing NM heap using Memory analyser, OOM was caused by ContainerMetrics
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org