You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-issues@hadoop.apache.org by "Miklos Szegedi (JIRA)" <ji...@apache.org> on 2018/05/15 22:10:00 UTC
[jira] [Comment Edited] (YARN-4599) Set OOM control for memory
cgroups
[ https://issues.apache.org/jira/browse/YARN-4599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16476365#comment-16476365 ]
Miklos Szegedi edited comment on YARN-4599 at 5/15/18 10:09 PM:
----------------------------------------------------------------
Thanks for the review [~snemeth].
{quote} [Question] In constructor: If both {{controlPhysicalMemory}} and {{controlVirtualMemory}} is on, you only warn log a line.
{quote}
{{this.controlPhysicalMemory = controlPhysicalMemory && !controlVirtualMemory;}} virtual will override
{quote}G.) In run(): I'm curious whether it can happen that in the while loop's statement, {{events.read}} would read more data than 8 bytes or is it perfectly safe to rely on that on every read, at most 8 bytes will be read?
{quote}
It cannot return more than the buffer size that is 8.
H) I like to have big logic together.
{quote}I.) In run(): {{throw new YarnRuntimeException("OOM was not resolved in time.");}}} --> I would include how much time was spent in the log message for better troubleshooting.
{quote}
The watchdog function handles this.
{quote}B.) In run(): Call to...
{quote}
It does not matter, it wraps into two lines anyways.
{quote}C.) In {{killContainerIfOOM()}}:
{quote}
Not sure what you mean here. Yes there is a conversion, since the request is in MB and the limit is in bytes.
{quote}A.) {{testConstructorHandler():(}}
{quote}
Good catch.
was (Author: miklos.szegedi@cloudera.com):
Thanks for the review [~snemeth].
{quote}{quote} [Question] In constructor: If both {{controlPhysicalMemory}} and {{controlVirtualMemory}} is on, you only warn log a line.
{quote}{quote}
{{this.controlPhysicalMemory = controlPhysicalMemory && !controlVirtualMemory;}} virtual will override
{quote}G.) In run(): I'm curious whether it can happen that in the while loop's statement, {{events.read}} would read more data than 8 bytes or is it perfectly safe to rely on that on every read, at most 8 bytes will be read?
{quote}
It cannot return more than the buffer size that is 8.
H) I like to have big logic together.
{quote}I.) In run(): {{throw new YarnRuntimeException("OOM was not resolved in time.");}}} --> I would include how much time was spent in the log message for better troubleshooting.
{quote}
The watchdog function handles this.
{quote}B.) In run(): Call to...
{quote}
It does not matter, it wraps into two lines anyways.
{quote}C.) In {{killContainerIfOOM()}}:
{quote}
Not sure what you mean here. Yes there is a conversion, since the request is in MB and the limit is in bytes.
{quote}A.) {{testConstructorHandler():(}}
{quote}
Good catch.
> Set OOM control for memory cgroups
> ----------------------------------
>
> Key: YARN-4599
> URL: https://issues.apache.org/jira/browse/YARN-4599
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: nodemanager
> Affects Versions: 2.9.0
> Reporter: Karthik Kambatla
> Assignee: Miklos Szegedi
> Priority: Major
> Labels: oct16-medium
> Attachments: Elastic Memory Control in YARN.pdf, YARN-4599.000.patch, YARN-4599.001.patch, YARN-4599.002.patch, YARN-4599.003.patch, YARN-4599.004.patch, YARN-4599.005.patch, YARN-4599.006.patch, YARN-4599.007.patch, YARN-4599.sandflee.patch, yarn-4599-not-so-useful.patch
>
>
> YARN-1856 adds memory cgroups enforcing support. We should also explicitly set OOM control so that containers are not killed as soon as they go over their usage. Today, one could set the swappiness to control this, but clusters with swap turned off exist.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org