You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-dev@hadoop.apache.org by "Scott Chen (JIRA)" <ji...@apache.org> on 2009/11/08 10:04:32 UTC

[jira] Resolved: (MAPREDUCE-1181) Enforce RSS memory limit in TaskMemoryManagerThread

     [ https://issues.apache.org/jira/browse/MAPREDUCE-1181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Scott Chen resolved MAPREDUCE-1181.
-----------------------------------

    Resolution: Invalid

> Enforce RSS memory limit in TaskMemoryManagerThread
> ---------------------------------------------------
>
>                 Key: MAPREDUCE-1181
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1181
>             Project: Hadoop Map/Reduce
>          Issue Type: New Feature
>          Components: tasktracker
>    Affects Versions: 0.20.1
>            Reporter: Scott Chen
>             Fix For: 0.20.1
>
>         Attachments: MAPREDUCE-1181.patch
>
>
> TaskMemoryManagerThread will periodically check the rss memory usage of every task. If the memory usage exceeds the specified threshold, the task will be killed. Also if the total rss memory of all tasks exceeds (total amount of memory - specified reserved memory). The task with least progress will be killed to recover the reserved rss memory.
> This is similar to the virtual memory limit provided by TaskMemoryManagerThread. But now the limit is for rss memory. This new feature allow us to avoid page swapping which is prone to error.
> The following are the related configurations
> mapreduce.reduce.memory.rss.mb   // RSS memory allowed for a reduce task
> mapreduce.map.memory.rss.mb       // RSS memory allowed for a map task
> mapreduce.tasktracker.reserved.memory.rss.mb     // RSS memory reserved (not for tasks) on a tasktracker

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.