You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "dhruba borthakur (JIRA)" <ji...@apache.org> on 2007/12/04 00:55:43 UTC
[jira] Created: (HADOOP-2340) Limiting memory usage on namenode
Limiting memory usage on namenode
---------------------------------
Key: HADOOP-2340
URL: https://issues.apache.org/jira/browse/HADOOP-2340
Project: Hadoop
Issue Type: Improvement
Components: dfs
Reporter: dhruba borthakur
When memory usage is too high, the namenode could refuse creation of new files. This would still crash applications, but it would keep the filesystem itself from crashing in a way that is hard to recover while folks remove excessive (presumably) small files.
http://java.sun.com/javase/6/docs/api/java/lang/management/MemoryPoolMXBean.html#UsageThreshold
Other resource limitations (other than memory) are CPU, network and disk. We thought that we do not need to monitor those resources. The monitoring of critical resources and the policy of what action to take can be outside the actual Namenode process itself.
There are two reasons that cause memory pressure on the Namenode. One is the creation of a large number of files. This reduces the free memory pool and the GC has to work even harder to recycle memory. The other reason is when a burst of RPCs arrive at the Namenode (especially Block reports). This spurt causes free memory to reduce dramatically within a couple of seconds and makes GC work harder. And we know that when GC runs hard, the server threads in the JVM starve for CPU, causing timeouts on clients.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-2340) Limiting memory usage on namenode
Posted by "dhruba borthakur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-2340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12548056 ]
dhruba borthakur commented on HADOOP-2340:
------------------------------------------
One line of reasoning is that if we never timeout client RPC requests (HADOOP-2188), then the above situation will not occur. A GC run on the Namenode will cause clients to block and slowdown. My feeling is that we should observe the system post-2188 and then decide whether (and policy) we need to monitor Namenode resources.
> Limiting memory usage on namenode
> ---------------------------------
>
> Key: HADOOP-2340
> URL: https://issues.apache.org/jira/browse/HADOOP-2340
> Project: Hadoop
> Issue Type: Improvement
> Components: dfs
> Reporter: dhruba borthakur
>
> When memory usage is too high, the namenode could refuse creation of new files. This would still crash applications, but it would keep the filesystem itself from crashing in a way that is hard to recover while folks remove excessive (presumably) small files.
> http://java.sun.com/javase/6/docs/api/java/lang/management/MemoryPoolMXBean.html#UsageThreshold
> Other resource limitations (other than memory) are CPU, network and disk. We thought that we do not need to monitor those resources. The monitoring of critical resources and the policy of what action to take can be outside the actual Namenode process itself.
> There are two reasons that cause memory pressure on the Namenode. One is the creation of a large number of files. This reduces the free memory pool and the GC has to work even harder to recycle memory. The other reason is when a burst of RPCs arrive at the Namenode (especially Block reports). This spurt causes free memory to reduce dramatically within a couple of seconds and makes GC work harder. And we know that when GC runs hard, the server threads in the JVM starve for CPU, causing timeouts on clients.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.