You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Hong Tang (JIRA)" <ji...@apache.org> on 2009/05/13 10:52:45 UTC
[jira] Created: (HADOOP-5817) Preventing node from swapping
Preventing node from swapping
-----------------------------
Key: HADOOP-5817
URL: https://issues.apache.org/jira/browse/HADOOP-5817
Project: Hadoop Core
Issue Type: Improvement
Components: mapred
Reporter: Hong Tang
When a node swaps, it slows everything: maps running on that node, reducers fetching output from the node, and DFS clients reading from the DN. We should just treat it the same way as if OS exhausts memory and kill some tasks to free up memory.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5817) Preventing node from swapping
Posted by "Hong Tang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12708833#action_12708833 ]
Hong Tang commented on HADOOP-5817:
-----------------------------------
Normally, this condition does not happen often. However, we have seen it happens due to various reasons: mis-configured heap size (HADOOP-5725) or processes running on that node not under control by MapReduce.
> Preventing node from swapping
> -----------------------------
>
> Key: HADOOP-5817
> URL: https://issues.apache.org/jira/browse/HADOOP-5817
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred
> Reporter: Hong Tang
>
> When a node swaps, it slows everything: maps running on that node, reducers fetching output from the node, and DFS clients reading from the DN. We should just treat it the same way as if OS exhausts memory and kill some tasks to free up memory.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5817) Preventing node from swapping
Posted by "Hong Tang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12708835#action_12708835 ]
Hong Tang commented on HADOOP-5817:
-----------------------------------
However, when it happens, all jobs that have tasks running on that node, or access data from the slow node will suffer.
> Preventing node from swapping
> -----------------------------
>
> Key: HADOOP-5817
> URL: https://issues.apache.org/jira/browse/HADOOP-5817
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred
> Reporter: Hong Tang
>
> When a node swaps, it slows everything: maps running on that node, reducers fetching output from the node, and DFS clients reading from the DN. We should just treat it the same way as if OS exhausts memory and kill some tasks to free up memory.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.