You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Yoram Arnon (JIRA)" <ji...@apache.org> on 2006/04/22 00:58:05 UTC
[jira] Created: (HADOOP-159) name node at 100% cpu, making
redundant replications
name node at 100% cpu, making redundant replications
----------------------------------------------------
Key: HADOOP-159
URL: http://issues.apache.org/jira/browse/HADOOP-159
Project: Hadoop
Type: Bug
Components: dfs
Versions: 0.1.1
Reporter: Yoram Arnon
Assigned to: Sameer Paranjpye
Priority: Blocker
Fix For: 0.2
some hours after adding some new nodes to the cluster, the name node went into a state where it's consuming 100% cpu.
The log file keeps logging messages of the forms
060421 155049 Obsoleting block blk_8093115169359854355
060421 155049 Pending transfer (block blk_-6965677235456960523) from node1383:50010 to 2 destinations
060421 155049 Block report from node1283:50010: 2140 blocks.
060421 155049 Redundant addStoredBlock request received for block blk_-6836937139917042917 on node node1143:50010
many DFS operations time out, making useful work impossible.
restarting dfs solved the problem for a while, but it came back within an hour.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
[jira] Commented: (HADOOP-159) name node at 100% cpu, making
redundant replications
Posted by "Sameer Paranjpye (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/HADOOP-159?page=comments#action_12377265 ]
Sameer Paranjpye commented on HADOOP-159:
-----------------------------------------
Is this problem still seen? Some of the committed patches appear to have resolved this issue, no?
> name node at 100% cpu, making redundant replications
> ----------------------------------------------------
>
> Key: HADOOP-159
> URL: http://issues.apache.org/jira/browse/HADOOP-159
> Project: Hadoop
> Type: Bug
> Components: dfs
> Versions: 0.1.1
> Reporter: Yoram Arnon
> Assignee: Sameer Paranjpye
> Priority: Blocker
> Fix For: 0.2
>
> some hours after adding some new nodes to the cluster, the name node went into a state where it's consuming 100% cpu.
> The log file keeps logging messages of the forms
> 060421 155049 Obsoleting block blk_8093115169359854355
> 060421 155049 Pending transfer (block blk_-6965677235456960523) from node1383:50010 to 2 destinations
> 060421 155049 Block report from node1283:50010: 2140 blocks.
> 060421 155049 Redundant addStoredBlock request received for block blk_-6836937139917042917 on node node1143:50010
> many DFS operations time out, making useful work impossible.
> restarting dfs solved the problem for a while, but it came back within an hour.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
[jira] Resolved: (HADOOP-159) name node at 100% cpu, making
redundant replications
Posted by "Sameer Paranjpye (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/HADOOP-159?page=all ]
Sameer Paranjpye resolved HADOOP-159:
-------------------------------------
Fix Version: 0.2
(was: 0.3)
Resolution: Duplicate
Looks like HADOOP-178 addresses this. Let's monitor and re-open if it manifests again.
> name node at 100% cpu, making redundant replications
> ----------------------------------------------------
>
> Key: HADOOP-159
> URL: http://issues.apache.org/jira/browse/HADOOP-159
> Project: Hadoop
> Type: Bug
> Components: dfs
> Versions: 0.1.1
> Reporter: Yoram Arnon
> Assignee: Sameer Paranjpye
> Priority: Blocker
> Fix For: 0.2
>
> some hours after adding some new nodes to the cluster, the name node went into a state where it's consuming 100% cpu.
> The log file keeps logging messages of the forms
> 060421 155049 Obsoleting block blk_8093115169359854355
> 060421 155049 Pending transfer (block blk_-6965677235456960523) from node1383:50010 to 2 destinations
> 060421 155049 Block report from node1283:50010: 2140 blocks.
> 060421 155049 Redundant addStoredBlock request received for block blk_-6836937139917042917 on node node1143:50010
> many DFS operations time out, making useful work impossible.
> restarting dfs solved the problem for a while, but it came back within an hour.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
[jira] Updated: (HADOOP-159) name node at 100% cpu, making
redundant replications
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/HADOOP-159?page=all ]
Doug Cutting updated HADOOP-159:
--------------------------------
Fix Version: 0.3
(was: 0.2)
> name node at 100% cpu, making redundant replications
> ----------------------------------------------------
>
> Key: HADOOP-159
> URL: http://issues.apache.org/jira/browse/HADOOP-159
> Project: Hadoop
> Type: Bug
> Components: dfs
> Versions: 0.1.1
> Reporter: Yoram Arnon
> Assignee: Sameer Paranjpye
> Priority: Blocker
> Fix For: 0.3
>
> some hours after adding some new nodes to the cluster, the name node went into a state where it's consuming 100% cpu.
> The log file keeps logging messages of the forms
> 060421 155049 Obsoleting block blk_8093115169359854355
> 060421 155049 Pending transfer (block blk_-6965677235456960523) from node1383:50010 to 2 destinations
> 060421 155049 Block report from node1283:50010: 2140 blocks.
> 060421 155049 Redundant addStoredBlock request received for block blk_-6836937139917042917 on node node1143:50010
> many DFS operations time out, making useful work impossible.
> restarting dfs solved the problem for a while, but it came back within an hour.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
[jira] Commented: (HADOOP-159) name node at 100% cpu, making
redundant replications
Posted by "Yoram Arnon (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/HADOOP-159?page=comments#action_12377280 ]
Yoram Arnon commented on HADOOP-159:
------------------------------------
It was observed just twice, in the same timeframe.
I'm not aware of any patches that attempt to address this. Logging was added though.
Let's wait a while before closing, unless there's a specific fix.
> name node at 100% cpu, making redundant replications
> ----------------------------------------------------
>
> Key: HADOOP-159
> URL: http://issues.apache.org/jira/browse/HADOOP-159
> Project: Hadoop
> Type: Bug
> Components: dfs
> Versions: 0.1.1
> Reporter: Yoram Arnon
> Assignee: Sameer Paranjpye
> Priority: Blocker
> Fix For: 0.2
>
> some hours after adding some new nodes to the cluster, the name node went into a state where it's consuming 100% cpu.
> The log file keeps logging messages of the forms
> 060421 155049 Obsoleting block blk_8093115169359854355
> 060421 155049 Pending transfer (block blk_-6965677235456960523) from node1383:50010 to 2 destinations
> 060421 155049 Block report from node1283:50010: 2140 blocks.
> 060421 155049 Redundant addStoredBlock request received for block blk_-6836937139917042917 on node node1143:50010
> many DFS operations time out, making useful work impossible.
> restarting dfs solved the problem for a while, but it came back within an hour.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira