You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Wendy Chien (JIRA)" <ji...@apache.org> on 2006/09/07 04:02:23 UTC
[jira] Created: (HADOOP-514) namenode heartbeat interval should be
configurable
namenode heartbeat interval should be configurable
--------------------------------------------------
Key: HADOOP-514
URL: http://issues.apache.org/jira/browse/HADOOP-514
Project: Hadoop
Issue Type: Bug
Components: dfs
Reporter: Wendy Chien
The namenode heartbeat interval is currently a constant in FSConstants. It should be configurable instead. Things which currently rely on the heartbeat occuring fairly frequently should be disassociated.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HADOOP-514) namenode heartbeat interval should
be configurable
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/HADOOP-514?page=comments#action_12433453 ]
Doug Cutting commented on HADOOP-514:
-------------------------------------
I agree that the heartbeat should be configurable. Setting it automatically would be nice. But we must not be careful to let the heartbeat interval get too long, as other actions are dependent on it. Currently block replications and deletions ride on heartbeat responses. So the heartbeat interval determines the speed of re-replication and deletion.
> namenode heartbeat interval should be configurable
> --------------------------------------------------
>
> Key: HADOOP-514
> URL: http://issues.apache.org/jira/browse/HADOOP-514
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Reporter: Wendy Chien
>
> The namenode heartbeat interval is currently a constant in FSConstants. It should be configurable instead. Things which currently rely on the heartbeat occuring fairly frequently should be disassociated.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HADOOP-514) namenode heartbeat interval should
be configurable
Posted by "Yoram Arnon (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/HADOOP-514?page=comments#action_12433334 ]
Yoram Arnon commented on HADOOP-514:
------------------------------------
using the analogy of the animal kingdom - the larger the beast, the slower the heartbeat, it would perhaps be optimal if the heartbeat interval were proportional to the size of the cluster, rather than configurable.
One less config item, one less thing to get wrong.
namenode could inform datanodes what the cluster size is.
A formula such as interval=max(nodes/10, 1) seconds could work.
> namenode heartbeat interval should be configurable
> --------------------------------------------------
>
> Key: HADOOP-514
> URL: http://issues.apache.org/jira/browse/HADOOP-514
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Reporter: Wendy Chien
>
> The namenode heartbeat interval is currently a constant in FSConstants. It should be configurable instead. Things which currently rely on the heartbeat occuring fairly frequently should be disassociated.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Assigned: (HADOOP-514) namenode heartbeat interval should be
configurable
Posted by "Milind Bhandarkar (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/HADOOP-514?page=all ]
Milind Bhandarkar reassigned HADOOP-514:
----------------------------------------
Assignee: Milind Bhandarkar
> namenode heartbeat interval should be configurable
> --------------------------------------------------
>
> Key: HADOOP-514
> URL: http://issues.apache.org/jira/browse/HADOOP-514
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Reporter: Wendy Chien
> Assigned To: Milind Bhandarkar
>
> The namenode heartbeat interval is currently a constant in FSConstants. It should be configurable instead. Things which currently rely on the heartbeat occuring fairly frequently should be disassociated.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HADOOP-514) namenode heartbeat interval should
be configurable
Posted by "Yoram Arnon (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/HADOOP-514?page=comments#action_12433496 ]
Yoram Arnon commented on HADOOP-514:
------------------------------------
agreed, the dfs needs to be responsive.
but the larger the system, the more frequently datanodes hit the name node.
a block that awaits re-replication will get replicated to the first node that sends a heartbeat and makes sense as a replication destination. Overall the system is very responsive even though each individual data node sends infrequent heartbeats.
as for deletions, they can wait a bit, especially with undelete functionality in mind.
The formula max(nodes/10, 1) maintains heartbeat rate at a constant 10 per second at the namenode. We can tweak the denominator to change that rate, but this seems like a good rate to get responsiveness, while limiting the load on the namenode.
> namenode heartbeat interval should be configurable
> --------------------------------------------------
>
> Key: HADOOP-514
> URL: http://issues.apache.org/jira/browse/HADOOP-514
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Reporter: Wendy Chien
>
> The namenode heartbeat interval is currently a constant in FSConstants. It should be configurable instead. Things which currently rely on the heartbeat occuring fairly frequently should be disassociated.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HADOOP-514) namenode heartbeat interval should
be configurable
Posted by "Milind Bhandarkar (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/HADOOP-514?page=comments#action_12441304 ]
Milind Bhandarkar commented on HADOOP-514:
------------------------------------------
Work started by milind.
> namenode heartbeat interval should be configurable
> --------------------------------------------------
>
> Key: HADOOP-514
> URL: http://issues.apache.org/jira/browse/HADOOP-514
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Reporter: Wendy Chien
> Assigned To: Milind Bhandarkar
>
> The namenode heartbeat interval is currently a constant in FSConstants. It should be configurable instead. Things which currently rely on the heartbeat occuring fairly frequently should be disassociated.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (HADOOP-514) namenode heartbeat interval should be
configurable
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/HADOOP-514?page=all ]
Doug Cutting updated HADOOP-514:
--------------------------------
Status: Resolved (was: Patch Available)
Resolution: Fixed
I just committed this. Thanks, Milind!
> namenode heartbeat interval should be configurable
> --------------------------------------------------
>
> Key: HADOOP-514
> URL: http://issues.apache.org/jira/browse/HADOOP-514
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.7.0
> Reporter: Wendy Chien
> Assigned To: Milind Bhandarkar
> Fix For: 0.8.0
>
> Attachments: heartbeat.patch
>
>
> The namenode heartbeat interval is currently a constant in FSConstants. It should be configurable instead. Things which currently rely on the heartbeat occuring fairly frequently should be disassociated.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (HADOOP-514) namenode heartbeat interval should be
configurable
Posted by "Milind Bhandarkar (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/HADOOP-514?page=all ]
Milind Bhandarkar updated HADOOP-514:
-------------------------------------
Status: Patch Available (was: Open)
Fix Version/s: 0.8.0
Affects Version/s: 0.7.0
Attached patch.
> namenode heartbeat interval should be configurable
> --------------------------------------------------
>
> Key: HADOOP-514
> URL: http://issues.apache.org/jira/browse/HADOOP-514
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.7.0
> Reporter: Wendy Chien
> Assigned To: Milind Bhandarkar
> Fix For: 0.8.0
>
> Attachments: heartbeat.patch
>
>
> The namenode heartbeat interval is currently a constant in FSConstants. It should be configurable instead. Things which currently rely on the heartbeat occuring fairly frequently should be disassociated.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (HADOOP-514) namenode heartbeat interval should be
configurable
Posted by "Milind Bhandarkar (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/HADOOP-514?page=all ]
Milind Bhandarkar updated HADOOP-514:
-------------------------------------
Attachment: heartbeat.patch
Mde heartbeat configurable, in seconds, though.
> namenode heartbeat interval should be configurable
> --------------------------------------------------
>
> Key: HADOOP-514
> URL: http://issues.apache.org/jira/browse/HADOOP-514
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.7.0
> Reporter: Wendy Chien
> Assigned To: Milind Bhandarkar
> Fix For: 0.8.0
>
> Attachments: heartbeat.patch
>
>
> The namenode heartbeat interval is currently a constant in FSConstants. It should be configurable instead. Things which currently rely on the heartbeat occuring fairly frequently should be disassociated.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira