You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Ted Dunning (JIRA)" <ji...@apache.org> on 2008/03/18 01:46:25 UTC
[jira] Created: (HADOOP-3034) Need to be able to evacuate a
datanode
Need to be able to evacuate a datanode
--------------------------------------
Key: HADOOP-3034
URL: https://issues.apache.org/jira/browse/HADOOP-3034
Project: Hadoop Core
Issue Type: Improvement
Reporter: Ted Dunning
It would be very helpful if there were some way to evacuate data from one or more nodes.
This scenario arise fairly often when several nodes need to be powered down at nearly the same time. Currently, they can only be taken down a few at a time (n-1 nodes at a time where n is the replication factor) and then you have to wait until all files on these nodes have been replicated.
One implementation would be to be to allow the nodes in question be put into read only mode and mark all blocks on those nodes as not counting as replicants. This should cause the namenode to copy these blocks and as soon as fsck shows no under-replicated files, the nodes will be known to be clear for power-down.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3034) Need to be able to evacuate a
datanode
Posted by "Ted Dunning (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12579700#action_12579700 ]
Ted Dunning commented on HADOOP-3034:
-------------------------------------
Yes it does look like a decommission feature.
(you say potato, I say potahto, you say decomission, I say evacuate)
Let's call the whole thing off.
> Need to be able to evacuate a datanode
> --------------------------------------
>
> Key: HADOOP-3034
> URL: https://issues.apache.org/jira/browse/HADOOP-3034
> Project: Hadoop Core
> Issue Type: Improvement
> Reporter: Ted Dunning
>
> It would be very helpful if there were some way to evacuate data from one or more nodes.
> This scenario arise fairly often when several nodes need to be powered down at nearly the same time. Currently, they can only be taken down a few at a time (n-1 nodes at a time where n is the replication factor) and then you have to wait until all files on these nodes have been replicated.
> One implementation would be to be to allow the nodes in question be put into read only mode and mark all blocks on those nodes as not counting as replicants. This should cause the namenode to copy these blocks and as soon as fsck shows no under-replicated files, the nodes will be known to be clear for power-down.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3034) Need to be able to evacuate a
datanode
Posted by "Konstantin Shvachko (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12579695#action_12579695 ]
Konstantin Shvachko commented on HADOOP-3034:
---------------------------------------------
Sounds like decommission feature.
http://wiki.apache.org/hadoop/FAQ#17
> Need to be able to evacuate a datanode
> --------------------------------------
>
> Key: HADOOP-3034
> URL: https://issues.apache.org/jira/browse/HADOOP-3034
> Project: Hadoop Core
> Issue Type: Improvement
> Reporter: Ted Dunning
>
> It would be very helpful if there were some way to evacuate data from one or more nodes.
> This scenario arise fairly often when several nodes need to be powered down at nearly the same time. Currently, they can only be taken down a few at a time (n-1 nodes at a time where n is the replication factor) and then you have to wait until all files on these nodes have been replicated.
> One implementation would be to be to allow the nodes in question be put into read only mode and mark all blocks on those nodes as not counting as replicants. This should cause the namenode to copy these blocks and as soon as fsck shows no under-replicated files, the nodes will be known to be clear for power-down.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (HADOOP-3034) Need to be able to evacuate a
datanode
Posted by "Nigel Daley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Nigel Daley resolved HADOOP-3034.
---------------------------------
Resolution: Won't Fix
> Need to be able to evacuate a datanode
> --------------------------------------
>
> Key: HADOOP-3034
> URL: https://issues.apache.org/jira/browse/HADOOP-3034
> Project: Hadoop Core
> Issue Type: Improvement
> Reporter: Ted Dunning
>
> It would be very helpful if there were some way to evacuate data from one or more nodes.
> This scenario arise fairly often when several nodes need to be powered down at nearly the same time. Currently, they can only be taken down a few at a time (n-1 nodes at a time where n is the replication factor) and then you have to wait until all files on these nodes have been replicated.
> One implementation would be to be to allow the nodes in question be put into read only mode and mark all blocks on those nodes as not counting as replicants. This should cause the namenode to copy these blocks and as soon as fsck shows no under-replicated files, the nodes will be known to be clear for power-down.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Reopened: (HADOOP-3034) Need to be able to evacuate a
datanode
Posted by "Nigel Daley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Nigel Daley reopened HADOOP-3034:
---------------------------------
> Need to be able to evacuate a datanode
> --------------------------------------
>
> Key: HADOOP-3034
> URL: https://issues.apache.org/jira/browse/HADOOP-3034
> Project: Hadoop Core
> Issue Type: Improvement
> Reporter: Ted Dunning
>
> It would be very helpful if there were some way to evacuate data from one or more nodes.
> This scenario arise fairly often when several nodes need to be powered down at nearly the same time. Currently, they can only be taken down a few at a time (n-1 nodes at a time where n is the replication factor) and then you have to wait until all files on these nodes have been replicated.
> One implementation would be to be to allow the nodes in question be put into read only mode and mark all blocks on those nodes as not counting as replicants. This should cause the namenode to copy these blocks and as soon as fsck shows no under-replicated files, the nodes will be known to be clear for power-down.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (HADOOP-3034) Need to be able to evacuate a
datanode
Posted by "Ted Dunning (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ted Dunning resolved HADOOP-3034.
---------------------------------
Resolution: Fixed
Konstantin is correct that this is a bug in my vocabulary, not a missing feature in Hadoop.
> Need to be able to evacuate a datanode
> --------------------------------------
>
> Key: HADOOP-3034
> URL: https://issues.apache.org/jira/browse/HADOOP-3034
> Project: Hadoop Core
> Issue Type: Improvement
> Reporter: Ted Dunning
>
> It would be very helpful if there were some way to evacuate data from one or more nodes.
> This scenario arise fairly often when several nodes need to be powered down at nearly the same time. Currently, they can only be taken down a few at a time (n-1 nodes at a time where n is the replication factor) and then you have to wait until all files on these nodes have been replicated.
> One implementation would be to be to allow the nodes in question be put into read only mode and mark all blocks on those nodes as not counting as replicants. This should cause the namenode to copy these blocks and as soon as fsck shows no under-replicated files, the nodes will be known to be clear for power-down.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.