You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "WenJin Ma (JIRA)" <ji...@apache.org> on 2013/02/01 03:39:12 UTC
[jira] [Commented] (HDFS-4450) Duplicate data node on the name node
after formatting data node
[ https://issues.apache.org/jira/browse/HDFS-4450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13568403#comment-13568403 ]
WenJin Ma commented on HDFS-4450:
---------------------------------
{quote}
Are you seeing duplicate datanode issue? If so, please post the logs. Per HDFS-3224, as long as the ipaddr + xfer port remains the same, even if the storage ID changes, duplicate datanode entry should not occur.
{quote}
1.I deploy a test cluster，have 2 namenodes(active/standby), 3 datanodes.
Under normal circumstances datanodes such as normal.bmp
2.Fomart datanode dn0 and restart it. hdfs web UI such as exception.bmp.
3.Namenode not found opteration for remove old datanode.
2013-02-01 10:11:10,522 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* NameSystem.registerDatanode: node registration from DatanodeRegistration(10.28.176.234, storageID=DS-685519412-10.28.176.234-50010-1359684666375, infoPort=50075, ipcPort=50020, storageInfo=lv=-40;cid=CID-fe3b5079-a34a-4912-b8a8-50443d038749;nsid=1321646662;c=0) storage DS-685519412-10.28.176.234-50010-1359684666375
2013-02-01 10:11:10,555 INFO org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: BLOCK* processReport: Received first block report from 10.28.176.234:50010 after becoming active. Its block contents are no longer considered stale.
2013-02-01 10:11:10,555 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* processReport: from DatanodeRegistration(10.28.176.234, storageID=DS-685519412-10.28.176.234-50010-1359684666375, infoPort=50075, ipcPort=50020, storageInfo=lv=-40;cid=CID-fe3b5079-a34a-4912-b8a8-50443d038749;nsid=1321646662;c=0), blocks: 0, processing time: 0 msecs
> Duplicate data node on the name node after formatting data node
> ---------------------------------------------------------------
>
> Key: HDFS-4450
> URL: https://issues.apache.org/jira/browse/HDFS-4450
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: namenode
> Affects Versions: 2.0.2-alpha
> Reporter: WenJin Ma
> Attachments: exception.bmp, normal.bmp
>
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> Duplicate data node on the name node after formatting data node。
> When we registered data node，use nodeReg.getXferPort() to find DatanodeDescriptor.
> {code}
> DatanodeDescriptor nodeN = host2DatanodeMap.getDatanodeByXferAddr(
> nodeReg.getIpAddr(), nodeReg.getXferPort());
> {code}
> but add data node use node.getIpAddr().
> {code}
> /** add node to the map
> * return true if the node is added; false otherwise.
> */
> boolean add(DatanodeDescriptor node) {
> hostmapLock.writeLock().lock();
> try {
> if (node==null || contains(node)) {
> return false;
> }
>
> String ipAddr = node.getIpAddr();
> DatanodeDescriptor[] nodes = map.get(ipAddr);
> DatanodeDescriptor[] newNodes;
> if (nodes==null) {
> newNodes = new DatanodeDescriptor[1];
> newNodes[0]=node;
> } else { // rare case: more than one datanode on the host
> newNodes = new DatanodeDescriptor[nodes.length+1];
> System.arraycopy(nodes, 0, newNodes, 0, nodes.length);
> newNodes[nodes.length] = node;
> }
> map.put(ipAddr, newNodes);
> return true;
> } finally {
> hostmapLock.writeLock().unlock();
> }
> }
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira