You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Wei Xu (JIRA)" <ji...@apache.org> on 2008/08/01 00:44:31 UTC
[jira] Created: (HADOOP-3885) Namenode blockMap not updated when
datanode invalidates a block on heart beat
Namenode blockMap not updated when datanode invalidates a block on heart beat
-----------------------------------------------------------------------------
Key: HADOOP-3885
URL: https://issues.apache.org/jira/browse/HADOOP-3885
Project: Hadoop Core
Issue Type: Bug
Components: dfs
Affects Versions: 0.18.0
Reporter: Wei Xu
Here are related logs. dn_10_251_107_213 deleted the block immediately after it received it (because of over-replication). However, the information is not reported to namenode and name node ask this DN to invalidate the block again, causing the error "Unexpected error trying to delete block blk_-1201258788540100829. BlockInfo not found in volumeMap.".
dn_10_251_107_213 | 080722 145554 2497 INFO dfs.DataNode$DataXceiver: Receiving block blk_-1201258788540100829 src: /10.251.67.214:54770 dest: /10.251.67.214:50010
dn_10_251_193_205 | 080722 145554 2468 INFO dfs.DataNode$DataXceiver: Receiving block blk_-1201258788540100829 src: /10.251.107.213:36718 dest: /10.251.107.213:50010
dn_10_251_67_214 | 080722 145554 2306 INFO dfs.DataNode$DataXceiver: Receiving block blk_-1201258788540100829 src: /10.251.67.214:60599 dest: /10.251.67.214:50010
nn_10_251_210_161 | 080722 145554 30 INFO dfs.FSNamesystem: BLOCK* NameSystem.allocateBlock: /user/root/rand/_temporary/_task_200807221347_0001_m_000092_1/part-00092. blk_-1201258788540100829
dn_10_251_107_213 | 080722 145656 2498 INFO dfs.DataNode$PacketResponder: Received block blk_-1201258788540100829 of size 29767698 from /10.251.67.214
dn_10_251_193_205 | 080722 145656 2469 INFO dfs.DataNode$PacketResponder: Received block blk_-1201258788540100829 of size 29767698 from /10.251.107.213
dn_10_251_67_214 | 080722 145656 2307 INFO dfs.DataNode$PacketResponder: Received block blk_-1201258788540100829 of size 29767698 from /10.251.67.214
dn_10_251_107_213 | 080722 145656 2498 INFO dfs.DataNode$PacketResponder: PacketResponder 1 for block blk_-1201258788540100829 terminating
dn_10_251_193_205 | 080722 145656 2469 INFO dfs.DataNode$PacketResponder: PacketResponder 0 for block blk_-1201258788540100829 terminating
dn_10_251_67_214 | 080722 145656 2307 INFO dfs.DataNode$PacketResponder: PacketResponder 2 for block blk_-1201258788540100829 terminating
nn_10_251_210_161 | 080722 145656 19 INFO dfs.FSNamesystem: BLOCK* ask 10.251.67.214:50010 to replicate blk_-1201258788540100829 to datanode(s) 10.251.107.49:50010
nn_10_251_210_161 | 080722 145656 31 INFO dfs.FSNamesystem: BLOCK* NameSystem.addStoredBlock: blockMap updated: 10.251.67.214:50010 is added to blk_-1201258788540100829 size 29767698
nn_10_251_210_161 | 080722 145656 31 INFO dfs.FSNamesystem: BLOCK* NameSystem.addStoredBlock: blockMap updated: 10.251.107.213:50010 is added to blk_-1201258788540100829 size 29767698
nn_10_251_210_161 | 080722 145656 26 INFO dfs.FSNamesystem: BLOCK* NameSystem.addStoredBlock: blockMap updated: 10.251.193.205:50010 is added to blk_-1201258788540100829 size 29767698
dn_10_251_107_49 | 080722 145658 2484 INFO dfs.DataNode$DataXceiver: Receiving block blk_-1201258788540100829 src: /10.251.67.214:42237 dest: /10.251.67.214:50010
dn_10_251_67_214 | 080722 145658 18 INFO dfs.DataNode: 10.251.67.214:50010 Starting thread to transfer block blk_-1201258788540100829 to 10.251.107.49:50010
dn_10_251_107_49 | 080722 145705 2484 INFO dfs.DataNode$DataXceiver: Received block blk_-1201258788540100829 src: /10.251.67.214:42237 dest: /10.251.67.214:50010 of size 29767698
dn_10_251_67_214 | 080722 145705 2342 INFO dfs.DataNode$DataTransfer: 10.251.67.214:50010:Transmitted block blk_-1201258788540100829 to /10.251.107.49:50010
nn_10_251_210_161 | 080722 145705 34 INFO dfs.FSNamesystem: BLOCK* NameSystem.addStoredBlock: blockMap updated: 10.251.107.49:50010 is added to blk_-1201258788540100829 size 29767698
dn_10_251_107_213 | 080722 145710 19 INFO dfs.FSDataset: Deleting block blk_-1201258788540100829 file /mnt/hadoop/dfs/data/current/subdir11/blk_-1201258788540100829
nn_10_251_210_161 | 080722 150220 34 INFO dfs.FSNamesystem: BLOCK* NameSystem.delete: blk_-1201258788540100829 is added to invalidSet of 10.251.67.214:50010
nn_10_251_210_161 | 080722 150220 34 INFO dfs.FSNamesystem: BLOCK* NameSystem.delete: blk_-1201258788540100829 is added to invalidSet of 10.251.107.213:50010
nn_10_251_210_161 | 080722 150220 34 INFO dfs.FSNamesystem: BLOCK* NameSystem.delete: blk_-1201258788540100829 is added to invalidSet of 10.251.193.205:50010
nn_10_251_210_161 | 080722 150220 34 INFO dfs.FSNamesystem: BLOCK* NameSystem.delete: blk_-1201258788540100829 is added to invalidSet of 10.251.107.49:50010
dn_10_251_193_205 | 080722 150224 19 INFO dfs.FSDataset: Deleting block blk_-1201258788540100829 file /mnt/hadoop/dfs/data/current/subdir51/blk_-1201258788540100829
dn_10_251_107_49 | 080722 150226 19 INFO dfs.FSDataset: Deleting block blk_-1201258788540100829 file /mnt/hadoop/dfs/data/current/subdir24/blk_-1201258788540100829
dn_10_251_107_213 | 080722 150228 19 WARN dfs.FSDataset: Unexpected error trying to delete block blk_-1201258788540100829. BlockInfo not found in volumeMap.
dn_10_251_67_214 | 080722 150253 18 INFO dfs.FSDataset: Deleting block blk_-1201258788540100829 file /mnt/hadoop/dfs/data/current/subdir43/blk_-1201258788540100829
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3885) Namenode blockMap not updated when
datanode invalidates a block on heart beat
Posted by "Hairong Kuang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12657637#action_12657637 ]
Hairong Kuang commented on HADOOP-3885:
---------------------------------------
One more note: excluding excessive replicas when counting live replicas are fixed at HADOOP-4643. If nobody against it, I will mark this issue as "not fixed".
> Namenode blockMap not updated when datanode invalidates a block on heart beat
> -----------------------------------------------------------------------------
>
> Key: HADOOP-3885
> URL: https://issues.apache.org/jira/browse/HADOOP-3885
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.18.0
> Reporter: Wei Xu
>
> Here are related logs. dn_10_251_107_213 deleted the block immediately after it received it (because of over-replication). However, the information is not reported to namenode and name node ask this DN to invalidate the block again, causing the error "Unexpected error trying to delete block blk_-1201258788540100829. BlockInfo not found in volumeMap.".
> dn_10_251_107_213 | 080722 145554 2497 INFO dfs.DataNode$DataXceiver: Receiving block blk_-1201258788540100829 src: /10.251.67.214:54770 dest: /10.251.67.214:50010
> dn_10_251_193_205 | 080722 145554 2468 INFO dfs.DataNode$DataXceiver: Receiving block blk_-1201258788540100829 src: /10.251.107.213:36718 dest: /10.251.107.213:50010
> dn_10_251_67_214 | 080722 145554 2306 INFO dfs.DataNode$DataXceiver: Receiving block blk_-1201258788540100829 src: /10.251.67.214:60599 dest: /10.251.67.214:50010
> nn_10_251_210_161 | 080722 145554 30 INFO dfs.FSNamesystem: BLOCK* NameSystem.allocateBlock: /user/root/rand/_temporary/_task_200807221347_0001_m_000092_1/part-00092. blk_-1201258788540100829
> dn_10_251_107_213 | 080722 145656 2498 INFO dfs.DataNode$PacketResponder: Received block blk_-1201258788540100829 of size 29767698 from /10.251.67.214
> dn_10_251_193_205 | 080722 145656 2469 INFO dfs.DataNode$PacketResponder: Received block blk_-1201258788540100829 of size 29767698 from /10.251.107.213
> dn_10_251_67_214 | 080722 145656 2307 INFO dfs.DataNode$PacketResponder: Received block blk_-1201258788540100829 of size 29767698 from /10.251.67.214
> dn_10_251_107_213 | 080722 145656 2498 INFO dfs.DataNode$PacketResponder: PacketResponder 1 for block blk_-1201258788540100829 terminating
> dn_10_251_193_205 | 080722 145656 2469 INFO dfs.DataNode$PacketResponder: PacketResponder 0 for block blk_-1201258788540100829 terminating
> dn_10_251_67_214 | 080722 145656 2307 INFO dfs.DataNode$PacketResponder: PacketResponder 2 for block blk_-1201258788540100829 terminating
> nn_10_251_210_161 | 080722 145656 19 INFO dfs.FSNamesystem: BLOCK* ask 10.251.67.214:50010 to replicate blk_-1201258788540100829 to datanode(s) 10.251.107.49:50010
> nn_10_251_210_161 | 080722 145656 31 INFO dfs.FSNamesystem: BLOCK* NameSystem.addStoredBlock: blockMap updated: 10.251.67.214:50010 is added to blk_-1201258788540100829 size 29767698
> nn_10_251_210_161 | 080722 145656 31 INFO dfs.FSNamesystem: BLOCK* NameSystem.addStoredBlock: blockMap updated: 10.251.107.213:50010 is added to blk_-1201258788540100829 size 29767698
> nn_10_251_210_161 | 080722 145656 26 INFO dfs.FSNamesystem: BLOCK* NameSystem.addStoredBlock: blockMap updated: 10.251.193.205:50010 is added to blk_-1201258788540100829 size 29767698
> dn_10_251_107_49 | 080722 145658 2484 INFO dfs.DataNode$DataXceiver: Receiving block blk_-1201258788540100829 src: /10.251.67.214:42237 dest: /10.251.67.214:50010
> dn_10_251_67_214 | 080722 145658 18 INFO dfs.DataNode: 10.251.67.214:50010 Starting thread to transfer block blk_-1201258788540100829 to 10.251.107.49:50010
> dn_10_251_107_49 | 080722 145705 2484 INFO dfs.DataNode$DataXceiver: Received block blk_-1201258788540100829 src: /10.251.67.214:42237 dest: /10.251.67.214:50010 of size 29767698
> dn_10_251_67_214 | 080722 145705 2342 INFO dfs.DataNode$DataTransfer: 10.251.67.214:50010:Transmitted block blk_-1201258788540100829 to /10.251.107.49:50010
> nn_10_251_210_161 | 080722 145705 34 INFO dfs.FSNamesystem: BLOCK* NameSystem.addStoredBlock: blockMap updated: 10.251.107.49:50010 is added to blk_-1201258788540100829 size 29767698
> dn_10_251_107_213 | 080722 145710 19 INFO dfs.FSDataset: Deleting block blk_-1201258788540100829 file /mnt/hadoop/dfs/data/current/subdir11/blk_-1201258788540100829
> nn_10_251_210_161 | 080722 150220 34 INFO dfs.FSNamesystem: BLOCK* NameSystem.delete: blk_-1201258788540100829 is added to invalidSet of 10.251.67.214:50010
> nn_10_251_210_161 | 080722 150220 34 INFO dfs.FSNamesystem: BLOCK* NameSystem.delete: blk_-1201258788540100829 is added to invalidSet of 10.251.107.213:50010
> nn_10_251_210_161 | 080722 150220 34 INFO dfs.FSNamesystem: BLOCK* NameSystem.delete: blk_-1201258788540100829 is added to invalidSet of 10.251.193.205:50010
> nn_10_251_210_161 | 080722 150220 34 INFO dfs.FSNamesystem: BLOCK* NameSystem.delete: blk_-1201258788540100829 is added to invalidSet of 10.251.107.49:50010
> dn_10_251_193_205 | 080722 150224 19 INFO dfs.FSDataset: Deleting block blk_-1201258788540100829 file /mnt/hadoop/dfs/data/current/subdir51/blk_-1201258788540100829
> dn_10_251_107_49 | 080722 150226 19 INFO dfs.FSDataset: Deleting block blk_-1201258788540100829 file /mnt/hadoop/dfs/data/current/subdir24/blk_-1201258788540100829
> dn_10_251_107_213 | 080722 150228 19 WARN dfs.FSDataset: Unexpected error trying to delete block blk_-1201258788540100829. BlockInfo not found in volumeMap.
> dn_10_251_67_214 | 080722 150253 18 INFO dfs.FSDataset: Deleting block blk_-1201258788540100829 file /mnt/hadoop/dfs/data/current/subdir43/blk_-1201258788540100829
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3885) Namenode blockMap not updated when
datanode invalidates a block on heart beat
Posted by "Hairong Kuang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12657617#action_12657617 ]
Hairong Kuang commented on HADOOP-3885:
---------------------------------------
The first deletion at 251_107_213 is recorded at ExcessReplicateMap. So it does not count as a valid replica. NN still keeps it in blocksMap since NN has not received confirmation from the datanode yet.
Were the following four deletions caused by the deletion of the file? When deleting a file, NN deletes all replicas belonged to this file no matter if they are already scheduled to delete or not. I think this is a safe solution since NN has not heard any confirmation yet so NN is not sure if the replicas are indeed deleted or not.
> Namenode blockMap not updated when datanode invalidates a block on heart beat
> -----------------------------------------------------------------------------
>
> Key: HADOOP-3885
> URL: https://issues.apache.org/jira/browse/HADOOP-3885
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.18.0
> Reporter: Wei Xu
>
> Here are related logs. dn_10_251_107_213 deleted the block immediately after it received it (because of over-replication). However, the information is not reported to namenode and name node ask this DN to invalidate the block again, causing the error "Unexpected error trying to delete block blk_-1201258788540100829. BlockInfo not found in volumeMap.".
> dn_10_251_107_213 | 080722 145554 2497 INFO dfs.DataNode$DataXceiver: Receiving block blk_-1201258788540100829 src: /10.251.67.214:54770 dest: /10.251.67.214:50010
> dn_10_251_193_205 | 080722 145554 2468 INFO dfs.DataNode$DataXceiver: Receiving block blk_-1201258788540100829 src: /10.251.107.213:36718 dest: /10.251.107.213:50010
> dn_10_251_67_214 | 080722 145554 2306 INFO dfs.DataNode$DataXceiver: Receiving block blk_-1201258788540100829 src: /10.251.67.214:60599 dest: /10.251.67.214:50010
> nn_10_251_210_161 | 080722 145554 30 INFO dfs.FSNamesystem: BLOCK* NameSystem.allocateBlock: /user/root/rand/_temporary/_task_200807221347_0001_m_000092_1/part-00092. blk_-1201258788540100829
> dn_10_251_107_213 | 080722 145656 2498 INFO dfs.DataNode$PacketResponder: Received block blk_-1201258788540100829 of size 29767698 from /10.251.67.214
> dn_10_251_193_205 | 080722 145656 2469 INFO dfs.DataNode$PacketResponder: Received block blk_-1201258788540100829 of size 29767698 from /10.251.107.213
> dn_10_251_67_214 | 080722 145656 2307 INFO dfs.DataNode$PacketResponder: Received block blk_-1201258788540100829 of size 29767698 from /10.251.67.214
> dn_10_251_107_213 | 080722 145656 2498 INFO dfs.DataNode$PacketResponder: PacketResponder 1 for block blk_-1201258788540100829 terminating
> dn_10_251_193_205 | 080722 145656 2469 INFO dfs.DataNode$PacketResponder: PacketResponder 0 for block blk_-1201258788540100829 terminating
> dn_10_251_67_214 | 080722 145656 2307 INFO dfs.DataNode$PacketResponder: PacketResponder 2 for block blk_-1201258788540100829 terminating
> nn_10_251_210_161 | 080722 145656 19 INFO dfs.FSNamesystem: BLOCK* ask 10.251.67.214:50010 to replicate blk_-1201258788540100829 to datanode(s) 10.251.107.49:50010
> nn_10_251_210_161 | 080722 145656 31 INFO dfs.FSNamesystem: BLOCK* NameSystem.addStoredBlock: blockMap updated: 10.251.67.214:50010 is added to blk_-1201258788540100829 size 29767698
> nn_10_251_210_161 | 080722 145656 31 INFO dfs.FSNamesystem: BLOCK* NameSystem.addStoredBlock: blockMap updated: 10.251.107.213:50010 is added to blk_-1201258788540100829 size 29767698
> nn_10_251_210_161 | 080722 145656 26 INFO dfs.FSNamesystem: BLOCK* NameSystem.addStoredBlock: blockMap updated: 10.251.193.205:50010 is added to blk_-1201258788540100829 size 29767698
> dn_10_251_107_49 | 080722 145658 2484 INFO dfs.DataNode$DataXceiver: Receiving block blk_-1201258788540100829 src: /10.251.67.214:42237 dest: /10.251.67.214:50010
> dn_10_251_67_214 | 080722 145658 18 INFO dfs.DataNode: 10.251.67.214:50010 Starting thread to transfer block blk_-1201258788540100829 to 10.251.107.49:50010
> dn_10_251_107_49 | 080722 145705 2484 INFO dfs.DataNode$DataXceiver: Received block blk_-1201258788540100829 src: /10.251.67.214:42237 dest: /10.251.67.214:50010 of size 29767698
> dn_10_251_67_214 | 080722 145705 2342 INFO dfs.DataNode$DataTransfer: 10.251.67.214:50010:Transmitted block blk_-1201258788540100829 to /10.251.107.49:50010
> nn_10_251_210_161 | 080722 145705 34 INFO dfs.FSNamesystem: BLOCK* NameSystem.addStoredBlock: blockMap updated: 10.251.107.49:50010 is added to blk_-1201258788540100829 size 29767698
> dn_10_251_107_213 | 080722 145710 19 INFO dfs.FSDataset: Deleting block blk_-1201258788540100829 file /mnt/hadoop/dfs/data/current/subdir11/blk_-1201258788540100829
> nn_10_251_210_161 | 080722 150220 34 INFO dfs.FSNamesystem: BLOCK* NameSystem.delete: blk_-1201258788540100829 is added to invalidSet of 10.251.67.214:50010
> nn_10_251_210_161 | 080722 150220 34 INFO dfs.FSNamesystem: BLOCK* NameSystem.delete: blk_-1201258788540100829 is added to invalidSet of 10.251.107.213:50010
> nn_10_251_210_161 | 080722 150220 34 INFO dfs.FSNamesystem: BLOCK* NameSystem.delete: blk_-1201258788540100829 is added to invalidSet of 10.251.193.205:50010
> nn_10_251_210_161 | 080722 150220 34 INFO dfs.FSNamesystem: BLOCK* NameSystem.delete: blk_-1201258788540100829 is added to invalidSet of 10.251.107.49:50010
> dn_10_251_193_205 | 080722 150224 19 INFO dfs.FSDataset: Deleting block blk_-1201258788540100829 file /mnt/hadoop/dfs/data/current/subdir51/blk_-1201258788540100829
> dn_10_251_107_49 | 080722 150226 19 INFO dfs.FSDataset: Deleting block blk_-1201258788540100829 file /mnt/hadoop/dfs/data/current/subdir24/blk_-1201258788540100829
> dn_10_251_107_213 | 080722 150228 19 WARN dfs.FSDataset: Unexpected error trying to delete block blk_-1201258788540100829. BlockInfo not found in volumeMap.
> dn_10_251_67_214 | 080722 150253 18 INFO dfs.FSDataset: Deleting block blk_-1201258788540100829 file /mnt/hadoop/dfs/data/current/subdir43/blk_-1201258788540100829
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3885) Namenode blockMap not updated when
datanode invalidates a block on heart beat
Posted by "Hairong Kuang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12657622#action_12657622 ]
Hairong Kuang commented on HADOOP-3885:
---------------------------------------
All I meant to say is that what's described in this jira is the expected behavior of dfs. It does not cause any data lose except for some harmless deletion warnings at datanodes. One improvement we could do is to make the deletion confirmation back to NN sooner. Currently it waits for the next block report to send the confirmation back. But in between a block deletion is scheduled to remove and the deletion confirmation comes back, the reported issue is likely to happen.
> Namenode blockMap not updated when datanode invalidates a block on heart beat
> -----------------------------------------------------------------------------
>
> Key: HADOOP-3885
> URL: https://issues.apache.org/jira/browse/HADOOP-3885
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.18.0
> Reporter: Wei Xu
>
> Here are related logs. dn_10_251_107_213 deleted the block immediately after it received it (because of over-replication). However, the information is not reported to namenode and name node ask this DN to invalidate the block again, causing the error "Unexpected error trying to delete block blk_-1201258788540100829. BlockInfo not found in volumeMap.".
> dn_10_251_107_213 | 080722 145554 2497 INFO dfs.DataNode$DataXceiver: Receiving block blk_-1201258788540100829 src: /10.251.67.214:54770 dest: /10.251.67.214:50010
> dn_10_251_193_205 | 080722 145554 2468 INFO dfs.DataNode$DataXceiver: Receiving block blk_-1201258788540100829 src: /10.251.107.213:36718 dest: /10.251.107.213:50010
> dn_10_251_67_214 | 080722 145554 2306 INFO dfs.DataNode$DataXceiver: Receiving block blk_-1201258788540100829 src: /10.251.67.214:60599 dest: /10.251.67.214:50010
> nn_10_251_210_161 | 080722 145554 30 INFO dfs.FSNamesystem: BLOCK* NameSystem.allocateBlock: /user/root/rand/_temporary/_task_200807221347_0001_m_000092_1/part-00092. blk_-1201258788540100829
> dn_10_251_107_213 | 080722 145656 2498 INFO dfs.DataNode$PacketResponder: Received block blk_-1201258788540100829 of size 29767698 from /10.251.67.214
> dn_10_251_193_205 | 080722 145656 2469 INFO dfs.DataNode$PacketResponder: Received block blk_-1201258788540100829 of size 29767698 from /10.251.107.213
> dn_10_251_67_214 | 080722 145656 2307 INFO dfs.DataNode$PacketResponder: Received block blk_-1201258788540100829 of size 29767698 from /10.251.67.214
> dn_10_251_107_213 | 080722 145656 2498 INFO dfs.DataNode$PacketResponder: PacketResponder 1 for block blk_-1201258788540100829 terminating
> dn_10_251_193_205 | 080722 145656 2469 INFO dfs.DataNode$PacketResponder: PacketResponder 0 for block blk_-1201258788540100829 terminating
> dn_10_251_67_214 | 080722 145656 2307 INFO dfs.DataNode$PacketResponder: PacketResponder 2 for block blk_-1201258788540100829 terminating
> nn_10_251_210_161 | 080722 145656 19 INFO dfs.FSNamesystem: BLOCK* ask 10.251.67.214:50010 to replicate blk_-1201258788540100829 to datanode(s) 10.251.107.49:50010
> nn_10_251_210_161 | 080722 145656 31 INFO dfs.FSNamesystem: BLOCK* NameSystem.addStoredBlock: blockMap updated: 10.251.67.214:50010 is added to blk_-1201258788540100829 size 29767698
> nn_10_251_210_161 | 080722 145656 31 INFO dfs.FSNamesystem: BLOCK* NameSystem.addStoredBlock: blockMap updated: 10.251.107.213:50010 is added to blk_-1201258788540100829 size 29767698
> nn_10_251_210_161 | 080722 145656 26 INFO dfs.FSNamesystem: BLOCK* NameSystem.addStoredBlock: blockMap updated: 10.251.193.205:50010 is added to blk_-1201258788540100829 size 29767698
> dn_10_251_107_49 | 080722 145658 2484 INFO dfs.DataNode$DataXceiver: Receiving block blk_-1201258788540100829 src: /10.251.67.214:42237 dest: /10.251.67.214:50010
> dn_10_251_67_214 | 080722 145658 18 INFO dfs.DataNode: 10.251.67.214:50010 Starting thread to transfer block blk_-1201258788540100829 to 10.251.107.49:50010
> dn_10_251_107_49 | 080722 145705 2484 INFO dfs.DataNode$DataXceiver: Received block blk_-1201258788540100829 src: /10.251.67.214:42237 dest: /10.251.67.214:50010 of size 29767698
> dn_10_251_67_214 | 080722 145705 2342 INFO dfs.DataNode$DataTransfer: 10.251.67.214:50010:Transmitted block blk_-1201258788540100829 to /10.251.107.49:50010
> nn_10_251_210_161 | 080722 145705 34 INFO dfs.FSNamesystem: BLOCK* NameSystem.addStoredBlock: blockMap updated: 10.251.107.49:50010 is added to blk_-1201258788540100829 size 29767698
> dn_10_251_107_213 | 080722 145710 19 INFO dfs.FSDataset: Deleting block blk_-1201258788540100829 file /mnt/hadoop/dfs/data/current/subdir11/blk_-1201258788540100829
> nn_10_251_210_161 | 080722 150220 34 INFO dfs.FSNamesystem: BLOCK* NameSystem.delete: blk_-1201258788540100829 is added to invalidSet of 10.251.67.214:50010
> nn_10_251_210_161 | 080722 150220 34 INFO dfs.FSNamesystem: BLOCK* NameSystem.delete: blk_-1201258788540100829 is added to invalidSet of 10.251.107.213:50010
> nn_10_251_210_161 | 080722 150220 34 INFO dfs.FSNamesystem: BLOCK* NameSystem.delete: blk_-1201258788540100829 is added to invalidSet of 10.251.193.205:50010
> nn_10_251_210_161 | 080722 150220 34 INFO dfs.FSNamesystem: BLOCK* NameSystem.delete: blk_-1201258788540100829 is added to invalidSet of 10.251.107.49:50010
> dn_10_251_193_205 | 080722 150224 19 INFO dfs.FSDataset: Deleting block blk_-1201258788540100829 file /mnt/hadoop/dfs/data/current/subdir51/blk_-1201258788540100829
> dn_10_251_107_49 | 080722 150226 19 INFO dfs.FSDataset: Deleting block blk_-1201258788540100829 file /mnt/hadoop/dfs/data/current/subdir24/blk_-1201258788540100829
> dn_10_251_107_213 | 080722 150228 19 WARN dfs.FSDataset: Unexpected error trying to delete block blk_-1201258788540100829. BlockInfo not found in volumeMap.
> dn_10_251_67_214 | 080722 150253 18 INFO dfs.FSDataset: Deleting block blk_-1201258788540100829 file /mnt/hadoop/dfs/data/current/subdir43/blk_-1201258788540100829
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.