You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by Christian Kunz <ck...@yahoo-inc.com> on 2009/09/04 21:06:54 UTC
CommitBlockSynchronization issue
Using hadoop-0.18.3,
One of our files with size 273592 changed size to an incorrect value 262144
due to commitBlockSynchronization.
Would that have been fixed by HADOOP-5134?
Thanks,
-Christian
Here are the relevant log messages of the namenode
log.2009-09-01:2009-09-01 20:52:45,451 INFO
org.apache.hadoop.dfs.StateChange: BLOCK*
NameSystem.allocateBlock:
fileName.blk_2101888387690035515_63606967
log.2009-09-01:2009-09-01 20:57:43,535 INFO
org.apache.hadoop.dfs.StateChange: BLOCK*
NameSystem.addStoredBlock: blockMap updated: xxx.yyy.zzz.234:54610 is added
to blk_2101888387690035515_63606967 size 273592
log.2009-09-01:2009-09-01 20:57:47,847 INFO
org.apache.hadoop.dfs.StateChange: BLOCK*
NameSystem.addStoredBlock: blockMap updated: xxx.yyy.zzz.250:54610 is added
to blk_2101888387690035515_63606967 size 273592
log.2009-09-01:2009-09-01 20:58:01,480 INFO
org.apache.hadoop.fs.FSNamesystem:
commitBlockSynchronization(lastblock=blk_2101888387690035515_63606967,
newgenerationstamp=63637360, newlength=262144,
newtargets=[xxx.yyy.zzz.234:54610])
log.2009-09-01:2009-09-01 20:58:01,482 INFO
org.apache.hadoop.fs.FSNamesystem:
commitBlockSynchronization(blk_2101888387690035515_63637360) successful
log.2009-09-01:2009-09-01 21:01:07,508 INFO
org.apache.hadoop.fs.FSNamesystem:
commitBlockSynchronization(lastblock=blk_2101888387690035515_63637360,
newgenerationstamp=63639267, newlength=262144,
newtargets=[xxx.yyy.zzz.234:54610])
log.2009-09-01:2009-09-01 21:01:07,512 INFO
org.apache.hadoop.fs.FSNamesystem:
commitBlockSynchronization(blk_2101888387690035515_63639267) successful
log.2009-09-01:2009-09-01 21:01:08,946 INFO
org.apache.hadoop.dfs.StateChange: BLOCK*
ask xxx.yyy.zzz.234:54610 to replicate blk_2101888387690035515_63639267 to
datanode(s) xxx.yyy.zzz.230:54610 xxx.yyy.zzz.115:54610
log.2009-09-01:2009-09-01 21:01:12,061 INFO
org.apache.hadoop.dfs.StateChange: BLOCK*
NameSystem.addStoredBlock: blockMap updated: xxx.yyy.zzz.230:54610 is added
to blk_2101888387690035515_63639267 size 262144
log.2009-09-01:2009-09-01 21:01:12,097 INFO
org.apache.hadoop.dfs.StateChange: BLOCK*
NameSystem.addStoredBlock: blockMap updated: xxx.yyy.zzz.115:54610 is added
to blk_2101888387690035515_63639267 size 262144
log.2009-09-01:2009-09-01 21:35:03,362 INFO
org.apache.hadoop.dfs.StateChange: BLOCK*
NameSystem.processReport: block blk_2101888387690035515_63637360 on
xxx.yyy.zzz.250:54610 size 273592 does not belong to any file.
log.2009-09-01:2009-09-01 21:35:06,344 INFO
org.apache.hadoop.dfs.StateChange: BLOCK*
ask xxx.yyy.zzz.250:54610 to delete blk_2101888387690035515_63637360