You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Brian Bockelman <bb...@math.unl.edu> on 2008/12/06 02:22:03 UTC
Block not found during commitBlockSynchronization
Hey,
I'm seeing this message repeated over and over in my logs:
2008-12-05 19:20:00,534 INFO
org.apache.hadoop.hdfs.server.namenode.FSNamesystem:
commitBlockSynchronization(lastblock=blk_-4236881263392665762_88597,
newgenerationstamp=0, newlength=0, newtargets=[])
2008-12-05 19:20:00,534 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 29 on 9000, call
commitBlockSynchronization(blk_-4236881263392665762_88597, 0, 0,
false, true, [Lorg.apache.hadoop.hdfs.protocol.DatanodeID;@67537412)
from 172.16.1.184:57586: error: java.io.IOException: Block
(=blk_-4236881263392665762_88597) not found
java.io.IOException: Block (=blk_-4236881263392665762_88597) not found
at
org
.apache
.hadoop
.hdfs
.server
.namenode.FSNamesystem.commitBlockSynchronization(FSNamesystem.java:
1898)
at
org
.apache
.hadoop
.hdfs
.server.namenode.NameNode.commitBlockSynchronization(NameNode.java:410)
at sun.reflect.GeneratedMethodAccessor21.invoke(Unknown Source)
at
sun
.reflect
.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:
25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:452)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:892)
What can I do to debug?
Brian
Re: Block not found during commitBlockSynchronization
Posted by Brian Bockelman <bb...@cse.unl.edu>.
This is 0.19.0.
Grepping around, it appears that message for this block has been
printed 1-5 Hz throughout all our logs (oldest logs are 12-3). Has
happened about .5 million times. If I grep for the
"nextGenerationStamp" error message, it's happened .4M times.
Anything else I can provide?
Brian
On Dec 5, 2008, at 8:31 PM, Tsz Wo (Nicholas), Sze wrote:
> Which version are you using?
>
> Calling commitBlockSynchronization(...) with newgenerationstamp=0,
> newlength=0, newtargets=[] does not look normal. You may check the
> namenode log and the client log about the block
> blk_-4236881263392665762.
>
> Nicholas Sze
>
>
>
>
> ----- Original Message ----
>> From: Brian Bockelman <bb...@math.unl.edu>
>> To: core-user@hadoop.apache.org
>> Sent: Friday, December 5, 2008 5:22:03 PM
>> Subject: Block not found during commitBlockSynchronization
>>
>> Hey,
>>
>> I'm seeing this message repeated over and over in my logs:
>>
>> 2008-12-05 19:20:00,534 INFO
>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem:
>> commitBlockSynchronization(lastblock=blk_-4236881263392665762_88597,
>> newgenerationstamp=0, newlength=0, newtargets=[])
>> 2008-12-05 19:20:00,534 INFO org.apache.hadoop.ipc.Server: IPC
>> Server handler 29
>> on 9000, call
>> commitBlockSynchronization(blk_-4236881263392665762_88597, 0, 0,
>> false, true,
>> [Lorg.apache.hadoop.hdfs.protocol.DatanodeID;@67537412) from
>> 172.16.1.184:57586: error: java.io.IOException: Block
>> (=blk_-4236881263392665762_88597) not found
>> java.io.IOException: Block (=blk_-4236881263392665762_88597) not
>> found
>> at
>> org
>> .apache
>> .hadoop
>> .hdfs
>> .server
>> .namenode.FSNamesystem.commitBlockSynchronization(FSNamesystem.java:
>> 1898)
>> at
>> org
>> .apache
>> .hadoop
>> .hdfs
>> .server.namenode.NameNode.commitBlockSynchronization(NameNode.java:
>> 410)
>> at sun.reflect.GeneratedMethodAccessor21.invoke(Unknown Source)
>> at
>> sun
>> .reflect
>> .DelegatingMethodAccessorImpl
>> .invoke(DelegatingMethodAccessorImpl.java:25)
>> at java.lang.reflect.Method.invoke(Method.java:597)
>> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:452)
>> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:892)
>>
>> What can I do to debug?
>>
>> Brian
Re: Block not found during commitBlockSynchronization
Posted by "Tsz Wo (Nicholas), Sze" <s2...@yahoo.com>.
Which version are you using?
Calling commitBlockSynchronization(...) with newgenerationstamp=0, newlength=0, newtargets=[] does not look normal. You may check the namenode log and the client log about the block blk_-4236881263392665762.
Nicholas Sze
----- Original Message ----
> From: Brian Bockelman <bb...@math.unl.edu>
> To: core-user@hadoop.apache.org
> Sent: Friday, December 5, 2008 5:22:03 PM
> Subject: Block not found during commitBlockSynchronization
>
> Hey,
>
> I'm seeing this message repeated over and over in my logs:
>
> 2008-12-05 19:20:00,534 INFO
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem:
> commitBlockSynchronization(lastblock=blk_-4236881263392665762_88597,
> newgenerationstamp=0, newlength=0, newtargets=[])
> 2008-12-05 19:20:00,534 INFO org.apache.hadoop.ipc.Server: IPC Server handler 29
> on 9000, call commitBlockSynchronization(blk_-4236881263392665762_88597, 0, 0,
> false, true, [Lorg.apache.hadoop.hdfs.protocol.DatanodeID;@67537412) from
> 172.16.1.184:57586: error: java.io.IOException: Block
> (=blk_-4236881263392665762_88597) not found
> java.io.IOException: Block (=blk_-4236881263392665762_88597) not found
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.commitBlockSynchronization(FSNamesystem.java:1898)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.commitBlockSynchronization(NameNode.java:410)
> at sun.reflect.GeneratedMethodAccessor21.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:452)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:892)
>
> What can I do to debug?
>
> Brian