You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Hairong Kuang (JIRA)" <ji...@apache.org> on 2009/02/23 22:06:02 UTC
[jira] Commented: (HADOOP-5311) Write pipeline recovery fails
[ https://issues.apache.org/jira/browse/HADOOP-5311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12676063#action_12676063 ]
Hairong Kuang commented on HADOOP-5311:
---------------------------------------
Here is more information regarding this failure. The log came from the same primary datanode:
org.apache.hadoop.hdfs.server.datanode.DataNode: oldblock=blk_1415000632081498137_954380(length=31016448), newblock=blk_1415000632081498137_989001(length=31016448), datanode=XX
org.apache.hadoop.hdfs.server.datanode.DataNode: Received block blk_1415000632081498137_989001 of size 31016448 as part of lease recovery.
It looks that pipeline recovery succeeded at the primary datanode. The new generation stamp is 989001. But the client saw this recovery as a failure and used the old generation stamp 954380 to recover the error:
WARN org.apache.hadoop.hdfs.DFSClient: Error Recovery for block blk_1415000632081498137_954380 failed because recovery from primary datanode XX failed 4 times. Will retry...
This retry went on forever.
> Write pipeline recovery fails
> -----------------------------
>
> Key: HADOOP-5311
> URL: https://issues.apache.org/jira/browse/HADOOP-5311
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.18.0
> Reporter: Hairong Kuang
> Fix For: 0.18.4
>
>
> A write pipeline recovery fails on the error below:
> INFO org.apache.hadoop.ipc.Server: IPC Server handler 2 on 53006, call recoverBlock(blk_1415000632081498137_954380, false, [Lorg.apache.hadoop.hdfs.protocol.DatanodeInfo;
> @4ec82dc6) from XX: error: org.apache.hadoop.ipc.RemoteException: java.io.IOException: blk_1415000632081498137_954380 is already commited, storedBlock == null.
> at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.nextGenerationStampForBlock(FSNamesystem.java:4487)
> at org.apache.hadoop.hdfs.server.namenode.NameNode.nextGenerationStamp(NameNode.java:473)
> at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:508)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:959)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:955)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:953)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.