IOException in offerService

[Yanbo Liang <ya...@gmail.com>]

DataNode may received IOException in heartbeat sending process. When
running TestStartup, I got the following warn log:

2012-11-13 01:28:19,217 WARN  datanode.DataNode
(BPServiceActor.java:offerService(596)) - IOException in offerService
java.io.IOException: Failed on local exception: java.io.IOException:
Response is null.; Host Details : local host is: "orange/127.0.0.1";
destination host is: "orange":38705;
        at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:759)
        at org.apache.hadoop.ipc.Client.call(Client.java:1163)
        at
org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:202)
        at $Proxy14.sendHeartbeat(Unknown Source)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:601)
        at
org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:163)
        at
org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:82)
        at $Proxy14.sendHeartbeat(Unknown Source)
        at
org.apache.hadoop.hdfs.protocolPB.DatanodeProtocolClientSideTranslatorPB.sendHeartbeat(DatanodeProtocolClientSideTranslatorPB.java:167)
        at
org.apache.hadoop.hdfs.server.datanode.BPServiceActor.sendHeartBeat(BPServiceActor.java:445)
        at
org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:525)
        at
org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:676)
        at java.lang.Thread.run(Thread.java:722)
Caused by: java.io.IOException: Response is null.
        at
org.apache.hadoop.ipc.Client$Connection.receiveResponse(Client.java:884)
        at org.apache.hadoop.ipc.Client$Connection.run(Client.java:812)

>From the log, I can infer that DN send heartbeat to NN and got response is
null.
But the workflow of process the response at DN just log it and without
other action.
I wonder which caused this result? The NameNode has no DatanodeCommand to
send for this DataNode, protocolbuffer related problem or other reason?
Is it make sense that the DN just log this Exception and without other
action?

Thanks,
Yanbo