[jira] [Created] (HADOOP-10216) Unnecessary disk check triggered when socket operation has problem.

["MaoYuan Xian (JIRA)" <ji...@apache.org>]

MaoYuan Xian created HADOOP-10216:
-------------------------------------

             Summary: Unnecessary disk check triggered when socket operation has problem.
                 Key: HADOOP-10216
                 URL: https://issues.apache.org/jira/browse/HADOOP-10216
             Project: Hadoop Common
          Issue Type: Improvement
          Components: fs
    Affects Versions: 1.1.2
            Reporter: MaoYuan Xian


When BlockReceiver transfer data fails, it can be found SocketOutputStream translates the exception as IOException with the message "The stream is closed":
2014-01-06 11:48:04,716 WARN org.apache.hadoop.hdfs.server.datanode.DataNode: IOException in BlockReceiver.run():
java.io.IOException: The stream is closed
        at org.apache.hadoop.net.SocketOutputStream.write
        at java.io.BufferedOutputStream.flushBuffer
        at java.io.BufferedOutputStream.flush
        at java.io.DataOutputStream.flush
        at org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.run
        at java.lang.Thread.run

Which makes the checkDiskError method of DataNode called and triggers the disk scan.

Can we make the modifications like below in checkDiskError to avoiding this unneccessary disk scan operations?:

{code}
--- a/src/hdfs/org/apache/hadoop/hdfs/server/datanode/DataNode.java
+++ b/src/hdfs/org/apache/hadoop/hdfs/server/datanode/DataNode.java
@@ -938,7 +938,8 @@ public class DataNode extends Configured
          || e.getMessage().startsWith("An established connection was aborted")
          || e.getMessage().startsWith("Broken pipe")
          || e.getMessage().startsWith("Connection reset")
-         || e.getMessage().contains("java.nio.channels.SocketChannel")) {
+         || e.getMessage().contains("java.nio.channels.SocketChannel")
+         || e.getMessage().startsWith("The stream is closed")) {
       LOG.info("Not checking disk as checkDiskError was called on a network" +
         " related exception"); 
       return;
{code}



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)