[jira] [Resolved] (HADOOP-10216) Unnecessary disk check triggered when socket operation has problem.

["MaoYuan Xian (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HADOOP-10216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

MaoYuan Xian resolved HADOOP-10216.
-----------------------------------

    Resolution: Invalid

> Unnecessary disk check triggered when socket operation has problem.
> -------------------------------------------------------------------
>
>                 Key: HADOOP-10216
>                 URL: https://issues.apache.org/jira/browse/HADOOP-10216
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: fs
>    Affects Versions: 1.1.2
>            Reporter: MaoYuan Xian
>
> When BlockReceiver transfer data fails, it can be found SocketOutputStream translates the exception as IOException with the message "The stream is closed":
> 2014-01-06 11:48:04,716 WARN org.apache.hadoop.hdfs.server.datanode.DataNode: IOException in BlockReceiver.run():
> java.io.IOException: The stream is closed
>         at org.apache.hadoop.net.SocketOutputStream.write
>         at java.io.BufferedOutputStream.flushBuffer
>         at java.io.BufferedOutputStream.flush
>         at java.io.DataOutputStream.flush
>         at org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.run
>         at java.lang.Thread.run
> Which makes the checkDiskError method of DataNode called and triggers the disk scan.
> Can we make the modifications like below in checkDiskError to avoiding this unneccessary disk scan operations?:
> {code}
> --- a/src/hdfs/org/apache/hadoop/hdfs/server/datanode/DataNode.java
> +++ b/src/hdfs/org/apache/hadoop/hdfs/server/datanode/DataNode.java
> @@ -938,7 +938,8 @@ public class DataNode extends Configured
>           || e.getMessage().startsWith("An established connection was aborted")
>           || e.getMessage().startsWith("Broken pipe")
>           || e.getMessage().startsWith("Connection reset")
> -         || e.getMessage().contains("java.nio.channels.SocketChannel")) {
> +         || e.getMessage().contains("java.nio.channels.SocketChannel")
> +         || e.getMessage().startsWith("The stream is closed")) {
>        LOG.info("Not checking disk as checkDiskError was called on a network" +
>          " related exception"); 
>        return;
> {code}



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)