[jira] [Created] (HDFS-7191) WebHDFS prematurely closes connections under high concurrent loads

["Haohui Mai (JIRA)" <ji...@apache.org>]

Haohui Mai created HDFS-7191:
--------------------------------

             Summary: WebHDFS prematurely closes connections under high concurrent loads
                 Key: HDFS-7191
                 URL: https://issues.apache.org/jira/browse/HDFS-7191
             Project: Hadoop HDFS
          Issue Type: Bug
            Reporter: Haohui Mai
            Priority: Critical


We're seeing the DN prematurely closes APPEND connections:

{noformat]
2014-09-22 23:53:12,721 WARN org.apache.hadoop.hdfs.web.resources.ExceptionHandler: INTERNAL_SERVER_ERROR
java.nio.channels.CancelledKeyException
        at sun.nio.ch.SelectionKeyImpl.ensureValid(SelectionKeyImpl.java:55)
        at sun.nio.ch.SelectionKeyImpl.interestOps(SelectionKeyImpl.java:59)
        at org.mortbay.io.nio.SelectChannelEndPoint.updateKey(SelectChannelEndPoint.java:325)
        at org.mortbay.io.nio.SelectChannelEndPoint.blockReadable(SelectChannelEndPoint.java:242)
        at org.mortbay.jetty.HttpParser$Input.blockForContent(HttpParser.java:1169)
        at org.mortbay.jetty.HttpParser$Input.read(HttpParser.java:1122)
        at java.io.InputStream.read(InputStream.java:85)
        at org.apache.hadoop.io.IOUtils.copyBytes(IOUtils.java:84)
        at org.apache.hadoop.hdfs.server.datanode.web.resources.DatanodeWebHdfsMethods.put(DatanodeWebHdfsMethods.java:239)
        at org.apache.hadoop.hdfs.server.datanode.web.resources.DatanodeWebHdfsMethods.access$000(DatanodeWebHdfsMethods.java:87)
        at org.apache.hadoop.hdfs.server.datanode.web.resources.DatanodeWebHdfsMethods$1.run(DatanodeWebHdfsMethods.java:205)
        at org.apache.hadoop.hdfs.server.datanode.web.resources.DatanodeWebHdfsMethods$1.run(DatanodeWebHdfsMethods.java:202)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:396)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
        at org.apache.hadoop.hdfs.server.datanode.web.resources.DatanodeWebHdfsMethods.put(DatanodeWebHdfsMethods.java:202)
{noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)