You are viewing a plain text version of this content. The canonical link for it is here.

Posted to hdfs-dev@hadoop.apache.org by "Daryn Sharp (JIRA)" <ji...@apache.org> on 2014/08/28 20:22:08 UTC

[jira] [Created] (HDFS-6967) DNs may OOM under high webhdfs load

Daryn Sharp created HDFS-6967:
---------------------------------

             Summary: DNs may OOM under high webhdfs load
                 Key: HDFS-6967
                 URL: https://issues.apache.org/jira/browse/HDFS-6967
             Project: Hadoop HDFS
          Issue Type: Bug
          Components: datanode, webhdfs
    Affects Versions: 2.0.0-alpha, 3.0.0
            Reporter: Daryn Sharp


Webhdfs uses jetty.  The size of the request thread pool is limited, but jetty will accept and queue infinite connections.  Every queued connection is "heavy" with buffers, etc.  Unlike data streamer connections, thousands of webhdfs connections will quickly OOM a DN.  The accepted requests must be bounded and excess clients rejected so they retry on a new DN.



--
This message was sent by Atlassian JIRA
(v6.2#6252)