You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "Vinod Kumar Vavilapalli (JIRA)" <ji...@apache.org> on 2016/06/05 06:01:07 UTC
[jira] [Updated] (HDFS-7607) Use random rack-local node for webhdfs
opens to avoid OOM on DNs
[ https://issues.apache.org/jira/browse/HDFS-7607?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vinod Kumar Vavilapalli updated HDFS-7607:
------------------------------------------
Target Version/s: (was: 2.8.0)
Not much going on here for a long time, dropping from 2.8.0.
Not putting any target-version either anymore, let's target this depending on when there is patch activity.
> Use random rack-local node for webhdfs opens to avoid OOM on DNs
> ----------------------------------------------------------------
>
> Key: HDFS-7607
> URL: https://issues.apache.org/jira/browse/HDFS-7607
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: namenode, webhdfs
> Affects Versions: 2.0.0-alpha
> Reporter: Daryn Sharp
> Assignee: Daryn Sharp
> Priority: Critical
>
> Webhdfs currently redirects a client to the DN that physically has one of the
> replicas. Unlike the hdfs data streamer protocol which can easily handle hundreds or thousands of connections, jetty has poor performance under heavy load. Webhdfs clients can easily overwhelm the DNs and likely cause OOMs or excessive GC.
> The NN should redirect the client to a rack-local location to distribute the webhdfs load across multiple hosts. The rack can then use the lightweight streamer protocol.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org