You are viewing a plain text version of this content. The canonical link for it is here.

Posted to common-user@hadoop.apache.org by "Tom Deckers (tdeckers)" <td...@cisco.com> on 2008/03/04 15:27:53 UTC

Hadoop / HDFS over WAN

How well does HDFS perform over WAN links?  Any best practices to take
into account?

 

Thanks!
Tom.

Re: Hadoop / HDFS over WAN

Posted by Raghu Angadi <ra...@yahoo-inc.com>.

I don't think there are any known deployments of Hadoop over WAN. There 
aren't any WAN specific tweaks or configuration settings present that I 
know of.

Hadoop apps tend to be data intensive. Any more details what the 
configuration likely to be? Will HDFS itself be across WAN?

Some exmpale tweaks could be, if you have a high latency and high 
bandwidth across WAN, each socket connection might need use large 
recv/send buffers for TCP sockets to mask latency.

Raghu.

Tom Deckers (tdeckers) wrote:
> How well does HDFS perform over WAN links?  Any best practices to take
> into account?
> 
>  
> 
> Thanks!
> Tom.
> 
>