Happy World IPv6 Day!

[Brian Bockelman <bb...@cse.unl.edu>]

Hi,

Well, I feel compelled to ask: what are folks opinions about the IPv6-readiness of Hadoop?

Doing some searching around, I think it's stated up-front that Hadoop is not IPv6 ready, and there's little interest in IPv6 support as Hadoop is meant to be within the data center (which I suppose stays on IPv4?).  This leads to two questions:

1) How "un-ready" is the stack for IPv6?  Is it a matter of sending a committer into a dark room for 6 months?  Or are there protocol issues?
2) How strongly do folks feel about not even attempting IPv4?

Alright - everyone go to work, ponder these questions, and attempt to get an IPv6 address.

Brian

Re: Happy World IPv6 Day!

[Steve Loughran <st...@apache.org>]

On 06/08/2011 02:25 PM, Brian Bockelman wrote:
> Hi,
>
> Well, I feel compelled to ask: what are folks opinions about the IPv6-readiness of Hadoop?
>
> Doing some searching around, I think it's stated up-front that Hadoop is not IPv6 ready, and there's little interest in IPv6 support as Hadoop is meant to be within the data center (which I suppose stays on IPv4?).  This leads to two questions:
>
> 1) How "un-ready" is the stack for IPv6?  Is it a matter of sending a committer into a dark room for 6 months?  Or are there protocol issues?
> 2) How strongly do folks feel about not even attempting IPv4?
>
> Alright - everyone go to work, ponder these questions, and attempt to get an IPv6 address.
>

funnily enough, I'm just forcing IPv6 off on multiple Java processes on 
a new RHEL6 machine, which I found deeply ironic
http://jira.smartfrog.org/jira/browse/SFOS-1601

I did some of the (hostname, IP address) caching code, primarily to get 
it working at home on a mess of a network (netgear aren't the 
recommended vendors of hadoop ToR switches). It doesn't like IPv4 there, 
but that's just an echo of the lack of IPv6 support.

Places you need to play
  -Jetty. Works w/ IPv6
  -RPC mechanisms
  -DFS

I wouldn't send someone into a dark room for 6 mo, though it is 
something that could/should be done on a branch by whoever is 
interested, and it will complicate testing. How big is the IPv6 cluster 
you have for people to test on, Brian?

What would make sense would be to slowly add it it up: RPC, HDFS, then 
MR, layers on top (Hive, HBase, etc), and see what performance hit you 
get by going to IPv6. Addresses get bigger, if they are cached in the NN 
and JT that could limit cluster size.

The big question is: do we need IPv6? I've just turned it off in my 
code, and most datacentres will only need a set of IPv4 addresses. Where 
IPv6 matters is when the organisation is going to IPv6, and they want to 
provide external access to the whole FS.