You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-user@hadoop.apache.org by Fei Hu <hu...@gmail.com> on 2014/12/12 19:11:42 UTC
Re: Hadoop Installation on Multihomed Networks
I solved the problem by changing the hosts file as follows:
10.10.0.10 10.5.0.10 yngcr10nc01
Thanks,
Fei
> On Nov 11, 2014, at 11:58 AM, daemeon reiydelle <da...@gmail.com> wrote:
>
> You may want to consider configuring host names that embed the subnet in the host name itself (e.g. foo50, foo40, for foo via each of the 50 and 40 subnets). ssh key file contents & etc may have to be fiddled with a bit
>
>
> .......
> “The race is not to the swift,
> nor the battle to the strong,
> but to those who can see it coming and jump aside.” - Hunter Thompson
>
> Daemeon C.M. Reiydelle
> USA (+1) 415.501.0198
> London (+44) (0) 20 8144 9872
>
> On Tue, Nov 11, 2014 at 7:47 AM, Fei Hu <hufei68@gmail.com <ma...@gmail.com>> wrote:
> Hi,
>
> I am trying to install Hadoop1.0.4 on multihomed networks. I have done it as the link http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-hdfs/HdfsMultihoming.html <http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-hdfs/HdfsMultihoming.html>. But it still doesn’t work.
>
> The datanode could not work. Its log is as the following. In my hdfs-site.xml, I set the ip:10.50.0.10 for datanode, but in the following report, host = yngcr10nc01/10.10.0.10 <http://10.10.0.10/>.I think it may be because in /etc/hosts file, I add the pair of ip and hostname:10.10.0.10 YNGCR10NC01 before.
>
> The problem now is that I could not add 10.50.0.10 YNGCR10NC01 into hosts file, because 10.10.0.10 YNGCR10NC01 is necessary for another program.
>
> Is there any way to solve the problem on multihomed networks?
>
> Thanks in advance,
> Fei Hu
>
>
>
>
>
> 2014-11-11 04:20:28,228 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: STARTUP_MSG:
> /************************************************************
> STARTUP_MSG: Starting DataNode
> STARTUP_MSG: host = yngcr10nc01/10.10.0.10 <http://10.10.0.10/>
> STARTUP_MSG: args = []
> STARTUP_MSG: version = 1.0.4
> STARTUP_MSG: build = https://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.0 <https://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.0> -r 1393290; compiled by 'hortonfo' on Wed Oct 3 05:13:58 UTC 2012
> ************************************************************/
> 2014-11-11 04:20:28,436 INFO org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from hadoop-metrics2.properties
> 2014-11-11 04:20:28,446 INFO org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source MetricsSystem,sub=Stats registered.
> 2014-11-11 04:20:28,447 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot period at 10 second(s).
> 2014-11-11 04:20:28,447 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: DataNode metrics system started
> 2014-11-11 04:20:28,572 INFO org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source ugi registered.
> 2014-11-11 04:20:28,607 INFO org.apache.hadoop.util.NativeCodeLoader: Loaded the native-hadoop library
> 2014-11-11 04:20:29,870 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: yngcr11hm01/10.50.0.5:9000 <http://10.50.0.5:9000/>. Already tried 0 time(s).
> 2014-11-11 04:20:30,871 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: yngcr11hm01/10.50.0.5:9000 <http://10.50.0.5:9000/>. Already tried 1 time(s).
> 2014-11-11 04:20:31,872 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: yngcr11hm01/10.50.0.5:9000 <http://10.50.0.5:9000/>. Already tried 2 time(s).
> 2014-11-11 04:20:32,873 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: yngcr11hm01/10.50.0.5:9000 <http://10.50.0.5:9000/>. Already tried 3 time(s).
> 2014-11-11 04:20:33,874 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: yngcr11hm01/10.50.0.5:9000 <http://10.50.0.5:9000/>. Already tried 4 time(s).
> 2014-11-11 04:20:34,875 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: yngcr11hm01/10.50.0.5:9000 <http://10.50.0.5:9000/>. Already tried 5 time(s).
> 2014-11-11 04:20:35,876 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: yngcr11hm01/10.50.0.5:9000 <http://10.50.0.5:9000/>. Already tried 6 time(s).
> 2014-11-11 04:20:36,877 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: yngcr11hm01/10.50.0.5:9000 <http://10.50.0.5:9000/>. Already tried 7 time(s).
> 2014-11-11 04:20:37,877 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: yngcr11hm01/10.50.0.5:9000 <http://10.50.0.5:9000/>. Already tried 8 time(s).
> 2014-11-11 04:20:38,879 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: yngcr11hm01/10.50.0.5:9000 <http://10.50.0.5:9000/>. Already tried 9 time(s).
> 2014-11-11 04:20:38,884 ERROR org.apache.hadoop.hdfs.server.datanode.DataNode: java.io.IOException: Call to yngcr11hm01/10.50.0.5:9000 <http://10.50.0.5:9000/> failed on local exception: java.net.NoRouteToHostException: No route to host
> at org.apache.hadoop.ipc.Client.wrapException(Client.java:1107)
> at org.apache.hadoop.ipc.Client.call(Client.java:1075)
> at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:225)
> at com.sun.proxy.$Proxy5.getProtocolVersion(Unknown Source)
> at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:396)
> at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:370)
> at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:429)
> at org.apache.hadoop.ipc.RPC.waitForProxy(RPC.java:331)
> at org.apache.hadoop.ipc.RPC.waitForProxy(RPC.java:296)
> at org.apache.hadoop.hdfs.server.datanode.DataNode.startDataNode(DataNode.java:356)
> at org.apache.hadoop.hdfs.server.datanode.DataNode.<init>(DataNode.java:299)
> at org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNode.java:1582)
> at org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:1521)
> at org.apache.hadoop.hdfs.server.datanode.DataNode.createDataNode(DataNode.java:1539)
> at org.apache.hadoop.hdfs.server.datanode.DataNode.secureMain(DataNode.java:1665)
> at org.apache.hadoop.hdfs.server.datanode.DataNode.main(DataNode.java:1682)
> Caused by: java.net.NoRouteToHostException: No route to host
> at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
> at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:597)
> at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
> at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:489)
> at org.apache.hadoop.ipc.Client$Connection.setupConnection(Client.java:434)
> at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:560)
> at org.apache.hadoop.ipc.Client$Connection.access$2000(Client.java:184)
> at org.apache.hadoop.ipc.Client.getConnection(Client.java:1206)
> at org.apache.hadoop.ipc.Client.call(Client.java:1050)
> ... 14 more
>
> 2014-11-11 04:20:38,887 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: SHUTDOWN_MSG:
> /************************************************************
> SHUTDOWN_MSG: Shutting down DataNode at yngcr10nc01/10.10.0.10 <http://10.10.0.10/>
> ************************************************************/
>
> ifconfig:
> brpriv Link encap:Ethernet HWaddr EC:F4:BB:C4:86:2A
> inet addr:10.100.0.10 Bcast:10.100.255.255 Mask:255.255.0.0
> inet6 addr: fe80::eef4:bbff:fec4:862a/64 Scope:Link
> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
> RX packets:1493 errors:0 dropped:0 overruns:0 frame:0
> TX packets:18 errors:0 dropped:0 overruns:0 carrier:0
> collisions:0 txqueuelen:0
> RX bytes:73898 (72.1 KiB) TX bytes:1068 (1.0 KiB)
>
> brpub Link encap:Ethernet HWaddr EC:F4:BB:C4:86:28
> inet addr:10.10.0.10 Bcast:10.10.255.255 Mask:255.255.0.0
> inet6 addr: fe80::eef4:bbff:fec4:8628/64 Scope:Link
> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
> RX packets:1274476 errors:0 dropped:0 overruns:0 frame:0
> TX packets:1013239 errors:0 dropped:0 overruns:0 carrier:0
> collisions:0 txqueuelen:0
> RX bytes:1488923684 (1.3 GiB) TX bytes:1142276754 (1.0 GiB)
>
> brpub:1 Link encap:Ethernet HWaddr EC:F4:BB:C4:86:28
> inet addr:10.10.3.10 Bcast:10.10.255.255 Mask:255.255.0.0
> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
>
> em1 Link encap:Ethernet HWaddr EC:F4:BB:C4:86:28
> inet6 addr: fe80::eef4:bbff:fec4:8628/64 Scope:Link
> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
> RX packets:6676964 errors:0 dropped:0 overruns:0 frame:0
> TX packets:1337128 errors:0 dropped:0 overruns:0 carrier:0
> collisions:0 txqueuelen:1000
> RX bytes:5882865885 (5.4 GiB) TX bytes:1163653184 (1.0 GiB)
>
> em2 Link encap:Ethernet HWaddr EC:F4:BB:C4:86:2A
> inet6 addr: fe80::eef4:bbff:fec4:862a/64 Scope:Link
> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
> RX packets:16608 errors:0 dropped:0 overruns:0 frame:0
> TX packets:22 errors:0 dropped:0 overruns:0 carrier:0
> collisions:0 txqueuelen:1000
> RX bytes:1004640 (981.0 KiB) TX bytes:1356 (1.3 KiB)
>
> em3 Link encap:Ethernet HWaddr EC:F4:BB:C4:86:2C
> inet addr:10.50.0.10 Bcast:10.50.0.255 Mask:255.255.255.0
> inet6 addr: fe80::eef4:bbff:fec4:862c/64 Scope:Link
> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
> RX packets:132716 errors:0 dropped:0 overruns:0 frame:0
> TX packets:103082 errors:0 dropped:0 overruns:0 carrier:0
> collisions:0 txqueuelen:1000
> RX bytes:78672375 (75.0 MiB) TX bytes:14882729 (14.1 MiB)
> Memory:db680000-db700000
>
> lo Link encap:Local Loopback
> inet addr:127.0.0.1 Mask:255.0.0.0
> inet6 addr: ::1/128 Scope:Host
> UP LOOPBACK RUNNING MTU:16436 Metric:1
> RX packets:974 errors:0 dropped:0 overruns:0 frame:0
> TX packets:974 errors:0 dropped:0 overruns:0 carrier:0
> collisions:0 txqueuelen:0
> RX bytes:118148 (115.3 KiB) TX bytes:118148 (115.3 KiB)
>
> p5p1 Link encap:Ethernet HWaddr 00:0A:F7:59:34:B0
> inet addr:10.30.0.10 Bcast:10.30.255.255 Mask:255.255.0.0
> inet6 addr: fe80::20a:f7ff:fe59:34b0/64 Scope:Link
> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
> RX packets:39190 errors:0 dropped:0 overruns:0 frame:0
> TX packets:14 errors:0 dropped:0 overruns:0 carrier:0
> collisions:0 txqueuelen:1000
> RX bytes:15408562 (14.6 MiB) TX bytes:956 (956.0 b)
> Interrupt:40 Memory:d1000000-d17fffff
>
> p5p2 Link encap:Ethernet HWaddr 00:0A:F7:59:34:B2
> inet addr:10.30.2.10 Bcast:10.30.255.255 Mask:255.255.0.0
> inet6 addr: fe80::20a:f7ff:fe59:34b2/64 Scope:Link
> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
> RX packets:192233 errors:0 dropped:0 overruns:0 frame:0
> TX packets:7 errors:0 dropped:0 overruns:0 carrier:0
> collisions:0 txqueuelen:1000
> RX bytes:12305366 (11.7 MiB) TX bytes:606 (606.0 b)
> Interrupt:44 Memory:d2000000-d27fffff
>
> virbr0 Link encap:Ethernet HWaddr 52:54:00:12:84:4C
> inet addr:192.168.122.1 Bcast:192.168.122.255 Mask:255.255.255.0
> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
> RX packets:0 errors:0 dropped:0 overruns:0 frame:0
> TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
> collisions:0 txqueuelen:0
> RX bytes:0 (0.0 b) TX bytes:0 (0.0 b)
>
>
>