You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Khalil Honsali <k....@gmail.com> on 2007/11/23 10:31:46 UTC
(repost3) Problem: [multi-node setup] addresss + DNS + ipc.client {query mailist = {!0}
Greetings;
I followed the excellent tutorials on the wiki, everything worked fine for
the single node version,
but for the multi-node setup (four nodes, including master), I had to use ip
addresses instead of fully qualified domain names in the
hadoop-site.xml(see appendix)
otherwise I get the error "Retrying connect to server" from both datanode
and task tracker's ipc.Client.
Connectivity is ok, ssh, host, traceroute, iptables all ok
I upgraded to hadoop-1.15 and removed ipv6 to kill the doubt , still the
problem is there...
What I am suspicious about, is the address retrieved by the ipcClient, which
is printed in the datanode info log as follow:
2007-11-22 14:52:09,734 INFO org.apache.hadoop.ipc.Client: Retrying connect
to server: speech02.matlab.nitech.ac.jp/133.68.15.85:54310. Already tried 10
time(s).
2007-11-22 14:52:10,742 INFO org.apache.hadoop.ipc.RPC: Server at
speech02.matlab.nitech.ac.jp/133.68.15.85:54310 not available yet, Zzzzz...
as you can see , it is a combination of fully qualified domain name + / +
ip address, i wonder if it is used as is....
even if I use strict ip addressing in the setup , hadoop still manages to
use the fully qualified name in the log (how?) like here:
133.68.15.85: starting tasktracker, logging to
/PFS/hadoop-0.15.0/bin/../logs/hadoop-hadoopsan-tasktracker-
speech02.matlab.nitech.ac.jp.out
Many thanks in advance for your time
K. Honsali
// PS \\ On a 'healthy' setup, I also got 'false' results from the {
test.jar mapredtest 5 50 }
Mismatch! Pos=1, i=0, val=1, dist[i]=9
Mismatch! Pos=2, i=1, val=2, dist[i]=9
Mismatch! Pos=3, i=2, val=3, dist[i]=11
Mismatch! Pos=4, i=3, val=4, dist[i]=7
Cannot read entry 4
Original sum: 50
Recomputed sum: 10
Success=false
Appendix:
---------------
healthy hadoop-site.xml excerpt (using ip)
<property>
<name> fs.default.name</name>
<value>hdfs://133.68.15.85:54310</value>
</property>
//*all the rest is default , apart from map tasks = 4*10 and reduce tasks =
4*2
<property>
<name> mapred.job.tracker</name>
<value>133.68.15.85:54311</value>
</property>
unhealthy hadoop-site.xml:
<property>
<name>fs.default.name</name>
<value>hdfs://speech02.matlab.nitech.ac.jp:54310</value>
</property>
<property>
<name>mapred.job.tracker</name>
<value>speech02.matlab.nitech.ac.jp:54311</value>
</property>
--
Re: (repost3) Problem: [multi-node setup] addresss + DNS + ipc.client
{query mailist = {!0}
Posted by Raghu Angadi <ra...@yahoo-inc.com>.
Khalil Honsali wrote:
> Greetings;
>
> I followed the excellent tutorials on the wiki, everything worked fine for
> the single node version,
> but for the multi-node setup (four nodes, including master), I had to use ip
> addresses instead of fully qualified domain names in the
> hadoop-site.xml(see appendix)
> otherwise I get the error "Retrying connect to server" from both datanode
> and task tracker's ipc.Client.
You didn't attach the config that does not work. Check the NameNode log.
Most likely reason is that resolving the hostname you configured would
have resulted in unexpected IP on the Namenode.
Raghu.
Re: Fwd: (repost3) Problem: [multi-node setup] addresss + DNS + ipc.client
{query mailist = {!0}
Posted by Sagar Naik <sa...@visvo.com>.
Hey,
Client log calls toString function on InetSocketAddress which in turn
calls InetAddress's toString().
InetAddress forms a string of the type : hostname + "/" + ip address
and the InetSocketAddress appends a ":" + port
So logs prints the connection info rightly.
pl double check the conf on client and servers (shud be same)
If not, my guess wud be some n/w issues.
You cud write a simple Java socket program to test it out.
Hope this helps.
khalil honsali wrote:
> Greetings;
>
> I followed the excellent tutorials on the wiki, everything worked fine for
> the single node version,
> but for the multi-node setup (four nodes, including master), I had to use ip
> addresses instead of fully qualified domain names in the
> hadoop-site.xml(see appendix)
> otherwise I get the error "Retrying connect to server" from both datanode
> and task tracker's ipc.Client.
>
> Connectivity is ok, ssh, host, traceroute, iptables all ok
> I upgraded to hadoop-1.15 and removed ipv6 to kill the doubt , still the
> problem is there...
>
> What I am suspicious about, is the address retrieved by the ipcClient, which
> is printed in the datanode info log as follow:
>
> 2007-11-22 14:52:09,734 INFO org.apache.hadoop.ipc.Client: Retrying connect
> to server: speech02.matlab.nitech.ac.jp/133.68.15.85:54310. Already tried 10
> time(s).
> 2007-11-22 14:52:10,742 INFO org.apache.hadoop.ipc.RPC: Server at
> speech02.matlab.nitech.ac.jp/133.68.15.85:54310 not available yet, Zzzzz...
>
> as you can see , it is a combination of fully qualified domain name + / +
> ip address, i wonder if it is used as is....
> even if I use strict ip addressing in the setup , hadoop still manages to
> use the fully qualified name in the log (how?) like here:
>
> 133.68.15.85: starting tasktracker, logging to
> /PFS/hadoop-0.15.0/bin/../logs/hadoop-hadoopsan-tasktracker-
> speech02.matlab.nitech.ac.jp.out
>
>
>
>
> Many thanks in advance for your time
>
> K. Honsali
>
>
>
> // PS \\ On a 'healthy' setup, I also got 'false' results from the {
> test.jar mapredtest 5 50 }
> Mismatch! Pos=1, i=0, val=1, dist[i]=9
> Mismatch! Pos=2, i=1, val=2, dist[i]=9
> Mismatch! Pos=3, i=2, val=3, dist[i]=11
> Mismatch! Pos=4, i=3, val=4, dist[i]=7
> Cannot read entry 4
> Original sum: 50
> Recomputed sum: 10
> Success=false
>
>
> Appendix:
> ---------------
> healthy hadoop-site.xml excerpt (using ip)
> <property>
> <name> fs.default.name</name>
> <value>hdfs://133.68.15.85:54310</value>
> </property>
> //*all the rest is default , apart from map tasks = 4*10 and reduce tasks =
> 4*2
> <property>
> <name> mapred.job.tracker</name>
> <value>133.68.15.85:54311</value>
> </property>
>
> unhealthy hadoop-site.xml:
> <property>
> <name>fs.default.name</name>
> <value>hdfs://speech02.matlab.nitech.ac.jp:54310</value>
> </property>
> <property>
> <name>mapred.job.tracker</name>
> <value>speech02.matlab.nitech.ac.jp:54311</value>
> </property>
> --
>
>
--
This message has been scanned for viruses and
dangerous content and is believed to be clean.
Fwd: (repost3) Problem: [multi-node setup] addresss + DNS + ipc.client {query mailist = {!0}
Posted by khalil honsali <ma...@gmail.com>.
Greetings;
I followed the excellent tutorials on the wiki, everything worked fine for
the single node version,
but for the multi-node setup (four nodes, including master), I had to use ip
addresses instead of fully qualified domain names in the
hadoop-site.xml(see appendix)
otherwise I get the error "Retrying connect to server" from both datanode
and task tracker's ipc.Client.
Connectivity is ok, ssh, host, traceroute, iptables all ok
I upgraded to hadoop-1.15 and removed ipv6 to kill the doubt , still the
problem is there...
What I am suspicious about, is the address retrieved by the ipcClient, which
is printed in the datanode info log as follow:
2007-11-22 14:52:09,734 INFO org.apache.hadoop.ipc.Client: Retrying connect
to server: speech02.matlab.nitech.ac.jp/133.68.15.85:54310. Already tried 10
time(s).
2007-11-22 14:52:10,742 INFO org.apache.hadoop.ipc.RPC: Server at
speech02.matlab.nitech.ac.jp/133.68.15.85:54310 not available yet, Zzzzz...
as you can see , it is a combination of fully qualified domain name + / +
ip address, i wonder if it is used as is....
even if I use strict ip addressing in the setup , hadoop still manages to
use the fully qualified name in the log (how?) like here:
133.68.15.85: starting tasktracker, logging to
/PFS/hadoop-0.15.0/bin/../logs/hadoop-hadoopsan-tasktracker-
speech02.matlab.nitech.ac.jp.out
Many thanks in advance for your time
K. Honsali
// PS \\ On a 'healthy' setup, I also got 'false' results from the {
test.jar mapredtest 5 50 }
Mismatch! Pos=1, i=0, val=1, dist[i]=9
Mismatch! Pos=2, i=1, val=2, dist[i]=9
Mismatch! Pos=3, i=2, val=3, dist[i]=11
Mismatch! Pos=4, i=3, val=4, dist[i]=7
Cannot read entry 4
Original sum: 50
Recomputed sum: 10
Success=false
Appendix:
---------------
healthy hadoop-site.xml excerpt (using ip)
<property>
<name> fs.default.name</name>
<value>hdfs://133.68.15.85:54310</value>
</property>
//*all the rest is default , apart from map tasks = 4*10 and reduce tasks =
4*2
<property>
<name> mapred.job.tracker</name>
<value>133.68.15.85:54311</value>
</property>
unhealthy hadoop-site.xml:
<property>
<name>fs.default.name</name>
<value>hdfs://speech02.matlab.nitech.ac.jp:54310</value>
</property>
<property>
<name>mapred.job.tracker</name>
<value>speech02.matlab.nitech.ac.jp:54311</value>
</property>
--