You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@nutch.apache.org by Ken Krugler <kk...@krugle.net> on 2006/01/14 18:33:28 UTC
Error running MapReduce - Host key verification failed
Hi all,
We've got the nutch-2006-01-12.tar version of Nutch, and are trying
to run it on three machines.
192.168.0.100 is the "master" machine, where we run the JobTracker
and NameNode processes.
192.168.0.101 and 192.168.0.103 are the "slave" machines, where we
rung the TaskTracker and DataNode processes.
When we fire off the the Nuch daemons with ./bin/start-all.sh, we get
the following error right away:
192.168.0.103: rsync from 192.168.0.100:/home/crawler/nutch
192.168.0.103: Host key verification failed.
192.168.0.103: rsync: connection unexpectedly closed (0 bytes
received so far) [receiver]
192.168.0.103: rsync error: error in rsync protocol data stream (code
12) at io.c(420)
192.168.0.103: starting datanode, logging to
/home/crawler/tmp/logs/nutch-crawler-datanode-crawlerw3.log
When I dump the datanode logfile from this 192.168.0.103 machine, I get:
060114 082814 10 parsing file:/home/crawler/nutch/conf/nutch-default.xml
060114 082814 10 parsing file:/home/crawler/nutch/conf/nutch-site.xml
060114 082814 10 Opened server at 50010
060114 082814 11 Starting DataNode in: /home/crawler/tmp/ndfs/data
060114 082814 11 using BLOCKREPORT_INTERVAL of 3314538msec
060114 082814 11 Exception: java.net.ConnectException: Connection refused
060114 082814 11 Lost connection to namenode. Retrying...
060114 082819 11 using BLOCKREPORT_INTERVAL of 3314538msec
060114 082819 11 Exception: java.net.ConnectException: Connection refused
060114 082819 11 Lost connection to namenode. Retrying...
060114 082824 11 using BLOCKREPORT_INTERVAL of 3314538msec
060114 082824 11 Exception: java.net.ConnectException: Connection refused
060114 082824 11 Lost connection to namenode. Retrying...
060114 082829 11 using BLOCKREPORT_INTERVAL of 3314538msec
060114 082829 12 Client connection to 192.168.0.100:8009: starting
When I dump the tasktracker logfile from 192.168.0.103, I get:
060114 082832 parsing file:/home/crawler/nutch/conf/nutch-default.xml
060114 082832 parsing file:/home/crawler/nutch/conf/nutch-site.xml
060114 082832 Server listener on port 50050: starting
060114 082832 Server handler 0 on 50050: starting
060114 082832 Server handler 1 on 50050: starting
060114 082832 Server listener on port 50040: starting
060114 082832 Server handler 0 on 50040: starting
060114 082832 Server handler 1 on 50040: starting
060114 082832 Lost connection to JobTracker
[main1/192.168.0.100:8010]. ex=java.net.ConnectException: Connection
refused Retrying...
It seems like the 192.168.0.103 machine doesn't have the right
settings for connecting to the 192.168.0.100 machine. Is there a way
to check this outside of running Nutch?
Thanks,
-- Ken
--
Ken Krugler
Krugle, Inc.
+1 530-470-9200
Re: Error running MapReduce - Host key verification failed
Posted by Gal Nitzan <gn...@usa.net>.
Hi Ken,
I am not such an expert on linux, however I have solved this problem
with ssh key based authentication.
http://www.cyberciti.biz/nixcraft/vivek/blogger/2004/05/ssh-public-key-based-authentication.php
after this operation you should be able to connect to all your nodes
from your namenode/jobtracker
Hope it helps.
G.
On Sat, 2006-01-14 at 09:33 -0800, Ken Krugler wrote:
> Hi all,
>
> We've got the nutch-2006-01-12.tar version of Nutch, and are trying
> to run it on three machines.
>
> 192.168.0.100 is the "master" machine, where we run the JobTracker
> and NameNode processes.
>
> 192.168.0.101 and 192.168.0.103 are the "slave" machines, where we
> rung the TaskTracker and DataNode processes.
>
> When we fire off the the Nuch daemons with ./bin/start-all.sh, we get
> the following error right away:
>
> 192.168.0.103: rsync from 192.168.0.100:/home/crawler/nutch
> 192.168.0.103: Host key verification failed.
> 192.168.0.103: rsync: connection unexpectedly closed (0 bytes
> received so far) [receiver]
> 192.168.0.103: rsync error: error in rsync protocol data stream (code
> 12) at io.c(420)
> 192.168.0.103: starting datanode, logging to
> /home/crawler/tmp/logs/nutch-crawler-datanode-crawlerw3.log
>
> When I dump the datanode logfile from this 192.168.0.103 machine, I get:
>
> 060114 082814 10 parsing file:/home/crawler/nutch/conf/nutch-default.xml
> 060114 082814 10 parsing file:/home/crawler/nutch/conf/nutch-site.xml
> 060114 082814 10 Opened server at 50010
> 060114 082814 11 Starting DataNode in: /home/crawler/tmp/ndfs/data
> 060114 082814 11 using BLOCKREPORT_INTERVAL of 3314538msec
> 060114 082814 11 Exception: java.net.ConnectException: Connection refused
> 060114 082814 11 Lost connection to namenode. Retrying...
> 060114 082819 11 using BLOCKREPORT_INTERVAL of 3314538msec
> 060114 082819 11 Exception: java.net.ConnectException: Connection refused
> 060114 082819 11 Lost connection to namenode. Retrying...
> 060114 082824 11 using BLOCKREPORT_INTERVAL of 3314538msec
> 060114 082824 11 Exception: java.net.ConnectException: Connection refused
> 060114 082824 11 Lost connection to namenode. Retrying...
> 060114 082829 11 using BLOCKREPORT_INTERVAL of 3314538msec
> 060114 082829 12 Client connection to 192.168.0.100:8009: starting
>
> When I dump the tasktracker logfile from 192.168.0.103, I get:
>
> 060114 082832 parsing file:/home/crawler/nutch/conf/nutch-default.xml
> 060114 082832 parsing file:/home/crawler/nutch/conf/nutch-site.xml
> 060114 082832 Server listener on port 50050: starting
> 060114 082832 Server handler 0 on 50050: starting
> 060114 082832 Server handler 1 on 50050: starting
> 060114 082832 Server listener on port 50040: starting
> 060114 082832 Server handler 0 on 50040: starting
> 060114 082832 Server handler 1 on 50040: starting
> 060114 082832 Lost connection to JobTracker
> [main1/192.168.0.100:8010]. ex=java.net.ConnectException: Connection
> refused Retrying...
>
> It seems like the 192.168.0.103 machine doesn't have the right
> settings for connecting to the 192.168.0.100 machine. Is there a way
> to check this outside of running Nutch?
>
> Thanks,
>
> -- Ken