You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@nutch.apache.org by Vishal Shah <vi...@rediff.co.in> on 2007/01/05 15:01:41 UTC
Can't start datanode on slaves (hadoop 0.9.1, nutch nightly build)
Hi,
Even I am getting the same error when I moved from hadoop 0.5 to
0.9.1. The remote datanodes cannot connect to the namenode server, and
remote tasktrackers cannot connect to the jobtracker - I get the same
error.
I followed the nutch-hadoop tutorial while setting up my system. The
namenode was formatted before starting dfs.
Any ideas as to how to debug this problem? I am sure it is not a
network setup issue since the system works with the same config using
nutch0.8, hadoop 0.5.
Also, if someone could send us a sample hadoop-site.xml for
hadoop/nutch 0.9, that would be great.
Thanks,
-vishal.
-----Original Message-----
From: Shailendra Mudgal [mailto:mudgal.shailendra@gmail.com]
Sent: Thursday, January 04, 2007 9:19 PM
To: nutch-user@lucene.apache.org
Subject: nutch 0.9 does not recognize slaves
Hi,
We upgraded our nutch system to 0.9 from 0.8. We are trying to install
nutch0.9 on two machine.
But after running the start-all.sh, it shows only one active node in the
UI
instead of showing 2 nodes.
On server side it is showing each service started successfully.
But at client side the logs are as :
log file name : hadoop-nutch-datanode-websearch3.log
2007-01-04 23:04:00,902 INFO conf.Configuration - parsing
file:/data/nutch/nutch0.9/search/conf/hadoop-default.xml
2007-01-04 23:04:01,072 INFO ipc.Client -
org.apache.hadoop.io.ObjectWritableConnection culler maxidletime= 1000ms
2007-01-04 23:04:01,073 INFO ipc.Client -
org.apache.hadoop.io.ObjectWritable Connection Culler: starting
2007-01-04 23:04:01,108 INFO ipc.Client - Retrying connect to server:
websearch1/10.50.12.220:9000. Already tried 1 time(s).
2007-01-04 23:04:02,110 INFO ipc.Client - Retrying connect to server:
websearch1/10.50.12.220:9000. Already tried 2 time(s).
2007-01-04 23:04:03,120 INFO ipc.Client - Retrying connect to server:
websearch1/10.50.12.220:9000. Already tried 3 time(s).
2007-01-04 23:04:04,130 INFO ipc.Client - Retrying connect to server:
websearch1/10.50.12.220:9000. Already tried 4 time(s).
2007-01-04 23:04:05,140 INFO ipc.Client - Retrying connect to server:
websearch1/10.50.12.220:9000. Already tried 5 time(s).
2007-01-04 23:04:06,150 INFO ipc.Client - Retrying connect to server:
websearch1/10.50.12.220:9000. Already tried 6 time(s).
2007-01-04 23:04:07,160 INFO ipc.Client - Retrying connect to server:
websearch1/10.50.12.220:9000. Already tried 7 time(s).
2007-01-04 23:04:08,170 INFO ipc.Client - Retrying connect to server:
websearch1/10.50.12.220:9000. Already tried 8 time(s).
2007-01-04 23:04:09,180 INFO ipc.Client - Retrying connect to server:
websearch1/10.50.12.220:9000. Already tried 9 time(s).
2007-01-04 23:04:10,190 INFO ipc.Client - Retrying connect to server:
websearch1/10.50.12.220:9000. Already tried 10 time(s).
2007-01-04 23:04:11,200 INFO ipc.RPC - Server at
websearch1/10.50.12.220:9000 not available yet, Zzzzz...
2007-01-04 23:04:12,210 INFO ipc.Client - Retrying connect to server:
websearch1/10.50.12.220:9000. Already tried 1 time(s).
2007-01-04 23:04:13,220 INFO ipc.Client - Retrying connect to server:
websearch1/10.50.12.220:9000. Already tried 2 time(s).
2007-01-04 23:04:14,230 INFO ipc.Client - Retrying connect to server:
websearch1/10.50.12.220:9000. Already tried 3 time(s).
2007-01-04 23:04:15,240 INFO ipc.Client - Retrying connect to server:
websearch1/10.50.12.220:9000. Already tried 4 time(s).
2007-01-04 23:04:16,250 INFO ipc.Client - Retrying connect to server:
websearch1/10.50.12.220:9000. Already tried 5 time(s).
2007-01-04 23:04:17,260 INFO ipc.Client - Retrying connect to server:
websearch1/10.50.12.220:9000. Already tried 6 time(s).
2007-01-04 23:04:18,270 INFO ipc.Client - Retrying connect to server:
websearch1/10.50.12.220:9000. Already tried 7 time(s).
2007-01-04 23:04:19,280 INFO ipc.Client - Retrying connect to server:
websearch1/10.50.12.220:9000. Already tried 8 time(s).
2007-01-04 23:04:20,290 INFO ipc.Client - Retrying connect to server:
websearch1/10.50.12.220:9000. Already tried 9 time(s).
2007-01-04 23:04:21,300 INFO ipc.Client - Retrying connect to server:
websearch1/10.50.12.220:9000. Already tried 10 time(s)
log file name : hadoop-nutch-tasktracker-websearch3.log
2007-01-04 23:04:03,482 INFO conf.Configuration - parsing
file:/data/nutch/nutch0.9/search/conf/hadoop-default.xml
2007-01-04 23:04:03,570 INFO conf.Configuration - parsing
file:/data/nutch/nutch0.9/search/conf/mapred-default.xml
2007-01-04 23:04:03,680 INFO util.Credential - Checking Resource
aliases
2007-01-04 23:04:03,744 INFO http.HttpServer - Version Jetty/5.1.4
2007-01-04 23:04:04,619 INFO util.Container - Started
org.mortbay.jetty.servlet.WebApplicationHandler@39e5b5
2007-01-04 23:04:04,673 INFO util.Container - Started
WebApplicationContext[/,/]
2007-01-04 23:04:04,673 INFO util.Container - Started
HttpContext[/logs,/logs]
2007-01-04 23:04:04,673 INFO util.Container - Started
HttpContext[/static,/static]
2007-01-04 23:04:04,677 INFO http.SocketListener - Started
SocketListener
on 0.0.0.0:50060
2007-01-04 23:04:04,677 INFO util.Container - Started
org.mortbay.jetty.Server@d1fa5
2007-01-04 23:04:04,712 INFO ipc.Server - IPC Server listener on 50050:
starting
2007-01-04 23:04:04,713 INFO ipc.Server - IPC Server handler 0 on
50050:
starting
2007-01-04 23:04:04,713 INFO ipc.Server - IPC Server handler 1 on
50050:
starting
2007-01-04 23:04:04,714 INFO ipc.Server - IPC Server handler 2 on
50050:
starting
2007-01-04 23:04:04,714 INFO ipc.Server - IPC Server handler 3 on
50050:
starting
2007-01-04 23:04:04,714 INFO ipc.Server - IPC Server handler 4 on
50050:
starting
2007-01-04 23:04:04,714 INFO ipc.Server - IPC Server handler 5 on
50050:
starting
2007-01-04 23:04:04,714 INFO ipc.Server - IPC Server handler 6 on
50050:
starting
2007-01-04 23:04:04,714 INFO ipc.Server - IPC Server handler 7 on
50050:
starting
2007-01-04 23:04:04,714 INFO ipc.Server - IPC Server handler 8 on
50050:
starting
2007-01-04 23:04:04,715 INFO ipc.Server - IPC Server handler 10 on
50050:
starting
2007-01-04 23:04:04,714 INFO ipc.Server - IPC Server handler 9 on
50050:
starting
2007-01-04 23:04:04,715 INFO ipc.Server - IPC Server handler 11 on
50050:
starting
2007-01-04 23:04:04,715 INFO ipc.Server - IPC Server handler 12 on
50050:
starting
2007-01-04 23:04:04,715 INFO ipc.Server - IPC Server handler 13 on
50050:
starting
2007-01-04 23:04:04,715 INFO ipc.Server - IPC Server handler 14 on
50050:
starting
2007-01-04 23:04:04,715 INFO ipc.Server - IPC Server handler 15 on
50050:
starting
2007-01-04 23:04:04,715 INFO ipc.Server - IPC Server handler 16 on
50050:
starting
2007-01-04 23:04:04,716 INFO ipc.Server - IPC Server handler 17 on
50050:
starting
2007-01-04 23:04:04,716 INFO ipc.Server - IPC Server handler 18 on
50050:
starting
2007-01-04 23:04:04,716 INFO mapred.TaskTracker - Starting tracker
tracker_websearch3:50050
2007-01-04 23:04:04,716 INFO ipc.Server - IPC Server handler 19 on
50050:
starting
2007-01-04 23:04:04,723 INFO ipc.Client -
org.apache.hadoop.io.ObjectWritableConnection culler maxidletime= 1000ms
2007-01-04 23:04:04,723 INFO ipc.Client -
org.apache.hadoop.io.ObjectWritable Connection Culler: starting
2007-01-04 23:04:04,744 INFO ipc.Client - Retrying connect to server:
websearch1/10.50.12.220:9001. Already tried 1 time(s).
2007-01-04 23:04:05,750 INFO ipc.Client - Retrying connect to server:
websearch1/10.50.12.220:9001. Already tried 2 time(s).
2007-01-04 23:04:06,760 INFO ipc.Client - Retrying connect to server:
websearch1/10.50.12.220:9001. Already tried 3 time(s).
2007-01-04 23:04:07,770 INFO ipc.Client - Retrying connect to server:
websearch1/10.50.12.220:9001. Already tried 4 time(s).
2007-01-04 23:04:08,780 INFO ipc.Client - Retrying connect to server:
websearch1/10.50.12.220:9001. Already tried 5 time(s).
2007-01-04 23:04:09,790 INFO ipc.Client - Retrying connect to server:
websearch1/10.50.12.220:9001. Already tried 6 time(s).
2007-01-04 23:04:10,800 INFO ipc.Client - Retrying connect to server:
websearch1/10.50.12.220:9001. Already tried 7 time(s).
2007-01-04 23:04:11,810 INFO ipc.Client - Retrying connect to server:
websearch1/10.50.12.220:9001. Already tried 8 time(s).
2007-01-04 23:04:12,820 INFO ipc.Client - Retrying connect to server:
websearch1/10.50.12.220:9001. Already tried 9 time(s).
2007-01-04 23:04:13,830 INFO ipc.Client - Retrying connect to server:
websearch1/10.50.12.220:9001. Already tried 10 time(s).
2007-01-04 23:04:14,840 INFO ipc.RPC - Server at
websearch1/10.50.12.220:9001 not available yet, Zzzzz...
2007-01-04 23:04:15,850 INFO ipc.Client - Retrying connect to server:
websearch1/10.50.12.220:9001. Already tried 1 time(s).
2007-01-04 23:04:16,860 INFO ipc.Client - Retrying connect to server:
websearch1/10.50.12.220:9001. Already tried 2 time(s).
2007-01-04 23:04:17,870 INFO ipc.Client - Retrying connect to server:
websearch1/10.50.12.220:9001. Already tried 3 time(s).
2007-01-04 23:04:18,880 INFO ipc.Client - Retrying connect to server:
websearch1/10.50.12.220:9001. Already tried 4 time(s).
2007-01-04 23:04:19,890 INFO ipc.Client - Retrying connect to server:
websearch1/10.50.12.220:9001. Already tried 5 time(s).
2007-01-04 23:04:20,900 INFO ipc.Client - Retrying connect to server:
websearch1/10.50.12.220:9001. Already tried 6 time(s).
2007-01-04 23:04:21,910 INFO ipc.Client - Retrying connect to server:
websearch1/10.50.12.220:9001. Already tried 7 time(s).
2007-01-04 23:04:22,920 INFO ipc.Client - Retrying connect to server:
websearch1/10.50.12.220:9001. Already tried 8 time(s).
2007-01-04 23:04:23,930 INFO ipc.Client - Retrying connect to server:
websearch1/10.50.12.220:9001. Already tried 9 time(s).
2007-01-04 23:04:24,940 INFO ipc.Client - Retrying connect to server:
websearch1/10.50.12.220:9001. Already tried 10 time(s)
The content of hadoop-site.xml is as follows :
<configuration>
<property>
<name>fs.default.name</name>
<value>websearch1:9000</value>
</property>
<property>
<name>mapred.job.tracker</name>
<value>websearch1:9001</value>
</property>
<property>
<name>mapred.tasktracker.tasks.maximum </name>
<value>20</value>
</property>
<property>
<name>mapred.map.tasks</name>
<value>60</value>
</property>
<property>
<name>mapred.reduce.tasks</name>
<value>6</value>
</property>
<property>
<name>dfs.name.dir</name>
<value>/data/nutch/nutch0.9/filesystem0.9/name</value>
</property>
<property>
<name>dfs.data.dir</name>
<value>/data/nutch/nutch0.9/filesystem0.9/data</value>
</property>
<property>
<name> dfs.replication</name>
<value>1</value>
</property>
</configuration>
Does anybody has the idea about this problem?