You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@nutch.apache.org by zzcgiacomini <zz...@echo.fr> on 2006/04/28 13:00:26 UTC
Connection refused tasktracker on slave machine
Hello everybody,
I am new to Nutch, I just start evaluating it a couple of days ago....
I have installed the yestarday nightlty build on two machines for
testing, one is running as a master
the second one is my only slave right now.
The ssh on the two machines has been configured properly so I can login
with no password between them
On the Master machine I have started to crawl.
Hadoop the DFS is working fine I can see from the logs that the slave
machines is receiving blocks from the master.
My problem is the tasktraker on the slave machine. When started it get
connected to the jobtracker on the master machine
but as soon as this late one seams to dispatch tasks to the slave then I
get the following error (see log below)
From the code in TaskTracker.java:756 I can not deduce much more that
is a FSError
Any helps ?
060428 120134 parsing
jar:file:/ke/disk10/nutch-0.8-dev/lib/hadoop-0.1.1.jar!/hadoop-default.xml
060428 120134 parsing file:/ke/disk10/nutch-0.8-dev/conf/hadoop-site.xml
060428 120134 Starting tracker tracker_61301
060428 120134 parsing
jar:file:/ke/disk10/nutch-0.8-dev/lib/hadoop-0.1.1.jar!/hadoop-default.xml
060428 120134 parsing
jar:file:/ke/disk10/nutch-0.8-dev/lib/hadoop-0.1.1.jar!/mapred-default.xml
060428 120134 parsing file:/ke/disk10/nutch-0.8-dev/conf/hadoop-site.xml
060428 120134 Server listener on port 50050: starting
060428 120134 Server handler 0 on 50050: starting
060428 120134 Server handler 1 on 50050: starting
060428 120134 Server listener on port 50040: starting
060428 120134 Server handler 0 on 50040: starting
060428 120134 parsing
jar:file:/ke/disk10/nutch-0.8-dev/lib/hadoop-0.1.1.jar!/hadoop-default.xml
060428 120134 parsing
jar:file:/ke/disk10/nutch-0.8-dev/lib/hadoop-0.1.1.jar!/mapred-default.xml
060428 120134 parsing file:/ke/disk10/nutch-0.8-dev/conf/hadoop-site.xml
060428 120134 Server handler 1 on 50040: starting
060428 120134 Client connection to 10.234.57.38:9011: starting
060428 120304 parsing
jar:file:/ke/disk10/nutch-0.8-dev/lib/hadoop-0.1.1.jar!/hadoop-default.xml
060428 120304 parsing
jar:file:/ke/disk10/nutch-0.8-dev/lib/hadoop-0.1.1.jar!/mapred-default.xml
060428 120304 parsing file:/ke/disk10/nutch-0.8-dev/conf/hadoop-site.xml
060428 120304 Lost connection to JobTracker
[bas025.dev.gen01.ke.wanadoo.fr/10.234.57.38:9011]. Retrying...
java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:333)
at
java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:195)
at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
at java.net.Socket.connect(Socket.java:507)
at java.net.Socket.connect(Socket.java:457)
at java.net.Socket.<init>(Socket.java:365)
at java.net.Socket.<init>(Socket.java:207)
at org.apache.hadoop.ipc.Client$Connection.<init>(Client.java:114)
at org.apache.hadoop.ipc.Client.getConnection(Client.java:352)
at org.apache.hadoop.ipc.Client.call(Client.java:290)
at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:141)
at org.apache.hadoop.dfs.$Proxy1.isDir(Unknown Source)
at org.apache.hadoop.dfs.DFSClient.isDirectory(DFSClient.java:127)
at
org.apache.hadoop.dfs.DistributedFileSystem.isDirectory(DistributedFileSystem.java:108)
at
org.apache.hadoop.dfs.DistributedFileSystem.copyToLocalFile(DistributedFileSystem.java:216)
at
org.apache.hadoop.mapred.TaskTracker$TaskInProgress.localizeTask(TaskTracker.java:397)
at
org.apache.hadoop.mapred.TaskTracker$TaskInProgress.<init>(TaskTracker.java:383)
at
org.apache.hadoop.mapred.TaskTracker.offerService(TaskTracker.java:270)
at org.apache.hadoop.mapred.TaskTracker.run(TaskTracker.java:336)
at org.apache.hadoop.mapred.TaskTracker.main(TaskTracker.jav a:756)
060428 120309 parsing
jar:file:/ke/disk10/nutch-0.8-dev/lib/hadoop-0.1.1.jar!/hadoop-default.xml
....
....
Re: Connection refused tasktracker on slave machine
Posted by zzcgiacomini <zz...@echo.fr>.
I got it solved by recompiling nutch using the new hadoop-0.2-dev.jar
from the nightly instead
of using the hadoop-0.1.1.jar originally in nutch/trunk/libs
-Corrado
zzcgiacomini wrote:
> Hello everybody,
> I am new to Nutch, I just start evaluating it a couple of days ago....
> I have installed the yestarday nightlty build on two machines for
> testing, one is running as a master
> the second one is my only slave right now. The ssh on the two machines
> has been configured properly so I can login with no password between them
> On the Master machine I have started to crawl.
> Hadoop the DFS is working fine I can see from the logs that the slave
> machines is receiving blocks from the master.
>
> My problem is the tasktraker on the slave machine. When started it get
> connected to the jobtracker on the master machine
> but as soon as this late one seams to dispatch tasks to the slave then
> I get the following error (see log below)
> From the code in TaskTracker.java:756 I can not deduce much more that
> is a FSError
>
>
> Any helps ?
>
> 060428 120134 parsing
> jar:file:/ke/disk10/nutch-0.8-dev/lib/hadoop-0.1.1.jar!/hadoop-default.xml
>
> 060428 120134 parsing file:/ke/disk10/nutch-0.8-dev/conf/hadoop-site.xml
> 060428 120134 Starting tracker tracker_61301
> 060428 120134 parsing
> jar:file:/ke/disk10/nutch-0.8-dev/lib/hadoop-0.1.1.jar!/hadoop-default.xml
>
> 060428 120134 parsing
> jar:file:/ke/disk10/nutch-0.8-dev/lib/hadoop-0.1.1.jar!/mapred-default.xml
>
> 060428 120134 parsing file:/ke/disk10/nutch-0.8-dev/conf/hadoop-site.xml
> 060428 120134 Server listener on port 50050: starting
> 060428 120134 Server handler 0 on 50050: starting
> 060428 120134 Server handler 1 on 50050: starting
> 060428 120134 Server listener on port 50040: starting
> 060428 120134 Server handler 0 on 50040: starting
> 060428 120134 parsing
> jar:file:/ke/disk10/nutch-0.8-dev/lib/hadoop-0.1.1.jar!/hadoop-default.xml
>
> 060428 120134 parsing
> jar:file:/ke/disk10/nutch-0.8-dev/lib/hadoop-0.1.1.jar!/mapred-default.xml
>
> 060428 120134 parsing file:/ke/disk10/nutch-0.8-dev/conf/hadoop-site.xml
> 060428 120134 Server handler 1 on 50040: starting
> 060428 120134 Client connection to 10.234.57.38:9011: starting
> 060428 120304 parsing
> jar:file:/ke/disk10/nutch-0.8-dev/lib/hadoop-0.1.1.jar!/hadoop-default.xml
>
> 060428 120304 parsing
> jar:file:/ke/disk10/nutch-0.8-dev/lib/hadoop-0.1.1.jar!/mapred-default.xml
>
> 060428 120304 parsing file:/ke/disk10/nutch-0.8-dev/conf/hadoop-site.xml
> 060428 120304 Lost connection to JobTracker
> [bas025.dev.gen01.ke.wanadoo.fr/10.234.57.38:9011]. Retrying...
> java.net.ConnectException: Connection refused
> at java.net.PlainSocketImpl.socketConnect(Native Method)
> at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:333)
> at
> java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:195)
> at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:182)
> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
> at java.net.Socket.connect(Socket.java:507)
> at java.net.Socket.connect(Socket.java:457)
> at java.net.Socket.<init>(Socket.java:365)
> at java.net.Socket.<init>(Socket.java:207)
> at org.apache.hadoop.ipc.Client$Connection.<init>(Client.java:114)
> at org.apache.hadoop.ipc.Client.getConnection(Client.java:352)
> at org.apache.hadoop.ipc.Client.call(Client.java:290)
> at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:141)
> at org.apache.hadoop.dfs.$Proxy1.isDir(Unknown Source)
> at org.apache.hadoop.dfs.DFSClient.isDirectory(DFSClient.java:127)
> at
> org.apache.hadoop.dfs.DistributedFileSystem.isDirectory(DistributedFileSystem.java:108)
>
> at
> org.apache.hadoop.dfs.DistributedFileSystem.copyToLocalFile(DistributedFileSystem.java:216)
>
> at
> org.apache.hadoop.mapred.TaskTracker$TaskInProgress.localizeTask(TaskTracker.java:397)
>
> at
> org.apache.hadoop.mapred.TaskTracker$TaskInProgress.<init>(TaskTracker.java:383)
>
> at
> org.apache.hadoop.mapred.TaskTracker.offerService(TaskTracker.java:270)
> at org.apache.hadoop.mapred.TaskTracker.run(TaskTracker.java:336)
> at org.apache.hadoop.mapred.TaskTracker.main(TaskTracker.jav
> a:756)
> 060428 120309 parsing
> jar:file:/ke/disk10/nutch-0.8-dev/lib/hadoop-0.1.1.jar!/hadoop-default.xml
>
> ....
> ....
>
>