You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@nutch.apache.org by "Mr. Udatny" <ru...@rosa.com> on 2005/11/22 16:22:24 UTC
ndfs / Lost connection to namenode
i use the nutch 0.7.1 distribution
starting namenode seems to be ok (as i can see from the logs)
051122 161211 parsing
file:/home/rude/workspace/nutch-0.7.1/conf/nutch-default.xml
051122 161212 parsing
file:/home/rude/workspace/nutch-0.7.1/conf/nutch-site.xml
051122 161212 Server listener on port 8009: starting
051122 161212 Server handler on 8009: starting
051122 161212 Server handler on 8009: starting
051122 161212 Server handler on 8009: starting
051122 161212 Server handler on 8009: starting
051122 161212 Server handler on 8009: starting
051122 161212 Server handler on 8009: starting
051122 161212 Server handler on 8009: starting
051122 161212 Server handler on 8009: starting
051122 161212 Server handler on 8009: starting
051122 161212 Server handler on 8009: starting
when trying to start ndfs datanode i get the following error:
(i changed the code to show the exception stacktrace)
051122 161220 parsing
file:/home/rude/workspace/nutch-0.7.1/conf/nutch-default.xml
051122 161220 parsing
file:/home/rude/workspace/nutch-0.7.1/conf/nutch-site.xml
051122 161220 Using [/tmp/nutch/ndfs/data] directory for data storage.
051122 161220 Opened server at 7000
051122 161220 Lost connection to namenode [/10.100.100.181:8009].
Retrying...
java.util.NoSuchElementException
at java.util.StringTokenizer.nextToken(StringTokenizer.java:332)
at org.apache.nutch.ndfs.DF.<init>(DF.java:51)
at org.apache.nutch.ndfs.FSDataset.getCapacity(FSDataset.java:204)
at org.apache.nutch.ndfs.NDFS$DataNode.offerService(NDFS.java:476)
at org.apache.nutch.ndfs.NDFS$DataNode.main(NDFS.java:922)
051122 161225 Lost connection to namenode [/10.100.100.181:8009].
Retrying...
java.util.NoSuchElementException
at java.util.StringTokenizer.nextToken(StringTokenizer.java:332)
at org.apache.nutch.ndfs.DF.<init>(DF.java:51)
at org.apache.nutch.ndfs.FSDataset.getCapacity(FSDataset.java:204)
at org.apache.nutch.ndfs.NDFS$DataNode.offerService(NDFS.java:476)
at org.apache.nutch.ndfs.NDFS$DataNode.main(NDFS.java:922)
i could connect to the namenode by just telnetting there:
telnet 10.100.100.181 8009
Trying 10.100.100.181...
Connected to 10.100.100.181 (10.100.100.181).
Escape character is '^]'.
...
further i can use the testclient to create directories and list the
contents of the ndfs. if it comes to a "-put" another exception arises:
[rude@ws-rude nutch-0.7.1]$ ./bin/nutch org.apache.nutch.fs.TestClient -du /
051122 161956 parsing
file:/home/rude/workspace/nutch-0.7.1/conf/nutch-default.xml051122
161956 parsing file:/home/rude/workspace/nutch-0.7.1/conf/nutch-site.xml
051122 161956 No FS indicated, using default:10.100.100.181:8009
051122 161956 Client connection to 10.100.100.181:8009: starting
Found 1 items
/test 0
[rude@ws-rude nutch-0.7.1]$ ./bin/nutch org.apache.nutch.fs.TestClient
-put ./nutch-0.7.1.jar /testfile
051122 162013 parsing
file:/home/rude/workspace/nutch-0.7.1/conf/nutch-default.xml051122
162013 parsing file:/home/rude/workspace/nutch-0.7.1/conf/nutch-site.xml
051122 162013 No FS indicated, using default:10.100.100.181:8009
051122 162013 Client connection to 10.100.100.181:8009: starting
Exception in thread "main" java.io.IOException: Could not obtain new
output block for file /testfile
at
org.apache.nutch.ndfs.NDFSClient$NameNodeCaller.getNewOutputBlock(NDFSClient.java:921)
at
org.apache.nutch.ndfs.NDFSClient$NDFSOutputStream.nextBlockOutputStream(NDFSClient.java:616)
at
org.apache.nutch.ndfs.NDFSClient$NDFSOutputStream.<init>(NDFSClient.java:597)
at org.apache.nutch.ndfs.NDFSClient.create(NDFSClient.java:85)
at org.apache.nutch.fs.NDFSFileSystem.create(NDFSFileSystem.java:76)
at org.apache.nutch.fs.NDFSFileSystem.create(NDFSFileSystem.java:71)
at
org.apache.nutch.fs.NDFSFileSystem.doFromLocalFile(NDFSFileSystem.java:183)
at
org.apache.nutch.fs.NDFSFileSystem.copyFromLocalFile(NDFSFileSystem.java:158)
at org.apache.nutch.fs.TestClient.copyFromLocal(TestClient.java:46)
at org.apache.nutch.fs.TestClient.main(TestClient.java:234)
[rude@ws-rude nutch-0.7.1]$
who tells me what i do wrong?
regards,
ud
Re: ndfs / Lost connection to namenode
Posted by "Mr. Udatny" <ru...@rosa.com>.
i checked out the latest mapred branch version from svn
and it seems to work!
java -cp ./:./classes/:./conf org.apache.nutch.ndfs.NameNode &
java -cp ./:./classes/:./conf org.apache.nutch.ndfs.DataNode &
java -cp ./:./classes/:./conf org.apache.nutch.fs.TestClient -put
somebigfile /testfile
java -cp ./:./classes/:./conf org.apache.nutch.fs.TestClient -report
cool.
ud
Mr. Udatny wrote:
> i use the nutch 0.7.1 distribution
>
> starting namenode seems to be ok (as i can see from the logs)
>
> 051122 161211 parsing
> file:/home/rude/workspace/nutch-0.7.1/conf/nutch-default.xml
> 051122 161212 parsing
> file:/home/rude/workspace/nutch-0.7.1/conf/nutch-site.xml
> 051122 161212 Server listener on port 8009: starting
> 051122 161212 Server handler on 8009: starting
> 051122 161212 Server handler on 8009: starting
> 051122 161212 Server handler on 8009: starting
> 051122 161212 Server handler on 8009: starting
> 051122 161212 Server handler on 8009: starting
> 051122 161212 Server handler on 8009: starting
> 051122 161212 Server handler on 8009: starting
> 051122 161212 Server handler on 8009: starting
> 051122 161212 Server handler on 8009: starting
> 051122 161212 Server handler on 8009: starting
>
>
> when trying to start ndfs datanode i get the following error:
> (i changed the code to show the exception stacktrace)
>
> 051122 161220 parsing
> file:/home/rude/workspace/nutch-0.7.1/conf/nutch-default.xml
> 051122 161220 parsing
> file:/home/rude/workspace/nutch-0.7.1/conf/nutch-site.xml
> 051122 161220 Using [/tmp/nutch/ndfs/data] directory for data storage.
> 051122 161220 Opened server at 7000
> 051122 161220 Lost connection to namenode [/10.100.100.181:8009].
> Retrying...
> java.util.NoSuchElementException
> at java.util.StringTokenizer.nextToken(StringTokenizer.java:332)
> at org.apache.nutch.ndfs.DF.<init>(DF.java:51)
> at org.apache.nutch.ndfs.FSDataset.getCapacity(FSDataset.java:204)
> at org.apache.nutch.ndfs.NDFS$DataNode.offerService(NDFS.java:476)
> at org.apache.nutch.ndfs.NDFS$DataNode.main(NDFS.java:922)
> 051122 161225 Lost connection to namenode [/10.100.100.181:8009].
> Retrying...
> java.util.NoSuchElementException
> at java.util.StringTokenizer.nextToken(StringTokenizer.java:332)
> at org.apache.nutch.ndfs.DF.<init>(DF.java:51)
> at org.apache.nutch.ndfs.FSDataset.getCapacity(FSDataset.java:204)
> at org.apache.nutch.ndfs.NDFS$DataNode.offerService(NDFS.java:476)
> at org.apache.nutch.ndfs.NDFS$DataNode.main(NDFS.java:922)
>
>
> i could connect to the namenode by just telnetting there:
> telnet 10.100.100.181 8009
> Trying 10.100.100.181...
> Connected to 10.100.100.181 (10.100.100.181).
> Escape character is '^]'.
> ...
>
>
> further i can use the testclient to create directories and list the
> contents of the ndfs. if it comes to a "-put" another exception arises:
> [rude@ws-rude nutch-0.7.1]$ ./bin/nutch org.apache.nutch.fs.TestClient
> -du /
> 051122 161956 parsing
> file:/home/rude/workspace/nutch-0.7.1/conf/nutch-default.xml051122
> 161956 parsing file:/home/rude/workspace/nutch-0.7.1/conf/nutch-site.xml
> 051122 161956 No FS indicated, using default:10.100.100.181:8009
> 051122 161956 Client connection to 10.100.100.181:8009: starting
> Found 1 items
> /test 0
> [rude@ws-rude nutch-0.7.1]$ ./bin/nutch org.apache.nutch.fs.TestClient
> -put ./nutch-0.7.1.jar /testfile
> 051122 162013 parsing
> file:/home/rude/workspace/nutch-0.7.1/conf/nutch-default.xml051122
> 162013 parsing file:/home/rude/workspace/nutch-0.7.1/conf/nutch-site.xml
> 051122 162013 No FS indicated, using default:10.100.100.181:8009
> 051122 162013 Client connection to 10.100.100.181:8009: starting
> Exception in thread "main" java.io.IOException: Could not obtain new
> output block for file /testfile
> at
> org.apache.nutch.ndfs.NDFSClient$NameNodeCaller.getNewOutputBlock(NDFSClient.java:921)
>
> at
> org.apache.nutch.ndfs.NDFSClient$NDFSOutputStream.nextBlockOutputStream(NDFSClient.java:616)
>
> at
> org.apache.nutch.ndfs.NDFSClient$NDFSOutputStream.<init>(NDFSClient.java:597)
>
> at org.apache.nutch.ndfs.NDFSClient.create(NDFSClient.java:85)
> at
> org.apache.nutch.fs.NDFSFileSystem.create(NDFSFileSystem.java:76)
> at
> org.apache.nutch.fs.NDFSFileSystem.create(NDFSFileSystem.java:71)
> at
> org.apache.nutch.fs.NDFSFileSystem.doFromLocalFile(NDFSFileSystem.java:183)
>
> at
> org.apache.nutch.fs.NDFSFileSystem.copyFromLocalFile(NDFSFileSystem.java:158)
>
> at
> org.apache.nutch.fs.TestClient.copyFromLocal(TestClient.java:46)
> at org.apache.nutch.fs.TestClient.main(TestClient.java:234)
> [rude@ws-rude nutch-0.7.1]$
>
>
>
> who tells me what i do wrong?
>
> regards,
>
> ud
>
>