You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@nutch.apache.org by Rafit Izhak_Ratzin <sa...@hotmail.com> on 2006/02/07 21:30:26 UTC
new svn version:NoClassDefFoundError - JobTracker
Hi,
I am trying to run with the new svn version (375414), I am working under
nutch/trunk directory.
When I ran the next command "bin/hadoop jobtracker" or "bin/hadoop-daemon.sh
start jobtracker"
I got the next message,
Exception in thread "main" java.lang.NoClassDefFoundError:
org/mortbay/http/HttpListener
at org.apache.hadoop.mapred.JobTracker.<init>(JobTracker.java:303)
at
org.apache.hadoop.mapred.JobTracker.startTracker(JobTracker.java:50)
at org.apache.hadoop.mapred.JobTracker.main(JobTracker.java:813)
Thanks,
Rafit
_________________________________________________________________
FREE pop-up blocking with the new MSN Toolbar - get it now!
http://toolbar.msn.click-url.com/go/onm00200415ave/direct/01/
Re: new svn version:NoClassDefFoundError - JobTracker
Posted by Mike Smith <mi...@gmail.com>.
I could make it work in a strange way. There should be a problem with hadoop
jar file.
I downloaded the hadoop project and made the project using ant. then I could
start the jobtracker successfully, but when I removed the build folder and
just used the hadoop jar file, it failed again. So I copied all the
hadoop/lib files into nutch/lib and I also copied the build folder of
hadoop into nutch folder. Now nutch is using build of hadoop (the
classes) instead of its jar file!!! But it worked fine.
I just started a crawl over 70000 seeds and it is going through! I will let
you know the results.
Thanks, Mike
On 2/7/06, Mike Smith <mi...@gmail.com> wrote:
>
> The problem is that jetty jar files are missing from the SVN., I replaced
> the Jetty jar files but I get another exception:
>
> 060207 123447 Property 'sun.cpu.isalist' is
> Exception in thread "main" java.lang.NullPointerException
> at org.apache.hadoop.mapred.JobTrackerInfoServer.<init>(
> JobTrackerInfoServer.java:56)
> at org.apache.hadoop.mapred.JobTracker.<init>(JobTracker.java:303)
> at org.apache.hadoop.mapred.JobTracker.startTracker(
> JobTracker.java:50)
> at org.apache.hadoop.mapred.JobTracker.main (JobTracker.java:813)
>
> Thanks, Mike
>
>
> On 2/7/06, Rafit Izhak_Ratzin <sa...@hotmail.com> wrote:
> >
> > Hi,
> >
> > I am trying to run with the new svn version (375414), I am working under
> > nutch/trunk directory.
> >
> > When I ran the next command "bin/hadoop jobtracker" or "bin/hadoop-
> > daemon.sh
> > start jobtracker"
> > I got the next message,
> >
> > Exception in thread "main" java.lang.NoClassDefFoundError :
> > org/mortbay/http/HttpListener
> > at org.apache.hadoop.mapred.JobTracker.<init>(JobTracker.java
> > :303)
> > at
> > org.apache.hadoop.mapred.JobTracker.startTracker(JobTracker.java:50)
> > at org.apache.hadoop.mapred.JobTracker.main (JobTracker.java:813)
> >
> > Thanks,
> > Rafit
> >
> > _________________________________________________________________
> > FREE pop-up blocking with the new MSN Toolbar - get it now!
> > http://toolbar.msn.click-url.com/go/onm00200415ave/direct/01/
> >
> >
>
Re: new svn version:NoClassDefFoundError - JobTracker
Posted by Rafit Izhak_Ratzin <sa...@hotmail.com>.
After copying teh build directory from hadoop to nutch....
I can run the crawl cycle, however I get the next Exeption (in jobtracking
file) lot of times:
060207 215603 Server connection on port 50020 from IP...
caught: java.lang.RuntimeException: java.lang.ClassNotFoundException:
org.apache.nutch.mapred.TaskTrackerStatus
java.lang.RuntimeException: java.lang.ClassNotFoundException:
org.apache.nutch.mapred.TaskTrackerStatus
at
org.apache.hadoop.io.ObjectWritable.readObject(ObjectWritable.java:183)
at org.apache.hadoop.ipc.RPC$Invocation.readFields(RPC.java:88)
at org.apache.hadoop.ipc.Server$Connection.run(Server.java:138)
060207 215603 Server connection on port 50020 from IP... : exiting
060207 215708 Server connection on port 50020 from IP...: starting
>From: Mike Smith <mi...@gmail.com>
>Reply-To: nutch-user@lucene.apache.org
>To: nutch-user@lucene.apache.org
>Subject: Re: new svn version:NoClassDefFoundError - JobTracker
>Date: Tue, 7 Feb 2006 15:45:26 -0800
>
>I finally finished my first successfull experinece with nutch/hadoop, I
>started from 70,000 seeds and this is results of 1 cycle crawl.
>
>060207 153956 Statistics for CrawlDb: t1/crawldb
>060207 153956 TOTAL urls: 850726
>060207 153956 avg score: 1.037
>060207 153956 max score: 133.269
>060207 153956 min score: 1.0
>060207 153956 retry 0: 849588
>060207 153956 retry 1: 1138
>060207 153956 status 1 (DB_unfetched): 788522
>060207 153956 status 2 (DB_fetched): 60703
>060207 153956 status 3 (DB_gone): 1501
>060207 153956 CrawlDb statistics: done
>
>But, I still have problem with hadoop.jar. I have to use the built classes
>instead!
>
>Thanks, Mike
>
>
>
>
>
>On 2/7/06, Mike Smith <mi...@gmail.com> wrote:
> >
> > Rafit,
> >
> > get the hadoop project make that and use the build folder instead of the
> > jar file. It will work fine then. Something probabely is missing from
>the
> > hadoop jar.
> >
> > M
> >
> >
> > On 2/7/06, Rafit Izhak_Ratzin <sa...@hotmail.com> wrote:
> > >
> > > I am still getting teh next Exception:
> > >
> > > Exception in thread "main" java.lang.NullPointerException
> > > at
> > > org.apache.hadoop.mapred.JobTrackerInfoServer.<init>(
> > > JobTrackerInfoServer.java:56)
> > > at org.apache.hadoop.mapred.JobTracker.<init>(JobTracker.java
> > > :303)
> > > at
> > > org.apache.hadoop.mapred.JobTracker.startTracker (JobTracker.java:50)
> > > at
>org.apache.hadoop.mapred.JobTracker.main(JobTracker.java:813)
> > > 060207 161923 Server handler 9 on 50020: starting
> > >
> > >
> > >
> > >
> > >
> > > >From: Doug Cutting < cutting@apache.org>
> > > >Reply-To: nutch-user@lucene.apache.org
> > > >To: nutch-user@lucene.apache.org
> > > >Subject: Re: new svn version:NoClassDefFoundError - JobTracker
> > > >Date: Tue, 07 Feb 2006 13:04:37 -0800
> > > >
> > > >Mike Smith wrote:
> > > >>The problem is that jetty jar files are missing from the SVN., I
> > > replaced
> > > >>the Jetty jar files but I get another exception:
> > > >
> > > >I just restored the jetty lib and the jetty-ext libs. Does that
>help?
> > > >
> > > >Doug
> > >
> > > _________________________________________________________________
> > > FREE pop-up blocking with the new MSN Toolbar - get it now!
> > > http://toolbar.msn.click-url.com/go/onm00200415ave/direct/01/
> > >
> > >
> >
_________________________________________________________________
Express yourself instantly with MSN Messenger! Download today it's FREE!
http://messenger.msn.click-url.com/go/onm00200471ave/direct/01/
Re: new svn version:NoClassDefFoundError - JobTracker
Posted by Mike Smith <mi...@gmail.com>.
I finally finished my first successfull experinece with nutch/hadoop, I
started from 70,000 seeds and this is results of 1 cycle crawl.
060207 153956 Statistics for CrawlDb: t1/crawldb
060207 153956 TOTAL urls: 850726
060207 153956 avg score: 1.037
060207 153956 max score: 133.269
060207 153956 min score: 1.0
060207 153956 retry 0: 849588
060207 153956 retry 1: 1138
060207 153956 status 1 (DB_unfetched): 788522
060207 153956 status 2 (DB_fetched): 60703
060207 153956 status 3 (DB_gone): 1501
060207 153956 CrawlDb statistics: done
But, I still have problem with hadoop.jar. I have to use the built classes
instead!
Thanks, Mike
On 2/7/06, Mike Smith <mi...@gmail.com> wrote:
>
> Rafit,
>
> get the hadoop project make that and use the build folder instead of the
> jar file. It will work fine then. Something probabely is missing from the
> hadoop jar.
>
> M
>
>
> On 2/7/06, Rafit Izhak_Ratzin <sa...@hotmail.com> wrote:
> >
> > I am still getting teh next Exception:
> >
> > Exception in thread "main" java.lang.NullPointerException
> > at
> > org.apache.hadoop.mapred.JobTrackerInfoServer.<init>(
> > JobTrackerInfoServer.java:56)
> > at org.apache.hadoop.mapred.JobTracker.<init>(JobTracker.java
> > :303)
> > at
> > org.apache.hadoop.mapred.JobTracker.startTracker (JobTracker.java:50)
> > at org.apache.hadoop.mapred.JobTracker.main(JobTracker.java:813)
> > 060207 161923 Server handler 9 on 50020: starting
> >
> >
> >
> >
> >
> > >From: Doug Cutting < cutting@apache.org>
> > >Reply-To: nutch-user@lucene.apache.org
> > >To: nutch-user@lucene.apache.org
> > >Subject: Re: new svn version:NoClassDefFoundError - JobTracker
> > >Date: Tue, 07 Feb 2006 13:04:37 -0800
> > >
> > >Mike Smith wrote:
> > >>The problem is that jetty jar files are missing from the SVN., I
> > replaced
> > >>the Jetty jar files but I get another exception:
> > >
> > >I just restored the jetty lib and the jetty-ext libs. Does that help?
> > >
> > >Doug
> >
> > _________________________________________________________________
> > FREE pop-up blocking with the new MSN Toolbar - get it now!
> > http://toolbar.msn.click-url.com/go/onm00200415ave/direct/01/
> >
> >
>
Re: new svn version:NoClassDefFoundError - JobTracker
Posted by Mike Smith <mi...@gmail.com>.
Rafit,
get the hadoop project make that and use the build folder instead of the jar
file. It will work fine then. Something probabely is missing from the hadoop
jar.
M
On 2/7/06, Rafit Izhak_Ratzin <sa...@hotmail.com> wrote:
>
> I am still getting teh next Exception:
>
> Exception in thread "main" java.lang.NullPointerException
> at
> org.apache.hadoop.mapred.JobTrackerInfoServer.<init>(
> JobTrackerInfoServer.java:56)
> at org.apache.hadoop.mapred.JobTracker.<init>(JobTracker.java:303)
> at
> org.apache.hadoop.mapred.JobTracker.startTracker(JobTracker.java:50)
> at org.apache.hadoop.mapred.JobTracker.main(JobTracker.java:813)
> 060207 161923 Server handler 9 on 50020: starting
>
>
>
>
>
> >From: Doug Cutting <cu...@apache.org>
> >Reply-To: nutch-user@lucene.apache.org
> >To: nutch-user@lucene.apache.org
> >Subject: Re: new svn version:NoClassDefFoundError - JobTracker
> >Date: Tue, 07 Feb 2006 13:04:37 -0800
> >
> >Mike Smith wrote:
> >>The problem is that jetty jar files are missing from the SVN., I
> replaced
> >>the Jetty jar files but I get another exception:
> >
> >I just restored the jetty lib and the jetty-ext libs. Does that help?
> >
> >Doug
>
> _________________________________________________________________
> FREE pop-up blocking with the new MSN Toolbar - get it now!
> http://toolbar.msn.click-url.com/go/onm00200415ave/direct/01/
>
>
Re: new svn version:NoClassDefFoundError - JobTracker
Posted by Rafit Izhak_Ratzin <sa...@hotmail.com>.
I am still getting teh next Exception:
Exception in thread "main" java.lang.NullPointerException
at
org.apache.hadoop.mapred.JobTrackerInfoServer.<init>(JobTrackerInfoServer.java:56)
at org.apache.hadoop.mapred.JobTracker.<init>(JobTracker.java:303)
at
org.apache.hadoop.mapred.JobTracker.startTracker(JobTracker.java:50)
at org.apache.hadoop.mapred.JobTracker.main(JobTracker.java:813)
060207 161923 Server handler 9 on 50020: starting
>From: Doug Cutting <cu...@apache.org>
>Reply-To: nutch-user@lucene.apache.org
>To: nutch-user@lucene.apache.org
>Subject: Re: new svn version:NoClassDefFoundError - JobTracker
>Date: Tue, 07 Feb 2006 13:04:37 -0800
>
>Mike Smith wrote:
>>The problem is that jetty jar files are missing from the SVN., I replaced
>>the Jetty jar files but I get another exception:
>
>I just restored the jetty lib and the jetty-ext libs. Does that help?
>
>Doug
_________________________________________________________________
FREE pop-up blocking with the new MSN Toolbar - get it now!
http://toolbar.msn.click-url.com/go/onm00200415ave/direct/01/
Re: new svn version:NoClassDefFoundError - JobTracker
Posted by Doug Cutting <cu...@apache.org>.
Mike Smith wrote:
> The problem is that jetty jar files are missing from the SVN., I replaced
> the Jetty jar files but I get another exception:
I just restored the jetty lib and the jetty-ext libs. Does that help?
Doug
Re: new svn version:NoClassDefFoundError - JobTracker
Posted by Mike Smith <mi...@gmail.com>.
The problem is that jetty jar files are missing from the SVN., I replaced
the Jetty jar files but I get another exception:
060207 123447 Property 'sun.cpu.isalist' is
Exception in thread "main" java.lang.NullPointerException
at org.apache.hadoop.mapred.JobTrackerInfoServer.<init>(
JobTrackerInfoServer.java:56)
at org.apache.hadoop.mapred.JobTracker.<init>(JobTracker.java:303)
at org.apache.hadoop.mapred.JobTracker.startTracker(JobTracker.java
:50)
at org.apache.hadoop.mapred.JobTracker.main(JobTracker.java:813)
Thanks, Mike
On 2/7/06, Rafit Izhak_Ratzin <sa...@hotmail.com> wrote:
>
> Hi,
>
> I am trying to run with the new svn version (375414), I am working under
> nutch/trunk directory.
>
> When I ran the next command "bin/hadoop jobtracker" or "bin/hadoop-
> daemon.sh
> start jobtracker"
> I got the next message,
>
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/mortbay/http/HttpListener
> at org.apache.hadoop.mapred.JobTracker.<init>(JobTracker.java:303)
> at
> org.apache.hadoop.mapred.JobTracker.startTracker(JobTracker.java:50)
> at org.apache.hadoop.mapred.JobTracker.main(JobTracker.java:813)
>
> Thanks,
> Rafit
>
> _________________________________________________________________
> FREE pop-up blocking with the new MSN Toolbar - get it now!
> http://toolbar.msn.click-url.com/go/onm00200415ave/direct/01/
>
>