You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@nutch.apache.org by Eugen Kochuev <eu...@lan23.net> on 2006/05/08 17:10:21 UTC

Intranet crawl trouble

Hello,

While trying to perform intranet crawl with nutch-0.8 (latest nightly
build) I always reveive:

060508 144906 Running job: job_gh23qd
060508 144906 c:/nutch/crawl/crawldb/current/part-00000/data:0+127
060508 144906 c:/nutch/crawl/segments/20060508144859/crawl_fetch/part-00000/data
:0+143
060508 144906 c:/nutch/crawl/segments/20060508144859/crawl_parse/part-00000:0+71
34
060508 144906 job_gh23qd
java.io.IOException: Target c:/nutch/Database/hadoop/mapred/local/reduce_g1xu1d/
map_fyymoa.out already exists
        at org.apache.hadoop.fs.FileUtil.checkDest(FileUtil.java:162)
        at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:62)
        at org.apache.hadoop.fs.LocalFileSystem.renameRaw(LocalFileSystem.java:1
91)
        at org.apache.hadoop.fs.FileSystem.rename(FileSystem.java:306)
        at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:1
01)
Exception in thread "main" java.io.IOException: Job failed!
        at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:341)
        at org.apache.nutch.crawl.CrawlDb.update(CrawlDb.java:54)
        at org.apache.nutch.crawl.Crawl.main(Crawl.java:114)

Re: Intranet crawl trouble

Posted by Eugen Kochuev <eu...@lan23.net>.
Tried recompiling nutch with the latest hadoop version (nightly build)
Still the same result.

> Hello,

> While trying to perform intranet crawl with nutch-0.8 (latest nightly
> build) I always reveive:

> 060508 144906 Running job: job_gh23qd
> 060508 144906 c:/nutch/crawl/crawldb/current/part-00000/data:0+127
> 060508 144906
> c:/nutch/crawl/segments/20060508144859/crawl_fetch/part-00000/data
> :0+143
> 060508 144906
> c:/nutch/crawl/segments/20060508144859/crawl_parse/part-00000:0+71
> 34
> 060508 144906 job_gh23qd
> java.io.IOException: Target
> c:/nutch/Database/hadoop/mapred/local/reduce_g1xu1d/
> map_fyymoa.out already exists
>         at org.apache.hadoop.fs.FileUtil.checkDest(FileUtil.java:162)
>         at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:62)
>         at
> org.apache.hadoop.fs.LocalFileSystem.renameRaw(LocalFileSystem.java:1
> 91)
>         at org.apache.hadoop.fs.FileSystem.rename(FileSystem.java:306)
>         at
> org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:1
> 01)
> Exception in thread "main" java.io.IOException: Job failed!
>         at
> org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:341)
>         at org.apache.nutch.crawl.CrawlDb.update(CrawlDb.java:54)
>         at org.apache.nutch.crawl.Crawl.main(Crawl.java:114)