0.8: NullPointerException Optimizing index when crawling

[ArentJan Banck <aj...@planet.nl>]

I have been running 0.7.1  without problems.
Today i build a 0.8-dev version from trunk. and ported my settings from 
0.7.1

When doing a crawl crawling starts, but it ends with a nullpointer 
exception.
Any hints what is going wrong /  how I should resolve this?

commandline used:
sh bin/nutch crawl urls -dir /nutch-0.8-dev/crawl -depth 1



Tail of the stacktrace:
060314 231802   Nutch Query Filter (org.apache.nutch.searcher.QueryFilter)
060314 231802 parsing 
jar:file:/C:/nutch-0.8-dev/lib/hadoop-0.1-dev.jar!/hadoop-default.xml
060314 231803 parsing 
jar:file:/C:/nutch-0.8-dev/lib/hadoop-0.1-dev.jar!/mapred-default.xml
060314 231803 parsing \tmp\hadoop\mapred\local\job_u1rb8.xml\localRunner
060314 231803 parsing 
jar:file:/C:/nutch-0.8-dev/lib/hadoop-0.1-dev.jar!/mapred-default.xml
060314 231803 parsing file:/C:/nutch-0.8-dev/conf/hadoop-site.xml
060314 231803 found resource common-terms.utf8 at 
file:/C:/nutch-0.8-dev/conf/common-terms.utf8
060314 231803 found resource common-terms.utf8 at 
file:/C:/nutch-0.8-dev/conf/common-terms.utf8
060314 231803 Optimizing index.
java.lang.NullPointerException
        at 
org.apache.nutch.indexer.Indexer$OutputFormat$1.write(Indexer.java:109)
        at 
org.apache.hadoop.mapred.ReduceTask$3.collect(ReduceTask.java:270)
        at org.apache.nutch.indexer.Indexer.reduce(Indexer.java:242)
        at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:283)
        at 
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:106)
060314 231803  map 100%  reduce 0%
Exception in thread "main" java.io.IOException: Job failed!
        at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:310)
        at org.apache.nutch.indexer.Indexer.index(Indexer.java:275)
        at org.apache.nutch.crawl.Crawl.main(Crawl.java:120)


Thanks,
Arent-Jan 

Re: 0.8: NullPointerException Optimizing index when crawling

[Marko Bauhardt <mb...@media-style.com>]

Am 14.03.2006 um 23:20 schrieb ArentJan Banck:

> java.lang.NullPointerException
>        at org.apache.nutch.indexer.Indexer$OutputFormat$1.write 
> (Indexer.java:109)

What for index plugins do you have configured in your nutch- 
default.xml or nutch-site.xml? Be sure that the index-basic plugin is  
included.

Marko