You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@nutch.apache.org by "Armel T. Nene" <ar...@idna-solutions.com> on 2007/01/23 20:26:35 UTC
is crawldb format in Nutch 0.8 compatible with Nutch0.7
Hi guys,
I am running in some nightmares when trying to iterate over values in the
Nutch 0.8.2 crawldb. I am getting some hadoop exception such as the
following:
07/01/23 18:33:56 INFO conf.Configuration: parsing
jar:file:/C:/nutch-0.8.2-dev/lib/hadoop-0.4.0-patched.jar!/hadoop-default.xm
l
07/01/23 18:33:56 INFO conf.Configuration: parsing
jar:file:/C:/nutch-0.8.2-dev/nutch-0.8.2-dev.jar!/nutch-default.xml
07/01/23 18:33:56 INFO conf.Configuration: parsing
jar:file:/C:/nutch-0.8.2-dev/nutch-0.8.2-dev.jar!/nutch-site.xml
Exception in thread "main" java.lang.ArithmeticException: / by zero
at
org.apache.hadoop.mapred.lib.HashPartitioner.getPartition(HashPartitioner.ja
va:33)
at
org.apache.hadoop.mapred.MapFileOutputFormat.getEntry(MapFileOutputFormat.ja
va:88)
at org.apache.nutch.crawl.CrawlDbReader.get(CrawlDbReader.java:321)
therefore, if I can iterate over the values contained in the crawldb using
Nutch 0.7 API, I should think this will fix the issue. So the question is;
is Nutch 0.8 backward compatible with Nutch 0.7.2
Thanks,
Armel
-------------------------------------------------
Armel T. Nene
iDNA Solutions
Tel: +44 (207) 257 6124
Mobile: +44 (788) 695 0483
<http://blog.idna-solutions.com/> http://blog.idna-solutions.com
Re: is crawldb format in Nutch 0.8 compatible with Nutch0.7
Posted by Andrzej Bialecki <ab...@getopt.org>.
Armel T. Nene wrote:
> therefore, if I can iterate over the values contained in the crawldb using
> Nutch 0.7 API, I should think this will fix the issue. So the question is;
>
>
>
> is Nutch 0.8 backward compatible with Nutch 0.7.2
>
This subject was discussed several times in the past. The answer is
still no - Nutch 0.8 is NOT compatible with 0.7. It is possible to write
converters for some of the data, but it's a lot of tedious work. The
recommended upgrade path is to dump your 0.7 WebDB to a text file, and
then inject this text file to a 0.8 CrawlDb.
--
Best regards,
Andrzej Bialecki <><
___. ___ ___ ___ _ _ __________________________________
[__ || __|__/|__||\/| Information Retrieval, Semantic Web
___|||__|| \| || | Embedded Unix, System Integration
http://www.sigram.com Contact: info at sigram dot com