You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@nutch.apache.org by karthik085 <ka...@gmail.com> on 2007/08/14 06:17:03 UTC
Re: Error on convert to 0.9 during mergesegs step
I put the property in hadoop-site.xml and still get the same error. Any other
thoughts would be helpful. I am converting from 0.7.2 to 0.9. Is it because
of that?
Andrzej Bialecki wrote:
>
> RP wrote:
>> Pardon my ignorance, but where and how do I do this..?? Nothing in
>> conf files that I can see as a switch....
>
>
> Ah-ha! you can't see it because it's a _hidden_ switch ... *evil grin*
>
> Joking aside ... Yes, sorry for the confusion - it's a setting in the
> default Hadoop config, and Nutch config contains only overrides ... so
> you need to put this explicitly into your hadoop-site.xml, like this:
>
> <property>
> <name>mapred.speculative.execution</name>
> <value>false</value>
> </property>
>
> If this fixes your problem, I'll put this property in the public sources.
>
> --
> Best regards,
> Andrzej Bialecki <><
> ___. ___ ___ ___ _ _ __________________________________
> [__ || __|__/|__||\/| Information Retrieval, Semantic Web
> ___|||__|| \| || | Embedded Unix, System Integration
> http://www.sigram.com Contact: info at sigram dot com
>
>
>
>
--
View this message in context: http://www.nabble.com/Error-on-convert-to-0.9-during-mergesegs-step-tf2828063.html#a12138519
Sent from the Nutch - User mailing list archive at Nabble.com.
Re: Error on convert to 0.9 during mergesegs step
Posted by karthik085 <ka...@gmail.com>.
Hmm..... Thanks.
Andrzej Bialecki wrote:
>
> karthik085 wrote:
>> I put the property in hadoop-site.xml and still get the same error. Any
>> other
>> thoughts would be helpful. I am converting from 0.7.2 to 0.9. Is it
>> because
>> of that?
>
> Convert tools support only migration from 0.8 to 0.9. Conversion from
> 0.7.x to 0.8 (or 0.9) is NOT supported.
>
> The recommended upgrade path is to dump your crawldb to a text file,
> extract urls, and inject them back to a 0.9-based system. Unfortunately,
> all other data is not (easily) upgradeable, so you will have to re-fetch
> everything.
>
>
> --
> Best regards,
> Andrzej Bialecki <><
> ___. ___ ___ ___ _ _ __________________________________
> [__ || __|__/|__||\/| Information Retrieval, Semantic Web
> ___|||__|| \| || | Embedded Unix, System Integration
> http://www.sigram.com Contact: info at sigram dot com
>
>
>
--
View this message in context: http://www.nabble.com/Error-on-convert-to-0.9-during-mergesegs-step-tf2828063.html#a12141947
Sent from the Nutch - User mailing list archive at Nabble.com.
Re: Error on convert to 0.9 during mergesegs step
Posted by Andrzej Bialecki <ab...@getopt.org>.
karthik085 wrote:
> I put the property in hadoop-site.xml and still get the same error. Any other
> thoughts would be helpful. I am converting from 0.7.2 to 0.9. Is it because
> of that?
Convert tools support only migration from 0.8 to 0.9. Conversion from
0.7.x to 0.8 (or 0.9) is NOT supported.
The recommended upgrade path is to dump your crawldb to a text file,
extract urls, and inject them back to a 0.9-based system. Unfortunately,
all other data is not (easily) upgradeable, so you will have to re-fetch
everything.
--
Best regards,
Andrzej Bialecki <><
___. ___ ___ ___ _ _ __________________________________
[__ || __|__/|__||\/| Information Retrieval, Semantic Web
___|||__|| \| || | Embedded Unix, System Integration
http://www.sigram.com Contact: info at sigram dot com