You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@nutch.apache.org by shrinivas patwardhan <sh...@gmail.com> on 2007/01/02 05:48:52 UTC

fetcher : some doubts

hello
 while fetching a certain fetchlist say about 2 - 3 million pages there are
some errors that might the fetcher process to stop
1: low disk space
2 : any severe error (like internet connection faliure for a long time )
3: some more like  java heap space
these are some of the resons that i faced ..
now straight answer would be that i should be taking care of all those
before i start fetching ..
i agree but in case the fetcher stops ...
Is there any way to continue fetching from where it stopped ?
if not can we all contribute towards that ?
i have been through the re fetching threads but wud that help me in this
case
example :
if i have fetched around a million pages and still some 2 million pages a
left to be fetched and the fetcher stops  due to low disk space is there any
way to continue from where i stopped after i organise everything (arrange
for another disk or free partition )

Thanks & Regards
Shrinivas Patwardhan