You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@nutch.apache.org by gekkokid <me...@gekkokid.org.uk> on 2005/09/20 02:26:37 UTC

regarding gal's faq proposal

is there a place where we can search the mailing list? that could be a short 
term solution

_gk
----- Original Message ----- 
From: "Gal Nitzan" <gn...@usa.net>
To: <nu...@lucene.apache.org>
Sent: Monday, September 19, 2005 11:37 PM
Subject: Re: Proposal: refuse to open partially trunc. MapFile, unless 
forced (Re: indexing is very very very slow)


> Andrzej Bialecki wrote:
>> Hi all,
>>
>>> Well I still get a very slow mergesegs:
>>
>>>>
>>>> >050917 043332  - data in segment index/segments/20050916014401 is
>>>> corrupt, using only 128115 entries.
>>
>> This is a common and recurring problem. What's worse is that an unfixed 
>> segment like this will destroy the performance of the search, too, not 
>> just the backend pre-processing.
>>
>> I propose to modify MapFile.Reader so that it refuses to open such file, 
>> and throws an Exception, unless a force=true flag is given. Tools that 
>> want to ignore this can do so, but all other tools will be able to make a 
>> conscious decision whether to fix it first, or to use it as such.
>>
>> If there are no objections, I will change it in the trunk/ in a couple of 
>> days.
>>
> Hi,
>
> I think it would be very confusing to old users as well as new users. 
> Throwing an exception when actually  a segment corruption is trivial and 
> can be fixed easily (now that I know how to do that :-)...
>
> Instead I would like to suggest building a FAQ for Nutch.
>
> I would like to propose myself  to build at least the skeleton for it.
>
> As a new user to Nutch I have run to so many problems and except this list 
> there was not much information elsewhere. So, I have all the answers fresh 
> in my mind and with some help from the rest of the nutch-users it can be 
> done without too much of a hustle.
>
> Besides, many people on this list contribute on their free time, I would 
> be happy to contribute to the success of this  project.
>
> Regards,
>
> Gal
>
>
>
>
>