Re: Failed to create text index reader for .frq file

[Nilesh Vijaywargiay <ni...@gmail.com>]

Hi Erick,

Last time, processing the case again on a different box didn't give the
missing file error. But now, after two months, we see this again. We are
using lucene 2.x and java 1.x I believe.

I am wondering as to why does lucene need this .frq file as we have never
seen this issue in house, but only at customers place. Is it optional and
if yes, how can I avoid lucene using it? We do not have access to the index
thus finding difficult to understand the core reason.

Do you know which code path in lucene triggers this demand for .frq file?

Thanks
Nilesh

On Tue, Sep 6, 2011 at 3:04 PM, Erick Erickson <er...@gmail.com>wrote:

> Hmmm, it rather looks like somehow some of your
> index files got deleted from your index directory.
> a .frq file is one of the files that contains document
> frequency information. The fact that Lucene
> can't find _166.frq indicates you've been using this
> index for a while and created a bunch of segments.
>
> There exists a program called checkindex that
> can be used to help figure out the state of your
> index. See:
> http://solr.pl/en/2011/01/17/checkindex-for-the-rescue/
>
> And if you're using Java7, you should be aware
> of:
>
> http://lucene.apache.org/solr/#28+July+2011+-+WARNING%3A+Index+corruption+and+crashes+in+Apache+Lucene+Core+%2F+Apache+Solr+with+Java+7
>
> What happens if you point Luke at your index?
>
> Best
> Erick
>
> On Tue, Sep 6, 2011 at 3:33 PM, Nilesh Vijaywargiay
> <ni...@gmail.com> wrote:
> > Hi,
> >
> > I am facing a error while processing a set of data.. the description says
> > "Failed to create text index reader for .../.../_166.frq (The System
> cannot
> > find the specified path).
> >
> > I have never seen this error in the past neither have I seen this file in
> > the index folder ever.  Does lucene take care of this file or we need
> some
> > add-on or third party software for this particular file?
> >
> > Thanks
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>

Re: Failed to create text index reader for .frq file

[Erick Erickson <er...@gmail.com>]

Here's a useful page:
http://lucene.apache.org/java/3_0_2/fileformats.html#file-names

>From that doc: "Contains the list of docs which contain each term
along with frequency"

which means that virtually any code path that calculates TF/IDF, i.e.
anything that
calculates relevancy will want this file. Or anything that tries to
analyze phrases.

It's really hard to understand why this file would disappear. My
suspicion is that
your client is somehow deleting it. But it's really hard to figure this kind
of thing out when you don't have access to the machine.

I can say that this hasn't been seen by other people that I know of,
and if it was it'd
be given lots of attention.

Best
Erick

On Thu, Nov 17, 2011 at 4:22 PM, Nilesh Vijaywargiay
<ni...@gmail.com> wrote:
> Hi Erick,
>
> Last time, processing the case again on a different box didn't give the
> missing file error. But now, after two months, we see this again. We are
> using lucene 2.x and java 1.x I believe.
>
> I am wondering as to why does lucene need this .frq file as we have never
> seen this issue in house, but only at customers place. Is it optional and
> if yes, how can I avoid lucene using it? We do not have access to the index
> thus finding difficult to understand the core reason.
>
> Do you know which code path in lucene triggers this demand for .frq file?
>
> Thanks
> Nilesh
>
> On Tue, Sep 6, 2011 at 3:04 PM, Erick Erickson <er...@gmail.com>wrote:
>
>> Hmmm, it rather looks like somehow some of your
>> index files got deleted from your index directory.
>> a .frq file is one of the files that contains document
>> frequency information. The fact that Lucene
>> can't find _166.frq indicates you've been using this
>> index for a while and created a bunch of segments.
>>
>> There exists a program called checkindex that
>> can be used to help figure out the state of your
>> index. See:
>> http://solr.pl/en/2011/01/17/checkindex-for-the-rescue/
>>
>> And if you're using Java7, you should be aware
>> of:
>>
>> http://lucene.apache.org/solr/#28+July+2011+-+WARNING%3A+Index+corruption+and+crashes+in+Apache+Lucene+Core+%2F+Apache+Solr+with+Java+7
>>
>> What happens if you point Luke at your index?
>>
>> Best
>> Erick
>>
>> On Tue, Sep 6, 2011 at 3:33 PM, Nilesh Vijaywargiay
>> <ni...@gmail.com> wrote:
>> > Hi,
>> >
>> > I am facing a error while processing a set of data.. the description says
>> > "Failed to create text index reader for .../.../_166.frq (The System
>> cannot
>> > find the specified path).
>> >
>> > I have never seen this error in the past neither have I seen this file in
>> > the index folder ever.  Does lucene take care of this file or we need
>> some
>> > add-on or third party software for this particular file?
>> >
>> > Thanks
>> >
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org