You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@lucene.apache.org by Michael McCandless <lu...@mikemccandless.com> on 2006/09/01 13:37:54 UTC
Re: read past EOF
> Yes I am sure only one writer at a time accessing index.
>
> no i am not getting any other exception.
>
> and there is no problem of disk space also.
>
> right now i have backcopy of indexes so whenever one index got corrupted
> i m replacing with backup one and starting the indexer again from that
> duration.
>
> Here is the script which i am using to move index after its built.
>
> - rm -rf backupindex/*
> - mv index backupindex;
> - mv newindex index;
> - mkdir newindex
> - cp -dpR index/* newindex/
> - touch index.done
> - echo "done";
>
> where "newindex" is the index which I am using for indexing...."index"
> which i am using for search purpose....and "backupindex" contains
> previous index.
It sounds like you're working with the index correctly, so I don't have
any other ideas on why you're getting CFS files that are truncated. I
would wory about the "cp" step filling up disk, but if you're nowhere
near filling up disk that's not the root cause here.
Does this happen intermittantly? Or it happened once and now it's gone?
Or is it easy to reproduce?
> Is there any way through which I can check if index is corrupt or
> not....right now because of this exception (read past EOF ) i made few
> changes in my code to check for corrupt index. But i am checking for
> corrupt index through optimizing...If in optimization of index i m
> getting IOException I am considering that index got corrupted or there
> is permission issue..
That's a great question. I don't know of existing tools for doing this
(anyone else?). Running optimize is likely a good test, so long as
there's more than 1 segment before optimize (so that it actually does
something).
Mike
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: read past EOF
Posted by Michael McCandless <lu...@mikemccandless.com>.
Bhavin Pandya wrote:
>> It sounds like you're working with the index correctly, so I don't
>> have any other ideas on why you're getting CFS files that are
>> truncated. I would wory about the "cp" step filling up disk, but if
>> you're nowhere near filling up disk that's not the root cause here.
>>
>
> I have found the cause of this problem... You were right .....
> Its because at perticular point of time my hard disk got full so It
> currupted index at that time but after that because of some batch
> process disk becomes empty enough so I was not able to find continuous
> exception like "no space left"...but when i gone through all the log i
> tracked it sucessfully.
Phew! Glad to hear you got down to the root cause and that in fact that
root cause was "outside" of Lucene :)
Mike
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: read past EOF
Posted by Bhavin Pandya <bh...@rediff.co.in>.
Hi Mike,
> It sounds like you're working with the index correctly, so I don't have
> any other ideas on why you're getting CFS files that are truncated. I
> would wory about the "cp" step filling up disk, but if you're nowhere near
> filling up disk that's not the root cause here.
>
I have found the cause of this problem... You were right .....
Its because at perticular point of time my hard disk got full so It
currupted index at that time but after that because of some batch process
disk becomes empty enough so I was not able to find continuous exception
like "no space left"...but when i gone through all the log i tracked it
sucessfully.
Thanks for your help....
- Bhavin pandya
----- Original Message -----
From: "Michael McCandless" <lu...@mikemccandless.com>
To: <ja...@lucene.apache.org>
Sent: Friday, September 01, 2006 5:07 PM
Subject: Re: read past EOF
>
>> Yes I am sure only one writer at a time accessing index.
>>
>> no i am not getting any other exception.
>>
>> and there is no problem of disk space also.
>>
>> right now i have backcopy of indexes so whenever one index got corrupted
>> i m replacing with backup one and starting the indexer again from that
>> duration.
>>
>> Here is the script which i am using to move index after its built.
>>
>> - rm -rf backupindex/*
>> - mv index backupindex;
>> - mv newindex index;
>> - mkdir newindex
>> - cp -dpR index/* newindex/
>> - touch index.done
>> - echo "done";
>>
>> where "newindex" is the index which I am using for indexing...."index"
>> which i am using for search purpose....and "backupindex" contains
>> previous index.
>
> It sounds like you're working with the index correctly, so I don't have
> any other ideas on why you're getting CFS files that are truncated. I
> would wory about the "cp" step filling up disk, but if you're nowhere near
> filling up disk that's not the root cause here.
>
> Does this happen intermittantly? Or it happened once and now it's gone?
> Or is it easy to reproduce?
>
>> Is there any way through which I can check if index is corrupt or
>> not....right now because of this exception (read past EOF ) i made few
>> changes in my code to check for corrupt index. But i am checking for
>> corrupt index through optimizing...If in optimization of index i m
>> getting IOException I am considering that index got corrupted or there is
>> permission issue..
>
> That's a great question. I don't know of existing tools for doing this
> (anyone else?). Running optimize is likely a good test, so long as
> there's more than 1 segment before optimize (so that it actually does
> something).
>
> Mike
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org