You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@lucene.apache.org by Bill Tschumy <bi...@otherwise.com> on 2005/04/08 19:26:49 UTC
Corrupted index
I had a customer report a corrupted Lucene index. He had copied the
index to backup storage, reformatted his drive, and then restored the
data. After that Lucene has trouble opening the index.
Here is the backtrace:
java.io.FileNotFoundException: /Users/bill/Desktop/IndexData/_v.fnm (No
such file or directory)
at java.io.RandomAccessFile.open(Native Method)
at java.io.RandomAccessFile.<init>(RandomAccessFile.java:204)
at
org.apache.lucene.store.FSInputStream$Descriptor.<init>(FSDirectory.java
:376)
at
org.apache.lucene.store.FSInputStream.<init>(FSDirectory.java:405)
at
org.apache.lucene.store.FSDirectory.openFile(FSDirectory.java:268)
at org.apache.lucene.index.FieldInfos.<init>(FieldInfos.java:53)
at
org.apache.lucene.index.SegmentReader.initialize(SegmentReader.java:
109)
at
org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:94)
at
org.apache.lucene.index.IndexReader$1.doBody(IndexReader.java:122)
at org.apache.lucene.store.Lock$With.run(Lock.java:109)
at
org.apache.lucene.index.IndexReader.open(IndexReader.java:111)
at
org.apache.lucene.index.IndexReader.open(IndexReader.java:106)
at
org.apache.lucene.search.IndexSearcher.<init>(IndexSearcher.java:43)
at
com.otherwise.parsnips.MySearcher.getSearcher(MySearcher.java:94)
at
com.otherwise.parsnips.IndexUpdater.checkIndexVersion(IndexUpdater.java:
35)
at com.otherwise.parsnips.Parsnips.initIndex(Parsnips.java:1101)
at
com.otherwise.parsnips.PreferencesDialog.checkIndexLoacation(Preferences
Dialog.java:212)
at
com.otherwise.parsnips.PreferencesDialog.actionPerformed(PreferencesDial
og.java:95)
at
javax.swing.AbstractButton.fireActionPerformed(AbstractButton.java:
1819)
at
javax.swing.AbstractButton$ForwardActionEvents.actionPerformed(AbstractB
utton.java:1872)
at
javax.swing.DefaultButtonModel.fireActionPerformed(DefaultButtonModel.ja
va:420)
at
javax.swing.DefaultButtonModel.setPressed(DefaultButtonModel.java:258)
at javax.swing.AbstractButton.doClick(AbstractButton.java:321)
at
javax.swing.plaf.basic.BasicRootPaneUI$DefaultAction.actionPerformed(Bas
icRootPaneUI.java
Here is what is in the index directory:
[pm1:~/Desktop/IndexData] ls -la
total 336
drwxr-xr-x 6 bill admin 204 8 Apr 08:04 ./
drwxr-xr-x 25 bill admin 850 8 Apr 10:10 ../
-rwxr-xr-x 1 bill admin 78695 7 Apr 10:02 _6r.cfs*
-rwxr-xr-x 1 bill admin 78371 7 Apr 10:03 _6t.cfs*
-rwxr-xr-x 1 bill admin 20 7 Apr 10:03 deletable*
-rwxr-xr-x 1 bill admin 83 7 Apr 10:03 segments*
The only thought I had was that he copied the data while the app was
still running and perhaps it was in an inconsistent state. Is this
possible? Any other way to track down what went wrong?
--
Bill Tschumy
Otherwise -- Austin, TX
http://www.otherwise.com
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Corrupted index
Posted by Doug Cutting <cu...@apache.org>.
Daniel Naber wrote:
> Yes, the *.cfs shows that this is a compound index which has *.fnm files
> only when it's being modified.
When creating a compound segment, a "segments" file is never written
that refers to the segment until the .cfs file is created and the .fnm
files are removed.
The real problem here looks to be that the segments file is out of sync,
and refers to a segment named 'v' when no such segment, compound or
otherwise, exists.
Doug
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Corrupted index
Posted by Daniel Naber <da...@t-online.de>.
On Friday 08 April 2005 19:26, Bill Tschumy wrote:
> The only thought I had was that he copied the data while the app was
> still running and perhaps it was in an inconsistent state.
Yes, the *.cfs shows that this is a compound index which has *.fnm files
only when it's being modified. You're not asking how to fix the index, but
that might be possible by modifying the segments file.
Regards
Daniel
--
http://www.danielnaber.de
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Corrupted index
Posted by Bill Tschumy <bi...@otherwise.com>.
Daniel,
Thanks for responding on this thread. I doubt the copy was made while
the index was being updated and I don't see any indication of a crash.
Just for my clarification, if I update the index, but don't close the
IndexWriter (because I may need it again soon), can the index on disk
be left in an inconsistent state? Do I need to close the IndexWriter
to assure the index is OK in disk?
It is imperative I figure out what is going on. I can't take the risk
of my customer's data becoming corrupted due to my error.
Bill
On Apr 8, 2005, at 4:59 PM, Daniel Naber wrote:
> On Friday 08 April 2005 23:51, Bill Tschumy wrote:
>
>> Would
>> this happen if there was a Writer that was not closed?
>
> Either the copy was done while the index was being updated, or the
> previous
> index update didn't finish (e.g. because it crashed before the index
> was
> closed).
>
> Regards
> Daniel
>
> --
> http://www.danielnaber.de
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
--
Bill Tschumy
Otherwise -- Austin, TX
http://www.otherwise.com
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Corrupted index
Posted by Daniel Naber <lu...@danielnaber.de>.
On Friday 08 April 2005 23:51, Bill Tschumy wrote:
> Would
> this happen if there was a Writer that was not closed?
Either the copy was done while the index was being updated, or the previous
index update didn't finish (e.g. because it crashed before the index was
closed).
Regards
Daniel
--
http://www.danielnaber.de
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Corrupted index
Posted by Doug Cutting <cu...@apache.org>.
Bill Tschumy wrote:
> So, did this happen because he copied the data while in an inconsistent
> state? I'm a bit surprised that an inconsistent index is ever left on
> disk (except for temporarily while something is being written). Would
> this happen if there was a Writer that was not closed?
An index should never be left in an inconsistent state. After a crash
lock files may need to be removed, but that's it.
However, backing up an index while it is being modified is tricky, since
it is a moving target. At any single point in time the index is
consistent, but the backup process is not instantaneous, and views the
index at a number of points in time.
Doug
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Corrupted index
Posted by Bill Tschumy <bi...@otherwise.com>.
So, did this happen because he copied the data while in an inconsistent
state? I'm a bit surprised that an inconsistent index is ever left on
disk (except for temporarily while something is being written). Would
this happen if there was a Writer that was not closed?
On Apr 8, 2005, at 1:22 PM, Daniel Naber wrote:
> On Friday 08 April 2005 19:26, Bill Tschumy wrote:
>
>> The only thought I had was that he copied the data while the app was
>> still running and perhaps it was in an inconsistent state.
>
> Yes, the *.cfs shows that this is a compound index which has *.fnm
> files
> only when it's being modified. You're not asking how to fix the index,
> but
> that might be possible by modifying the segments file.
>
> Regards
> Daniel
>
> --
> http://www.danielnaber.de
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
--
Bill Tschumy
Otherwise -- Austin, TX
http://www.otherwise.com
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Corrupted index
Posted by Daniel Naber <lu...@danielnaber.de>.
On Friday 08 April 2005 19:26, Bill Tschumy wrote:
> The only thought I had was that he copied the data while the app was
> still running and perhaps it was in an inconsistent state.
Yes, the *.cfs shows that this is a compound index which has *.fnm files
only when it's being modified. You're not asking how to fix the index, but
that might be possible by modifying the segments file.
Regards
Daniel
--
http://www.danielnaber.de
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org