You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@lucene.apache.org by Olivier Jaquemet <ol...@jalios.com> on 2006/03/22 16:08:28 UTC
FileNotFoundException: Corrupted Index?
Hi all,
We are using the last version of lucene (1.9.1), and sometimes we end up
with such error when opening one of the index our application uses:
java.io.FileNotFoundException: [...]/LuceneIndex/_ 46.fnm (No such file
or directory)
at java.io.RandomAccessFile.open(Native Method)
at java.io.RandomAccessFile.<init>(RandomAccessFile.java:204)
at
org.apache.lucene.store.FSIndexInput$Descriptor.<init>(FSDirectory.java:425)
at org.apache.lucene.store.FSIndexInput.<init>(FSDirectory.java:434)
at
org.apache.lucene.store.FSDirectory.openInput(FSDirectory.java:324)
at org.apache.lucene.index.FieldInfos.<init>(FieldInfos.java:56)
at
org.apache.lucene.index.SegmentReader.initialize(SegmentReader.java:144)
at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:129)
at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:110)
at
org.apache.lucene.index.IndexReader$1.doBody(IndexReader.java:154)
at org.apache.lucene.store.Lock$With.run(Lock.java:109)
at org.apache.lucene.index.IndexReader.open(IndexReader.java:143)
at org.apache.lucene.index.IndexReader.open(IndexReader.java:138)
The only solution available in this case being to completely remove and
recreate the index.
I have the "corrupted" index available for testing should you need it.
Apparently this corruption occurs if the JVM has crashed or was shutdown
too violently (kill -9)
I was wondering how a corruption of a lucene index could occur and how
to prevent it, fix it on reopening or in a last resort, detect it to be
able to recreate the index.
Note that I already have that kind of hook in the code for shutdown:
// clean writer reader and searcher correctly
Thread shutdown = new Thread() {
public void run() {
if (reader != null) {
try { reader.close(); }
catch (IOException ex){ /*empty*/ }
reader = null;
}
if (searcher != null) {
try { searcher.close(); }
catch (IOException ex){ /*empty*/ }
searcher = null;
}
}
};
Runtime.getRuntime().addShutdownHook(shutdown);
Or, on opening, code such as:
Directory indexDir = FSDirectory.getDirectory(luceneDir,
!IndexReader.indexExists(luceneDir));
IndexReader.unlock(indexDir); // unlock directory in case of
unproper shutdown
if (!IndexReader.indexExists(luceneDir)) {
writer = new IndexWriter(indexDir, analyzer, true);
writer.close();
}
Any suggestion or remark?
Thanks!
--
Olivier Jaquemet <ol...@jalios.com>
Re: FileNotFoundException: Corrupted Index? => Use jvm ShutdownHook
Posted by Olivier Jaquemet <ol...@jalios.com>.
Hi Otis,
Thanks for your reply.
I will also put the writer shutdown hook for this index, as you said.
I had already done that for other part of our code where we use other
lucene index, but thought it would not be needed for this special index
due to the fact that we rarely write on it. But this is a stupid thought
as the jvm can also be shutdown during those rare case... and this
corruption proves it..
I will watch if the problem still occurs and if it does not, I'll update
the wiki FAQ with the following code (left here for search history
purpose and for other users)
// clean writer reader and searcher correctly
Thread shutdown = new Thread() {
public void run() {
if (writer != null) {
try { writer.close(); }
catch (Exception ex){ /*empty*/ }
writer = null;
}
if (reader != null) {
try { reader.close(); }
catch (IOException ex){ /*empty*/ }
reader = null;
}
if (searcher != null) {
try { searcher.close(); }
catch (IOException ex){ /*empty*/ }
searcher = null;
}
}
};
Runtime.getRuntime().addShutdownHook(shutdown);
Otis Gospodnetic wrote:
> Hi Olivier,
>
> You have shutdown hooks for read-only operations. They won't corrupt your index. I'd add shutdown hooks for IndexWriter.
> If that fixes your problem, it would be great if you could add your shutdown hook code to the FAQ on the Wiki, or at least post it to java-user, so somebody else can put it there.
>
> Otis
>
> ----- Original Message ----
> From: Olivier Jaquemet <ol...@jalios.com>
> To: Lucene Java User ML <ja...@lucene.apache.org>
> Sent: Wednesday, March 22, 2006 10:08:28 AM
> Subject: FileNotFoundException: Corrupted Index?
>
> Hi all,
>
> We are using the last version of lucene (1.9.1), and sometimes we end up
> with such error when opening one of the index our application uses:
>
> java.io.FileNotFoundException: [...]/LuceneIndex/_ 46.fnm (No such file
> or directory)
> at java.io.RandomAccessFile.open(Native Method)
> at java.io.RandomAccessFile.<init>(RandomAccessFile.java:204)
> at
> org.apache.lucene.store.FSIndexInput$Descriptor.<init>(FSDirectory.java:425)
>
> at org.apache.lucene.store.FSIndexInput.<init>(FSDirectory.java:434)
> at
> org.apache.lucene.store.FSDirectory.openInput(FSDirectory.java:324)
> at org.apache.lucene.index.FieldInfos.<init>(FieldInfos.java:56)
> at
> org.apache.lucene.index.SegmentReader.initialize(SegmentReader.java:144)
> at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:129)
> at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:110)
> at
> org.apache.lucene.index.IndexReader$1.doBody(IndexReader.java:154)
> at org.apache.lucene.store.Lock$With.run(Lock.java:109)
> at org.apache.lucene.index.IndexReader.open(IndexReader.java:143)
> at org.apache.lucene.index.IndexReader.open(IndexReader.java:138)
>
> The only solution available in this case being to completely remove and
> recreate the index.
> I have the "corrupted" index available for testing should you need it.
>
> Apparently this corruption occurs if the JVM has crashed or was shutdown
> too violently (kill -9)
> I was wondering how a corruption of a lucene index could occur and how
> to prevent it, fix it on reopening or in a last resort, detect it to be
> able to recreate the index.
>
> Note that I already have that kind of hook in the code for shutdown:
>
> // clean writer reader and searcher correctly
> Thread shutdown = new Thread() {
> public void run() {
> if (reader != null) {
> try { reader.close(); }
> catch (IOException ex){ /*empty*/ }
> reader = null;
> }
> if (searcher != null) {
> try { searcher.close(); }
> catch (IOException ex){ /*empty*/ }
> searcher = null;
> }
> }
> };
> Runtime.getRuntime().addShutdownHook(shutdown);
>
> Or, on opening, code such as:
>
> Directory indexDir = FSDirectory.getDirectory(luceneDir,
> !IndexReader.indexExists(luceneDir));
> IndexReader.unlock(indexDir); // unlock directory in case of
> unproper shutdown
> if (!IndexReader.indexExists(luceneDir)) {
> writer = new IndexWriter(indexDir, analyzer, true);
> writer.close();
> }
>
> Any suggestion or remark?
>
> Thanks!
>
--
Olivier Jaquemet <ol...@jalios.com>
Ingénieur R&D Jalios S.A.
Tel: 01.39.23.92.83
http://www.jalios.com/
http://support.jalios.com/
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: FileNotFoundException: Corrupted Index?
Posted by Otis Gospodnetic <ot...@yahoo.com>.
Hi Olivier,
You have shutdown hooks for read-only operations. They won't corrupt your index. I'd add shutdown hooks for IndexWriter.
If that fixes your problem, it would be great if you could add your shutdown hook code to the FAQ on the Wiki, or at least post it to java-user, so somebody else can put it there.
Otis
----- Original Message ----
From: Olivier Jaquemet <ol...@jalios.com>
To: Lucene Java User ML <ja...@lucene.apache.org>
Sent: Wednesday, March 22, 2006 10:08:28 AM
Subject: FileNotFoundException: Corrupted Index?
Hi all,
We are using the last version of lucene (1.9.1), and sometimes we end up
with such error when opening one of the index our application uses:
java.io.FileNotFoundException: [...]/LuceneIndex/_ 46.fnm (No such file
or directory)
at java.io.RandomAccessFile.open(Native Method)
at java.io.RandomAccessFile.<init>(RandomAccessFile.java:204)
at
org.apache.lucene.store.FSIndexInput$Descriptor.<init>(FSDirectory.java:425)
at org.apache.lucene.store.FSIndexInput.<init>(FSDirectory.java:434)
at
org.apache.lucene.store.FSDirectory.openInput(FSDirectory.java:324)
at org.apache.lucene.index.FieldInfos.<init>(FieldInfos.java:56)
at
org.apache.lucene.index.SegmentReader.initialize(SegmentReader.java:144)
at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:129)
at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:110)
at
org.apache.lucene.index.IndexReader$1.doBody(IndexReader.java:154)
at org.apache.lucene.store.Lock$With.run(Lock.java:109)
at org.apache.lucene.index.IndexReader.open(IndexReader.java:143)
at org.apache.lucene.index.IndexReader.open(IndexReader.java:138)
The only solution available in this case being to completely remove and
recreate the index.
I have the "corrupted" index available for testing should you need it.
Apparently this corruption occurs if the JVM has crashed or was shutdown
too violently (kill -9)
I was wondering how a corruption of a lucene index could occur and how
to prevent it, fix it on reopening or in a last resort, detect it to be
able to recreate the index.
Note that I already have that kind of hook in the code for shutdown:
// clean writer reader and searcher correctly
Thread shutdown = new Thread() {
public void run() {
if (reader != null) {
try { reader.close(); }
catch (IOException ex){ /*empty*/ }
reader = null;
}
if (searcher != null) {
try { searcher.close(); }
catch (IOException ex){ /*empty*/ }
searcher = null;
}
}
};
Runtime.getRuntime().addShutdownHook(shutdown);
Or, on opening, code such as:
Directory indexDir = FSDirectory.getDirectory(luceneDir,
!IndexReader.indexExists(luceneDir));
IndexReader.unlock(indexDir); // unlock directory in case of
unproper shutdown
if (!IndexReader.indexExists(luceneDir)) {
writer = new IndexWriter(indexDir, analyzer, true);
writer.close();
}
Any suggestion or remark?
Thanks!
--
Olivier Jaquemet <ol...@jalios.com>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org