Lucene index reset/cleared after an unexpected powerdown - advice please

[Mark Cranness <ma...@orcon.net.nz>]

Hi All,

Re: Lucene index reset/cleared after an unexpected powerdown - advice please

A client site had an unexpected powerdown (and UPS failure), after which the Lucene index was empty or corrupted and has
lost its documents.
Users continued to work and add new documents, and the index now only contains those new documents.

The index files (after the crash) only add up to about 78KB (for 30 new documents), so it looks like there is nothing
left of the old index.

Our logging shows the last writer.UpdateDocument 8 minutes before the crash.
Our code Commits the index writer after each document update.
So the index should have been synced to disk

There is evidence against the possibility that a document update was in progress at the time of the crash, but was not
logged because the log was not flushed.

A bit of a long-shot, but does anybody have any advice on how to debug the truncation/clearing of the Lucene index?


Details:
Lucene.Net 3.0.3.0

Index is a Lucene.Net.Store.FSDirectory
_directory = FSDirectory.Open(new DirectoryInfo(luceneDir), new SingleInstanceLockFactory());

There is only one single (static) instance of the IndexWriter
_writer = new IndexWriter(_directory, analyzer, IndexWriter.MaxFieldLength.UNLIMITED);

.NET 4.5

Thanks,
Mark

Re: Lucene index reset/cleared after an unexpected powerdown - advice please

[Simon Svensson <si...@devhost.se>]

Hi,

I believe that this would be due to
https://issues.apache.org/jira/browse/LUCENE-3418 assuming that
Lucene.Net has the exact same behavior as Lucene 3.0.3.

// Simon


On 06/10/15 08:01, Mark Cranness wrote:
> Hi All,
> 
> Re: Lucene index reset/cleared after an unexpected powerdown - advice please
> 
> A client site had an unexpected powerdown (and UPS failure), after which the Lucene index was empty or corrupted and has
> lost its documents.
> Users continued to work and add new documents, and the index now only contains those new documents.
> 
> The index files (after the crash) only add up to about 78KB (for 30 new documents), so it looks like there is nothing
> left of the old index.
> 
> Our logging shows the last writer.UpdateDocument 8 minutes before the crash.
> Our code Commits the index writer after each document update.
> So the index should have been synced to disk
> 
> There is evidence against the possibility that a document update was in progress at the time of the crash, but was not
> logged because the log was not flushed.
> 
> A bit of a long-shot, but does anybody have any advice on how to debug the truncation/clearing of the Lucene index?
> 
> 
> Details:
> Lucene.Net 3.0.3.0
> 
> Index is a Lucene.Net.Store.FSDirectory
> _directory = FSDirectory.Open(new DirectoryInfo(luceneDir), new SingleInstanceLockFactory());
> 
> There is only one single (static) instance of the IndexWriter
> _writer = new IndexWriter(_directory, analyzer, IndexWriter.MaxFieldLength.UNLIMITED);
> 
> .NET 4.5
> 
> Thanks,
> Mark
> 
>