Lucene and Disk Snapshots

[Adam Talaat <ad...@illumen.org>]

Hello, 

I am trying to determine whether Lucene is compatible with a disk snapshot based backup strategy. I would like to take disk snapshots while Lucene is running. Can I be assured that committed data will remain consistent and recoverable for any snapshot? It is acceptable for me to lose documents which have not been committed, or a commit in-progress. Full ACID compliance isn't necessary, just that the Lucene index is consistent and recoverable regardless of when a snapshot is taken. 

I have seen bugfix reports related to consistent commits for earlier versions of Lucene. I have not been able to determine whether Lucene is generally considered safe in this respect. 

I plan to supplement the more frequent snapshot backups with logical/export backups. 

My specific usage scenario is Solr 1.4 hosted on a SAN. The SAN will be performing the snapshots. 

Thank you! 
Adam 

Re: Lucene and Disk Snapshots

[Shai Erera <se...@gmail.com>]

Yes, when you open an IndexWriter, make sure to use SnapshotDeletionPolicy.
When you want to snapshot a certain commit point, just call its snapshot()
method.

Note that I'm currently working on LUCENE-2481 which will allow SDP to
support multiple snapshots by ID. I'm using it for the same purposes as you
mention.

Shai

On Thu, May 27, 2010 at 7:58 PM, Adam Talaat <ad...@illumen.org> wrote:

> Hello,
>
> I am trying to determine whether Lucene is compatible with a disk snapshot
> based backup strategy. I would like to take disk snapshots while Lucene is
> running. Can I be assured that committed data will remain consistent and
> recoverable for any snapshot? It is acceptable for me to lose documents
> which have not been committed, or a commit in-progress. Full ACID compliance
> isn't necessary, just that the Lucene index is consistent and recoverable
> regardless of when a snapshot is taken.
>
> I have seen bugfix reports related to consistent commits for earlier
> versions of Lucene. I have not been able to determine whether Lucene is
> generally considered safe in this respect.
>
> I plan to supplement the more frequent snapshot backups with logical/export
> backups.
>
> My specific usage scenario is Solr 1.4 hosted on a SAN. The SAN will be
> performing the snapshots.
>
> Thank you!
> Adam
>
>
>