is snapshot/backup consistent (reliable) on the index server?

["Wu, Daniel" <Da...@sonyconnect.com>]

Hi,
 
I have some concern with solr using hard link to perform snapshot and
backup on the indexing box.
 
When referring to snapshot in contexts other than solr, it usually mean
some database state which can be revert to when necessary.  Since solr
use hard link to create snapshot on the master indexing box, the older
snapshot image could change if Lucene changed any file shared between
the index directory and the snapshot directory.  This would mean that
the snapshot or backup may not be the same state as when the snapshot or
backup were taken.
 
Can some one confirm or clarify?
 
Daniel Wu
 

Re: is snapshot/backup consistent (reliable) on the index server?

[Mike Klaas <mi...@gmail.com>]

No.  Hard links are alternative names for an inode: when lucene  
replaces a file, it is creating a new (underlying) inode/file, and  
the backup "link" points to the old one.

Don't think of hard links as "links", but additional logical names  
for the same physical data.

cheers,
-Mike

On 10-Aug-07, at 5:50 PM, Wu, Daniel wrote:

> So, does it mean the backup could have documents marked as deleted
> because they are deleted from the main index?
>
>
>
> -----Original Message-----
> From: yseeley@gmail.com [mailto:yseeley@gmail.com] On Behalf Of Yonik
> Seeley
> Sent: Friday, August 10, 2007 5:44 PM
> To: solr-user@lucene.apache.org
> Subject: Re: is snapshot/backup consistent (reliable) on the index
> server?
>
>
> On 8/10/07, Wu, Daniel <Da...@sonyconnect.com> wrote:
>> I have some concern with solr using hard link to perform snapshot and
>> backup on the indexing box.
>>
>> When referring to snapshot in contexts other than solr, it usually
>> mean some database state which can be revert to when necessary.   
>> Since
>
>> solr use hard link to create snapshot on the master indexing box, the
>> older snapshot image could change if Lucene changed any file shared
>> between the index directory and the snapshot directory.
>
> Right.  By design, lucene never changes index files it has written.  
> Some
> files, like .del files for deleted documents, can be completely  
> replaced
> however.
>
> -Yonik
>

RE: is snapshot/backup consistent (reliable) on the index server?

["Wu, Daniel" <Da...@sonyconnect.com>]

So, does it mean the backup could have documents marked as deleted
because they are deleted from the main index?

 

-----Original Message-----
From: yseeley@gmail.com [mailto:yseeley@gmail.com] On Behalf Of Yonik
Seeley
Sent: Friday, August 10, 2007 5:44 PM
To: solr-user@lucene.apache.org
Subject: Re: is snapshot/backup consistent (reliable) on the index
server?


On 8/10/07, Wu, Daniel <Da...@sonyconnect.com> wrote:
> I have some concern with solr using hard link to perform snapshot and 
> backup on the indexing box.
>
> When referring to snapshot in contexts other than solr, it usually 
> mean some database state which can be revert to when necessary.  Since

> solr use hard link to create snapshot on the master indexing box, the 
> older snapshot image could change if Lucene changed any file shared 
> between the index directory and the snapshot directory.

Right.  By design, lucene never changes index files it has written. Some
files, like .del files for deleted documents, can be completely replaced
however.

-Yonik

Re: is snapshot/backup consistent (reliable) on the index server?

[Yonik Seeley <yo...@apache.org>]

On 8/10/07, Wu, Daniel <Da...@sonyconnect.com> wrote:
> I have some concern with solr using hard link to perform snapshot and
> backup on the indexing box.
>
> When referring to snapshot in contexts other than solr, it usually mean
> some database state which can be revert to when necessary.  Since solr
> use hard link to create snapshot on the master indexing box, the older
> snapshot image could change if Lucene changed any file shared between
> the index directory and the snapshot directory.

Right.  By design, lucene never changes index files it has written.
Some files, like .del files for deleted documents, can be completely
replaced however.

-Yonik