Backing up large indexes

[Alexandre Leopoldo Gonçalves <a....@stela.org.br>]

Hi All,

We have a system with a lucene index with 100GB and growing fast. I 
wonder whether
there is an efficient way to backup it taking into account only the 
changes among old
and new version of the index, once after optimization process the name 
of the main
index file change.

Regards,

Alex


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org

Re: Backing up large indexes

[Mindaugas Žakšauskas <mi...@gmail.com>]

This might be irrelevant, but have you considered using ZFS? This file
system is designed to do what you need. Assuming you can trigger
events at the time after you have updated the index, you would have to
trigger new ZFS snapshot and place it elsewhere.

This might have some side effects though (hint: fs I/O speed), either
positive or negative depending on your current setup.

Regards,
Mindaugas

On Wed, Jul 22, 2009 at 2:33 PM, Alexandre Leopoldo
Gonçalves<a....@stela.org.br> wrote:
> <..>whether there is an efficient way to backup it taking into account only the changes <..>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org

Re: [?? Probable Spam] Re: Backing up large indexes

[Alexandre Leopoldo Gonçalves <a....@stela.org.br>]

Shai,

Thanks for the tip. I´ll start with it.

Alex

Shai Erera wrote:
> Hi Alex,
>
> You can start with this article:
> http://www.manning.com/free/green_HotBackupsLucene.html (you'll need to
> register w/ your email). It describes how one can write Hot Backups w/
> Lucene, and capture just the "delta" since the last backup.
>
> I'm about to try it myself, so if you get to do it before I am, I'd
> appreciate if you can post back some initial observations/impressions.
>
> Shai
>
> On Wed, Jul 22, 2009 at 4:33 PM, Alexandre Leopoldo Gonçalves <
> a.l.goncalves@stela.org.br> wrote:
>
>   
>> Hi All,
>>
>> We have a system with a lucene index with 100GB and growing fast. I wonder
>> whether
>> there is an efficient way to backup it taking into account only the changes
>> among old
>> and new version of the index, once after optimization process the name of
>> the main
>> index file change.
>>
>> Regards,
>>
>> Alex
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>>
>>     
>
> .
>
>   


-- 

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org

Re: Backing up large indexes

[Shai Erera <se...@gmail.com>]

Hi Alex,

You can start with this article:
http://www.manning.com/free/green_HotBackupsLucene.html (you'll need to
register w/ your email). It describes how one can write Hot Backups w/
Lucene, and capture just the "delta" since the last backup.

I'm about to try it myself, so if you get to do it before I am, I'd
appreciate if you can post back some initial observations/impressions.

Shai

On Wed, Jul 22, 2009 at 4:33 PM, Alexandre Leopoldo Gonçalves <
a.l.goncalves@stela.org.br> wrote:

> Hi All,
>
> We have a system with a lucene index with 100GB and growing fast. I wonder
> whether
> there is an efficient way to backup it taking into account only the changes
> among old
> and new version of the index, once after optimization process the name of
> the main
> index file change.
>
> Regards,
>
> Alex
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>