Proper way to backup solr.

[elmerfudd <na...@012.net.il>]

What is the proper way to backup Solr while running, in the index level and
in the node level.
How do you restore it afterwards ?


thanks ahead.



--
View this message in context: http://lucene.472066.n3.nabble.com/Proper-way-to-backup-solr-tp4168498.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Proper way to backup solr.

[Shawn Heisey <ap...@elyograg.org>]

On 11/11/2014 1:45 AM, elmerfudd wrote:
> First, I want to thank you for your response!
> 
> can you provide more information about the suggested hardlink  solution?
> What are the advantages and disadvantages using it?
> 
> can you provide an example please?
> 
> 
> meanwhile try to read about it and test it myself asap.

Something like this:

mkdir -p ${BACKUPDIR}/corename/index
rm -f ${BACKUPDIR}/corename/index/*
cp -pl ${SOLRHOME}/corename/data/index/* ${BACKUPDIR}/corename/index/.

This does not include necessary additional steps like renaming previous
backups to set up an auto-rotating archive.  Because only you know what
your requirements are when it comes to backup archives, you'll need to
fill that part in.  As already mentioned, the source and destination
must be on the same filesystem.

There are very few disadvantages to this solution.  It maintains
instantaneous backups of previous index states with as little overhead
as possible.

Note that if you have a filesystem with good built-in snapshot support
(typically zfs or btrfs), you can use filesystem snapshots instead, with
much the same effect.

Thanks,
Shawn

Re: Proper way to backup solr.

[elmerfudd <na...@012.net.il>]

First, I want to thank you for your response!

can you provide more information about the suggested hardlink  solution?
What are the advantages and disadvantages using it?

can you provide an example please?


meanwhile try to read about it and test it myself asap.


thanks!






--
View this message in context: http://lucene.472066.n3.nabble.com/Proper-way-to-backup-solr-tp4168498p4168714.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Proper way to backup solr.

[Shawn Heisey <ap...@elyograg.org>]

On 11/10/2014 12:00 AM, elmerfudd wrote:
> What is the proper way to backup Solr while running, in the index level and
> in the node level.
> How do you restore it afterwards ?

One way to back up is to use the backup feature of the replication
handler.  Note that this actually copies the files, so this can be very
slow if the index is large.

https://cwiki.apache.org/confluence/display/solr/Backing+Up

Restoring involves stopping Solr, deleting the current index directory
(and tlog directory if it exists), copying the backed up index directory
(will be as slow as making the backup was in the first place) into the
original location, and starting Solr back up.  This approach will work
on ANY operating system.

If you're on UNIX or a clone like Linux, there is an alternate backup
method that works a lot faster, using hardlinks.  If you make a hardlink
copy of the index directory (the copy must be on the same filesystem),
it will complete instantly and take up zero additional disk space, at
least until you update the original index.

Restoring it would similar to what I described above, but instead of a
traditional copy, you'd do another hardlink copy after deleting the
current index.  That would also complete instantly.

Due to the way that Lucene creates index files, the hardlink copies are
guaranteed to never be corrupted by ongoing index operations in the
original directory.

Thanks,
Shawn