Avoiding corrupted index

[Laurent Vaills <la...@gmail.com>]

Hi everyone,

We are using Solr 1.4.1 in my company and we need to do some backups of the
indexes.

After some googling, I'm quite confused about the differents ways of backing
up the index.

First, I tried the scripts provided in the Solr distribution without success
:
I untarred the apache-solr-1.4.1.tar.gz into /opt; then I launched but I get
this error :
$ /opt/apache-solr-1.4.1/src/scripts/backup
/opt/apache-solr-1.4.1/src/scripts/backup: line 26:
/opt/apache-solr-1.4.1/src/bin/scripts-util: No such file or directory
And that's true : there is no /opt/apache-solr-1.4.1/src/bin/scripts-util
but a /opt/apache-solr-1.4.1/src/scripts/scripts-util
Is this normal to distribute the scripts with a bad path ?

Then I discovered that these utility scripts were not distributed anymore
with the version 3.1.0 : were they not reliable ? can we get corrupted
backups with this scripts ?

Finally, we found the page about SolrReplication on the Solr wiki also this
post
http://stackoverflow.com/questions/3083314/solr-incremental-backup-on-real-time-system-with-heavy-indexand
in particular the answer advising to use the replication.
So we tried to use this replication mecanism (and call the URL on the slave
with the query parameters command="backup" and location="/backup") but this
method requires lots of i/o for big index.

Is it the best way to get not corrupted backup of the index ?

Is there another way to do the backup with Solr 3.1 ?

Thanks in advance for your time.

Regards,
Laurent

Re: Avoiding corrupted index

[Chris Hostetter <ho...@fucit.org>]

: First, I tried the scripts provided in the Solr distribution without success
	...
: And that's true : there is no /opt/apache-solr-1.4.1/src/bin/scripts-util
: but a /opt/apache-solr-1.4.1/src/scripts/scripts-util
: Is this normal to distribute the scripts with a bad path ?

it looks like you are trying to run the scripts from the "src" directory 
of the distro ... they are ment to be installed in a "bin" directory 
in your solr home dir (so they can locate the default data dir, etc...)

If you haven't seen them already...

http://wiki.apache.org/solr/CollectionDistribution
http://wiki.apache.org/solr/SolrCollectionDistributionScripts

: Then I discovered that these utility scripts were not distributed anymore
: with the version 3.1.0 : were they not reliable ? can we get corrupted
: backups with this scripts ?

no, as far as i know they work great.

they were not included in the *binary* distributions of Solr, but they 
were most certianly included in the *source* distributions ... i think 
that was actually an oversight ... 3.1 is hte first time we had a binary 
distibution, and there's no reason i know of why they shouldn't have been 
in both.

(in general, these scripts have fallen out of favor because they aren't as 
portable or as easy to test as the java based replication, so they are 
easy to forget)


-Hoss