You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Eva Lacy <ev...@lacy.ie> on 2012/11/20 18:31:49 UTC
Replication Backup
Hi All,
It takes a long time to reindex our database and so I'd like to be able to
backup the solr server.
I'm running solr 3.6.1 using tomcat on debian squeeze and I'd like to be
able to backup to a
windows server that contains the rest of our backups.
There isn't much free space on the solr server. Enough for maybe 3 backups.
Ideally I would like to be able to backup the index by pulling it similar
to how a solr slave does
but from the windows server. I heard that it uses something like rsync to
do that.
That way I could create a backup solution that backs up daily for 3 days,
then holds onto one of those every 10 days
or something similar.
Any ideas on this would be appreciated.
Eva
Re: Replication Backup
Posted by Eva Lacy <ev...@lacy.ie>.
There doesn't seem to be a lock file created by the snapshooter, it news up
a lock file but it never obtains the lock.
So there is no indication of when it is finished backing up the files.
On Sun, Nov 25, 2012 at 5:32 AM, Otis Gospodnetic <
otis.gospodnetic@gmail.com> wrote:
> Hi Eva,
>
> You just need a script that:
> * calls master with http://....replication?command=backup
> * copies the backup off of master and stores it somewhere
> * removes that backup from the master if you don't have enough disk for it
> there
>
> Otis
> --
> SOLR Performance Monitoring - http://sematext.com/spm/index.html
> Search Analytics - http://sematext.com/search-analytics/index.html
>
>
>
>
> On Wed, Nov 21, 2012 at 5:40 AM, Eva Lacy <ev...@lacy.ie> wrote:
>
> > Hi Otis,
> >
> > It seems to me that I'm going to have to write a script anyway that takes
> > handles the retention of the backups.
> > Plus it doesn't seem optimal that I would run a solr instance on that
> > server, taking up memory when I could probably
> > write a script that would pull all the data directly using the
> replication
> > handler.
> > My current thinking is that I could imitate a slave and pull the index
> > directly from the master by calling the replication
> > handler with http requests.
> > Does this seem reasonable?
> >
> > Eva
> >
> >
> > On Wed, Nov 21, 2012 at 3:29 AM, Otis Gospodnetic <
> > otis.gospodnetic@gmail.com> wrote:
> >
> > > Hi Eva,
> > >
> > > I think you just need to configure the Solr instance on your Windows
> and
> > > point it to your Solr master. It will then copy the index from the
> > master
> > > periodically.
> > > Please see http://search-lucene.com/?q=solr+replication+backup for
> some
> > > more info about doing backups - you don't need rsync. Once you set up
> > > Windows as described above, you can call the Solr replication handler
> on
> > it
> > > and tell it to make an index shapshot, which is basically a copy of the
> > > index at that point in time.
> > >
> > > Otis
> > > --
> > > Performance Monitoring - http://sematext.com/spm/index.html
> > > Search Analytics - http://sematext.com/search-analytics/index.html
> > >
> > >
> > >
> > >
> > > On Tue, Nov 20, 2012 at 12:31 PM, Eva Lacy <ev...@lacy.ie> wrote:
> > >
> > > > Hi All,
> > > >
> > > > It takes a long time to reindex our database and so I'd like to be
> able
> > > to
> > > > backup the solr server.
> > > > I'm running solr 3.6.1 using tomcat on debian squeeze and I'd like to
> > be
> > > > able to backup to a
> > > > windows server that contains the rest of our backups.
> > > >
> > > > There isn't much free space on the solr server. Enough for maybe 3
> > > backups.
> > > > Ideally I would like to be able to backup the index by pulling it
> > similar
> > > > to how a solr slave does
> > > > but from the windows server. I heard that it uses something like
> rsync
> > to
> > > > do that.
> > > > That way I could create a backup solution that backs up daily for 3
> > days,
> > > > then holds onto one of those every 10 days
> > > > or something similar.
> > > >
> > > > Any ideas on this would be appreciated.
> > > >
> > > > Eva
> > > >
> > >
> >
>
Re: Replication Backup
Posted by Otis Gospodnetic <ot...@gmail.com>.
Hi Eva,
You just need a script that:
* calls master with http://....replication?command=backup
* copies the backup off of master and stores it somewhere
* removes that backup from the master if you don't have enough disk for it
there
Otis
--
SOLR Performance Monitoring - http://sematext.com/spm/index.html
Search Analytics - http://sematext.com/search-analytics/index.html
On Wed, Nov 21, 2012 at 5:40 AM, Eva Lacy <ev...@lacy.ie> wrote:
> Hi Otis,
>
> It seems to me that I'm going to have to write a script anyway that takes
> handles the retention of the backups.
> Plus it doesn't seem optimal that I would run a solr instance on that
> server, taking up memory when I could probably
> write a script that would pull all the data directly using the replication
> handler.
> My current thinking is that I could imitate a slave and pull the index
> directly from the master by calling the replication
> handler with http requests.
> Does this seem reasonable?
>
> Eva
>
>
> On Wed, Nov 21, 2012 at 3:29 AM, Otis Gospodnetic <
> otis.gospodnetic@gmail.com> wrote:
>
> > Hi Eva,
> >
> > I think you just need to configure the Solr instance on your Windows and
> > point it to your Solr master. It will then copy the index from the
> master
> > periodically.
> > Please see http://search-lucene.com/?q=solr+replication+backup for some
> > more info about doing backups - you don't need rsync. Once you set up
> > Windows as described above, you can call the Solr replication handler on
> it
> > and tell it to make an index shapshot, which is basically a copy of the
> > index at that point in time.
> >
> > Otis
> > --
> > Performance Monitoring - http://sematext.com/spm/index.html
> > Search Analytics - http://sematext.com/search-analytics/index.html
> >
> >
> >
> >
> > On Tue, Nov 20, 2012 at 12:31 PM, Eva Lacy <ev...@lacy.ie> wrote:
> >
> > > Hi All,
> > >
> > > It takes a long time to reindex our database and so I'd like to be able
> > to
> > > backup the solr server.
> > > I'm running solr 3.6.1 using tomcat on debian squeeze and I'd like to
> be
> > > able to backup to a
> > > windows server that contains the rest of our backups.
> > >
> > > There isn't much free space on the solr server. Enough for maybe 3
> > backups.
> > > Ideally I would like to be able to backup the index by pulling it
> similar
> > > to how a solr slave does
> > > but from the windows server. I heard that it uses something like rsync
> to
> > > do that.
> > > That way I could create a backup solution that backs up daily for 3
> days,
> > > then holds onto one of those every 10 days
> > > or something similar.
> > >
> > > Any ideas on this would be appreciated.
> > >
> > > Eva
> > >
> >
>
Re: Replication Backup
Posted by Eva Lacy <ev...@lacy.ie>.
Hi Otis,
It seems to me that I'm going to have to write a script anyway that takes
handles the retention of the backups.
Plus it doesn't seem optimal that I would run a solr instance on that
server, taking up memory when I could probably
write a script that would pull all the data directly using the replication
handler.
My current thinking is that I could imitate a slave and pull the index
directly from the master by calling the replication
handler with http requests.
Does this seem reasonable?
Eva
On Wed, Nov 21, 2012 at 3:29 AM, Otis Gospodnetic <
otis.gospodnetic@gmail.com> wrote:
> Hi Eva,
>
> I think you just need to configure the Solr instance on your Windows and
> point it to your Solr master. It will then copy the index from the master
> periodically.
> Please see http://search-lucene.com/?q=solr+replication+backup for some
> more info about doing backups - you don't need rsync. Once you set up
> Windows as described above, you can call the Solr replication handler on it
> and tell it to make an index shapshot, which is basically a copy of the
> index at that point in time.
>
> Otis
> --
> Performance Monitoring - http://sematext.com/spm/index.html
> Search Analytics - http://sematext.com/search-analytics/index.html
>
>
>
>
> On Tue, Nov 20, 2012 at 12:31 PM, Eva Lacy <ev...@lacy.ie> wrote:
>
> > Hi All,
> >
> > It takes a long time to reindex our database and so I'd like to be able
> to
> > backup the solr server.
> > I'm running solr 3.6.1 using tomcat on debian squeeze and I'd like to be
> > able to backup to a
> > windows server that contains the rest of our backups.
> >
> > There isn't much free space on the solr server. Enough for maybe 3
> backups.
> > Ideally I would like to be able to backup the index by pulling it similar
> > to how a solr slave does
> > but from the windows server. I heard that it uses something like rsync to
> > do that.
> > That way I could create a backup solution that backs up daily for 3 days,
> > then holds onto one of those every 10 days
> > or something similar.
> >
> > Any ideas on this would be appreciated.
> >
> > Eva
> >
>
Re: Replication Backup
Posted by Otis Gospodnetic <ot...@gmail.com>.
Hi Eva,
I think you just need to configure the Solr instance on your Windows and
point it to your Solr master. It will then copy the index from the master
periodically.
Please see http://search-lucene.com/?q=solr+replication+backup for some
more info about doing backups - you don't need rsync. Once you set up
Windows as described above, you can call the Solr replication handler on it
and tell it to make an index shapshot, which is basically a copy of the
index at that point in time.
Otis
--
Performance Monitoring - http://sematext.com/spm/index.html
Search Analytics - http://sematext.com/search-analytics/index.html
On Tue, Nov 20, 2012 at 12:31 PM, Eva Lacy <ev...@lacy.ie> wrote:
> Hi All,
>
> It takes a long time to reindex our database and so I'd like to be able to
> backup the solr server.
> I'm running solr 3.6.1 using tomcat on debian squeeze and I'd like to be
> able to backup to a
> windows server that contains the rest of our backups.
>
> There isn't much free space on the solr server. Enough for maybe 3 backups.
> Ideally I would like to be able to backup the index by pulling it similar
> to how a solr slave does
> but from the windows server. I heard that it uses something like rsync to
> do that.
> That way I could create a backup solution that backs up daily for 3 days,
> then holds onto one of those every 10 days
> or something similar.
>
> Any ideas on this would be appreciated.
>
> Eva
>