You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Andy D'Arcy Jewell <an...@sysmicro.co.uk> on 2012/12/20 11:16:01 UTC
Pause and resume indexing on SolR 4 for backups
Hi all.
Can anyone advise me of a way to pause and resume SolR 4 so I can
perform a backup? I need to be able to revert to a usable (though not
necessarily complete) index after a crash or other "disaster" more
quickly than a re-index operation would yield.
I can't yet afford the "extravagance" of a separate SolR replica just
for backups, and I'm not sure if I'll ever have the luxury. I'm
currently running with just one node, be we are not yet live.
I can think of the following ways to do this, each with various downsides:
1) Just backup the existing index files whilst indexing continues
+ Easy
+ Fast
- Incomplete
- Potential for corruption? (e.g. partial files)
2) Stop/Start Tomcat
+ Easy
- Very slow and I/O, CPU intensive
- Client gets errors when trying to connect
3) Block/unblock SolR port with IpTables
+ Fast
- Client gets errors when trying to connect
- Have to wait for existing transactions to complete (not sure how,
maybe watch socket FD's in /proc)
4) Pause/Restart SolR service
+ Fast ? (hopefully)
- Client gets errors when trying to connect
In any event, the web app will have to gracefully handle unavailability
of SolR, probably by displaying a "down for maintenance" message, but
this should preferably be only a very short amount of time.
Can anyone comment on my proposed solutions above, or provide any
additional ones?
Thanks for any input you can provide!
-Andy
--
Andy D'Arcy Jewell
SysMicro Limited
Linux Support
E: andy.jewell@sysmicro.co.uk
W: www.sysmicro.co.uk
Re: Pause and resume indexing on SolR 4 for backups
Posted by Upayavira <uv...@odoko.co.uk>.
That's neat, but wouldn't that run on every commit? How would you use it
to, say, back up once a day?
Upayavira
On Thu, Dec 20, 2012, at 01:57 PM, Markus Jelsma wrote:
> You can use the postCommit event in updateHandler to execute a task.
>
> -----Original message-----
> > From:Upayavira <uv...@odoko.co.uk>
> > Sent: Thu 20-Dec-2012 14:45
> > To: solr-user@lucene.apache.org
> > Subject: Re: Pause and resume indexing on SolR 4 for backups
> >
> > The backup directory should just be a clone of the index files. I'm
> > curious to know whether it is a cp -r or a cp -lr that the replication
> > handler produces.
> >
> > You would prevent commits by telling your app not to commit. That is,
> > Solr only commits when it is *told* to.
> >
> > Unless you use autocommit, in which case I guess you could monitor your
> > logs for the last commit, and do your backup a 10 seconds after that.
> >
> > Upayavira
> >
> > On Thu, Dec 20, 2012, at 12:44 PM, Andy D'Arcy Jewell wrote:
> > > On 20/12/12 11:58, Upayavira wrote:
> > > > I've never used it, but the replication handler has an option:
> > > >
> > > > http://master_host:port/solr/replication?command=backup
> > > >
> > > > Which will take you a backup.
> > > I've looked at that this morning as suggested by Markus Jelsma. Looks
> > > good, but I'll have to work out how to use the resultant backup
> > > directory. I've been dealing with another unrelated issue in the
> > > mean-time and I haven't had a chance to look for any docu so far.
> > > > Also something to note, if you don't want to use the above, and you are
> > > > running on Unix, you can create fast 'hard link' clones of lucene
> > > > indexes. Doing:
> > > >
> > > > cp -lr data data.bak
> > > >
> > > > will copy your index instantly. If you can avoid doing this when a
> > > > commit is happening, then you'll have a good index copy, that will take
> > > > no space on your disk and be made instantly. This is because it just
> > > > copies the directory structure, not the files themselves, and given
> > > > files in a lucene index never change (they are only ever deleted or
> > > > replaced), this works as a good copy technique for backing up.
> > > That's the approach that Shawn Heisey proposed, and what I've been
> > > working towards, but it still leaves open the question of how to
> > > *pause* SolR or prevent commits during the backup (otherwise we have a
> > > potential race condition).
> > >
> > > -Andy
> > >
> > >
> > > --
> > > Andy D'Arcy Jewell
> > >
> > > SysMicro Limited
> > > Linux Support
> > > E: andy.jewell@sysmicro.co.uk
> > > W: www.sysmicro.co.uk
> > >
> >
RE: Pause and resume indexing on SolR 4 for backups
Posted by Markus Jelsma <ma...@openindex.io>.
You can use the postCommit event in updateHandler to execute a task.
-----Original message-----
> From:Upayavira <uv...@odoko.co.uk>
> Sent: Thu 20-Dec-2012 14:45
> To: solr-user@lucene.apache.org
> Subject: Re: Pause and resume indexing on SolR 4 for backups
>
> The backup directory should just be a clone of the index files. I'm
> curious to know whether it is a cp -r or a cp -lr that the replication
> handler produces.
>
> You would prevent commits by telling your app not to commit. That is,
> Solr only commits when it is *told* to.
>
> Unless you use autocommit, in which case I guess you could monitor your
> logs for the last commit, and do your backup a 10 seconds after that.
>
> Upayavira
>
> On Thu, Dec 20, 2012, at 12:44 PM, Andy D'Arcy Jewell wrote:
> > On 20/12/12 11:58, Upayavira wrote:
> > > I've never used it, but the replication handler has an option:
> > >
> > > http://master_host:port/solr/replication?command=backup
> > >
> > > Which will take you a backup.
> > I've looked at that this morning as suggested by Markus Jelsma. Looks
> > good, but I'll have to work out how to use the resultant backup
> > directory. I've been dealing with another unrelated issue in the
> > mean-time and I haven't had a chance to look for any docu so far.
> > > Also something to note, if you don't want to use the above, and you are
> > > running on Unix, you can create fast 'hard link' clones of lucene
> > > indexes. Doing:
> > >
> > > cp -lr data data.bak
> > >
> > > will copy your index instantly. If you can avoid doing this when a
> > > commit is happening, then you'll have a good index copy, that will take
> > > no space on your disk and be made instantly. This is because it just
> > > copies the directory structure, not the files themselves, and given
> > > files in a lucene index never change (they are only ever deleted or
> > > replaced), this works as a good copy technique for backing up.
> > That's the approach that Shawn Heisey proposed, and what I've been
> > working towards, but it still leaves open the question of how to
> > *pause* SolR or prevent commits during the backup (otherwise we have a
> > potential race condition).
> >
> > -Andy
> >
> >
> > --
> > Andy D'Arcy Jewell
> >
> > SysMicro Limited
> > Linux Support
> > E: andy.jewell@sysmicro.co.uk
> > W: www.sysmicro.co.uk
> >
>
Re: Pause and resume indexing on SolR 4 for backups
Posted by Upayavira <uv...@odoko.co.uk>.
Are you sure a commit didn't happen between? Also, a background merge
might have happened.
As to using a backup, you are right, just stop solr, put the snapshot
into index/data, and restart.
Upayavira
On Thu, Dec 20, 2012, at 05:16 PM, Andy D'Arcy Jewell wrote:
> On 20/12/12 13:38, Upayavira wrote:
> > The backup directory should just be a clone of the index files. I'm
> > curious to know whether it is a cp -r or a cp -lr that the replication
> > handler produces.
> >
> > You would prevent commits by telling your app not to commit. That is,
> > Solr only commits when it is *told* to.
> >
> > Unless you use autocommit, in which case I guess you could monitor your
> > logs for the last commit, and do your backup a 10 seconds after that.
> >
> >
> Hmm. Strange - the files created by the backup API don't seem to
> correlate exactly with the files stored under the solr data directory:
>
> andydj@me-solr01:~$ find /tmp/snapshot.20121220155853703/
> /tmp/snapshot.20121220155853703/
> /tmp/snapshot.20121220155853703/_2vq.fdx
> /tmp/snapshot.20121220155853703/_2vq_Lucene40_0.tim
> /tmp/snapshot.20121220155853703/segments_2vs
> /tmp/snapshot.20121220155853703/_2vq_nrm.cfs
> /tmp/snapshot.20121220155853703/_2vq.fnm
> /tmp/snapshot.20121220155853703/_2vq_nrm.cfe
> /tmp/snapshot.20121220155853703/_2vq_Lucene40_0.frq
> /tmp/snapshot.20121220155853703/_2vq.fdt
> /tmp/snapshot.20121220155853703/_2vq.si
> /tmp/snapshot.20121220155853703/_2vq_Lucene40_0.tip
> andydj@me-solr01:~$ find /var/lib/solr/data/index/
> /var/lib/solr/data/index/
> /var/lib/solr/data/index/_2w6_Lucene40_0.frq
> /var/lib/solr/data/index/_2w6.si
> /var/lib/solr/data/index/segments_2w8
> /var/lib/solr/data/index/write.lock
> /var/lib/solr/data/index/_2w6_nrm.cfs
> /var/lib/solr/data/index/_2w6.fdx
> /var/lib/solr/data/index/_2w6_Lucene40_0.tip
> /var/lib/solr/data/index/_2w6_nrm.cfe
> /var/lib/solr/data/index/segments.gen
> /var/lib/solr/data/index/_2w6.fnm
> /var/lib/solr/data/index/_2w6.fdt
> /var/lib/solr/data/index/_2w6_Lucene40_0.tim
>
> Am I correct in thinking that to restore from this backup, I would need
> to do the following?
>
> 1. Stop Tomcat (or maybe just solr)
> 2. Remove all files under /var/lib/solr/data/index/
> 3. Move/copy files from /tmp/snapshot.20121220155853703/ to
> /var/lib/solr/data/index/
> 4. Restart Tomcat (or just solr)
>
>
> Thanks everyone who's pitched in on this! Once I've got this working,
> I'll document it.
> -Andy
>
> --
> Andy D'Arcy Jewell
>
> SysMicro Limited
> Linux Support
> E: andy.jewell@sysmicro.co.uk
> W: www.sysmicro.co.uk
>
Re: Pause and resume indexing on SolR 4 for backups
Posted by Andy D'Arcy Jewell <an...@sysmicro.co.uk>.
On 20/12/12 13:38, Upayavira wrote:
> The backup directory should just be a clone of the index files. I'm
> curious to know whether it is a cp -r or a cp -lr that the replication
> handler produces.
>
> You would prevent commits by telling your app not to commit. That is,
> Solr only commits when it is *told* to.
>
> Unless you use autocommit, in which case I guess you could monitor your
> logs for the last commit, and do your backup a 10 seconds after that.
>
>
Hmm. Strange - the files created by the backup API don't seem to
correlate exactly with the files stored under the solr data directory:
andydj@me-solr01:~$ find /tmp/snapshot.20121220155853703/
/tmp/snapshot.20121220155853703/
/tmp/snapshot.20121220155853703/_2vq.fdx
/tmp/snapshot.20121220155853703/_2vq_Lucene40_0.tim
/tmp/snapshot.20121220155853703/segments_2vs
/tmp/snapshot.20121220155853703/_2vq_nrm.cfs
/tmp/snapshot.20121220155853703/_2vq.fnm
/tmp/snapshot.20121220155853703/_2vq_nrm.cfe
/tmp/snapshot.20121220155853703/_2vq_Lucene40_0.frq
/tmp/snapshot.20121220155853703/_2vq.fdt
/tmp/snapshot.20121220155853703/_2vq.si
/tmp/snapshot.20121220155853703/_2vq_Lucene40_0.tip
andydj@me-solr01:~$ find /var/lib/solr/data/index/
/var/lib/solr/data/index/
/var/lib/solr/data/index/_2w6_Lucene40_0.frq
/var/lib/solr/data/index/_2w6.si
/var/lib/solr/data/index/segments_2w8
/var/lib/solr/data/index/write.lock
/var/lib/solr/data/index/_2w6_nrm.cfs
/var/lib/solr/data/index/_2w6.fdx
/var/lib/solr/data/index/_2w6_Lucene40_0.tip
/var/lib/solr/data/index/_2w6_nrm.cfe
/var/lib/solr/data/index/segments.gen
/var/lib/solr/data/index/_2w6.fnm
/var/lib/solr/data/index/_2w6.fdt
/var/lib/solr/data/index/_2w6_Lucene40_0.tim
Am I correct in thinking that to restore from this backup, I would need
to do the following?
1. Stop Tomcat (or maybe just solr)
2. Remove all files under /var/lib/solr/data/index/
3. Move/copy files from /tmp/snapshot.20121220155853703/ to
/var/lib/solr/data/index/
4. Restart Tomcat (or just solr)
Thanks everyone who's pitched in on this! Once I've got this working,
I'll document it.
-Andy
--
Andy D'Arcy Jewell
SysMicro Limited
Linux Support
E: andy.jewell@sysmicro.co.uk
W: www.sysmicro.co.uk
Re: Pause and resume indexing on SolR 4 for backups
Posted by Upayavira <uv...@odoko.co.uk>.
The backup directory should just be a clone of the index files. I'm
curious to know whether it is a cp -r or a cp -lr that the replication
handler produces.
You would prevent commits by telling your app not to commit. That is,
Solr only commits when it is *told* to.
Unless you use autocommit, in which case I guess you could monitor your
logs for the last commit, and do your backup a 10 seconds after that.
Upayavira
On Thu, Dec 20, 2012, at 12:44 PM, Andy D'Arcy Jewell wrote:
> On 20/12/12 11:58, Upayavira wrote:
> > I've never used it, but the replication handler has an option:
> >
> > http://master_host:port/solr/replication?command=backup
> >
> > Which will take you a backup.
> I've looked at that this morning as suggested by Markus Jelsma. Looks
> good, but I'll have to work out how to use the resultant backup
> directory. I've been dealing with another unrelated issue in the
> mean-time and I haven't had a chance to look for any docu so far.
> > Also something to note, if you don't want to use the above, and you are
> > running on Unix, you can create fast 'hard link' clones of lucene
> > indexes. Doing:
> >
> > cp -lr data data.bak
> >
> > will copy your index instantly. If you can avoid doing this when a
> > commit is happening, then you'll have a good index copy, that will take
> > no space on your disk and be made instantly. This is because it just
> > copies the directory structure, not the files themselves, and given
> > files in a lucene index never change (they are only ever deleted or
> > replaced), this works as a good copy technique for backing up.
> That's the approach that Shawn Heisey proposed, and what I've been
> working towards, but it still leaves open the question of how to
> *pause* SolR or prevent commits during the backup (otherwise we have a
> potential race condition).
>
> -Andy
>
>
> --
> Andy D'Arcy Jewell
>
> SysMicro Limited
> Linux Support
> E: andy.jewell@sysmicro.co.uk
> W: www.sysmicro.co.uk
>
Re: Pause and resume indexing on SolR 4 for backups
Posted by Andy D'Arcy Jewell <an...@sysmicro.co.uk>.
On 20/12/12 11:58, Upayavira wrote:
> I've never used it, but the replication handler has an option:
>
> http://master_host:port/solr/replication?command=backup
>
> Which will take you a backup.
I've looked at that this morning as suggested by Markus Jelsma. Looks
good, but I'll have to work out how to use the resultant backup
directory. I've been dealing with another unrelated issue in the
mean-time and I haven't had a chance to look for any docu so far.
> Also something to note, if you don't want to use the above, and you are
> running on Unix, you can create fast 'hard link' clones of lucene
> indexes. Doing:
>
> cp -lr data data.bak
>
> will copy your index instantly. If you can avoid doing this when a
> commit is happening, then you'll have a good index copy, that will take
> no space on your disk and be made instantly. This is because it just
> copies the directory structure, not the files themselves, and given
> files in a lucene index never change (they are only ever deleted or
> replaced), this works as a good copy technique for backing up.
That's the approach that Shawn Heisey proposed, and what I've been
working towards, but it still leaves open the question of how to
*pause* SolR or prevent commits during the backup (otherwise we have a
potential race condition).
-Andy
--
Andy D'Arcy Jewell
SysMicro Limited
Linux Support
E: andy.jewell@sysmicro.co.uk
W: www.sysmicro.co.uk
Re: Pause and resume indexing on SolR 4 for backups
Posted by Upayavira <uv...@odoko.co.uk>.
I've never used it, but the replication handler has an option:
http://master_host:port/solr/replication?command=backup
Which will take you a backup.
Also something to note, if you don't want to use the above, and you are
running on Unix, you can create fast 'hard link' clones of lucene
indexes. Doing:
cp -lr data data.bak
will copy your index instantly. If you can avoid doing this when a
commit is happening, then you'll have a good index copy, that will take
no space on your disk and be made instantly. This is because it just
copies the directory structure, not the files themselves, and given
files in a lucene index never change (they are only ever deleted or
replaced), this works as a good copy technique for backing up.
Upayavira
On Thu, Dec 20, 2012, at 10:34 AM, Markus Jelsma wrote:
> You can use the replication handler to fetch a complete snapshot of the
> index over HTTP.
> http://wiki.apache.org/solr/SolrReplication#HTTP_API
>
>
> -----Original message-----
> > From:Andy D'Arcy Jewell <an...@sysmicro.co.uk>
> > Sent: Thu 20-Dec-2012 11:23
> > To: solr-user@lucene.apache.org
> > Subject: Pause and resume indexing on SolR 4 for backups
> >
> > Hi all.
> >
> > Can anyone advise me of a way to pause and resume SolR 4 so I can
> > perform a backup? I need to be able to revert to a usable (though not
> > necessarily complete) index after a crash or other "disaster" more
> > quickly than a re-index operation would yield.
> >
> > I can't yet afford the "extravagance" of a separate SolR replica just
> > for backups, and I'm not sure if I'll ever have the luxury. I'm
> > currently running with just one node, be we are not yet live.
> >
> > I can think of the following ways to do this, each with various downsides:
> >
> > 1) Just backup the existing index files whilst indexing continues
> > + Easy
> > + Fast
> > - Incomplete
> > - Potential for corruption? (e.g. partial files)
> >
> > 2) Stop/Start Tomcat
> > + Easy
> > - Very slow and I/O, CPU intensive
> > - Client gets errors when trying to connect
> >
> > 3) Block/unblock SolR port with IpTables
> > + Fast
> > - Client gets errors when trying to connect
> > - Have to wait for existing transactions to complete (not sure how,
> > maybe watch socket FD's in /proc)
> >
> > 4) Pause/Restart SolR service
> > + Fast ? (hopefully)
> > - Client gets errors when trying to connect
> >
> > In any event, the web app will have to gracefully handle unavailability
> > of SolR, probably by displaying a "down for maintenance" message, but
> > this should preferably be only a very short amount of time.
> >
> > Can anyone comment on my proposed solutions above, or provide any
> > additional ones?
> >
> > Thanks for any input you can provide!
> >
> > -Andy
> >
> > --
> > Andy D'Arcy Jewell
> >
> > SysMicro Limited
> > Linux Support
> > E: andy.jewell@sysmicro.co.uk
> > W: www.sysmicro.co.uk
> >
> >
RE: Pause and resume indexing on SolR 4 for backups
Posted by Markus Jelsma <ma...@openindex.io>.
You can use the replication handler to fetch a complete snapshot of the index over HTTP.
http://wiki.apache.org/solr/SolrReplication#HTTP_API
-----Original message-----
> From:Andy D'Arcy Jewell <an...@sysmicro.co.uk>
> Sent: Thu 20-Dec-2012 11:23
> To: solr-user@lucene.apache.org
> Subject: Pause and resume indexing on SolR 4 for backups
>
> Hi all.
>
> Can anyone advise me of a way to pause and resume SolR 4 so I can
> perform a backup? I need to be able to revert to a usable (though not
> necessarily complete) index after a crash or other "disaster" more
> quickly than a re-index operation would yield.
>
> I can't yet afford the "extravagance" of a separate SolR replica just
> for backups, and I'm not sure if I'll ever have the luxury. I'm
> currently running with just one node, be we are not yet live.
>
> I can think of the following ways to do this, each with various downsides:
>
> 1) Just backup the existing index files whilst indexing continues
> + Easy
> + Fast
> - Incomplete
> - Potential for corruption? (e.g. partial files)
>
> 2) Stop/Start Tomcat
> + Easy
> - Very slow and I/O, CPU intensive
> - Client gets errors when trying to connect
>
> 3) Block/unblock SolR port with IpTables
> + Fast
> - Client gets errors when trying to connect
> - Have to wait for existing transactions to complete (not sure how,
> maybe watch socket FD's in /proc)
>
> 4) Pause/Restart SolR service
> + Fast ? (hopefully)
> - Client gets errors when trying to connect
>
> In any event, the web app will have to gracefully handle unavailability
> of SolR, probably by displaying a "down for maintenance" message, but
> this should preferably be only a very short amount of time.
>
> Can anyone comment on my proposed solutions above, or provide any
> additional ones?
>
> Thanks for any input you can provide!
>
> -Andy
>
> --
> Andy D'Arcy Jewell
>
> SysMicro Limited
> Linux Support
> E: andy.jewell@sysmicro.co.uk
> W: www.sysmicro.co.uk
>
>
Re: Pause and resume indexing on SolR 4 for backups
Posted by Gora Mohanty <go...@mimirtech.com>.
On 20 December 2012 16:14, Andy D'Arcy Jewell
<an...@sysmicro.co.uk> wrote:
[...]
> It's attached to a web-app, which accepts uploads and will be available
> 24/7, with a global audience, so "pausing" it may be rather difficult (tho I
> may put this to the developer - it may for instance be possible if he has a
> small number of choke points for input into SolR).
[...]
It adds work for the web developer, but one could pause indexing,
put indexing requests into some kind of a queuing system, do the
backup, and flush the queue when the backup is done.
Regards,
Gora
Re: Pause and resume indexing on SolR 4 for backups
Posted by Andy D'Arcy Jewell <an...@sysmicro.co.uk>.
On 20/12/12 10:24, Gora Mohanty wrote:
>
> Unless I am missing something, the index is only being written to
> when you are adding/updating the index. So, the question is how
> is this being done in your case, and could you pause indexing for
> the duration of the backup?
>
> Regards,
> Gora
It's attached to a web-app, which accepts uploads and will be available
24/7, with a global audience, so "pausing" it may be rather difficult
(tho I may put this to the developer - it may for instance be possible
if he has a small number of choke points for input into SolR).
Thanks.
--
Andy D'Arcy Jewell
SysMicro Limited
Linux Support
T: 0844 9918804
M: 07961605631
E: andy.jewell@sysmicro.co.uk
W: www.sysmicro.co.uk
Re: Pause and resume indexing on SolR 4 for backups
Posted by Gora Mohanty <go...@mimirtech.com>.
On 20 December 2012 15:46, Andy D'Arcy Jewell
<an...@sysmicro.co.uk> wrote:
> Hi all.
>
> Can anyone advise me of a way to pause and resume SolR 4 so I can perform a
> backup? I need to be able to revert to a usable (though not necessarily
> complete) index after a crash or other "disaster" more quickly than a
> re-index operation would yield.
[...]
Unless I am missing something, the index is only being written to
when you are adding/updating the index. So, the question is how
is this being done in your case, and could you pause indexing for
the duration of the backup?
Regards,
Gora
Re: Pause and resume indexing on SolR 4 for backups
Posted by Andy D'Arcy Jewell <an...@sysmicro.co.uk>.
On 20/12/12 20:19, alxsss@aim.com wrote:
> Depending on your architecture, why not index the same data into two machines? One will be your prod another your backup?
Because we're trying to keep costs and complexity low whilst in the
development stage ;-)
But more seriously, this will obviously be a must sooner or later.
--
Andy D'Arcy Jewell
SysMicro Limited
Linux Support
T: 0844 9918804
M: 07961605631
E: andy.jewell@sysmicro.co.uk
W: www.sysmicro.co.uk
Re: Pause and resume indexing on SolR 4 for backups
Posted by al...@aim.com.
Depending on your architecture, why not index the same data into two machines? One will be your prod another your backup?
Thanks.
Alex.
-----Original Message-----
From: Upayavira <uv...@odoko.co.uk>
To: solr-user <so...@lucene.apache.org>
Sent: Thu, Dec 20, 2012 11:51 am
Subject: Re: Pause and resume indexing on SolR 4 for backups
You're saying that there's no chance to catch it in the middle of
writing the segments file?
Having said that, the segments file is pretty small, so the chance would
be pretty slim.
Upayavira
On Thu, Dec 20, 2012, at 06:45 PM, Lance Norskog wrote:
> To be clear: 1) is fine. Lucene index updates are carefully sequenced so
> that the index is never in a bogus state. All data files are written and
> flushed to disk, then the segments.* files are written that match the
> data files. You can capture the files with a set of hard links to create
> a backup.
>
> The CheckIndex program will verify the index backup.
> java -cp yourcopy/lucene-core-SOMETHING.jar
> org.apache.lucene.index.CheckIndex collection/data/index
>
> lucene-core-SOMETHING.jar is usually in the solr-webapp directory where
> Solr is unpacked.
>
> On 12/20/2012 02:16 AM, Andy D'Arcy Jewell wrote:
> > Hi all.
> >
> > Can anyone advise me of a way to pause and resume SolR 4 so I can
> > perform a backup? I need to be able to revert to a usable (though not
> > necessarily complete) index after a crash or other "disaster" more
> > quickly than a re-index operation would yield.
> >
> > I can't yet afford the "extravagance" of a separate SolR replica just
> > for backups, and I'm not sure if I'll ever have the luxury. I'm
> > currently running with just one node, be we are not yet live.
> >
> > I can think of the following ways to do this, each with various
> > downsides:
> >
> > 1) Just backup the existing index files whilst indexing continues
> > + Easy
> > + Fast
> > - Incomplete
> > - Potential for corruption? (e.g. partial files)
> >
> > 2) Stop/Start Tomcat
> > + Easy
> > - Very slow and I/O, CPU intensive
> > - Client gets errors when trying to connect
> >
> > 3) Block/unblock SolR port with IpTables
> > + Fast
> > - Client gets errors when trying to connect
> > - Have to wait for existing transactions to complete (not sure
> > how, maybe watch socket FD's in /proc)
> >
> > 4) Pause/Restart SolR service
> > + Fast ? (hopefully)
> > - Client gets errors when trying to connect
> >
> > In any event, the web app will have to gracefully handle
> > unavailability of SolR, probably by displaying a "down for
> > maintenance" message, but this should preferably be only a very short
> > amount of time.
> >
> > Can anyone comment on my proposed solutions above, or provide any
> > additional ones?
> >
> > Thanks for any input you can provide!
> >
> > -Andy
> >
>
Re: Pause and resume indexing on SolR 4 for backups
Posted by Upayavira <uv...@odoko.co.uk>.
You're saying that there's no chance to catch it in the middle of
writing the segments file?
Having said that, the segments file is pretty small, so the chance would
be pretty slim.
Upayavira
On Thu, Dec 20, 2012, at 06:45 PM, Lance Norskog wrote:
> To be clear: 1) is fine. Lucene index updates are carefully sequenced so
> that the index is never in a bogus state. All data files are written and
> flushed to disk, then the segments.* files are written that match the
> data files. You can capture the files with a set of hard links to create
> a backup.
>
> The CheckIndex program will verify the index backup.
> java -cp yourcopy/lucene-core-SOMETHING.jar
> org.apache.lucene.index.CheckIndex collection/data/index
>
> lucene-core-SOMETHING.jar is usually in the solr-webapp directory where
> Solr is unpacked.
>
> On 12/20/2012 02:16 AM, Andy D'Arcy Jewell wrote:
> > Hi all.
> >
> > Can anyone advise me of a way to pause and resume SolR 4 so I can
> > perform a backup? I need to be able to revert to a usable (though not
> > necessarily complete) index after a crash or other "disaster" more
> > quickly than a re-index operation would yield.
> >
> > I can't yet afford the "extravagance" of a separate SolR replica just
> > for backups, and I'm not sure if I'll ever have the luxury. I'm
> > currently running with just one node, be we are not yet live.
> >
> > I can think of the following ways to do this, each with various
> > downsides:
> >
> > 1) Just backup the existing index files whilst indexing continues
> > + Easy
> > + Fast
> > - Incomplete
> > - Potential for corruption? (e.g. partial files)
> >
> > 2) Stop/Start Tomcat
> > + Easy
> > - Very slow and I/O, CPU intensive
> > - Client gets errors when trying to connect
> >
> > 3) Block/unblock SolR port with IpTables
> > + Fast
> > - Client gets errors when trying to connect
> > - Have to wait for existing transactions to complete (not sure
> > how, maybe watch socket FD's in /proc)
> >
> > 4) Pause/Restart SolR service
> > + Fast ? (hopefully)
> > - Client gets errors when trying to connect
> >
> > In any event, the web app will have to gracefully handle
> > unavailability of SolR, probably by displaying a "down for
> > maintenance" message, but this should preferably be only a very short
> > amount of time.
> >
> > Can anyone comment on my proposed solutions above, or provide any
> > additional ones?
> >
> > Thanks for any input you can provide!
> >
> > -Andy
> >
>
Re: Pause and resume indexing on SolR 4 for backups
Posted by Lance Norskog <go...@gmail.com>.
To be clear: 1) is fine. Lucene index updates are carefully sequenced so
that the index is never in a bogus state. All data files are written and
flushed to disk, then the segments.* files are written that match the
data files. You can capture the files with a set of hard links to create
a backup.
The CheckIndex program will verify the index backup.
java -cp yourcopy/lucene-core-SOMETHING.jar
org.apache.lucene.index.CheckIndex collection/data/index
lucene-core-SOMETHING.jar is usually in the solr-webapp directory where
Solr is unpacked.
On 12/20/2012 02:16 AM, Andy D'Arcy Jewell wrote:
> Hi all.
>
> Can anyone advise me of a way to pause and resume SolR 4 so I can
> perform a backup? I need to be able to revert to a usable (though not
> necessarily complete) index after a crash or other "disaster" more
> quickly than a re-index operation would yield.
>
> I can't yet afford the "extravagance" of a separate SolR replica just
> for backups, and I'm not sure if I'll ever have the luxury. I'm
> currently running with just one node, be we are not yet live.
>
> I can think of the following ways to do this, each with various
> downsides:
>
> 1) Just backup the existing index files whilst indexing continues
> + Easy
> + Fast
> - Incomplete
> - Potential for corruption? (e.g. partial files)
>
> 2) Stop/Start Tomcat
> + Easy
> - Very slow and I/O, CPU intensive
> - Client gets errors when trying to connect
>
> 3) Block/unblock SolR port with IpTables
> + Fast
> - Client gets errors when trying to connect
> - Have to wait for existing transactions to complete (not sure
> how, maybe watch socket FD's in /proc)
>
> 4) Pause/Restart SolR service
> + Fast ? (hopefully)
> - Client gets errors when trying to connect
>
> In any event, the web app will have to gracefully handle
> unavailability of SolR, probably by displaying a "down for
> maintenance" message, but this should preferably be only a very short
> amount of time.
>
> Can anyone comment on my proposed solutions above, or provide any
> additional ones?
>
> Thanks for any input you can provide!
>
> -Andy
>