You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Mike Austin <mi...@juggle.com> on 2011/08/26 17:13:00 UTC
New IndexSearcher and autowarming
I would like to have the ability to keep requests from being slowed from new
document adds and commits by having a separate index that gets updated.
Basically a read-only and an updatable index. After the update index has
finished updating with new adds and commits, I'd like to switch the update
to the "live" read-only. At the same time, it would be nice to have the old
read-only index become "updated" with the now live read-only index before I
start this update process again.
1. Index1 is live and read-only and doesn't get slowed by updates
2. Index2 is updated with Index1 and gets new adds and commits
3. Index2 gets cache warming
4. Index2 becomes the live index read-only index
5. Index1 gets synced with Index2 so that when these steps start again, the
updating is happening on an updated index.
I know that this is possible but can't find a simple tutorial on how to do
this. By the way, I'm using SolrNet in a windows environment.
Thanks,
Mike
Re: New IndexSearcher and autowarming
Posted by Mike Austin <mi...@juggle.com>.
Hi Erick,
It might work. I've only worked with solr having one index on one server
over a year ago so I might need to just research more about the replication.
I am using windows and I remember that replication on windows had some
issues with scripts and hard links, however it looks like we have some new
good replication features with solr1.4.
For now, I wanted to do this on just one windows server since this is my
requirement. After your suggestion, I took a little more time to review:
http://wiki.apache.org/solr/SolrReplication. So based on what I want to do,
would the "Replication with MultiCore " section be what I need to do? But
this wouldn't be a master/slave setup would it since basically I want to
swap between two. I guess I could set up 3 indexes on the same server if
that's possible to use master/slave in that way, but that might take some
more space than I anticipated.
Thanks,
Mike
On Fri, Aug 26, 2011 at 12:08 PM, Erick Erickson <er...@gmail.com>wrote:
> Why doesn't standard replication with auto-warming work for you?
> You can control how often replication gets triggered by controlling
> your commit points and/or your replication interval. This seems easier
> than maintaining cores like your problem statement indicates.
>
> Best
> Erick
>
> On Fri, Aug 26, 2011 at 12:56 PM, simon <mt...@gmail.com> wrote:
> > The multicore API (see http://wiki.apache.org/solr/CoreAdmin ) allows
> you to
> > swap, unload, reload cores. That should allow you to do what you want,
> >
> > -Simon
> >
> > On Fri, Aug 26, 2011 at 11:13 AM, Mike Austin <mike.austin@juggle.com
> >wrote:
> >
> >> I would like to have the ability to keep requests from being slowed from
> >> new
> >> document adds and commits by having a separate index that gets updated.
> >> Basically a read-only and an updatable index. After the update index has
> >> finished updating with new adds and commits, I'd like to switch the
> update
> >> to the "live" read-only. At the same time, it would be nice to have the
> >> old
> >> read-only index become "updated" with the now live read-only index
> before I
> >> start this update process again.
> >>
> >> 1. Index1 is live and read-only and doesn't get slowed by updates
> >> 2. Index2 is updated with Index1 and gets new adds and commits
> >> 3. Index2 gets cache warming
> >> 4. Index2 becomes the live index read-only index
> >> 5. Index1 gets synced with Index2 so that when these steps start again,
> the
> >> updating is happening on an updated index.
> >>
> >> I know that this is possible but can't find a simple tutorial on how to
> do
> >> this. By the way, I'm using SolrNet in a windows environment.
> >>
> >> Thanks,
> >> Mike
> >>
> >
>
Re: New IndexSearcher and autowarming
Posted by Erick Erickson <er...@gmail.com>.
Why doesn't standard replication with auto-warming work for you?
You can control how often replication gets triggered by controlling
your commit points and/or your replication interval. This seems easier
than maintaining cores like your problem statement indicates.
Best
Erick
On Fri, Aug 26, 2011 at 12:56 PM, simon <mt...@gmail.com> wrote:
> The multicore API (see http://wiki.apache.org/solr/CoreAdmin ) allows you to
> swap, unload, reload cores. That should allow you to do what you want,
>
> -Simon
>
> On Fri, Aug 26, 2011 at 11:13 AM, Mike Austin <mi...@juggle.com>wrote:
>
>> I would like to have the ability to keep requests from being slowed from
>> new
>> document adds and commits by having a separate index that gets updated.
>> Basically a read-only and an updatable index. After the update index has
>> finished updating with new adds and commits, I'd like to switch the update
>> to the "live" read-only. At the same time, it would be nice to have the
>> old
>> read-only index become "updated" with the now live read-only index before I
>> start this update process again.
>>
>> 1. Index1 is live and read-only and doesn't get slowed by updates
>> 2. Index2 is updated with Index1 and gets new adds and commits
>> 3. Index2 gets cache warming
>> 4. Index2 becomes the live index read-only index
>> 5. Index1 gets synced with Index2 so that when these steps start again, the
>> updating is happening on an updated index.
>>
>> I know that this is possible but can't find a simple tutorial on how to do
>> this. By the way, I'm using SolrNet in a windows environment.
>>
>> Thanks,
>> Mike
>>
>
Re: New IndexSearcher and autowarming
Posted by simon <mt...@gmail.com>.
The multicore API (see http://wiki.apache.org/solr/CoreAdmin ) allows you to
swap, unload, reload cores. That should allow you to do what you want,
-Simon
On Fri, Aug 26, 2011 at 11:13 AM, Mike Austin <mi...@juggle.com>wrote:
> I would like to have the ability to keep requests from being slowed from
> new
> document adds and commits by having a separate index that gets updated.
> Basically a read-only and an updatable index. After the update index has
> finished updating with new adds and commits, I'd like to switch the update
> to the "live" read-only. At the same time, it would be nice to have the
> old
> read-only index become "updated" with the now live read-only index before I
> start this update process again.
>
> 1. Index1 is live and read-only and doesn't get slowed by updates
> 2. Index2 is updated with Index1 and gets new adds and commits
> 3. Index2 gets cache warming
> 4. Index2 becomes the live index read-only index
> 5. Index1 gets synced with Index2 so that when these steps start again, the
> updating is happening on an updated index.
>
> I know that this is possible but can't find a simple tutorial on how to do
> this. By the way, I'm using SolrNet in a windows environment.
>
> Thanks,
> Mike
>