You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Ron Poling <ro...@teamexpress.com> on 2013/01/23 20:32:10 UTC
Indexing question
Hello!
I'm new to solr and trying to figure out how to implement it in our environment. My question involves building the index. Our data does not lend itself to delta updates so we have to build the entire index each time. Is there some way to feed solr a file with all index records and tell it to throw away all current data and use only the new? I'm guessing that I could delete everything and add all the new records, but until the new index was built, solr would not be able to service my web app. I would like to build the new index in solr and then tell it to switch to it and remove the old one. Is that possible? Another way of doing the might be to update the current index with the new data and then delete everything that didn't get updated.
Any help here would be appreciated so I can focus on the things in the wiki that I need to before I start implementing. Thanks!
Re: Indexing question
Posted by Upayavira <uv...@odoko.co.uk>.
you can do this with cores. You can have one core to serve the public,
and one for indexing. Then, when you've finished updating your index,
you use the core admin handler to swap the cores around. Then you do the
same thing the following night. Doesn't require any file moving nor any
restarts of solr, just a call to an HTTP URL.
Upayavira
On Wed, Jan 23, 2013, at 07:51 PM, Alan Rykhus wrote:
> Hello,
>
> I do nightly builds for one of my sites. I build the new index in a
> parallel directory. When it is finished I move the old files to a backup
> directory(I only save one, delete the previous), move the new database
> files to the correct place, then stop and restart solr. It sees the new
> database and uses it.
>
> Moving the old files over, means I always have a quick backup if
> something goes wrong. It hasn't happened yet though.
>
> al
>
> On Wed, 2013-01-23 at 13:32 -0600, Ron Poling wrote:
> > Hello!
> >
> > I'm new to solr and trying to figure out how to implement it in our
> > environment. My question involves building the index. Our data does
> > not lend itself to delta updates so we have to build the entire index
> > each time. Is there some way to feed solr a file with all index
> > records and tell it to throw away all current data and use only the
> > new? I'm guessing that I could delete everything and add all the new
> > records, but until the new index was built, solr would not be able to
> > service my web app. I would like to build the new index in solr and
> > then tell it to switch to it and remove the old one. Is that possible?
> > Another way of doing the might be to update the current index with the
> > new data and then delete everything that didn't get updated.
> >
> > Any help here would be appreciated so I can focus on the things in the wiki that I need to before I start implementing. Thanks!
>
> --
> Alan Rykhus
> PALS, A Program of the Minnesota State Colleges and Universities
> (507)389-1975
> alan.rykhus@mnsu.edu
> "Be pleasant until ten o'clock in the morning and the rest of the day
> will take care of itself." ~ Elbert Hubbard
>
Re: Indexing question
Posted by Alan Rykhus <al...@mnsu.edu>.
Hello,
I do nightly builds for one of my sites. I build the new index in a
parallel directory. When it is finished I move the old files to a backup
directory(I only save one, delete the previous), move the new database
files to the correct place, then stop and restart solr. It sees the new
database and uses it.
Moving the old files over, means I always have a quick backup if
something goes wrong. It hasn't happened yet though.
al
On Wed, 2013-01-23 at 13:32 -0600, Ron Poling wrote:
> Hello!
>
> I'm new to solr and trying to figure out how to implement it in our
> environment. My question involves building the index. Our data does
> not lend itself to delta updates so we have to build the entire index
> each time. Is there some way to feed solr a file with all index
> records and tell it to throw away all current data and use only the
> new? I'm guessing that I could delete everything and add all the new
> records, but until the new index was built, solr would not be able to
> service my web app. I would like to build the new index in solr and
> then tell it to switch to it and remove the old one. Is that possible?
> Another way of doing the might be to update the current index with the
> new data and then delete everything that didn't get updated.
>
> Any help here would be appreciated so I can focus on the things in the wiki that I need to before I start implementing. Thanks!
--
Alan Rykhus
PALS, A Program of the Minnesota State Colleges and Universities
(507)389-1975
alan.rykhus@mnsu.edu
"Be pleasant until ten o'clock in the morning and the rest of the day
will take care of itself." ~ Elbert Hubbard
Re: Indexing question
Posted by Michael Della Bitta <mi...@appinions.com>.
Hi Ron,
If you turn off autoCommit and only commit after your delete and refresh,
the user's experience will be totally uninterrupted. Commits are used to
control visibility in a Solr index.
Michael Della Bitta
------------------------------------------------
Appinions
18 East 41st Street, 2nd Floor
New York, NY 10017-6271
www.appinions.com
Where Influence Isn’t a Game
On Wed, Jan 23, 2013 at 2:32 PM, Ron Poling <ro...@teamexpress.com>wrote:
> Hello!
>
> I'm new to solr and trying to figure out how to implement it in our
> environment. My question involves building the index. Our data does not
> lend itself to delta updates so we have to build the entire index each
> time. Is there some way to feed solr a file with all index records and tell
> it to throw away all current data and use only the new? I'm guessing that I
> could delete everything and add all the new records, but until the new
> index was built, solr would not be able to service my web app. I would like
> to build the new index in solr and then tell it to switch to it and remove
> the old one. Is that possible? Another way of doing the might be to update
> the current index with the new data and then delete everything that didn't
> get updated.
>
> Any help here would be appreciated so I can focus on the things in the
> wiki that I need to before I start implementing. Thanks!
>