You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@lucene.apache.org by "Beyer,Nathan" <NB...@CERNER.COM> on 2007/12/06 18:43:15 UTC
best practices for reloading an index for a searcher
I did some searching on the lucene site and wiki, but didn't quite find
what I was looking for in regards to a basic approach to how and when to
reload index data. I have a long running process that will be
continually indexing and concurrently searching the same index and I'm
looking for a basic approach to reloading the data for the searcher.
Perhaps I should verify that the data does need to be reloaded. Does a
manual reload of a changing index need to be performed for a searcher?
I'd appreciate any links to documents, source or some quick pseudo code.
Thanks.
-Nathan
----------------------------------------------------------------------
CONFIDENTIALITY NOTICE This message and any included attachments are from Cerner Corporation and are intended only for the addressee. The information contained in this message is confidential and may constitute inside or non-public information under international, federal, or state securities laws. Unauthorized forwarding, printing, copying, distribution, or use of such information is strictly prohibited and may be unlawful. If you are not the addressee, please promptly delete this message and notify the sender of the delivery error by e-mail or you may call Cerner's corporate offices in Kansas City, Missouri, U.S.A at (+1) (816)221-1024.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
RE: best practices for reloading an index for a searcher
Posted by "Beyer,Nathan" <NB...@CERNER.COM>.
That does help, thank you.
Does closing a reader, writer or searcher close the underlying
Directory? I've been operating under the assumption that it does not and
that I should be sharing the Directory instance as much as possible.
-Nathan
-----Original Message-----
From: Erick Erickson [mailto:erickerickson@gmail.com]
Sent: Thursday, December 06, 2007 12:10 PM
To: java-user@lucene.apache.org
Subject: Re: best practices for reloading an index for a searcher
If by reload you mean closing and opening the reader, then yes. You need
to do this in order to see the changes since the *last* time you opened
the reader.
Think of it as the reader taking a snapshot of the index and using that
for its lifetime.
Be aware that opening a reader (and running the first searches) is an
expensive operation and shouldn't be done unnecessarily. I guess it
all depends upon your requirements for how soon changes are available
for search.
As for guidelines, to quote the famous wise ones.... it depends. The
real question is how quickly your users require seeing recent changes.
One technique for handling these updates is to periodically open a new
reader
*without* closing the old one. Fire a few warm-up queries at the new
searcher,
then close the old one and start using the new one. There are some
coordination issues here..... Also be aware that this will chew up some
disk space....
Hope this helps
Erick
On Dec 6, 2007 12:43 PM, Beyer,Nathan <NB...@cerner.com> wrote:
> I did some searching on the lucene site and wiki, but didn't quite
find
> what I was looking for in regards to a basic approach to how and when
to
> reload index data. I have a long running process that will be
> continually indexing and concurrently searching the same index and I'm
> looking for a basic approach to reloading the data for the searcher.
> Perhaps I should verify that the data does need to be reloaded. Does a
> manual reload of a changing index need to be performed for a searcher?
>
> I'd appreciate any links to documents, source or some quick pseudo
code.
>
> Thanks.
>
> -Nathan
>
----------------------------------------------------------------------
CONFIDENTIALITY NOTICE This message and any included attachments are from Cerner Corporation and are intended only for the addressee. The information contained in this message is confidential and may constitute inside or non-public information under international, federal, or state securities laws. Unauthorized forwarding, printing, copying, distribution, or use of such information is strictly prohibited and may be unlawful. If you are not the addressee, please promptly delete this message and notify the sender of the delivery error by e-mail or you may call Cerner's corporate offices in Kansas City, Missouri, U.S.A at (+1) (816)221-1024.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: best practices for reloading an index for a searcher
Posted by Erick Erickson <er...@gmail.com>.
If by reload you mean closing and opening the reader, then yes. You need
to do this in order to see the changes since the *last* time you opened
the reader.
Think of it as the reader taking a snapshot of the index and using that
for its lifetime.
Be aware that opening a reader (and running the first searches) is an
expensive operation and shouldn't be done unnecessarily. I guess it
all depends upon your requirements for how soon changes are available
for search.
As for guidelines, to quote the famous wise ones.... it depends. The
real question is how quickly your users require seeing recent changes.
One technique for handling these updates is to periodically open a new
reader
*without* closing the old one. Fire a few warm-up queries at the new
searcher,
then close the old one and start using the new one. There are some
coordination issues here..... Also be aware that this will chew up some
disk space....
Hope this helps
Erick
On Dec 6, 2007 12:43 PM, Beyer,Nathan <NB...@cerner.com> wrote:
> I did some searching on the lucene site and wiki, but didn't quite find
> what I was looking for in regards to a basic approach to how and when to
> reload index data. I have a long running process that will be
> continually indexing and concurrently searching the same index and I'm
> looking for a basic approach to reloading the data for the searcher.
> Perhaps I should verify that the data does need to be reloaded. Does a
> manual reload of a changing index need to be performed for a searcher?
>
> I'd appreciate any links to documents, source or some quick pseudo code.
>
> Thanks.
>
> -Nathan
>
> ----------------------------------------------------------------------
> CONFIDENTIALITY NOTICE This message and any included attachments are from
> Cerner Corporation and are intended only for the addressee. The information
> contained in this message is confidential and may constitute inside or
> non-public information under international, federal, or state securities
> laws. Unauthorized forwarding, printing, copying, distribution, or use of
> such information is strictly prohibited and may be unlawful. If you are not
> the addressee, please promptly delete this message and notify the sender of
> the delivery error by e-mail or you may call Cerner's corporate offices in
> Kansas City, Missouri, U.S.A at (+1) (816)221-1024.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>