You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-dev@lucene.apache.org by jason rutherglen <ja...@yahoo.com> on 2006/04/26 19:19:55 UTC
Re: GData, updateable IndexSearcher
Hi Doug,
Thanks for the info, makes sense.
> In particular, it supports scaling the number of *readers* well.
Yes this is very true and a good architecture and in fact because Java comes in 64-bit flavors allows for a smaller number of machines as per 32-bit built C systems that have memory limitations like the current Google architecture.
> Yes. Folks have developed incrementally updateable IndexSearchers before, but none is yet part of Lucene.
Interesting, does this mean there is a plan for incrementally updateable IndexSearchers to become part of Lucene? Are there any negatives to updateable IndexSearchers?
Thanks,
Jason
----- Original Message ----
From: Doug Cutting <cu...@apache.org>
To: solr-dev@lucene.apache.org
Sent: Tuesday, April 25, 2006 9:04:47 PM
Subject: Re: GData
jason rutherglen wrote:
> Ah ok, think I found it: org.apache.nutch.indexer.FsDirectory no?
>
> Couldn't this be used in Solr and distribute all the data rather than master/slave it?
It's possible to search a Lucene index that lives in Hadoop's DFS, but
not recommended. It's very slow. It's much faster to copy the index to
a local drive.
The rsync approach, of only transmitting index diffs, is a very
efficient way to distribute an index. In particular, it supports
scaling the number of *readers* well.
For read/write stuff (e.g. a calendar) such scaling might not be
paramount. Rather, you might be happy to route all requests for a
particular calendar to a particular server. The index/database could
still be somehow replicated/synced, in case that server dies, but a
single server can probably handle all requests for a particular
index/database. And keeping things coherent is much simpler in this case.
Doug
Re: GData, updateable IndexSearcher
Posted by Yonik Seeley <ys...@gmail.com>.
On 4/26/06, jason rutherglen <ja...@yahoo.com> wrote:
> As this would be used with Solr I wonder if there would be a good way to also update the Solr caches.
Other than re-executing queries that generated the results? Probably not.
The nice thing about knowing exactly when the view of an index changes
(and having it only change once in a while), is that you can do very
aggressive caching.
If you want an IndexSearcher who's view of the index changed every
second (for example), I don't think Solr's type of caching would be
useful at all (or even possible, if you have big caches and
autowarming).
-Yonik
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Re: GData, updateable IndexSearcher
Posted by Yonik Seeley <ys...@gmail.com>.
On 4/26/06, jason rutherglen <ja...@yahoo.com> wrote:
> As this would be used with Solr I wonder if there would be a good way to also update the Solr caches.
Other than re-executing queries that generated the results? Probably not.
The nice thing about knowing exactly when the view of an index changes
(and having it only change once in a while), is that you can do very
aggressive caching.
If you want an IndexSearcher who's view of the index changed every
second (for example), I don't think Solr's type of caching would be
useful at all (or even possible, if you have big caches and
autowarming).
-Yonik
Re: GData, updateable IndexSearcher
Posted by Yonik Seeley <ys...@gmail.com>.
On 4/27/06, Robert Engels <re...@ix.netcom.com> wrote:
> I thought each segment maintained its own list of deleted documents
Right.
> (since segments are WRITE ONCE
Yes, but deletions are the exception to that rule. Once written,
segment files never change, except for the file that tracks deleted
documents for that segment.
Hence, if the segment name is the same, you should be able to count on
everything being unchanged *except* for which documents are deleted.
-Yonik
http://incubator.apache.org/solr Solr, the open-source Lucene search server
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Re: GData, updateable IndexSearcher
Posted by Chuck Williams <ch...@manawiz.com>.
If I'm following this correctly, it omits a related issue which is the
need to periodically close and reopen the IndexWriter in order to flush
its internal RAMDirectory, and similarly for the IndexReader used for
delete. Is there any good solution to avoid these as well?
My app has an IndexManager class that is somewhat like the built-in
IndexModifier, except that it contains all index operations:
IndexSearcher, IndexReader for search, IndexReader for delete,
IndexWriter and IndexUpdater (my own class that covers many more use
cases than the contributed patch of the same name). IndexManager tracks
whether or not the index has changed, and spawns a RefreshThread that
periodically commits all updates (close and reopen whatever is open of
IndexWriter, IndexReader for delete and IndexUpdater) and then reopens
the searcher and search reader.
The notion of incrementally updating the searcher is great. Is there
any way to also avoid closing the writer and delete reader?
Chuck
Doug Cutting wrote on 04/26/2006 10:44 AM:
> jason rutherglen wrote:
>> I was thinking you implied that you knew of someone who had
>> customized their own, but it was a closed source solution. And if so
>> then you would know how that project faired.
>
> I don't recall the details, but I know folks have discussed this
> previously, and probably even posted patches, but I don't think any of
> the patches was ready to commit.
>
>> Wouldn't there also need to be a hack on the IndexWriter to keep
>> track of new segments?
>
> I think the 'public static IndexReader.reopen(IndexReader old)' method
> I proposed can easily compare the current list of segments for the
> directory of old to those that old already has open, and determine
> which can be reused and which new segments must be opened. Deletions
> would be a little tricky to track. If a segment has had deletions,
> then a new SegmentReader could be cloned from the old, sharing
> everything but the deletions, which could be re-read from disk. This
> would invalidate cached filters for segments that had deletions.
>
> You could even try to figure out what documents have been deleted,
> then update filters incrementally. That would be fastest, but more
> complicated.
>
> Doug
>
>> ----- Original Message ----
>> From: Doug Cutting <cu...@apache.org>
>> To: solr-dev@lucene.apache.org
>> Sent: Wednesday, April 26, 2006 11:27:44 AM
>> Subject: Re: GData, updateable IndexSearcher
>>
>> jason rutherglen wrote:
>>
>>> Interesting, does this mean there is a plan for incrementally
>>> updateable IndexSearchers to become part of Lucene?
>>
>>
>> In general, there is no plan for Lucene. If someone implements a
>> generally useful, efficient, feature in a back-compatible, easy to
>> use, manner, and submits it as a patch, then it becomes a part of
>> Lucene. That's the way Lucene changes. Since we don't pay anyone, we
>> can't make plans and assign tasks. So if you're particularly
>> interested in this feature, you might search the archives to find
>> past efforts, or simply try to implement it yourself.
>>
>> I think a good approach would be to create a new IndexSearcher
>> instance based on an existing one, that shares IndexReaders.
>> Similarly, one should be able to create a new IndexReader based on an
>> existing one. This would be a MultiReader that shares many of the
>> same SegmentReaders.
>>
>> Things get a little tricky after this.
>>
>> Lucene caches filters based on the IndexReader. So filters would
>> need to be re-created. Ideally these could be incrementally
>> re-created, but that might be difficult. What might be simpler would
>> be to use a MultiSearcher constructed with an IndexSearcher per
>> SegmentReader, avoiding the use of MultiReader. Then the caches
>> would still work. This would require making a few things public that
>> are not at present. Perhaps adding a 'MultiReader.getSubReaders()'
>> method, combined with an 'static IndexReader.reopen(IndexReader)'
>> method. The latter would return a new MultiReader that shared
>> SegmentReaders with the old version. Then one could use
>> getSubReaders() on the new multi reader to extract the current set to
>> use when constructing a MultiSearcher.
>>
>> Another tricky bit is figuring out when to close readers.
>>
>> Does this make sense? This discussion should probably move to the
>> lucene-dev list.
>>
>>
>>> Are there any negatives to updateable IndexSearchers?
>>
>>
>> Not if implemented well!
>>
>> Doug
>>
>>
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-dev-help@lucene.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Re: GData, updateable IndexSearcher
Posted by jason rutherglen <ja...@yahoo.com>.
> I think the 'public static IndexReader.reopen(IndexReader old)' method I proposed can easily compare the current list of segments for the directory of old to those that old already has open, and determine which can be reused and which new segments must be opened.
This makes sense. Could you describe how the new segments would be known, or where in the code they can be loaded? Where in the design would there need to be synchronization blocks?
----- Original Message ----
From: Doug Cutting <cu...@apache.org>
To: java-dev@lucene.apache.org
Sent: Wednesday, April 26, 2006 1:44:08 PM
Subject: Re: GData, updateable IndexSearcher
jason rutherglen wrote:
> I was thinking you implied that you knew of someone who had customized their own, but it was a closed source solution. And if so then you would know how that project faired.
I don't recall the details, but I know folks have discussed this
previously, and probably even posted patches, but I don't think any of
the patches was ready to commit.
> Wouldn't there also need to be a hack on the IndexWriter to keep track of new segments?
I think the 'public static IndexReader.reopen(IndexReader old)' method I
proposed can easily compare the current list of segments for the
directory of old to those that old already has open, and determine which
can be reused and which new segments must be opened. Deletions would be
a little tricky to track. If a segment has had deletions, then a new
SegmentReader could be cloned from the old, sharing everything but the
deletions, which could be re-read from disk. This would invalidate
cached filters for segments that had deletions.
You could even try to figure out what documents have been deleted, then
update filters incrementally. That would be fastest, but more complicated.
Doug
> ----- Original Message ----
> From: Doug Cutting <cu...@apache.org>
> To: solr-dev@lucene.apache.org
> Sent: Wednesday, April 26, 2006 11:27:44 AM
> Subject: Re: GData, updateable IndexSearcher
>
> jason rutherglen wrote:
>
>>Interesting, does this mean there is a plan for incrementally updateable IndexSearchers to become part of Lucene?
>
>
> In general, there is no plan for Lucene. If someone implements a
> generally useful, efficient, feature in a back-compatible, easy to use,
> manner, and submits it as a patch, then it becomes a part of Lucene.
> That's the way Lucene changes. Since we don't pay anyone, we can't make
> plans and assign tasks. So if you're particularly interested in this
> feature, you might search the archives to find past efforts, or simply
> try to implement it yourself.
>
> I think a good approach would be to create a new IndexSearcher instance
> based on an existing one, that shares IndexReaders. Similarly, one
> should be able to create a new IndexReader based on an existing one.
> This would be a MultiReader that shares many of the same SegmentReaders.
>
> Things get a little tricky after this.
>
> Lucene caches filters based on the IndexReader. So filters would need
> to be re-created. Ideally these could be incrementally re-created, but
> that might be difficult. What might be simpler would be to use a
> MultiSearcher constructed with an IndexSearcher per SegmentReader,
> avoiding the use of MultiReader. Then the caches would still work.
> This would require making a few things public that are not at present.
> Perhaps adding a 'MultiReader.getSubReaders()' method, combined with an
> 'static IndexReader.reopen(IndexReader)' method. The latter would
> return a new MultiReader that shared SegmentReaders with the old
> version. Then one could use getSubReaders() on the new multi reader to
> extract the current set to use when constructing a MultiSearcher.
>
> Another tricky bit is figuring out when to close readers.
>
> Does this make sense? This discussion should probably move to the
> lucene-dev list.
>
>
>>Are there any negatives to updateable IndexSearchers?
>
>
> Not if implemented well!
>
> Doug
>
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
RE: GData, updateable IndexSearcher
Posted by Robert Engels <re...@ix.netcom.com>.
Doug can you please elaborate on this.
I thought each segment maintained its own list of deleted documents (since
segments are WRITE ONCE, and when that segment is merged or optimized it
would "go away" anyway, as the deleted documents are removed.
In my reopen() implementation, I check to see if the existing segment name
is the same as an already open segment, and then just use the existing
SegmentInfo object (since it should still have reference to its deleted
documents).
For example,
Index has 3 (1-3) segments. A new document is written that causes a segment
to be created (4). A reopen would retain the SegmentInfo for 1-3, and create
a new one for 4.
It would be no different if segment 2 had deleted documents when the
creation of segment 4 occurs, segment 2 is not modified in this case.
If adding the new document, which creates a new segment, caused a merge,
segment 2 would be rewritten (and the deletions processed), so the segment
name for 2 would no longer be valid anyway, and the SegmentInfo would not
reused.
I've had this code in production for almost 2 years and have not seen any
problems - trying to get a handle on the possibility that our code may be
"unstable".
-----Original Message-----
From: Doug Cutting [mailto:cutting@apache.org]
Sent: Wednesday, April 26, 2006 3:44 PM
To: java-dev@lucene.apache.org
Subject: Re: GData, updateable IndexSearcher
jason rutherglen wrote:
> I was thinking you implied that you knew of someone who had customized
their own, but it was a closed source solution. And if so then you would
know how that project faired.
I don't recall the details, but I know folks have discussed this
previously, and probably even posted patches, but I don't think any of
the patches was ready to commit.
> Wouldn't there also need to be a hack on the IndexWriter to keep track of
new segments?
I think the 'public static IndexReader.reopen(IndexReader old)' method I
proposed can easily compare the current list of segments for the
directory of old to those that old already has open, and determine which
can be reused and which new segments must be opened. Deletions would be
a little tricky to track. If a segment has had deletions, then a new
SegmentReader could be cloned from the old, sharing everything but the
deletions, which could be re-read from disk. This would invalidate
cached filters for segments that had deletions.
You could even try to figure out what documents have been deleted, then
update filters incrementally. That would be fastest, but more complicated.
Doug
> ----- Original Message ----
> From: Doug Cutting <cu...@apache.org>
> To: solr-dev@lucene.apache.org
> Sent: Wednesday, April 26, 2006 11:27:44 AM
> Subject: Re: GData, updateable IndexSearcher
>
> jason rutherglen wrote:
>
>>Interesting, does this mean there is a plan for incrementally updateable
IndexSearchers to become part of Lucene?
>
>
> In general, there is no plan for Lucene. If someone implements a
> generally useful, efficient, feature in a back-compatible, easy to use,
> manner, and submits it as a patch, then it becomes a part of Lucene.
> That's the way Lucene changes. Since we don't pay anyone, we can't make
> plans and assign tasks. So if you're particularly interested in this
> feature, you might search the archives to find past efforts, or simply
> try to implement it yourself.
>
> I think a good approach would be to create a new IndexSearcher instance
> based on an existing one, that shares IndexReaders. Similarly, one
> should be able to create a new IndexReader based on an existing one.
> This would be a MultiReader that shares many of the same SegmentReaders.
>
> Things get a little tricky after this.
>
> Lucene caches filters based on the IndexReader. So filters would need
> to be re-created. Ideally these could be incrementally re-created, but
> that might be difficult. What might be simpler would be to use a
> MultiSearcher constructed with an IndexSearcher per SegmentReader,
> avoiding the use of MultiReader. Then the caches would still work.
> This would require making a few things public that are not at present.
> Perhaps adding a 'MultiReader.getSubReaders()' method, combined with an
> 'static IndexReader.reopen(IndexReader)' method. The latter would
> return a new MultiReader that shared SegmentReaders with the old
> version. Then one could use getSubReaders() on the new multi reader to
> extract the current set to use when constructing a MultiSearcher.
>
> Another tricky bit is figuring out when to close readers.
>
> Does this make sense? This discussion should probably move to the
> lucene-dev list.
>
>
>>Are there any negatives to updateable IndexSearchers?
>
>
> Not if implemented well!
>
> Doug
>
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Re: GData, updateable IndexSearcher
Posted by Doug Cutting <cu...@apache.org>.
jason rutherglen wrote:
> I was thinking you implied that you knew of someone who had customized their own, but it was a closed source solution. And if so then you would know how that project faired.
I don't recall the details, but I know folks have discussed this
previously, and probably even posted patches, but I don't think any of
the patches was ready to commit.
> Wouldn't there also need to be a hack on the IndexWriter to keep track of new segments?
I think the 'public static IndexReader.reopen(IndexReader old)' method I
proposed can easily compare the current list of segments for the
directory of old to those that old already has open, and determine which
can be reused and which new segments must be opened. Deletions would be
a little tricky to track. If a segment has had deletions, then a new
SegmentReader could be cloned from the old, sharing everything but the
deletions, which could be re-read from disk. This would invalidate
cached filters for segments that had deletions.
You could even try to figure out what documents have been deleted, then
update filters incrementally. That would be fastest, but more complicated.
Doug
> ----- Original Message ----
> From: Doug Cutting <cu...@apache.org>
> To: solr-dev@lucene.apache.org
> Sent: Wednesday, April 26, 2006 11:27:44 AM
> Subject: Re: GData, updateable IndexSearcher
>
> jason rutherglen wrote:
>
>>Interesting, does this mean there is a plan for incrementally updateable IndexSearchers to become part of Lucene?
>
>
> In general, there is no plan for Lucene. If someone implements a
> generally useful, efficient, feature in a back-compatible, easy to use,
> manner, and submits it as a patch, then it becomes a part of Lucene.
> That's the way Lucene changes. Since we don't pay anyone, we can't make
> plans and assign tasks. So if you're particularly interested in this
> feature, you might search the archives to find past efforts, or simply
> try to implement it yourself.
>
> I think a good approach would be to create a new IndexSearcher instance
> based on an existing one, that shares IndexReaders. Similarly, one
> should be able to create a new IndexReader based on an existing one.
> This would be a MultiReader that shares many of the same SegmentReaders.
>
> Things get a little tricky after this.
>
> Lucene caches filters based on the IndexReader. So filters would need
> to be re-created. Ideally these could be incrementally re-created, but
> that might be difficult. What might be simpler would be to use a
> MultiSearcher constructed with an IndexSearcher per SegmentReader,
> avoiding the use of MultiReader. Then the caches would still work.
> This would require making a few things public that are not at present.
> Perhaps adding a 'MultiReader.getSubReaders()' method, combined with an
> 'static IndexReader.reopen(IndexReader)' method. The latter would
> return a new MultiReader that shared SegmentReaders with the old
> version. Then one could use getSubReaders() on the new multi reader to
> extract the current set to use when constructing a MultiSearcher.
>
> Another tricky bit is figuring out when to close readers.
>
> Does this make sense? This discussion should probably move to the
> lucene-dev list.
>
>
>>Are there any negatives to updateable IndexSearchers?
>
>
> Not if implemented well!
>
> Doug
>
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Re: GData, updateable IndexSearcher
Posted by jason rutherglen <ja...@yahoo.com>.
This originated on the Solr mailing list.
> That's the way Lucene changes.
I was thinking you implied that you knew of someone who had customized their own, but it was a closed source solution. And if so then you would know how that project faired.
I definitely sounds like an interesting project, it will take me several days to digest the design you described. As this would be used with Solr I wonder if there would be a good way to also update the Solr caches. Wouldn't there also need to be a hack on the IndexWriter to keep track of new segments?
----- Original Message ----
From: Doug Cutting <cu...@apache.org>
To: solr-dev@lucene.apache.org
Sent: Wednesday, April 26, 2006 11:27:44 AM
Subject: Re: GData, updateable IndexSearcher
jason rutherglen wrote:
> Interesting, does this mean there is a plan for incrementally updateable IndexSearchers to become part of Lucene?
In general, there is no plan for Lucene. If someone implements a
generally useful, efficient, feature in a back-compatible, easy to use,
manner, and submits it as a patch, then it becomes a part of Lucene.
That's the way Lucene changes. Since we don't pay anyone, we can't make
plans and assign tasks. So if you're particularly interested in this
feature, you might search the archives to find past efforts, or simply
try to implement it yourself.
I think a good approach would be to create a new IndexSearcher instance
based on an existing one, that shares IndexReaders. Similarly, one
should be able to create a new IndexReader based on an existing one.
This would be a MultiReader that shares many of the same SegmentReaders.
Things get a little tricky after this.
Lucene caches filters based on the IndexReader. So filters would need
to be re-created. Ideally these could be incrementally re-created, but
that might be difficult. What might be simpler would be to use a
MultiSearcher constructed with an IndexSearcher per SegmentReader,
avoiding the use of MultiReader. Then the caches would still work.
This would require making a few things public that are not at present.
Perhaps adding a 'MultiReader.getSubReaders()' method, combined with an
'static IndexReader.reopen(IndexReader)' method. The latter would
return a new MultiReader that shared SegmentReaders with the old
version. Then one could use getSubReaders() on the new multi reader to
extract the current set to use when constructing a MultiSearcher.
Another tricky bit is figuring out when to close readers.
Does this make sense? This discussion should probably move to the
lucene-dev list.
> Are there any negatives to updateable IndexSearchers?
Not if implemented well!
Doug
Re: GData, updateable IndexSearcher
Posted by jason rutherglen <ja...@yahoo.com>.
This originated on the Solr mailing list.
> That's the way Lucene changes.
I was thinking you implied that you knew of someone who had customized their own, but it was a closed source solution. And if so then you would know how that project faired.
I definitely sounds like an interesting project, it will take me several days to digest the design you described. As this would be used with Solr I wonder if there would be a good way to also update the Solr caches. Wouldn't there also need to be a hack on the IndexWriter to keep track of new segments?
----- Original Message ----
From: Doug Cutting <cu...@apache.org>
To: solr-dev@lucene.apache.org
Sent: Wednesday, April 26, 2006 11:27:44 AM
Subject: Re: GData, updateable IndexSearcher
jason rutherglen wrote:
> Interesting, does this mean there is a plan for incrementally updateable IndexSearchers to become part of Lucene?
In general, there is no plan for Lucene. If someone implements a
generally useful, efficient, feature in a back-compatible, easy to use,
manner, and submits it as a patch, then it becomes a part of Lucene.
That's the way Lucene changes. Since we don't pay anyone, we can't make
plans and assign tasks. So if you're particularly interested in this
feature, you might search the archives to find past efforts, or simply
try to implement it yourself.
I think a good approach would be to create a new IndexSearcher instance
based on an existing one, that shares IndexReaders. Similarly, one
should be able to create a new IndexReader based on an existing one.
This would be a MultiReader that shares many of the same SegmentReaders.
Things get a little tricky after this.
Lucene caches filters based on the IndexReader. So filters would need
to be re-created. Ideally these could be incrementally re-created, but
that might be difficult. What might be simpler would be to use a
MultiSearcher constructed with an IndexSearcher per SegmentReader,
avoiding the use of MultiReader. Then the caches would still work.
This would require making a few things public that are not at present.
Perhaps adding a 'MultiReader.getSubReaders()' method, combined with an
'static IndexReader.reopen(IndexReader)' method. The latter would
return a new MultiReader that shared SegmentReaders with the old
version. Then one could use getSubReaders() on the new multi reader to
extract the current set to use when constructing a MultiSearcher.
Another tricky bit is figuring out when to close readers.
Does this make sense? This discussion should probably move to the
lucene-dev list.
> Are there any negatives to updateable IndexSearchers?
Not if implemented well!
Doug
Re: GData, updateable IndexSearcher
Posted by Doug Cutting <cu...@apache.org>.
jason rutherglen wrote:
> Interesting, does this mean there is a plan for incrementally updateable IndexSearchers to become part of Lucene?
In general, there is no plan for Lucene. If someone implements a
generally useful, efficient, feature in a back-compatible, easy to use,
manner, and submits it as a patch, then it becomes a part of Lucene.
That's the way Lucene changes. Since we don't pay anyone, we can't make
plans and assign tasks. So if you're particularly interested in this
feature, you might search the archives to find past efforts, or simply
try to implement it yourself.
I think a good approach would be to create a new IndexSearcher instance
based on an existing one, that shares IndexReaders. Similarly, one
should be able to create a new IndexReader based on an existing one.
This would be a MultiReader that shares many of the same SegmentReaders.
Things get a little tricky after this.
Lucene caches filters based on the IndexReader. So filters would need
to be re-created. Ideally these could be incrementally re-created, but
that might be difficult. What might be simpler would be to use a
MultiSearcher constructed with an IndexSearcher per SegmentReader,
avoiding the use of MultiReader. Then the caches would still work.
This would require making a few things public that are not at present.
Perhaps adding a 'MultiReader.getSubReaders()' method, combined with an
'static IndexReader.reopen(IndexReader)' method. The latter would
return a new MultiReader that shared SegmentReaders with the old
version. Then one could use getSubReaders() on the new multi reader to
extract the current set to use when constructing a MultiSearcher.
Another tricky bit is figuring out when to close readers.
Does this make sense? This discussion should probably move to the
lucene-dev list.
> Are there any negatives to updateable IndexSearchers?
Not if implemented well!
Doug