You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@lucene.apache.org by Daniel Noll <da...@nuix.com> on 2007/04/20 03:27:01 UTC
IndexReader method semantics
Hi all.
I'm considering making a kind of IndexReader where each time terms() is
called it might return a different sequence even though the reader
hasn't been reopened.
Would that kind of thing conceivably cause problems anywhere else in the
framework? Which is to ask, do other parts of the framework implement
their own caching of the TermEnum, or is all the caching implemented in
the reader and TermEnum implementations?
Daniel
--
Daniel Noll
Nuix Pty Ltd
Suite 79, 89 Jones St, Ultimo NSW 2007, Australia Ph: +61 2 9280 0699
Web: http://nuix.com/ Fax: +61 2 9212 6902
This message is intended only for the named recipient. If you are not
the intended recipient you are notified that disclosing, copying,
distributing or taking any action in reliance on the contents of this
message or attachment is strictly prohibited.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: IndexReader method semantics
Posted by Daniel Noll <da...@nuix.com>.
Chris Hostetter wrote:
> : Basically I'm thinking of writing a different kind of IndexReader which
> : uses a database to return fake terms for things like tags. The idea
> : would be that it can be slotted in alongside a real index via
> : ParallelReader in order to provide the fast-changing part. So it would
> : be possible to implement real queries on a database rather than the
>
> how will you deal with docid renumbering as segments merge? (or can you
> assume no deletions in your use cases?)
Yeah, in our use case the documents are never deleted (when your data is
evidence it's not a good idea to delete any. ;-))
Our database already keys on Lucene IDs, a decision made ages ago for
performance reasons; calling hits.doc(i) is much slower than hits.id(i)
plus we're trying to build filters based of database records where the
indexes are huge so it wasn't very practical to keep reading documents.
Daniel
--
Daniel Noll
Nuix Pty Ltd
Suite 79, 89 Jones St, Ultimo NSW 2007, Australia Ph: +61 2 9280 0699
Web: http://nuix.com/ Fax: +61 2 9212 6902
This message is intended only for the named recipient. If you are not
the intended recipient you are notified that disclosing, copying,
distributing or taking any action in reliance on the contents of this
message or attachment is strictly prohibited.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: IndexReader method semantics
Posted by Chris Hostetter <ho...@fucit.org>.
: Basically I'm thinking of writing a different kind of IndexReader which
: uses a database to return fake terms for things like tags. The idea
: would be that it can be slotted in alongside a real index via
: ParallelReader in order to provide the fast-changing part. So it would
: be possible to implement real queries on a database rather than the
how will you deal with docid renumbering as segments merge? (or can you
assume no deletions in your use cases?)
-Hoss
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: IndexReader method semantics
Posted by Daniel Noll <da...@nuix.com>.
Chris Hostetter wrote:
> : I'm considering making a kind of IndexReader where each time terms() is
> : called it might return a different sequence even though the reader
> : hasn't been reopened.
>
> okay *this* got my attention ... what are you working on?
Basically I'm thinking of writing a different kind of IndexReader which
uses a database to return fake terms for things like tags. The idea
would be that it can be slotted in alongside a real index via
ParallelReader in order to provide the fast-changing part. So it would
be possible to implement real queries on a database rather than the
approach at present which is to use filters and ConstantScoreQuery.
It's going to be highly experimental anyway.
> : Would that kind of thing conceivably cause problems anywhere else in the
> : framework? Which is to ask, do other parts of the framework implement
> : their own caching of the TermEnum, or is all the caching implemented in
> : the reader and TermEnum implementations?
>
> the only thing like this i can think of is some TermDocs freq caching in
> TermScorer.
That sounds like a good thing since it means the tags changing *during*
the query might not upset things (this was my other concern about trying
to do something crazy like this.)
Daniel
--
Daniel Noll
Nuix Pty Ltd
Suite 79, 89 Jones St, Ultimo NSW 2007, Australia Ph: +61 2 9280 0699
Web: http://nuix.com/ Fax: +61 2 9212 6902
This message is intended only for the named recipient. If you are not
the intended recipient you are notified that disclosing, copying,
distributing or taking any action in reliance on the contents of this
message or attachment is strictly prohibited.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: IndexReader method semantics
Posted by Chris Hostetter <ho...@fucit.org>.
: I'm considering making a kind of IndexReader where each time terms() is
: called it might return a different sequence even though the reader
: hasn't been reopened.
okay *this* got my attention ... what are you working on?
: Would that kind of thing conceivably cause problems anywhere else in the
: framework? Which is to ask, do other parts of the framework implement
: their own caching of the TermEnum, or is all the caching implemented in
: the reader and TermEnum implementations?
the only thing like this i can think of is some TermDocs freq caching in
TermScorer.
-Hoss
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org