Using 2nd Index to constraing Search

[Mike Upshon <Mi...@aftermail.com>]

Hi

Just starting to evaluate Lucene and hope somone can answer this
question.

I am looking at using Lucene to index a very large databse. There is a
documents table and a few other tables that define what users can view
what documents. My question is, is it posible to have an index of the
full text contents of the documents and another index that contains the
document id's and the user id's and then use the 2nd index to qualify
the full text search over the document table. The reason I want to do
this is to reduce the numbers of documents that the full text query will
run.

E.g. if there are 50 million documents in the documents table and 20,000
usesr if the full text search was able to be restricted by the user the
full text search over the documents would be run over a substantialy
smaller set.

Thanks


Mike

---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-user-help@jakarta.apache.org

Re: Using 2nd Index to constraing Search

[Paul Elschot <pa...@xs4all.nl>]

On Friday 27 August 2004 20:10, Mike Upshon wrote:
> Hi
>
> Just starting to evaluate Lucene and hope somone can answer this
> question.
>
> I am looking at using Lucene to index a very large databse. There is a
> documents table and a few other tables that define what users can view
> what documents. My question is, is it posible to have an index of the

The normal way of doing that is to:
- make a list of all doc id's for the user.
- from this list construct a Filter for use in the full text index.
Sort the doc id's, use an IndexReader on the full text index, construct
a Term for each doc id, walk the termDocs() for the Term, and set
a bit in the filter to allow the document number for the doc id.
- keep this filter to restrict the searches for the user by
IndexSearcher.search(Query,Filter)
- rebuild the filter when the doc id's for the user change, or when
the full text index changes (a document deletion followed
by an optimize or an add can change any other document's number).

Hmm, this is getting to be a FAQ.

> full text contents of the documents and another index that contains the
> document id's and the user id's and then use the 2nd index to qualify
> the full text search over the document table. The reason I want to do
> this is to reduce the numbers of documents that the full text query will
> run.

Regards,
Paul Elschot


---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-user-help@jakarta.apache.org