Re: Filters - at what stage are they applied?

[Michael McCandless <lu...@mikemccandless.com>]

Prior to 2.4, the search runs first and then the filter.  Ie, search  
does all the work to produce docIDs that match it, and then per docID  
the filter is checked.

As of 2.4, they actually sort of play leap-frog, document by  
document.  First, was ask the filter for its first matching docID,  
then we ask the query to skip to that docID, which either accepts that  
docID or moves to the next docID that the query accepts.  Then we ask  
the filter to skip to that docID, etc.

It's actually not clear that the approach in 2.4 is best, eg here:

     https://issues.apache.org/jira/browse/LUCENE-1536

I found that if the filter can support random access (ie, it's backed  
by a bit array) then we can get far better performance in many cases  
by using a random access API very early (for each Term) instead of the  
iterator/skipTo API used in 2.4.  EG this is exactly how deleted  
documents (which really is just a filter) are applied.

Also this issue:

     https://issues.apache.org/jira/browse/LUCENE-1345

we are exploring simply making filter another (AND'd) clause in a  
BooleanQuery.

So, exactly how Lucene combines constraints from filters & queries is  
still very much in flux...

Mike

Joel Halbert wrote:

> Hi,
>
> By way of clarification, when a filter is used with a search query, is
> the filter applied only to documents that matched the search query  
> or is
> it applied to all documents in the index before the query is executed?
>
> Rgs,
>
> Joel
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org