You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@lucene.apache.org by "Nader, John P" <jo...@cengage.com> on 2010/09/10 18:23:51 UTC
Filters do not work with MultiSearcher?
We are attempting to perform a filtered search on two indices joined by a MultiSearcher. Unfortunately, it appears there is an issue in the lucene code that is causing the filter to be simply reused at the starting ordinal for each individual index instead of being augmented by the starting document identifier. We are hoping there is an alternate API that will allow us to perform a filtered search on multiple indices.
For example, we have two indices with three documents each, and a filter containing only doc ID 1. When we perform a filtered search on a MultiSearcher that joins these two indices, we get two documents back (1, 4), where we were expecting only the one. This is because the MultiSearcher, instead of starting at doc ID 3 for the second index, is interpreting the filter individually for each index.
We are using Lucene 3.0.2. The API we see this behavior with is MultiSearcher.search(Query, Filter, nDocs) with a MatchAllDocsQuery and the filter code pasted below:
public class OpenBitSetFilter extends Filter {
private OpenBitSet openBitSet;
public OpenBitSetFilter(OpenBitSet openBitSet) {
this.openBitSet = openBitSet;
}
public DocIdSet getDocIdSet(IndexReader indexReader) throws IOException {
return openBitSet;
}
}
RE: Filters do not work with MultiSearcher?
Posted by "Nader, John P" <jo...@cengage.com>.
Thanks. Now I understand.
We changed my implementation to hold multiple bit sets, one per index reader. Then we return the correct bit set based upon which reader is being passed in.
The results are exactly what we need. This is our first use of the MultiSearcher (as if that wasn't obvious) and our assumption was that a MultiReader was being passed in.
Thank you for your quick response and useful insights.
-John
-----Original Message-----
From: yseeley@gmail.com [mailto:yseeley@gmail.com] On Behalf Of Yonik Seeley
Sent: Friday, September 10, 2010 12:32 PM
To: java-user@lucene.apache.org
Subject: Re: Filters do not work with MultiSearcher?
This is "working as designed".
Note this method:
public DocIdSet getDocIdSet(IndexReader indexReader) throws IOException {
return openBitSet;
}
You must pay attention to the IndexReader passed - and the DocIdSet
returned must always be based on that reader (and the first document
of every reader is always 0). So returning the same DocIdSet each
time is not valid and will result in errors.
-Yonik
http://lucenerevolution.org Lucene/Solr Conference, Boston Oct 7-8
On Fri, Sep 10, 2010 at 12:23 PM, Nader, John P <jo...@cengage.com> wrote:
> We are attempting to perform a filtered search on two indices joined by a MultiSearcher. Unfortunately, it appears there is an issue in the lucene code that is causing the filter to be simply reused at the starting ordinal for each individual index instead of being augmented by the starting document identifier. We are hoping there is an alternate API that will allow us to perform a filtered search on multiple indices.
>
> For example, we have two indices with three documents each, and a filter containing only doc ID 1. When we perform a filtered search on a MultiSearcher that joins these two indices, we get two documents back (1, 4), where we were expecting only the one. This is because the MultiSearcher, instead of starting at doc ID 3 for the second index, is interpreting the filter individually for each index.
>
> We are using Lucene 3.0.2. The API we see this behavior with is MultiSearcher.search(Query, Filter, nDocs) with a MatchAllDocsQuery and the filter code pasted below:
>
> public class OpenBitSetFilter extends Filter {
>
> private OpenBitSet openBitSet;
>
>
> public OpenBitSetFilter(OpenBitSet openBitSet) {
> this.openBitSet = openBitSet;
> }
>
> public DocIdSet getDocIdSet(IndexReader indexReader) throws IOException {
> return openBitSet;
> }
>
> }
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Filters do not work with MultiSearcher?
Posted by Yonik Seeley <yo...@lucidimagination.com>.
This is "working as designed".
Note this method:
public DocIdSet getDocIdSet(IndexReader indexReader) throws IOException {
return openBitSet;
}
You must pay attention to the IndexReader passed - and the DocIdSet
returned must always be based on that reader (and the first document
of every reader is always 0). So returning the same DocIdSet each
time is not valid and will result in errors.
-Yonik
http://lucenerevolution.org Lucene/Solr Conference, Boston Oct 7-8
On Fri, Sep 10, 2010 at 12:23 PM, Nader, John P <jo...@cengage.com> wrote:
> We are attempting to perform a filtered search on two indices joined by a MultiSearcher. Unfortunately, it appears there is an issue in the lucene code that is causing the filter to be simply reused at the starting ordinal for each individual index instead of being augmented by the starting document identifier. We are hoping there is an alternate API that will allow us to perform a filtered search on multiple indices.
>
> For example, we have two indices with three documents each, and a filter containing only doc ID 1. When we perform a filtered search on a MultiSearcher that joins these two indices, we get two documents back (1, 4), where we were expecting only the one. This is because the MultiSearcher, instead of starting at doc ID 3 for the second index, is interpreting the filter individually for each index.
>
> We are using Lucene 3.0.2. The API we see this behavior with is MultiSearcher.search(Query, Filter, nDocs) with a MatchAllDocsQuery and the filter code pasted below:
>
> public class OpenBitSetFilter extends Filter {
>
> private OpenBitSet openBitSet;
>
>
> public OpenBitSetFilter(OpenBitSet openBitSet) {
> this.openBitSet = openBitSet;
> }
>
> public DocIdSet getDocIdSet(IndexReader indexReader) throws IOException {
> return openBitSet;
> }
>
> }
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org