You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@lucene.apache.org by Edwin Tang <em...@yahoo.com> on 2004/11/17 01:20:55 UTC
Need help with filtering
Hello,
I have been using DateFilter to limit my search results to a certain date
range. I am now asked to replace this filter with one where my search results
have document IDs greater than a given document ID. This document ID is
assigned during indexing and is a Keyword field.
I've browsed around the FAQs and archives and see that I can either use
QueryFilter or BooleanQuery. I've tried both approaches to limit the document
ID range, but am getting the BooleanQuery.TooManyClauses exception in both
cases. I've also tried bumping max number of clauses via setMaxClauseCount(),
but that number has gotten pretty big.
Is there another approach to this? Or am I setting this up incorrectly? Snippet
of one of my approaches follows:
queryFilter = new QueryFilter(new RangeQuery(new Term("id", sLastSearchedId),
null, false));
docs = searcher.search(parser.parse(sSearchPhrase), queryFilter,
utility.iMaxResults, new Sort(sortFields));
Thanks in advance,
Ed
__________________________________
Do you Yahoo!?
Meet the all-new My Yahoo! - Try it today!
http://my.yahoo.com
---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-user-help@jakarta.apache.org
Re: Need help with filtering
Posted by Nader Henein <ns...@bayt.net>.
Well if the document ID is number (even if it isn't really) you could
use a range query, or just rebuild your index using that specific filed
as a sorted field but if it numeric be aware that if you use integer it
limits how high your numbers can get.
nader
Edwin Tang wrote:
>Hello,
>
>I have been using DateFilter to limit my search results to a certain date
>range. I am now asked to replace this filter with one where my search results
>have document IDs greater than a given document ID. This document ID is
>assigned during indexing and is a Keyword field.
>
>I've browsed around the FAQs and archives and see that I can either use
>QueryFilter or BooleanQuery. I've tried both approaches to limit the document
>ID range, but am getting the BooleanQuery.TooManyClauses exception in both
>cases. I've also tried bumping max number of clauses via setMaxClauseCount(),
>but that number has gotten pretty big.
>
>Is there another approach to this? Or am I setting this up incorrectly? Snippet
>of one of my approaches follows:
>queryFilter = new QueryFilter(new RangeQuery(new Term("id", sLastSearchedId),
>null, false));
>docs = searcher.search(parser.parse(sSearchPhrase), queryFilter,
>utility.iMaxResults, new Sort(sortFields));
>
>Thanks in advance,
>Ed
>
>
>
>__________________________________
>Do you Yahoo!?
>Meet the all-new My Yahoo! - Try it today!
>http://my.yahoo.com
>
>
>
>---------------------------------------------------------------------
>To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
>For additional commands, e-mail: lucene-user-help@jakarta.apache.org
>
>
>
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-user-help@jakarta.apache.org
Re: Need help with filtering
Posted by Edwin Tang <em...@yahoo.com>.
Ah... recoding DateFilter. I will look into this today. Thanks for the help.
Ed
--- Paul Elschot <pa...@xs4all.nl> wrote:
> On Wednesday 17 November 2004 01:20, Edwin Tang wrote:
> > Hello,
> >
> > I have been using DateFilter to limit my search results to a certain date
> > range. I am now asked to replace this filter with one where my search
> results
> > have document IDs greater than a given document ID. This document ID is
> > assigned during indexing and is a Keyword field.
> >
> > I've browsed around the FAQs and archives and see that I can either use
> > QueryFilter or BooleanQuery. I've tried both approaches to limit the
> document
> > ID range, but am getting the BooleanQuery.TooManyClauses exception in both
> > cases. I've also tried bumping max number of clauses via
> setMaxClauseCount(),
> > but that number has gotten pretty big.
> >
> > Is there another approach to this? ...
>
> Recoding DateFilter to a DocumentIdFilter should be straightforward.
>
> The trick is to use only one document enumerator at a time for all
> terms. Document enumerators take buffer space, and that is the
> reason why BooleanQuery has an exception for too many clauses.
>
> Regards,
> Paul
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: lucene-user-help@jakarta.apache.org
>
>
__________________________________
Do you Yahoo!?
The all-new My Yahoo! - Get yours free!
http://my.yahoo.com
---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-user-help@jakarta.apache.org
Re: Need help with filtering
Posted by Edwin Tang <em...@yahoo.com>.
Hello again,
I've modified DateFilter to filter out document IDs as suggested. All seems to
be running well until I tried a specific test case. All my documents have IDs
in the 400,000 range. If I set my lower limit to 5, nothing comes back. After
examining the code, I found the issue to be at the following line:
TermEnum enumerator = reader.terms(new Term(field, start));
Is there a way to retrieve a set of documents with IDs using a Integer
comparison versus a String comparison? If I set "start" to 0, I get everything,
but that's not very efficient.
Thanks in advance,
Ed
--- Paul Elschot <pa...@xs4all.nl> wrote:
> On Wednesday 17 November 2004 01:20, Edwin Tang wrote:
> > Hello,
> >
> > I have been using DateFilter to limit my search results to a certain date
> > range. I am now asked to replace this filter with one where my search
> results
> > have document IDs greater than a given document ID. This document ID is
> > assigned during indexing and is a Keyword field.
> >
> > I've browsed around the FAQs and archives and see that I can either use
> > QueryFilter or BooleanQuery. I've tried both approaches to limit the
> document
> > ID range, but am getting the BooleanQuery.TooManyClauses exception in both
> > cases. I've also tried bumping max number of clauses via
> setMaxClauseCount(),
> > but that number has gotten pretty big.
> >
> > Is there another approach to this? ...
>
> Recoding DateFilter to a DocumentIdFilter should be straightforward.
>
> The trick is to use only one document enumerator at a time for all
> terms. Document enumerators take buffer space, and that is the
> reason why BooleanQuery has an exception for too many clauses.
>
> Regards,
> Paul
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: lucene-user-help@jakarta.apache.org
>
>
__________________________________________________
Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam protection around
http://mail.yahoo.com
---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-user-help@jakarta.apache.org
Re: Need help with filtering
Posted by Paul Elschot <pa...@xs4all.nl>.
On Wednesday 17 November 2004 01:20, Edwin Tang wrote:
> Hello,
>
> I have been using DateFilter to limit my search results to a certain date
> range. I am now asked to replace this filter with one where my search
results
> have document IDs greater than a given document ID. This document ID is
> assigned during indexing and is a Keyword field.
>
> I've browsed around the FAQs and archives and see that I can either use
> QueryFilter or BooleanQuery. I've tried both approaches to limit the
document
> ID range, but am getting the BooleanQuery.TooManyClauses exception in both
> cases. I've also tried bumping max number of clauses via
setMaxClauseCount(),
> but that number has gotten pretty big.
>
> Is there another approach to this? ...
Recoding DateFilter to a DocumentIdFilter should be straightforward.
The trick is to use only one document enumerator at a time for all
terms. Document enumerators take buffer space, and that is the
reason why BooleanQuery has an exception for too many clauses.
Regards,
Paul
---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-user-help@jakarta.apache.org