Finding docs which contain at least x of the queryterms

[Barbara Krausz <bk...@web.de>]

>
Hi,

Consider a Query with e.g. 4 terms (t1,t2,t3,t4). I want to retrieve all 
documents which contain at least e.g. 3 of the queryterms. How can I 
implement this?
The first idea is to use BooleanQueries such as
(t1 and t2 and t3 and t4) or (t1 and t2 and t3) or(t1 and t2 and t4) or 
(t1 and t3 and t4).....

But the perfomance is not very good when I have 20 queryterms and I want 
to retrieve all docs which contain at least 15 of the terms.
Can I modify the skipto-algorithm in ConjunctionScorer in order to 
achieve this?

Thanks
Barbara

PS: Has anybody written a Statistics-class which says how many term and 
different terms are  in  the index.  And perhaps computes the mean 
length of the documents in the index with the standard deviation?

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org

utf-8 & Lucene 1.4.3 & Solaris &windows

[gaudinat <ar...@healthonnet.org>]

Hi,

I haven't got no utf-8 index when I use Lucene with Solaris while my 
characters are OK under windows. My indexing program is the same and it 
uses lucene 1.4.3.

Is someone have an Idea to help me?

Regards,

Arnaud.

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org

Re: Finding docs which contain at least x of the queryterms

[Paul Elschot <pa...@xs4all.nl>]

On Wednesday 25 May 2005 13:00, Barbara Krausz wrote:
> 
> >
> Hi,
> 
> Consider a Query with e.g. 4 terms (t1,t2,t3,t4). I want to retrieve all 
> documents which contain at least e.g. 3 of the queryterms. How can I 
> implement this?
> The first idea is to use BooleanQueries such as
> (t1 and t2 and t3 and t4) or (t1 and t2 and t3) or(t1 and t2 and t4) or 
> (t1 and t3 and t4).....
> 
> But the perfomance is not very good when I have 20 queryterms and I want 
> to retrieve all docs which contain at least 15 of the terms.
> Can I modify the skipto-algorithm in ConjunctionScorer in order to 
> achieve this?

I don't think so, but in case you can describe a method to do this,
please share it.

In the svn trunk there is a DisjunctionSumScorer that has the
minimum number of subquery matchers as a constructor parameter:

http://svn.apache.org/viewcvs.cgi/lucene/java/trunk/src/java/org/apache/lucene/search/

It has this javadoc comment in the advanceAfterCurrent method:
* @todo Investigate whether it is possible to use skipTo() when
* the minimum number of matchers is bigger than one, ie. try and use the
* character of ConjunctionScorer for the minimum number of matchers.

The constructor parameter is not used (even in the trunk), so you'll have
to write the code to use it yourself. I'd recommend to start from the trunk
and extend BooleanQuery for this.

Regards,
Paul Elschot


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org

Re: Finding docs which contain at least x of the queryterms

[Erik Hatcher <er...@ehatchersolutions.com>]

On May 25, 2005, at 7:00 AM, Barbara Krausz wrote:
> Hi,
>
> Consider a Query with e.g. 4 terms (t1,t2,t3,t4). I want to  
> retrieve all documents which contain at least e.g. 3 of the  
> queryterms. How can I implement this?
> The first idea is to use BooleanQueries such as
> (t1 and t2 and t3 and t4) or (t1 and t2 and t3) or(t1 and t2 and  
> t4) or (t1 and t3 and t4).....
>
> But the perfomance is not very good when I have 20 queryterms and I  
> want to retrieve all docs which contain at least 15 of the terms.
> Can I modify the skipto-algorithm in ConjunctionScorer in order to  
> achieve this?
>
> Thanks
> Barbara
>
> PS: Has anybody written a Statistics-class which says how many term  
> and different terms are  in  the index.  And perhaps computes the  
> mean length of the documents in the index with the standard deviation?

There is an interesting trick you can play with a custom Similarity  
class on a BooleanQuery - check out the coord method.  This could be  
used to ensure that an "overlap" of 3 is mandatory for a match, for  
example.

I'll leave the details of this as an exercise to the reader for the  
moment.

     Erik


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org