Re: Solr/Lucene MoreLikeThis with RangeQuery

[Chris Hostetter <ho...@fucit.org>]

: Solr/Lucene. I am in a situation where I think that I can improve the
: quality of the LikeThis-documents significantly by restricting the
: MoreLikeThis-query to documents where one field has its term in a
: specified range. That is, I would like to add a RangeQuery to the
: default MoreLikeThis query.
	[...]
: I would like to also add a range restriction as,
: 
:   rq = new ConstantScoreRangeQuery("time",startTimeString,endTimeString,true,true);
:   query.add(rq, BooleanClause.Occur.MUST);
: 
: This is all made in
:   contrib/queries/src/java/org/apache/lucene/search/similar/MoreLikeThis.java
: 
: However, this does not work at all when running from Solr (no MLT
: suggestions are returned). I suspect that the problem is that the

If i'm understanding you correctly, you modified the MoreLikeThis class to 
include your new clause (using those two lines above) correct?

If you aren't getting any results, i suspect it may be an issues of term 
value encoding ... is your "time" field a Solr DateField?  what is the 
value of startTimeString and endTimeString? ... if you replace all of the 
MLT Query logic so that it's *just* the ConstantScoreRangeQuery do you get 
any results?

: does not perform a standard query, but a getDocList:
: 
:   results.docList = searcher.getDocList(mltQuery, filters, null,
: start, rows, flags);
: 
: and that this type of query does not handle a RangeQuery. Is this
: correct, or what is the problem with adding a RangeQuery? Should it be

a RangeQuery will work just fine.  but in general the type of problem you 
are trying to solve could be more generally dealt with if the MLT code had 
a way to let people specify "filter" queries (like the existing "fq" 
param) to be applied tothe MLT logic -- that way they wouldn't contribute 
to the relevancy ... it seems like it would be pretty easy to add a 
"mlt.fq" param for this purpose if you wanted to appraoch the problem 
thatway as a more generic path -- but i'm not too familiar with the MLT 
code to say for certain waht would be required, and I know the code is 
probably more complicated then it should be with the MoreLikeThisHandler 
and the MoreLikeThisComponent (i think there's a MoreLikeThisHelper that 
they share or something)



-Hoss

Re: Solr/Lucene MoreLikeThis with RangeQuery

[Clas Rydergren <ni...@gmail.com>]

Hi,

Thanks for the help.

>If i'm understanding you correctly, you modified the MoreLikeThis class to
>include your new clause (using those two lines above) correct?

Yes.

The time field is a "long" and so is the range variables, so the
problem should not be related to that. If I construct the query by
adding a ConstantScoreRangeQuery, nothing more, no results are
returned. But I have not tried to add it to the filter part of the
mlt-handler; I suspect that this would solve the problem.

However, after trying more alternatives, I think that adding a

&fq=time:[1230922259744+TO+1231440659744]

to the mlt-url-request seems to actually add a time filter to the
constructed MLT-query:

Query:(+kategori:nyheter titel:moderbolaget^2.0 artikel:moderbolaget
titel:pininfarin^2.0 artikel:pininfarin titel:bilbygg^1.9725448
artikel:bilbygg^0.9862724 titel:huvudäg^1.9257689
artikel:huvudäg^0.9628844 titel:uddevall^1.9054867
artikel:uddevall^0.95274335 titel:majoritet^1.71646
artikel:majoritet^0.85823 titel:volvo^1.6696839
artikel:volvo^0.83484197 titel:italiensk^1.5226858
artikel:italiensk^0.7613429)~5

So, a mlt.fq does not seems to be necessary to implement since the fq
filter seems to be passed to the mlt-query.

To use a long for the time field rather than a Field.Date is probably
bad, but it seems to work at least for testing. So, I think that my
problem is solved. Thanks!

/Clas






On Fri, Jan 9, 2009 at 2:40 AM, Chris Hostetter
<ho...@fucit.org> wrote:
>
> : Solr/Lucene. I am in a situation where I think that I can improve the
> : quality of the LikeThis-documents significantly by restricting the
> : MoreLikeThis-query to documents where one field has its term in a
> : specified range. That is, I would like to add a RangeQuery to the
> : default MoreLikeThis query.
>        [...]
> : I would like to also add a range restriction as,
> :
> :   rq = new ConstantScoreRangeQuery("time",startTimeString,endTimeString,true,true);
> :   query.add(rq, BooleanClause.Occur.MUST);
> :
> : This is all made in
> :   contrib/queries/src/java/org/apache/lucene/search/similar/MoreLikeThis.java
> :
> : However, this does not work at all when running from Solr (no MLT
> : suggestions are returned). I suspect that the problem is that the
>
> If i'm understanding you correctly, you modified the MoreLikeThis class to
> include your new clause (using those two lines above) correct?
>
> If you aren't getting any results, i suspect it may be an issues of term
> value encoding ... is your "time" field a Solr DateField?  what is the
> value of startTimeString and endTimeString? ... if you replace all of the
> MLT Query logic so that it's *just* the ConstantScoreRangeQuery do you get
> any results?
>
> : does not perform a standard query, but a getDocList:
> :
> :   results.docList = searcher.getDocList(mltQuery, filters, null,
> : start, rows, flags);
> :
> : and that this type of query does not handle a RangeQuery. Is this
> : correct, or what is the problem with adding a RangeQuery? Should it be
>
> a RangeQuery will work just fine.  but in general the type of problem you
> are trying to solve could be more generally dealt with if the MLT code had
> a way to let people specify "filter" queries (like the existing "fq"
> param) to be applied tothe MLT logic -- that way they wouldn't contribute
> to the relevancy ... it seems like it would be pretty easy to add a
> "mlt.fq" param for this purpose if you wanted to appraoch the problem
> thatway as a more generic path -- but i'm not too familiar with the MLT
> code to say for certain waht would be required, and I know the code is
> probably more complicated then it should be with the MoreLikeThisHandler
> and the MoreLikeThisComponent (i think there's a MoreLikeThisHelper that
> they share or something)
>
>
>
> -Hoss
>
>