You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Jason Rennie <jr...@gmail.com> on 2008/08/04 18:50:05 UTC
diversity in results
Is there any option in solr to encourage diversity in the results? Our solr
index has millions of products, many of which are quite similar to each
other. Even something simple like max 50% text overlap in successive
results would be valuable. Does something like this exist in solr or are
there any plans to add it?
Thanks,
Jason
--
Jason Rennie
Head of Machine Learning Technologies, StyleFeeder
http://www.stylefeeder.com/
Samantha's blog & pictures: http://samanthalyrarennie.blogspot.com/
Re: diversity in results
Posted by Jason Rennie <jr...@gmail.com>.
Thanks for the pointers. Looks interesting, at least as a starting point
for something more sophisticated.
Cheers,
Jason
On Mon, Aug 4, 2008 at 4:38 PM, Grant Ingersoll <gs...@apache.org> wrote:
> See https://issues.apache.org/jira/browse/SOLR-236 and
> http://wiki.apache.org/solr/FieldCollapsing, but I gather it has been
> languishing. I also don't think it will do anything as extensive as the
> text similarity question you are asking (50% overlap) but I have not tried
> it.
>
> -Grant
Re: diversity in results
Posted by Grant Ingersoll <gs...@apache.org>.
See https://issues.apache.org/jira/browse/SOLR-236 and http://wiki.apache.org/solr/FieldCollapsing
, but I gather it has been languishing. I also don't think it will do
anything as extensive as the text similarity question you are asking
(50% overlap) but I have not tried it.
-Grant
On Aug 4, 2008, at 12:50 PM, Jason Rennie wrote:
> Is there any option in solr to encourage diversity in the results?
> Our solr
> index has millions of products, many of which are quite similar to
> each
> other. Even something simple like max 50% text overlap in successive
> results would be valuable. Does something like this exist in solr
> or are
> there any plans to add it?
>
> Thanks,
>
> Jason
>
> --
> Jason Rennie
> Head of Machine Learning Technologies, StyleFeeder
> http://www.stylefeeder.com/
> Samantha's blog & pictures: http://samanthalyrarennie.blogspot.com/
Re: diversity in results
Posted by Grant Ingersoll <gs...@apache.org>.
On Aug 4, 2008, at 6:17 PM, Jason Rennie wrote:
> Does the MLT handler simply select a few high tfidf terms from the
> doc and
> use them as a query? Sounds like a useful tool. Do you know
> anything about
> relevant performance issues? I noticed that the Solr MoreLikeThis
> wiki page
> recommends turning on TermVectors for corresponding fields. Can
> lucene not
> easily return term counts for a document with the standard indexing
> b/c it's
> term-based (i.e. "inverted").
Correct.
> Does TermVectors=true cause solr/lucene to
> store an additional doc-based index?
Yes, that is correct
Re: diversity in results
Posted by Jason Rennie <jr...@gmail.com>.
Does the MLT handler simply select a few high tfidf terms from the doc and
use them as a query? Sounds like a useful tool. Do you know anything about
relevant performance issues? I noticed that the Solr MoreLikeThis wiki page
recommends turning on TermVectors for corresponding fields. Can lucene not
easily return term counts for a document with the standard indexing b/c it's
term-based (i.e. "inverted"). Does TermVectors=true cause solr/lucene to
store an additional doc-based index?
Thanks,
Jason
On Mon, Aug 4, 2008 at 5:06 PM, Brian Whitman <br...@variogr.am>wrote:
> not out of the box, but I would use the mlt handler on the first result and
> remove all the ones that appear in both the MLT and query response.
>
> B
>
>
--
Jason Rennie
Head of Machine Learning Technologies, StyleFeeder
http://www.stylefeeder.com/
Samantha's blog & pictures: http://samanthalyrarennie.blogspot.com/
Re: diversity in results
Posted by Brian Whitman <br...@variogr.am>.
On Aug 4, 2008, at 12:50 PM, Jason Rennie wrote:
> Is there any option in solr to encourage diversity in the results?
> Our solr
> index has millions of products, many of which are quite similar to
> each
> other. Even something simple like max 50% text overlap in successive
> results would be valuable. Does something like this exist in solr
> or are
> there any plans to add it?
>
not out of the box, but I would use the mlt handler on the first
result and remove all the ones that appear in both the MLT and query
response.
B