You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Bruno René Santos <br...@gmail.com> on 2013/08/02 18:13:46 UTC
Random sort with groups
Hello,
We are trying to make a random sort with a query where we are grouping
results by a field. On one example we always have one of the results on the
top of the query result. We sort first by the score and then by a random
field.
We think that is because this group has 214 items inside and the second
group only has 6 items. That way the random sort on the first group would
get the first group like 99% of times on the first place of the results.
Can someone confirm this behaviour?
Regards
--
Bruno René Santos
Lisboa - Portugal
Re: Random sort with groups
Posted by Erick Erickson <er...@gmail.com>.
Why can't you shuffle in the app? Once you have the groups returned, _how_
you display them is up to you. And you can sort the groups based on fields.
Best
Erick
On Fri, Aug 2, 2013 at 7:15 PM, Bruno René Santos <br...@gmail.com>wrote:
> Hi Erick,
>
> Yes we have lots of documents with the same score. We turned off IDF and on
> some fields also the norms and frequencies. And so on a business point of
> view we need to shuffle those results in order to give the opportunity to
> any document with the highest score to be in first.
>
> The problem here is that when we group results the same group always stays
> on top (the group with the highest number of items but with the same score
> than the second, first has 214 second has 2 items) and we need it to
> shuffle between those two.
>
> On a different query there is shuffling when the the groups with the
> highest score have a small number of items (1-3).
>
> We need shuffling on the first example also.
>
> Regards
>
>
> On Fri, Aug 2, 2013 at 9:37 PM, Erick Erickson <erickerickson@gmail.com
> >wrote:
>
> > bq: We sort first by the score and then by a random field.
> >
> > This is highly unlikely to produce anything useful. Your
> > secondary sort (random field) will only be used if your
> > primary sort (score) matches exactly, which is usually
> > quite rare. Unless you're somehow doing a constant-score
> > query (e.g. *:*)....
> >
> > So either you have a very unusual corpus that is scoring
> > lots and lots of docs exactly the same or there's something
> > I'm not understanding.
> >
> > Best
> > Erick
> >
> >
> > On Fri, Aug 2, 2013 at 12:13 PM, Bruno René Santos <brunorene@gmail.com
> > >wrote:
> >
> > > Hello,
> > >
> > > We are trying to make a random sort with a query where we are grouping
> > > results by a field. On one example we always have one of the results on
> > the
> > > top of the query result. We sort first by the score and then by a
> random
> > > field.
> > >
> > > We think that is because this group has 214 items inside and the second
> > > group only has 6 items. That way the random sort on the first group
> would
> > > get the first group like 99% of times on the first place of the
> results.
> > >
> > > Can someone confirm this behaviour?
> > >
> > > Regards
> > >
> > > --
> > > Bruno René Santos
> > > Lisboa - Portugal
> > >
> >
>
>
>
> --
> Bruno René Santos
> Lisboa - Portugal
>
Re: Random sort with groups
Posted by Bruno René Santos <br...@gmail.com>.
Hi Erick,
Yes we have lots of documents with the same score. We turned off IDF and on
some fields also the norms and frequencies. And so on a business point of
view we need to shuffle those results in order to give the opportunity to
any document with the highest score to be in first.
The problem here is that when we group results the same group always stays
on top (the group with the highest number of items but with the same score
than the second, first has 214 second has 2 items) and we need it to
shuffle between those two.
On a different query there is shuffling when the the groups with the
highest score have a small number of items (1-3).
We need shuffling on the first example also.
Regards
On Fri, Aug 2, 2013 at 9:37 PM, Erick Erickson <er...@gmail.com>wrote:
> bq: We sort first by the score and then by a random field.
>
> This is highly unlikely to produce anything useful. Your
> secondary sort (random field) will only be used if your
> primary sort (score) matches exactly, which is usually
> quite rare. Unless you're somehow doing a constant-score
> query (e.g. *:*)....
>
> So either you have a very unusual corpus that is scoring
> lots and lots of docs exactly the same or there's something
> I'm not understanding.
>
> Best
> Erick
>
>
> On Fri, Aug 2, 2013 at 12:13 PM, Bruno René Santos <brunorene@gmail.com
> >wrote:
>
> > Hello,
> >
> > We are trying to make a random sort with a query where we are grouping
> > results by a field. On one example we always have one of the results on
> the
> > top of the query result. We sort first by the score and then by a random
> > field.
> >
> > We think that is because this group has 214 items inside and the second
> > group only has 6 items. That way the random sort on the first group would
> > get the first group like 99% of times on the first place of the results.
> >
> > Can someone confirm this behaviour?
> >
> > Regards
> >
> > --
> > Bruno René Santos
> > Lisboa - Portugal
> >
>
--
Bruno René Santos
Lisboa - Portugal
Re: Random sort with groups
Posted by Erick Erickson <er...@gmail.com>.
bq: We sort first by the score and then by a random field.
This is highly unlikely to produce anything useful. Your
secondary sort (random field) will only be used if your
primary sort (score) matches exactly, which is usually
quite rare. Unless you're somehow doing a constant-score
query (e.g. *:*)....
So either you have a very unusual corpus that is scoring
lots and lots of docs exactly the same or there's something
I'm not understanding.
Best
Erick
On Fri, Aug 2, 2013 at 12:13 PM, Bruno René Santos <br...@gmail.com>wrote:
> Hello,
>
> We are trying to make a random sort with a query where we are grouping
> results by a field. On one example we always have one of the results on the
> top of the query result. We sort first by the score and then by a random
> field.
>
> We think that is because this group has 214 items inside and the second
> group only has 6 items. That way the random sort on the first group would
> get the first group like 99% of times on the first place of the results.
>
> Can someone confirm this behaviour?
>
> Regards
>
> --
> Bruno René Santos
> Lisboa - Portugal
>