You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Peter Keegan <pe...@gmail.com> on 2014/07/23 15:31:20 UTC
Question about ReRankQuery
I'm looking at how 'ReRankQuery' works. If the main query has a Sort
criteria, it is only used to sort the first pass results. The QueryScorer
used in the second pass only reorders the ScoreDocs based on score and
docid, but doesn't use the original Sort fields. If the Sort criteria is
'score desc, myfield asc', I would expect 'myfield' to break score ties
from the second pass after rescoring.
Is this a bug or the intended behavior?
Thanks,
Peter
Re: Question about ReRankQuery
Posted by Joel Bernstein <jo...@gmail.com>.
Blog on the RankQuery API
http://heliosearch.org/solrs-new-rankquery-feature/
Joel Bernstein
Search Engineer at Heliosearch
On Wed, Jul 23, 2014 at 3:27 PM, Joel Bernstein <jo...@gmail.com> wrote:
> The ReRankingQParserPlugin uses the Lucene QueryRescorer, which only uses
> the score from the re-rank query when re-ranking the top N documents.
>
> The ReRanklingQParserPlugin is built as a RankQuery plugin so you can swap
> in your own implementation. Patches are also welcome for the existing
> implementation.
>
> Joel Bernstein
> Search Engineer at Heliosearch
>
>
> On Wed, Jul 23, 2014 at 11:37 AM, Peter Keegan <pe...@gmail.com>
> wrote:
>
>> See http://heliosearch.org/solrs-new-re-ranking-feature/
>>
>>
>> On Wed, Jul 23, 2014 at 11:27 AM, Erick Erickson <erickerickson@gmail.com
>> >
>> wrote:
>>
>> > I'm having a little trouble understanding the use-case here. Why use
>> > re-ranking?
>> > Isn't this just combining the original query with the second query with
>> an
>> > AND
>> > and using the original sort?
>> >
>> > At the end, you have your original list in it's original order, with
>> > (potentially) some
>> > documents removed that don't satisfy the secondary query.
>> >
>> > Or I'm missing the boat entirely.
>> >
>> > Best,
>> > Erick
>> >
>> >
>> > On Wed, Jul 23, 2014 at 6:31 AM, Peter Keegan <pe...@gmail.com>
>> > wrote:
>> >
>> > > I'm looking at how 'ReRankQuery' works. If the main query has a Sort
>> > > criteria, it is only used to sort the first pass results. The
>> QueryScorer
>> > > used in the second pass only reorders the ScoreDocs based on score and
>> > > docid, but doesn't use the original Sort fields. If the Sort criteria
>> is
>> > > 'score desc, myfield asc', I would expect 'myfield' to break score
>> ties
>> > > from the second pass after rescoring.
>> > >
>> > > Is this a bug or the intended behavior?
>> > >
>> > > Thanks,
>> > > Peter
>> > >
>> >
>>
>
>
Re: Question about ReRankQuery
Posted by Joel Bernstein <jo...@gmail.com>.
I like the FieldSortedHitQueue idea. If you want to work up a patch for
that, it would be great.
Joel Bernstein
Search Engineer at Heliosearch
On Wed, Jul 23, 2014 at 5:17 PM, Peter Keegan <pe...@gmail.com>
wrote:
> > The ReRankingQParserPlugin uses the Lucene QueryRescorer, which only uses
> the score from the re-rank query when re-ranking the top N documents.
>
> Understood, but if the re-rank scores produce new ties, wouldn't you want
> to resort them with the FieldSortedHitQueue?
>
> Anyway, I was looking to reimplement the ScaleScoreQParser PostFilter
> plugin with RankQuery, and would need to implement the behavior of the
> DelegateCollector there for handling multiple sort fields.
>
> Peter
>
> On Wednesday, July 23, 2014, Joel Bernstein <jo...@gmail.com> wrote:
>
> > The ReRankingQParserPlugin uses the Lucene QueryRescorer, which only uses
> > the score from the re-rank query when re-ranking the top N documents.
> >
> > The ReRanklingQParserPlugin is built as a RankQuery plugin so you can
> swap
> > in your own implementation. Patches are also welcome for the existing
> > implementation.
> >
> > Joel Bernstein
> > Search Engineer at Heliosearch
> >
> >
> > On Wed, Jul 23, 2014 at 11:37 AM, Peter Keegan <peterlkeegan@gmail.com
> > <javascript:;>>
> > wrote:
> >
> > > See http://heliosearch.org/solrs-new-re-ranking-feature/
> > >
> > >
> > > On Wed, Jul 23, 2014 at 11:27 AM, Erick Erickson <
> > erickerickson@gmail.com <javascript:;>>
> > > wrote:
> > >
> > > > I'm having a little trouble understanding the use-case here. Why use
> > > > re-ranking?
> > > > Isn't this just combining the original query with the second query
> with
> > > an
> > > > AND
> > > > and using the original sort?
> > > >
> > > > At the end, you have your original list in it's original order, with
> > > > (potentially) some
> > > > documents removed that don't satisfy the secondary query.
> > > >
> > > > Or I'm missing the boat entirely.
> > > >
> > > > Best,
> > > > Erick
> > > >
> > > >
> > > > On Wed, Jul 23, 2014 at 6:31 AM, Peter Keegan <
> peterlkeegan@gmail.com
> > <javascript:;>>
> > > > wrote:
> > > >
> > > > > I'm looking at how 'ReRankQuery' works. If the main query has a
> Sort
> > > > > criteria, it is only used to sort the first pass results. The
> > > QueryScorer
> > > > > used in the second pass only reorders the ScoreDocs based on score
> > and
> > > > > docid, but doesn't use the original Sort fields. If the Sort
> criteria
> > > is
> > > > > 'score desc, myfield asc', I would expect 'myfield' to break score
> > ties
> > > > > from the second pass after rescoring.
> > > > >
> > > > > Is this a bug or the intended behavior?
> > > > >
> > > > > Thanks,
> > > > > Peter
> > > > >
> > > >
> > >
> >
>
Re: Question about ReRankQuery
Posted by Peter Keegan <pe...@gmail.com>.
> The ReRankingQParserPlugin uses the Lucene QueryRescorer, which only uses
the score from the re-rank query when re-ranking the top N documents.
Understood, but if the re-rank scores produce new ties, wouldn't you want
to resort them with the FieldSortedHitQueue?
Anyway, I was looking to reimplement the ScaleScoreQParser PostFilter
plugin with RankQuery, and would need to implement the behavior of the
DelegateCollector there for handling multiple sort fields.
Peter
On Wednesday, July 23, 2014, Joel Bernstein <jo...@gmail.com> wrote:
> The ReRankingQParserPlugin uses the Lucene QueryRescorer, which only uses
> the score from the re-rank query when re-ranking the top N documents.
>
> The ReRanklingQParserPlugin is built as a RankQuery plugin so you can swap
> in your own implementation. Patches are also welcome for the existing
> implementation.
>
> Joel Bernstein
> Search Engineer at Heliosearch
>
>
> On Wed, Jul 23, 2014 at 11:37 AM, Peter Keegan <peterlkeegan@gmail.com
> <javascript:;>>
> wrote:
>
> > See http://heliosearch.org/solrs-new-re-ranking-feature/
> >
> >
> > On Wed, Jul 23, 2014 at 11:27 AM, Erick Erickson <
> erickerickson@gmail.com <javascript:;>>
> > wrote:
> >
> > > I'm having a little trouble understanding the use-case here. Why use
> > > re-ranking?
> > > Isn't this just combining the original query with the second query with
> > an
> > > AND
> > > and using the original sort?
> > >
> > > At the end, you have your original list in it's original order, with
> > > (potentially) some
> > > documents removed that don't satisfy the secondary query.
> > >
> > > Or I'm missing the boat entirely.
> > >
> > > Best,
> > > Erick
> > >
> > >
> > > On Wed, Jul 23, 2014 at 6:31 AM, Peter Keegan <peterlkeegan@gmail.com
> <javascript:;>>
> > > wrote:
> > >
> > > > I'm looking at how 'ReRankQuery' works. If the main query has a Sort
> > > > criteria, it is only used to sort the first pass results. The
> > QueryScorer
> > > > used in the second pass only reorders the ScoreDocs based on score
> and
> > > > docid, but doesn't use the original Sort fields. If the Sort criteria
> > is
> > > > 'score desc, myfield asc', I would expect 'myfield' to break score
> ties
> > > > from the second pass after rescoring.
> > > >
> > > > Is this a bug or the intended behavior?
> > > >
> > > > Thanks,
> > > > Peter
> > > >
> > >
> >
>
Re: Question about ReRankQuery
Posted by Joel Bernstein <jo...@gmail.com>.
The ReRankingQParserPlugin uses the Lucene QueryRescorer, which only uses
the score from the re-rank query when re-ranking the top N documents.
The ReRanklingQParserPlugin is built as a RankQuery plugin so you can swap
in your own implementation. Patches are also welcome for the existing
implementation.
Joel Bernstein
Search Engineer at Heliosearch
On Wed, Jul 23, 2014 at 11:37 AM, Peter Keegan <pe...@gmail.com>
wrote:
> See http://heliosearch.org/solrs-new-re-ranking-feature/
>
>
> On Wed, Jul 23, 2014 at 11:27 AM, Erick Erickson <er...@gmail.com>
> wrote:
>
> > I'm having a little trouble understanding the use-case here. Why use
> > re-ranking?
> > Isn't this just combining the original query with the second query with
> an
> > AND
> > and using the original sort?
> >
> > At the end, you have your original list in it's original order, with
> > (potentially) some
> > documents removed that don't satisfy the secondary query.
> >
> > Or I'm missing the boat entirely.
> >
> > Best,
> > Erick
> >
> >
> > On Wed, Jul 23, 2014 at 6:31 AM, Peter Keegan <pe...@gmail.com>
> > wrote:
> >
> > > I'm looking at how 'ReRankQuery' works. If the main query has a Sort
> > > criteria, it is only used to sort the first pass results. The
> QueryScorer
> > > used in the second pass only reorders the ScoreDocs based on score and
> > > docid, but doesn't use the original Sort fields. If the Sort criteria
> is
> > > 'score desc, myfield asc', I would expect 'myfield' to break score ties
> > > from the second pass after rescoring.
> > >
> > > Is this a bug or the intended behavior?
> > >
> > > Thanks,
> > > Peter
> > >
> >
>
Re: Question about ReRankQuery
Posted by Peter Keegan <pe...@gmail.com>.
See http://heliosearch.org/solrs-new-re-ranking-feature/
On Wed, Jul 23, 2014 at 11:27 AM, Erick Erickson <er...@gmail.com>
wrote:
> I'm having a little trouble understanding the use-case here. Why use
> re-ranking?
> Isn't this just combining the original query with the second query with an
> AND
> and using the original sort?
>
> At the end, you have your original list in it's original order, with
> (potentially) some
> documents removed that don't satisfy the secondary query.
>
> Or I'm missing the boat entirely.
>
> Best,
> Erick
>
>
> On Wed, Jul 23, 2014 at 6:31 AM, Peter Keegan <pe...@gmail.com>
> wrote:
>
> > I'm looking at how 'ReRankQuery' works. If the main query has a Sort
> > criteria, it is only used to sort the first pass results. The QueryScorer
> > used in the second pass only reorders the ScoreDocs based on score and
> > docid, but doesn't use the original Sort fields. If the Sort criteria is
> > 'score desc, myfield asc', I would expect 'myfield' to break score ties
> > from the second pass after rescoring.
> >
> > Is this a bug or the intended behavior?
> >
> > Thanks,
> > Peter
> >
>
Re: Question about ReRankQuery
Posted by Erick Erickson <er...@gmail.com>.
I'm having a little trouble understanding the use-case here. Why use
re-ranking?
Isn't this just combining the original query with the second query with an
AND
and using the original sort?
At the end, you have your original list in it's original order, with
(potentially) some
documents removed that don't satisfy the secondary query.
Or I'm missing the boat entirely.
Best,
Erick
On Wed, Jul 23, 2014 at 6:31 AM, Peter Keegan <pe...@gmail.com>
wrote:
> I'm looking at how 'ReRankQuery' works. If the main query has a Sort
> criteria, it is only used to sort the first pass results. The QueryScorer
> used in the second pass only reorders the ScoreDocs based on score and
> docid, but doesn't use the original Sort fields. If the Sort criteria is
> 'score desc, myfield asc', I would expect 'myfield' to break score ties
> from the second pass after rescoring.
>
> Is this a bug or the intended behavior?
>
> Thanks,
> Peter
>