You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@lucene.apache.org by pdaures <pa...@gmail.com> on 2010/02/22 09:54:02 UTC
Boost Problem (again), need example !
Hi,
I know that there are many topics about scoring issues, but I didn't find an
answer in the topics.
This is the problem :
Imagine I'm a teacher, and I have to index all the results, comments and
score about students.
Student :
String name (eg : John Smith)
String comments : (eg: John is a good student, but he needs to be more self
confident bla bla bla)
float score (eg : 98)
I have to index all the students and when I use the search class, I want to
get first the best students. So, if John Smith is a better student than John
Mickael, when I search "John" I want to have John Smith BEFORE John Mickeal.
To do that, I'm using BooleanQuery to search in name and comment fields.
First, I thought I could use the function Document.setBoost(float boost)
while indexing student, with boost = Student.score. But the result was not
what I was expected, it didn't work correctly.
Then I thought I could use a FunctionQuery to search :
FunctionQuery functionQuery = new FunctionQuery(new
ReverseOrdFieldSource("score"));
But the result was still incorrect.
I don't know what I'm doing wrong. Could you help me to find a solution ?
Thank you :)
--
View this message in context: http://old.nabble.com/Boost-Problem-%28again%29%2C-need-example-%21-tp27684388p27684388.html
Sent from the Lucene - Java Users mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Boost Problem (again), need example !
Posted by Erick Erickson <er...@gmail.com>.
I still don't understand why a simple sort as suggested by Ian wouldn't
work.
It'd be a lot more reliable than fiddling with doc scores if you want a
strict
ordering on a particular field (make sure it's untokenized though).
Erick
On Mon, Feb 22, 2010 at 8:19 AM, pdaures <pa...@gmail.com> wrote:
>
> It WORKS !
>
> Thank you so much, I spent a lot of time trying to do that, thank you again
> !
>
>
> Uwe Schindler wrote:
> >
> > The simple fix for that is to wrap the subQuery using: new
> > ConstantScoreQuery(new QueryWrapperFilter(query)) - after that its score
> > is constant and the ValueSource only scores.
> >
> > I recommend to use NumericField for indexing this boost (no storing
> > needed, only indexing, precisionStep=Integer.MAX_VALUE). Else (if using
> > standard Field) the boost field does not need to be "stored", it must be
> > indexed as NOT_ANALYZED.
> >
> > -----
> > Uwe Schindler
> > H.-H.-Meier-Allee 63, D-28213 Bremen
> > http://www.thetaphi.de
> > eMail: uwe@thetaphi.de
> >
> >
> >> -----Original Message-----
> >> From: Ian Lea [mailto:ian.lea@gmail.com]
> >> Sent: Monday, February 22, 2010 12:26 PM
> >> To: java-user@lucene.apache.org
> >> Subject: Re: Boost Problem (again), need example !
> >>
> >> boostField needs to be indexed to be used in the FieldScoreQuery.
> >>
> >> Are you now using one of the the latest releases that Uwe mentioned,
> >> with fixes for CustomScoreQuery?
> >>
> >> And unless you provide your own implementation of
> >> CustomScoreQuery.customScore() I think that you are still not
> >> guaranteed to get what you want since the default implementation is to
> >> calculate the score as subQueryScore * valSrcScore.
> >>
> >>
> >> --
> >> Ian.
> >>
> >>
> >> On Mon, Feb 22, 2010 at 11:00 AM, pdaures <pa...@gmail.com>
> >> wrote:
> >> >
> >> > HI !
> >> > Thank you for your help.
> >> > I think I don't use CustomScoreQuery correctly when I do a "search".
> >> >
> >> > BooleanQuery combinedQuery = new BooleanQuery();
> >> > combinedQuery.add(textQuery, Occur.MUST);
> >> > combinedQuery.add(titleQuery, Occur.MUST);
> >> >
> >> > CustomScoreQuery customQuery = new CustomScoreQuery(combinedQuery,new
> >> > FieldScoreQuery(BOOST_FIELD,Type.INT));
> >> >
> >> > indexSearcher.search(..., customQuery, ....).
> >> >
> >> > in order to index the BOOST_FIELD, I do that :
> >> > Field boostField = new Field(BOOST_FIELD, Integer.toString(boost),
> >> > Field.Store.YES, Field.Index.ANALYZED.NO);
> >> >
> >> >
> >> > Is that correct ?
> >> > Thank you
> >> >
> >> >
> >> >
> >> >
> >> > Uwe Schindler wrote:
> >> >>
> >> >> It's CustomScoreQuery in 2.9 and 3.0.
> >> >>
> >> >> Please wait for 2.9.2 and 3.0.1 for an important API change in this
> >> >> experimental query type to work correct with the new per-segment-
> >> search!
> >> >> You can test the release artifacts of both new versions here:
> >> >> http://people.apache.org/~uschindler/staging-area/lucene-292-301-
> >> take2-rev912433/
> >> >>
> >> >> With e.g. ValueSourceQuery you can score your documents using a
> >> separate
> >> >> numeric field from your documents (it uses FieldCache).
> >> >>
> >> >> Uwe
> >> >>
> >> >> -----
> >> >> Uwe Schindler
> >> >> H.-H.-Meier-Allee 63, D-28213 Bremen
> >> >> http://www.thetaphi.de
> >> >> eMail: uwe@thetaphi.de
> >> >>
> >> >>> -----Original Message-----
> >> >>> From: Ian Lea [mailto:ian.lea@gmail.com]
> >> >>> Sent: Monday, February 22, 2010 10:33 AM
> >> >>> To: java-user@lucene.apache.org
> >> >>> Subject: Re: Boost Problem (again), need example !
> >> >>>
> >> >>> Can't you simply sort by descending score (your score, not
> >> lucene's)?
> >> >>> Seems to me that would give you what you are asking for.
> >> >>>
> >> >>> The setBoost() method is unlikely to work consistently because it
> >> only
> >> >>> infuences the score rather than setting it. If your John Mickeal
> >> doc
> >> >>> happens to have a higher lucene score, because of the normal
> >> >>> idf/tf/etc stuff, then the setBoost() with a higher value for John
> >> >>> Smith may well not be enough to force John Smith to the top.
> >> >>>
> >> >>> I don't know enough about function queries to help you much there
> >> but
> >> >>> FieldScoreQuery might work. I can't see any sign of class
> >> >>> FunctionQuery in the 3.0.0 core package so am not clear what that
> >> is.
> >> >>>
> >> >>>
> >> >>> --
> >> >>> Ian.
> >> >>>
> >> >>>
> >> >>>
> >> >>> On Mon, Feb 22, 2010 at 8:54 AM, pdaures <pa...@gmail.com>
> >> >>> wrote:
> >> >>> >
> >> >>> > Hi,
> >> >>> > I know that there are many topics about scoring issues, but I
> >> didn't
> >> >>> find an
> >> >>> > answer in the topics.
> >> >>> > This is the problem :
> >> >>> > Imagine I'm a teacher, and I have to index all the results,
> >> comments
> >> >>> and
> >> >>> > score about students.
> >> >>> >
> >> >>> > Student :
> >> >>> > String name (eg : John Smith)
> >> >>> > String comments : (eg: John is a good student, but he needs to be
> >> >>> more self
> >> >>> > confident bla bla bla)
> >> >>> > float score (eg : 98)
> >> >>> >
> >> >>> > I have to index all the students and when I use the search class,
> >> I
> >> >>> want to
> >> >>> > get first the best students. So, if John Smith is a better
> >> student
> >> >>> than John
> >> >>> > Mickael, when I search "John" I want to have John Smith BEFORE
> >> John
> >> >>> Mickeal.
> >> >>> >
> >> >>> > To do that, I'm using BooleanQuery to search in name and comment
> >> >>> fields.
> >> >>> >
> >> >>> > First, I thought I could use the function Document.setBoost(float
> >> >>> boost)
> >> >>> > while indexing student, with boost = Student.score. But the
> >> result
> >> >>> was not
> >> >>> > what I was expected, it didn't work correctly.
> >> >>> >
> >> >>> > Then I thought I could use a FunctionQuery to search :
> >> >>> > FunctionQuery functionQuery = new FunctionQuery(new
> >> >>> > ReverseOrdFieldSource("score"));
> >> >>> > But the result was still incorrect.
> >> >>> >
> >> >>> > I don't know what I'm doing wrong. Could you help me to find a
> >> >>> solution ?
> >> >>> > Thank you :)
> >> >>> > --
> >> >>> > View this message in context: http://old.nabble.com/Boost-
> >> Problem-
> >> >>> %28again%29%2C-need-example-%21-tp27684388p27684388.html
> >> >>> > Sent from the Lucene - Java Users mailing list archive at
> >> Nabble.com.
> >> >>> >
> >> >>> >
> >> >>> > -----------------------------------------------------------------
> >> ----
> >> >>> > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> >> >>> > For additional commands, e-mail: java-user-help@lucene.apache.org
> >> >>> >
> >> >>> >
> >> >>>
> >> >>> -------------------------------------------------------------------
> >> --
> >> >>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> >> >>> For additional commands, e-mail: java-user-help@lucene.apache.org
> >> >>
> >> >>
> >> >>
> >> >> --------------------------------------------------------------------
> >> -
> >> >> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> >> >> For additional commands, e-mail: java-user-help@lucene.apache.org
> >> >>
> >> >>
> >> >>
> >> >
> >> > --
> >> > View this message in context: http://old.nabble.com/Boost-Problem-
> >> %28again%29%2C-need-example-%21-tp27684388p27685594.html
> >> > Sent from the Lucene - Java Users mailing list archive at Nabble.com.
> >> >
> >> >
> >> > ---------------------------------------------------------------------
> >> > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> >> > For additional commands, e-mail: java-user-help@lucene.apache.org
> >> >
> >> >
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> >> For additional commands, e-mail: java-user-help@lucene.apache.org
> >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> > For additional commands, e-mail: java-user-help@lucene.apache.org
> >
> >
> >
>
> --
> View this message in context:
> http://old.nabble.com/Boost-Problem-%28again%29%2C-need-example-%21-tp27684388p27687065.html
> Sent from the Lucene - Java Users mailing list archive at Nabble.com.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
RE: Boost Problem (again), need example !
Posted by pdaures <pa...@gmail.com>.
It WORKS !
Thank you so much, I spent a lot of time trying to do that, thank you again
!
Uwe Schindler wrote:
>
> The simple fix for that is to wrap the subQuery using: new
> ConstantScoreQuery(new QueryWrapperFilter(query)) - after that its score
> is constant and the ValueSource only scores.
>
> I recommend to use NumericField for indexing this boost (no storing
> needed, only indexing, precisionStep=Integer.MAX_VALUE). Else (if using
> standard Field) the boost field does not need to be "stored", it must be
> indexed as NOT_ANALYZED.
>
> -----
> Uwe Schindler
> H.-H.-Meier-Allee 63, D-28213 Bremen
> http://www.thetaphi.de
> eMail: uwe@thetaphi.de
>
>
>> -----Original Message-----
>> From: Ian Lea [mailto:ian.lea@gmail.com]
>> Sent: Monday, February 22, 2010 12:26 PM
>> To: java-user@lucene.apache.org
>> Subject: Re: Boost Problem (again), need example !
>>
>> boostField needs to be indexed to be used in the FieldScoreQuery.
>>
>> Are you now using one of the the latest releases that Uwe mentioned,
>> with fixes for CustomScoreQuery?
>>
>> And unless you provide your own implementation of
>> CustomScoreQuery.customScore() I think that you are still not
>> guaranteed to get what you want since the default implementation is to
>> calculate the score as subQueryScore * valSrcScore.
>>
>>
>> --
>> Ian.
>>
>>
>> On Mon, Feb 22, 2010 at 11:00 AM, pdaures <pa...@gmail.com>
>> wrote:
>> >
>> > HI !
>> > Thank you for your help.
>> > I think I don't use CustomScoreQuery correctly when I do a "search".
>> >
>> > BooleanQuery combinedQuery = new BooleanQuery();
>> > combinedQuery.add(textQuery, Occur.MUST);
>> > combinedQuery.add(titleQuery, Occur.MUST);
>> >
>> > CustomScoreQuery customQuery = new CustomScoreQuery(combinedQuery,new
>> > FieldScoreQuery(BOOST_FIELD,Type.INT));
>> >
>> > indexSearcher.search(..., customQuery, ....).
>> >
>> > in order to index the BOOST_FIELD, I do that :
>> > Field boostField = new Field(BOOST_FIELD, Integer.toString(boost),
>> > Field.Store.YES, Field.Index.ANALYZED.NO);
>> >
>> >
>> > Is that correct ?
>> > Thank you
>> >
>> >
>> >
>> >
>> > Uwe Schindler wrote:
>> >>
>> >> It's CustomScoreQuery in 2.9 and 3.0.
>> >>
>> >> Please wait for 2.9.2 and 3.0.1 for an important API change in this
>> >> experimental query type to work correct with the new per-segment-
>> search!
>> >> You can test the release artifacts of both new versions here:
>> >> http://people.apache.org/~uschindler/staging-area/lucene-292-301-
>> take2-rev912433/
>> >>
>> >> With e.g. ValueSourceQuery you can score your documents using a
>> separate
>> >> numeric field from your documents (it uses FieldCache).
>> >>
>> >> Uwe
>> >>
>> >> -----
>> >> Uwe Schindler
>> >> H.-H.-Meier-Allee 63, D-28213 Bremen
>> >> http://www.thetaphi.de
>> >> eMail: uwe@thetaphi.de
>> >>
>> >>> -----Original Message-----
>> >>> From: Ian Lea [mailto:ian.lea@gmail.com]
>> >>> Sent: Monday, February 22, 2010 10:33 AM
>> >>> To: java-user@lucene.apache.org
>> >>> Subject: Re: Boost Problem (again), need example !
>> >>>
>> >>> Can't you simply sort by descending score (your score, not
>> lucene's)?
>> >>> Seems to me that would give you what you are asking for.
>> >>>
>> >>> The setBoost() method is unlikely to work consistently because it
>> only
>> >>> infuences the score rather than setting it. If your John Mickeal
>> doc
>> >>> happens to have a higher lucene score, because of the normal
>> >>> idf/tf/etc stuff, then the setBoost() with a higher value for John
>> >>> Smith may well not be enough to force John Smith to the top.
>> >>>
>> >>> I don't know enough about function queries to help you much there
>> but
>> >>> FieldScoreQuery might work. I can't see any sign of class
>> >>> FunctionQuery in the 3.0.0 core package so am not clear what that
>> is.
>> >>>
>> >>>
>> >>> --
>> >>> Ian.
>> >>>
>> >>>
>> >>>
>> >>> On Mon, Feb 22, 2010 at 8:54 AM, pdaures <pa...@gmail.com>
>> >>> wrote:
>> >>> >
>> >>> > Hi,
>> >>> > I know that there are many topics about scoring issues, but I
>> didn't
>> >>> find an
>> >>> > answer in the topics.
>> >>> > This is the problem :
>> >>> > Imagine I'm a teacher, and I have to index all the results,
>> comments
>> >>> and
>> >>> > score about students.
>> >>> >
>> >>> > Student :
>> >>> > String name (eg : John Smith)
>> >>> > String comments : (eg: John is a good student, but he needs to be
>> >>> more self
>> >>> > confident bla bla bla)
>> >>> > float score (eg : 98)
>> >>> >
>> >>> > I have to index all the students and when I use the search class,
>> I
>> >>> want to
>> >>> > get first the best students. So, if John Smith is a better
>> student
>> >>> than John
>> >>> > Mickael, when I search "John" I want to have John Smith BEFORE
>> John
>> >>> Mickeal.
>> >>> >
>> >>> > To do that, I'm using BooleanQuery to search in name and comment
>> >>> fields.
>> >>> >
>> >>> > First, I thought I could use the function Document.setBoost(float
>> >>> boost)
>> >>> > while indexing student, with boost = Student.score. But the
>> result
>> >>> was not
>> >>> > what I was expected, it didn't work correctly.
>> >>> >
>> >>> > Then I thought I could use a FunctionQuery to search :
>> >>> > FunctionQuery functionQuery = new FunctionQuery(new
>> >>> > ReverseOrdFieldSource("score"));
>> >>> > But the result was still incorrect.
>> >>> >
>> >>> > I don't know what I'm doing wrong. Could you help me to find a
>> >>> solution ?
>> >>> > Thank you :)
>> >>> > --
>> >>> > View this message in context: http://old.nabble.com/Boost-
>> Problem-
>> >>> %28again%29%2C-need-example-%21-tp27684388p27684388.html
>> >>> > Sent from the Lucene - Java Users mailing list archive at
>> Nabble.com.
>> >>> >
>> >>> >
>> >>> > -----------------------------------------------------------------
>> ----
>> >>> > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> >>> > For additional commands, e-mail: java-user-help@lucene.apache.org
>> >>> >
>> >>> >
>> >>>
>> >>> -------------------------------------------------------------------
>> --
>> >>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> >>> For additional commands, e-mail: java-user-help@lucene.apache.org
>> >>
>> >>
>> >>
>> >> --------------------------------------------------------------------
>> -
>> >> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> >> For additional commands, e-mail: java-user-help@lucene.apache.org
>> >>
>> >>
>> >>
>> >
>> > --
>> > View this message in context: http://old.nabble.com/Boost-Problem-
>> %28again%29%2C-need-example-%21-tp27684388p27685594.html
>> > Sent from the Lucene - Java Users mailing list archive at Nabble.com.
>> >
>> >
>> > ---------------------------------------------------------------------
>> > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> > For additional commands, e-mail: java-user-help@lucene.apache.org
>> >
>> >
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
>
--
View this message in context: http://old.nabble.com/Boost-Problem-%28again%29%2C-need-example-%21-tp27684388p27687065.html
Sent from the Lucene - Java Users mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
RE: Boost Problem (again), need example !
Posted by Uwe Schindler <uw...@thetaphi.de>.
The simple fix for that is to wrap the subQuery using: new ConstantScoreQuery(new QueryWrapperFilter(query)) - after that its score is constant and the ValueSource only scores.
I recommend to use NumericField for indexing this boost (no storing needed, only indexing, precisionStep=Integer.MAX_VALUE). Else (if using standard Field) the boost field does not need to be "stored", it must be indexed as NOT_ANALYZED.
-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: uwe@thetaphi.de
> -----Original Message-----
> From: Ian Lea [mailto:ian.lea@gmail.com]
> Sent: Monday, February 22, 2010 12:26 PM
> To: java-user@lucene.apache.org
> Subject: Re: Boost Problem (again), need example !
>
> boostField needs to be indexed to be used in the FieldScoreQuery.
>
> Are you now using one of the the latest releases that Uwe mentioned,
> with fixes for CustomScoreQuery?
>
> And unless you provide your own implementation of
> CustomScoreQuery.customScore() I think that you are still not
> guaranteed to get what you want since the default implementation is to
> calculate the score as subQueryScore * valSrcScore.
>
>
> --
> Ian.
>
>
> On Mon, Feb 22, 2010 at 11:00 AM, pdaures <pa...@gmail.com>
> wrote:
> >
> > HI !
> > Thank you for your help.
> > I think I don't use CustomScoreQuery correctly when I do a "search".
> >
> > BooleanQuery combinedQuery = new BooleanQuery();
> > combinedQuery.add(textQuery, Occur.MUST);
> > combinedQuery.add(titleQuery, Occur.MUST);
> >
> > CustomScoreQuery customQuery = new CustomScoreQuery(combinedQuery,new
> > FieldScoreQuery(BOOST_FIELD,Type.INT));
> >
> > indexSearcher.search(..., customQuery, ....).
> >
> > in order to index the BOOST_FIELD, I do that :
> > Field boostField = new Field(BOOST_FIELD, Integer.toString(boost),
> > Field.Store.YES, Field.Index.ANALYZED.NO);
> >
> >
> > Is that correct ?
> > Thank you
> >
> >
> >
> >
> > Uwe Schindler wrote:
> >>
> >> It's CustomScoreQuery in 2.9 and 3.0.
> >>
> >> Please wait for 2.9.2 and 3.0.1 for an important API change in this
> >> experimental query type to work correct with the new per-segment-
> search!
> >> You can test the release artifacts of both new versions here:
> >> http://people.apache.org/~uschindler/staging-area/lucene-292-301-
> take2-rev912433/
> >>
> >> With e.g. ValueSourceQuery you can score your documents using a
> separate
> >> numeric field from your documents (it uses FieldCache).
> >>
> >> Uwe
> >>
> >> -----
> >> Uwe Schindler
> >> H.-H.-Meier-Allee 63, D-28213 Bremen
> >> http://www.thetaphi.de
> >> eMail: uwe@thetaphi.de
> >>
> >>> -----Original Message-----
> >>> From: Ian Lea [mailto:ian.lea@gmail.com]
> >>> Sent: Monday, February 22, 2010 10:33 AM
> >>> To: java-user@lucene.apache.org
> >>> Subject: Re: Boost Problem (again), need example !
> >>>
> >>> Can't you simply sort by descending score (your score, not
> lucene's)?
> >>> Seems to me that would give you what you are asking for.
> >>>
> >>> The setBoost() method is unlikely to work consistently because it
> only
> >>> infuences the score rather than setting it. If your John Mickeal
> doc
> >>> happens to have a higher lucene score, because of the normal
> >>> idf/tf/etc stuff, then the setBoost() with a higher value for John
> >>> Smith may well not be enough to force John Smith to the top.
> >>>
> >>> I don't know enough about function queries to help you much there
> but
> >>> FieldScoreQuery might work. I can't see any sign of class
> >>> FunctionQuery in the 3.0.0 core package so am not clear what that
> is.
> >>>
> >>>
> >>> --
> >>> Ian.
> >>>
> >>>
> >>>
> >>> On Mon, Feb 22, 2010 at 8:54 AM, pdaures <pa...@gmail.com>
> >>> wrote:
> >>> >
> >>> > Hi,
> >>> > I know that there are many topics about scoring issues, but I
> didn't
> >>> find an
> >>> > answer in the topics.
> >>> > This is the problem :
> >>> > Imagine I'm a teacher, and I have to index all the results,
> comments
> >>> and
> >>> > score about students.
> >>> >
> >>> > Student :
> >>> > String name (eg : John Smith)
> >>> > String comments : (eg: John is a good student, but he needs to be
> >>> more self
> >>> > confident bla bla bla)
> >>> > float score (eg : 98)
> >>> >
> >>> > I have to index all the students and when I use the search class,
> I
> >>> want to
> >>> > get first the best students. So, if John Smith is a better
> student
> >>> than John
> >>> > Mickael, when I search "John" I want to have John Smith BEFORE
> John
> >>> Mickeal.
> >>> >
> >>> > To do that, I'm using BooleanQuery to search in name and comment
> >>> fields.
> >>> >
> >>> > First, I thought I could use the function Document.setBoost(float
> >>> boost)
> >>> > while indexing student, with boost = Student.score. But the
> result
> >>> was not
> >>> > what I was expected, it didn't work correctly.
> >>> >
> >>> > Then I thought I could use a FunctionQuery to search :
> >>> > FunctionQuery functionQuery = new FunctionQuery(new
> >>> > ReverseOrdFieldSource("score"));
> >>> > But the result was still incorrect.
> >>> >
> >>> > I don't know what I'm doing wrong. Could you help me to find a
> >>> solution ?
> >>> > Thank you :)
> >>> > --
> >>> > View this message in context: http://old.nabble.com/Boost-
> Problem-
> >>> %28again%29%2C-need-example-%21-tp27684388p27684388.html
> >>> > Sent from the Lucene - Java Users mailing list archive at
> Nabble.com.
> >>> >
> >>> >
> >>> > -----------------------------------------------------------------
> ----
> >>> > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> >>> > For additional commands, e-mail: java-user-help@lucene.apache.org
> >>> >
> >>> >
> >>>
> >>> -------------------------------------------------------------------
> --
> >>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> >>> For additional commands, e-mail: java-user-help@lucene.apache.org
> >>
> >>
> >>
> >> --------------------------------------------------------------------
> -
> >> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> >> For additional commands, e-mail: java-user-help@lucene.apache.org
> >>
> >>
> >>
> >
> > --
> > View this message in context: http://old.nabble.com/Boost-Problem-
> %28again%29%2C-need-example-%21-tp27684388p27685594.html
> > Sent from the Lucene - Java Users mailing list archive at Nabble.com.
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> > For additional commands, e-mail: java-user-help@lucene.apache.org
> >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Boost Problem (again), need example !
Posted by Ian Lea <ia...@gmail.com>.
boostField needs to be indexed to be used in the FieldScoreQuery.
Are you now using one of the the latest releases that Uwe mentioned,
with fixes for CustomScoreQuery?
And unless you provide your own implementation of
CustomScoreQuery.customScore() I think that you are still not
guaranteed to get what you want since the default implementation is to
calculate the score as subQueryScore * valSrcScore.
--
Ian.
On Mon, Feb 22, 2010 at 11:00 AM, pdaures <pa...@gmail.com> wrote:
>
> HI !
> Thank you for your help.
> I think I don't use CustomScoreQuery correctly when I do a "search".
>
> BooleanQuery combinedQuery = new BooleanQuery();
> combinedQuery.add(textQuery, Occur.MUST);
> combinedQuery.add(titleQuery, Occur.MUST);
>
> CustomScoreQuery customQuery = new CustomScoreQuery(combinedQuery,new
> FieldScoreQuery(BOOST_FIELD,Type.INT));
>
> indexSearcher.search(..., customQuery, ....).
>
> in order to index the BOOST_FIELD, I do that :
> Field boostField = new Field(BOOST_FIELD, Integer.toString(boost),
> Field.Store.YES, Field.Index.ANALYZED.NO);
>
>
> Is that correct ?
> Thank you
>
>
>
>
> Uwe Schindler wrote:
>>
>> It's CustomScoreQuery in 2.9 and 3.0.
>>
>> Please wait for 2.9.2 and 3.0.1 for an important API change in this
>> experimental query type to work correct with the new per-segment-search!
>> You can test the release artifacts of both new versions here:
>> http://people.apache.org/~uschindler/staging-area/lucene-292-301-take2-rev912433/
>>
>> With e.g. ValueSourceQuery you can score your documents using a separate
>> numeric field from your documents (it uses FieldCache).
>>
>> Uwe
>>
>> -----
>> Uwe Schindler
>> H.-H.-Meier-Allee 63, D-28213 Bremen
>> http://www.thetaphi.de
>> eMail: uwe@thetaphi.de
>>
>>> -----Original Message-----
>>> From: Ian Lea [mailto:ian.lea@gmail.com]
>>> Sent: Monday, February 22, 2010 10:33 AM
>>> To: java-user@lucene.apache.org
>>> Subject: Re: Boost Problem (again), need example !
>>>
>>> Can't you simply sort by descending score (your score, not lucene's)?
>>> Seems to me that would give you what you are asking for.
>>>
>>> The setBoost() method is unlikely to work consistently because it only
>>> infuences the score rather than setting it. If your John Mickeal doc
>>> happens to have a higher lucene score, because of the normal
>>> idf/tf/etc stuff, then the setBoost() with a higher value for John
>>> Smith may well not be enough to force John Smith to the top.
>>>
>>> I don't know enough about function queries to help you much there but
>>> FieldScoreQuery might work. I can't see any sign of class
>>> FunctionQuery in the 3.0.0 core package so am not clear what that is.
>>>
>>>
>>> --
>>> Ian.
>>>
>>>
>>>
>>> On Mon, Feb 22, 2010 at 8:54 AM, pdaures <pa...@gmail.com>
>>> wrote:
>>> >
>>> > Hi,
>>> > I know that there are many topics about scoring issues, but I didn't
>>> find an
>>> > answer in the topics.
>>> > This is the problem :
>>> > Imagine I'm a teacher, and I have to index all the results, comments
>>> and
>>> > score about students.
>>> >
>>> > Student :
>>> > String name (eg : John Smith)
>>> > String comments : (eg: John is a good student, but he needs to be
>>> more self
>>> > confident bla bla bla)
>>> > float score (eg : 98)
>>> >
>>> > I have to index all the students and when I use the search class, I
>>> want to
>>> > get first the best students. So, if John Smith is a better student
>>> than John
>>> > Mickael, when I search "John" I want to have John Smith BEFORE John
>>> Mickeal.
>>> >
>>> > To do that, I'm using BooleanQuery to search in name and comment
>>> fields.
>>> >
>>> > First, I thought I could use the function Document.setBoost(float
>>> boost)
>>> > while indexing student, with boost = Student.score. But the result
>>> was not
>>> > what I was expected, it didn't work correctly.
>>> >
>>> > Then I thought I could use a FunctionQuery to search :
>>> > FunctionQuery functionQuery = new FunctionQuery(new
>>> > ReverseOrdFieldSource("score"));
>>> > But the result was still incorrect.
>>> >
>>> > I don't know what I'm doing wrong. Could you help me to find a
>>> solution ?
>>> > Thank you :)
>>> > --
>>> > View this message in context: http://old.nabble.com/Boost-Problem-
>>> %28again%29%2C-need-example-%21-tp27684388p27684388.html
>>> > Sent from the Lucene - Java Users mailing list archive at Nabble.com.
>>> >
>>> >
>>> > ---------------------------------------------------------------------
>>> > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>> > For additional commands, e-mail: java-user-help@lucene.apache.org
>>> >
>>> >
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>>
>>
>
> --
> View this message in context: http://old.nabble.com/Boost-Problem-%28again%29%2C-need-example-%21-tp27684388p27685594.html
> Sent from the Lucene - Java Users mailing list archive at Nabble.com.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
RE: Boost Problem (again), need example !
Posted by pdaures <pa...@gmail.com>.
HI !
Thank you for your help.
I think I don't use CustomScoreQuery correctly when I do a "search".
BooleanQuery combinedQuery = new BooleanQuery();
combinedQuery.add(textQuery, Occur.MUST);
combinedQuery.add(titleQuery, Occur.MUST);
CustomScoreQuery customQuery = new CustomScoreQuery(combinedQuery,new
FieldScoreQuery(BOOST_FIELD,Type.INT));
indexSearcher.search(..., customQuery, ....).
in order to index the BOOST_FIELD, I do that :
Field boostField = new Field(BOOST_FIELD, Integer.toString(boost),
Field.Store.YES, Field.Index.ANALYZED.NO);
Is that correct ?
Thank you
Uwe Schindler wrote:
>
> It's CustomScoreQuery in 2.9 and 3.0.
>
> Please wait for 2.9.2 and 3.0.1 for an important API change in this
> experimental query type to work correct with the new per-segment-search!
> You can test the release artifacts of both new versions here:
> http://people.apache.org/~uschindler/staging-area/lucene-292-301-take2-rev912433/
>
> With e.g. ValueSourceQuery you can score your documents using a separate
> numeric field from your documents (it uses FieldCache).
>
> Uwe
>
> -----
> Uwe Schindler
> H.-H.-Meier-Allee 63, D-28213 Bremen
> http://www.thetaphi.de
> eMail: uwe@thetaphi.de
>
>> -----Original Message-----
>> From: Ian Lea [mailto:ian.lea@gmail.com]
>> Sent: Monday, February 22, 2010 10:33 AM
>> To: java-user@lucene.apache.org
>> Subject: Re: Boost Problem (again), need example !
>>
>> Can't you simply sort by descending score (your score, not lucene's)?
>> Seems to me that would give you what you are asking for.
>>
>> The setBoost() method is unlikely to work consistently because it only
>> infuences the score rather than setting it. If your John Mickeal doc
>> happens to have a higher lucene score, because of the normal
>> idf/tf/etc stuff, then the setBoost() with a higher value for John
>> Smith may well not be enough to force John Smith to the top.
>>
>> I don't know enough about function queries to help you much there but
>> FieldScoreQuery might work. I can't see any sign of class
>> FunctionQuery in the 3.0.0 core package so am not clear what that is.
>>
>>
>> --
>> Ian.
>>
>>
>>
>> On Mon, Feb 22, 2010 at 8:54 AM, pdaures <pa...@gmail.com>
>> wrote:
>> >
>> > Hi,
>> > I know that there are many topics about scoring issues, but I didn't
>> find an
>> > answer in the topics.
>> > This is the problem :
>> > Imagine I'm a teacher, and I have to index all the results, comments
>> and
>> > score about students.
>> >
>> > Student :
>> > String name (eg : John Smith)
>> > String comments : (eg: John is a good student, but he needs to be
>> more self
>> > confident bla bla bla)
>> > float score (eg : 98)
>> >
>> > I have to index all the students and when I use the search class, I
>> want to
>> > get first the best students. So, if John Smith is a better student
>> than John
>> > Mickael, when I search "John" I want to have John Smith BEFORE John
>> Mickeal.
>> >
>> > To do that, I'm using BooleanQuery to search in name and comment
>> fields.
>> >
>> > First, I thought I could use the function Document.setBoost(float
>> boost)
>> > while indexing student, with boost = Student.score. But the result
>> was not
>> > what I was expected, it didn't work correctly.
>> >
>> > Then I thought I could use a FunctionQuery to search :
>> > FunctionQuery functionQuery = new FunctionQuery(new
>> > ReverseOrdFieldSource("score"));
>> > But the result was still incorrect.
>> >
>> > I don't know what I'm doing wrong. Could you help me to find a
>> solution ?
>> > Thank you :)
>> > --
>> > View this message in context: http://old.nabble.com/Boost-Problem-
>> %28again%29%2C-need-example-%21-tp27684388p27684388.html
>> > Sent from the Lucene - Java Users mailing list archive at Nabble.com.
>> >
>> >
>> > ---------------------------------------------------------------------
>> > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> > For additional commands, e-mail: java-user-help@lucene.apache.org
>> >
>> >
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
>
--
View this message in context: http://old.nabble.com/Boost-Problem-%28again%29%2C-need-example-%21-tp27684388p27685594.html
Sent from the Lucene - Java Users mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
RE: Boost Problem (again), need example !
Posted by Uwe Schindler <uw...@thetaphi.de>.
It's CustomScoreQuery in 2.9 and 3.0.
Please wait for 2.9.2 and 3.0.1 for an important API change in this experimental query type to work correct with the new per-segment-search! You can test the release artifacts of both new versions here: http://people.apache.org/~uschindler/staging-area/lucene-292-301-take2-rev912433/
With e.g. ValueSourceQuery you can score your documents using a separate numeric field from your documents (it uses FieldCache).
Uwe
-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: uwe@thetaphi.de
> -----Original Message-----
> From: Ian Lea [mailto:ian.lea@gmail.com]
> Sent: Monday, February 22, 2010 10:33 AM
> To: java-user@lucene.apache.org
> Subject: Re: Boost Problem (again), need example !
>
> Can't you simply sort by descending score (your score, not lucene's)?
> Seems to me that would give you what you are asking for.
>
> The setBoost() method is unlikely to work consistently because it only
> infuences the score rather than setting it. If your John Mickeal doc
> happens to have a higher lucene score, because of the normal
> idf/tf/etc stuff, then the setBoost() with a higher value for John
> Smith may well not be enough to force John Smith to the top.
>
> I don't know enough about function queries to help you much there but
> FieldScoreQuery might work. I can't see any sign of class
> FunctionQuery in the 3.0.0 core package so am not clear what that is.
>
>
> --
> Ian.
>
>
>
> On Mon, Feb 22, 2010 at 8:54 AM, pdaures <pa...@gmail.com>
> wrote:
> >
> > Hi,
> > I know that there are many topics about scoring issues, but I didn't
> find an
> > answer in the topics.
> > This is the problem :
> > Imagine I'm a teacher, and I have to index all the results, comments
> and
> > score about students.
> >
> > Student :
> > String name (eg : John Smith)
> > String comments : (eg: John is a good student, but he needs to be
> more self
> > confident bla bla bla)
> > float score (eg : 98)
> >
> > I have to index all the students and when I use the search class, I
> want to
> > get first the best students. So, if John Smith is a better student
> than John
> > Mickael, when I search "John" I want to have John Smith BEFORE John
> Mickeal.
> >
> > To do that, I'm using BooleanQuery to search in name and comment
> fields.
> >
> > First, I thought I could use the function Document.setBoost(float
> boost)
> > while indexing student, with boost = Student.score. But the result
> was not
> > what I was expected, it didn't work correctly.
> >
> > Then I thought I could use a FunctionQuery to search :
> > FunctionQuery functionQuery = new FunctionQuery(new
> > ReverseOrdFieldSource("score"));
> > But the result was still incorrect.
> >
> > I don't know what I'm doing wrong. Could you help me to find a
> solution ?
> > Thank you :)
> > --
> > View this message in context: http://old.nabble.com/Boost-Problem-
> %28again%29%2C-need-example-%21-tp27684388p27684388.html
> > Sent from the Lucene - Java Users mailing list archive at Nabble.com.
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> > For additional commands, e-mail: java-user-help@lucene.apache.org
> >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Boost Problem (again), need example !
Posted by Ian Lea <ia...@gmail.com>.
Can't you simply sort by descending score (your score, not lucene's)?
Seems to me that would give you what you are asking for.
The setBoost() method is unlikely to work consistently because it only
infuences the score rather than setting it. If your John Mickeal doc
happens to have a higher lucene score, because of the normal
idf/tf/etc stuff, then the setBoost() with a higher value for John
Smith may well not be enough to force John Smith to the top.
I don't know enough about function queries to help you much there but
FieldScoreQuery might work. I can't see any sign of class
FunctionQuery in the 3.0.0 core package so am not clear what that is.
--
Ian.
On Mon, Feb 22, 2010 at 8:54 AM, pdaures <pa...@gmail.com> wrote:
>
> Hi,
> I know that there are many topics about scoring issues, but I didn't find an
> answer in the topics.
> This is the problem :
> Imagine I'm a teacher, and I have to index all the results, comments and
> score about students.
>
> Student :
> String name (eg : John Smith)
> String comments : (eg: John is a good student, but he needs to be more self
> confident bla bla bla)
> float score (eg : 98)
>
> I have to index all the students and when I use the search class, I want to
> get first the best students. So, if John Smith is a better student than John
> Mickael, when I search "John" I want to have John Smith BEFORE John Mickeal.
>
> To do that, I'm using BooleanQuery to search in name and comment fields.
>
> First, I thought I could use the function Document.setBoost(float boost)
> while indexing student, with boost = Student.score. But the result was not
> what I was expected, it didn't work correctly.
>
> Then I thought I could use a FunctionQuery to search :
> FunctionQuery functionQuery = new FunctionQuery(new
> ReverseOrdFieldSource("score"));
> But the result was still incorrect.
>
> I don't know what I'm doing wrong. Could you help me to find a solution ?
> Thank you :)
> --
> View this message in context: http://old.nabble.com/Boost-Problem-%28again%29%2C-need-example-%21-tp27684388p27684388.html
> Sent from the Lucene - Java Users mailing list archive at Nabble.com.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org