You are viewing a plain text version of this content. The canonical link for it is here.

Posted to java-user@lucene.apache.org by Yatin Soni <ya...@interinfosystems.com> on 2007/06/14 18:53:35 UTC

Lucene Search result (scoring )

Hi,

We are using Lucene as search engine and I have a question regarding the scoring of search results, I had given a example for it,

Example :-->

suppose we have four Items on which we have indexed,
////////////////////////////////////////////////////////////////////////////////////////////////////////////////
1)demo --> contents: Jumbo Numbers
2)demo2 --> contents: Jumbo Numbers
3)demo3 --> contents: Jumbo Numbers
4)demo4 --> contents: Jumbo Numbers

Search Results for query ->"Jumbo"

demo: Score: 0.3884282
demo2: Score: 0.3884282
demo3: Score: 0.3884282
demo4: Score: 0.3884282
////////////////////////////////////////////////////////////////////////////////////////////////////////////////
Test 1 >> Change the contents of items

1)demo --> contents: Jumbo Numbers Stamps Numbers
2)demo2 --> contents: Jumbo Numbers Stamps Jumbo
3)demo3 --> contents: Jumbo Numbers Numbers Test
4)demo4 --> contents: Jumbo Numbers Stamps Jumbo

Search Results for query ->"Jumbo"

demo2: Score: 0.54932046
demo4: Score: 0.54932046
demo: Score: 0.3884282
demo3: Score: 0.3884282

# In the Test 1 items(demo2 and demo4) both contains 2 occurrence of "Jumbo" and rest items(demo and demo3)
have only one occurrence of "Jumbo" so result are coming as per the high score.
////////////////////////////////////////////////////////////////////////////////////////////////////////////////
Test 2 >> Change the contents of items

1)demo --> contents: Jumbo Numbers Stamps Numbers
2)demo2 --> contents: Jumbo Numbers Stamps Jumbo Parquetry Block Super Set
3)demo3 --> contents: Jumbo Numbers Numbers Test
4)demo4 --> contents: Jumbo Numbers Stamps Jumbo

Search Results for query ->"Jumbo"

demo4: Score: 0.5493204
demo: Score: 0.3884282
demo3: Score: 0.3884282
demo2: Score: 0.3433253

# Now in the Test 2 We are changing the contents of demo2, and by searching the same query "Jumbo" the results
are different from the Test 1.And the score for demo2 is also less as compare to the items (demo and demo3)
which have less occurrence of "Jumbo" as compare to demo2.
////////////////////////////////////////////////////////////////////////////////////////////////////////////////

According to the example search results are coming in the following sequence:---->
1. demo4: Score: 0.5493204
2. demo: Score: 0.3884282
3. demo3: Score: 0.3884282
4. demo2: Score: 0.3433253

but we are expecting in the following sequence :--->
1. demo4
2. demo2
3. demo
4. demo3

1. demo2
2. demo4
3. demo
4. demo3

because demo2 contains the two occurrence of "Jumbo" which is higher than the items demo and demo3. I have sorted the results with their RELEVANCE but then also
results were coming in the same sequence.

So my QUESTION is that can we make desire sequence as per the occurrence of a particular word? Can we make demo2 above the demo and demo3 in the search results?

So I need help on this issue....

Thanks,
Yatin

Re: Lucene Search result (scoring )

Posted by Chris Hostetter <ho...@fucit.org>.

: I had tried with Explaination but didn't get the desired results.Can you
: give me the brief demo code based on the result order by the no of matching
: terms .

the Explanation class will not change your scores to give you results in
any particular way you might want -- it just explains what factors are
contributing to the scores you get, so you can change the appropraite
settings run your query again and see the effects.

this is where reading about how lucene scoring (via hte URL i sent)
becomes important so you cna understand what explain is telling you...

: > the "explain" method on a Searcher, and the Explanation classes can
: > explain everything baout how/why a particular document in a particular
: > index gets a particular score for a particular search.  The only tricky
: > thing about it is understanding that it refers to the "raw" scores (what
: > you seem to be quoting look like "normalized" scores from the Hits class,
: > but if your cheif concern is the relative order of hte results this is
: > largely irrelevent)
: >
: > When looking at the Explanation info, many of the terms may not make a
: > lot of sense .. this is where the scoring.html document can fill in the
: > gaps and explain the explaination...
: >
: > http://lucene.apache.org/java/docs/scoring.html



-Hoss


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org

Re: Lucene Search result (scoring )

Posted by Yatin Soni <ya...@interinfosystems.com>.

Hi Hoss,

I had tried with Explaination but didn't get the desired results.Can you
give me the brief demo code based on the result order by the no of matching
terms .

Thanks,
Yatin

----- Original Message -----
From: "Chris Hostetter" <ho...@fucit.org>
To: <ja...@lucene.apache.org>
Sent: 16, 06, 2007 8:47 AM
Subject: Re: Lucene Search result (scoring )


>
> the "explain" method on a Searcher, and the Explanation classes can
> explain everything baout how/why a particular document in a particular
> index gets a particular score for a particular search.  The only tricky
> thing about it is understanding that it refers to the "raw" scores (what
> you seem to be quoting look like "normalized" scores from the Hits class,
> but if your cheif concern is the relative order of hte results this is
> largely irrelevent)
>
> When looking at the Explanation info, many of the terms may not make a
> lot of sense .. this is where the scoring.html document can fill in the
> gaps and explain the explaination...
>
> http://lucene.apache.org/java/docs/scoring.html
>
>
>
>
>
> -Hoss
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>



________________________________________________________
S Naidu  Domain Name Registration, Web Hosting, Email Solutions, Website Designing, 
Ecommerce website development, Payment Gateway,  
Contact 91 22 28123113 / 91 22 28129404 Mumbai http://www.snaidu.com

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org

Re: Lucene Search result (scoring )

Posted by Chris Hostetter <ho...@fucit.org>.

the "explain" method on a Searcher, and the Explanation classes can
explain everything baout how/why a particular document in a particular
index gets a particular score for a particular search.  The only tricky
thing about it is understanding that it refers to the "raw" scores (what
you seem to be quoting look like "normalized" scores from the Hits class,
but if your cheif concern is the relative order of hte results this is
largely irrelevent)

When looking at the Explanation info, many of the terms may not make a
lot of sense .. this is where the scoring.html document can fill in the
gaps and explain the explaination...

http://lucene.apache.org/java/docs/scoring.html





-Hoss


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org

Re: Lucene Search result (scoring )

Posted by Donna L Gresh <gr...@us.ibm.com>.

Your examples are a little confusing to read. However, I think one thing
that you need to know is that the score (by "default") depends on more
than just the number of hits. It also depends on the length of the
document the hits are in. For example, matching two words in a
two-word-long document will generate a higher score than matching the same
two words in a one-hundred-word-long document. In order to change the
scoring, you will have to override some of the standard methods. There are
those on the mailing list much more qualified to discuss this than I am,
as I am new myself.

Donna Gresh

"Yatin Soni" <ya...@interinfosystems.com>
06/14/2007 12:53 PM
Please respond to
java-user@lucene.apache.org

To
<ja...@lucene.apache.org>
cc

Subject
Lucene Search result (scoring )

Hi,

We are using Lucene as search engine and I have a question regarding
the scoring of search results, I had given a example for it,

Example :-->

Search Results for query ->"Jumbo"

1)demo --> contents: Jumbo Numbers Stamps Numbers
2)demo2 --> contents: Jumbo Numbers Stamps Jumbo
3)demo3 --> contents: Jumbo Numbers Numbers Test
4)demo4 --> contents: Jumbo Numbers Stamps Jumbo

Search Results for query ->"Jumbo"

demo2: Score: 0.54932046
demo4: Score: 0.54932046
demo: Score: 0.3884282
demo3: Score: 0.3884282

# In the Test 1 items(demo2 and demo4) both contains 2 occurrence of
"Jumbo" and rest items(demo and demo3)
have only one occurrence of "Jumbo" so result are coming as per the high
score.
////////////////////////////////////////////////////////////////////////////////////////////////////////////////
Test 2 >> Change the contents of items

Search Results for query ->"Jumbo"

demo4: Score: 0.5493204
demo: Score: 0.3884282
demo3: Score: 0.3884282
demo2: Score: 0.3433253

# Now in the Test 2 We are changing the contents of demo2, and by
searching the same query "Jumbo" the results
are different from the Test 1.And the score for demo2 is also less as
compare to the items (demo and demo3)
which have less occurrence of "Jumbo" as compare to demo2.
////////////////////////////////////////////////////////////////////////////////////////////////////////////////

According to the example search results are coming in the following
sequence:---->
1. demo4: Score: 0.5493204
2. demo: Score: 0.3884282
3. demo3: Score: 0.3884282
4. demo2: Score: 0.3433253

but we are expecting in the following sequence :--->
1. demo4
2. demo2
3. demo
4. demo3

1. demo2
2. demo4
3. demo
4. demo3

because demo2 contains the two occurrence of "Jumbo" which is higher than
the items demo and demo3. I have sorted the results with their RELEVANCE
but then also
results were coming in the same sequence.

So my QUESTION is that can we make desire sequence as per the occurrence
of a particular word? Can we make demo2 above the demo and demo3 in the
search results?

So I need help on this issue....

Thanks,
Yatin