You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@lucene.apache.org by Se...@emc.com on 2007/12/06 03:05:09 UTC
does the MultiSearcher class calculate IDF properly?
I tried the following. Creating 2 different indexes, search each
individually and print score details and compare to searching both
indexes with MulitSearcher and printing score details.
The "docFreq" value printed don't seem right - is this just a problem
with using Explain together with the MultiSearcher?
Code is like:
MultiSearcher multi = new MultiSearcher(searchables);
Hits hits = multi.search(query);
for(int i=0; i<hits.length(); i++)
{
Explanation expl = multi.explain(query, hits.id(i));
System.out.println(expl.toString());
}
Output:
id = 14 score = 0.071
0.07073946 = (MATCH) fieldWeight(contents:climate in 2), product of:
1.0 = tf(termFreq(contents:climate)=1)
1.8109303 = idf(docFreq=1)
0.0390625 = fieldNorm(field=contents, doc=2)
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
RE: does the MultiSearcher class calculate IDF properly?
Posted by Se...@emc.com.
Thank you for the response. I logged a bug
https://issues.apache.org/jira/browse/LUCENE-1087
-----Original Message-----
From: Chris Hostetter [mailto:hossman_lucene@fucit.org]
Sent: Friday, December 07, 2007 10:30 PM
To: java-user@lucene.apache.org
Subject: Re: does the MultiSearcher class calculate IDF properly?
a quick glance at the code seems to indicate that MultiSearcher has code
for calcuating the docFreq accross all of the Searchables when searching
(or when the docFreq method is explicitly called) but that explain
method
just delegates to Searchable that the specific docid came from.
if you compare that Explanation score you got with the score returned by
a HitCollector (or TopDocs) they probably won't match.
So i would say "yes MultiSearcher calculates IDF properly, but
MultiSeracher.explain is broken. Please file a bug about this, i can't
think of an easy way to fix it, but it certianly seems broken to me.
: Subject: does the MultiSearcher class calculate IDF properly?
:
: I tried the following. Creating 2 different indexes, search each
: individually and print score details and compare to searching both
: indexes with MulitSearcher and printing score details.
:
: The "docFreq" value printed don't seem right - is this just a problem
: with using Explain together with the MultiSearcher?
:
:
: Code is like:
: MultiSearcher multi = new MultiSearcher(searchables);
: Hits hits = multi.search(query);
: for(int i=0; i<hits.length(); i++)
: {
: Explanation expl = multi.explain(query, hits.id(i));
: System.out.println(expl.toString());
: }
:
:
: Output:
: id = 14 score = 0.071
: 0.07073946 = (MATCH) fieldWeight(contents:climate in 2), product of:
: 1.0 = tf(termFreq(contents:climate)=1)
: 1.8109303 = idf(docFreq=1)
: 0.0390625 = fieldNorm(field=contents, doc=2)
:
: ---------------------------------------------------------------------
: To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
: For additional commands, e-mail: java-user-help@lucene.apache.org
:
-Hoss
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: does the MultiSearcher class calculate IDF properly?
Posted by Chris Hostetter <ho...@fucit.org>.
a quick glance at the code seems to indicate that MultiSearcher has code
for calcuating the docFreq accross all of the Searchables when searching
(or when the docFreq method is explicitly called) but that explain method
just delegates to Searchable that the specific docid came from.
if you compare that Explanation score you got with the score returned by
a HitCollector (or TopDocs) they probably won't match.
So i would say "yes MultiSearcher calculates IDF properly, but
MultiSeracher.explain is broken. Please file a bug about this, i can't
think of an easy way to fix it, but it certianly seems broken to me.
: Subject: does the MultiSearcher class calculate IDF properly?
:
: I tried the following. Creating 2 different indexes, search each
: individually and print score details and compare to searching both
: indexes with MulitSearcher and printing score details.
:
: The "docFreq" value printed don't seem right - is this just a problem
: with using Explain together with the MultiSearcher?
:
:
: Code is like:
: MultiSearcher multi = new MultiSearcher(searchables);
: Hits hits = multi.search(query);
: for(int i=0; i<hits.length(); i++)
: {
: Explanation expl = multi.explain(query, hits.id(i));
: System.out.println(expl.toString());
: }
:
:
: Output:
: id = 14 score = 0.071
: 0.07073946 = (MATCH) fieldWeight(contents:climate in 2), product of:
: 1.0 = tf(termFreq(contents:climate)=1)
: 1.8109303 = idf(docFreq=1)
: 0.0390625 = fieldNorm(field=contents, doc=2)
:
: ---------------------------------------------------------------------
: To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
: For additional commands, e-mail: java-user-help@lucene.apache.org
:
-Hoss
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org