post-normalization score filter

[Chris Fraschetti <fr...@gmail.com>]

I've seen a few posts which reference using a HitCollector to help
filter the returned document's scores.. but are there any
implementations out there that perform this filter of results by score
after they've already been normalized, short of walking the hits
array.... If i was to walk hits until i reach my threshold, just
checking the hits.score() ... what am I looking performance wise, are
the scores all calculated at this point and I'm simply accessing them
or are they normalized on the fly?

Obviously I can trim my results a bit for results that score a million
documents only allow the first few thousand as results, but even in
the case of where I might get two thousand results, the lower end
results have such minimal scores in comparison to the earlier that it
seems unintuitive for them to be accessible.

thanks as always!

-- 
___________________________________________________
Chris Fraschetti
e fraschetti@gmail.com

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org

Re: post-normalization score filter

[Chris Hostetter <ho...@fucit.org>]

It doesn't look like there were any replies to this while i've been away,
so i just wanted to point out that this isn't really a practicle thing to
do because the score's don't have any meaningfulll absolute value (ie: you
can't compare the scores from one search with the scores of another).  the
first Hit will allways have a score of "1" and the score of the other hits
will be normalized against the first one.

I've added this to the FAQ as "Can I filter by score?"
http://wiki.apache.org/jakarta-lucene/LuceneFAQ
http://wiki.apache.org/jakarta-lucene/LuceneFAQ#head-912c1f237bb00259185353182948e5935f0c2f03

: Date: Sat, 13 Aug 2005 16:49:12 -0700
: From: Chris Fraschetti <fr...@gmail.com>
: Reply-To: java-user@lucene.apache.org
: To: java-user@lucene.apache.org
: Subject: post-normalization score filter
:
: I've seen a few posts which reference using a HitCollector to help
: filter the returned document's scores.. but are there any
: implementations out there that perform this filter of results by score
: after they've already been normalized, short of walking the hits
: array.... If i was to walk hits until i reach my threshold, just
: checking the hits.score() ... what am I looking performance wise, are
: the scores all calculated at this point and I'm simply accessing them
: or are they normalized on the fly?
:
: Obviously I can trim my results a bit for results that score a million
: documents only allow the first few thousand as results, but even in
: the case of where I might get two thousand results, the lower end
: results have such minimal scores in comparison to the earlier that it
: seems unintuitive for them to be accessible.
:
: thanks as always!
:
: --
: ___________________________________________________
: Chris Fraschetti
: e fraschetti@gmail.com
:
: ---------------------------------------------------------------------
: To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
: For additional commands, e-mail: java-user-help@lucene.apache.org
:



-Hoss


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org