Negative boosts

[Derek Watson <wa...@wayspa.com>]

Hello,

I have been developing a new search application based on Solr (Very
nice!) using dismax. We are using query-time boosts to provide better
search results for user queries and index-time boosts to promote
certain documents over others.

My question is about the latter: We have a "position" field available
at index time that is an integer value, 0 being the 1st position, 1
being the 2nd position, 99 being the hundredth, etc.  How do I get
Solr to return documents in that same order? Is it possible to apply a
negative boost?

<doc boost="-0.0">...</doc>
<doc boost="-0.1">...</doc>
<doc boost="-0.2">...</doc>

I notice in the documentation for parseFieldBoosts that the routine
"Doesn't care if boost info is negative, you're on your own.", but
what does that mean?

There is a desire to preserve the order as 0..xx and not reverse it
(which would be the obvious choice - x becomes 0, 0 becomes x) because
we are adding multiple sets of positions to the index, and I want the
first position of each set to be equal (zero).

This weighting is important to us for user queries as well as filtered
views. Is there another way to get what I'm looking for?

Thanks,
Derek

Re: Negative boosts

[Chris Hostetter <ho...@fucit.org>]

: > function that will give me a bigger boost for field values closer to
: > zero?
:
: The syntax could possibly change in the future, but look at the syntax in
: the javadoc for parseFunction in
: http://incubator.apache.org/solr/docs/api/org/apache/solr/search/QueryParsing.html
:
: Then when using the query parser syntax, use _val_ to activate it.
: So you get things like _val_:linear(myfield,1,2)

two things to note:

  1) if you are using dismax, the "bf" param parses functions directly, so
you don't need the _val_ hack.
  2) the "rord" function lets you access the "reverse ordinal" value of a
field, so the higher the value, the lower the number returned by hte
function.  if you have a garunteed sequential ordering for your field,
then you'll get an even distribution out of rord(fieldName)





-Hoss

Re: Negative boosts

[Yonik Seeley <yo...@apache.org>]

On 12/14/06, Derek Watson <wa...@wayspa.com> wrote:
> >
> > If you want documents returned in the same order as a field, it's
> > easy... you sort!
> > If you want the value of a field to influence a score, not determine
> > the exact sort order, you can use FunctionQuery (currently hacked into
> > the query parser as _val_:myfield)
>
> That seems like what I want -- boosting and not sorting. Is there a
> function that will give me a bigger boost for field values closer to
> zero?

The syntax could possibly change in the future, but look at the syntax in
the javadoc for parseFunction in
http://incubator.apache.org/solr/docs/api/org/apache/solr/search/QueryParsing.html

Then when using the query parser syntax, use _val_ to activate it.
So you get things like _val_:linear(myfield,1,2)

Use debug on the query admin page to see the score explanations generated.

-Yonik

Re: Negative boosts

[Derek Watson <wa...@wayspa.com>]

>
> If you want documents returned in the same order as a field, it's
> easy... you sort!
> If you want the value of a field to influence a score, not determine
> the exact sort order, you can use FunctionQuery (currently hacked into
> the query parser as _val_:myfield)

That seems like what I want -- boosting and not sorting. Is there a
function that will give me a bigger boost for field values closer to
zero?

Re: Negative boosts

[Yonik Seeley <yo...@apache.org>]

On 12/14/06, Derek Watson <wa...@wayspa.com> wrote:
> I have been developing a new search application based on Solr (Very
> nice!) using dismax. We are using query-time boosts to provide better
> search results for user queries and index-time boosts to promote
> certain documents over others.
>
> My question is about the latter: We have a "position" field available
> at index time that is an integer value, 0 being the 1st position, 1
> being the 2nd position, 99 being the hundredth, etc.  How do I get
> Solr to return documents in that same order?

If you want documents returned in the same order as a field, it's
easy... you sort!
If you want the value of a field to influence a score, not determine
the exact sort order, you can use FunctionQuery (currently hacked into
the query parser as _val_:myfield)

> Is it possible to apply a
> negative boost?
>
> <doc boost="-0.0">...</doc>
> <doc boost="-0.1">...</doc>
> <doc boost="-0.2">...</doc>
>
> I notice in the documentation for parseFieldBoosts that the routine
> "Doesn't care if boost info is negative, you're on your own.", but
> what does that mean?

AFAIK, negative scores and boosts at query time aren't really
prohibited in Lucene, but neither are they really supported.  I would
avoid them.

Negative boosts would be mapped to zero boosts during indexing... this
is due to the way norms are represented in the Lucene index:
http://svn.apache.org/viewvc/lucene/java/trunk/src/java/org/apache/lucene/util/SmallFloat.java?view=markup
Note that there are only 3 mantissa bits in the 8 bit float, so small
distinctions at index time are effectively discarded.


-Yonik