Need another way to boost relavence of recent content

[Shawn Heisey <so...@elyograg.org>]

I implemented the date boosting function outline here, placed into the 
boost parameter in the request handler:

http://wiki.apache.org/solr/FunctionQuery#Date_Boosting

Today it was reported to me that this boosting is producing terrible 
results.  A close look at the description reveals that this isn't so 
much a boost on new content as it is a negative boost on old content. 
This isn't what I want.

What I need to have happen is that content added today gets a small (and 
easily configurable) boost, content added yesterday gets a slightly 
smaller boost, tapering down to approximately two to four weeks in the 
past, at which point there would be no boost at all.  A document that's 
a month old would have the same boost (none) as a document that's 50 
years old.

Can someone figure out the formula for this?

Thanks,
Shawn

RE: Need another way to boost relavence of recent content

[Markus Jelsma <ma...@openindex.io>]

Hi,

Depending on use case the functions max, min, scale and map can be used really well to regulate the output of recip. Check their docs and you'll surely work it out. Perhaps scale will work best for you.

Cheers

 
 
-----Original message-----
> From:Shawn Heisey <so...@elyograg.org>
> Sent: Mon 14-Jan-2013 22:50
> To: solr-user@lucene.apache.org
> Subject: Need another way to boost relavence of recent content
> 
> I implemented the date boosting function outline here, placed into the 
> boost parameter in the request handler:
> 
> http://wiki.apache.org/solr/FunctionQuery#Date_Boosting
> 
> Today it was reported to me that this boosting is producing terrible 
> results.  A close look at the description reveals that this isn't so 
> much a boost on new content as it is a negative boost on old content. 
> This isn't what I want.
> 
> What I need to have happen is that content added today gets a small (and 
> easily configurable) boost, content added yesterday gets a slightly 
> smaller boost, tapering down to approximately two to four weeks in the 
> past, at which point there would be no boost at all.  A document that's 
> a month old would have the same boost (none) as a document that's 50 
> years old.
> 
> Can someone figure out the formula for this?
> 
> Thanks,
> Shawn
> 

Re: Need another way to boost relavence of recent content

[Shawn Heisey <so...@elyograg.org>]

On 1/14/2013 2:56 PM, Markus Jelsma wrote:
> Depending on use case the functions max, min, scale and map can be used really well to regulate the output of recip. Check their docs and you'll surely work it out. Perhaps scale will work best for you.

I need someone to sanity check my work here.

Here's my existing boost:

     <str name="boost">recip(ms(NOW/DAY,pd),3.16e-11,1,1)</str>

After a careful look at your advice and the functions available, this is 
what I have come up with:

min(recip(abs(ms(NOW/HOUR,pd)),3.85e-10,1.25,1),0.625)

To get the second value for the recip function, I figured out how many 
milliseconds were in 30 days, then inverted that.  If I understand 
everything correctly, this will result in boost values from 1.25 for 
docs created right now to 0.625 for docs created >= 30 days ago.  If I 
need to adjust 1.25 to X, then I also need to adjust the 0.625 value to 
0.5 times X.

Does that look OK?

Thanks,
Shawn

RE: Need another way to boost relavence of recent content

[Markus Jelsma <ma...@openindex.io>]

Hi,

Depending on use case the functions max, min, scale and map can be used really well to regulate the output of recip. Check their docs and you'll surely work it out. Perhaps scale will work best for you.

Cheers
 
-----Original message-----
> From:Shawn Heisey <so...@elyograg.org>
> Sent: Mon 14-Jan-2013 22:50
> To: solr-user@lucene.apache.org
> Subject: Need another way to boost relavence of recent content
> 
> I implemented the date boosting function outline here, placed into the 
> boost parameter in the request handler:
> 
> http://wiki.apache.org/solr/FunctionQuery#Date_Boosting
> 
> Today it was reported to me that this boosting is producing terrible 
> results.  A close look at the description reveals that this isn't so 
> much a boost on new content as it is a negative boost on old content. 
> This isn't what I want.
> 
> What I need to have happen is that content added today gets a small (and 
> easily configurable) boost, content added yesterday gets a slightly 
> smaller boost, tapering down to approximately two to four weeks in the 
> past, at which point there would be no boost at all.  A document that's 
> a month old would have the same boost (none) as a document that's 50 
> years old.
> 
> Can someone figure out the formula for this?
> 
> Thanks,
> Shawn
>