Document Decay

[brad anderson <so...@gmail.com>]

Hi,

I'm looking for a way to have the score of documents decay over time. I want
older documents to have a lower score than newer documents.

I noted the ReciprocalFloatFunction class. In an example it seemed to be
doing just this when you set the function to be:

     recip(ms(NOW,mydatefield),3.16e-11,1,1)

This is supposed to degrade the score to half its value if the mydatefield
is 1 year older  than the current date. My question with this is, is it
making the document score go down to 0.5 or is it making the document score
1/2 of its original value.

i.e.

The document has score 0.8

Will the score be 0.4 or 0.5 after using this function?

Also, are there better alternatives to deal with document decay?

Thanks,
Stephen

Re: Document Decay

[Grant Ingersoll <gs...@apache.org>]

On Dec 4, 2009, at 1:56 AM, brad anderson wrote:

> Hi,
> 
> I'm looking for a way to have the score of documents decay over time. I want
> older documents to have a lower score than newer documents.
> 
> I noted the ReciprocalFloatFunction class. In an example it seemed to be
> doing just this when you set the function to be:
> 
>     recip(ms(NOW,mydatefield),3.16e-11,1,1)
> 
> This is supposed to degrade the score to half its value if the mydatefield
> is 1 year older  than the current date. My question with this is, is it
> making the document score go down to 0.5 or is it making the document score
> 1/2 of its original value.
> 
> i.e.
> 
> The document has score 0.8
> 
> Will the score be 0.4 or 0.5 after using this function?


Actually, the value of the function gets added (it can be multiplied, too, with other params) to the score for the document.  You can see this by adding a &debugQuery=true value to your request which allows you to examine the explains.

> 
> Also, are there better alternatives to deal with document decay?

Some people like a different decay that does something like:  today is better than yesterday, yesterday is better than last week and last week is better than last month, etc. (in a non-linear way).  Todo this, you would need to implement your own function, I think.

--------------------------
Grant Ingersoll
http://www.lucidimagination.com/

Search the Lucene ecosystem (Lucene/Solr/Nutch/Mahout/Tika/Droids) using Solr/Lucene:
http://www.lucidimagination.com/search