How to boost some documents at query-time ?

[Fabrice Estiévenart <fa...@cetic.be>]

Hello,

I need to sort my hits according to a rate of "popularity" which 
dynamically and periodically changes. Thus, I can't store this 
popularity in the index and I have to get it, from memory, at query-time.

Is it possible with Solr ? Thank you.

Fabrice

Re: How to boost some documents at query-time ?

[Chris Hostetter <ho...@fucit.org>]

: - The CustomScoreQuery of Lucene :
: http://lucene.apache.org/java/2_4_0/api/org/apache/lucene/search/function/package-summary.html

this is a variation on the FunctionQuery class in Solr, which uses the 
ValueSource API Shalin refered to -- at present the only way to implement 
a ValueSource is either using an index field or an ExternalFile ... but if 
you someone already have the popularity values in memory, you can 
certainly implement your own ValueSource as Shalin suggested...

: > Something like this should be possible by creating your own ValueSource.
: > Look at the ExternalFileField and FileFloatSource in Solr as an example.
: > Instead of loading from a file, you can load it from some other source and
: > refresh it periodically.
: > 
: > You won't be able to sort on it using the sort parameter until [1] is
: > complete. However, you can achieve a similar effect by boosting documents by
: > a function of popularity using function queries in the "q" parameter.
: > 
: > However, if you can re-index your documents periodically, just add the
: > popularity value into the document itself and avoid a complicated system. We
: > do this for a multi-million document index (storing daily popularity as a
: > field). We also have another smaller index which we re-index every 15
: > minutes with the latest popularity values.
: > 
: > [1] - https://issues.apache.org/jira/browse/SOLR-1297
: > 
: >   
: 
: 
: -- 
: Fabrice Esti�venart, Ing�nieur R&D, CETIC
: T�l : +32 (0)71/49.07.28
: Web : http://www.cetic.be
: 



-Hoss

Re: How to boost some documents at query-time ?

[Fabrice Estiévenart <fa...@cetic.be>]

Thank you for your suggestions. Since my initial email, I have 
found/guessed two other indications :

- The CustomScoreQuery of Lucene : 
http://lucene.apache.org/java/2_4_0/api/org/apache/lucene/search/function/package-summary.html
- To create a temporary index, at query-time, containing the documents 
that match the query, with the current "popularity" values

Not sure these options will realize what I wish, any opinion ?

Fabrice

Shalin Shekhar Mangar a écrit :
> 2009/8/12 Fabrice Estiévenart <fa...@cetic.be>
>
>   
>> Hello,
>>
>> I need to sort my hits according to a rate of "popularity" which
>> dynamically and periodically changes. Thus, I can't store this popularity in
>> the index and I have to get it, from memory, at query-time.
>>
>> Is it possible with Solr ? Thank you.
>>
>>     
>
> Not out of the box.
>
> Something like this should be possible by creating your own ValueSource.
> Look at the ExternalFileField and FileFloatSource in Solr as an example.
> Instead of loading from a file, you can load it from some other source and
> refresh it periodically.
>
> You won't be able to sort on it using the sort parameter until [1] is
> complete. However, you can achieve a similar effect by boosting documents by
> a function of popularity using function queries in the "q" parameter.
>
> However, if you can re-index your documents periodically, just add the
> popularity value into the document itself and avoid a complicated system. We
> do this for a multi-million document index (storing daily popularity as a
> field). We also have another smaller index which we re-index every 15
> minutes with the latest popularity values.
>
> [1] - https://issues.apache.org/jira/browse/SOLR-1297
>
>   


-- 
Fabrice Estiévenart, Ingénieur R&D, CETIC
Tél : +32 (0)71/49.07.28
Web : http://www.cetic.be

Re: How to boost some documents at query-time ?

[Shalin Shekhar Mangar <sh...@gmail.com>]

2009/8/12 Fabrice Estiévenart <fa...@cetic.be>

> Hello,
>
> I need to sort my hits according to a rate of "popularity" which
> dynamically and periodically changes. Thus, I can't store this popularity in
> the index and I have to get it, from memory, at query-time.
>
> Is it possible with Solr ? Thank you.
>

Not out of the box.

Something like this should be possible by creating your own ValueSource.
Look at the ExternalFileField and FileFloatSource in Solr as an example.
Instead of loading from a file, you can load it from some other source and
refresh it periodically.

You won't be able to sort on it using the sort parameter until [1] is
complete. However, you can achieve a similar effect by boosting documents by
a function of popularity using function queries in the "q" parameter.

However, if you can re-index your documents periodically, just add the
popularity value into the document itself and avoid a complicated system. We
do this for a multi-million document index (storing daily popularity as a
field). We also have another smaller index which we re-index every 15
minutes with the latest popularity values.

[1] - https://issues.apache.org/jira/browse/SOLR-1297

-- 
Regards,
Shalin Shekhar Mangar.

Re: How to boost some documents at query-time ?

[Avlesh Singh <av...@gmail.com>]

>
> Is it possible with Solr ?
>
My understanding says, No.

I am waiting to be surprised.

Cheers
Avlesh

2009/8/12 Fabrice Estiévenart <fa...@cetic.be>

> Hello,
>
> I need to sort my hits according to a rate of "popularity" which
> dynamically and periodically changes. Thus, I can't store this popularity in
> the index and I have to get it, from memory, at query-time.
>
> Is it possible with Solr ? Thank you.
>
> Fabrice
>