query score across ALL docs

[Peyman Faratin <pe...@robustlinks.com>]

Hi

What is the best way to retrieve the score of a query across ALL documents in the index? i.e.

given:

1) docs,  [A,B,C,D,E,...M]  of M dimensions

2) Query q

searcher outputs (efficiently)

1) the score of q across _all_ M dimensional documents, ordered by index number. i.e

score(q) = [A=0.1,B=0.0,....M=0.76]

Currently the searcher outputs the top N matches, where (often) N <<M in cases of large indices.  My index is ~9MM docs. Using a custom collector will not work. 

Any advice would be much appreciated

Peyman

Re: query score across ALL docs

[Walter Underwood <wu...@wunderwood.org>]

If you want to do *anything* across all matches, you probably should be using a relational database. Search engines, like Solr, are optimized for just the best matches. Fetching all matches is likely to be slow. Relational databases are optimized for working with the whole set of matches.

wunder
Search Guy, Chegg

On Mar 28, 2012, at 7:03 PM, Peyman Faratin wrote:

> Hi
> 
> What is the best way to retrieve the score of a query across ALL documents in the index? i.e.
> 
> given:
> 
> 1) docs,  [A,B,C,D,E,...M]  of M dimensions
> 
> 2) Query q
> 
> searcher outputs (efficiently)
> 
> 1) the score of q across _all_ M dimensional documents, ordered by index number. i.e
> 
> score(q) = [A=0.1,B=0.0,....M=0.76]
> 
> Currently the searcher outputs the top N matches, where (often) N <<M in cases of large indices.  My index is ~9MM docs. Using a custom collector will not work. 
> 
> Any advice would be much appreciated
> 
> Peyman