Custom Sorting Algorithm

[wojtekpia <wo...@hotmail.com>]

Is an easy way to choose/create an alternate sorting algorithm? I'm
frequently dealing with large result sets (a few million results) and I
might be able to benefit domain knowledge in my sort.
-- 
View this message in context: http://www.nabble.com/Custom-Sorting-Algorithm-tp21837721p21837721.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Custom Sorting Algorithm

[Yonik Seeley <ys...@gmail.com>]

On Wed, Feb 4, 2009 at 4:45 PM, wojtekpia <wo...@hotmail.com> wrote:
> Ok, so maybe a better question is: should I bother trying to change the
> "sorting" algorithm? I'm concerned that with large data sets, sorting
> becomes a severe bottleneck (this is an assumption, I haven't profiled
> anything to verify).

No... Lucene/Solr never sorts the complete result set.
If you ask for the top 10 results, a priority queue (heap) of the
current top 10 results is maintained... far more efficient and
scalable than sorting all the hits at the end.

-Yonik

Re: Custom Sorting Algorithm

[wojtekpia <wo...@hotmail.com>]

Ok, so maybe a better question is: should I bother trying to change the
"sorting" algorithm? I'm concerned that with large data sets, sorting
becomes a severe bottleneck (this is an assumption, I haven't profiled
anything to verify). Does it become a severe bottleneck? Do you know if
alternate sort algorithms have been tried during Lucene development? 



markrmiller wrote:
> 
> It would not be simple to use a new algorithm. The current 
> implementation takes place at the Lucene level and uses a priority 
> queue. When you ask for the top n results, a priority queue of size n is 
> filled with all of the matching documents. The ordering in the priority 
> queue is the sort. The on Sort method orders by relevance score - the 
> Sort method orders by field, relevance, or doc id.
> 

-- 
View this message in context: http://www.nabble.com/Custom-Sorting-Algorithm-tp21837721p21840299.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Custom Sorting Algorithm

[Mark Miller <ma...@gmail.com>]

It would not be simple to use a new algorithm. The current 
implementation takes place at the Lucene level and uses a priority 
queue. When you ask for the top n results, a priority queue of size n is 
filled with all of the matching documents. The ordering in the priority 
queue is the sort. The on Sort method orders by relevance score - the 
Sort method orders by field, relevance, or doc id.

- Mark

wojtekpia wrote:
> That's not quite what I meant. I'm not looking for a custom comparator, I'm
> looking for a custom sorting algorithm. Is there a way to use quick sort or
> merge sort or... rather than the current algorithm? Also, what is the
> current algorithm?
>
>
> Otis Gospodnetic wrote:
>   
>> You can use one of the exiting function queries (if they fit your need) or
>> write a custom function query to reorder the results of a query.
>>
>>
>>     
>
>   

Re: Custom Sorting Algorithm

[wojtekpia <wo...@hotmail.com>]

That's not quite what I meant. I'm not looking for a custom comparator, I'm
looking for a custom sorting algorithm. Is there a way to use quick sort or
merge sort or... rather than the current algorithm? Also, what is the
current algorithm?


Otis Gospodnetic wrote:
> 
> 
> You can use one of the exiting function queries (if they fit your need) or
> write a custom function query to reorder the results of a query.
> 
> 

-- 
View this message in context: http://www.nabble.com/Custom-Sorting-Algorithm-tp21837721p21838804.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Custom Sorting Algorithm

[Otis Gospodnetic <ot...@yahoo.com>]

Hi,

You can use one of the exiting function queries (if they fit your need) or write a custom function query to reorder the results of a query.

Otis
--
Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch



----- Original Message ----
> From: wojtekpia <wo...@hotmail.com>
> To: solr-user@lucene.apache.org
> Sent: Wednesday, February 4, 2009 2:28:56 PM
> Subject: Custom Sorting Algorithm
> 
> 
> Is an easy way to choose/create an alternate sorting algorithm? I'm
> frequently dealing with large result sets (a few million results) and I
> might be able to benefit domain knowledge in my sort.
> -- 
> View this message in context: 
> http://www.nabble.com/Custom-Sorting-Algorithm-tp21837721p21837721.html
> Sent from the Solr - User mailing list archive at Nabble.com.