Learning to Rank algorithms in Lucene

[Ajinkya Kale <ka...@gmail.com>]

Are there any existing packages/examples or prior experience on using
Learning to Rank (or Machine Learned Ranking) algorithms as custom
Scorer/Ranker for lucene or solr ?
How do people deploy Learning to Rank models with Lucene backends ?

--ajinkya

Re: Learning to Rank algorithms in Lucene

[Koji Sekiguchi <ko...@rondhuit.com>]

Hi ajinkya,

In last week, I had a technical talk about NLP4L at Lucene/Solr meetup:

http://www.meetup.com/Downtown-SF-Apache-Lucene-Solr-Meetup/events/223899054/

In my talk, I told about the implementation idea of Learning to Rank using Lucene.
Please take a look at page 48 to 50 of the following slides:

http://www.slideshare.net/KojiSekiguchi/an-introduction-to-nlp4l

In NLP4L project, we're going to develop functions that are described in the slides

Thanks,

Koji


On 2015/08/18 16:34, Ajinkya Kale wrote:
> Are there any existing packages/examples or prior experience on using
> Learning to Rank (or Machine Learned Ranking) algorithms as custom
> Scorer/Ranker for lucene or solr ?
> How do people deploy Learning to Rank models with Lucene backends ?
>
> --ajinkya
>



---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org

Re: Learning to Rank algorithms in Lucene

[Ahmet Arslan <io...@yahoo.com.INVALID>]

Hi Ajinkya,

I don't think there exists any production-ready LtR-Lucene/Solr setup.

LtR simply re-rank top N (typically 1000) documents. 
Fetching top N documents is what we do today with Lucene.

There is an API for re-rank in Lucene/Solr but no LtR support yet.
https://cwiki.apache.org/confluence/display/solr/Query+Re-Ranking

Here are the difficulties/problems :

* LtR requires training data (probably labelled by humans)
* It is hard to decide the feature set. Also it differs from system to system.
* Query-dependeny features must be calculated for the top N documents at query/retrieval time, which may be slow.

Today, generally function queries are used to combine recency, popularity, star, product/document quality, price, etc into scoring function.
This approach is unsupervised therefore requires no training data.

Ahmet



On Tuesday, August 18, 2015 10:34 AM, Ajinkya Kale <ka...@gmail.com> wrote:
Are there any existing packages/examples or prior experience on using
Learning to Rank (or Machine Learned Ranking) algorithms as custom
Scorer/Ranker for lucene or solr ?
How do people deploy Learning to Rank models with Lucene backends ?

--ajinkya

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org

Re: Learning to Rank algorithms in Lucene

[Sachin Kulkarni <ku...@hawk.iit.edu>]

Where do you plan to use it?
So far there is no built in learning to rank implementations in Lucene at
least.
There are suggestions to include those.
I do not know about Solr.

I worked on research projects on Learning to Rank algorithms and I had used
Lucene to generate the features which then I ran through
my own learning to rank algorithms externally.

I am curious if anyone else has tried this.

On Tue, Aug 18, 2015 at 2:34 AM, Ajinkya Kale <ka...@gmail.com> wrote:

> Are there any existing packages/examples or prior experience on using
> Learning to Rank (or Machine Learned Ranking) algorithms as custom
> Scorer/Ranker for lucene or solr ?
> How do people deploy Learning to Rank models with Lucene backends ?
>
> --ajinkya
>