You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2016/05/27 14:36:13 UTC
[jira] [Commented] (SOLR-8542) Integrate Learning to Rank into Solr
[ https://issues.apache.org/jira/browse/SOLR-8542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15304130#comment-15304130 ]
ASF GitHub Bot commented on SOLR-8542:
--------------------------------------
GitHub user mnilsson23 opened a pull request:
https://github.com/apache/lucene-solr/pull/40
SOLR-8542: Integrate Learning to Rank into Solr
Solr Learning to Rank (LTR) provides a way for you to extract features
directly inside Solr for use in training a machine learned model. You
can then deploy that model to Solr and use it to rerank your top X
search results. This concept was previously presented by the authors at
Lucene/Solr Revolution 2015.
See the [README](https://github.com/bloomberg/lucene-solr/tree/master-ltr-plugin-release/solr/contrib/ltr) for more information on how to get started.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/bloomberg/lucene-solr master-ltr-plugin-release
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/lucene-solr/pull/40.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #40
----
commit 073de9b2719abe91e106119b23b977e521e8b32f
Author: Diego Ceccarelli <dc...@bloomberg.net>
Date: 2016-01-13T22:29:17Z
SOLR-8542: Integrate Learning to Rank into Solr
Solr Learning to Rank (LTR) provides a way for you to extract features
directly inside Solr for use in training a machine learned model. You
can then deploy that model to Solr and use it to rerank your top X
search results. This concept was previously presented by the authors at
Lucene/Solr Revolution 2015
commit b2bbe8c13122280ee5a76149bfb55fd1b7324279
Author: Michael Nilsson <mn...@bloomberg.net>
Date: 2016-05-25T22:13:05Z
Learning to Rank plugin updates
- Updated our documentation about the training phase and how to train a real model for those that are not familiar with this process. We provided a step by step example building a rankSVM model externally, and supplied a sample script which does this using liblinear.
- Formatted the code based on the lucene eclipse style
- Updated the hashCode and equals functions of the ModelQuery as [~Alessandro.Benedetti] pointed out
- Renamed ModelMetadata, the class you would subclass to add a new model for scoring docs, to LTRScoringAlgorithm
- Cleaned up the LTRScoringAlgorithm to no longer have a type parameter
- Added IntelliJ support. Thank you [~Alessandro.Benedetti] for adding it
- Renamed mstore and fstore endpoints to feature-store and model-store as per [~Upayavira]'s suggestion
- Added support for default efi parameters using the same Solr standard in solrconfig. When defining a feature in the config, put $\{isFromManchester:0\} to get 0 as a default, and you won't have to specify it in the request's efi params. Thanks for the enhancement suggestion [~Alessandro.Benedetti]
- Removed the fv=true param requirement for extracting features.
- You do not have to provide a "dummy model" first for extracting features, so you can request the transformer without the need of an rq ranking query. Inside the transformer you can provide a store=myFeatureStore param, and it will extract all features from that feature store directly. You can also provide local efi params if needed when extracting without an rq.
----
> Integrate Learning to Rank into Solr
> ------------------------------------
>
> Key: SOLR-8542
> URL: https://issues.apache.org/jira/browse/SOLR-8542
> Project: Solr
> Issue Type: New Feature
> Reporter: Joshua Pantony
> Assignee: Christine Poerschke
> Priority: Minor
> Attachments: README.md, README.md, SOLR-8542-branch_5x.patch, SOLR-8542-trunk.patch
>
>
> This is a ticket to integrate learning to rank machine learning models into Solr. Solr Learning to Rank (LTR) provides a way for you to extract features directly inside Solr for use in training a machine learned model. You can then deploy that model to Solr and use it to rerank your top X search results. This concept was previously presented by the authors at Lucene/Solr Revolution 2015 ( http://www.slideshare.net/lucidworks/learning-to-rank-in-solr-presented-by-michael-nilsson-diego-ceccarelli-bloomberg-lp ).
> The attached code was jointly worked on by Joshua Pantony, Michael Nilsson, David Grohmann and Diego Ceccarelli.
> Any chance this could make it into a 5x release? We've also attached documentation as a github MD file, but are happy to convert to a desired format.
> h3. Test the plugin with solr/example/techproducts in 6 steps
> Solr provides some simple example of indices. In order to test the plugin with
> the techproducts example please follow these steps
> h4. 1. compile solr and the examples
> cd solr
> ant dist
> ant example
> h4. 2. run the example
> ./bin/solr -e techproducts
> h4. 3. stop it and install the plugin:
>
> ./bin/solr stop
> mkdir example/techproducts/solr/techproducts/lib
> cp build/contrib/ltr/lucene-ltr-6.0.0-SNAPSHOT.jar example/techproducts/solr/techproducts/lib/
> cp contrib/ltr/example/solrconfig.xml example/techproducts/solr/techproducts/conf/
> h4. 4. run the example again
>
> ./bin/solr -e techproducts
> h4. 5. index some features and a model
> curl -XPUT 'http://localhost:8983/solr/techproducts/schema/fstore' --data-binary "@./contrib/ltr/example/techproducts-features.json" -H 'Content-type:application/json'
> curl -XPUT 'http://localhost:8983/solr/techproducts/schema/mstore' --data-binary "@./contrib/ltr/example/techproducts-model.json" -H 'Content-type:application/json'
> h4. 6. have fun !
> *access to the default feature store*
> http://localhost:8983/solr/techproducts/schema/fstore/_DEFAULT_
> *access to the model store*
> http://localhost:8983/solr/techproducts/schema/mstore
> *perform a query using the model, and retrieve the features*
> http://localhost:8983/solr/techproducts/query?indent=on&q=test&wt=json&rq={!ltr%20model=svm%20reRankDocs=25%20efi.query=%27test%27}&fl=*,[features],price,score,name&fv=true
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org