[jira] [Commented] (SOLR-11250) Add new LTR model which loads the model definition from the external resource

["Yuki Yano (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SOLR-11250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16131583#comment-16131583 ] 

Yuki Yano commented on SOLR-11250:
----------------------------------

[~cpoerschke]
Thank you for revising the patch! I agree with your proposal and try refactoring codes based on new patch.

By the way, I'm wandering whether we need to revise {{getFeatures}} and {{getNorms}} which affect {{toLTRScoringModelMap}}. 

Current implementation only overwrites {{getParams}} which tends to grow with large model (ex. large ensemble trees). On the other hand, because it delegates {{getFeatures}} and {{getNorms}} to wrapped model, we still upload features and norms of the  wrapped model to ZooKeeper. This can be problem when uploading huge linear model which may contain many features.

As for search engines, I think this is very rare case because extracting many features is too costly. Do you think we need to consider the above case with this patch?

> Add new LTR model which loads the model definition from the external resource
> -----------------------------------------------------------------------------
>
>                 Key: SOLR-11250
>                 URL: https://issues.apache.org/jira/browse/SOLR-11250
>             Project: Solr
>          Issue Type: Improvement
>      Security Level: Public(Default Security Level. Issues are Public) 
>          Components: contrib - LTR
>            Reporter: Yuki Yano
>            Priority: Minor
>         Attachments: SOLR-11250_master.patch, SOLR-11250.patch
>
>
> We add new model which contains only the location of the external model and loads it during the initialization.
> By this procedure, large models which are difficult to upload to ZooKeeper can be available.
> The new model works as the wrapper of existing models, and deligates APIs to them.
> We add two classes by this patch:
> * {{ExternalModel}} : a base class for models with external resources.
> * {{URIExternalModel}} : an implementation of {{ExternalModel}} which loads the external model from specified URI (ex. file:, http:, etc.).
> For example, if you have a model on the local disk "file:///var/models/myModel.json", the definition of {{URIExternalModel}} will be like the following.
> {code}
> {
>   "class" : "org.apache.solr.ltr.model.URIExternalModel",
>   "name" : "myURIExternalModel",
>   "features" : [],
>   "params" : {
>     "uri" : "file:///var/models/myModel.json"
>   }
> }
> {code}
> If you use LTR with {{model=myURIExternalModel}}, the model of {{myModel.json}} will be used for scoring documents.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org