You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Dariusz Wojtas <dw...@gmail.com> on 2018/01/12 14:40:37 UTC
LTR and working with feature stores
Hi,
I am working with the LTR rescoring.
Works beautifully, but I am curious about something.
How do I specify the feature store in a way different than using the
[features] syntax?
[features store=yourFeatureStore]
I have a range of models in my custom feature store, with plenty of
features implemented.
I have found that when I call LTR with model using only two features, Solr
still executes them all.
My setup in solrconfig.xml
-
<str name="fl">id,score,why_score:[explain style=nl],[features
store=store_incidentDB]</str>
<str name="rq">{!ltr reRankDocs=$reRankDocs model=simpleModelA}</str>
--
simpleModel above only uses LinearModel with 2 features.
What do I see in results?
In response I can see it has executed ALL features (there are values
calculated) in section:
1) -> response -> result -> doc -> <str name="[features]">HERE</str>
In addition, there is my model executed and only TWO features of the
executed model are presented in:
2) -> response -> debug -> explain
Why do I see all features being executed, if the specified model only
contains two features?
I tried to reduce 'fl' to:
<str name="fl">id,score,why_score:[explain style=nl]</str>
and id works as expected then:
1. additional features are not executed (correct)
2. my model works, only two features of the selected model (correct)
And the final questions for this long email are:
1. why does it execute all features when i specify 'store'?
2. how do I specify the 'store', if I have more stores, but do not want to
execute all their features?
Best regards,
Dariusz Wojtas
Re: LTR and working with feature stores
Posted by Dariusz Wojtas <dw...@gmail.com>.
Ups,
Diego, I have just read your answer again.
Now I see that it is [features] element that triggers calculation of all
store features.
That gives hope model only executed the features it needs ;)
Best regards,
Dariusz Wojtas
On Sat, Jan 13, 2018 at 11:12 PM, Dariusz Wojtas <dw...@gmail.com> wrote:
> Hi,
>
> Thanks for the response, I understand that all features from the given
> store are calculated, no matter if they are used or not.
> OK, spread features across different models.
> But what if different models share some features?
> Creating copies of feature definitions in different stores, one per model,
> is erroneous ...
> Having several models in one store, some of them use only part of these
> features - that seems 'expensive' ;)
>
> Simple syntax evolution would be very helpful, to give {!ltr} optional
> 'store' parameter. It could override the current features store, is
> specified.
> {!ltr reRankDocs=25 store=storeA model=simpleModelA}
>
> And {!ltr} executes 'model based calculation', not 'store based
> calculation'. Model knows what featues are required.
> Why are all features executed?
>
> Best regards,
> Dariusz Wojtas
>
>
> On Sat, Jan 13, 2018 at 4:03 PM, Diego Ceccarelli <
> diego.ceccarelli@gmail.com> wrote:
>
>> Hi Dariusz,
>>
>> On Jan 12, 2018 14:40, "Dariusz Wojtas" <dw...@gmail.com> wrote:
>>
>> Hi,
>>
>> I am working with the LTR rescoring.
>> Works beautifully, but I am curious about something.
>> How do I specify the feature store in a way different than using the
>> [features] syntax?
>> [features store=yourFeatureStore]
>>
>>
>>
>> What is the problem with this syntax? If the problem is the name of the
>> field, you can also call it by doing fl=title,authors,myfield=[features
>> store=yourFeatureStore]
>> I can't think of alternative ways..
>>
>>
>>
>> I have a range of models in my custom feature store, with plenty of
>> features implemented.
>> I have found that when I call LTR with model using only two features, Solr
>> still executes them all.
>>
>> My setup in solrconfig.xml
>> -
>> <str name="fl">id,score,why_score:[explain style=nl],[features
>> store=store_incidentDB]</str>
>> <str name="rq">{!ltr reRankDocs=$reRankDocs model=simpleModelA}</str>
>> --
>>
>> simpleModel above only uses LinearModel with 2 features.
>>
>>
>> What do I see in results?
>> In response I can see it has executed ALL features (there are values
>> calculated) in section:
>> 1) -> response -> result -> doc -> <str name="[features]">HERE</str>
>>
>> In addition, there is my model executed and only TWO features of the
>> executed model are presented in:
>>
>>
>> It is intended, the reason is that usually you want to execute your model
>> and at the same time log a *superset* of the features to train the next
>> model. If you want to compute only the features of the model you can
>> define
>> a featureStore that matches exactly the features that you have in the
>> model.
>>
>> 2) -> response -> debug -> explain
>>
>> Why do I see all features being executed, if the specified model only
>> contains two features?
>>
>> I tried to reduce 'fl' to:
>> <str name="fl">id,score,why_score:[explain style=nl]</str>
>> and id works as expected then:
>> 1. additional features are not executed (correct)
>> 2. my model works, only two features of the selected model (correct)
>>
>> And the final questions for this long email are:
>> 1. why does it execute all features when i specify 'store'?
>> 2. how do I specify the 'store', if I have more stores, but do not want to
>> execute all their features?
>>
>>
>> Just define a feature store that matches the features that you have in the
>> model. Please note that the featureStore that you specify in fl=
>> [features]
>> field **will not** affect the reranking (the model will compute only the
>> features that are specified in the model json file), you should ask for
>> the
>> [features] only if you want to log them.
>> Please do not hesitate to ask if something is not clear ;)
>>
>> Cheers,
>> Diego
>>
>>
>>
>> Best regards,
>> Dariusz Wojtas
>>
>
>
Re: LTR and working with feature stores
Posted by Dariusz Wojtas <dw...@gmail.com>.
Hi,
Thanks for the response, I understand that all features from the given
store are calculated, no matter if they are used or not.
OK, spread features across different models.
But what if different models share some features?
Creating copies of feature definitions in different stores, one per model,
is erroneous ...
Having several models in one store, some of them use only part of these
features - that seems 'expensive' ;)
Simple syntax evolution would be very helpful, to give {!ltr} optional
'store' parameter. It could override the current features store, is
specified.
{!ltr reRankDocs=25 store=storeA model=simpleModelA}
And {!ltr} executes 'model based calculation', not 'store based
calculation'. Model knows what featues are required.
Why are all features executed?
Best regards,
Dariusz Wojtas
On Sat, Jan 13, 2018 at 4:03 PM, Diego Ceccarelli <
diego.ceccarelli@gmail.com> wrote:
> Hi Dariusz,
>
> On Jan 12, 2018 14:40, "Dariusz Wojtas" <dw...@gmail.com> wrote:
>
> Hi,
>
> I am working with the LTR rescoring.
> Works beautifully, but I am curious about something.
> How do I specify the feature store in a way different than using the
> [features] syntax?
> [features store=yourFeatureStore]
>
>
>
> What is the problem with this syntax? If the problem is the name of the
> field, you can also call it by doing fl=title,authors,myfield=[features
> store=yourFeatureStore]
> I can't think of alternative ways..
>
>
>
> I have a range of models in my custom feature store, with plenty of
> features implemented.
> I have found that when I call LTR with model using only two features, Solr
> still executes them all.
>
> My setup in solrconfig.xml
> -
> <str name="fl">id,score,why_score:[explain style=nl],[features
> store=store_incidentDB]</str>
> <str name="rq">{!ltr reRankDocs=$reRankDocs model=simpleModelA}</str>
> --
>
> simpleModel above only uses LinearModel with 2 features.
>
>
> What do I see in results?
> In response I can see it has executed ALL features (there are values
> calculated) in section:
> 1) -> response -> result -> doc -> <str name="[features]">HERE</str>
>
> In addition, there is my model executed and only TWO features of the
> executed model are presented in:
>
>
> It is intended, the reason is that usually you want to execute your model
> and at the same time log a *superset* of the features to train the next
> model. If you want to compute only the features of the model you can define
> a featureStore that matches exactly the features that you have in the
> model.
>
> 2) -> response -> debug -> explain
>
> Why do I see all features being executed, if the specified model only
> contains two features?
>
> I tried to reduce 'fl' to:
> <str name="fl">id,score,why_score:[explain style=nl]</str>
> and id works as expected then:
> 1. additional features are not executed (correct)
> 2. my model works, only two features of the selected model (correct)
>
> And the final questions for this long email are:
> 1. why does it execute all features when i specify 'store'?
> 2. how do I specify the 'store', if I have more stores, but do not want to
> execute all their features?
>
>
> Just define a feature store that matches the features that you have in the
> model. Please note that the featureStore that you specify in fl= [features]
> field **will not** affect the reranking (the model will compute only the
> features that are specified in the model json file), you should ask for the
> [features] only if you want to log them.
> Please do not hesitate to ask if something is not clear ;)
>
> Cheers,
> Diego
>
>
>
> Best regards,
> Dariusz Wojtas
>
Re: LTR and working with feature stores
Posted by Diego Ceccarelli <di...@gmail.com>.
Hi Dariusz,
On Jan 12, 2018 14:40, "Dariusz Wojtas" <dw...@gmail.com> wrote:
Hi,
I am working with the LTR rescoring.
Works beautifully, but I am curious about something.
How do I specify the feature store in a way different than using the
[features] syntax?
[features store=yourFeatureStore]
What is the problem with this syntax? If the problem is the name of the
field, you can also call it by doing fl=title,authors,myfield=[features
store=yourFeatureStore]
I can't think of alternative ways..
I have a range of models in my custom feature store, with plenty of
features implemented.
I have found that when I call LTR with model using only two features, Solr
still executes them all.
My setup in solrconfig.xml
-
<str name="fl">id,score,why_score:[explain style=nl],[features
store=store_incidentDB]</str>
<str name="rq">{!ltr reRankDocs=$reRankDocs model=simpleModelA}</str>
--
simpleModel above only uses LinearModel with 2 features.
What do I see in results?
In response I can see it has executed ALL features (there are values
calculated) in section:
1) -> response -> result -> doc -> <str name="[features]">HERE</str>
In addition, there is my model executed and only TWO features of the
executed model are presented in:
It is intended, the reason is that usually you want to execute your model
and at the same time log a *superset* of the features to train the next
model. If you want to compute only the features of the model you can define
a featureStore that matches exactly the features that you have in the model.
2) -> response -> debug -> explain
Why do I see all features being executed, if the specified model only
contains two features?
I tried to reduce 'fl' to:
<str name="fl">id,score,why_score:[explain style=nl]</str>
and id works as expected then:
1. additional features are not executed (correct)
2. my model works, only two features of the selected model (correct)
And the final questions for this long email are:
1. why does it execute all features when i specify 'store'?
2. how do I specify the 'store', if I have more stores, but do not want to
execute all their features?
Just define a feature store that matches the features that you have in the
model. Please note that the featureStore that you specify in fl= [features]
field **will not** affect the reranking (the model will compute only the
features that are specified in the model json file), you should ask for the
[features] only if you want to log them.
Please do not hesitate to ask if something is not clear ;)
Cheers,
Diego
Best regards,
Dariusz Wojtas