You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@predictionio.apache.org by Lin Amy <li...@gmail.com> on 2016/12/16 15:07:14 UTC
A/B test on recommendation engine
Hello everyone,
I want to do some A/B test on the algorithm, and have found the following
method:
Run two engines in parallel (with different port of course), with the same
appid.
However, I want to use some load balance to redirect the traffic from
clients to the engines, and also tracking the recommendation algorithm (in
order to analyse the result). And here the question comes: is there any way
to modify the http response for query? (So that I can track the query
result)
Thank you for any advice!
Best regards,
Amy
Re: A/B test on recommendation engine
Posted by Gustavo Frederico <gu...@thinkwrap.com>.
An alternative to changing the template would be to put a 'proxy app'
between each PIO instance and the client, perhaps using Express/NodeJS,
that attaches the instance ID to the response. The benefit is that you
don't have to change the template, so upgrades and maintenance are easier.
Gustavo
On Sun, Dec 18, 2016 at 8:16 AM, Lin Amy <li...@gmail.com> wrote:
> Yes I am using universal recommender template, and thank you for helping
> me solve it!
>
> Best regards,
> Amy
>
> Pat Ferrel <pa...@occamsmachete.com> 於 2016年12月18日 週日 上午3:53寫道：
>
>> What template are you using? The results are usually formatted in
>> Engine.scala like the Universal Recommender here: https://github.com/
>> actionml/universal-recommender/blob/master/src/
>> main/scala/Engine.scala#L65 I think you can just add a string ID to each
>> template that tells the version that returns results. This seems a bit
>> heavy handed but if it’s the only thing you can do—so be it.
>>
>>
>>
>> On Dec 17, 2016, at 8:00 AM, Lin Amy <li...@gmail.com> wrote:
>>
>> Because there will be a load-balancer, so I would like to modify the
>> Serving results.
>> I'll try to figure it out, thanks!
>>
>> Best regards,
>> Amy
>>
>> Pat Ferrel <pa...@occamsmachete.com> 於 2016年12月17日 週六 上午5:59寫道：
>>
>> Amy asked: "is there any way to modify the http response for query? (So
>> that I can track the query result)"
>>
>> What do you want to modify? You know which one you queried from the port
>> number. The longer answer is that to modify the result or the response body
>> you will need to modify the template you are using.
>>
>>
>>
>> On Dec 16, 2016, at 7:07 AM, Lin Amy <li...@gmail.com> wrote:
>>
>> Hello everyone,
>>
>> I want to do some A/B test on the algorithm, and have found the following
>> method:
>> Run two engines in parallel (with different port of course), with the
>> same appid.
>>
>> However, I want to use some load balance to redirect the traffic from
>> clients to the engines, and also tracking the recommendation algorithm (in
>> order to analyse the result). And here the question comes: is there any way
>> to modify the http response for query? (So that I can track the query
>> result)
>>
>> Thank you for any advice!
>>
>> Best regards,
>> Amy
>>
>>
>>
Re: A/B test on recommendation engine
Posted by Lin Amy <li...@gmail.com>.
Yes I am using universal recommender template, and thank you for helping me
solve it!
Best regards,
Amy
Pat Ferrel <pa...@occamsmachete.com> 於 2016年12月18日 週日 上午3:53寫道：
> What template are you using? The results are usually formatted in
> Engine.scala like the Universal Recommender here:
> https://github.com/actionml/universal-recommender/blob/master/src/main/scala/Engine.scala#L65 I
> think you can just add a string ID to each template that tells the version
> that returns results. This seems a bit heavy handed but if it’s the only
> thing you can do—so be it.
>
>
>
> On Dec 17, 2016, at 8:00 AM, Lin Amy <li...@gmail.com> wrote:
>
> Because there will be a load-balancer, so I would like to modify the
> Serving results.
> I'll try to figure it out, thanks!
>
> Best regards,
> Amy
>
> Pat Ferrel <pa...@occamsmachete.com> 於 2016年12月17日 週六 上午5:59寫道：
>
> Amy asked: "is there any way to modify the http response for query? (So
> that I can track the query result)"
>
> What do you want to modify? You know which one you queried from the port
> number. The longer answer is that to modify the result or the response body
> you will need to modify the template you are using.
>
>
>
> On Dec 16, 2016, at 7:07 AM, Lin Amy <li...@gmail.com> wrote:
>
> Hello everyone,
>
> I want to do some A/B test on the algorithm, and have found the following
> method:
> Run two engines in parallel (with different port of course), with the same
> appid.
>
> However, I want to use some load balance to redirect the traffic from
> clients to the engines, and also tracking the recommendation algorithm (in
> order to analyse the result). And here the question comes: is there any way
> to modify the http response for query? (So that I can track the query
> result)
>
> Thank you for any advice!
>
> Best regards,
> Amy
>
>
>
Re: A/B test on recommendation engine
Posted by Pat Ferrel <pa...@occamsmachete.com>.
What template are you using? The results are usually formatted in Engine.scala like the Universal Recommender here: https://github.com/actionml/universal-recommender/blob/master/src/main/scala/Engine.scala#L65 <https://github.com/actionml/universal-recommender/blob/master/src/main/scala/Engine.scala#L65> I think you can just add a string ID to each template that tells the version that returns results. This seems a bit heavy handed but if it’s the only thing you can do—so be it.
On Dec 17, 2016, at 8:00 AM, Lin Amy <li...@gmail.com> wrote:
Because there will be a load-balancer, so I would like to modify the Serving results.
I'll try to figure it out, thanks!
Best regards,
Amy
Pat Ferrel <pat@occamsmachete.com <ma...@occamsmachete.com>> 於 2016年12月17日 週六 上午5:59寫道：
Amy asked: "is there any way to modify the http response for query? (So that I can track the query result)"
What do you want to modify? You know which one you queried from the port number. The longer answer is that to modify the result or the response body you will need to modify the template you are using.
On Dec 16, 2016, at 7:07 AM, Lin Amy <linamy85@gmail.com <ma...@gmail.com>> wrote:
Hello everyone,
I want to do some A/B test on the algorithm, and have found the following method:
Run two engines in parallel (with different port of course), with the same appid.
However, I want to use some load balance to redirect the traffic from clients to the engines, and also tracking the recommendation algorithm (in order to analyse the result). And here the question comes: is there any way to modify the http response for query? (So that I can track the query result)
Thank you for any advice!
Best regards,
Amy
Re: A/B test on recommendation engine
Posted by Lin Amy <li...@gmail.com>.
Because there will be a load-balancer, so I would like to modify the
Serving results.
I'll try to figure it out, thanks!
Best regards,
Amy
Pat Ferrel <pa...@occamsmachete.com> 於 2016年12月17日 週六 上午5:59寫道：
Amy asked: "is there any way to modify the http response for query? (So
that I can track the query result)"
What do you want to modify? You know which one you queried from the port
number. The longer answer is that to modify the result or the response body
you will need to modify the template you are using.
On Dec 16, 2016, at 7:07 AM, Lin Amy <li...@gmail.com> wrote:
Hello everyone,
I want to do some A/B test on the algorithm, and have found the following
method:
Run two engines in parallel (with different port of course), with the same
appid.
However, I want to use some load balance to redirect the traffic from
clients to the engines, and also tracking the recommendation algorithm (in
order to analyse the result). And here the question comes: is there any way
to modify the http response for query? (So that I can track the query
result)
Thank you for any advice!
Best regards,
Amy
Re: A/B test on recommendation engine
Posted by Pat Ferrel <pa...@occamsmachete.com>.
Amy asked: "is there any way to modify the http response for query? (So that I can track the query result)"
What do you want to modify? You know which one you queried from the port number. The longer answer is that to modify the result or the response body you will need to modify the template you are using.
On Dec 16, 2016, at 7:07 AM, Lin Amy <li...@gmail.com> wrote:
Hello everyone,
I want to do some A/B test on the algorithm, and have found the following method:
Run two engines in parallel (with different port of course), with the same appid.
However, I want to use some load balance to redirect the traffic from clients to the engines, and also tracking the recommendation algorithm (in order to analyse the result). And here the question comes: is there any way to modify the http response for query? (So that I can track the query result)
Thank you for any advice!
Best regards,
Amy