You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@mahout.apache.org by Bala Rajagopal <ba...@hotmail.com> on 2011/08/19 21:39:01 UTC
Recommending items with temporal restrictions
Hi,
My team is working on building a recommendation system to recommend items for the following use cases:1. Based on User similarity (using org.apache.mahout.cf.taste.hadoop.item.RecommenderJob as the Base)2. Based on item similarity
The part where it gets tricky is that we have a temporal restriction on our items (they are valid only for x days). So in the ideal case, the recommender should/can use the rating information on all our historical items, but will never recommend any items that are not temporally available. Based on the historical rating information, we need the list of best matches from the temporally available items.
Apart from ideas that involve any pre/post processing activities to filter temporally invalid item recommendations, we were reaching out to find if somebody out here has ever dealt with a similar requirement and has found an easier solution to deal with this edge case.
Any piece of advice, word of caution or streak of brilliance is more than welcome.
Thanks in advance.
Bala
Re: Recommending items with temporal restrictions
Posted by Sean Owen <sr...@gmail.com>.
Look at AggregateAndRecommendReducer, which is the final stage in the
distributed item-based computation. Look at the end of the
reduceBooleanData() / reduceNonBooleanData() methods. This is where
the final results are output. You can just cause it to skip any items
you don't care about.
I think you could do this check a little earlier, in a prior stage,
and save some computation. But this simple change would definitely
work, as a start.
On Mon, Aug 22, 2011 at 4:05 PM, Varnit Khanna <va...@gmail.com> wrote:
> On Sat, Aug 20, 2011 at 2:31 AM, Sean Owen <sr...@gmail.com> wrote:
>> The non-distributed code handles this case with Rescorers. There is no
>> equivalent in the distributed implementation.
>>
>> However it's fairly easy to hack this into the code. You just need to
>> modify the final stage where recommendations are computed to reject
>> items that are not new enough. They will have been used for similarity
>> calculations already, but then you can filter them out of
>> recommendations here.
> What do you mean by "final stage"? Did you mean have a post process
> which rejects old items produced by RecommenderJob or modify one of
> steps in RecommenderJob? If you meant the latter can you provide any
> documentation?
>
> Thanks
> -varnit
>
Re: Recommending items with temporal restrictions
Posted by Varnit Khanna <va...@gmail.com>.
On Sat, Aug 20, 2011 at 2:31 AM, Sean Owen <sr...@gmail.com> wrote:
> The non-distributed code handles this case with Rescorers. There is no
> equivalent in the distributed implementation.
>
> However it's fairly easy to hack this into the code. You just need to
> modify the final stage where recommendations are computed to reject
> items that are not new enough. They will have been used for similarity
> calculations already, but then you can filter them out of
> recommendations here.
What do you mean by "final stage"? Did you mean have a post process
which rejects old items produced by RecommenderJob or modify one of
steps in RecommenderJob? If you meant the latter can you provide any
documentation?
Thanks
-varnit
Re: Recommending items with temporal restrictions
Posted by Sean Owen <sr...@gmail.com>.
The non-distributed code handles this case with Rescorers. There is no
equivalent in the distributed implementation.
However it's fairly easy to hack this into the code. You just need to
modify the final stage where recommendations are computed to reject
items that are not new enough. They will have been used for similarity
calculations already, but then you can filter them out of
recommendations here.
Sean
On Fri, Aug 19, 2011 at 8:39 PM, Bala Rajagopal <ba...@hotmail.com> wrote:
>
> Hi,
> My team is working on building a recommendation system to recommend items for the following use cases:1. Based on User similarity (using org.apache.mahout.cf.taste.hadoop.item.RecommenderJob as the Base)2. Based on item similarity
> The part where it gets tricky is that we have a temporal restriction on our items (they are valid only for x days). So in the ideal case, the recommender should/can use the rating information on all our historical items, but will never recommend any items that are not temporally available. Based on the historical rating information, we need the list of best matches from the temporally available items.
> Apart from ideas that involve any pre/post processing activities to filter temporally invalid item recommendations, we were reaching out to find if somebody out here has ever dealt with a similar requirement and has found an easier solution to deal with this edge case.
> Any piece of advice, word of caution or streak of brilliance is more than welcome.
> Thanks in advance.
> Bala
RE: Recommending items with temporal restrictions
Posted by Bala Rajagopal <ba...@hotmail.com>.
Thanks Lance,
For your golden words " Algorithms are seductive to programmers". Can't tell you how many papers we have read to find out if we "fit into" any of them.
In response to your question, Yes, we do maintain a history of who (users) likes what (items) in the form of their ratings. That would go in as input to the algorithm to make the recommendation. That was what I meant when I wrote "Based on the historical rating information, we need the list of best matches from the temporally available items". So we are looking to use the historical "taste" data to make recommendations on the new items.
Bala
> Date: Sat, 20 Aug 2011 17:39:57 -0700
> Subject: Re: Recommending items with temporal restrictions
> From: goksron@gmail.com
> To: user@mahout.apache.org
>
> Do you keep history of user & item actions? Even if an item is old, it
> has a correlation with users who are still active. User A,B,C buy X. X
> goes out of date. Users A,B,C buy Y. Y goes out of date. Users A,B buy
> Z. Should you recommend Z to C?
>
> Algorithms are seductive to programmers. Start simple, understand your
> data and use a feedback loop to watch your responses. After this, you
> can pick&choose algorithms based on what makes sense in your context.
>
> On Sat, Aug 20, 2011 at 2:30 AM, Danny Bickson <da...@gmail.com> wrote:
> > I advise taking a lot in some of the related papers:
> > A) Liang Xiong, Xi Chen, Tzu-Kuo Huang, Jeff Schneider, Jaime G.
> > Carbonell, Temporal Collaborative Filtering with Bayesian
> > Probabilistic Tensor Factorization. In Proceedings of SIAM Data
> > Mining, 2010.
> > B) Yehuda Koren. Collaborative Filtering with Temporal Dynamics.
> > http://research.yahoo.com/files/kdd-fp074-koren.pdf
> > C) Yahoo! Music Recommendations: Modeling Music Ratings with Temporal
> > Dynamics and Item Taxonomy. Gideon Dror, Noam Koenigstein and Yehuda
> > Koren
> > ACM Conference on Recommender Systems (RecSys), 2011
> >)
> > All of the above papers bin ratings into time slots, and have the
> > flexibility to support temporal effects. In other words, the linear
> > model can learn availability of items per time bins
> > and not recommend items that do not exist in a certain time. (I assume
> > that item availability can be mapped to discrete time bins).
> >
> > Hope this helps,
> >
> > DB
> >
> >>
> >> Hi,
> >> My team is working on building a recommendation system to recommend items for the following use cases:1. Based on User similarity (using org.apache.mahout.cf.taste.hadoop.item.RecommenderJob as the Base)2. Based on item similarity
> >> The part where it gets tricky is that we have a temporal restriction on our items (they are valid only for x days). So in the ideal case, the recommender should/can use the rating information on all our historical items, but will never recommend any items that are not temporally available. Based on the historical rating information, we need the list of best matches from the temporally available items.
> >> Apart from ideas that involve any pre/post processing activities to filter temporally invalid item recommendations, we were reaching out to find if somebody out here has ever dealt with a similar requirement and has found an easier solution to deal with this edge case.
> >> Any piece of advice, word of caution or streak of brilliance is more than welcome.
> >> Thanks in advance.
> >> Bala
> >
>
>
>
> --
> Lance Norskog
> goksron@gmail.com
Re: Recommending items with temporal restrictions
Posted by Lance Norskog <go...@gmail.com>.
Do you keep history of user & item actions? Even if an item is old, it
has a correlation with users who are still active. User A,B,C buy X. X
goes out of date. Users A,B,C buy Y. Y goes out of date. Users A,B buy
Z. Should you recommend Z to C?
Algorithms are seductive to programmers. Start simple, understand your
data and use a feedback loop to watch your responses. After this, you
can pick&choose algorithms based on what makes sense in your context.
On Sat, Aug 20, 2011 at 2:30 AM, Danny Bickson <da...@gmail.com> wrote:
> I advise taking a lot in some of the related papers:
> A) Liang Xiong, Xi Chen, Tzu-Kuo Huang, Jeff Schneider, Jaime G.
> Carbonell, Temporal Collaborative Filtering with Bayesian
> Probabilistic Tensor Factorization. In Proceedings of SIAM Data
> Mining, 2010.
> B) Yehuda Koren. Collaborative Filtering with Temporal Dynamics.
> http://research.yahoo.com/files/kdd-fp074-koren.pdf
> C) Yahoo! Music Recommendations: Modeling Music Ratings with Temporal
> Dynamics and Item Taxonomy. Gideon Dror, Noam Koenigstein and Yehuda
> Koren
> ACM Conference on Recommender Systems (RecSys), 2011
>)
> All of the above papers bin ratings into time slots, and have the
> flexibility to support temporal effects. In other words, the linear
> model can learn availability of items per time bins
> and not recommend items that do not exist in a certain time. (I assume
> that item availability can be mapped to discrete time bins).
>
> Hope this helps,
>
> DB
>
>>
>> Hi,
>> My team is working on building a recommendation system to recommend items for the following use cases:1. Based on User similarity (using org.apache.mahout.cf.taste.hadoop.item.RecommenderJob as the Base)2. Based on item similarity
>> The part where it gets tricky is that we have a temporal restriction on our items (they are valid only for x days). So in the ideal case, the recommender should/can use the rating information on all our historical items, but will never recommend any items that are not temporally available. Based on the historical rating information, we need the list of best matches from the temporally available items.
>> Apart from ideas that involve any pre/post processing activities to filter temporally invalid item recommendations, we were reaching out to find if somebody out here has ever dealt with a similar requirement and has found an easier solution to deal with this edge case.
>> Any piece of advice, word of caution or streak of brilliance is more than welcome.
>> Thanks in advance.
>> Bala
>
--
Lance Norskog
goksron@gmail.com
RE: Recommending items with temporal restrictions
Posted by Bala Rajagopal <ba...@hotmail.com>.
Thanks DB,
We will add these to our "have to read" list. We are hoping to get feedback before we start tuning the recommendation algorithm to get better results. So we were looking out for an "easy" button that we can use to put this out in the wilderness to get some feedback. I am sure this would come in handy when we get to the tune-up phase.
Thanks,
Bala
> From: danny.bickson@gmail.com
> Date: Sat, 20 Aug 2011 12:30:14 +0300
> Subject: Re: Recommending items with temporal restrictions
> To: user@mahout.apache.org
>
> I advise taking a lot in some of the related papers:
> A) Liang Xiong, Xi Chen, Tzu-Kuo Huang, Jeff Schneider, Jaime G.
> Carbonell, Temporal Collaborative Filtering with Bayesian
> Probabilistic Tensor Factorization. In Proceedings of SIAM Data
> Mining, 2010.
> B) Yehuda Koren. Collaborative Filtering with Temporal Dynamics.
> http://research.yahoo.com/files/kdd-fp074-koren.pdf
> C) Yahoo! Music Recommendations: Modeling Music Ratings with Temporal
> Dynamics and Item Taxonomy. Gideon Dror, Noam Koenigstein and Yehuda
> Koren
> ACM Conference on Recommender Systems (RecSys), 2011
>
> All of the above papers bin ratings into time slots, and have the
> flexibility to support temporal effects. In other words, the linear
> model can learn availability of items per time bins
> and not recommend items that do not exist in a certain time. (I assume
> that item availability can be mapped to discrete time bins).
>
> Hope this helps,
>
> DB
>
> >
> > Hi,
> > My team is working on building a recommendation system to recommend items for the following use cases:1. Based on User similarity (using org.apache.mahout.cf.taste.hadoop.item.RecommenderJob as the Base)2. Based on item similarity
> > The part where it gets tricky is that we have a temporal restriction on our items (they are valid only for x days). So in the ideal case, the recommender should/can use the rating information on all our historical items, but will never recommend any items that are not temporally available. Based on the historical rating information, we need the list of best matches from the temporally available items.
> > Apart from ideas that involve any pre/post processing activities to filter temporally invalid item recommendations, we were reaching out to find if somebody out here has ever dealt with a similar requirement and has found an easier solution to deal with this edge case.
> > Any piece of advice, word of caution or streak of brilliance is more than welcome.
> > Thanks in advance.
> > Bala
Re: Recommending items with temporal restrictions
Posted by Danny Bickson <da...@gmail.com>.
I advise taking a lot in some of the related papers:
A) Liang Xiong, Xi Chen, Tzu-Kuo Huang, Jeff Schneider, Jaime G.
Carbonell, Temporal Collaborative Filtering with Bayesian
Probabilistic Tensor Factorization. In Proceedings of SIAM Data
Mining, 2010.
B) Yehuda Koren. Collaborative Filtering with Temporal Dynamics.
http://research.yahoo.com/files/kdd-fp074-koren.pdf
C) Yahoo! Music Recommendations: Modeling Music Ratings with Temporal
Dynamics and Item Taxonomy. Gideon Dror, Noam Koenigstein and Yehuda
Koren
ACM Conference on Recommender Systems (RecSys), 2011
All of the above papers bin ratings into time slots, and have the
flexibility to support temporal effects. In other words, the linear
model can learn availability of items per time bins
and not recommend items that do not exist in a certain time. (I assume
that item availability can be mapped to discrete time bins).
Hope this helps,
DB
>
> Hi,
> My team is working on building a recommendation system to recommend items for the following use cases:1. Based on User similarity (using org.apache.mahout.cf.taste.hadoop.item.RecommenderJob as the Base)2. Based on item similarity
> The part where it gets tricky is that we have a temporal restriction on our items (they are valid only for x days). So in the ideal case, the recommender should/can use the rating information on all our historical items, but will never recommend any items that are not temporally available. Based on the historical rating information, we need the list of best matches from the temporally available items.
> Apart from ideas that involve any pre/post processing activities to filter temporally invalid item recommendations, we were reaching out to find if somebody out here has ever dealt with a similar requirement and has found an easier solution to deal with this edge case.
> Any piece of advice, word of caution or streak of brilliance is more than welcome.
> Thanks in advance.
> Bala