You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@mahout.apache.org by Tim Smith <ti...@hotmail.com> on 2014/01/11 02:27:20 UTC
Item recommendation w/o users or preferences
Say I have a retail organization that doesn't sell a diverse set of products, eg 2000, but has many small transactions. Also say that I don't have any user or preference information. Is it reasonable to use pattern mining (market baskets) and recommend items based on a set of thresholds for support, confidence, and lift? If not, what are my options?
RE: Item recommendation w/o users or preferences
Posted by Rachel Owsley <Ra...@safeway.com>.
No, Tim. Sorry, if I was not clear. I was referring to the probability that someone would purchase an item, given all the previous items they have purchased. A/B testing is a different subject. If you have any further questions, please feel free to send me an email.
Best,
Rachel
________________________________________
From: Tim Smith [timsmith_s@hotmail.com]
Sent: Friday, January 10, 2014 8:05 PM
To: user@mahout.apache.org
Subject: RE: Item recommendation w/o users or preferences
Very awesome, thank you! I am twisting the support knob right now!
Sequential analysis in the sense that I A/B test my recommendations and feed the conversion rates back into my next set of recommendations or something else?
> From: Rachel.Owsley@safeway.com
> To: user@mahout.apache.org
> Subject: RE: Item recommendation w/o users or preferences
> Date: Sat, 11 Jan 2014 03:53:41 +0000
>
> My mail crossed with yours. Try market basket analysis and sequential analysis. With the market basket analysis, there are often a lot of frequent basket combinations that are not that useful. You may want to lower the support to get some more infrequent combinations, but up the confidence level.
>
> Good luck.
>
> Rachel
> ________________________________________
> From: Tim Smith [timsmith_s@hotmail.com]
> Sent: Friday, January 10, 2014 7:39 PM
> To: user@mahout.apache.org
> Subject: RE: Item recommendation w/o users or preferences
>
> Yes, thank you - read through it and several of the item and user recommendation examples. The objective is to recommend based on the current basket - given no users/preferences (but I do have a history of transactions) - I have been able to leverage the item mining algorithm to calculate support and confidence values. When I use a support threshold of 10% and group by product and sort descending on confidence I am left we a ranking of item combos. Basically a top N list by item that I would use to drive the recommendations. In the actual use case, the requirement is not to recommend a product every time, rather the most likely products based on a given basket - with my arbitrary thresholds, I would expect to exclude some baskets.
>
> > From: NiMartin@pssd.com
> > To: user@mahout.apache.org
> > Subject: RE: Item recommendation w/o users or preferences
> > Date: Sat, 11 Jan 2014 03:08:30 +0000
> >
> > I think the key question is what is the desired outcome? If you don't have users (customers) for which you'd like to generate recommendations that really handcuffs you from a recommendation standpoint.
> >
> > I'd recommend starting with a read through this: http://mahout.apache.org/users/recommender/recommender-first-timer-faq.html to get a feel for what Mahout does in the recommendation space.
> >
> > -----Original Message-----
> > From: Tim Smith [mailto:timsmith_s@hotmail.com]
> > Sent: Friday, January 10, 2014 8:27 PM
> > To: user@mahout.apache.org
> > Subject: Item recommendation w/o users or preferences
> >
> > Say I have a retail organization that doesn't sell a diverse set of products, eg 2000, but has many small transactions. Also say that I don't have any user or preference information. Is it reasonable to use pattern mining (market baskets) and recommend items based on a set of thresholds for support, confidence, and lift? If not, what are my options?
> >
>
> "Email Firewall" made the following annotations.
> ------------------------------------------------------------------------------
>
> Warning:
> All e-mail sent to this address will be received by the corporate e-mail system, and is subject to archival and review by someone other than the recipient. This e-mail may contain proprietary information and is intended only for the use of the intended recipient(s). If the reader of this message is not the intended recipient(s), you are notified that you have received this message in error and that any review, dissemination, distribution or copying of this message is strictly prohibited. If you have received this message in error, please notify the sender immediately.
>
> ==============================================================================
>
"Email Firewall" made the following annotations.
------------------------------------------------------------------------------
Warning:
All e-mail sent to this address will be received by the corporate e-mail system, and is subject to archival and review by someone other than the recipient. This e-mail may contain proprietary information and is intended only for the use of the intended recipient(s). If the reader of this message is not the intended recipient(s), you are notified that you have received this message in error and that any review, dissemination, distribution or copying of this message is strictly prohibited. If you have received this message in error, please notify the sender immediately.
==============================================================================
RE: Item recommendation w/o users or preferences
Posted by Rachel Owsley <Ra...@safeway.com>.
Thanks, Tim. Any time.
Best,
Rachel
________________________________________
From: Ted Dunning [ted.dunning@gmail.com]
Sent: Friday, January 10, 2014 8:09 PM
To: user@mahout.apache.org
Subject: Re: Item recommendation w/o users or preferences
This talk of support and overlap smacks of very poor coocccurrence analysis.
See http://tdunning.blogspot.com/2008/03/surprise-and-coincidence.html for
a better option.
On Fri, Jan 10, 2014 at 8:05 PM, Tim Smith <ti...@hotmail.com> wrote:
> Very awesome, thank you! I am twisting the support knob right now!
>
> Sequential analysis in the sense that I A/B test my recommendations and
> feed the conversion rates back into my next set of recommendations or
> something else?
>
> > From: Rachel.Owsley@safeway.com
> > To: user@mahout.apache.org
> > Subject: RE: Item recommendation w/o users or preferences
> > Date: Sat, 11 Jan 2014 03:53:41 +0000
> >
> > My mail crossed with yours. Try market basket analysis and sequential
> analysis. With the market basket analysis, there are often a lot of
> frequent basket combinations that are not that useful. You may want to
> lower the support to get some more infrequent combinations, but up the
> confidence level.
> >
> > Good luck.
> >
> > Rachel
> > ________________________________________
> > From: Tim Smith [timsmith_s@hotmail.com]
> > Sent: Friday, January 10, 2014 7:39 PM
> > To: user@mahout.apache.org
> > Subject: RE: Item recommendation w/o users or preferences
> >
> > Yes, thank you - read through it and several of the item and user
> recommendation examples. The objective is to recommend based on the
> current basket - given no users/preferences (but I do have a history of
> transactions) - I have been able to leverage the item mining algorithm to
> calculate support and confidence values. When I use a support threshold of
> 10% and group by product and sort descending on confidence I am left we a
> ranking of item combos. Basically a top N list by item that I would use to
> drive the recommendations. In the actual use case, the requirement is not
> to recommend a product every time, rather the most likely products based on
> a given basket - with my arbitrary thresholds, I would expect to exclude
> some baskets.
> >
> > > From: NiMartin@pssd.com
> > > To: user@mahout.apache.org
> > > Subject: RE: Item recommendation w/o users or preferences
> > > Date: Sat, 11 Jan 2014 03:08:30 +0000
> > >
> > > I think the key question is what is the desired outcome? If you don't
> have users (customers) for which you'd like to generate recommendations
> that really handcuffs you from a recommendation standpoint.
> > >
> > > I'd recommend starting with a read through this:
> http://mahout.apache.org/users/recommender/recommender-first-timer-faq.htmlto get a feel for what Mahout does in the recommendation space.
> > >
> > > -----Original Message-----
> > > From: Tim Smith [mailto:timsmith_s@hotmail.com]
> > > Sent: Friday, January 10, 2014 8:27 PM
> > > To: user@mahout.apache.org
> > > Subject: Item recommendation w/o users or preferences
> > >
> > > Say I have a retail organization that doesn't sell a diverse set of
> products, eg 2000, but has many small transactions. Also say that I don't
> have any user or preference information. Is it reasonable to use pattern
> mining (market baskets) and recommend items based on a set of thresholds
> for support, confidence, and lift? If not, what are my options?
> > >
> >
> > "Email Firewall" made the following annotations.
> >
> ------------------------------------------------------------------------------
> >
> > Warning:
> > All e-mail sent to this address will be received by the corporate e-mail
> system, and is subject to archival and review by someone other than the
> recipient. This e-mail may contain proprietary information and is intended
> only for the use of the intended recipient(s). If the reader of this
> message is not the intended recipient(s), you are notified that you have
> received this message in error and that any review, dissemination,
> distribution or copying of this message is strictly prohibited. If you
> have received this message in error, please notify the sender immediately.
> >
> >
> ==============================================================================
> >
>
>
"Email Firewall" made the following annotations.
------------------------------------------------------------------------------
Warning:
All e-mail sent to this address will be received by the corporate e-mail system, and is subject to archival and review by someone other than the recipient. This e-mail may contain proprietary information and is intended only for the use of the intended recipient(s). If the reader of this message is not the intended recipient(s), you are notified that you have received this message in error and that any review, dissemination, distribution or copying of this message is strictly prohibited. If you have received this message in error, please notify the sender immediately.
==============================================================================
Re: Item recommendation w/o users or preferences
Posted by Johannes Schulte <jo...@gmail.com>.
Hey,
since you are already using basket analysis terms like support, confidence
and lift it might be easier for you to think of the llr score as a "better
lift" since it automatically puts a penalty on seldom items (you usually
use support in classic mba for that).
So, you would use the same 4 counts you use for lift calculation
(cooccurrence,2*marginals, totals), and you do this for every combination
of items that occur in a basket
On Sat, Jan 11, 2014 at 10:38 PM, Tim Smith <ti...@hotmail.com> wrote:
> > Is it about how to arrange your data to use this computation? The
> > references below might help with that.
>
> Yes, I read and tried the recommendation examples from MIA and there is a
> mention of item to item similarity, but I am not sure what form the file
> should take. The examples are along the lines of userid,itemid,value
>
> In section 6.2 of MIA we are multiplying the Co-occur matrix X User
> preferences = Recommendations (top of page 97), so if I do not have
> preferences should
> I just default them all to the same value? Taken together with your
> previous comments, is this how I should be preparing my data?
>
> Raw Sample Data (format: Transaction|Item)
> 123|Sun Glasses
> 124|Sun Glasses
> 124|Sun Glass Case
> 125|Sun Glass Case
> 126|Sun Glasses
> 126|Glass Repair Kit
> 127|Glass Repair Kit
>
> Are you suggesting that I just simply use (format: userid|item|value)
> 123|Sun Glasses|1
> 124|Sun Glasses|1
> 124|Sun Glass Case|1
> 125|Sun Glass Case|1
> 126|Sun Glasses|1
> 126|Glass Repair Kit|1
> 127|Glass Repair Kit|1
>
> > Is it regarding the specifics of how you do the computation? I can help
> > with that, but would need a pointer to the difficulty.
>
> Not quite yet. I am working through the intuition first, I'll fight
> through the math once, if ever, the fog clears
>
Re: Item recommendation w/o users or preferences
Posted by Pat Ferrel <pa...@occamsmachete.com>.
Haven’t read the whole thread but it sounds like you just need some simple start-here info…
To do collaborative filtering you must have user-id, item-id, action/weight.
For a minimal commerce CF cooccurrence recommender this is typically something like:
user-id, item-id, 1=purchase
To use Mahout you will have to translate the ids into positive integers. Treat these like keys to an item-id or user-id lookup. So input into Mahout will be:
user-id-key, item-id-key, 1
You can do CF with anonymous user-ids, meaning an individual took the action but you don’t know who. However to use this data you will have to have some way of tying an id to a real person. Using transactions ids as a proxy for user-ids will work in the training data but once you want to make a recommendation you will have to know some real user history to allow the recommender to compare it with transactions.
Then you calculate recommendations using some Mahout recommender. If you are using the hadoop version the output will be a row per user-id-key that will contain some number of recommendation item-id-keys and their recommendation weight for sorting purposes. You then write your own retrieval code to get the recs for a given user-id-key, since they are all pre-calculated and in a Sequence File. If you are using the in-memory recommender you can ask for recs for a given user-id-key and get the list returned.
You can also use transaction data alone to make anonymous recommendations, but that is market basket analysis. In that case you have:
transaction-id-key, item-id-key, 1
Then at recommendation time you have a list of items in a single basket. There are several ways to get this to work so I’ll stop here unless it’s what you need, in which case let us know.
On Jan 11, 2014, at 1:38 PM, Tim Smith <ti...@hotmail.com> wrote:
> Is it about how to arrange your data to use this computation? The
> references below might help with that.
Yes, I read and tried the recommendation examples from MIA and there is a mention of item to item similarity, but I am not sure what form the file should take. The examples are along the lines of userid,itemid,value
In section 6.2 of MIA we are multiplying the Co-occur matrix X User
preferences = Recommendations (top of page 97), so if I do not have preferences should
I just default them all to the same value? Taken together with your previous comments, is this how I should be preparing my data?
Raw Sample Data (format: Transaction|Item)
123|Sun Glasses
124|Sun Glasses
124|Sun Glass Case
125|Sun Glass Case
126|Sun Glasses
126|Glass Repair Kit
127|Glass Repair Kit
Are you suggesting that I just simply use (format: userid|item|value)
123|Sun Glasses|1
124|Sun Glasses|1
124|Sun Glass Case|1
125|Sun Glass Case|1
126|Sun Glasses|1
126|Glass Repair Kit|1
127|Glass Repair Kit|1
> Is it regarding the specifics of how you do the computation? I can help
> with that, but would need a pointer to the difficulty.
Not quite yet. I am working through the intuition first, I'll fight through the math once, if ever, the fog clears
RE: Item recommendation w/o users or preferences
Posted by Tim Smith <ti...@hotmail.com>.
> Is it about how to arrange your data to use this computation? The
> references below might help with that.
Yes, I read and tried the recommendation examples from MIA and there is a mention of item to item similarity, but I am not sure what form the file should take. The examples are along the lines of userid,itemid,value
In section 6.2 of MIA we are multiplying the Co-occur matrix X User
preferences = Recommendations (top of page 97), so if I do not have preferences should
I just default them all to the same value? Taken together with your previous comments, is this how I should be preparing my data?
Raw Sample Data (format: Transaction|Item)
123|Sun Glasses
124|Sun Glasses
124|Sun Glass Case
125|Sun Glass Case
126|Sun Glasses
126|Glass Repair Kit
127|Glass Repair Kit
Are you suggesting that I just simply use (format: userid|item|value)
123|Sun Glasses|1
124|Sun Glasses|1
124|Sun Glass Case|1
125|Sun Glass Case|1
126|Sun Glasses|1
126|Glass Repair Kit|1
127|Glass Repair Kit|1
> Is it regarding the specifics of how you do the computation? I can help
> with that, but would need a pointer to the difficulty.
Not quite yet. I am working through the intuition first, I'll fight through the math once, if ever, the fog clears
Re: Item recommendation w/o users or preferences
Posted by Ted Dunning <te...@gmail.com>.
Tim,
Can you be more specific about which bits are missing?
Is it about the rationale for the log-likelihood ratio test? If so, that
rationale is simply that the algorithm is simple and empirically has been
shown to produce excellent results across a large array of applications
(over a thousand references in the literature now).
Is it regarding the specifics of how you do the computation? I can help
with that, but would need a pointer to the difficulty.
Is it about how to arrange your data to use this computation? The
references below might help with that.
Is it about the use of indicators for recommendations? If so, here are
references that might make that more clear:
http://www.linkedin.com/in/sebastianschelter
http://ssc.io/wp-content/uploads/2013/02/cf-mahout.pdf
http://isabel-drost.de/hadoop/slides/collabMahout.pdf
http://www.slideshare.net/tdunning/building-multimodal-recommendation-engines-using-search-engines
https://www.youtube.com/watch?v=fWR1T2pY08Y
On Sat, Jan 11, 2014 at 7:05 AM, Tim Smith <ti...@hotmail.com> wrote:
> Ted - I read your blog post and the paper you refer to/wrote and several
> other authors that sited your paper, but I am having trouble with the
> overall approach for putting all the pieces together. Would you mind
> giving me a nudge with the intuition? I realize you have been doing this
> for almost 30 years; however, I am just a cave man... Thank you.
>
> > From: ted.dunning@gmail.com
> > Date: Fri, 10 Jan 2014 20:09:38 -0800
> > Subject: Re: Item recommendation w/o users or preferences
> > To: user@mahout.apache.org
> >
> > This talk of support and overlap smacks of very poor coocccurrence
> analysis.
> >
> > See http://tdunning.blogspot.com/2008/03/surprise-and-coincidence.htmlfor
> > a better option.
> >
> >
> > On Fri, Jan 10, 2014 at 8:05 PM, Tim Smith <ti...@hotmail.com>
> wrote:
> >
> > > Very awesome, thank you! I am twisting the support knob right now!
> > >
> > > Sequential analysis in the sense that I A/B test my recommendations and
> > > feed the conversion rates back into my next set of recommendations or
> > > something else?
> > >
> > > > From: Rachel.Owsley@safeway.com
> > > > To: user@mahout.apache.org
> > > > Subject: RE: Item recommendation w/o users or preferences
> > > > Date: Sat, 11 Jan 2014 03:53:41 +0000
> > > >
> > > > My mail crossed with yours. Try market basket analysis and sequential
> > > analysis. With the market basket analysis, there are often a lot of
> > > frequent basket combinations that are not that useful. You may want to
> > > lower the support to get some more infrequent combinations, but up the
> > > confidence level.
> > > >
> > > > Good luck.
> > > >
> > > > Rachel
> > > > ________________________________________
> > > > From: Tim Smith [timsmith_s@hotmail.com]
> > > > Sent: Friday, January 10, 2014 7:39 PM
> > > > To: user@mahout.apache.org
> > > > Subject: RE: Item recommendation w/o users or preferences
> > > >
> > > > Yes, thank you - read through it and several of the item and user
> > > recommendation examples. The objective is to recommend based on the
> > > current basket - given no users/preferences (but I do have a history of
> > > transactions) - I have been able to leverage the item mining algorithm
> to
> > > calculate support and confidence values. When I use a support
> threshold of
> > > 10% and group by product and sort descending on confidence I am left
> we a
> > > ranking of item combos. Basically a top N list by item that I would
> use to
> > > drive the recommendations. In the actual use case, the requirement is
> not
> > > to recommend a product every time, rather the most likely products
> based on
> > > a given basket - with my arbitrary thresholds, I would expect to
> exclude
> > > some baskets.
> > > >
> > > > > From: NiMartin@pssd.com
> > > > > To: user@mahout.apache.org
> > > > > Subject: RE: Item recommendation w/o users or preferences
> > > > > Date: Sat, 11 Jan 2014 03:08:30 +0000
> > > > >
> > > > > I think the key question is what is the desired outcome? If you
> don't
> > > have users (customers) for which you'd like to generate recommendations
> > > that really handcuffs you from a recommendation standpoint.
> > > > >
> > > > > I'd recommend starting with a read through this:
> > >
> http://mahout.apache.org/users/recommender/recommender-first-timer-faq.htmltoget a feel for what Mahout does in the recommendation space.
> > > > >
> > > > > -----Original Message-----
> > > > > From: Tim Smith [mailto:timsmith_s@hotmail.com]
> > > > > Sent: Friday, January 10, 2014 8:27 PM
> > > > > To: user@mahout.apache.org
> > > > > Subject: Item recommendation w/o users or preferences
> > > > >
> > > > > Say I have a retail organization that doesn't sell a diverse set of
> > > products, eg 2000, but has many small transactions. Also say that I
> don't
> > > have any user or preference information. Is it reasonable to use
> pattern
> > > mining (market baskets) and recommend items based on a set of
> thresholds
> > > for support, confidence, and lift? If not, what are my options?
> > > > >
> > > >
> > > > "Email Firewall" made the following annotations.
> > > >
> > >
> ------------------------------------------------------------------------------
> > > >
> > > > Warning:
> > > > All e-mail sent to this address will be received by the corporate
> e-mail
> > > system, and is subject to archival and review by someone other than the
> > > recipient. This e-mail may contain proprietary information and is
> intended
> > > only for the use of the intended recipient(s). If the reader of this
> > > message is not the intended recipient(s), you are notified that you
> have
> > > received this message in error and that any review, dissemination,
> > > distribution or copying of this message is strictly prohibited. If you
> > > have received this message in error, please notify the sender
> immediately.
> > > >
> > > >
> > >
> ==============================================================================
> > > >
> > >
> > >
>
>
RE: Item recommendation w/o users or preferences
Posted by Tim Smith <ti...@hotmail.com>.
Ted - I read your blog post and the paper you refer to/wrote and several other authors that sited your paper, but I am having trouble with the overall approach for putting all the pieces together. Would you mind giving me a nudge with the intuition? I realize you have been doing this for almost 30 years; however, I am just a cave man... Thank you.
> From: ted.dunning@gmail.com
> Date: Fri, 10 Jan 2014 20:09:38 -0800
> Subject: Re: Item recommendation w/o users or preferences
> To: user@mahout.apache.org
>
> This talk of support and overlap smacks of very poor coocccurrence analysis.
>
> See http://tdunning.blogspot.com/2008/03/surprise-and-coincidence.html for
> a better option.
>
>
> On Fri, Jan 10, 2014 at 8:05 PM, Tim Smith <ti...@hotmail.com> wrote:
>
> > Very awesome, thank you! I am twisting the support knob right now!
> >
> > Sequential analysis in the sense that I A/B test my recommendations and
> > feed the conversion rates back into my next set of recommendations or
> > something else?
> >
> > > From: Rachel.Owsley@safeway.com
> > > To: user@mahout.apache.org
> > > Subject: RE: Item recommendation w/o users or preferences
> > > Date: Sat, 11 Jan 2014 03:53:41 +0000
> > >
> > > My mail crossed with yours. Try market basket analysis and sequential
> > analysis. With the market basket analysis, there are often a lot of
> > frequent basket combinations that are not that useful. You may want to
> > lower the support to get some more infrequent combinations, but up the
> > confidence level.
> > >
> > > Good luck.
> > >
> > > Rachel
> > > ________________________________________
> > > From: Tim Smith [timsmith_s@hotmail.com]
> > > Sent: Friday, January 10, 2014 7:39 PM
> > > To: user@mahout.apache.org
> > > Subject: RE: Item recommendation w/o users or preferences
> > >
> > > Yes, thank you - read through it and several of the item and user
> > recommendation examples. The objective is to recommend based on the
> > current basket - given no users/preferences (but I do have a history of
> > transactions) - I have been able to leverage the item mining algorithm to
> > calculate support and confidence values. When I use a support threshold of
> > 10% and group by product and sort descending on confidence I am left we a
> > ranking of item combos. Basically a top N list by item that I would use to
> > drive the recommendations. In the actual use case, the requirement is not
> > to recommend a product every time, rather the most likely products based on
> > a given basket - with my arbitrary thresholds, I would expect to exclude
> > some baskets.
> > >
> > > > From: NiMartin@pssd.com
> > > > To: user@mahout.apache.org
> > > > Subject: RE: Item recommendation w/o users or preferences
> > > > Date: Sat, 11 Jan 2014 03:08:30 +0000
> > > >
> > > > I think the key question is what is the desired outcome? If you don't
> > have users (customers) for which you'd like to generate recommendations
> > that really handcuffs you from a recommendation standpoint.
> > > >
> > > > I'd recommend starting with a read through this:
> > http://mahout.apache.org/users/recommender/recommender-first-timer-faq.htmlto get a feel for what Mahout does in the recommendation space.
> > > >
> > > > -----Original Message-----
> > > > From: Tim Smith [mailto:timsmith_s@hotmail.com]
> > > > Sent: Friday, January 10, 2014 8:27 PM
> > > > To: user@mahout.apache.org
> > > > Subject: Item recommendation w/o users or preferences
> > > >
> > > > Say I have a retail organization that doesn't sell a diverse set of
> > products, eg 2000, but has many small transactions. Also say that I don't
> > have any user or preference information. Is it reasonable to use pattern
> > mining (market baskets) and recommend items based on a set of thresholds
> > for support, confidence, and lift? If not, what are my options?
> > > >
> > >
> > > "Email Firewall" made the following annotations.
> > >
> > ------------------------------------------------------------------------------
> > >
> > > Warning:
> > > All e-mail sent to this address will be received by the corporate e-mail
> > system, and is subject to archival and review by someone other than the
> > recipient. This e-mail may contain proprietary information and is intended
> > only for the use of the intended recipient(s). If the reader of this
> > message is not the intended recipient(s), you are notified that you have
> > received this message in error and that any review, dissemination,
> > distribution or copying of this message is strictly prohibited. If you
> > have received this message in error, please notify the sender immediately.
> > >
> > >
> > ==============================================================================
> > >
> >
> >
RE: Item recommendation w/o users or preferences
Posted by Tim Smith <ti...@hotmail.com>.
Thank you Ted. Excellent reference. Back to the drawing board...
> From: ted.dunning@gmail.com
> Date: Fri, 10 Jan 2014 20:09:38 -0800
> Subject: Re: Item recommendation w/o users or preferences
> To: user@mahout.apache.org
>
> This talk of support and overlap smacks of very poor coocccurrence analysis.
>
> See http://tdunning.blogspot.com/2008/03/surprise-and-coincidence.html for
> a better option.
>
>
> On Fri, Jan 10, 2014 at 8:05 PM, Tim Smith <ti...@hotmail.com> wrote:
>
> > Very awesome, thank you! I am twisting the support knob right now!
> >
> > Sequential analysis in the sense that I A/B test my recommendations and
> > feed the conversion rates back into my next set of recommendations or
> > something else?
> >
> > > From: Rachel.Owsley@safeway.com
> > > To: user@mahout.apache.org
> > > Subject: RE: Item recommendation w/o users or preferences
> > > Date: Sat, 11 Jan 2014 03:53:41 +0000
> > >
> > > My mail crossed with yours. Try market basket analysis and sequential
> > analysis. With the market basket analysis, there are often a lot of
> > frequent basket combinations that are not that useful. You may want to
> > lower the support to get some more infrequent combinations, but up the
> > confidence level.
> > >
> > > Good luck.
> > >
> > > Rachel
> > > ________________________________________
> > > From: Tim Smith [timsmith_s@hotmail.com]
> > > Sent: Friday, January 10, 2014 7:39 PM
> > > To: user@mahout.apache.org
> > > Subject: RE: Item recommendation w/o users or preferences
> > >
> > > Yes, thank you - read through it and several of the item and user
> > recommendation examples. The objective is to recommend based on the
> > current basket - given no users/preferences (but I do have a history of
> > transactions) - I have been able to leverage the item mining algorithm to
> > calculate support and confidence values. When I use a support threshold of
> > 10% and group by product and sort descending on confidence I am left we a
> > ranking of item combos. Basically a top N list by item that I would use to
> > drive the recommendations. In the actual use case, the requirement is not
> > to recommend a product every time, rather the most likely products based on
> > a given basket - with my arbitrary thresholds, I would expect to exclude
> > some baskets.
> > >
> > > > From: NiMartin@pssd.com
> > > > To: user@mahout.apache.org
> > > > Subject: RE: Item recommendation w/o users or preferences
> > > > Date: Sat, 11 Jan 2014 03:08:30 +0000
> > > >
> > > > I think the key question is what is the desired outcome? If you don't
> > have users (customers) for which you'd like to generate recommendations
> > that really handcuffs you from a recommendation standpoint.
> > > >
> > > > I'd recommend starting with a read through this:
> > http://mahout.apache.org/users/recommender/recommender-first-timer-faq.htmlto get a feel for what Mahout does in the recommendation space.
> > > >
> > > > -----Original Message-----
> > > > From: Tim Smith [mailto:timsmith_s@hotmail.com]
> > > > Sent: Friday, January 10, 2014 8:27 PM
> > > > To: user@mahout.apache.org
> > > > Subject: Item recommendation w/o users or preferences
> > > >
> > > > Say I have a retail organization that doesn't sell a diverse set of
> > products, eg 2000, but has many small transactions. Also say that I don't
> > have any user or preference information. Is it reasonable to use pattern
> > mining (market baskets) and recommend items based on a set of thresholds
> > for support, confidence, and lift? If not, what are my options?
> > > >
> > >
> > > "Email Firewall" made the following annotations.
> > >
> > ------------------------------------------------------------------------------
> > >
> > > Warning:
> > > All e-mail sent to this address will be received by the corporate e-mail
> > system, and is subject to archival and review by someone other than the
> > recipient. This e-mail may contain proprietary information and is intended
> > only for the use of the intended recipient(s). If the reader of this
> > message is not the intended recipient(s), you are notified that you have
> > received this message in error and that any review, dissemination,
> > distribution or copying of this message is strictly prohibited. If you
> > have received this message in error, please notify the sender immediately.
> > >
> > >
> > ==============================================================================
> > >
> >
> >
Re: Item recommendation w/o users or preferences
Posted by Ted Dunning <te...@gmail.com>.
This talk of support and overlap smacks of very poor coocccurrence analysis.
See http://tdunning.blogspot.com/2008/03/surprise-and-coincidence.html for
a better option.
On Fri, Jan 10, 2014 at 8:05 PM, Tim Smith <ti...@hotmail.com> wrote:
> Very awesome, thank you! I am twisting the support knob right now!
>
> Sequential analysis in the sense that I A/B test my recommendations and
> feed the conversion rates back into my next set of recommendations or
> something else?
>
> > From: Rachel.Owsley@safeway.com
> > To: user@mahout.apache.org
> > Subject: RE: Item recommendation w/o users or preferences
> > Date: Sat, 11 Jan 2014 03:53:41 +0000
> >
> > My mail crossed with yours. Try market basket analysis and sequential
> analysis. With the market basket analysis, there are often a lot of
> frequent basket combinations that are not that useful. You may want to
> lower the support to get some more infrequent combinations, but up the
> confidence level.
> >
> > Good luck.
> >
> > Rachel
> > ________________________________________
> > From: Tim Smith [timsmith_s@hotmail.com]
> > Sent: Friday, January 10, 2014 7:39 PM
> > To: user@mahout.apache.org
> > Subject: RE: Item recommendation w/o users or preferences
> >
> > Yes, thank you - read through it and several of the item and user
> recommendation examples. The objective is to recommend based on the
> current basket - given no users/preferences (but I do have a history of
> transactions) - I have been able to leverage the item mining algorithm to
> calculate support and confidence values. When I use a support threshold of
> 10% and group by product and sort descending on confidence I am left we a
> ranking of item combos. Basically a top N list by item that I would use to
> drive the recommendations. In the actual use case, the requirement is not
> to recommend a product every time, rather the most likely products based on
> a given basket - with my arbitrary thresholds, I would expect to exclude
> some baskets.
> >
> > > From: NiMartin@pssd.com
> > > To: user@mahout.apache.org
> > > Subject: RE: Item recommendation w/o users or preferences
> > > Date: Sat, 11 Jan 2014 03:08:30 +0000
> > >
> > > I think the key question is what is the desired outcome? If you don't
> have users (customers) for which you'd like to generate recommendations
> that really handcuffs you from a recommendation standpoint.
> > >
> > > I'd recommend starting with a read through this:
> http://mahout.apache.org/users/recommender/recommender-first-timer-faq.htmlto get a feel for what Mahout does in the recommendation space.
> > >
> > > -----Original Message-----
> > > From: Tim Smith [mailto:timsmith_s@hotmail.com]
> > > Sent: Friday, January 10, 2014 8:27 PM
> > > To: user@mahout.apache.org
> > > Subject: Item recommendation w/o users or preferences
> > >
> > > Say I have a retail organization that doesn't sell a diverse set of
> products, eg 2000, but has many small transactions. Also say that I don't
> have any user or preference information. Is it reasonable to use pattern
> mining (market baskets) and recommend items based on a set of thresholds
> for support, confidence, and lift? If not, what are my options?
> > >
> >
> > "Email Firewall" made the following annotations.
> >
> ------------------------------------------------------------------------------
> >
> > Warning:
> > All e-mail sent to this address will be received by the corporate e-mail
> system, and is subject to archival and review by someone other than the
> recipient. This e-mail may contain proprietary information and is intended
> only for the use of the intended recipient(s). If the reader of this
> message is not the intended recipient(s), you are notified that you have
> received this message in error and that any review, dissemination,
> distribution or copying of this message is strictly prohibited. If you
> have received this message in error, please notify the sender immediately.
> >
> >
> ==============================================================================
> >
>
>
RE: Item recommendation w/o users or preferences
Posted by Tim Smith <ti...@hotmail.com>.
Very awesome, thank you! I am twisting the support knob right now!
Sequential analysis in the sense that I A/B test my recommendations and feed the conversion rates back into my next set of recommendations or something else?
> From: Rachel.Owsley@safeway.com
> To: user@mahout.apache.org
> Subject: RE: Item recommendation w/o users or preferences
> Date: Sat, 11 Jan 2014 03:53:41 +0000
>
> My mail crossed with yours. Try market basket analysis and sequential analysis. With the market basket analysis, there are often a lot of frequent basket combinations that are not that useful. You may want to lower the support to get some more infrequent combinations, but up the confidence level.
>
> Good luck.
>
> Rachel
> ________________________________________
> From: Tim Smith [timsmith_s@hotmail.com]
> Sent: Friday, January 10, 2014 7:39 PM
> To: user@mahout.apache.org
> Subject: RE: Item recommendation w/o users or preferences
>
> Yes, thank you - read through it and several of the item and user recommendation examples. The objective is to recommend based on the current basket - given no users/preferences (but I do have a history of transactions) - I have been able to leverage the item mining algorithm to calculate support and confidence values. When I use a support threshold of 10% and group by product and sort descending on confidence I am left we a ranking of item combos. Basically a top N list by item that I would use to drive the recommendations. In the actual use case, the requirement is not to recommend a product every time, rather the most likely products based on a given basket - with my arbitrary thresholds, I would expect to exclude some baskets.
>
> > From: NiMartin@pssd.com
> > To: user@mahout.apache.org
> > Subject: RE: Item recommendation w/o users or preferences
> > Date: Sat, 11 Jan 2014 03:08:30 +0000
> >
> > I think the key question is what is the desired outcome? If you don't have users (customers) for which you'd like to generate recommendations that really handcuffs you from a recommendation standpoint.
> >
> > I'd recommend starting with a read through this: http://mahout.apache.org/users/recommender/recommender-first-timer-faq.html to get a feel for what Mahout does in the recommendation space.
> >
> > -----Original Message-----
> > From: Tim Smith [mailto:timsmith_s@hotmail.com]
> > Sent: Friday, January 10, 2014 8:27 PM
> > To: user@mahout.apache.org
> > Subject: Item recommendation w/o users or preferences
> >
> > Say I have a retail organization that doesn't sell a diverse set of products, eg 2000, but has many small transactions. Also say that I don't have any user or preference information. Is it reasonable to use pattern mining (market baskets) and recommend items based on a set of thresholds for support, confidence, and lift? If not, what are my options?
> >
>
> "Email Firewall" made the following annotations.
> ------------------------------------------------------------------------------
>
> Warning:
> All e-mail sent to this address will be received by the corporate e-mail system, and is subject to archival and review by someone other than the recipient. This e-mail may contain proprietary information and is intended only for the use of the intended recipient(s). If the reader of this message is not the intended recipient(s), you are notified that you have received this message in error and that any review, dissemination, distribution or copying of this message is strictly prohibited. If you have received this message in error, please notify the sender immediately.
>
> ==============================================================================
>
RE: Item recommendation w/o users or preferences
Posted by Rachel Owsley <Ra...@safeway.com>.
My mail crossed with yours. Try market basket analysis and sequential analysis. With the market basket analysis, there are often a lot of frequent basket combinations that are not that useful. You may want to lower the support to get some more infrequent combinations, but up the confidence level.
Good luck.
Rachel
________________________________________
From: Tim Smith [timsmith_s@hotmail.com]
Sent: Friday, January 10, 2014 7:39 PM
To: user@mahout.apache.org
Subject: RE: Item recommendation w/o users or preferences
Yes, thank you - read through it and several of the item and user recommendation examples. The objective is to recommend based on the current basket - given no users/preferences (but I do have a history of transactions) - I have been able to leverage the item mining algorithm to calculate support and confidence values. When I use a support threshold of 10% and group by product and sort descending on confidence I am left we a ranking of item combos. Basically a top N list by item that I would use to drive the recommendations. In the actual use case, the requirement is not to recommend a product every time, rather the most likely products based on a given basket - with my arbitrary thresholds, I would expect to exclude some baskets.
> From: NiMartin@pssd.com
> To: user@mahout.apache.org
> Subject: RE: Item recommendation w/o users or preferences
> Date: Sat, 11 Jan 2014 03:08:30 +0000
>
> I think the key question is what is the desired outcome? If you don't have users (customers) for which you'd like to generate recommendations that really handcuffs you from a recommendation standpoint.
>
> I'd recommend starting with a read through this: http://mahout.apache.org/users/recommender/recommender-first-timer-faq.html to get a feel for what Mahout does in the recommendation space.
>
> -----Original Message-----
> From: Tim Smith [mailto:timsmith_s@hotmail.com]
> Sent: Friday, January 10, 2014 8:27 PM
> To: user@mahout.apache.org
> Subject: Item recommendation w/o users or preferences
>
> Say I have a retail organization that doesn't sell a diverse set of products, eg 2000, but has many small transactions. Also say that I don't have any user or preference information. Is it reasonable to use pattern mining (market baskets) and recommend items based on a set of thresholds for support, confidence, and lift? If not, what are my options?
>
"Email Firewall" made the following annotations.
------------------------------------------------------------------------------
Warning:
All e-mail sent to this address will be received by the corporate e-mail system, and is subject to archival and review by someone other than the recipient. This e-mail may contain proprietary information and is intended only for the use of the intended recipient(s). If the reader of this message is not the intended recipient(s), you are notified that you have received this message in error and that any review, dissemination, distribution or copying of this message is strictly prohibited. If you have received this message in error, please notify the sender immediately.
==============================================================================
RE: Item recommendation w/o users or preferences
Posted by Tim Smith <ti...@hotmail.com>.
Yes, thank you - read through it and several of the item and user recommendation examples. The objective is to recommend based on the current basket - given no users/preferences (but I do have a history of transactions) - I have been able to leverage the item mining algorithm to calculate support and confidence values. When I use a support threshold of 10% and group by product and sort descending on confidence I am left we a ranking of item combos. Basically a top N list by item that I would use to drive the recommendations. In the actual use case, the requirement is not to recommend a product every time, rather the most likely products based on a given basket - with my arbitrary thresholds, I would expect to exclude some baskets.
> From: NiMartin@pssd.com
> To: user@mahout.apache.org
> Subject: RE: Item recommendation w/o users or preferences
> Date: Sat, 11 Jan 2014 03:08:30 +0000
>
> I think the key question is what is the desired outcome? If you don't have users (customers) for which you'd like to generate recommendations that really handcuffs you from a recommendation standpoint.
>
> I'd recommend starting with a read through this: http://mahout.apache.org/users/recommender/recommender-first-timer-faq.html to get a feel for what Mahout does in the recommendation space.
>
> -----Original Message-----
> From: Tim Smith [mailto:timsmith_s@hotmail.com]
> Sent: Friday, January 10, 2014 8:27 PM
> To: user@mahout.apache.org
> Subject: Item recommendation w/o users or preferences
>
> Say I have a retail organization that doesn't sell a diverse set of products, eg 2000, but has many small transactions. Also say that I don't have any user or preference information. Is it reasonable to use pattern mining (market baskets) and recommend items based on a set of thresholds for support, confidence, and lift? If not, what are my options?
>
RE: Item recommendation w/o users or preferences
Posted by "Martin, Nick" <Ni...@pssd.com>.
I think the key question is what is the desired outcome? If you don't have users (customers) for which you'd like to generate recommendations that really handcuffs you from a recommendation standpoint.
I'd recommend starting with a read through this: http://mahout.apache.org/users/recommender/recommender-first-timer-faq.html to get a feel for what Mahout does in the recommendation space.
-----Original Message-----
From: Tim Smith [mailto:timsmith_s@hotmail.com]
Sent: Friday, January 10, 2014 8:27 PM
To: user@mahout.apache.org
Subject: Item recommendation w/o users or preferences
Say I have a retail organization that doesn't sell a diverse set of products, eg 2000, but has many small transactions. Also say that I don't have any user or preference information. Is it reasonable to use pattern mining (market baskets) and recommend items based on a set of thresholds for support, confidence, and lift? If not, what are my options?
RE: Item recommendation w/o users or preferences
Posted by Rachel Owsley <Ra...@safeway.com>.
Thanks for the info. I think your best bet is market basket analysis and looking for the frequently bought baskets and strong relationships between items. A sequential analysis might also help. Neither one of these would be real-time, in that you'd have to already have the frequent itemsets generated and would just be pairing them with the POS items.
So not exactly a recommender in this case, but it might get you some lift-- and that's really all that matters. Yeah-- we do have cash customers, but having the loyalty card helps because knowing what the loyalty customers buy and tracking that history, you can infer to a certain extent what the cash customers may buy.
________________________________________
From: Tim Smith [timsmith_s@hotmail.com]
Sent: Friday, January 10, 2014 8:01 PM
To: user@mahout.apache.org
Subject: RE: Item recommendation w/o users or preferences
Excellent question. Given who you work for, just assume a customer comes into a retail location and goes to pay at the checkout. They do not identity themselves (no loyalty/club card) and use cash (trying to make the point that we have no idea who this consumer is right at this moment, and may never will). So rather than having Catalina print out coupons after the fact, say I want to make an offer right there at the POS during their transaction. I realize that this is a bit problematic at a grocery store, but our scenario has a clerk behind a counter with these items close at hand. So all I have is their current basket and the baskets of previous anonymous purchases. Clear?
> From: Rachel.Owsley@safeway.com
> To: user@mahout.apache.org
> Subject: RE: Item recommendation w/o users or preferences
> Date: Sat, 11 Jan 2014 03:49:53 +0000
>
> Hi Tim,
>
> By not having user or preference information, it's not clear to me-- do you mean you have no demographic information, but you have email or some IP address-- some way to track the user?
>
> It is possible to generate recommendations on purchase history, by looking at the user's transactions and inferring a preference from what they buy the most frequently. I used to work for a company that had transaction history, but it was anonymized-- all the user's activity was tied to an anonymous token. They didn't even have the name or gender. If you know a customer's card #, you could relate the card # as their "user_id" and use the count or monetary value of their transactions for a specific item as a preference for that item. Try something like conditional probability-- the probability that you will buy one thing given that you bought another. By generating a set of pairs (item a being the user has bought, and item b being the one they have not purchased), you can determine the probability that they will by item b, given that they bought item A.
>
> Still, if you know nothing about a person at all, and don't even have a way to distinguish them on your website, then recommendation won't really help much because how will you actually give the user recommendations? You could consider using market basket analysis to tell you what other items a person might put in his/her cart. I've done market basket analysis before. It is necessary to do a lot of "pruning" with market basket analysis, because a lot of the frequent pairs are not very useful. But through some careful analysis, you may find interesting combinations of items that will help your business in terms of cross selling/promotion. I am looking at sequential basket analysis right now. If I buy items x1 through x4, what is the probability that a certain item will be the next one? You might be able to use something market basket (fpgrowth) or maybe a markov model to determine the next item in sequence.
>
> Good luck with this. If you could share the type of data you do have available, it would be helpful.
>
> Rachel
> ________________________________________
> From: Tim Smith [timsmith_s@hotmail.com]
> Sent: Friday, January 10, 2014 5:27 PM
> To: user@mahout.apache.org
> Subject: Item recommendation w/o users or preferences
>
> Say I have a retail organization that doesn't sell a diverse set of products, eg 2000, but has many small transactions. Also say that I don't have any user or preference information. Is it reasonable to use pattern mining (market baskets) and recommend items based on a set of thresholds for support, confidence, and lift? If not, what are my options?
>
> "Email Firewall" made the following annotations.
> ------------------------------------------------------------------------------
>
> Warning:
> All e-mail sent to this address will be received by the corporate e-mail system, and is subject to archival and review by someone other than the recipient. This e-mail may contain proprietary information and is intended only for the use of the intended recipient(s). If the reader of this message is not the intended recipient(s), you are notified that you have received this message in error and that any review, dissemination, distribution or copying of this message is strictly prohibited. If you have received this message in error, please notify the sender immediately.
>
> ==============================================================================
>
"Email Firewall" made the following annotations.
------------------------------------------------------------------------------
Warning:
All e-mail sent to this address will be received by the corporate e-mail system, and is subject to archival and review by someone other than the recipient. This e-mail may contain proprietary information and is intended only for the use of the intended recipient(s). If the reader of this message is not the intended recipient(s), you are notified that you have received this message in error and that any review, dissemination, distribution or copying of this message is strictly prohibited. If you have received this message in error, please notify the sender immediately.
==============================================================================
Re: Item recommendation w/o users or preferences
Posted by Ted Dunning <te...@gmail.com>.
These are just short user histories.
Standard item-item cooccurrence analysis should work fine.
If you have any longer histories, that will help even more.
On Fri, Jan 10, 2014 at 8:01 PM, Tim Smith <ti...@hotmail.com> wrote:
> Excellent question. Given who you work for, just assume a customer comes
> into a retail location and goes to pay at the checkout. They do not
> identity themselves (no loyalty/club card) and use cash (trying to make the
> point that we have no idea who this consumer is right at this moment, and
> may never will). So rather than having Catalina print out coupons after
> the fact, say I want to make an offer right there at the POS during their
> transaction. I realize that this is a bit problematic at a grocery store,
> but our scenario has a clerk behind a counter with these items close at
> hand. So all I have is their current basket and the baskets of previous
> anonymous purchases. Clear?
>
> > From: Rachel.Owsley@safeway.com
> > To: user@mahout.apache.org
> > Subject: RE: Item recommendation w/o users or preferences
> > Date: Sat, 11 Jan 2014 03:49:53 +0000
> >
> > Hi Tim,
> >
> > By not having user or preference information, it's not clear to me-- do
> you mean you have no demographic information, but you have email or some IP
> address-- some way to track the user?
> >
> > It is possible to generate recommendations on purchase history, by
> looking at the user's transactions and inferring a preference from what
> they buy the most frequently. I used to work for a company that had
> transaction history, but it was anonymized-- all the user's activity was
> tied to an anonymous token. They didn't even have the name or gender. If
> you know a customer's card #, you could relate the card # as their
> "user_id" and use the count or monetary value of their transactions for a
> specific item as a preference for that item. Try something like conditional
> probability-- the probability that you will buy one thing given that you
> bought another. By generating a set of pairs (item a being the user has
> bought, and item b being the one they have not purchased), you can
> determine the probability that they will by item b, given that they bought
> item A.
> >
> > Still, if you know nothing about a person at all, and don't even have a
> way to distinguish them on your website, then recommendation won't really
> help much because how will you actually give the user recommendations? You
> could consider using market basket analysis to tell you what other items a
> person might put in his/her cart. I've done market basket analysis before.
> It is necessary to do a lot of "pruning" with market basket analysis,
> because a lot of the frequent pairs are not very useful. But through some
> careful analysis, you may find interesting combinations of items that will
> help your business in terms of cross selling/promotion. I am looking at
> sequential basket analysis right now. If I buy items x1 through x4, what is
> the probability that a certain item will be the next one? You might be able
> to use something market basket (fpgrowth) or maybe a markov model to
> determine the next item in sequence.
> >
> > Good luck with this. If you could share the type of data you do have
> available, it would be helpful.
> >
> > Rachel
> > ________________________________________
> > From: Tim Smith [timsmith_s@hotmail.com]
> > Sent: Friday, January 10, 2014 5:27 PM
> > To: user@mahout.apache.org
> > Subject: Item recommendation w/o users or preferences
> >
> > Say I have a retail organization that doesn't sell a diverse set of
> products, eg 2000, but has many small transactions. Also say that I don't
> have any user or preference information. Is it reasonable to use pattern
> mining (market baskets) and recommend items based on a set of thresholds
> for support, confidence, and lift? If not, what are my options?
> >
> > "Email Firewall" made the following annotations.
> >
> ------------------------------------------------------------------------------
> >
> > Warning:
> > All e-mail sent to this address will be received by the corporate e-mail
> system, and is subject to archival and review by someone other than the
> recipient. This e-mail may contain proprietary information and is intended
> only for the use of the intended recipient(s). If the reader of this
> message is not the intended recipient(s), you are notified that you have
> received this message in error and that any review, dissemination,
> distribution or copying of this message is strictly prohibited. If you
> have received this message in error, please notify the sender immediately.
> >
> >
> ==============================================================================
> >
>
>
RE: Item recommendation w/o users or preferences
Posted by Tim Smith <ti...@hotmail.com>.
Excellent question. Given who you work for, just assume a customer comes into a retail location and goes to pay at the checkout. They do not identity themselves (no loyalty/club card) and use cash (trying to make the point that we have no idea who this consumer is right at this moment, and may never will). So rather than having Catalina print out coupons after the fact, say I want to make an offer right there at the POS during their transaction. I realize that this is a bit problematic at a grocery store, but our scenario has a clerk behind a counter with these items close at hand. So all I have is their current basket and the baskets of previous anonymous purchases. Clear?
> From: Rachel.Owsley@safeway.com
> To: user@mahout.apache.org
> Subject: RE: Item recommendation w/o users or preferences
> Date: Sat, 11 Jan 2014 03:49:53 +0000
>
> Hi Tim,
>
> By not having user or preference information, it's not clear to me-- do you mean you have no demographic information, but you have email or some IP address-- some way to track the user?
>
> It is possible to generate recommendations on purchase history, by looking at the user's transactions and inferring a preference from what they buy the most frequently. I used to work for a company that had transaction history, but it was anonymized-- all the user's activity was tied to an anonymous token. They didn't even have the name or gender. If you know a customer's card #, you could relate the card # as their "user_id" and use the count or monetary value of their transactions for a specific item as a preference for that item. Try something like conditional probability-- the probability that you will buy one thing given that you bought another. By generating a set of pairs (item a being the user has bought, and item b being the one they have not purchased), you can determine the probability that they will by item b, given that they bought item A.
>
> Still, if you know nothing about a person at all, and don't even have a way to distinguish them on your website, then recommendation won't really help much because how will you actually give the user recommendations? You could consider using market basket analysis to tell you what other items a person might put in his/her cart. I've done market basket analysis before. It is necessary to do a lot of "pruning" with market basket analysis, because a lot of the frequent pairs are not very useful. But through some careful analysis, you may find interesting combinations of items that will help your business in terms of cross selling/promotion. I am looking at sequential basket analysis right now. If I buy items x1 through x4, what is the probability that a certain item will be the next one? You might be able to use something market basket (fpgrowth) or maybe a markov model to determine the next item in sequence.
>
> Good luck with this. If you could share the type of data you do have available, it would be helpful.
>
> Rachel
> ________________________________________
> From: Tim Smith [timsmith_s@hotmail.com]
> Sent: Friday, January 10, 2014 5:27 PM
> To: user@mahout.apache.org
> Subject: Item recommendation w/o users or preferences
>
> Say I have a retail organization that doesn't sell a diverse set of products, eg 2000, but has many small transactions. Also say that I don't have any user or preference information. Is it reasonable to use pattern mining (market baskets) and recommend items based on a set of thresholds for support, confidence, and lift? If not, what are my options?
>
> "Email Firewall" made the following annotations.
> ------------------------------------------------------------------------------
>
> Warning:
> All e-mail sent to this address will be received by the corporate e-mail system, and is subject to archival and review by someone other than the recipient. This e-mail may contain proprietary information and is intended only for the use of the intended recipient(s). If the reader of this message is not the intended recipient(s), you are notified that you have received this message in error and that any review, dissemination, distribution or copying of this message is strictly prohibited. If you have received this message in error, please notify the sender immediately.
>
> ==============================================================================
>
RE: Item recommendation w/o users or preferences
Posted by Rachel Owsley <Ra...@safeway.com>.
Hi Tim,
By not having user or preference information, it's not clear to me-- do you mean you have no demographic information, but you have email or some IP address-- some way to track the user?
It is possible to generate recommendations on purchase history, by looking at the user's transactions and inferring a preference from what they buy the most frequently. I used to work for a company that had transaction history, but it was anonymized-- all the user's activity was tied to an anonymous token. They didn't even have the name or gender. If you know a customer's card #, you could relate the card # as their "user_id" and use the count or monetary value of their transactions for a specific item as a preference for that item. Try something like conditional probability-- the probability that you will buy one thing given that you bought another. By generating a set of pairs (item a being the user has bought, and item b being the one they have not purchased), you can determine the probability that they will by item b, given that they bought item A.
Still, if you know nothing about a person at all, and don't even have a way to distinguish them on your website, then recommendation won't really help much because how will you actually give the user recommendations? You could consider using market basket analysis to tell you what other items a person might put in his/her cart. I've done market basket analysis before. It is necessary to do a lot of "pruning" with market basket analysis, because a lot of the frequent pairs are not very useful. But through some careful analysis, you may find interesting combinations of items that will help your business in terms of cross selling/promotion. I am looking at sequential basket analysis right now. If I buy items x1 through x4, what is the probability that a certain item will be the next one? You might be able to use something market basket (fpgrowth) or maybe a markov model to determine the next item in sequence.
Good luck with this. If you could share the type of data you do have available, it would be helpful.
Rachel
________________________________________
From: Tim Smith [timsmith_s@hotmail.com]
Sent: Friday, January 10, 2014 5:27 PM
To: user@mahout.apache.org
Subject: Item recommendation w/o users or preferences
Say I have a retail organization that doesn't sell a diverse set of products, eg 2000, but has many small transactions. Also say that I don't have any user or preference information. Is it reasonable to use pattern mining (market baskets) and recommend items based on a set of thresholds for support, confidence, and lift? If not, what are my options?
"Email Firewall" made the following annotations.
------------------------------------------------------------------------------
Warning:
All e-mail sent to this address will be received by the corporate e-mail system, and is subject to archival and review by someone other than the recipient. This e-mail may contain proprietary information and is intended only for the use of the intended recipient(s). If the reader of this message is not the intended recipient(s), you are notified that you have received this message in error and that any review, dissemination, distribution or copying of this message is strictly prohibited. If you have received this message in error, please notify the sender immediately.
==============================================================================
Re: Item recommendation w/o users or preferences
Posted by Ted Dunning <te...@gmail.com>.
Yes. Since each transaction contains several items, you might as well call
that a row in the history matrix and go from there to cooccurrence analysis
or matrix factorization (cooccurrence is easier and just as accurate if you
have enough data).
As Rachel mentions, you also can sometimes string together multiple
transactions using additional data such as loyalty programs. That just
makes things better. Your assumptions preclude this, but it is often
possible, nevertheless.
The goal is to make recommendations in real-time at the point of sale just
as you suggest.
On Fri, Jan 10, 2014 at 10:21 PM, Tim Smith <ti...@hotmail.com> wrote:
> Not sure I follow - are suggesting that the transaction could be thought
> of as a proxy for a user? Therefore, each transaction is essentially a
> user? And for what end?
>
> > From: ted.dunning@gmail.com
> > Date: Fri, 10 Jan 2014 20:06:43 -0800
> > Subject: Re: Item recommendation w/o users or preferences
> > To: user@mahout.apache.org
> >
> > How is it that you have many transactions and have no user information?
> >
> > I thought that transactions were user information?
> >
> >
> > On Fri, Jan 10, 2014 at 5:27 PM, Tim Smith <ti...@hotmail.com>
> wrote:
> >
> > > Say I have a retail organization that doesn't sell a diverse set of
> > > products, eg 2000, but has many small transactions. Also say that I
> don't
> > > have any user or preference information. Is it reasonable to use
> pattern
> > > mining (market baskets) and recommend items based on a set of
> thresholds
> > > for support, confidence, and lift? If not, what are my options?
> > >
>
RE: Item recommendation w/o users or preferences
Posted by Tim Smith <ti...@hotmail.com>.
Not sure I follow - are suggesting that the transaction could be thought of as a proxy for a user? Therefore, each transaction is essentially a user? And for what end?
> From: ted.dunning@gmail.com
> Date: Fri, 10 Jan 2014 20:06:43 -0800
> Subject: Re: Item recommendation w/o users or preferences
> To: user@mahout.apache.org
>
> How is it that you have many transactions and have no user information?
>
> I thought that transactions were user information?
>
>
> On Fri, Jan 10, 2014 at 5:27 PM, Tim Smith <ti...@hotmail.com> wrote:
>
> > Say I have a retail organization that doesn't sell a diverse set of
> > products, eg 2000, but has many small transactions. Also say that I don't
> > have any user or preference information. Is it reasonable to use pattern
> > mining (market baskets) and recommend items based on a set of thresholds
> > for support, confidence, and lift? If not, what are my options?
> >
Re: Item recommendation w/o users or preferences
Posted by Ted Dunning <te...@gmail.com>.
How is it that you have many transactions and have no user information?
I thought that transactions were user information?
On Fri, Jan 10, 2014 at 5:27 PM, Tim Smith <ti...@hotmail.com> wrote:
> Say I have a retail organization that doesn't sell a diverse set of
> products, eg 2000, but has many small transactions. Also say that I don't
> have any user or preference information. Is it reasonable to use pattern
> mining (market baskets) and recommend items based on a set of thresholds
> for support, confidence, and lift? If not, what are my options?
>