You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@mahout.apache.org by Pat Ferrel <pa...@gmail.com> on 2013/02/09 20:50:57 UTC
Implicit preferences
I'd like to experiment with using several types of implicit preference values with recommenders. I have purchases as an implicit pref of high strength. I'd like to see if add-to-cart, view-product-details, impressions-seen, etc. can increase offline precision in holdout tests. These less than obvious implicit prefs will get a much lower value than purchase and i'll experiment with different mixes. The problem is that some of these prefs will indicate that the user, for whom I'm getting recs, has expressed a preference.
Using these implicit prefs seems reasonable in finding similarity of taste between users but presents several problems. 1) how to encode the prefs, each impression-seen will increase the strength of preference of a user for an item but the recommender framework replaces the preference value for items preferred more than once, doesn't it? 2) AFAIK the current recommender framework will return recs only for items that the user in question has expressed no preference for. If I use something like view-product-details or impressions-seen, I will be removing anything the user has seen from the recs, which is not what I want in this experiment.
Has anyone tried something like this? I'm not convinced that these other implicit preferences will add anything to the recommender, just trying to find out.
Re: Implicit preferences
Posted by Pat Ferrel <pa...@gmail.com>.
What if I treated each item+pref-type as if they were separate items, separate dimensions? This would use weaker measures for user similarity along with strong purchase related prefs. But it avoids the problem of returning only recs for things with no expressed preference by the user in question. In fact I'd leave in the current filtering. I'd still get recs based on other item+pref-type dimensions but throwing them out and only returning item+purchase based recs would have the affect of filtering out things already purchased by the user.
Intuitively this seems to account for the affect: you looked at certain items and eventually purchased one and I looked at the same items so I might like what you purchased. It also seems to work better in the existing mahout framework.
On Feb 9, 2013, at 11:50 AM, Pat Ferrel <pa...@gmail.com> wrote:
I'd like to experiment with using several types of implicit preference values with recommenders. I have purchases as an implicit pref of high strength. I'd like to see if add-to-cart, view-product-details, impressions-seen, etc. can increase offline precision in holdout tests. These less than obvious implicit prefs will get a much lower value than purchase and i'll experiment with different mixes. The problem is that some of these prefs will indicate that the user, for whom I'm getting recs, has expressed a preference.
Using these implicit prefs seems reasonable in finding similarity of taste between users but presents several problems. 1) how to encode the prefs, each impression-seen will increase the strength of preference of a user for an item but the recommender framework replaces the preference value for items preferred more than once, doesn't it? 2) AFAIK the current recommender framework will return recs only for items that the user in question has expressed no preference for. If I use something like view-product-details or impressions-seen, I will be removing anything the user has seen from the recs, which is not what I want in this experiment.
Has anyone tried something like this? I'm not convinced that these other implicit preferences will add anything to the recommender, just trying to find out.
Re: Implicit preferences
Posted by Ken Krugler <kk...@transpac.com>.
On Feb 10, 2013, at 2:39pm, Johannes Schulte wrote:
> Hi,
>
> i am currently implementing a system of the same kind, LLR sparsified
> "term"-cooccurrence vectors in lucene (since not a day goes by where i see
> Ted praising this).
> There are not only views and purchases, but also search terms, facets and a
> lot more textual information to be included in the cooccurrence matrix (as
> "input").
> That's why i went with the feature hashing framework in mahout. This gives
> small (hd/mem) user profiles and allows for reusing the vectors for click
> prediction and/or clustering. The main difference is that there's only two
> fields in lucene with a lot of terms (Numbers), representing the features.
> Two fields because i think predicting views (besides purchases) might in
> some cases be better than predicting nothing.
> I don't think it should make a big differing in scoring because in a
> vector space model used by most engines it's just, well a vector space and
> i don't know if the field norm make sense after stripping values from the
> term vectors with the LLR threshold.
>
> @Ted
>> It is handy to simply use the binary values of the sparsified versions of
>> these and let the search engine handle the weighting of different
>> components at query time.
>
> Do you really want to omit the cooccurrence counts which would become the
> term frequecies? how would the engine then weight different inputs against
> each other?
>
> And,
> if anyone knows a
> 1. smarter way to index the cooccurrence counts in lucene than a
> tokenstream that emits a word k times for a cooccurrence count of k
I haven't been following this discussion, but in general using payloads is a way of providing additional information about a term that can be used for scoring.
> 2. way to avoid treating the (hashed) vector column indices as terms but
> reusing them? It's a bit weird hashing to an int and then having the lucene
> term dictionary treating them as string, mapping to another int
Is there a performance/size issue here?
Also I'm assuming you're using the solr.TrieIntField field type (not the string-ified value).
-- Ken
> On Sun, Feb 10, 2013 at 6:36 PM, Ted Dunning <te...@gmail.com> wrote:
>
>> Actually treating the different interactions separately can lead to very
>> good recommendations. The only issue is that the interactions are no
>> longer dyadic.
>>
>> If you think about it, having two different kinds of interactions is like
>> adjoining interaction matrices for the two different kinds of interaction.
>> Suppose that you have user x views in matrix A and you have user x
>> purchases in matrix B. The complete interaction matrix of user x (views +
>> purchases) is [A | B].
>>
>> When you compute cooccurrence in this matrix, you get
>>
>> [A | B] = [ A' ] [ A' A A' B ]
>> [A | B]' [A | B] = [ ] [A | B] = [ ]
>> [A | B] = [ B' ] [ B' A B' B ]
>>
>> This matrix is (view + purchase) x (view + purchase). But we don't care
>> about predicting views so we only really need a matrix that is purchase x
>> (view
>> + purchase). This is just the bottom part of the matrix above, or [ B'A |
>> B'B ]. When you produce purchase recommendations r_p by multiplying by a
>> mixed view and purchase history vector h which has a view part h_v and a
>> purchase part h_p, you get
>>
>> r_p = [ B' A B' B ] h = B'A h_v + B'B h_p
>>
>> That is a prediction of purchases based on past views and past purchase.
>>
>> Note that this general form applies for both decomposition techniques such
>> as SVD, ALS and LLL as well as for sparsification techniques such as the
>> LLR sparsification. All that changes is the mechanics of how you do the
>> multiplications. Weighting of components works the same as well.
>>
>> What is very different here is that we have a component of cross
>> recommendation. That is the B'A in the formula above. This is very
>> different from a normal recommendation and cannot be computed with the
>> simple self-join that we normally have in Mahout cooccurrence computation
>> and also very different from the decompositions that we normally do. It
>> isn't hard to adapt the Mahout computations, however.
>>
>> When implementing a recommendation using a search engine as the base, these
>> same techniques also work extremely well in my experience. What happens is
>> that for each item that you would like to recommend, you would have one
>> field that has components of B'A and one field that has components of B'B.
>> It is handy to simply use the binary values of the sparsified versions of
>> these and let the search engine handle the weighting of different
>> components at query time. Having these components separated into different
>> fields in the search index seems to help quite a lot, which makes a fair
>> bit of sense.
>>
>> On Sun, Feb 10, 2013 at 9:55 AM, Sean Owen <sr...@gmail.com> wrote:
>>>
>>> I think you'd have to hack the code to not exclude previously-seen items,
>>> or at least, not of the type you wish to consider. Yes you would also
>> have
>>> to hack it to add rather than replace existing values. Or for test
>>> purposes, just do the adding yourself before inputting the data.
>>>
>>> My hunch is that it will hurt non-trivially to treat different
>> interaction
>>> types as different items. You probably want to predict that someone who
>>> viewed a product over and over is likely to buy it, but this would only
>>> weakly tend to occur if the bought-item is not the same thing as the
>>> viewed-item. You'd learn they go together but not as strongly as ought to
>>> be obvious from the fact that they're the same. Still, interesting
>> thought.
>>>
>>> There ought to be some 'signal' in this data, just a question of how much
>>> vs noise. A purchase means much more than a page view of course; it's not
>>> as subject to noise. Finding a means to use that info is probably going
>> to
>>> help.
>>>
>>>
>>>
>>>
>>> On Sat, Feb 9, 2013 at 7:50 PM, Pat Ferrel <pa...@gmail.com> wrote:
>>>
>>>> I'd like to experiment with using several types of implicit preference
>>>> values with recommenders. I have purchases as an implicit pref of high
>>>> strength. I'd like to see if add-to-cart, view-product-details,
>>>> impressions-seen, etc. can increase offline precision in holdout tests.
>>>> These less than obvious implicit prefs will get a much lower value than
>>>> purchase and i'll experiment with different mixes. The problem is that
>> some
>>>> of these prefs will indicate that the user, for whom I'm getting recs,
>> has
>>>> expressed a preference.
>>>>
>>>> Using these implicit prefs seems reasonable in finding similarity of
>> taste
>>>> between users but presents several problems. 1) how to encode the
>> prefs,
>>>> each impression-seen will increase the strength of preference of a user
>> for
>>>> an item but the recommender framework replaces the preference value for
>>>> items preferred more than once, doesn't it? 2) AFAIK the current
>>>> recommender framework will return recs only for items that the user in
>>>> question has expressed no preference for. If I use something like
>>>> view-product-details or impressions-seen, I will be removing anything
>> the
>>>> user has seen from the recs, which is not what I want in this
>> experiment.
>>>>
>>>> Has anyone tried something like this? I'm not convinced that these
>> other
>>>> implicit preferences will add anything to the recommender, just trying
>> to
>>>> find out.
>>
--------------------------
Ken Krugler
+1 530-210-6378
http://www.scaleunlimited.com
custom big data solutions & training
Hadoop, Cascading, Cassandra & Solr
Re: Implicit preferences
Posted by Ted Dunning <te...@gmail.com>.
If you are using rootLLR, then a threshold of 10 represents (roughly) 10
standard deviations. This is a big threshold.
What I generally do is threshold to a level that either makes the resulting
pairs be composed of a weak majority of plausible terms (if I understand
the domain) or simply to drive to a level of sparsity. Both seem about the
same.
I also pretty much always also down-sample the number of items per user.
This has two motivations. One is simply pragmatics. The other is that it
decreases the influence of bots and other pathological users.
On Mon, Feb 11, 2013 at 2:57 AM, Johannes Schulte <
johannes.schulte@gmail.com> wrote:
>
>
> I am also thresholding the counts with LLR. Every time i do this I take a
> threshold of 10 since I loosely remember it being about the 99% margin of
> confidence in the chi square distribution. I got no clue however if anybody
> wants something like 99% for recommendations or if 50% might be a better
> value. What's your experience on that?
>
> And do you apply a limit on the total number of docs per term, since there
> could be big boolean queries tearing down the performance?
>
> Thanks for all the input!
Re: Implicit preferences
Posted by Johannes Schulte <jo...@gmail.com>.
@Ted: Thanks for the input!
@Ken
I think the boosting is buried in getSpanScore() calling super.score() but
a quick look at the code didn't give me the exact reason. I remember me
setting includeSpanScore to false and the boost disappearing. The use of
payloads however was only necessary because i put the similarity scores in
there and then summed them up. In the "new" system i am gonna use no
payloads (so far).
Thanks for the hint with the int field. I am using plain lucene but the
semantics should be the same. I didnt know you could turn off the bonus
features
On Mon, Feb 11, 2013 at 7:27 PM, Ken Krugler <kk...@transpac.com>wrote:
>
> On Feb 11, 2013, at 1:57am, Johannes Schulte wrote:
>
> > @Ken
> > Thanks for the hints...
> > I am coming from a payload based system so I am aware if them, however in
> > the lucene 3.6 branch boosting and payloads didn't work together (if you
> > set PayloadTermQuery.setIncludeSpanScore to false, they were ignored)
>
> I assume you're talking about passing false for the includeSpanScore
> parameter in the PayloadTermQuery constructor, yes?
>
> Anyway, I'm surprised you ran into this issue. In the 3.6.0 source for
> PayloadTermQuery, the getScore() method is:
>
> @Override
> public float score() throws IOException {
>
> return includeSpanScore ? getSpanScore() * getPayloadScore()
> : getPayloadScore();
> }
>
> So I would assume that you'd get the payload score (as expected). But I
> haven't actually tried to validate this.
>
> > Besides that, there is no performance issue here so far so it's probably
> a
> > fine way to go, i was just curious...as for the IntField / TrieIntField,
> > all the range query / ordering benefits of it are overhead since the
> > integers just represent random indices into a vector. I might look into
> > indexing the integer bytes rather than the string separation…
>
> I was proposing you use:
>
> <fieldType name="int" class="solr.TrieIntField" precisionStep="0"
> positionIncrementGap="0"/>
>
> Which doesn't generate the extra values that make range queries faster,
> but should store the data more efficiently.
>
> -- Ken
>
> > @Ted
> > You are probably right with chosing 1 as term frequency, i forgot that
> the
> > most interesting information comes from the idf probably and using
> > cooccurrence counts as term frequency might make the combination with
> text
> > searches infeasible since the values lie in some totally different range.
> > Also I forgot that idf is per field so i might go for separating the
> hashed
> > values into their originating fields (search tern, item_id, category_id)
> .
> > This would still alllow to recombine them later when a use profile has to
> > be constructed.
> >
> > I like to threshold with LLR. That gives me a binary matrix. Then I
> >> directly index that.
> >> The search engine provides very nice weights at this point. I don't
> feel
> >> the need to adjust those weights because they have roughly the same
> form as
> >> learned weights are likely to have and because learning those weights
> would
> >> almost certainly result in over-fitting unless I go to quite a lot of
> >> trouble.
> >> Also, I have heard that at least one head-to-head test found that the
> >> native Solr term weighting actually out-performed several more intricate
> >> and explicit weighting schemes. That can't be taken as evidence that
> >> Solr's weightings would perform better than whatever you have in mind,
> but
> >> it does provide interesting meta-evidence that the probability that a
> >> reasonably smart dev team is definitely not guaranteed to beat Solr's
> >> weighting by a large margin. When you sit down to architect your
> system,
> >> you need to make decisions about where to spend your time and evidence
> like
> >> that is helpful to guess how much effort it would take to achieve
> different
> >> levels of performance.
> >
> >
> >
> > I am also thresholding the counts with LLR. Every time i do this I take a
> > threshold of 10 since I loosely remember it being about the 99% margin
> of
> > confidence in the chi square distribution. I got no clue however if
> anybody
> > wants something like 99% for recommendations or if 50% might be a better
> > value. What's your experience on that?
> >
> > And do you apply a limit on the total number of docs per term, since
> there
> > could be big boolean queries tearing down the performance?
> >
> > Thanks for all the input!
> >
> >
> >
> > On Mon, Feb 11, 2013 at 7:20 AM, Ted Dunning <te...@gmail.com>
> wrote:
> >
> >> On Sun, Feb 10, 2013 at 3:39 PM, Johannes Schulte <
> >> johannes.schulte@gmail.com> wrote:
> >>
> >>> ...
> >>> i am currently implementing a system of the same kind, LLR sparsified
> >>> "term"-cooccurrence vectors in lucene (since not a day goes by where i
> >> see
> >>> Ted praising this).
> >>>
> >>
> >> (turns red)
> >>
> >>
> >>> There are not only views and purchases, but also search terms, facets
> >> and a
> >>> lot more textual information to be included in the cooccurrence matrix
> >> (as
> >>> "input").
> >>> That's why i went with the feature hashing framework in mahout. This
> >> gives
> >>> small (hd/mem) user profiles and allows for reusing the vectors for
> click
> >>> prediction and/or clustering.
> >>
> >>
> >> This is a reasonable choice. For recommendations, you might want to use
> >> direct encoding since it can be simpler to build a search index for
> >> recommending.
> >>
> >>
> >>> The main difference is that there's only two
> >>> fields in lucene with a lot of terms (Numbers), representing the
> >> features.
> >>> Two fields because i think predicting views (besides purchases) might
> in
> >>> some cases be better than predicting nothing.
> >>>
> >>
> >> OK.
> >>
> >>
> >>> I don't think it should make a big differing in scoring because in a
> >>> vector space model used by most engines it's just, well a vector space
> >> and
> >>> i don't know if the field norm make sense after stripping values from
> the
> >>> term vectors with the LLR threshold.
> >>>
> >>
> >> Having separate fields is going to give separate total term counts.
> That
> >> seems better to me, but I have to confess I have never rigorously tested
> >> that.
> >>
> >>
> >>> @Ted
> >>>> It is handy to simply use the binary values of the sparsified versions
> >> of
> >>>> these and let the search engine handle the weighting of different
> >>>> components at query time.
> >>>
> >>> Do you really want to omit the cooccurrence counts which would become
> the
> >>> term frequecies? how would the engine then weight different inputs
> >> against
> >>> each other?
> >>>
> >>
> >> I like to threshold with LLR. That gives me a binary matrix. Then I
> >> directly index that.
> >>
> >> The search engine provides very nice weights at this point. I don't
> feel
> >> the need to adjust those weights because they have roughly the same
> form as
> >> learned weights are likely to have and because learning those weights
> would
> >> almost certainly result in over-fitting unless I go to quite a lot of
> >> trouble.
> >>
> >> Also, I have heard that at least one head-to-head test found that the
> >> native Solr term weighting actually out-performed several more intricate
> >> and explicit weighting schemes. That can't be taken as evidence that
> >> Solr's weightings would perform better than whatever you have in mind,
> but
> >> it does provide interesting meta-evidence that the probability that a
> >> reasonably smart dev team is definitely not guaranteed to beat Solr's
> >> weighting by a large margin. When you sit down to architect your
> system,
> >> you need to make decisions about where to spend your time and evidence
> like
> >> that is helpful to guess how much effort it would take to achieve
> different
> >> levels of performance.
> >>
> >> And, if anyone knows a
> >>> 1. smarter way to index the cooccurrence counts in lucene than a
> >>> tokenstream that emits a word k times for a cooccurrence count of k
> >>>
> >>
> >> You can use payloads or you can boost individual terms.
> >>
> >>
> >>> 2. way to avoid treating the (hashed) vector column indices as terms
> but
> >>> reusing them? It's a bit weird hashing to an int and then having the
> >> lucene
> >>> term dictionary treating them as string, mapping to another int
> >>>
> >>
> >> Why do we care about this? These tokens get put onto documents that
> have
> >> additional data to help them make sense, but why do we care if the
> tokens
> >> look like numbers?
> >>
>
> --------------------------------------------
> http://about.me/kkrugler
> +1 530-210-6378
>
>
>
>
> --------------------------
> Ken Krugler
> +1 530-210-6378
> http://www.scaleunlimited.com
> custom big data solutions & training
> Hadoop, Cascading, Cassandra & Solr
>
>
>
>
>
>
Re: Implicit preferences
Posted by Ken Krugler <kk...@transpac.com>.
On Feb 11, 2013, at 1:57am, Johannes Schulte wrote:
> @Ken
> Thanks for the hints...
> I am coming from a payload based system so I am aware if them, however in
> the lucene 3.6 branch boosting and payloads didn't work together (if you
> set PayloadTermQuery.setIncludeSpanScore to false, they were ignored)
I assume you're talking about passing false for the includeSpanScore parameter in the PayloadTermQuery constructor, yes?
Anyway, I'm surprised you ran into this issue. In the 3.6.0 source for PayloadTermQuery, the getScore() method is:
@Override
public float score() throws IOException {
return includeSpanScore ? getSpanScore() * getPayloadScore()
: getPayloadScore();
}
So I would assume that you'd get the payload score (as expected). But I haven't actually tried to validate this.
> Besides that, there is no performance issue here so far so it's probably a
> fine way to go, i was just curious...as for the IntField / TrieIntField,
> all the range query / ordering benefits of it are overhead since the
> integers just represent random indices into a vector. I might look into
> indexing the integer bytes rather than the string separation…
I was proposing you use:
<fieldType name="int" class="solr.TrieIntField" precisionStep="0" positionIncrementGap="0"/>
Which doesn't generate the extra values that make range queries faster, but should store the data more efficiently.
-- Ken
> @Ted
> You are probably right with chosing 1 as term frequency, i forgot that the
> most interesting information comes from the idf probably and using
> cooccurrence counts as term frequency might make the combination with text
> searches infeasible since the values lie in some totally different range.
> Also I forgot that idf is per field so i might go for separating the hashed
> values into their originating fields (search tern, item_id, category_id) .
> This would still alllow to recombine them later when a use profile has to
> be constructed.
>
> I like to threshold with LLR. That gives me a binary matrix. Then I
>> directly index that.
>> The search engine provides very nice weights at this point. I don't feel
>> the need to adjust those weights because they have roughly the same form as
>> learned weights are likely to have and because learning those weights would
>> almost certainly result in over-fitting unless I go to quite a lot of
>> trouble.
>> Also, I have heard that at least one head-to-head test found that the
>> native Solr term weighting actually out-performed several more intricate
>> and explicit weighting schemes. That can't be taken as evidence that
>> Solr's weightings would perform better than whatever you have in mind, but
>> it does provide interesting meta-evidence that the probability that a
>> reasonably smart dev team is definitely not guaranteed to beat Solr's
>> weighting by a large margin. When you sit down to architect your system,
>> you need to make decisions about where to spend your time and evidence like
>> that is helpful to guess how much effort it would take to achieve different
>> levels of performance.
>
>
>
> I am also thresholding the counts with LLR. Every time i do this I take a
> threshold of 10 since I loosely remember it being about the 99% margin of
> confidence in the chi square distribution. I got no clue however if anybody
> wants something like 99% for recommendations or if 50% might be a better
> value. What's your experience on that?
>
> And do you apply a limit on the total number of docs per term, since there
> could be big boolean queries tearing down the performance?
>
> Thanks for all the input!
>
>
>
> On Mon, Feb 11, 2013 at 7:20 AM, Ted Dunning <te...@gmail.com> wrote:
>
>> On Sun, Feb 10, 2013 at 3:39 PM, Johannes Schulte <
>> johannes.schulte@gmail.com> wrote:
>>
>>> ...
>>> i am currently implementing a system of the same kind, LLR sparsified
>>> "term"-cooccurrence vectors in lucene (since not a day goes by where i
>> see
>>> Ted praising this).
>>>
>>
>> (turns red)
>>
>>
>>> There are not only views and purchases, but also search terms, facets
>> and a
>>> lot more textual information to be included in the cooccurrence matrix
>> (as
>>> "input").
>>> That's why i went with the feature hashing framework in mahout. This
>> gives
>>> small (hd/mem) user profiles and allows for reusing the vectors for click
>>> prediction and/or clustering.
>>
>>
>> This is a reasonable choice. For recommendations, you might want to use
>> direct encoding since it can be simpler to build a search index for
>> recommending.
>>
>>
>>> The main difference is that there's only two
>>> fields in lucene with a lot of terms (Numbers), representing the
>> features.
>>> Two fields because i think predicting views (besides purchases) might in
>>> some cases be better than predicting nothing.
>>>
>>
>> OK.
>>
>>
>>> I don't think it should make a big differing in scoring because in a
>>> vector space model used by most engines it's just, well a vector space
>> and
>>> i don't know if the field norm make sense after stripping values from the
>>> term vectors with the LLR threshold.
>>>
>>
>> Having separate fields is going to give separate total term counts. That
>> seems better to me, but I have to confess I have never rigorously tested
>> that.
>>
>>
>>> @Ted
>>>> It is handy to simply use the binary values of the sparsified versions
>> of
>>>> these and let the search engine handle the weighting of different
>>>> components at query time.
>>>
>>> Do you really want to omit the cooccurrence counts which would become the
>>> term frequecies? how would the engine then weight different inputs
>> against
>>> each other?
>>>
>>
>> I like to threshold with LLR. That gives me a binary matrix. Then I
>> directly index that.
>>
>> The search engine provides very nice weights at this point. I don't feel
>> the need to adjust those weights because they have roughly the same form as
>> learned weights are likely to have and because learning those weights would
>> almost certainly result in over-fitting unless I go to quite a lot of
>> trouble.
>>
>> Also, I have heard that at least one head-to-head test found that the
>> native Solr term weighting actually out-performed several more intricate
>> and explicit weighting schemes. That can't be taken as evidence that
>> Solr's weightings would perform better than whatever you have in mind, but
>> it does provide interesting meta-evidence that the probability that a
>> reasonably smart dev team is definitely not guaranteed to beat Solr's
>> weighting by a large margin. When you sit down to architect your system,
>> you need to make decisions about where to spend your time and evidence like
>> that is helpful to guess how much effort it would take to achieve different
>> levels of performance.
>>
>> And, if anyone knows a
>>> 1. smarter way to index the cooccurrence counts in lucene than a
>>> tokenstream that emits a word k times for a cooccurrence count of k
>>>
>>
>> You can use payloads or you can boost individual terms.
>>
>>
>>> 2. way to avoid treating the (hashed) vector column indices as terms but
>>> reusing them? It's a bit weird hashing to an int and then having the
>> lucene
>>> term dictionary treating them as string, mapping to another int
>>>
>>
>> Why do we care about this? These tokens get put onto documents that have
>> additional data to help them make sense, but why do we care if the tokens
>> look like numbers?
>>
--------------------------------------------
http://about.me/kkrugler
+1 530-210-6378
--------------------------
Ken Krugler
+1 530-210-6378
http://www.scaleunlimited.com
custom big data solutions & training
Hadoop, Cascading, Cassandra & Solr
Re: Implicit preferences
Posted by Johannes Schulte <jo...@gmail.com>.
@Ken
Thanks for the hints...
I am coming from a payload based system so I am aware if them, however in
the lucene 3.6 branch boosting and payloads didn't work together (if you
set PayloadTermQuery.setIncludeSpanScore to false, they were ignored)
Besides that, there is no performance issue here so far so it's probably a
fine way to go, i was just curious...as for the IntField / TrieIntField,
all the range query / ordering benefits of it are overhead since the
integers just represent random indices into a vector. I might look into
indexing the integer bytes rather than the string separation...
@Ted
You are probably right with chosing 1 as term frequency, i forgot that the
most interesting information comes from the idf probably and using
cooccurrence counts as term frequency might make the combination with text
searches infeasible since the values lie in some totally different range.
Also I forgot that idf is per field so i might go for separating the hashed
values into their originating fields (search tern, item_id, category_id) .
This would still alllow to recombine them later when a use profile has to
be constructed.
I like to threshold with LLR. That gives me a binary matrix. Then I
> directly index that.
> The search engine provides very nice weights at this point. I don't feel
> the need to adjust those weights because they have roughly the same form as
> learned weights are likely to have and because learning those weights would
> almost certainly result in over-fitting unless I go to quite a lot of
> trouble.
> Also, I have heard that at least one head-to-head test found that the
> native Solr term weighting actually out-performed several more intricate
> and explicit weighting schemes. That can't be taken as evidence that
> Solr's weightings would perform better than whatever you have in mind, but
> it does provide interesting meta-evidence that the probability that a
> reasonably smart dev team is definitely not guaranteed to beat Solr's
> weighting by a large margin. When you sit down to architect your system,
> you need to make decisions about where to spend your time and evidence like
> that is helpful to guess how much effort it would take to achieve different
> levels of performance.
I am also thresholding the counts with LLR. Every time i do this I take a
threshold of 10 since I loosely remember it being about the 99% margin of
confidence in the chi square distribution. I got no clue however if anybody
wants something like 99% for recommendations or if 50% might be a better
value. What's your experience on that?
And do you apply a limit on the total number of docs per term, since there
could be big boolean queries tearing down the performance?
Thanks for all the input!
On Mon, Feb 11, 2013 at 7:20 AM, Ted Dunning <te...@gmail.com> wrote:
> On Sun, Feb 10, 2013 at 3:39 PM, Johannes Schulte <
> johannes.schulte@gmail.com> wrote:
>
> > ...
> > i am currently implementing a system of the same kind, LLR sparsified
> > "term"-cooccurrence vectors in lucene (since not a day goes by where i
> see
> > Ted praising this).
> >
>
> (turns red)
>
>
> > There are not only views and purchases, but also search terms, facets
> and a
> > lot more textual information to be included in the cooccurrence matrix
> (as
> > "input").
> > That's why i went with the feature hashing framework in mahout. This
> gives
> > small (hd/mem) user profiles and allows for reusing the vectors for click
> > prediction and/or clustering.
>
>
> This is a reasonable choice. For recommendations, you might want to use
> direct encoding since it can be simpler to build a search index for
> recommending.
>
>
> > The main difference is that there's only two
> > fields in lucene with a lot of terms (Numbers), representing the
> features.
> > Two fields because i think predicting views (besides purchases) might in
> > some cases be better than predicting nothing.
> >
>
> OK.
>
>
> > I don't think it should make a big differing in scoring because in a
> > vector space model used by most engines it's just, well a vector space
> and
> > i don't know if the field norm make sense after stripping values from the
> > term vectors with the LLR threshold.
> >
>
> Having separate fields is going to give separate total term counts. That
> seems better to me, but I have to confess I have never rigorously tested
> that.
>
>
> > @Ted
> > > It is handy to simply use the binary values of the sparsified versions
> of
> > >these and let the search engine handle the weighting of different
> > >components at query time.
> >
> > Do you really want to omit the cooccurrence counts which would become the
> > term frequecies? how would the engine then weight different inputs
> against
> > each other?
> >
>
> I like to threshold with LLR. That gives me a binary matrix. Then I
> directly index that.
>
> The search engine provides very nice weights at this point. I don't feel
> the need to adjust those weights because they have roughly the same form as
> learned weights are likely to have and because learning those weights would
> almost certainly result in over-fitting unless I go to quite a lot of
> trouble.
>
> Also, I have heard that at least one head-to-head test found that the
> native Solr term weighting actually out-performed several more intricate
> and explicit weighting schemes. That can't be taken as evidence that
> Solr's weightings would perform better than whatever you have in mind, but
> it does provide interesting meta-evidence that the probability that a
> reasonably smart dev team is definitely not guaranteed to beat Solr's
> weighting by a large margin. When you sit down to architect your system,
> you need to make decisions about where to spend your time and evidence like
> that is helpful to guess how much effort it would take to achieve different
> levels of performance.
>
> And, if anyone knows a
> > 1. smarter way to index the cooccurrence counts in lucene than a
> > tokenstream that emits a word k times for a cooccurrence count of k
> >
>
> You can use payloads or you can boost individual terms.
>
>
> > 2. way to avoid treating the (hashed) vector column indices as terms but
> > reusing them? It's a bit weird hashing to an int and then having the
> lucene
> > term dictionary treating them as string, mapping to another int
> >
>
> Why do we care about this? These tokens get put onto documents that have
> additional data to help them make sense, but why do we care if the tokens
> look like numbers?
>
Re: Implicit preferences
Posted by Ted Dunning <te...@gmail.com>.
On Sun, Feb 10, 2013 at 3:39 PM, Johannes Schulte <
johannes.schulte@gmail.com> wrote:
> ...
> i am currently implementing a system of the same kind, LLR sparsified
> "term"-cooccurrence vectors in lucene (since not a day goes by where i see
> Ted praising this).
>
(turns red)
> There are not only views and purchases, but also search terms, facets and a
> lot more textual information to be included in the cooccurrence matrix (as
> "input").
> That's why i went with the feature hashing framework in mahout. This gives
> small (hd/mem) user profiles and allows for reusing the vectors for click
> prediction and/or clustering.
This is a reasonable choice. For recommendations, you might want to use
direct encoding since it can be simpler to build a search index for
recommending.
> The main difference is that there's only two
> fields in lucene with a lot of terms (Numbers), representing the features.
> Two fields because i think predicting views (besides purchases) might in
> some cases be better than predicting nothing.
>
OK.
> I don't think it should make a big differing in scoring because in a
> vector space model used by most engines it's just, well a vector space and
> i don't know if the field norm make sense after stripping values from the
> term vectors with the LLR threshold.
>
Having separate fields is going to give separate total term counts. That
seems better to me, but I have to confess I have never rigorously tested
that.
> @Ted
> > It is handy to simply use the binary values of the sparsified versions of
> >these and let the search engine handle the weighting of different
> >components at query time.
>
> Do you really want to omit the cooccurrence counts which would become the
> term frequecies? how would the engine then weight different inputs against
> each other?
>
I like to threshold with LLR. That gives me a binary matrix. Then I
directly index that.
The search engine provides very nice weights at this point. I don't feel
the need to adjust those weights because they have roughly the same form as
learned weights are likely to have and because learning those weights would
almost certainly result in over-fitting unless I go to quite a lot of
trouble.
Also, I have heard that at least one head-to-head test found that the
native Solr term weighting actually out-performed several more intricate
and explicit weighting schemes. That can't be taken as evidence that
Solr's weightings would perform better than whatever you have in mind, but
it does provide interesting meta-evidence that the probability that a
reasonably smart dev team is definitely not guaranteed to beat Solr's
weighting by a large margin. When you sit down to architect your system,
you need to make decisions about where to spend your time and evidence like
that is helpful to guess how much effort it would take to achieve different
levels of performance.
And, if anyone knows a
> 1. smarter way to index the cooccurrence counts in lucene than a
> tokenstream that emits a word k times for a cooccurrence count of k
>
You can use payloads or you can boost individual terms.
> 2. way to avoid treating the (hashed) vector column indices as terms but
> reusing them? It's a bit weird hashing to an int and then having the lucene
> term dictionary treating them as string, mapping to another int
>
Why do we care about this? These tokens get put onto documents that have
additional data to help them make sense, but why do we care if the tokens
look like numbers?
Re: Implicit preferences
Posted by Johannes Schulte <jo...@gmail.com>.
Hi,
i am currently implementing a system of the same kind, LLR sparsified
"term"-cooccurrence vectors in lucene (since not a day goes by where i see
Ted praising this).
There are not only views and purchases, but also search terms, facets and a
lot more textual information to be included in the cooccurrence matrix (as
"input").
That's why i went with the feature hashing framework in mahout. This gives
small (hd/mem) user profiles and allows for reusing the vectors for click
prediction and/or clustering. The main difference is that there's only two
fields in lucene with a lot of terms (Numbers), representing the features.
Two fields because i think predicting views (besides purchases) might in
some cases be better than predicting nothing.
I don't think it should make a big differing in scoring because in a
vector space model used by most engines it's just, well a vector space and
i don't know if the field norm make sense after stripping values from the
term vectors with the LLR threshold.
@Ted
> It is handy to simply use the binary values of the sparsified versions of
>these and let the search engine handle the weighting of different
>components at query time.
Do you really want to omit the cooccurrence counts which would become the
term frequecies? how would the engine then weight different inputs against
each other?
And,
if anyone knows a
1. smarter way to index the cooccurrence counts in lucene than a
tokenstream that emits a word k times for a cooccurrence count of k
2. way to avoid treating the (hashed) vector column indices as terms but
reusing them? It's a bit weird hashing to an int and then having the lucene
term dictionary treating them as string, mapping to another int
I'd be thankful for input
On Sun, Feb 10, 2013 at 6:36 PM, Ted Dunning <te...@gmail.com> wrote:
> Actually treating the different interactions separately can lead to very
> good recommendations. The only issue is that the interactions are no
> longer dyadic.
>
> If you think about it, having two different kinds of interactions is like
> adjoining interaction matrices for the two different kinds of interaction.
> Suppose that you have user x views in matrix A and you have user x
> purchases in matrix B. The complete interaction matrix of user x (views +
> purchases) is [A | B].
>
> When you compute cooccurrence in this matrix, you get
>
> [A | B] = [ A' ] [ A' A A' B ]
> [A | B]' [A | B] = [ ] [A | B] = [ ]
> [A | B] = [ B' ] [ B' A B' B ]
>
> This matrix is (view + purchase) x (view + purchase). But we don't care
> about predicting views so we only really need a matrix that is purchase x
> (view
> + purchase). This is just the bottom part of the matrix above, or [ B'A |
> B'B ]. When you produce purchase recommendations r_p by multiplying by a
> mixed view and purchase history vector h which has a view part h_v and a
> purchase part h_p, you get
>
> r_p = [ B' A B' B ] h = B'A h_v + B'B h_p
>
> That is a prediction of purchases based on past views and past purchase.
>
> Note that this general form applies for both decomposition techniques such
> as SVD, ALS and LLL as well as for sparsification techniques such as the
> LLR sparsification. All that changes is the mechanics of how you do the
> multiplications. Weighting of components works the same as well.
>
> What is very different here is that we have a component of cross
> recommendation. That is the B'A in the formula above. This is very
> different from a normal recommendation and cannot be computed with the
> simple self-join that we normally have in Mahout cooccurrence computation
> and also very different from the decompositions that we normally do. It
> isn't hard to adapt the Mahout computations, however.
>
> When implementing a recommendation using a search engine as the base, these
> same techniques also work extremely well in my experience. What happens is
> that for each item that you would like to recommend, you would have one
> field that has components of B'A and one field that has components of B'B.
> It is handy to simply use the binary values of the sparsified versions of
> these and let the search engine handle the weighting of different
> components at query time. Having these components separated into different
> fields in the search index seems to help quite a lot, which makes a fair
> bit of sense.
>
> On Sun, Feb 10, 2013 at 9:55 AM, Sean Owen <sr...@gmail.com> wrote:
> >
> > I think you'd have to hack the code to not exclude previously-seen items,
> > or at least, not of the type you wish to consider. Yes you would also
> have
> > to hack it to add rather than replace existing values. Or for test
> > purposes, just do the adding yourself before inputting the data.
> >
> > My hunch is that it will hurt non-trivially to treat different
> interaction
> > types as different items. You probably want to predict that someone who
> > viewed a product over and over is likely to buy it, but this would only
> > weakly tend to occur if the bought-item is not the same thing as the
> > viewed-item. You'd learn they go together but not as strongly as ought to
> > be obvious from the fact that they're the same. Still, interesting
> thought.
> >
> > There ought to be some 'signal' in this data, just a question of how much
> > vs noise. A purchase means much more than a page view of course; it's not
> > as subject to noise. Finding a means to use that info is probably going
> to
> > help.
> >
> >
> >
> >
> > On Sat, Feb 9, 2013 at 7:50 PM, Pat Ferrel <pa...@gmail.com> wrote:
> >
> > > I'd like to experiment with using several types of implicit preference
> > > values with recommenders. I have purchases as an implicit pref of high
> > > strength. I'd like to see if add-to-cart, view-product-details,
> > > impressions-seen, etc. can increase offline precision in holdout tests.
> > > These less than obvious implicit prefs will get a much lower value than
> > > purchase and i'll experiment with different mixes. The problem is that
> some
> > > of these prefs will indicate that the user, for whom I'm getting recs,
> has
> > > expressed a preference.
> > >
> > > Using these implicit prefs seems reasonable in finding similarity of
> taste
> > > between users but presents several problems. 1) how to encode the
> prefs,
> > > each impression-seen will increase the strength of preference of a user
> for
> > > an item but the recommender framework replaces the preference value for
> > > items preferred more than once, doesn't it? 2) AFAIK the current
> > > recommender framework will return recs only for items that the user in
> > > question has expressed no preference for. If I use something like
> > > view-product-details or impressions-seen, I will be removing anything
> the
> > > user has seen from the recs, which is not what I want in this
> experiment.
> > >
> > > Has anyone tried something like this? I'm not convinced that these
> other
> > > implicit preferences will add anything to the recommender, just trying
> to
> > > find out.
>
Re: Implicit preferences
Posted by Ted Dunning <te...@gmail.com>.
Yes you are learning two matrices but the cost scales with the input data size anyway so learning the two matrices costs the same as learning one matrix on the combined day set.
You could argue that the search index is larger but the index is usually so small you don't even need more than one search engine instance.
Sent from my iPhone
On Feb 10, 2013, at 2:18 PM, Sean Owen <sr...@gmail.com> wrote:
> Yeah I bet it does actually work well... but aren't you basically spending
> an extra step to make the "item-item" matrix, to relearn that bought-X and
> viewed-X go together? yeah you learn a lot more along the way, as this is
> item-based recommendation at heart. It seems like you could add back that
> knowledge.
>
>
> On Sun, Feb 10, 2013 at 5:36 PM, Ted Dunning <te...@gmail.com> wrote:
>
>> Actually treating the different interactions separately can lead to very
>> good recommendations. The only issue is that the interactions are no
>> longer dyadic.
>>
>> If you think about it, having two different kinds of interactions is like
>> adjoining interaction matrices for the two different kinds of interaction.
>> Suppose that you have user x views in matrix A and you have user x
>> purchases in matrix B. The complete interaction matrix of user x (views +
>> purchases) is [A | B].
>>
>> When you compute cooccurrence in this matrix, you get
>>
>> [A | B] = [ A' ] [ A' A A' B ]
>> [A | B]' [A | B] = [ ] [A | B] = [ ]
>> [A | B] = [ B' ] [ B' A B' B ]
>>
>> This matrix is (view + purchase) x (view + purchase). But we don't care
>> about predicting views so we only really need a matrix that is purchase x
>> (view
>> + purchase). This is just the bottom part of the matrix above, or [ B'A |
>> B'B ]. When you produce purchase recommendations r_p by multiplying by a
>> mixed view and purchase history vector h which has a view part h_v and a
>> purchase part h_p, you get
>>
>> r_p = [ B' A B' B ] h = B'A h_v + B'B h_p
>>
>> That is a prediction of purchases based on past views and past purchase.
>>
>> Note that this general form applies for both decomposition techniques such
>> as SVD, ALS and LLL as well as for sparsification techniques such as the
>> LLR sparsification. All that changes is the mechanics of how you do the
>> multiplications. Weighting of components works the same as well.
>>
>> What is very different here is that we have a component of cross
>> recommendation. That is the B'A in the formula above. This is very
>> different from a normal recommendation and cannot be computed with the
>> simple self-join that we normally have in Mahout cooccurrence computation
>> and also very different from the decompositions that we normally do. It
>> isn't hard to adapt the Mahout computations, however.
>>
>> When implementing a recommendation using a search engine as the base, these
>> same techniques also work extremely well in my experience. What happens is
>> that for each item that you would like to recommend, you would have one
>> field that has components of B'A and one field that has components of B'B.
>> It is handy to simply use the binary values of the sparsified versions of
>> these and let the search engine handle the weighting of different
>> components at query time. Having these components separated into different
>> fields in the search index seems to help quite a lot, which makes a fair
>> bit of sense.
>>
>> On Sun, Feb 10, 2013 at 9:55 AM, Sean Owen <sr...@gmail.com> wrote:
>>>
>>> I think you'd have to hack the code to not exclude previously-seen items,
>>> or at least, not of the type you wish to consider. Yes you would also
>> have
>>> to hack it to add rather than replace existing values. Or for test
>>> purposes, just do the adding yourself before inputting the data.
>>>
>>> My hunch is that it will hurt non-trivially to treat different
>> interaction
>>> types as different items. You probably want to predict that someone who
>>> viewed a product over and over is likely to buy it, but this would only
>>> weakly tend to occur if the bought-item is not the same thing as the
>>> viewed-item. You'd learn they go together but not as strongly as ought to
>>> be obvious from the fact that they're the same. Still, interesting
>> thought.
>>>
>>> There ought to be some 'signal' in this data, just a question of how much
>>> vs noise. A purchase means much more than a page view of course; it's not
>>> as subject to noise. Finding a means to use that info is probably going
>> to
>>> help.
>>>
>>>
>>>
>>>
>>> On Sat, Feb 9, 2013 at 7:50 PM, Pat Ferrel <pa...@gmail.com> wrote:
>>>
>>>> I'd like to experiment with using several types of implicit preference
>>>> values with recommenders. I have purchases as an implicit pref of high
>>>> strength. I'd like to see if add-to-cart, view-product-details,
>>>> impressions-seen, etc. can increase offline precision in holdout tests.
>>>> These less than obvious implicit prefs will get a much lower value than
>>>> purchase and i'll experiment with different mixes. The problem is that
>> some
>>>> of these prefs will indicate that the user, for whom I'm getting recs,
>> has
>>>> expressed a preference.
>>>>
>>>> Using these implicit prefs seems reasonable in finding similarity of
>> taste
>>>> between users but presents several problems. 1) how to encode the
>> prefs,
>>>> each impression-seen will increase the strength of preference of a user
>> for
>>>> an item but the recommender framework replaces the preference value for
>>>> items preferred more than once, doesn't it? 2) AFAIK the current
>>>> recommender framework will return recs only for items that the user in
>>>> question has expressed no preference for. If I use something like
>>>> view-product-details or impressions-seen, I will be removing anything
>> the
>>>> user has seen from the recs, which is not what I want in this
>> experiment.
>>>>
>>>> Has anyone tried something like this? I'm not convinced that these
>> other
>>>> implicit preferences will add anything to the recommender, just trying
>> to
>>>> find out.
>>
Re: Implicit preferences
Posted by Sean Owen <sr...@gmail.com>.
Yeah I bet it does actually work well... but aren't you basically spending
an extra step to make the "item-item" matrix, to relearn that bought-X and
viewed-X go together? yeah you learn a lot more along the way, as this is
item-based recommendation at heart. It seems like you could add back that
knowledge.
On Sun, Feb 10, 2013 at 5:36 PM, Ted Dunning <te...@gmail.com> wrote:
> Actually treating the different interactions separately can lead to very
> good recommendations. The only issue is that the interactions are no
> longer dyadic.
>
> If you think about it, having two different kinds of interactions is like
> adjoining interaction matrices for the two different kinds of interaction.
> Suppose that you have user x views in matrix A and you have user x
> purchases in matrix B. The complete interaction matrix of user x (views +
> purchases) is [A | B].
>
> When you compute cooccurrence in this matrix, you get
>
> [A | B] = [ A' ] [ A' A A' B ]
> [A | B]' [A | B] = [ ] [A | B] = [ ]
> [A | B] = [ B' ] [ B' A B' B ]
>
> This matrix is (view + purchase) x (view + purchase). But we don't care
> about predicting views so we only really need a matrix that is purchase x
> (view
> + purchase). This is just the bottom part of the matrix above, or [ B'A |
> B'B ]. When you produce purchase recommendations r_p by multiplying by a
> mixed view and purchase history vector h which has a view part h_v and a
> purchase part h_p, you get
>
> r_p = [ B' A B' B ] h = B'A h_v + B'B h_p
>
> That is a prediction of purchases based on past views and past purchase.
>
> Note that this general form applies for both decomposition techniques such
> as SVD, ALS and LLL as well as for sparsification techniques such as the
> LLR sparsification. All that changes is the mechanics of how you do the
> multiplications. Weighting of components works the same as well.
>
> What is very different here is that we have a component of cross
> recommendation. That is the B'A in the formula above. This is very
> different from a normal recommendation and cannot be computed with the
> simple self-join that we normally have in Mahout cooccurrence computation
> and also very different from the decompositions that we normally do. It
> isn't hard to adapt the Mahout computations, however.
>
> When implementing a recommendation using a search engine as the base, these
> same techniques also work extremely well in my experience. What happens is
> that for each item that you would like to recommend, you would have one
> field that has components of B'A and one field that has components of B'B.
> It is handy to simply use the binary values of the sparsified versions of
> these and let the search engine handle the weighting of different
> components at query time. Having these components separated into different
> fields in the search index seems to help quite a lot, which makes a fair
> bit of sense.
>
> On Sun, Feb 10, 2013 at 9:55 AM, Sean Owen <sr...@gmail.com> wrote:
> >
> > I think you'd have to hack the code to not exclude previously-seen items,
> > or at least, not of the type you wish to consider. Yes you would also
> have
> > to hack it to add rather than replace existing values. Or for test
> > purposes, just do the adding yourself before inputting the data.
> >
> > My hunch is that it will hurt non-trivially to treat different
> interaction
> > types as different items. You probably want to predict that someone who
> > viewed a product over and over is likely to buy it, but this would only
> > weakly tend to occur if the bought-item is not the same thing as the
> > viewed-item. You'd learn they go together but not as strongly as ought to
> > be obvious from the fact that they're the same. Still, interesting
> thought.
> >
> > There ought to be some 'signal' in this data, just a question of how much
> > vs noise. A purchase means much more than a page view of course; it's not
> > as subject to noise. Finding a means to use that info is probably going
> to
> > help.
> >
> >
> >
> >
> > On Sat, Feb 9, 2013 at 7:50 PM, Pat Ferrel <pa...@gmail.com> wrote:
> >
> > > I'd like to experiment with using several types of implicit preference
> > > values with recommenders. I have purchases as an implicit pref of high
> > > strength. I'd like to see if add-to-cart, view-product-details,
> > > impressions-seen, etc. can increase offline precision in holdout tests.
> > > These less than obvious implicit prefs will get a much lower value than
> > > purchase and i'll experiment with different mixes. The problem is that
> some
> > > of these prefs will indicate that the user, for whom I'm getting recs,
> has
> > > expressed a preference.
> > >
> > > Using these implicit prefs seems reasonable in finding similarity of
> taste
> > > between users but presents several problems. 1) how to encode the
> prefs,
> > > each impression-seen will increase the strength of preference of a user
> for
> > > an item but the recommender framework replaces the preference value for
> > > items preferred more than once, doesn't it? 2) AFAIK the current
> > > recommender framework will return recs only for items that the user in
> > > question has expressed no preference for. If I use something like
> > > view-product-details or impressions-seen, I will be removing anything
> the
> > > user has seen from the recs, which is not what I want in this
> experiment.
> > >
> > > Has anyone tried something like this? I'm not convinced that these
> other
> > > implicit preferences will add anything to the recommender, just trying
> to
> > > find out.
>
Re: Cross recommendation
Posted by Ted Dunning <te...@gmail.com>.
On Fri, Feb 22, 2013 at 9:48 AM, Pat Ferrel <pa...@gmail.com> wrote:
> ... lots of stuff I can't answer right now ...
>
> Then the question is how to blend B'A h_v with B'B h_p?
>
Add the scores. With coherent weighting schemes that should work just
fine.
To do this, get the two recommendation lists and add scores for the common
elements and merge results with scores unchanged for the rest.
Re: Cross recommendation
Posted by Sean Owen <sr...@gmail.com>.
I may not be 100% following the thread, but:
Similarity metrics won't care whether some items are really actions and
some items are items. The math is the same. The problem which you may be
alluding to is the one I mentioned earlier -- there is no connection
between item and item-action in the model, when there plainly is in real
life. The upside is what Ted mention: you get to treat actions like views
separately from purchases, and yes it's also certain those aren't the same
thing in real life. YMMV.
The piece of code you're playing with has nothing to do with latent factor
models and won't learn weights. It's going to assume by default that all
items (+actions) are equal.
(user+action,item) doesn't make sense. You compute item-item similarity
from (user,item+action) data. Some of the results are really item-action
similarities or action-action. It may be useful, maybe not, to know these
things too but you can just look at item-item if you want.
On Sun, Feb 24, 2013 at 4:39 PM, Pat Ferrel <pa...@gmail.com> wrote:
> Yes I understand that you need (user, item+action) input for user based
> recs returned from recommender.recommend(userID, n).
>
> But can you expect item similarity to work with the same input? I am fuzzy
> about how item similarity is calculated in cf/taste.
>
> I was expecting to train one recommender with (user, item+action) and call
> recommender1.recommend(userID, n) to get recs but also train another
> recommender with (user+action, item) to get recommender2.mostSimilarItems(
> itemID, n). I realize it's a hack but that aside is this second recommender
> required? I'd expect it to return items that use all actions to calculate
> similarity and therefore will use view information to improve the
> similarity calculation.
>
> No?
>
>
> On Feb 23, 2013, at 10:06 PM, Ted Dunning <te...@gmail.com> wrote:
>
> No.
>
> It is uniformly better to have (item+action, user). In fact, I would
> prefer to have it the other way around when describing it to match the
> matrix row x column convention.
>
> (user, item+action) where action is binary leads to A = [A_1 | A_2] = user
> by 2xitem. The alternative of (user+action, item) leads to
>
> [ A_1 ]
> A = [ ] = 2xuser by item
> [ A_2 ]
>
> This last form doesn't have a uniform set of users to connect the items
> together. When you compute the cooccurrence matrix you get A_1' A_1 + A_2'
> A_2 which gives you recommendations from 1=>1 and from 2=>2, but no
> recommendations 1=>2 or 2=>1. Thus, no cross recommendations.
>
>
>
> On Sat, Feb 23, 2013 at 10:39 AM, Pat Ferrel <pa...@gmail.com> wrote:
>
> > But the discussion below lead me to realize that cf/taste is doing
> > something in addition to [B'B] h_p, which returns user history based
> recs.
> > I'm getting better results currently from item similarity based recs,
> which
> > I blend with user-history based recs. To get item similarity based recs
> > cf/taste is using a similarity metric and I'd guess that it uses the
> input
> > matrix to get these results (something like the dot product for cosine).
> > For item similarity should I create a training set of (item,
> user+action)?
>
>
Re: Cross recommendation
Posted by Ted Dunning <te...@gmail.com>.
On Sun, Feb 24, 2013 at 8:39 AM, Pat Ferrel <pa...@gmail.com> wrote:
> ...
> But can you expect item similarity to work with the same input? I am fuzzy
> about how item similarity is calculated in cf/taste.
>
The cooccurrence oriented item-based recommenders should be fine.
I was expecting to train one recommender with (user, item+action) and call
> recommender1.recommend(userID, n) to get recs but also train another
> recommender with (user+action, item) to get recommender2.mostSimilarItems(
> itemID, n). I realize it's a hack but that aside is this second recommender
> required? I'd expect it to return items that use all actions to calculate
> similarity and therefore will use view information to improve the
> similarity calculation.
>
The second recommender is redundant.
Re: Cross recommendation
Posted by Pat Ferrel <pa...@gmail.com>.
Yes I understand that you need (user, item+action) input for user based recs returned from recommender.recommend(userID, n).
But can you expect item similarity to work with the same input? I am fuzzy about how item similarity is calculated in cf/taste.
I was expecting to train one recommender with (user, item+action) and call recommender1.recommend(userID, n) to get recs but also train another recommender with (user+action, item) to get recommender2.mostSimilarItems( itemID, n). I realize it's a hack but that aside is this second recommender required? I'd expect it to return items that use all actions to calculate similarity and therefore will use view information to improve the similarity calculation.
No?
On Feb 23, 2013, at 10:06 PM, Ted Dunning <te...@gmail.com> wrote:
No.
It is uniformly better to have (item+action, user). In fact, I would
prefer to have it the other way around when describing it to match the
matrix row x column convention.
(user, item+action) where action is binary leads to A = [A_1 | A_2] = user
by 2xitem. The alternative of (user+action, item) leads to
[ A_1 ]
A = [ ] = 2xuser by item
[ A_2 ]
This last form doesn't have a uniform set of users to connect the items
together. When you compute the cooccurrence matrix you get A_1' A_1 + A_2'
A_2 which gives you recommendations from 1=>1 and from 2=>2, but no
recommendations 1=>2 or 2=>1. Thus, no cross recommendations.
On Sat, Feb 23, 2013 at 10:39 AM, Pat Ferrel <pa...@gmail.com> wrote:
> But the discussion below lead me to realize that cf/taste is doing
> something in addition to [B'B] h_p, which returns user history based recs.
> I'm getting better results currently from item similarity based recs, which
> I blend with user-history based recs. To get item similarity based recs
> cf/taste is using a similarity metric and I'd guess that it uses the input
> matrix to get these results (something like the dot product for cosine).
> For item similarity should I create a training set of (item, user+action)?
Re: Cross recommendation
Posted by Ted Dunning <te...@gmail.com>.
No.
It is uniformly better to have (item+action, user). In fact, I would
prefer to have it the other way around when describing it to match the
matrix row x column convention.
(user, item+action) where action is binary leads to A = [A_1 | A_2] = user
by 2xitem. The alternative of (user+action, item) leads to
[ A_1 ]
A = [ ] = 2xuser by item
[ A_2 ]
This last form doesn't have a uniform set of users to connect the items
together. When you compute the cooccurrence matrix you get A_1' A_1 + A_2'
A_2 which gives you recommendations from 1=>1 and from 2=>2, but no
recommendations 1=>2 or 2=>1. Thus, no cross recommendations.
On Sat, Feb 23, 2013 at 10:39 AM, Pat Ferrel <pa...@gmail.com> wrote:
> But the discussion below lead me to realize that cf/taste is doing
> something in addition to [B'B] h_p, which returns user history based recs.
> I'm getting better results currently from item similarity based recs, which
> I blend with user-history based recs. To get item similarity based recs
> cf/taste is using a similarity metric and I'd guess that it uses the input
> matrix to get these results (something like the dot product for cosine).
> For item similarity should I create a training set of (item, user+action)?
Re: Cross recommendation
Posted by Pat Ferrel <pa...@gmail.com>.
This is exactly what I was trying to say in an earlier email. It's an easy first cut. I understand that it doesn't allow adjusting weights separately.
I'd create a training set of (item+action, user). So [B'B] will have four values for a single item, This will use the views for similarity of user but would also return view recs, which I could filter out.
But the discussion below lead me to realize that cf/taste is doing something in addition to [B'B] h_p, which returns user history based recs. I'm getting better results currently from item similarity based recs, which I blend with user-history based recs. To get item similarity based recs cf/taste is using a similarity metric and I'd guess that it uses the input matrix to get these results (something like the dot product for cosine). For item similarity should I create a training set of (item, user+action)?
Thanks for sticking with this. I suspect many recommender users will eventually get to the point where they want to use more than one implicit preference action. Especially in cases where you don't have a lot of cf type data. Using extra actions gives more ways to find a signal.
On Feb 22, 2013, at 3:46 PM, Ted Dunning <te...@gmail.com> wrote:
The absolutely minimal page is to give different ids to the different
actions against the same item. Then analyze as normal, but ignore
recommendations to item/action combos that you can't deal with. This
computes all four parts of the combined item recommendation matrix which is
roughly twice as much work as you need to do and it also doesn't let you
adjust weightings separately.
But it is probably the simplest way to get going with cross recommendation.
On Fri, Feb 22, 2013 at 9:48 AM, Pat Ferrel <pa...@gmail.com> wrote:
> Therefore my discussion was assuming the use of the entire mahout cf/taste
> framework even in the retrieving of recs. In that light B'B h_p is just
> another way of stating the usual train with user and items purchased then
> get recs for users and so that part of recs is covered. Since this also
> supports item similarity based queries (no user in the query) I'm covered.
>
> As to the B'A h_v part, isn't that just replacing where cf/taste would
> calc B'B, the self-join matrix, with the result of B'A? To use cf/taste you
> would ingest the user and items viewed data to create h_v for all users,
> but instead of allowing cf/taste to calculate B'B you would replace it with
> B'A. Then at query time taste would take a user and return purchase recs by
> calculating B'A h_v? isn't this correct? I hope someone comments on this
> because it is the route I plan to explore.
>
> The downside is that without lucene I would have two (or more) sets of
> recs to blend. I can make lucene return the raw recs fields but not sure
> how to return similarity based queries with lucene and don't really want to
> tackle that just yet (keep it simple?)
>
> Also in using cf/taste I don't need to create rows of the combined matrix,
> I can treat them as independent recommenders, which means I can tune them
> independently. I still have questions about how to generate the view values
> of A but that's another discussion.
>
Re: Cross recommendation
Posted by Ted Dunning <te...@gmail.com>.
The absolutely minimal page is to give different ids to the different
actions against the same item. Then analyze as normal, but ignore
recommendations to item/action combos that you can't deal with. This
computes all four parts of the combined item recommendation matrix which is
roughly twice as much work as you need to do and it also doesn't let you
adjust weightings separately.
But it is probably the simplest way to get going with cross recommendation.
On Fri, Feb 22, 2013 at 9:48 AM, Pat Ferrel <pa...@gmail.com> wrote:
> Therefore my discussion was assuming the use of the entire mahout cf/taste
> framework even in the retrieving of recs. In that light B'B h_p is just
> another way of stating the usual train with user and items purchased then
> get recs for users and so that part of recs is covered. Since this also
> supports item similarity based queries (no user in the query) I'm covered.
>
> As to the B'A h_v part, isn't that just replacing where cf/taste would
> calc B'B, the self-join matrix, with the result of B'A? To use cf/taste you
> would ingest the user and items viewed data to create h_v for all users,
> but instead of allowing cf/taste to calculate B'B you would replace it with
> B'A. Then at query time taste would take a user and return purchase recs by
> calculating B'A h_v? isn't this correct? I hope someone comments on this
> because it is the route I plan to explore.
>
> The downside is that without lucene I would have two (or more) sets of
> recs to blend. I can make lucene return the raw recs fields but not sure
> how to return similarity based queries with lucene and don't really want to
> tackle that just yet (keep it simple?)
>
> Also in using cf/taste I don't need to create rows of the combined matrix,
> I can treat them as independent recommenders, which means I can tune them
> independently. I still have questions about how to generate the view values
> of A but that's another discussion.
>
Re: Cross recommendation
Posted by Pat Ferrel <pa...@gmail.com>.
My plan was to NOT use lucene to start with though I see the benefits. This is because I want to experiment with weighting--doing idf, no weighting, and with a non-log idf. Also I want to experiment with temporal decay of recomendability and maybe blend item similarity based results in certain cases. Getting the raw recs is important to the experiment so I'd like to use mahout cf/taste if possible.
Therefore my discussion was assuming the use of the entire mahout cf/taste framework even in the retrieving of recs. In that light B'B h_p is just another way of stating the usual train with user and items purchased then get recs for users and so that part of recs is covered. Since this also supports item similarity based queries (no user in the query) I'm covered.
As to the B'A h_v part, isn't that just replacing where cf/taste would calc B'B, the self-join matrix, with the result of B'A? To use cf/taste you would ingest the user and items viewed data to create h_v for all users, but instead of allowing cf/taste to calculate B'B you would replace it with B'A. Then at query time taste would take a user and return purchase recs by calculating B'A h_v? isn't this correct? I hope someone comments on this because it is the route I plan to explore.
The downside is that without lucene I would have two (or more) sets of recs to blend. I can make lucene return the raw recs fields but not sure how to return similarity based queries with lucene and don't really want to tackle that just yet (keep it simple?)
Also in using cf/taste I don't need to create rows of the combined matrix, I can treat them as independent recommenders, which means I can tune them independently. I still have questions about how to generate the view values of A but that's another discussion.
Then the question is how to blend B'A h_v with B'B h_p?
The range of both of these will be identical. Each row of [B'A | B'B] corresponds to a document. One field (the view=>purchase indicators) contains a row of B'A and another field (the purchase=>purchase indicators) will contain a row of B'B.
The query will ultimately contain two fields corresponding to recent views and recent purchases. The search engine will combine the scores from these intelligently without any effort on your part. You can tune how this works, but I haven't ever found that very useful.
Re: Cross recommendation
Posted by Ted Dunning <te...@gmail.com>.
On Thu, Feb 21, 2013 at 12:13 PM, Pat Ferrel <pa...@gmail.com> wrote:
> I am quite interested in trying this but have a few questions.
>
> To use/abuse mahout to do this:
>
> A and B can be thought of as having the same size, in other words they
> must be constructed to have the same dimension definitions (userID for
> rows, itemID for columns) as well as row and column rank.
>
I generally only assume the same userID space (i.e. same number of rows).
> So A and B should be constructed to have the same users and items, even
> though a column or row in either matrix may be 0. Some work will be needed
> to produce this so I want to be sure I understand.
>
Less work if you only assume user identity.
If you co-group your transaction data, then you commonly will get a list of
A transactions and a list of B transactions for each user which is all you
should need.
The conclusion is B'A h_v + B'B h_p will produce view-based cross
> recommendations along with purchase based recs.
>
This term only produces purchase recommendations. If you want both, you
need to include the A'A h_v + A'B h_p part of the vector.
> Using mahout I'd get purchase based recs by constructing the usual
> purchase matrix [user x item] and getting recs from the framework, as I'm
> already doing. Assuming here that B'B is what you are calling the self-join?
>
Yes.
> So the primary work to be done is to calculate B'A. Since the CF/taste
> framework does the self-join to prepare for making recs for users, I would
> have to replace the self-joined matrix with B'A then allow the rest of the
> framework to produce recs in the usual way as if it had calculated the
> self-joined matrix. Does this seem reasonable?
>
Yes. In pig, there is a co-group that would be helpful. Alternately, you
can group views and purchases separately and then join. Co-group saves a
map-reduce step.
> Then the question is how to blend B'A h_v with B'B h_p?
>
The range of both of these will be identical. Each row of [B'A | B'B]
corresponds to a document. One field (the view=>purchase indicators)
contains a row of B'A and another field (the purchase=>purchase indicators)
will contain a row of B'B.
The query will ultimately contain two fields corresponding to recent views
and recent purchases. The search engine will combine the scores from these
intelligently without any effort on your part. You can tune how this
works, but I haven't ever found that very useful.
Since I'm using values = 1 or 0 the strengths will be on the same scale,
> but are they really comparable?
>
Missing from this formulation are the weights that the search engine places
on things. This has to do with how many of items have each indicator.
Common indicators will have little weight and rare ones great weight. For
instance, it might be that everybody (well, all the car buffs, anyway)
looks at the Ferrari's, but few buy them. This would mean that the views
would have little weight but the purchase would have a large weight.
I'd be inclined to try sorting B'A recs + B'B recs by strength and try some
> other experiments with blending techniques then look at eval metrics.
>
Start simple.
Remember that these are systems with feedback. That means that once the
system goes live (and if you have dithering set up) it will quickly tune
away the false-positive mistakes it is making by showing the events in
question and seeing that they don't lead to success. This closed loop
nature makes excessive refinement in weighting schemes largely unnecessary.
Cross recommendation
Posted by Pat Ferrel <pa...@gmail.com>.
I am quite interested in trying this but have a few questions.
To use/abuse mahout to do this:
A and B can be thought of as having the same size, in other words they must be constructed to have the same dimension definitions (userID for rows, itemID for columns) as well as row and column rank. So A and B should be constructed to have the same users and items, even though a column or row in either matrix may be 0. Some work will be needed to produce this so I want to be sure I understand.
The conclusion is B'A h_v + B'B h_p will produce view-based cross recommendations along with purchase based recs.
Using mahout I'd get purchase based recs by constructing the usual purchase matrix [user x item] and getting recs from the framework, as I'm already doing. Assuming here that B'B is what you are calling the self-join?
So the primary work to be done is to calculate B'A. Since the CF/taste framework does the self-join to prepare for making recs for users, I would have to replace the self-joined matrix with B'A then allow the rest of the framework to produce recs in the usual way as if it had calculated the self-joined matrix. Does this seem reasonable?
Then the question is how to blend B'A h_v with B'B h_p? Since I'm using values = 1 or 0 the strengths will be on the same scale, but are they really comparable? I'd be inclined to try sorting B'A recs + B'B recs by strength and try some other experiments with blending techniques then look at eval metrics.
On Feb 10, 2013, at 9:36 AM, Ted Dunning <te...@gmail.com> wrote:
Actually treating the different interactions separately can lead to very
good recommendations. The only issue is that the interactions are no
longer dyadic.
If you think about it, having two different kinds of interactions is like
adjoining interaction matrices for the two different kinds of interaction.
Suppose that you have user x views in matrix A and you have user x
purchases in matrix B. The complete interaction matrix of user x (views +
purchases) is [A | B].
When you compute cooccurrence in this matrix, you get
[A | B] = [ A' ] [ A' A A' B ]
[A | B]' [A | B] = [ ] [A | B] = [ ]
[A | B] = [ B' ] [ B' A B' B ]
This matrix is (view + purchase) x (view + purchase). But we don't care
about predicting views so we only really need a matrix that is purchase x (view
+ purchase). This is just the bottom part of the matrix above, or [ B'A |
B'B ]. When you produce purchase recommendations r_p by multiplying by a
mixed view and purchase history vector h which has a view part h_v and a
purchase part h_p, you get
r_p = [ B' A B' B ] h = B'A h_v + B'B h_p
That is a prediction of purchases based on past views and past purchase.
Note that this general form applies for both decomposition techniques such
as SVD, ALS and LLL as well as for sparsification techniques such as the
LLR sparsification. All that changes is the mechanics of how you do the
multiplications. Weighting of components works the same as well.
What is very different here is that we have a component of cross
recommendation. That is the B'A in the formula above. This is very
different from a normal recommendation and cannot be computed with the
simple self-join that we normally have in Mahout cooccurrence computation
and also very different from the decompositions that we normally do. It
isn't hard to adapt the Mahout computations, however.
When implementing a recommendation using a search engine as the base, these
same techniques also work extremely well in my experience. What happens is
that for each item that you would like to recommend, you would have one
field that has components of B'A and one field that has components of B'B.
It is handy to simply use the binary values of the sparsified versions of
these and let the search engine handle the weighting of different
components at query time. Having these components separated into different
fields in the search index seems to help quite a lot, which makes a fair
bit of sense.
On Sun, Feb 10, 2013 at 9:55 AM, Sean Owen <sr...@gmail.com> wrote:
>
> I think you'd have to hack the code to not exclude previously-seen items,
> or at least, not of the type you wish to consider. Yes you would also have
> to hack it to add rather than replace existing values. Or for test
> purposes, just do the adding yourself before inputting the data.
>
> My hunch is that it will hurt non-trivially to treat different interaction
> types as different items. You probably want to predict that someone who
> viewed a product over and over is likely to buy it, but this would only
> weakly tend to occur if the bought-item is not the same thing as the
> viewed-item. You'd learn they go together but not as strongly as ought to
> be obvious from the fact that they're the same. Still, interesting
thought.
>
> There ought to be some 'signal' in this data, just a question of how much
> vs noise. A purchase means much more than a page view of course; it's not
> as subject to noise. Finding a means to use that info is probably going to
> help.
>
>
>
>
> On Sat, Feb 9, 2013 at 7:50 PM, Pat Ferrel <pa...@gmail.com> wrote:
>
>> I'd like to experiment with using several types of implicit preference
>> values with recommenders. I have purchases as an implicit pref of high
>> strength. I'd like to see if add-to-cart, view-product-details,
>> impressions-seen, etc. can increase offline precision in holdout tests.
>> These less than obvious implicit prefs will get a much lower value than
>> purchase and i'll experiment with different mixes. The problem is that
some
>> of these prefs will indicate that the user, for whom I'm getting recs,
has
>> expressed a preference.
>>
>> Using these implicit prefs seems reasonable in finding similarity of
taste
>> between users but presents several problems. 1) how to encode the prefs,
>> each impression-seen will increase the strength of preference of a user
for
>> an item but the recommender framework replaces the preference value for
>> items preferred more than once, doesn't it? 2) AFAIK the current
>> recommender framework will return recs only for items that the user in
>> question has expressed no preference for. If I use something like
>> view-product-details or impressions-seen, I will be removing anything
the
>> user has seen from the recs, which is not what I want in this
experiment.
>>
>> Has anyone tried something like this? I'm not convinced that these other
>> implicit preferences will add anything to the recommender, just trying
to
>> find out.
Re: Implicit preferences
Posted by Ted Dunning <te...@gmail.com>.
On Tue, Feb 12, 2013 at 9:18 AM, Pat Ferrel <pa...@gmail.com> wrote:
> Maybe I'm splitting hairs here but I was thinking of using the frequency
> in the views. Maybe above some number of view = 1, below = 0.
>
> In any case this just seems like a different way of constructing A, no?
>
Correct.
In general, getting the best surrogate for your measure of engagement is
one of the hardest things to do.
Frequency might be right, but isn't very general. For videos, I have used
>30 second views. For music, time works, but is very complicated. For web
pages, time spent in focus might be useful (but you have to worry about
people who open lots of pages in new tabs... thus the focus qualifier).
The trick with all of these is that you want to avoid good headlines or
descriptions that lead to bad content.
Surrogates are usually necessary for training rec engines because the real
target is either unobservable (user liked it) or too rare (purchases,
perhaps) or takes too long to measure (user takes meat-space action over a
period of weeks).
>
> On Feb 10, 2013, at 9:36 AM, Ted Dunning <te...@gmail.com> wrote:
>
> Actually treating the different interactions separately can lead to very
> good recommendations. The only issue is that the interactions are no
> longer dyadic.
>
> If you think about it, having two different kinds of interactions is like
> adjoining interaction matrices for the two different kinds of interaction.
> Suppose that you have user x views in matrix A and you have user x
> purchases in matrix B. The complete interaction matrix of user x (views +
> purchases) is [A | B].
>
> When you compute cooccurrence in this matrix, you get
>
> [A | B] = [ A' ] [ A' A A' B ]
> [A | B]' [A | B] = [ ] [A | B] = [ ]
> [A | B] = [ B' ] [ B' A B' B ]
>
> This matrix is (view + purchase) x (view + purchase). But we don't care
> about predicting views so we only really need a matrix that is purchase x
> (view
> + purchase). This is just the bottom part of the matrix above, or [ B'A |
> B'B ]. When you produce purchase recommendations r_p by multiplying by a
> mixed view and purchase history vector h which has a view part h_v and a
> purchase part h_p, you get
>
> r_p = [ B' A B' B ] h = B'A h_v + B'B h_p
>
> That is a prediction of purchases based on past views and past purchase.
>
> Note that this general form applies for both decomposition techniques such
> as SVD, ALS and LLL as well as for sparsification techniques such as the
> LLR sparsification. All that changes is the mechanics of how you do the
> multiplications. Weighting of components works the same as well.
>
> What is very different here is that we have a component of cross
> recommendation. That is the B'A in the formula above. This is very
> different from a normal recommendation and cannot be computed with the
> simple self-join that we normally have in Mahout cooccurrence computation
> and also very different from the decompositions that we normally do. It
> isn't hard to adapt the Mahout computations, however.
>
> When implementing a recommendation using a search engine as the base, these
> same techniques also work extremely well in my experience. What happens is
> that for each item that you would like to recommend, you would have one
> field that has components of B'A and one field that has components of B'B.
> It is handy to simply use the binary values of the sparsified versions of
> these and let the search engine handle the weighting of different
> components at query time. Having these components separated into different
> fields in the search index seems to help quite a lot, which makes a fair
> bit of sense.
>
> On Sun, Feb 10, 2013 at 9:55 AM, Sean Owen <sr...@gmail.com> wrote:
> >
> > I think you'd have to hack the code to not exclude previously-seen items,
> > or at least, not of the type you wish to consider. Yes you would also
> have
> > to hack it to add rather than replace existing values. Or for test
> > purposes, just do the adding yourself before inputting the data.
> >
> > My hunch is that it will hurt non-trivially to treat different
> interaction
> > types as different items. You probably want to predict that someone who
> > viewed a product over and over is likely to buy it, but this would only
> > weakly tend to occur if the bought-item is not the same thing as the
> > viewed-item. You'd learn they go together but not as strongly as ought to
> > be obvious from the fact that they're the same. Still, interesting
> thought.
> >
> > There ought to be some 'signal' in this data, just a question of how much
> > vs noise. A purchase means much more than a page view of course; it's not
> > as subject to noise. Finding a means to use that info is probably going
> to
> > help.
> >
> >
> >
> >
> > On Sat, Feb 9, 2013 at 7:50 PM, Pat Ferrel <pa...@gmail.com> wrote:
> >
> >> I'd like to experiment with using several types of implicit preference
> >> values with recommenders. I have purchases as an implicit pref of high
> >> strength. I'd like to see if add-to-cart, view-product-details,
> >> impressions-seen, etc. can increase offline precision in holdout tests.
> >> These less than obvious implicit prefs will get a much lower value than
> >> purchase and i'll experiment with different mixes. The problem is that
> some
> >> of these prefs will indicate that the user, for whom I'm getting recs,
> has
> >> expressed a preference.
> >>
> >> Using these implicit prefs seems reasonable in finding similarity of
> taste
> >> between users but presents several problems. 1) how to encode the prefs,
> >> each impression-seen will increase the strength of preference of a user
> for
> >> an item but the recommender framework replaces the preference value for
> >> items preferred more than once, doesn't it? 2) AFAIK the current
> >> recommender framework will return recs only for items that the user in
> >> question has expressed no preference for. If I use something like
> >> view-product-details or impressions-seen, I will be removing anything
> the
> >> user has seen from the recs, which is not what I want in this
> experiment.
> >>
> >> Has anyone tried something like this? I'm not convinced that these other
> >> implicit preferences will add anything to the recommender, just trying
> to
> >> find out.
>
>
Re: Implicit preferences
Posted by Pat Ferrel <pa...@gmail.com>.
Maybe I'm splitting hairs here but I was thinking of using the frequency in the views. Maybe above some number of view = 1, below = 0.
In any case this just seems like a different way of constructing A, no?
On Feb 10, 2013, at 9:36 AM, Ted Dunning <te...@gmail.com> wrote:
Actually treating the different interactions separately can lead to very
good recommendations. The only issue is that the interactions are no
longer dyadic.
If you think about it, having two different kinds of interactions is like
adjoining interaction matrices for the two different kinds of interaction.
Suppose that you have user x views in matrix A and you have user x
purchases in matrix B. The complete interaction matrix of user x (views +
purchases) is [A | B].
When you compute cooccurrence in this matrix, you get
[A | B] = [ A' ] [ A' A A' B ]
[A | B]' [A | B] = [ ] [A | B] = [ ]
[A | B] = [ B' ] [ B' A B' B ]
This matrix is (view + purchase) x (view + purchase). But we don't care
about predicting views so we only really need a matrix that is purchase x (view
+ purchase). This is just the bottom part of the matrix above, or [ B'A |
B'B ]. When you produce purchase recommendations r_p by multiplying by a
mixed view and purchase history vector h which has a view part h_v and a
purchase part h_p, you get
r_p = [ B' A B' B ] h = B'A h_v + B'B h_p
That is a prediction of purchases based on past views and past purchase.
Note that this general form applies for both decomposition techniques such
as SVD, ALS and LLL as well as for sparsification techniques such as the
LLR sparsification. All that changes is the mechanics of how you do the
multiplications. Weighting of components works the same as well.
What is very different here is that we have a component of cross
recommendation. That is the B'A in the formula above. This is very
different from a normal recommendation and cannot be computed with the
simple self-join that we normally have in Mahout cooccurrence computation
and also very different from the decompositions that we normally do. It
isn't hard to adapt the Mahout computations, however.
When implementing a recommendation using a search engine as the base, these
same techniques also work extremely well in my experience. What happens is
that for each item that you would like to recommend, you would have one
field that has components of B'A and one field that has components of B'B.
It is handy to simply use the binary values of the sparsified versions of
these and let the search engine handle the weighting of different
components at query time. Having these components separated into different
fields in the search index seems to help quite a lot, which makes a fair
bit of sense.
On Sun, Feb 10, 2013 at 9:55 AM, Sean Owen <sr...@gmail.com> wrote:
>
> I think you'd have to hack the code to not exclude previously-seen items,
> or at least, not of the type you wish to consider. Yes you would also have
> to hack it to add rather than replace existing values. Or for test
> purposes, just do the adding yourself before inputting the data.
>
> My hunch is that it will hurt non-trivially to treat different interaction
> types as different items. You probably want to predict that someone who
> viewed a product over and over is likely to buy it, but this would only
> weakly tend to occur if the bought-item is not the same thing as the
> viewed-item. You'd learn they go together but not as strongly as ought to
> be obvious from the fact that they're the same. Still, interesting
thought.
>
> There ought to be some 'signal' in this data, just a question of how much
> vs noise. A purchase means much more than a page view of course; it's not
> as subject to noise. Finding a means to use that info is probably going to
> help.
>
>
>
>
> On Sat, Feb 9, 2013 at 7:50 PM, Pat Ferrel <pa...@gmail.com> wrote:
>
>> I'd like to experiment with using several types of implicit preference
>> values with recommenders. I have purchases as an implicit pref of high
>> strength. I'd like to see if add-to-cart, view-product-details,
>> impressions-seen, etc. can increase offline precision in holdout tests.
>> These less than obvious implicit prefs will get a much lower value than
>> purchase and i'll experiment with different mixes. The problem is that
some
>> of these prefs will indicate that the user, for whom I'm getting recs,
has
>> expressed a preference.
>>
>> Using these implicit prefs seems reasonable in finding similarity of
taste
>> between users but presents several problems. 1) how to encode the prefs,
>> each impression-seen will increase the strength of preference of a user
for
>> an item but the recommender framework replaces the preference value for
>> items preferred more than once, doesn't it? 2) AFAIK the current
>> recommender framework will return recs only for items that the user in
>> question has expressed no preference for. If I use something like
>> view-product-details or impressions-seen, I will be removing anything
the
>> user has seen from the recs, which is not what I want in this
experiment.
>>
>> Has anyone tried something like this? I'm not convinced that these other
>> implicit preferences will add anything to the recommender, just trying
to
>> find out.
Re: Implicit preferences
Posted by Ted Dunning <te...@gmail.com>.
Actually treating the different interactions separately can lead to very
good recommendations. The only issue is that the interactions are no
longer dyadic.
If you think about it, having two different kinds of interactions is like
adjoining interaction matrices for the two different kinds of interaction.
Suppose that you have user x views in matrix A and you have user x
purchases in matrix B. The complete interaction matrix of user x (views +
purchases) is [A | B].
When you compute cooccurrence in this matrix, you get
[A | B] = [ A' ] [ A' A A' B ]
[A | B]' [A | B] = [ ] [A | B] = [ ]
[A | B] = [ B' ] [ B' A B' B ]
This matrix is (view + purchase) x (view + purchase). But we don't care
about predicting views so we only really need a matrix that is purchase x (view
+ purchase). This is just the bottom part of the matrix above, or [ B'A |
B'B ]. When you produce purchase recommendations r_p by multiplying by a
mixed view and purchase history vector h which has a view part h_v and a
purchase part h_p, you get
r_p = [ B' A B' B ] h = B'A h_v + B'B h_p
That is a prediction of purchases based on past views and past purchase.
Note that this general form applies for both decomposition techniques such
as SVD, ALS and LLL as well as for sparsification techniques such as the
LLR sparsification. All that changes is the mechanics of how you do the
multiplications. Weighting of components works the same as well.
What is very different here is that we have a component of cross
recommendation. That is the B'A in the formula above. This is very
different from a normal recommendation and cannot be computed with the
simple self-join that we normally have in Mahout cooccurrence computation
and also very different from the decompositions that we normally do. It
isn't hard to adapt the Mahout computations, however.
When implementing a recommendation using a search engine as the base, these
same techniques also work extremely well in my experience. What happens is
that for each item that you would like to recommend, you would have one
field that has components of B'A and one field that has components of B'B.
It is handy to simply use the binary values of the sparsified versions of
these and let the search engine handle the weighting of different
components at query time. Having these components separated into different
fields in the search index seems to help quite a lot, which makes a fair
bit of sense.
On Sun, Feb 10, 2013 at 9:55 AM, Sean Owen <sr...@gmail.com> wrote:
>
> I think you'd have to hack the code to not exclude previously-seen items,
> or at least, not of the type you wish to consider. Yes you would also have
> to hack it to add rather than replace existing values. Or for test
> purposes, just do the adding yourself before inputting the data.
>
> My hunch is that it will hurt non-trivially to treat different interaction
> types as different items. You probably want to predict that someone who
> viewed a product over and over is likely to buy it, but this would only
> weakly tend to occur if the bought-item is not the same thing as the
> viewed-item. You'd learn they go together but not as strongly as ought to
> be obvious from the fact that they're the same. Still, interesting
thought.
>
> There ought to be some 'signal' in this data, just a question of how much
> vs noise. A purchase means much more than a page view of course; it's not
> as subject to noise. Finding a means to use that info is probably going to
> help.
>
>
>
>
> On Sat, Feb 9, 2013 at 7:50 PM, Pat Ferrel <pa...@gmail.com> wrote:
>
> > I'd like to experiment with using several types of implicit preference
> > values with recommenders. I have purchases as an implicit pref of high
> > strength. I'd like to see if add-to-cart, view-product-details,
> > impressions-seen, etc. can increase offline precision in holdout tests.
> > These less than obvious implicit prefs will get a much lower value than
> > purchase and i'll experiment with different mixes. The problem is that
some
> > of these prefs will indicate that the user, for whom I'm getting recs,
has
> > expressed a preference.
> >
> > Using these implicit prefs seems reasonable in finding similarity of
taste
> > between users but presents several problems. 1) how to encode the prefs,
> > each impression-seen will increase the strength of preference of a user
for
> > an item but the recommender framework replaces the preference value for
> > items preferred more than once, doesn't it? 2) AFAIK the current
> > recommender framework will return recs only for items that the user in
> > question has expressed no preference for. If I use something like
> > view-product-details or impressions-seen, I will be removing anything
the
> > user has seen from the recs, which is not what I want in this
experiment.
> >
> > Has anyone tried something like this? I'm not convinced that these other
> > implicit preferences will add anything to the recommender, just trying
to
> > find out.
Re: Implicit preferences
Posted by Sean Owen <sr...@gmail.com>.
I think you'd have to hack the code to not exclude previously-seen items,
or at least, not of the type you wish to consider. Yes you would also have
to hack it to add rather than replace existing values. Or for test
purposes, just do the adding yourself before inputting the data.
My hunch is that it will hurt non-trivially to treat different interaction
types as different items. You probably want to predict that someone who
viewed a product over and over is likely to buy it, but this would only
weakly tend to occur if the bought-item is not the same thing as the
viewed-item. You'd learn they go together but not as strongly as ought to
be obvious from the fact that they're the same. Still, interesting thought.
There ought to be some 'signal' in this data, just a question of how much
vs noise. A purchase means much more than a page view of course; it's not
as subject to noise. Finding a means to use that info is probably going to
help.
On Sat, Feb 9, 2013 at 7:50 PM, Pat Ferrel <pa...@gmail.com> wrote:
> I'd like to experiment with using several types of implicit preference
> values with recommenders. I have purchases as an implicit pref of high
> strength. I'd like to see if add-to-cart, view-product-details,
> impressions-seen, etc. can increase offline precision in holdout tests.
> These less than obvious implicit prefs will get a much lower value than
> purchase and i'll experiment with different mixes. The problem is that some
> of these prefs will indicate that the user, for whom I'm getting recs, has
> expressed a preference.
>
> Using these implicit prefs seems reasonable in finding similarity of taste
> between users but presents several problems. 1) how to encode the prefs,
> each impression-seen will increase the strength of preference of a user for
> an item but the recommender framework replaces the preference value for
> items preferred more than once, doesn't it? 2) AFAIK the current
> recommender framework will return recs only for items that the user in
> question has expressed no preference for. If I use something like
> view-product-details or impressions-seen, I will be removing anything the
> user has seen from the recs, which is not what I want in this experiment.
>
> Has anyone tried something like this? I'm not convinced that these other
> implicit preferences will add anything to the recommender, just trying to
> find out.