market basket analysis of low sales volume products

[Si Chen <si...@opensourcestrategies.com>]

Hi everybody,

I'd like to do some market basket analysis to suggest cross-sells, but many
of the products are very low sales volume items, so in the past the results
weren't that useful.

Do you think it would make sense to do market basket analysis at more
aggregate levels, for example by brand, product keywords, and product
categories, to develop a set of heuristic rules?  Then we can use those
rules to say that even if we haven't sold product X, because it has brand
A, category B, or type C, then it should be cross-sold with some other
products.

Does that sound like a reasonable strategy?  Has anybody ever tried this?

-- 
Si Chen
Open Source Strategies, Inc.
sichen@opensourcestrategies.com
http://www.OpenSourceStrategies.com
LinkedIn: http://www.linkedin.com/in/opentaps
Twitter: http://twitter.com/opentaps

Re: market basket analysis of low sales volume products

[Ted Dunning <te...@gmail.com>]

On Mon, Mar 24, 2014 at 4:46 PM, Si Chen <si...@opensourcestrategies.com>wrote:

> Thanks everybody for your feedback!  I thought more about it, and basically
> our issue is that we have a lot of SKU's per brand, so there's not a lot of
> repeat sales of the same SKU's to make SKU to SKU market basket analysis
> useful.  We can move it up to the brand level, but then I'm concerned that
> the results would be statistically significant but not very useful.  Some
> brands have 5K to 10K SKU's, so a cross-sell of brand A to brand B would
> still leave thousands of cross-sell candidates being suggested.
>

This sounds right.


>
> Thinking about it further, maybe what we should try to do is to divide our
> catalog into small units, finer than a brand but more aggregated than an
> individual SKU.  Maybe if we could partition our catalog into sub-units of
> 100 or so products each, then try to do market basket analysis at the
> levels of these units?  The products in each sub-unit would be related to
> each other, but they wouldn't be necessarily be bought together.
>

Good idea.


> Should we try to do it this way?  Is there a way to partition a catalog
> analytically, or would we have to do it based on human knowledge?
>

Well, yes and no.

The only way to analytically group SKU's into reasonably sensible groups is
to look at something like cooccurrence.  You said that doesn't work so you
have a dead end.

Do you have any way to connect market baskets together?

Other than that, external information will be required.

That said, I wouldn't limit the analysis to a single level.  If you have
Brand > X > SKU, then I would consider each occurrence of a SKU to be an
occurrence of X and of the Brand as well.

Re: market basket analysis of low sales volume products

[Si Chen <si...@opensourcestrategies.com>]

Thanks everybody for your feedback!  I thought more about it, and basically
our issue is that we have a lot of SKU's per brand, so there's not a lot of
repeat sales of the same SKU's to make SKU to SKU market basket analysis
useful.  We can move it up to the brand level, but then I'm concerned that
the results would be statistically significant but not very useful.  Some
brands have 5K to 10K SKU's, so a cross-sell of brand A to brand B would
still leave thousands of cross-sell candidates being suggested.

Thinking about it further, maybe what we should try to do is to divide our
catalog into small units, finer than a brand but more aggregated than an
individual SKU.  Maybe if we could partition our catalog into sub-units of
100 or so products each, then try to do market basket analysis at the
levels of these units?  The products in each sub-unit would be related to
each other, but they wouldn't be necessarily be bought together.

Should we try to do it this way?  Is there a way to partition a catalog
analytically, or would we have to do it based on human knowledge?


On Thu, Mar 20, 2014 at 8:28 PM, Ted Dunning <te...@gmail.com> wrote:

> I have done the equivalent thing with music (moving up from track to album
> to artist) with very good results.
>
>
>
> On Thu, Mar 20, 2014 at 5:58 PM, Martin, Nick <Ni...@pssd.com> wrote:
>
> > I can tell you my experience is that it's absolutely informative to take
> a
> > look at running the recommendation stuff on things other than items
> > (brands, categories, sub-categories, etc.). If you're in a multi-brand
> > environment it can give you a great view into brand pen by customer
> groups
> > pretty quickly. Instead of items just assign your categories (or brands,
> or
> > types, etc.) an ID and pass them through the recommendation algos.
> And/or,
> > if you'd like (and you have the metadata available), you can do the same
> > with customer segments/groups/etc.
> >
> > If you start to see deficiencies in brand spread for customers you don't
> > expect (or, even, don't like) you can inject that feedback into your
> > process. A good place to control that kind of thing is in the filter file
> > and items file - here you can control what items (or categories, or
> > sub-categories, or brands) make it into your output. You could even go so
> > far as to exclude low-margin items, only generate recs for categories in
> a
> > specific brand for which you're currently trying to increase penetration,
> > etc.
> >
> > Long answer, but I strongly suggest it's a "yes" and based on experience
> > dealing with this stuff day-to-day.
> >
> > Come to think of it, I think I owe a write-up on this whole kind of
> > thing...
> >
> > -----Original Message-----
> > From: Si Chen [mailto:sichen@opensourcestrategies.com]
> > Sent: Thursday, March 20, 2014 8:15 PM
> > To: user@mahout.apache.org
> > Subject: market basket analysis of low sales volume products
> >
> > Hi everybody,
> >
> > I'd like to do some market basket analysis to suggest cross-sells, but
> > many of the products are very low sales volume items, so in the past the
> > results weren't that useful.
> >
> > Do you think it would make sense to do market basket analysis at more
> > aggregate levels, for example by brand, product keywords, and product
> > categories, to develop a set of heuristic rules?  Then we can use those
> > rules to say that even if we haven't sold product X, because it has brand
> > A, category B, or type C, then it should be cross-sold with some other
> > products.
> >
> > Does that sound like a reasonable strategy?  Has anybody ever tried this?
> >
> > --
> > Si Chen
> > Open Source Strategies, Inc.
> > sichen@opensourcestrategies.com
> > http://www.OpenSourceStrategies.com
> > LinkedIn: http://www.linkedin.com/in/opentaps
> > Twitter: http://twitter.com/opentaps
> >
>



-- 
Si Chen
Open Source Strategies, Inc.
sichen@opensourcestrategies.com
http://www.OpenSourceStrategies.com
LinkedIn: http://www.linkedin.com/in/opentaps
Twitter: http://twitter.com/opentaps

Re: market basket analysis of low sales volume products

[Ted Dunning <te...@gmail.com>]

I have done the equivalent thing with music (moving up from track to album
to artist) with very good results.



On Thu, Mar 20, 2014 at 5:58 PM, Martin, Nick <Ni...@pssd.com> wrote:

> I can tell you my experience is that it's absolutely informative to take a
> look at running the recommendation stuff on things other than items
> (brands, categories, sub-categories, etc.). If you're in a multi-brand
> environment it can give you a great view into brand pen by customer groups
> pretty quickly. Instead of items just assign your categories (or brands, or
> types, etc.) an ID and pass them through the recommendation algos. And/or,
> if you'd like (and you have the metadata available), you can do the same
> with customer segments/groups/etc.
>
> If you start to see deficiencies in brand spread for customers you don't
> expect (or, even, don't like) you can inject that feedback into your
> process. A good place to control that kind of thing is in the filter file
> and items file - here you can control what items (or categories, or
> sub-categories, or brands) make it into your output. You could even go so
> far as to exclude low-margin items, only generate recs for categories in a
> specific brand for which you're currently trying to increase penetration,
> etc.
>
> Long answer, but I strongly suggest it's a "yes" and based on experience
> dealing with this stuff day-to-day.
>
> Come to think of it, I think I owe a write-up on this whole kind of
> thing...
>
> -----Original Message-----
> From: Si Chen [mailto:sichen@opensourcestrategies.com]
> Sent: Thursday, March 20, 2014 8:15 PM
> To: user@mahout.apache.org
> Subject: market basket analysis of low sales volume products
>
> Hi everybody,
>
> I'd like to do some market basket analysis to suggest cross-sells, but
> many of the products are very low sales volume items, so in the past the
> results weren't that useful.
>
> Do you think it would make sense to do market basket analysis at more
> aggregate levels, for example by brand, product keywords, and product
> categories, to develop a set of heuristic rules?  Then we can use those
> rules to say that even if we haven't sold product X, because it has brand
> A, category B, or type C, then it should be cross-sold with some other
> products.
>
> Does that sound like a reasonable strategy?  Has anybody ever tried this?
>
> --
> Si Chen
> Open Source Strategies, Inc.
> sichen@opensourcestrategies.com
> http://www.OpenSourceStrategies.com
> LinkedIn: http://www.linkedin.com/in/opentaps
> Twitter: http://twitter.com/opentaps
>

RE: market basket analysis of low sales volume products

["Martin, Nick" <Ni...@pssd.com>]

I can tell you my experience is that it's absolutely informative to take a look at running the recommendation stuff on things other than items (brands, categories, sub-categories, etc.). If you're in a multi-brand environment it can give you a great view into brand pen by customer groups pretty quickly. Instead of items just assign your categories (or brands, or types, etc.) an ID and pass them through the recommendation algos. And/or, if you'd like (and you have the metadata available), you can do the same with customer segments/groups/etc. 

If you start to see deficiencies in brand spread for customers you don't expect (or, even, don't like) you can inject that feedback into your process. A good place to control that kind of thing is in the filter file and items file - here you can control what items (or categories, or sub-categories, or brands) make it into your output. You could even go so far as to exclude low-margin items, only generate recs for categories in a specific brand for which you're currently trying to increase penetration, etc.

Long answer, but I strongly suggest it's a "yes" and based on experience dealing with this stuff day-to-day. 

Come to think of it, I think I owe a write-up on this whole kind of thing...

-----Original Message-----
From: Si Chen [mailto:sichen@opensourcestrategies.com] 
Sent: Thursday, March 20, 2014 8:15 PM
To: user@mahout.apache.org
Subject: market basket analysis of low sales volume products

Hi everybody,

I'd like to do some market basket analysis to suggest cross-sells, but many of the products are very low sales volume items, so in the past the results weren't that useful.

Do you think it would make sense to do market basket analysis at more aggregate levels, for example by brand, product keywords, and product categories, to develop a set of heuristic rules?  Then we can use those rules to say that even if we haven't sold product X, because it has brand A, category B, or type C, then it should be cross-sold with some other products.

Does that sound like a reasonable strategy?  Has anybody ever tried this?

--
Si Chen
Open Source Strategies, Inc.
sichen@opensourcestrategies.com
http://www.OpenSourceStrategies.com
LinkedIn: http://www.linkedin.com/in/opentaps
Twitter: http://twitter.com/opentaps

Re: market basket analysis of low sales volume products

[Andrew Musselman <an...@gmail.com>]

Yes that can help, depending on what you're doing.  Want to talk more about it?

> On Mar 20, 2014, at 5:14 PM, Si Chen <si...@opensourcestrategies.com> wrote:
> 
> Hi everybody,
> 
> I'd like to do some market basket analysis to suggest cross-sells, but many
> of the products are very low sales volume items, so in the past the results
> weren't that useful.
> 
> Do you think it would make sense to do market basket analysis at more
> aggregate levels, for example by brand, product keywords, and product
> categories, to develop a set of heuristic rules?  Then we can use those
> rules to say that even if we haven't sold product X, because it has brand
> A, category B, or type C, then it should be cross-sold with some other
> products.
> 
> Does that sound like a reasonable strategy?  Has anybody ever tried this?
> 
> -- 
> Si Chen
> Open Source Strategies, Inc.
> sichen@opensourcestrategies.com
> http://www.OpenSourceStrategies.com
> LinkedIn: http://www.linkedin.com/in/opentaps
> Twitter: http://twitter.com/opentaps