You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@mahout.apache.org by bejoy ks <be...@hotmail.com> on 2010/11/12 10:57:54 UTC
FW: Mahout - Help needed - files with no preferences and
integarting mahout with Hadoop
Hi Mahout Experts,
I'm totally new into mahout and badly indeed of some expert opinions. I
wanted
to implement a recommendation engine using mahout and hadoop, as the
initial
stage I just tried a recommender class with mahout alone which worked
fine . I’m
using eclipse for my development and to test I just ran the code from my
eclipse like a simple java class with the required jars already added on
to project build path. The code I used for testing User Similarity
recommendation is as follows
public static void main(String[] args) throws TasteException
{
String recsFile="D://mahoutFiles//SampleData.txt";
int
neighbourhoodSize=7;
long
ssoId=206008129;
int
noOfRecommendations=5;
try {
FileDataModel
dataModel = new FileDataModel(new File(recsFile));
UserSimilarity
userSimilarity = new PearsonCorrelationSimilarity(dataModel);
UserNeighborhood
neighborhood =new NearestNUserNeighborhood(neighbourhoodSize, userSimilarity,
dataModel);
Recommender
recommender =new GenericUserBasedRecommender(dataModel, neighborhood,
userSimilarity);
List<RecommendedItem>
recommendations =recommender.recommend(ssoId, noOfRecommendations);
for
(RecommendedItem recommendedItem : recommendations) {
System.out.println(recommendedItem.getItemID());
System.out.print(" Item:
"+dataModel.getItemIDAsString(recommendedItem.getItemID()));
System.out.println(" and
value: "+recommendedItem.getValue());
}
} catch (IOException e)
{
e.printStackTrace();
} catch (TasteException
e) {
e.printStackTrace();
}
}
The code I used for testing Item Similarity recommendation is as follows
public static void main(String[] args) {
String recsFile="D://mahoutFiles//SampleData.txt";
long ssoId=206008129;
int noOfRecommendations=5;
String itemId="GE-CORPPROG-IMLP-PTOOLS";
try {
//item similarity recommendation based on User
FileDataModel dataModel = new FileDataModel(new File(recsFile));
ItemSimilarity itemSimilarity = new LogLikelihoodSimilarity(dataModel);
ItemBasedRecommender recommender =new GenericItemBasedRecommender(dataModel, itemSimilarity);
List<RecommendedItem> recommendations =recommender.recommend(ssoId, noOfRecommendations);
for (RecommendedItem recommendedItem : recommendations) {
System.out.println(recommendedItem);
}
//item similarity recommendation based on item
recommender =new GenericItemBasedRecommender(dataModel, itemSimilarity);
recommendations= recommender.mostSimilarItems(dataModel.readItemIDFromString(itemId), noOfRecommendations);
System.out.println("Item recommendations based on Item");
for (RecommendedItem recommendedItem : recommendations) {
System.out.println(recommendedItem);
}
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (TasteException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
The issues or concerns I have are as follows.
Problem 1:
This code produces relevant
recommendations when I have given my file in the format <User id>, <item id>, <preference value>.
But in my actual sample file I have just the user id and Item id alone, no
preference value. On my reference since it is kind of a Boolean preference, I tried
using PearsonCorrelationSimilarity
as well as TanimotoCoefficientSimilarity but it is not giving me any
recommendations. My actual file format that I'd want to process would be in the format <user id>,
<item id>, . I can see it is processing all users in my file( from my
eclipse console logs) but no recommendations are being produced as output.
Problem 2:
I
want to use my mahout recommender in Hadoop distributed environment. I don’t have
a clue about it. I tried web help but most blogs refer me to org.apache.mahout.cf.taste.hadoop.item.RecommenderJob,
the code
available with mahout src. Unfortunately I’m not able to get much from
the code
as it is a bit complicated for a starter like me. (I have done a couple
of Map reduce project on text files processing before ). Could someone
please help me
out with very basic simple code snippets to run my User Similarity and
Item Similarity Recommendation code
on hadoop environment .
It would be much
helpful if you could share me with some informative tutorials which
could make my initial steps in Mahout comfortable before i could start exploring in
depth.
Thank you
Thanks and Regards
Bejoy.K.S
RE: Mahout - Help needed - files with no preferences and
integarting mahout with Hadoop
Posted by bejoy ks <be...@hotmail.com>.
Thanks Owen. It'd test my code the same way.
Thanks and Regards
Bejoy.K.S
> Date: Fri, 12 Nov 2010 15:45:16 +0000
> Subject: Re: Mahout - Help needed - files with no preferences and integarting mahout with Hadoop
> From: srowen@gmail.com
> To: user@mahout.apache.org
>
> It looks reasonable to me; what works best for your data will only be
> revealed with some testing of different algorithms. For example try
> LogLikelihoodSimilarity.
>
> On Fri, Nov 12, 2010 at 3:41 PM, bejoy ks <be...@hotmail.com> wrote:
>
> >
> > Thanks a lot Owen. One more small favor,hope its fine for you.
> > I'd like to get your suggestion on my implementation of my Item
> > based recommendation over the same data set.(data set with no
> > preference value).
> >
> > The sample code i have used is given below, it works fine with
> > recommendations being produced. Could you please look into it and just
> > verify whether the use of ItemSimilarity and ItemBasedRecommender are the
> > best suited ones for Boolean data sets.
> >
> >
> >
> > //item similarity recommendation based on User Id
> >
> > FileDataModel dataModel = new FileDataModel(new File(recsFile));
> >
> > ItemSimilarity itemSimilarity = new
> > TanimotoCoefficientSimilarity(dataModel);
> >
> > ItemBasedRecommender recommender =new
> > GenericItemBasedRecommender(dataModel, itemSimilarity);
> >
> > List<RecommendedItem> recommendations
> > =recommender.recommend(userId, noOfRecommendations);
> >
> > System.out.println(recommendations);
> >
> >
> >
> > //item similarity recommendation based on Item Id
> >
> > recommendations= recommender.mostSimilarItems(itemId,
> > noOfRecommendations);
> >
> > System.out.println(recommendations);
> >
> >
> >
> > Please let me know you suggestions and comments on the code snippet. Is
> > this the right way to so Item based recommendations on Boolean data
> > sets?
> >
> > Thanks and Regards
> > Bejoy.K.S
> >
> >
> >
> >
> > > Date: Fri, 12 Nov 2010 15:34:03 +0000
> > > Subject: Re: Mahout - Help needed - files with no preferences and
> > integarting mahout with Hadoop
> > > From: srowen@gmail.com
> > > To: user@mahout.apache.org
> > >
> > > http://manning.com/owen
> > >
> > > On Fri, Nov 12, 2010 at 3:16 PM, bejoy ks <be...@hotmail.com> wrote:
> > >
> > > >
> > > > Ok that'd be great Owen , if you could point me to the book 'Mahout in
> > > > Action' . I'm bit interested to know more on the possibilities
> > available
> > > > with mahout and also the right usage of similarities, recommenders
> >
> >
Re: Mahout - Help needed - files with no preferences and integarting
mahout with Hadoop
Posted by Sean Owen <sr...@gmail.com>.
It looks reasonable to me; what works best for your data will only be
revealed with some testing of different algorithms. For example try
LogLikelihoodSimilarity.
On Fri, Nov 12, 2010 at 3:41 PM, bejoy ks <be...@hotmail.com> wrote:
>
> Thanks a lot Owen. One more small favor,hope its fine for you.
> I'd like to get your suggestion on my implementation of my Item
> based recommendation over the same data set.(data set with no
> preference value).
>
> The sample code i have used is given below, it works fine with
> recommendations being produced. Could you please look into it and just
> verify whether the use of ItemSimilarity and ItemBasedRecommender are the
> best suited ones for Boolean data sets.
>
>
>
> //item similarity recommendation based on User Id
>
> FileDataModel dataModel = new FileDataModel(new File(recsFile));
>
> ItemSimilarity itemSimilarity = new
> TanimotoCoefficientSimilarity(dataModel);
>
> ItemBasedRecommender recommender =new
> GenericItemBasedRecommender(dataModel, itemSimilarity);
>
> List<RecommendedItem> recommendations
> =recommender.recommend(userId, noOfRecommendations);
>
> System.out.println(recommendations);
>
>
>
> //item similarity recommendation based on Item Id
>
> recommendations= recommender.mostSimilarItems(itemId,
> noOfRecommendations);
>
> System.out.println(recommendations);
>
>
>
> Please let me know you suggestions and comments on the code snippet. Is
> this the right way to so Item based recommendations on Boolean data
> sets?
>
> Thanks and Regards
> Bejoy.K.S
>
>
>
>
> > Date: Fri, 12 Nov 2010 15:34:03 +0000
> > Subject: Re: Mahout - Help needed - files with no preferences and
> integarting mahout with Hadoop
> > From: srowen@gmail.com
> > To: user@mahout.apache.org
> >
> > http://manning.com/owen
> >
> > On Fri, Nov 12, 2010 at 3:16 PM, bejoy ks <be...@hotmail.com> wrote:
> >
> > >
> > > Ok that'd be great Owen , if you could point me to the book 'Mahout in
> > > Action' . I'm bit interested to know more on the possibilities
> available
> > > with mahout and also the right usage of similarities, recommenders
>
>
RE: Mahout - Help needed - files with no preferences and
integarting mahout with Hadoop
Posted by bejoy ks <be...@hotmail.com>.
Thanks a lot Owen. One more small favor,hope its fine for you.
I'd like to get your suggestion on my implementation of my Item
based recommendation over the same data set.(data set with no
preference value).
The sample code i have used is given below, it works fine with
recommendations being produced. Could you please look into it and just
verify whether the use of ItemSimilarity and ItemBasedRecommender are the
best suited ones for Boolean data sets.
//item similarity recommendation based on User Id
FileDataModel dataModel = new FileDataModel(new File(recsFile));
ItemSimilarity itemSimilarity = new TanimotoCoefficientSimilarity(dataModel);
ItemBasedRecommender recommender =new GenericItemBasedRecommender(dataModel, itemSimilarity);
List<RecommendedItem> recommendations =recommender.recommend(userId, noOfRecommendations);
System.out.println(recommendations);
//item similarity recommendation based on Item Id
recommendations= recommender.mostSimilarItems(itemId, noOfRecommendations);
System.out.println(recommendations);
Please let me know you suggestions and comments on the code snippet. Is
this the right way to so Item based recommendations on Boolean data
sets?
Thanks and Regards
Bejoy.K.S
> Date: Fri, 12 Nov 2010 15:34:03 +0000
> Subject: Re: Mahout - Help needed - files with no preferences and integarting mahout with Hadoop
> From: srowen@gmail.com
> To: user@mahout.apache.org
>
> http://manning.com/owen
>
> On Fri, Nov 12, 2010 at 3:16 PM, bejoy ks <be...@hotmail.com> wrote:
>
> >
> > Ok that'd be great Owen , if you could point me to the book 'Mahout in
> > Action' . I'm bit interested to know more on the possibilities available
> > with mahout and also the right usage of similarities, recommenders
Re: Mahout - Help needed - files with no preferences and integarting
mahout with Hadoop
Posted by Sean Owen <sr...@gmail.com>.
http://manning.com/owen
On Fri, Nov 12, 2010 at 3:16 PM, bejoy ks <be...@hotmail.com> wrote:
>
> Ok that'd be great Owen , if you could point me to the book 'Mahout in
> Action' . I'm bit interested to know more on the possibilities available
> with mahout and also the right usage of similarities, recommenders
RE: Mahout - Help needed - files with no preferences and
integarting mahout with Hadoop
Posted by bejoy ks <be...@hotmail.com>.
Ok that'd be great Owen , if you could point me to the book 'Mahout in Action' . I'm bit interested to know more on the possibilities available with mahout and also the right usage of similarities, recommenders etc. I'd spend more time on the java doc get my understanding crystal clear on the RecommenderJob class and convert my code into one usable in hadoop distributed environment . I'd get back to you in case i need more details on the RecommenderJob class.
Thanks a lot for your clarifications and support.
Thanks and Regards
Bejoy.K.S
> Date: Fri, 12 Nov 2010 13:33:01 +0000
> Subject: Re: Mahout - Help needed - files with no preferences and integarting mahout with Hadoop
> From: srowen@gmail.com
> To: user@mahout.apache.org
>
> Yes, if you have no data for a user, you can't make recommendations. If you
> have a little data, you can make only a few, if any, weak recommendations.
> The framework won't return very weak recommendations.
>
> For RecommenderJob -- just read the javadoc, which will tell you how to run
> it.
>
> I'll also point you to the book Mahout In Action. I wrote up a detailed
> treatment of the Hadoop-based recommender job and also the other
> non-distributed code. You don't need the book to use the code -- the javadoc
> explains it all -- but if you're deeply interested it will add a lot of
> value to you.
>
> On Fri, Nov 12, 2010 at 12:31 PM, bejoy ks <be...@hotmail.com> wrote:
>
> >
> > Thanks Owen. Thanks a lot.
> > You were right, I'm having close to 5000 <User Id>,<Item Id> sets. And my
> > code is producing recommendations for some User Ids where as for some it
> > does't.
> > For those User Ids that didn't give any recommendations had only a few
> > preferences compared to others, but all users do have some preferences.
> > Guess i need more data to get much accurate recommendations for all users.
> > Is there any better implementations of this code? any suggestions from your
> > end?
> >
> > Also, I'd like to get this same code running on my hadoop cluster. Could
> > you please help me out with some documents or simple code snippets that
> > could lead my way. I tried web help but most blogs refer me to
> > org.apache.mahout.cf.taste.hadoop.item.RecommenderJob, the code available
> > with mahout src. Unfortunately I’m not able to get much from the code as it
> > is a bit complicated for a starter like me. (I have done a couple of Map
> > reduce project on text files processing before, that is my hadoop MR
> > experience ).
> >
> > It would be much helpful if you could share me with some informative
> > tutorials whichcould make my initial steps in Mahout comfortable before i
> > could start exploring in depth. I'm from India and I'm eagerly waiting for
> > the Indian edition of 'Mahout In Action' which is not in stores yet.
> >
> > Steven,
> > Thank you for your support.
> >
> > Thanks and Regards
> > Bejoy.K.S
> >
> >
> >
> >
> > > Date: Fri, 12 Nov 2010 12:06:47 +0000
> > > Subject: Re: Mahout - Help needed - files with no preferences and
> > integarting mahout with Hadoop
> > > From: srowen@gmail.com
> > > To: user@mahout.apache.org
> > >
> > > I think you'd have to debug to get more insight. To me it looks OK. Do
> > you
> > > have enough data? Maybe your user has few or no prefs, which means
> > nothing
> > > can be recommended.
> > >
> > >
> > > On Fri, Nov 12, 2010 at 12:02 PM, bejoy ks <be...@hotmail.com> wrote:
> > >
> > > >
> > > > Hi Steven
> > > > I tried my User Similarity recommendation using
> > > > GenericBooleanPrefUserBasedRecommender as you suggested but
> > unfortunaley it
> > > > is not resolving my issue. I can see in logs that it is processing the
> > > > entire file(<UserId, ItemId>) but then in a few seconds the application
> > is
> > > > terminating without giving me any recommendations. The code i tried is
> > > >
> > > > FileDataModel dataModel = new FileDataModel(new
> > File(recsFile));
> > > > UserSimilarity userSimilarity = new
> > > > TanimotoCoefficientSimilarity(dataModel);
> > > > UserNeighborhood neighborhood =new
> > > > NearestNUserNeighborhood(neighbourhoodSize, userSimilarity, dataModel);
> > > > Recommender recommender =new
> > > > GenericBooleanPrefUserBasedRecommender(dataModel, neighborhood,
> > > > userSimilarity);
> > > > List<RecommendedItem> recommendations
> > > > =recommender.recommend(userId, noOfRecommendations);
> > > > System.out.println(recommendations);
> > > >
> > > > I tried the same code with PearsonCorrelationSimilarity as well. Still
> > it
> > > > not giving me any recommendations.
> > > > Is there any thing else I should change in my code to get in
> > > > recommendations?
> > > >
> > > > Thanks and Regards
> > > > Bejoy.K.S
> > > >
> > > >
> > > >
> > > >
> > > > > Date: Fri, 12 Nov 2010 10:28:16 +0000
> > > > > Subject: Re: FW: Mahout - Help needed - files with no preferences and
> > > > integarting mahout with Hadoop
> > > > > From: sbourke@gmail.com
> > > > > To: user@mahout.apache.org
> > > > >
> > > > > Try using the generic boolean pref user based recommender instead of
> > the
> > > > > generic user based recommender when using binary data.
> > > > >
> > > > >
> > > >
> > http://mahout.apache.org/javadoc/core/org/apache/mahout/cf/taste/impl/recommender/GenericBooleanPrefUserBasedRecommender.html
> > > > >
> > > > >
> > > > >
> > > > > On Fri, Nov 12, 2010 at 9:57 AM, bejoy ks <be...@hotmail.com>
> > wrote:
> > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > Hi Mahout Experts,
> > > > > >
> > > > > >
> > > > > > I'm totally new into mahout and badly indeed of some expert
> > opinions. I
> > > > > > wanted
> > > > > > to implement a recommendation engine using mahout and hadoop, as
> > the
> > > > > > initial
> > > > > > stage I just tried a recommender class with mahout alone which
> > worked
> > > > > > fine . I’m
> > > > > > using eclipse for my development and to test I just ran the code
> > from
> > > > my
> > > > > > eclipse like a simple java class with the required jars already
> > added
> > > > on
> > > > > > to project build path. The code I used for testing User Similarity
> > > > > > recommendation is as follows
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > public static void main(String[] args) throws TasteException
> > > > > > {
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > String recsFile="D://mahoutFiles//SampleData.txt";
> > > > > >
> > > > > >
> > > > > > int
> > > > > > neighbourhoodSize=7;
> > > > > >
> > > > > >
> > > > > > long
> > > > > > ssoId=206008129;
> > > > > >
> > > > > >
> > > > > > int
> > > > > > noOfRecommendations=5;
> > > > > >
> > > > > >
> > > > > > try {
> > > > > >
> > > > > >
> > > > > > FileDataModel
> > > > > > dataModel = new FileDataModel(new File(recsFile));
> > > > > >
> > > > > >
> > > > > > UserSimilarity
> > > > > > userSimilarity = new PearsonCorrelationSimilarity(dataModel);
> > > > > >
> > > > > >
> > > > > > UserNeighborhood
> > > > > > neighborhood =new NearestNUserNeighborhood(neighbourhoodSize,
> > > > > > userSimilarity,
> > > > > > dataModel);
> > > > > >
> > > > > >
> > > > > > Recommender
> > > > > > recommender =new GenericUserBasedRecommender(dataModel,
> > neighborhood,
> > > > > > userSimilarity);
> > > > > >
> > > > > >
> > > > > > List<RecommendedItem>
> > > > > > recommendations =recommender.recommend(ssoId, noOfRecommendations);
> > > > > >
> > > > > >
> > > > > > for
> > > > > > (RecommendedItem recommendedItem : recommendations) {
> > > > > >
> > > > > >
> > > > > >
> > System.out.println(recommendedItem.getItemID());
> > > > > >
> > > > > >
> > > > > > System.out.print(" Item:
> > > > > > "+dataModel.getItemIDAsString(recommendedItem.getItemID()));
> > > > > >
> > > > > >
> > > > > > System.out.println(" and
> > > > > > value: "+recommendedItem.getValue());
> > > > > >
> > > > > >
> > > > > > }
> > > > > >
> > > > > >
> > > > > > } catch (IOException e)
> > > > > > {
> > > > > >
> > > > > >
> > > > > > e.printStackTrace();
> > > > > >
> > > > > >
> > > > > > } catch (TasteException
> > > > > > e) {
> > > > > >
> > > > > >
> > > > > > e.printStackTrace();
> > > > > >
> > > > > >
> > > > > > }
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > }
> > > > > >
> > > > > >
> > > > > > The code I used for testing Item Similarity recommendation is as
> > > > follows
> > > > > >
> > > > > > public static void main(String[] args) {
> > > > > >
> > > > > > String recsFile="D://mahoutFiles//SampleData.txt";
> > > > > >
> > > > > > long ssoId=206008129;
> > > > > >
> > > > > > int noOfRecommendations=5;
> > > > > >
> > > > > > String itemId="GE-CORPPROG-IMLP-PTOOLS";
> > > > > >
> > > > > > try {
> > > > > >
> > > > > > //item similarity recommendation based on User
> > > > > >
> > > > > > FileDataModel dataModel = new FileDataModel(new
> > > > File(recsFile));
> > > > > >
> > > > > > ItemSimilarity itemSimilarity = new
> > > > > > LogLikelihoodSimilarity(dataModel);
> > > > > >
> > > > > > ItemBasedRecommender recommender =new
> > > > > > GenericItemBasedRecommender(dataModel, itemSimilarity);
> > > > > >
> > > > > > List<RecommendedItem> recommendations
> > > > > > =recommender.recommend(ssoId, noOfRecommendations);
> > > > > >
> > > > > >
> > > > > >
> > > > > > for (RecommendedItem recommendedItem : recommendations)
> > {
> > > > > >
> > > > > > System.out.println(recommendedItem);
> > > > > >
> > > > > > }
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > //item similarity recommendation based on item
> > > > > >
> > > > > > recommender =new GenericItemBasedRecommender(dataModel,
> > > > > > itemSimilarity);
> > > > > >
> > > > > > recommendations=
> > > > > >
> > recommender.mostSimilarItems(dataModel.readItemIDFromString(itemId),
> > > > > > noOfRecommendations);
> > > > > >
> > > > > > System.out.println("Item recommendations based on
> > Item");
> > > > > >
> > > > > > for (RecommendedItem recommendedItem : recommendations)
> > {
> > > > > >
> > > > > > System.out.println(recommendedItem);
> > > > > >
> > > > > > }
> > > > > >
> > > > > > } catch (IOException e) {
> > > > > >
> > > > > > // TODO Auto-generated catch block
> > > > > >
> > > > > > e.printStackTrace();
> > > > > >
> > > > > > } catch (TasteException e) {
> > > > > >
> > > > > > // TODO Auto-generated catch block
> > > > > >
> > > > > > e.printStackTrace();
> > > > > >
> > > > > > }
> > > > > >
> > > > > >
> > > > > >
> > > > > > }
> > > > > >
> > > > > >
> > > > > > The issues or concerns I have are as follows.
> > > > > >
> > > > > >
> > > > > >
> > > > > > Problem 1:
> > > > > >
> > > > > >
> > > > > > This code produces relevant
> > > > > > recommendations when I have given my file in the format <User id>,
> > > > <item
> > > > > > id>, <preference value>.
> > > > > > But in my actual sample file I have just the user id and Item id
> > alone,
> > > > no
> > > > > > preference value. On my reference since it is kind of a Boolean
> > > > preference,
> > > > > > I tried
> > > > > > using PearsonCorrelationSimilarity
> > > > > > as well as TanimotoCoefficientSimilarity but it is not giving me
> > any
> > > > > > recommendations. My actual file format that I'd want to process
> > would
> > > > be in
> > > > > > the format <user id>,
> > > > > > <item id>, . I can see it is processing all users in my file( from
> > my
> > > > > > eclipse console logs) but no recommendations are being produced as
> > > > output.
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > Problem 2:
> > > > > >
> > > > > >
> > > > > > I
> > > > > > want to use my mahout recommender in Hadoop distributed
> > environment. I
> > > > > > don’t have
> > > > > > a clue about it. I tried web help but most blogs refer me to
> > > > > > org.apache.mahout.cf.taste.hadoop.item.RecommenderJob,
> > > > > > the code
> > > > > > available with mahout src. Unfortunately I’m not able to get much
> > from
> > > > > > the code
> > > > > > as it is a bit complicated for a starter like me. (I have done a
> > couple
> > > > > > of Map reduce project on text files processing before ). Could
> > someone
> > > > > > please help me
> > > > > > out with very basic simple code snippets to run my User Similarity
> > and
> > > > > > Item Similarity Recommendation code
> > > > > > on hadoop environment .
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > It would be much
> > > > > > helpful if you could share me with some informative tutorials which
> > > > > > could make my initial steps in Mahout comfortable before i could
> > start
> > > > > > exploring in
> > > > > > depth.
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > Thank you
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > Thanks and Regards
> > > > > > Bejoy.K.S
> > > > > >
> > > > > >
> > > > > >
> > > >
> > > >
> >
> >
Re: Mahout - Help needed - files with no preferences and integarting
mahout with Hadoop
Posted by Sean Owen <sr...@gmail.com>.
Yes, if you have no data for a user, you can't make recommendations. If you
have a little data, you can make only a few, if any, weak recommendations.
The framework won't return very weak recommendations.
For RecommenderJob -- just read the javadoc, which will tell you how to run
it.
I'll also point you to the book Mahout In Action. I wrote up a detailed
treatment of the Hadoop-based recommender job and also the other
non-distributed code. You don't need the book to use the code -- the javadoc
explains it all -- but if you're deeply interested it will add a lot of
value to you.
On Fri, Nov 12, 2010 at 12:31 PM, bejoy ks <be...@hotmail.com> wrote:
>
> Thanks Owen. Thanks a lot.
> You were right, I'm having close to 5000 <User Id>,<Item Id> sets. And my
> code is producing recommendations for some User Ids where as for some it
> does't.
> For those User Ids that didn't give any recommendations had only a few
> preferences compared to others, but all users do have some preferences.
> Guess i need more data to get much accurate recommendations for all users.
> Is there any better implementations of this code? any suggestions from your
> end?
>
> Also, I'd like to get this same code running on my hadoop cluster. Could
> you please help me out with some documents or simple code snippets that
> could lead my way. I tried web help but most blogs refer me to
> org.apache.mahout.cf.taste.hadoop.item.RecommenderJob, the code available
> with mahout src. Unfortunately I’m not able to get much from the code as it
> is a bit complicated for a starter like me. (I have done a couple of Map
> reduce project on text files processing before, that is my hadoop MR
> experience ).
>
> It would be much helpful if you could share me with some informative
> tutorials whichcould make my initial steps in Mahout comfortable before i
> could start exploring in depth. I'm from India and I'm eagerly waiting for
> the Indian edition of 'Mahout In Action' which is not in stores yet.
>
> Steven,
> Thank you for your support.
>
> Thanks and Regards
> Bejoy.K.S
>
>
>
>
> > Date: Fri, 12 Nov 2010 12:06:47 +0000
> > Subject: Re: Mahout - Help needed - files with no preferences and
> integarting mahout with Hadoop
> > From: srowen@gmail.com
> > To: user@mahout.apache.org
> >
> > I think you'd have to debug to get more insight. To me it looks OK. Do
> you
> > have enough data? Maybe your user has few or no prefs, which means
> nothing
> > can be recommended.
> >
> >
> > On Fri, Nov 12, 2010 at 12:02 PM, bejoy ks <be...@hotmail.com> wrote:
> >
> > >
> > > Hi Steven
> > > I tried my User Similarity recommendation using
> > > GenericBooleanPrefUserBasedRecommender as you suggested but
> unfortunaley it
> > > is not resolving my issue. I can see in logs that it is processing the
> > > entire file(<UserId, ItemId>) but then in a few seconds the application
> is
> > > terminating without giving me any recommendations. The code i tried is
> > >
> > > FileDataModel dataModel = new FileDataModel(new
> File(recsFile));
> > > UserSimilarity userSimilarity = new
> > > TanimotoCoefficientSimilarity(dataModel);
> > > UserNeighborhood neighborhood =new
> > > NearestNUserNeighborhood(neighbourhoodSize, userSimilarity, dataModel);
> > > Recommender recommender =new
> > > GenericBooleanPrefUserBasedRecommender(dataModel, neighborhood,
> > > userSimilarity);
> > > List<RecommendedItem> recommendations
> > > =recommender.recommend(userId, noOfRecommendations);
> > > System.out.println(recommendations);
> > >
> > > I tried the same code with PearsonCorrelationSimilarity as well. Still
> it
> > > not giving me any recommendations.
> > > Is there any thing else I should change in my code to get in
> > > recommendations?
> > >
> > > Thanks and Regards
> > > Bejoy.K.S
> > >
> > >
> > >
> > >
> > > > Date: Fri, 12 Nov 2010 10:28:16 +0000
> > > > Subject: Re: FW: Mahout - Help needed - files with no preferences and
> > > integarting mahout with Hadoop
> > > > From: sbourke@gmail.com
> > > > To: user@mahout.apache.org
> > > >
> > > > Try using the generic boolean pref user based recommender instead of
> the
> > > > generic user based recommender when using binary data.
> > > >
> > > >
> > >
> http://mahout.apache.org/javadoc/core/org/apache/mahout/cf/taste/impl/recommender/GenericBooleanPrefUserBasedRecommender.html
> > > >
> > > >
> > > >
> > > > On Fri, Nov 12, 2010 at 9:57 AM, bejoy ks <be...@hotmail.com>
> wrote:
> > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > Hi Mahout Experts,
> > > > >
> > > > >
> > > > > I'm totally new into mahout and badly indeed of some expert
> opinions. I
> > > > > wanted
> > > > > to implement a recommendation engine using mahout and hadoop, as
> the
> > > > > initial
> > > > > stage I just tried a recommender class with mahout alone which
> worked
> > > > > fine . I’m
> > > > > using eclipse for my development and to test I just ran the code
> from
> > > my
> > > > > eclipse like a simple java class with the required jars already
> added
> > > on
> > > > > to project build path. The code I used for testing User Similarity
> > > > > recommendation is as follows
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > public static void main(String[] args) throws TasteException
> > > > > {
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > String recsFile="D://mahoutFiles//SampleData.txt";
> > > > >
> > > > >
> > > > > int
> > > > > neighbourhoodSize=7;
> > > > >
> > > > >
> > > > > long
> > > > > ssoId=206008129;
> > > > >
> > > > >
> > > > > int
> > > > > noOfRecommendations=5;
> > > > >
> > > > >
> > > > > try {
> > > > >
> > > > >
> > > > > FileDataModel
> > > > > dataModel = new FileDataModel(new File(recsFile));
> > > > >
> > > > >
> > > > > UserSimilarity
> > > > > userSimilarity = new PearsonCorrelationSimilarity(dataModel);
> > > > >
> > > > >
> > > > > UserNeighborhood
> > > > > neighborhood =new NearestNUserNeighborhood(neighbourhoodSize,
> > > > > userSimilarity,
> > > > > dataModel);
> > > > >
> > > > >
> > > > > Recommender
> > > > > recommender =new GenericUserBasedRecommender(dataModel,
> neighborhood,
> > > > > userSimilarity);
> > > > >
> > > > >
> > > > > List<RecommendedItem>
> > > > > recommendations =recommender.recommend(ssoId, noOfRecommendations);
> > > > >
> > > > >
> > > > > for
> > > > > (RecommendedItem recommendedItem : recommendations) {
> > > > >
> > > > >
> > > > >
> System.out.println(recommendedItem.getItemID());
> > > > >
> > > > >
> > > > > System.out.print(" Item:
> > > > > "+dataModel.getItemIDAsString(recommendedItem.getItemID()));
> > > > >
> > > > >
> > > > > System.out.println(" and
> > > > > value: "+recommendedItem.getValue());
> > > > >
> > > > >
> > > > > }
> > > > >
> > > > >
> > > > > } catch (IOException e)
> > > > > {
> > > > >
> > > > >
> > > > > e.printStackTrace();
> > > > >
> > > > >
> > > > > } catch (TasteException
> > > > > e) {
> > > > >
> > > > >
> > > > > e.printStackTrace();
> > > > >
> > > > >
> > > > > }
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > }
> > > > >
> > > > >
> > > > > The code I used for testing Item Similarity recommendation is as
> > > follows
> > > > >
> > > > > public static void main(String[] args) {
> > > > >
> > > > > String recsFile="D://mahoutFiles//SampleData.txt";
> > > > >
> > > > > long ssoId=206008129;
> > > > >
> > > > > int noOfRecommendations=5;
> > > > >
> > > > > String itemId="GE-CORPPROG-IMLP-PTOOLS";
> > > > >
> > > > > try {
> > > > >
> > > > > //item similarity recommendation based on User
> > > > >
> > > > > FileDataModel dataModel = new FileDataModel(new
> > > File(recsFile));
> > > > >
> > > > > ItemSimilarity itemSimilarity = new
> > > > > LogLikelihoodSimilarity(dataModel);
> > > > >
> > > > > ItemBasedRecommender recommender =new
> > > > > GenericItemBasedRecommender(dataModel, itemSimilarity);
> > > > >
> > > > > List<RecommendedItem> recommendations
> > > > > =recommender.recommend(ssoId, noOfRecommendations);
> > > > >
> > > > >
> > > > >
> > > > > for (RecommendedItem recommendedItem : recommendations)
> {
> > > > >
> > > > > System.out.println(recommendedItem);
> > > > >
> > > > > }
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > //item similarity recommendation based on item
> > > > >
> > > > > recommender =new GenericItemBasedRecommender(dataModel,
> > > > > itemSimilarity);
> > > > >
> > > > > recommendations=
> > > > >
> recommender.mostSimilarItems(dataModel.readItemIDFromString(itemId),
> > > > > noOfRecommendations);
> > > > >
> > > > > System.out.println("Item recommendations based on
> Item");
> > > > >
> > > > > for (RecommendedItem recommendedItem : recommendations)
> {
> > > > >
> > > > > System.out.println(recommendedItem);
> > > > >
> > > > > }
> > > > >
> > > > > } catch (IOException e) {
> > > > >
> > > > > // TODO Auto-generated catch block
> > > > >
> > > > > e.printStackTrace();
> > > > >
> > > > > } catch (TasteException e) {
> > > > >
> > > > > // TODO Auto-generated catch block
> > > > >
> > > > > e.printStackTrace();
> > > > >
> > > > > }
> > > > >
> > > > >
> > > > >
> > > > > }
> > > > >
> > > > >
> > > > > The issues or concerns I have are as follows.
> > > > >
> > > > >
> > > > >
> > > > > Problem 1:
> > > > >
> > > > >
> > > > > This code produces relevant
> > > > > recommendations when I have given my file in the format <User id>,
> > > <item
> > > > > id>, <preference value>.
> > > > > But in my actual sample file I have just the user id and Item id
> alone,
> > > no
> > > > > preference value. On my reference since it is kind of a Boolean
> > > preference,
> > > > > I tried
> > > > > using PearsonCorrelationSimilarity
> > > > > as well as TanimotoCoefficientSimilarity but it is not giving me
> any
> > > > > recommendations. My actual file format that I'd want to process
> would
> > > be in
> > > > > the format <user id>,
> > > > > <item id>, . I can see it is processing all users in my file( from
> my
> > > > > eclipse console logs) but no recommendations are being produced as
> > > output.
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > Problem 2:
> > > > >
> > > > >
> > > > > I
> > > > > want to use my mahout recommender in Hadoop distributed
> environment. I
> > > > > don’t have
> > > > > a clue about it. I tried web help but most blogs refer me to
> > > > > org.apache.mahout.cf.taste.hadoop.item.RecommenderJob,
> > > > > the code
> > > > > available with mahout src. Unfortunately I’m not able to get much
> from
> > > > > the code
> > > > > as it is a bit complicated for a starter like me. (I have done a
> couple
> > > > > of Map reduce project on text files processing before ). Could
> someone
> > > > > please help me
> > > > > out with very basic simple code snippets to run my User Similarity
> and
> > > > > Item Similarity Recommendation code
> > > > > on hadoop environment .
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > It would be much
> > > > > helpful if you could share me with some informative tutorials which
> > > > > could make my initial steps in Mahout comfortable before i could
> start
> > > > > exploring in
> > > > > depth.
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > Thank you
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > Thanks and Regards
> > > > > Bejoy.K.S
> > > > >
> > > > >
> > > > >
> > >
> > >
>
>
RE: Mahout - Help needed - files with no preferences and
integarting mahout with Hadoop
Posted by bejoy ks <be...@hotmail.com>.
Thanks Owen. Thanks a lot.
You were right, I'm having close to 5000 <User Id>,<Item Id> sets. And my code is producing recommendations for some User Ids where as for some it does't.
For those User Ids that didn't give any recommendations had only a few preferences compared to others, but all users do have some preferences.
Guess i need more data to get much accurate recommendations for all users. Is there any better implementations of this code? any suggestions from your end?
Also, I'd like to get this same code running on my hadoop cluster. Could you please help me out with some documents or simple code snippets that could lead my way. I tried web help but most blogs refer me to org.apache.mahout.cf.taste.hadoop.item.RecommenderJob, the code available with mahout src. Unfortunately I’m not able to get much from the code as it is a bit complicated for a starter like me. (I have done a couple of Map reduce project on text files processing before, that is my hadoop MR experience ).
It would be much helpful if you could share me with some informative tutorials whichcould make my initial steps in Mahout comfortable before i could start exploring in depth. I'm from India and I'm eagerly waiting for the Indian edition of 'Mahout In Action' which is not in stores yet.
Steven,
Thank you for your support.
Thanks and Regards
Bejoy.K.S
> Date: Fri, 12 Nov 2010 12:06:47 +0000
> Subject: Re: Mahout - Help needed - files with no preferences and integarting mahout with Hadoop
> From: srowen@gmail.com
> To: user@mahout.apache.org
>
> I think you'd have to debug to get more insight. To me it looks OK. Do you
> have enough data? Maybe your user has few or no prefs, which means nothing
> can be recommended.
>
>
> On Fri, Nov 12, 2010 at 12:02 PM, bejoy ks <be...@hotmail.com> wrote:
>
> >
> > Hi Steven
> > I tried my User Similarity recommendation using
> > GenericBooleanPrefUserBasedRecommender as you suggested but unfortunaley it
> > is not resolving my issue. I can see in logs that it is processing the
> > entire file(<UserId, ItemId>) but then in a few seconds the application is
> > terminating without giving me any recommendations. The code i tried is
> >
> > FileDataModel dataModel = new FileDataModel(new File(recsFile));
> > UserSimilarity userSimilarity = new
> > TanimotoCoefficientSimilarity(dataModel);
> > UserNeighborhood neighborhood =new
> > NearestNUserNeighborhood(neighbourhoodSize, userSimilarity, dataModel);
> > Recommender recommender =new
> > GenericBooleanPrefUserBasedRecommender(dataModel, neighborhood,
> > userSimilarity);
> > List<RecommendedItem> recommendations
> > =recommender.recommend(userId, noOfRecommendations);
> > System.out.println(recommendations);
> >
> > I tried the same code with PearsonCorrelationSimilarity as well. Still it
> > not giving me any recommendations.
> > Is there any thing else I should change in my code to get in
> > recommendations?
> >
> > Thanks and Regards
> > Bejoy.K.S
> >
> >
> >
> >
> > > Date: Fri, 12 Nov 2010 10:28:16 +0000
> > > Subject: Re: FW: Mahout - Help needed - files with no preferences and
> > integarting mahout with Hadoop
> > > From: sbourke@gmail.com
> > > To: user@mahout.apache.org
> > >
> > > Try using the generic boolean pref user based recommender instead of the
> > > generic user based recommender when using binary data.
> > >
> > >
> > http://mahout.apache.org/javadoc/core/org/apache/mahout/cf/taste/impl/recommender/GenericBooleanPrefUserBasedRecommender.html
> > >
> > >
> > >
> > > On Fri, Nov 12, 2010 at 9:57 AM, bejoy ks <be...@hotmail.com> wrote:
> > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > Hi Mahout Experts,
> > > >
> > > >
> > > > I'm totally new into mahout and badly indeed of some expert opinions. I
> > > > wanted
> > > > to implement a recommendation engine using mahout and hadoop, as the
> > > > initial
> > > > stage I just tried a recommender class with mahout alone which worked
> > > > fine . I’m
> > > > using eclipse for my development and to test I just ran the code from
> > my
> > > > eclipse like a simple java class with the required jars already added
> > on
> > > > to project build path. The code I used for testing User Similarity
> > > > recommendation is as follows
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > public static void main(String[] args) throws TasteException
> > > > {
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > String recsFile="D://mahoutFiles//SampleData.txt";
> > > >
> > > >
> > > > int
> > > > neighbourhoodSize=7;
> > > >
> > > >
> > > > long
> > > > ssoId=206008129;
> > > >
> > > >
> > > > int
> > > > noOfRecommendations=5;
> > > >
> > > >
> > > > try {
> > > >
> > > >
> > > > FileDataModel
> > > > dataModel = new FileDataModel(new File(recsFile));
> > > >
> > > >
> > > > UserSimilarity
> > > > userSimilarity = new PearsonCorrelationSimilarity(dataModel);
> > > >
> > > >
> > > > UserNeighborhood
> > > > neighborhood =new NearestNUserNeighborhood(neighbourhoodSize,
> > > > userSimilarity,
> > > > dataModel);
> > > >
> > > >
> > > > Recommender
> > > > recommender =new GenericUserBasedRecommender(dataModel, neighborhood,
> > > > userSimilarity);
> > > >
> > > >
> > > > List<RecommendedItem>
> > > > recommendations =recommender.recommend(ssoId, noOfRecommendations);
> > > >
> > > >
> > > > for
> > > > (RecommendedItem recommendedItem : recommendations) {
> > > >
> > > >
> > > > System.out.println(recommendedItem.getItemID());
> > > >
> > > >
> > > > System.out.print(" Item:
> > > > "+dataModel.getItemIDAsString(recommendedItem.getItemID()));
> > > >
> > > >
> > > > System.out.println(" and
> > > > value: "+recommendedItem.getValue());
> > > >
> > > >
> > > > }
> > > >
> > > >
> > > > } catch (IOException e)
> > > > {
> > > >
> > > >
> > > > e.printStackTrace();
> > > >
> > > >
> > > > } catch (TasteException
> > > > e) {
> > > >
> > > >
> > > > e.printStackTrace();
> > > >
> > > >
> > > > }
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > }
> > > >
> > > >
> > > > The code I used for testing Item Similarity recommendation is as
> > follows
> > > >
> > > > public static void main(String[] args) {
> > > >
> > > > String recsFile="D://mahoutFiles//SampleData.txt";
> > > >
> > > > long ssoId=206008129;
> > > >
> > > > int noOfRecommendations=5;
> > > >
> > > > String itemId="GE-CORPPROG-IMLP-PTOOLS";
> > > >
> > > > try {
> > > >
> > > > //item similarity recommendation based on User
> > > >
> > > > FileDataModel dataModel = new FileDataModel(new
> > File(recsFile));
> > > >
> > > > ItemSimilarity itemSimilarity = new
> > > > LogLikelihoodSimilarity(dataModel);
> > > >
> > > > ItemBasedRecommender recommender =new
> > > > GenericItemBasedRecommender(dataModel, itemSimilarity);
> > > >
> > > > List<RecommendedItem> recommendations
> > > > =recommender.recommend(ssoId, noOfRecommendations);
> > > >
> > > >
> > > >
> > > > for (RecommendedItem recommendedItem : recommendations) {
> > > >
> > > > System.out.println(recommendedItem);
> > > >
> > > > }
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > //item similarity recommendation based on item
> > > >
> > > > recommender =new GenericItemBasedRecommender(dataModel,
> > > > itemSimilarity);
> > > >
> > > > recommendations=
> > > > recommender.mostSimilarItems(dataModel.readItemIDFromString(itemId),
> > > > noOfRecommendations);
> > > >
> > > > System.out.println("Item recommendations based on Item");
> > > >
> > > > for (RecommendedItem recommendedItem : recommendations) {
> > > >
> > > > System.out.println(recommendedItem);
> > > >
> > > > }
> > > >
> > > > } catch (IOException e) {
> > > >
> > > > // TODO Auto-generated catch block
> > > >
> > > > e.printStackTrace();
> > > >
> > > > } catch (TasteException e) {
> > > >
> > > > // TODO Auto-generated catch block
> > > >
> > > > e.printStackTrace();
> > > >
> > > > }
> > > >
> > > >
> > > >
> > > > }
> > > >
> > > >
> > > > The issues or concerns I have are as follows.
> > > >
> > > >
> > > >
> > > > Problem 1:
> > > >
> > > >
> > > > This code produces relevant
> > > > recommendations when I have given my file in the format <User id>,
> > <item
> > > > id>, <preference value>.
> > > > But in my actual sample file I have just the user id and Item id alone,
> > no
> > > > preference value. On my reference since it is kind of a Boolean
> > preference,
> > > > I tried
> > > > using PearsonCorrelationSimilarity
> > > > as well as TanimotoCoefficientSimilarity but it is not giving me any
> > > > recommendations. My actual file format that I'd want to process would
> > be in
> > > > the format <user id>,
> > > > <item id>, . I can see it is processing all users in my file( from my
> > > > eclipse console logs) but no recommendations are being produced as
> > output.
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > Problem 2:
> > > >
> > > >
> > > > I
> > > > want to use my mahout recommender in Hadoop distributed environment. I
> > > > don’t have
> > > > a clue about it. I tried web help but most blogs refer me to
> > > > org.apache.mahout.cf.taste.hadoop.item.RecommenderJob,
> > > > the code
> > > > available with mahout src. Unfortunately I’m not able to get much from
> > > > the code
> > > > as it is a bit complicated for a starter like me. (I have done a couple
> > > > of Map reduce project on text files processing before ). Could someone
> > > > please help me
> > > > out with very basic simple code snippets to run my User Similarity and
> > > > Item Similarity Recommendation code
> > > > on hadoop environment .
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > It would be much
> > > > helpful if you could share me with some informative tutorials which
> > > > could make my initial steps in Mahout comfortable before i could start
> > > > exploring in
> > > > depth.
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > Thank you
> > > >
> > > >
> > > >
> > > >
> > > > Thanks and Regards
> > > > Bejoy.K.S
> > > >
> > > >
> > > >
> >
> >
Re: Mahout - Help needed - files with no preferences and integarting
mahout with Hadoop
Posted by Sean Owen <sr...@gmail.com>.
I think you'd have to debug to get more insight. To me it looks OK. Do you
have enough data? Maybe your user has few or no prefs, which means nothing
can be recommended.
On Fri, Nov 12, 2010 at 12:02 PM, bejoy ks <be...@hotmail.com> wrote:
>
> Hi Steven
> I tried my User Similarity recommendation using
> GenericBooleanPrefUserBasedRecommender as you suggested but unfortunaley it
> is not resolving my issue. I can see in logs that it is processing the
> entire file(<UserId, ItemId>) but then in a few seconds the application is
> terminating without giving me any recommendations. The code i tried is
>
> FileDataModel dataModel = new FileDataModel(new File(recsFile));
> UserSimilarity userSimilarity = new
> TanimotoCoefficientSimilarity(dataModel);
> UserNeighborhood neighborhood =new
> NearestNUserNeighborhood(neighbourhoodSize, userSimilarity, dataModel);
> Recommender recommender =new
> GenericBooleanPrefUserBasedRecommender(dataModel, neighborhood,
> userSimilarity);
> List<RecommendedItem> recommendations
> =recommender.recommend(userId, noOfRecommendations);
> System.out.println(recommendations);
>
> I tried the same code with PearsonCorrelationSimilarity as well. Still it
> not giving me any recommendations.
> Is there any thing else I should change in my code to get in
> recommendations?
>
> Thanks and Regards
> Bejoy.K.S
>
>
>
>
> > Date: Fri, 12 Nov 2010 10:28:16 +0000
> > Subject: Re: FW: Mahout - Help needed - files with no preferences and
> integarting mahout with Hadoop
> > From: sbourke@gmail.com
> > To: user@mahout.apache.org
> >
> > Try using the generic boolean pref user based recommender instead of the
> > generic user based recommender when using binary data.
> >
> >
> http://mahout.apache.org/javadoc/core/org/apache/mahout/cf/taste/impl/recommender/GenericBooleanPrefUserBasedRecommender.html
> >
> >
> >
> > On Fri, Nov 12, 2010 at 9:57 AM, bejoy ks <be...@hotmail.com> wrote:
> >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > > Hi Mahout Experts,
> > >
> > >
> > > I'm totally new into mahout and badly indeed of some expert opinions. I
> > > wanted
> > > to implement a recommendation engine using mahout and hadoop, as the
> > > initial
> > > stage I just tried a recommender class with mahout alone which worked
> > > fine . I’m
> > > using eclipse for my development and to test I just ran the code from
> my
> > > eclipse like a simple java class with the required jars already added
> on
> > > to project build path. The code I used for testing User Similarity
> > > recommendation is as follows
> > >
> > >
> > >
> > >
> > >
> > > public static void main(String[] args) throws TasteException
> > > {
> > >
> > >
> > >
> > >
> > >
> > > String recsFile="D://mahoutFiles//SampleData.txt";
> > >
> > >
> > > int
> > > neighbourhoodSize=7;
> > >
> > >
> > > long
> > > ssoId=206008129;
> > >
> > >
> > > int
> > > noOfRecommendations=5;
> > >
> > >
> > > try {
> > >
> > >
> > > FileDataModel
> > > dataModel = new FileDataModel(new File(recsFile));
> > >
> > >
> > > UserSimilarity
> > > userSimilarity = new PearsonCorrelationSimilarity(dataModel);
> > >
> > >
> > > UserNeighborhood
> > > neighborhood =new NearestNUserNeighborhood(neighbourhoodSize,
> > > userSimilarity,
> > > dataModel);
> > >
> > >
> > > Recommender
> > > recommender =new GenericUserBasedRecommender(dataModel, neighborhood,
> > > userSimilarity);
> > >
> > >
> > > List<RecommendedItem>
> > > recommendations =recommender.recommend(ssoId, noOfRecommendations);
> > >
> > >
> > > for
> > > (RecommendedItem recommendedItem : recommendations) {
> > >
> > >
> > > System.out.println(recommendedItem.getItemID());
> > >
> > >
> > > System.out.print(" Item:
> > > "+dataModel.getItemIDAsString(recommendedItem.getItemID()));
> > >
> > >
> > > System.out.println(" and
> > > value: "+recommendedItem.getValue());
> > >
> > >
> > > }
> > >
> > >
> > > } catch (IOException e)
> > > {
> > >
> > >
> > > e.printStackTrace();
> > >
> > >
> > > } catch (TasteException
> > > e) {
> > >
> > >
> > > e.printStackTrace();
> > >
> > >
> > > }
> > >
> > >
> > >
> > >
> > >
> > > }
> > >
> > >
> > > The code I used for testing Item Similarity recommendation is as
> follows
> > >
> > > public static void main(String[] args) {
> > >
> > > String recsFile="D://mahoutFiles//SampleData.txt";
> > >
> > > long ssoId=206008129;
> > >
> > > int noOfRecommendations=5;
> > >
> > > String itemId="GE-CORPPROG-IMLP-PTOOLS";
> > >
> > > try {
> > >
> > > //item similarity recommendation based on User
> > >
> > > FileDataModel dataModel = new FileDataModel(new
> File(recsFile));
> > >
> > > ItemSimilarity itemSimilarity = new
> > > LogLikelihoodSimilarity(dataModel);
> > >
> > > ItemBasedRecommender recommender =new
> > > GenericItemBasedRecommender(dataModel, itemSimilarity);
> > >
> > > List<RecommendedItem> recommendations
> > > =recommender.recommend(ssoId, noOfRecommendations);
> > >
> > >
> > >
> > > for (RecommendedItem recommendedItem : recommendations) {
> > >
> > > System.out.println(recommendedItem);
> > >
> > > }
> > >
> > >
> > >
> > >
> > >
> > > //item similarity recommendation based on item
> > >
> > > recommender =new GenericItemBasedRecommender(dataModel,
> > > itemSimilarity);
> > >
> > > recommendations=
> > > recommender.mostSimilarItems(dataModel.readItemIDFromString(itemId),
> > > noOfRecommendations);
> > >
> > > System.out.println("Item recommendations based on Item");
> > >
> > > for (RecommendedItem recommendedItem : recommendations) {
> > >
> > > System.out.println(recommendedItem);
> > >
> > > }
> > >
> > > } catch (IOException e) {
> > >
> > > // TODO Auto-generated catch block
> > >
> > > e.printStackTrace();
> > >
> > > } catch (TasteException e) {
> > >
> > > // TODO Auto-generated catch block
> > >
> > > e.printStackTrace();
> > >
> > > }
> > >
> > >
> > >
> > > }
> > >
> > >
> > > The issues or concerns I have are as follows.
> > >
> > >
> > >
> > > Problem 1:
> > >
> > >
> > > This code produces relevant
> > > recommendations when I have given my file in the format <User id>,
> <item
> > > id>, <preference value>.
> > > But in my actual sample file I have just the user id and Item id alone,
> no
> > > preference value. On my reference since it is kind of a Boolean
> preference,
> > > I tried
> > > using PearsonCorrelationSimilarity
> > > as well as TanimotoCoefficientSimilarity but it is not giving me any
> > > recommendations. My actual file format that I'd want to process would
> be in
> > > the format <user id>,
> > > <item id>, . I can see it is processing all users in my file( from my
> > > eclipse console logs) but no recommendations are being produced as
> output.
> > >
> > >
> > >
> > >
> > >
> > > Problem 2:
> > >
> > >
> > > I
> > > want to use my mahout recommender in Hadoop distributed environment. I
> > > don’t have
> > > a clue about it. I tried web help but most blogs refer me to
> > > org.apache.mahout.cf.taste.hadoop.item.RecommenderJob,
> > > the code
> > > available with mahout src. Unfortunately I’m not able to get much from
> > > the code
> > > as it is a bit complicated for a starter like me. (I have done a couple
> > > of Map reduce project on text files processing before ). Could someone
> > > please help me
> > > out with very basic simple code snippets to run my User Similarity and
> > > Item Similarity Recommendation code
> > > on hadoop environment .
> > >
> > >
> > >
> > >
> > >
> > > It would be much
> > > helpful if you could share me with some informative tutorials which
> > > could make my initial steps in Mahout comfortable before i could start
> > > exploring in
> > > depth.
> > >
> > >
> > >
> > >
> > >
> > > Thank you
> > >
> > >
> > >
> > >
> > > Thanks and Regards
> > > Bejoy.K.S
> > >
> > >
> > >
>
>
RE: Mahout - Help needed - files with no preferences and
integarting mahout with Hadoop
Posted by bejoy ks <be...@hotmail.com>.
Hi Steven
I tried my User Similarity recommendation using GenericBooleanPrefUserBasedRecommender as you suggested but unfortunaley it is not resolving my issue. I can see in logs that it is processing the entire file(<UserId, ItemId>) but then in a few seconds the application is terminating without giving me any recommendations. The code i tried is
FileDataModel dataModel = new FileDataModel(new File(recsFile));
UserSimilarity userSimilarity = new TanimotoCoefficientSimilarity(dataModel);
UserNeighborhood neighborhood =new NearestNUserNeighborhood(neighbourhoodSize, userSimilarity, dataModel);
Recommender recommender =new GenericBooleanPrefUserBasedRecommender(dataModel, neighborhood, userSimilarity);
List<RecommendedItem> recommendations =recommender.recommend(userId, noOfRecommendations);
System.out.println(recommendations);
I tried the same code with PearsonCorrelationSimilarity as well. Still it not giving me any recommendations.
Is there any thing else I should change in my code to get in recommendations?
Thanks and Regards
Bejoy.K.S
> Date: Fri, 12 Nov 2010 10:28:16 +0000
> Subject: Re: FW: Mahout - Help needed - files with no preferences and integarting mahout with Hadoop
> From: sbourke@gmail.com
> To: user@mahout.apache.org
>
> Try using the generic boolean pref user based recommender instead of the
> generic user based recommender when using binary data.
>
> http://mahout.apache.org/javadoc/core/org/apache/mahout/cf/taste/impl/recommender/GenericBooleanPrefUserBasedRecommender.html
>
>
>
> On Fri, Nov 12, 2010 at 9:57 AM, bejoy ks <be...@hotmail.com> wrote:
>
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > Hi Mahout Experts,
> >
> >
> > I'm totally new into mahout and badly indeed of some expert opinions. I
> > wanted
> > to implement a recommendation engine using mahout and hadoop, as the
> > initial
> > stage I just tried a recommender class with mahout alone which worked
> > fine . I’m
> > using eclipse for my development and to test I just ran the code from my
> > eclipse like a simple java class with the required jars already added on
> > to project build path. The code I used for testing User Similarity
> > recommendation is as follows
> >
> >
> >
> >
> >
> > public static void main(String[] args) throws TasteException
> > {
> >
> >
> >
> >
> >
> > String recsFile="D://mahoutFiles//SampleData.txt";
> >
> >
> > int
> > neighbourhoodSize=7;
> >
> >
> > long
> > ssoId=206008129;
> >
> >
> > int
> > noOfRecommendations=5;
> >
> >
> > try {
> >
> >
> > FileDataModel
> > dataModel = new FileDataModel(new File(recsFile));
> >
> >
> > UserSimilarity
> > userSimilarity = new PearsonCorrelationSimilarity(dataModel);
> >
> >
> > UserNeighborhood
> > neighborhood =new NearestNUserNeighborhood(neighbourhoodSize,
> > userSimilarity,
> > dataModel);
> >
> >
> > Recommender
> > recommender =new GenericUserBasedRecommender(dataModel, neighborhood,
> > userSimilarity);
> >
> >
> > List<RecommendedItem>
> > recommendations =recommender.recommend(ssoId, noOfRecommendations);
> >
> >
> > for
> > (RecommendedItem recommendedItem : recommendations) {
> >
> >
> > System.out.println(recommendedItem.getItemID());
> >
> >
> > System.out.print(" Item:
> > "+dataModel.getItemIDAsString(recommendedItem.getItemID()));
> >
> >
> > System.out.println(" and
> > value: "+recommendedItem.getValue());
> >
> >
> > }
> >
> >
> > } catch (IOException e)
> > {
> >
> >
> > e.printStackTrace();
> >
> >
> > } catch (TasteException
> > e) {
> >
> >
> > e.printStackTrace();
> >
> >
> > }
> >
> >
> >
> >
> >
> > }
> >
> >
> > The code I used for testing Item Similarity recommendation is as follows
> >
> > public static void main(String[] args) {
> >
> > String recsFile="D://mahoutFiles//SampleData.txt";
> >
> > long ssoId=206008129;
> >
> > int noOfRecommendations=5;
> >
> > String itemId="GE-CORPPROG-IMLP-PTOOLS";
> >
> > try {
> >
> > //item similarity recommendation based on User
> >
> > FileDataModel dataModel = new FileDataModel(new File(recsFile));
> >
> > ItemSimilarity itemSimilarity = new
> > LogLikelihoodSimilarity(dataModel);
> >
> > ItemBasedRecommender recommender =new
> > GenericItemBasedRecommender(dataModel, itemSimilarity);
> >
> > List<RecommendedItem> recommendations
> > =recommender.recommend(ssoId, noOfRecommendations);
> >
> >
> >
> > for (RecommendedItem recommendedItem : recommendations) {
> >
> > System.out.println(recommendedItem);
> >
> > }
> >
> >
> >
> >
> >
> > //item similarity recommendation based on item
> >
> > recommender =new GenericItemBasedRecommender(dataModel,
> > itemSimilarity);
> >
> > recommendations=
> > recommender.mostSimilarItems(dataModel.readItemIDFromString(itemId),
> > noOfRecommendations);
> >
> > System.out.println("Item recommendations based on Item");
> >
> > for (RecommendedItem recommendedItem : recommendations) {
> >
> > System.out.println(recommendedItem);
> >
> > }
> >
> > } catch (IOException e) {
> >
> > // TODO Auto-generated catch block
> >
> > e.printStackTrace();
> >
> > } catch (TasteException e) {
> >
> > // TODO Auto-generated catch block
> >
> > e.printStackTrace();
> >
> > }
> >
> >
> >
> > }
> >
> >
> > The issues or concerns I have are as follows.
> >
> >
> >
> > Problem 1:
> >
> >
> > This code produces relevant
> > recommendations when I have given my file in the format <User id>, <item
> > id>, <preference value>.
> > But in my actual sample file I have just the user id and Item id alone, no
> > preference value. On my reference since it is kind of a Boolean preference,
> > I tried
> > using PearsonCorrelationSimilarity
> > as well as TanimotoCoefficientSimilarity but it is not giving me any
> > recommendations. My actual file format that I'd want to process would be in
> > the format <user id>,
> > <item id>, . I can see it is processing all users in my file( from my
> > eclipse console logs) but no recommendations are being produced as output.
> >
> >
> >
> >
> >
> > Problem 2:
> >
> >
> > I
> > want to use my mahout recommender in Hadoop distributed environment. I
> > don’t have
> > a clue about it. I tried web help but most blogs refer me to
> > org.apache.mahout.cf.taste.hadoop.item.RecommenderJob,
> > the code
> > available with mahout src. Unfortunately I’m not able to get much from
> > the code
> > as it is a bit complicated for a starter like me. (I have done a couple
> > of Map reduce project on text files processing before ). Could someone
> > please help me
> > out with very basic simple code snippets to run my User Similarity and
> > Item Similarity Recommendation code
> > on hadoop environment .
> >
> >
> >
> >
> >
> > It would be much
> > helpful if you could share me with some informative tutorials which
> > could make my initial steps in Mahout comfortable before i could start
> > exploring in
> > depth.
> >
> >
> >
> >
> >
> > Thank you
> >
> >
> >
> >
> > Thanks and Regards
> > Bejoy.K.S
> >
> >
> >
Re: FW: Mahout - Help needed - files with no preferences and
integarting mahout with Hadoop
Posted by Steven Bourke <sb...@gmail.com>.
Try using the generic boolean pref user based recommender instead of the
generic user based recommender when using binary data.
http://mahout.apache.org/javadoc/core/org/apache/mahout/cf/taste/impl/recommender/GenericBooleanPrefUserBasedRecommender.html
On Fri, Nov 12, 2010 at 9:57 AM, bejoy ks <be...@hotmail.com> wrote:
>
>
>
>
>
>
>
>
>
> Hi Mahout Experts,
>
>
> I'm totally new into mahout and badly indeed of some expert opinions. I
> wanted
> to implement a recommendation engine using mahout and hadoop, as the
> initial
> stage I just tried a recommender class with mahout alone which worked
> fine . I’m
> using eclipse for my development and to test I just ran the code from my
> eclipse like a simple java class with the required jars already added on
> to project build path. The code I used for testing User Similarity
> recommendation is as follows
>
>
>
>
>
> public static void main(String[] args) throws TasteException
> {
>
>
>
>
>
> String recsFile="D://mahoutFiles//SampleData.txt";
>
>
> int
> neighbourhoodSize=7;
>
>
> long
> ssoId=206008129;
>
>
> int
> noOfRecommendations=5;
>
>
> try {
>
>
> FileDataModel
> dataModel = new FileDataModel(new File(recsFile));
>
>
> UserSimilarity
> userSimilarity = new PearsonCorrelationSimilarity(dataModel);
>
>
> UserNeighborhood
> neighborhood =new NearestNUserNeighborhood(neighbourhoodSize,
> userSimilarity,
> dataModel);
>
>
> Recommender
> recommender =new GenericUserBasedRecommender(dataModel, neighborhood,
> userSimilarity);
>
>
> List<RecommendedItem>
> recommendations =recommender.recommend(ssoId, noOfRecommendations);
>
>
> for
> (RecommendedItem recommendedItem : recommendations) {
>
>
> System.out.println(recommendedItem.getItemID());
>
>
> System.out.print(" Item:
> "+dataModel.getItemIDAsString(recommendedItem.getItemID()));
>
>
> System.out.println(" and
> value: "+recommendedItem.getValue());
>
>
> }
>
>
> } catch (IOException e)
> {
>
>
> e.printStackTrace();
>
>
> } catch (TasteException
> e) {
>
>
> e.printStackTrace();
>
>
> }
>
>
>
>
>
> }
>
>
> The code I used for testing Item Similarity recommendation is as follows
>
> public static void main(String[] args) {
>
> String recsFile="D://mahoutFiles//SampleData.txt";
>
> long ssoId=206008129;
>
> int noOfRecommendations=5;
>
> String itemId="GE-CORPPROG-IMLP-PTOOLS";
>
> try {
>
> //item similarity recommendation based on User
>
> FileDataModel dataModel = new FileDataModel(new File(recsFile));
>
> ItemSimilarity itemSimilarity = new
> LogLikelihoodSimilarity(dataModel);
>
> ItemBasedRecommender recommender =new
> GenericItemBasedRecommender(dataModel, itemSimilarity);
>
> List<RecommendedItem> recommendations
> =recommender.recommend(ssoId, noOfRecommendations);
>
>
>
> for (RecommendedItem recommendedItem : recommendations) {
>
> System.out.println(recommendedItem);
>
> }
>
>
>
>
>
> //item similarity recommendation based on item
>
> recommender =new GenericItemBasedRecommender(dataModel,
> itemSimilarity);
>
> recommendations=
> recommender.mostSimilarItems(dataModel.readItemIDFromString(itemId),
> noOfRecommendations);
>
> System.out.println("Item recommendations based on Item");
>
> for (RecommendedItem recommendedItem : recommendations) {
>
> System.out.println(recommendedItem);
>
> }
>
> } catch (IOException e) {
>
> // TODO Auto-generated catch block
>
> e.printStackTrace();
>
> } catch (TasteException e) {
>
> // TODO Auto-generated catch block
>
> e.printStackTrace();
>
> }
>
>
>
> }
>
>
> The issues or concerns I have are as follows.
>
>
>
> Problem 1:
>
>
> This code produces relevant
> recommendations when I have given my file in the format <User id>, <item
> id>, <preference value>.
> But in my actual sample file I have just the user id and Item id alone, no
> preference value. On my reference since it is kind of a Boolean preference,
> I tried
> using PearsonCorrelationSimilarity
> as well as TanimotoCoefficientSimilarity but it is not giving me any
> recommendations. My actual file format that I'd want to process would be in
> the format <user id>,
> <item id>, . I can see it is processing all users in my file( from my
> eclipse console logs) but no recommendations are being produced as output.
>
>
>
>
>
> Problem 2:
>
>
> I
> want to use my mahout recommender in Hadoop distributed environment. I
> don’t have
> a clue about it. I tried web help but most blogs refer me to
> org.apache.mahout.cf.taste.hadoop.item.RecommenderJob,
> the code
> available with mahout src. Unfortunately I’m not able to get much from
> the code
> as it is a bit complicated for a starter like me. (I have done a couple
> of Map reduce project on text files processing before ). Could someone
> please help me
> out with very basic simple code snippets to run my User Similarity and
> Item Similarity Recommendation code
> on hadoop environment .
>
>
>
>
>
> It would be much
> helpful if you could share me with some informative tutorials which
> could make my initial steps in Mahout comfortable before i could start
> exploring in
> depth.
>
>
>
>
>
> Thank you
>
>
>
>
> Thanks and Regards
> Bejoy.K.S
>
>
>