You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mahout.apache.org by "Manuel Blechschmidt (Created) (JIRA)" <ji...@apache.org> on 2011/11/30 23:13:40 UTC
[jira] [Created] (MAHOUT-905) CachingUserSimilarity and
CachingItemSimilarity have wrong (far to small) default maxSizes
CachingUserSimilarity and CachingItemSimilarity have wrong (far to small) default maxSizes
------------------------------------------------------------------------------------------
Key: MAHOUT-905
URL: https://issues.apache.org/jira/browse/MAHOUT-905
Project: Mahout
Issue Type: Bug
Components: Collaborative Filtering
Affects Versions: 0.5
Environment: Mac OS X 10.6.8
java version "1.6.0_29"
Java(TM) SE Runtime Environment (build 1.6.0_29-b11-402-10M3527)
Java HotSpot(TM) 64-Bit Server VM (build 20.4-b02-402, mixed mode)
This does not matter should be reproducible on every system.
Reporter: Manuel Blechschmidt
Assignee: Sean Owen
Attachments: CachingSimilariyAdjustedDefaultSize.patch
I am currently tuning my recommender discussed here: http://thread.gmane.org/gmane.comp.apache.mahout.user/10433.
As a first step I wrapped my LogLikelihoodSimilarity with an CachingUserSimilarity. I used Java Visual VM to profile the calls. I recognized that I didn't get any performance benefits. So I had a look into the code.
Actually line 47 this(similarity, dataModel.getNumItems()); in CachingUserSimilarity.java is wrong. If we want to cache all item similarities we need a cache with (dataModel.getNumItems()*(dataModel.getNumItems()-1))/2 possible entries.
I am now doing this in the constructor. I attached a patch to adjust this in the trunk build.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAHOUT-905) CachingUserSimilarity and
CachingItemSimilarity have wrong (far to small) default maxSizes
Posted by "Manuel Blechschmidt (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Manuel Blechschmidt updated MAHOUT-905:
---------------------------------------
Status: Patch Available (was: Open)
Attache is a patch solving this issue.
> CachingUserSimilarity and CachingItemSimilarity have wrong (far to small) default maxSizes
> ------------------------------------------------------------------------------------------
>
> Key: MAHOUT-905
> URL: https://issues.apache.org/jira/browse/MAHOUT-905
> Project: Mahout
> Issue Type: Bug
> Components: Collaborative Filtering
> Affects Versions: 0.5
> Environment: Mac OS X 10.6.8
> java version "1.6.0_29"
> Java(TM) SE Runtime Environment (build 1.6.0_29-b11-402-10M3527)
> Java HotSpot(TM) 64-Bit Server VM (build 20.4-b02-402, mixed mode)
> This does not matter should be reproducible on every system.
> Reporter: Manuel Blechschmidt
> Assignee: Sean Owen
> Labels: caching, max, similarity
> Attachments: CachingSimilariyAdjustedDefaultSize.patch
>
> Original Estimate: 0.5h
> Remaining Estimate: 0.5h
>
> I am currently tuning my recommender discussed here: http://thread.gmane.org/gmane.comp.apache.mahout.user/10433.
> As a first step I wrapped my LogLikelihoodSimilarity with an CachingUserSimilarity. I used Java Visual VM to profile the calls. I recognized that I didn't get any performance benefits. So I had a look into the code.
> Actually line 47 this(similarity, dataModel.getNumItems()); in CachingUserSimilarity.java is wrong. If we want to cache all item similarities we need a cache with (dataModel.getNumItems()*(dataModel.getNumItems()-1))/2 possible entries.
> I am now doing this in the constructor. I attached a patch to adjust this in the trunk build.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAHOUT-905) CachingUserSimilarity and
CachingItemSimilarity have wrong (far to small) default maxSizes
Posted by "Manuel Blechschmidt (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Manuel Blechschmidt updated MAHOUT-905:
---------------------------------------
Attachment: CachingSimilariyAdjustedDefaultSize.patch
The attached patch fixes this issue.
> CachingUserSimilarity and CachingItemSimilarity have wrong (far to small) default maxSizes
> ------------------------------------------------------------------------------------------
>
> Key: MAHOUT-905
> URL: https://issues.apache.org/jira/browse/MAHOUT-905
> Project: Mahout
> Issue Type: Bug
> Components: Collaborative Filtering
> Affects Versions: 0.5
> Environment: Mac OS X 10.6.8
> java version "1.6.0_29"
> Java(TM) SE Runtime Environment (build 1.6.0_29-b11-402-10M3527)
> Java HotSpot(TM) 64-Bit Server VM (build 20.4-b02-402, mixed mode)
> This does not matter should be reproducible on every system.
> Reporter: Manuel Blechschmidt
> Assignee: Sean Owen
> Labels: caching, max, similarity
> Attachments: CachingSimilariyAdjustedDefaultSize.patch
>
> Original Estimate: 0.5h
> Remaining Estimate: 0.5h
>
> I am currently tuning my recommender discussed here: http://thread.gmane.org/gmane.comp.apache.mahout.user/10433.
> As a first step I wrapped my LogLikelihoodSimilarity with an CachingUserSimilarity. I used Java Visual VM to profile the calls. I recognized that I didn't get any performance benefits. So I had a look into the code.
> Actually line 47 this(similarity, dataModel.getNumItems()); in CachingUserSimilarity.java is wrong. If we want to cache all item similarities we need a cache with (dataModel.getNumItems()*(dataModel.getNumItems()-1))/2 possible entries.
> I am now doing this in the constructor. I attached a patch to adjust this in the trunk build.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAHOUT-905) CachingUserSimilarity and
CachingItemSimilarity have wrong (far to small) default maxSizes
Posted by "Sean Owen (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sean Owen updated MAHOUT-905:
-----------------------------
Resolution: Not A Problem
Status: Resolved (was: Patch Available)
> CachingUserSimilarity and CachingItemSimilarity have wrong (far to small) default maxSizes
> ------------------------------------------------------------------------------------------
>
> Key: MAHOUT-905
> URL: https://issues.apache.org/jira/browse/MAHOUT-905
> Project: Mahout
> Issue Type: Improvement
> Components: Collaborative Filtering
> Affects Versions: 0.5
> Environment: Mac OS X 10.6.8
> java version "1.6.0_29"
> Java(TM) SE Runtime Environment (build 1.6.0_29-b11-402-10M3527)
> Java HotSpot(TM) 64-Bit Server VM (build 20.4-b02-402, mixed mode)
> This does not matter should be reproducible on every system.
> Reporter: Manuel Blechschmidt
> Assignee: Sean Owen
> Priority: Minor
> Labels: caching, max, similarity
> Attachments: CachingSimilariyAdjustedDefaultSize.patch
>
> Original Estimate: 0.5h
> Remaining Estimate: 0.5h
>
> I am currently tuning my recommender discussed here: http://thread.gmane.org/gmane.comp.apache.mahout.user/10433.
> As a first step I wrapped my LogLikelihoodSimilarity with an CachingUserSimilarity. I used Java Visual VM to profile the calls. I recognized that I didn't get any performance benefits. So I had a look into the code.
> Actually line 47 this(similarity, dataModel.getNumItems()); in CachingUserSimilarity.java is wrong. If we want to cache all item similarities we need a cache with (dataModel.getNumItems()*(dataModel.getNumItems()-1))/2 possible entries.
> I am now doing this in the constructor. I attached a patch to adjust this in the trunk build.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAHOUT-905) CachingUserSimilarity and
CachingItemSimilarity have wrong (far to small) default maxSizes
Posted by "Sean Owen (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sean Owen updated MAHOUT-905:
-----------------------------
Priority: Minor (was: Major)
Issue Type: Improvement (was: Bug)
(This is hardly a bug!)
The cache is supposed to be much smaller than the universe of all possible things you might cache, since only a small fraction will represent most of the pairs that are computed. If you cache everything I think you'll find your hit rate drops as lots of the elements are never read a second time. I would rather not create such a massive cache by default, no, though you can of course set it however you like for your use case.
> CachingUserSimilarity and CachingItemSimilarity have wrong (far to small) default maxSizes
> ------------------------------------------------------------------------------------------
>
> Key: MAHOUT-905
> URL: https://issues.apache.org/jira/browse/MAHOUT-905
> Project: Mahout
> Issue Type: Improvement
> Components: Collaborative Filtering
> Affects Versions: 0.5
> Environment: Mac OS X 10.6.8
> java version "1.6.0_29"
> Java(TM) SE Runtime Environment (build 1.6.0_29-b11-402-10M3527)
> Java HotSpot(TM) 64-Bit Server VM (build 20.4-b02-402, mixed mode)
> This does not matter should be reproducible on every system.
> Reporter: Manuel Blechschmidt
> Assignee: Sean Owen
> Priority: Minor
> Labels: caching, max, similarity
> Attachments: CachingSimilariyAdjustedDefaultSize.patch
>
> Original Estimate: 0.5h
> Remaining Estimate: 0.5h
>
> I am currently tuning my recommender discussed here: http://thread.gmane.org/gmane.comp.apache.mahout.user/10433.
> As a first step I wrapped my LogLikelihoodSimilarity with an CachingUserSimilarity. I used Java Visual VM to profile the calls. I recognized that I didn't get any performance benefits. So I had a look into the code.
> Actually line 47 this(similarity, dataModel.getNumItems()); in CachingUserSimilarity.java is wrong. If we want to cache all item similarities we need a cache with (dataModel.getNumItems()*(dataModel.getNumItems()-1))/2 possible entries.
> I am now doing this in the constructor. I attached a patch to adjust this in the trunk build.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira