You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mahout.apache.org by "Sebastian Schelter (JIRA)" <ji...@apache.org> on 2011/06/17 16:28:47 UTC
[jira] [Created] (MAHOUT-736) Save one pass through the data in
ItemSimilarityJob and RecommenderJob by intelligently using counters
Save one pass through the data in ItemSimilarityJob and RecommenderJob by intelligently using counters
------------------------------------------------------------------------------------------------------
Key: MAHOUT-736
URL: https://issues.apache.org/jira/browse/MAHOUT-736
Project: Mahout
Issue Type: Improvement
Components: Collaborative Filtering
Affects Versions: 0.6
Reporter: Sebastian Schelter
Assignee: Sebastian Schelter
Attachments: MAHOUT-736.patch
Both ItemSimilarityJob and RecommenderJob have to find the overall number of users. This was done with an M/R pass over the data. Instead of that we can use a counter in a previous pass over the data (similar to the way the total number of ngrams if collected in CollocDriver) and thereby save one pass through the data.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAHOUT-736) Save one pass through the data in
ItemSimilarityJob and RecommenderJob by intelligently using counters
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13051746#comment-13051746 ]
Hudson commented on MAHOUT-736:
-------------------------------
Integrated in Mahout-Quality #893 (See [https://builds.apache.org/job/Mahout-Quality/893/])
MAHOUT-736 removing empty files
ssc : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1137458
Files :
* /mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/hadoop/similarity/item/CountUsersKeyWritable.java
* /mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/hadoop/similarity/item/CountUsersMapper.java
* /mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/hadoop/similarity/item/CountUsersReducer.java
* /mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/hadoop/similarity/item/CountUsersCombiner.java
> Save one pass through the data in ItemSimilarityJob and RecommenderJob by intelligently using counters
> ------------------------------------------------------------------------------------------------------
>
> Key: MAHOUT-736
> URL: https://issues.apache.org/jira/browse/MAHOUT-736
> Project: Mahout
> Issue Type: Improvement
> Components: Collaborative Filtering
> Affects Versions: 0.6
> Reporter: Sebastian Schelter
> Assignee: Sebastian Schelter
> Fix For: 0.6
>
> Attachments: MAHOUT-736.patch
>
>
> Both ItemSimilarityJob and RecommenderJob have to find the overall number of users. This was done with an M/R pass over the data. Instead of that we can use a counter in a previous pass over the data (similar to the way the total number of ngrams if collected in CollocDriver) and thereby save one pass through the data.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAHOUT-736) Save one pass through the data in
ItemSimilarityJob and RecommenderJob by intelligently using counters
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13051744#comment-13051744 ]
Hudson commented on MAHOUT-736:
-------------------------------
Integrated in Mahout-Quality #892 (See [https://builds.apache.org/job/Mahout-Quality/892/])
MAHOUT-736 Save one pass through the data in ItemSimilarityJob and RecommenderJob by intelligently using counters
ssc : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1137456
Files :
* /mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/hadoop/similarity/item/CountUsersKeyWritable.java
* /mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/hadoop/similarity/item/CountUsersMapper.java
* /mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/hadoop/similarity/item/CountUsersReducer.java
* /mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/hadoop/item/ToUserVectorReducer.java
* /mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/hadoop/TasteHadoopUtils.java
* /mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/hadoop/item/RecommenderJob.java
* /mahout/trunk/core/src/test/java/org/apache/mahout/cf/taste/hadoop/similarity/item/ItemSimilarityJobTest.java
* /mahout/trunk/core/src/test/java/org/apache/mahout/cf/taste/hadoop/item/ToUserVectorReducerTest.java
* /mahout/trunk/core/src/test/java/org/apache/mahout/cf/taste/hadoop/item/RecommenderJobTest.java
* /mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/hadoop/similarity/item/CountUsersCombiner.java
* /mahout/trunk/core/src/main/java/org/apache/mahout/cf/taste/hadoop/similarity/item/ItemSimilarityJob.java
> Save one pass through the data in ItemSimilarityJob and RecommenderJob by intelligently using counters
> ------------------------------------------------------------------------------------------------------
>
> Key: MAHOUT-736
> URL: https://issues.apache.org/jira/browse/MAHOUT-736
> Project: Mahout
> Issue Type: Improvement
> Components: Collaborative Filtering
> Affects Versions: 0.6
> Reporter: Sebastian Schelter
> Assignee: Sebastian Schelter
> Fix For: 0.6
>
> Attachments: MAHOUT-736.patch
>
>
> Both ItemSimilarityJob and RecommenderJob have to find the overall number of users. This was done with an M/R pass over the data. Instead of that we can use a counter in a previous pass over the data (similar to the way the total number of ngrams if collected in CollocDriver) and thereby save one pass through the data.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAHOUT-736) Save one pass through the data in
ItemSimilarityJob and RecommenderJob by intelligently using counters
Posted by "Sebastian Schelter (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sebastian Schelter updated MAHOUT-736:
--------------------------------------
Resolution: Fixed
Status: Resolved (was: Patch Available)
> Save one pass through the data in ItemSimilarityJob and RecommenderJob by intelligently using counters
> ------------------------------------------------------------------------------------------------------
>
> Key: MAHOUT-736
> URL: https://issues.apache.org/jira/browse/MAHOUT-736
> Project: Mahout
> Issue Type: Improvement
> Components: Collaborative Filtering
> Affects Versions: 0.6
> Reporter: Sebastian Schelter
> Assignee: Sebastian Schelter
> Fix For: 0.6
>
> Attachments: MAHOUT-736.patch
>
>
> Both ItemSimilarityJob and RecommenderJob have to find the overall number of users. This was done with an M/R pass over the data. Instead of that we can use a counter in a previous pass over the data (similar to the way the total number of ngrams if collected in CollocDriver) and thereby save one pass through the data.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAHOUT-736) Save one pass through the data in
ItemSimilarityJob and RecommenderJob by intelligently using counters
Posted by "Sebastian Schelter (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sebastian Schelter updated MAHOUT-736:
--------------------------------------
Fix Version/s: 0.6
Status: Patch Available (was: Open)
Will commit this in the next days if there are no objections.
> Save one pass through the data in ItemSimilarityJob and RecommenderJob by intelligently using counters
> ------------------------------------------------------------------------------------------------------
>
> Key: MAHOUT-736
> URL: https://issues.apache.org/jira/browse/MAHOUT-736
> Project: Mahout
> Issue Type: Improvement
> Components: Collaborative Filtering
> Affects Versions: 0.6
> Reporter: Sebastian Schelter
> Assignee: Sebastian Schelter
> Fix For: 0.6
>
> Attachments: MAHOUT-736.patch
>
>
> Both ItemSimilarityJob and RecommenderJob have to find the overall number of users. This was done with an M/R pass over the data. Instead of that we can use a counter in a previous pass over the data (similar to the way the total number of ngrams if collected in CollocDriver) and thereby save one pass through the data.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAHOUT-736) Save one pass through the data in
ItemSimilarityJob and RecommenderJob by intelligently using counters
Posted by "Sean Owen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13051660#comment-13051660 ]
Sean Owen commented on MAHOUT-736:
----------------------------------
Wow does that work? great trick then.
> Save one pass through the data in ItemSimilarityJob and RecommenderJob by intelligently using counters
> ------------------------------------------------------------------------------------------------------
>
> Key: MAHOUT-736
> URL: https://issues.apache.org/jira/browse/MAHOUT-736
> Project: Mahout
> Issue Type: Improvement
> Components: Collaborative Filtering
> Affects Versions: 0.6
> Reporter: Sebastian Schelter
> Assignee: Sebastian Schelter
> Fix For: 0.6
>
> Attachments: MAHOUT-736.patch
>
>
> Both ItemSimilarityJob and RecommenderJob have to find the overall number of users. This was done with an M/R pass over the data. Instead of that we can use a counter in a previous pass over the data (similar to the way the total number of ngrams if collected in CollocDriver) and thereby save one pass through the data.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAHOUT-736) Save one pass through the data in
ItemSimilarityJob and RecommenderJob by intelligently using counters
Posted by "Sebastian Schelter (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sebastian Schelter updated MAHOUT-736:
--------------------------------------
Attachment: MAHOUT-736.patch
> Save one pass through the data in ItemSimilarityJob and RecommenderJob by intelligently using counters
> ------------------------------------------------------------------------------------------------------
>
> Key: MAHOUT-736
> URL: https://issues.apache.org/jira/browse/MAHOUT-736
> Project: Mahout
> Issue Type: Improvement
> Components: Collaborative Filtering
> Affects Versions: 0.6
> Reporter: Sebastian Schelter
> Assignee: Sebastian Schelter
> Attachments: MAHOUT-736.patch
>
>
> Both ItemSimilarityJob and RecommenderJob have to find the overall number of users. This was done with an M/R pass over the data. Instead of that we can use a counter in a previous pass over the data (similar to the way the total number of ngrams if collected in CollocDriver) and thereby save one pass through the data.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira