You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mahout.apache.org by "Sebastian Schelter (JIRA)" <ji...@apache.org> on 2010/04/27 10:04:32 UTC
[jira] Created: (MAHOUT-386)
org.apache.mahout.cf.taste.hadoop.pseudo.RecommenderJob breaks when no
usersFile is supplied
org.apache.mahout.cf.taste.hadoop.pseudo.RecommenderJob breaks when no usersFile is supplied
--------------------------------------------------------------------------------------------
Key: MAHOUT-386
URL: https://issues.apache.org/jira/browse/MAHOUT-386
Project: Mahout
Issue Type: Bug
Components: Collaborative Filtering
Reporter: Sebastian Schelter
When one tries to run org.apache.mahout.cf.taste.hadoop.pseudo.RecommenderJob without a usersFile (the file containing the userIDs to generate recommendations for) it throws an Exception in the Reducer.
I took a look at the code and it seems that there are some needless things done:
org.apache.mahout.cf.taste.hadoop.pseudo.RecommenderReducer tries to create a set of userIDs to generate recommendations for and fails to do this if no usersFile is supplied, when it tries to parse the preferences file. As far as I understand the code the Mapper already maps out only the userIDs to generate recommendations for, so there the check in the reducer is not even necessary, right? Correct me if I'm wrong.
---------------------------------------------------------------------------
How to reproduce the error:
1) Create a file containing some sample preferences and call it mahout-testing.txt, mine looks like this
1,2,1
1,3,2
2,1,1
2,3,1
2) Run the Job without a usersFile, e.g.:
hadoop jar core/target/mahout-core-0.4-SNAPSHOT.job org.apache.mahout.cf.taste.hadoop.pseudo.RecommenderJob -Dmapred.input.dir=./mahout-testing.txt -Dmapred.output.dir=/tmp/mahout/out --tempDir /tmp/mahout/tmp --recommenderClassName org.apache.mahout.cf.taste.impl.recommender.GenericItemBasedRecommender
java.lang.RuntimeException: Error in configuring object
at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93)
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
at org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:426)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:411)
at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:215)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:88)
... 5 more
Caused by: java.lang.NumberFormatException: For input string: "1,2,1"
at java.lang.NumberFormatException.forInputString(NumberFormatException.java:48)
at java.lang.Long.parseLong(Long.java:419)
at java.lang.Long.parseLong(Long.java:468)
at org.apache.mahout.cf.taste.hadoop.pseudo.RecommenderReducer.configure(RecommenderReducer.java:88)
... 10 more
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (MAHOUT-386)
org.apache.mahout.cf.taste.hadoop.pseudo.RecommenderJob breaks when no
usersFile is supplied
Posted by "Sean Owen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sean Owen resolved MAHOUT-386.
------------------------------
Assignee: Sean Owen
Fix Version/s: 0.4
Resolution: Fixed
Yeah looks like the old mechanism was left in and it should not have been. I just removed the code in the reducer. You can try again from SVN.
> org.apache.mahout.cf.taste.hadoop.pseudo.RecommenderJob breaks when no usersFile is supplied
> --------------------------------------------------------------------------------------------
>
> Key: MAHOUT-386
> URL: https://issues.apache.org/jira/browse/MAHOUT-386
> Project: Mahout
> Issue Type: Bug
> Components: Collaborative Filtering
> Reporter: Sebastian Schelter
> Assignee: Sean Owen
> Fix For: 0.4
>
>
> When one tries to run org.apache.mahout.cf.taste.hadoop.pseudo.RecommenderJob without a usersFile (the file containing the userIDs to generate recommendations for) it throws an Exception in the Reducer.
> I took a look at the code and it seems that there are some needless things done:
> org.apache.mahout.cf.taste.hadoop.pseudo.RecommenderReducer tries to create a set of userIDs to generate recommendations for and fails to do this if no usersFile is supplied, when it tries to parse the preferences file. As far as I understand the code the Mapper already maps out only the userIDs to generate recommendations for, so there the check in the reducer is not even necessary, right? Correct me if I'm wrong.
> ---------------------------------------------------------------------------
> How to reproduce the error:
> 1) Create a file containing some sample preferences and call it mahout-testing.txt, mine looks like this
> 1,2,1
> 1,3,2
> 2,1,1
> 2,3,1
> 2) Run the Job without a usersFile, e.g.:
> hadoop jar core/target/mahout-core-0.4-SNAPSHOT.job org.apache.mahout.cf.taste.hadoop.pseudo.RecommenderJob -Dmapred.input.dir=./mahout-testing.txt -Dmapred.output.dir=/tmp/mahout/out --tempDir /tmp/mahout/tmp --recommenderClassName org.apache.mahout.cf.taste.impl.recommender.GenericItemBasedRecommender
> java.lang.RuntimeException: Error in configuring object
> at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93)
> at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
> at org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:426)
> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:411)
> at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:215)
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:88)
> ... 5 more
> Caused by: java.lang.NumberFormatException: For input string: "1,2,1"
> at java.lang.NumberFormatException.forInputString(NumberFormatException.java:48)
> at java.lang.Long.parseLong(Long.java:419)
> at java.lang.Long.parseLong(Long.java:468)
> at org.apache.mahout.cf.taste.hadoop.pseudo.RecommenderReducer.configure(RecommenderReducer.java:88)
> ... 10 more
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAHOUT-386)
org.apache.mahout.cf.taste.hadoop.pseudo.RecommenderJob breaks when no
usersFile is supplied
Posted by "Sebastian Schelter (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12861410#action_12861410 ]
Sebastian Schelter commented on MAHOUT-386:
-------------------------------------------
tested it again, works fine now :)
> org.apache.mahout.cf.taste.hadoop.pseudo.RecommenderJob breaks when no usersFile is supplied
> --------------------------------------------------------------------------------------------
>
> Key: MAHOUT-386
> URL: https://issues.apache.org/jira/browse/MAHOUT-386
> Project: Mahout
> Issue Type: Bug
> Components: Collaborative Filtering
> Reporter: Sebastian Schelter
> Assignee: Sean Owen
> Fix For: 0.4
>
>
> When one tries to run org.apache.mahout.cf.taste.hadoop.pseudo.RecommenderJob without a usersFile (the file containing the userIDs to generate recommendations for) it throws an Exception in the Reducer.
> I took a look at the code and it seems that there are some needless things done:
> org.apache.mahout.cf.taste.hadoop.pseudo.RecommenderReducer tries to create a set of userIDs to generate recommendations for and fails to do this if no usersFile is supplied, when it tries to parse the preferences file. As far as I understand the code the Mapper already maps out only the userIDs to generate recommendations for, so there the check in the reducer is not even necessary, right? Correct me if I'm wrong.
> ---------------------------------------------------------------------------
> How to reproduce the error:
> 1) Create a file containing some sample preferences and call it mahout-testing.txt, mine looks like this
> 1,2,1
> 1,3,2
> 2,1,1
> 2,3,1
> 2) Run the Job without a usersFile, e.g.:
> hadoop jar core/target/mahout-core-0.4-SNAPSHOT.job org.apache.mahout.cf.taste.hadoop.pseudo.RecommenderJob -Dmapred.input.dir=./mahout-testing.txt -Dmapred.output.dir=/tmp/mahout/out --tempDir /tmp/mahout/tmp --recommenderClassName org.apache.mahout.cf.taste.impl.recommender.GenericItemBasedRecommender
> java.lang.RuntimeException: Error in configuring object
> at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93)
> at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
> at org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:426)
> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:411)
> at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:215)
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:88)
> ... 5 more
> Caused by: java.lang.NumberFormatException: For input string: "1,2,1"
> at java.lang.NumberFormatException.forInputString(NumberFormatException.java:48)
> at java.lang.Long.parseLong(Long.java:419)
> at java.lang.Long.parseLong(Long.java:468)
> at org.apache.mahout.cf.taste.hadoop.pseudo.RecommenderReducer.configure(RecommenderReducer.java:88)
> ... 10 more
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.