You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@mahout.apache.org by "Rajpal, Harjeet Kumar" <Ha...@Honeywell.com> on 2009/12/01 07:50:41 UTC
RE: Mahout on Hadoop
Hello all,
This time when I reconfigured the Hadoop 0.20.1 the error has changed.
Now I am using Mahout 0.1. The error is:
09/12/01 12:07:01 INFO mapred.FileInputFormat: Total input paths to process : 1
09/12/01 12:07:04 INFO mapred.JobClient: Running job: job_200912011202_0002
09/12/01 12:07:05 INFO mapred.JobClient: map 0% reduce 0%
09/12/01 12:07:25 INFO mapred.JobClient: Task Id : attempt_200912011202_0002_m_000000_0, Status : FAILED
java.lang.RuntimeException: Error in configuring object
at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93)
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:354)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
at org.apache.hadoop.mapred.Child.main(Child.java:170)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:88)
... 5 more
Caused by: java.lang.RuntimeException: java.lang.RuntimeException: class org.apache.mahout.cf.taste.hadoop.RecommenderMapper not org.apache.hadoop.mapred.Mapper
at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:840)
at org.apache.hadoop.mapred.JobConf.getMapperClass(JobConf.java:771)
at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)
... 10 more
Caused by: java.lang.RuntimeException: class org.apache.mahout.cf.taste.hadoop.RecommenderMapper not org.apache.hadoop.mapred.Mapper
at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:834)
... 12 more
09/12/01 12:07:25 INFO mapred.JobClient: Task Id : attempt_200912011202_0002_m_000001_0, Status : FAILED
java.lang.RuntimeException: Error in configuring object
at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93)
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:354)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
at org.apache.hadoop.mapred.Child.main(Child.java:170)
I will Really appreciate if anybody can help.
Best Regards,
Harjeet Kumar Rajpal
-----Original Message-----
From: Rajpal, Harjeet Kumar [mailto:Harjeet.Kumar@Honeywell.com]
Sent: Monday, November 30, 2009 12:35 PM
To: mahout-user@lucene.apache.org
Subject: RE: Mahout on Hadoop
Hi,
Thanks for reply.
I tried mahout with Hadoop 0.18.3 now following error occurs
Command is:
HadoopAdmin@IE11DTXPJ9K2C1S ~/hadoop-0.18.3
$ bin/hadoop jar mahout-examples-0.2-SNAPSHOT.job org.apache.mahout.cf.taste.hadoop.RecommenderJob
org.apache.mahout.cf.taste.impl.recommender.slopeone.SlopeOneRecommenderTest 10 InputD/data1.txt InputD/data2.txt true
ERROR is :
java.lang.NoSuchMethodError: org.apache.hadoop.mapred.JobConf.setBooleanIfUnset(Ljava/lang/String;Z)
V
at org.apache.hadoop.mapreduce.Job.setUseNewAPI(Job.java:388)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:431)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:447)
at org.apache.mahout.cf.taste.hadoop.RecommenderJob.main(RecommenderJob.java:66)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:155)
at org.apache.hadoop.mapred.JobShell.run(JobShell.java:54)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
at org.apache.hadoop.mapred.JobShell.main(JobShell.java:68)
I hope it will clarify more on what is problem.
Thanks and regards,
Harjeet Kumar Rajpal
-----Original Message-----
From: Sean Owen [mailto:srowen@gmail.com]
Sent: Saturday, November 28, 2009 5:13 PM
To: mahout-user@lucene.apache.org
Subject: Re: Mahout on Hadoop
Yeah, this is an issue with Hadoop. You can see that when using the
new API, it's still looking for the old interfaces
(org.apache.hadoop.mapred.OutputFormat). I have also been waiting for
Hadoop to fix this.
On Sat, Nov 28, 2009 at 11:16 AM, Rajpal, Harjeet Kumar
<Ha...@honeywell.com> wrote:
> Hi,
>
> I am a newbie to Hadoop and Mahout. I have configured Hadoop on my
> machine and tried to run mahout on this.
>
> I tried following command.
>
> $ bin/hadoop jar mahout-core-0.2-SNAPSHOT.job
> org.apache.mahout.cf.taste.hadoop.RecommenderJob org.
>
> apache.mahout.cf.taste.impl.recommender.slopeone.SlopeOneRecommenderTest
> 10 InputD/data1.txt InputD
>
> /data2.txt true
>
>
>
> I got following error message:
>
>
>
> 09/11/28 16:22:25 WARN conf.Configuration: DEPRECATED: hadoop-site.xml
> found in the classpath. Usage of hadoop-site.xml is deprecated. Instead
> use core-site.xml, mapred-site.xml and hdfs-site.xml to override
> properties of core-default.xml, mapred-default.xml and hdfs-default.xml
> respectively
>
> 09/11/28 16:22:25 WARN mapred.JobClient: Use GenericOptionsParser for
> parsing the arguments. Applications should implement Tool for the same.
>
> Exception in thread "main" java.lang.RuntimeException:
> java.lang.RuntimeException: class
> org.apache.hadoop.mapreduce.lib.output.TextOutputFormat not
> org.apache.hadoop.mapred.OutputFormat
>
> at
> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:840)
>
> at
> org.apache.hadoop.mapred.JobConf.getOutputFormat(JobConf.java:488)
>
> at
> org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:772)
>
> at org.apache.hadoop.mapreduce.Job.submit(Job.java:432)
>
> at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:447)
>
> at
> org.apache.mahout.cf.taste.hadoop.RecommenderJob.main(RecommenderJob.jav
> a:66)
>
> Caused by: java.lang.RuntimeException: class
> org.apache.hadoop.mapreduce.lib.output.TextOutputFormat not
> org.apache.hadoop.mapred.OutputFormat
>
> at
> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:834)
>
> ... 5 more
>
> Can anybody tell How to resolve this.
>
> Thanks in advance
>
>
>
> Best Regards,
>
> Harjeet Kumar Rajpal
>
>
>
>
RE: Mahout on Hadoop
Posted by "Rajpal, Harjeet Kumar" <Ha...@Honeywell.com>.
I checked out new code and tried with this. you are right it is simpler
way. It is also working for me:). Thanks, carry on with the great work.
Now I am working more on understanding basics of both mahout and hadoop.
-----Original Message-----
From: Sean Owen [mailto:srowen@gmail.com]
Sent: Tuesday, December 08, 2009 10:10 PM
To: mahout-user@lucene.apache.org
Subject: Re: Mahout on Hadoop
The good news is you are just about to get this running. I think it
will work now.
The bad news is I have already changed this code a little bit in the
last week. For example, the arguments have changed. You now pass the
ratings data as --input, there is no --dataModelFile flag, and there
is no more user ID file. It's simpler and faster though.
On Tue, Dec 8, 2009 at 11:32 AM, Rajpal, Harjeet Kumar
<Ha...@honeywell.com> wrote:
> Yes, I did that. Its working now. Sorry for inconvenience.
> Thanks for your help. Even though questions were stupid, still you
were
> very helpful.
>
> -----Original Message-----
> From: Sean Owen [mailto:srowen@gmail.com]
> Sent: Tuesday, December 08, 2009 2:50 PM
> To: mahout-user@lucene.apache.org
> Subject: Re: Mahout on Hadoop
>
> Harjeet I think you need to read the documentation of the code, of the
> data you're using and of Hadoop. I think many of the questions you are
> asking are answerable from those resources.
>
> The userIdFile is not your ratings file. It is a file full of user IDs
> to recommend for.
>
> On Tue, Dec 8, 2009 at 7:05 AM, Rajpal, Harjeet Kumar
> <Ha...@honeywell.com> wrote:
>> I just read the code in Recommender and saw that userIdfile path is
>> given as input to mapred.input.dir. if I only give --userIdFile
> /input/
>> instead of --userIdFile /input/users.dat. then it still gives number
>> format error.
>>
>
Re: Mahout on Hadoop
Posted by Sean Owen <sr...@gmail.com>.
The good news is you are just about to get this running. I think it
will work now.
The bad news is I have already changed this code a little bit in the
last week. For example, the arguments have changed. You now pass the
ratings data as --input, there is no --dataModelFile flag, and there
is no more user ID file. It's simpler and faster though.
On Tue, Dec 8, 2009 at 11:32 AM, Rajpal, Harjeet Kumar
<Ha...@honeywell.com> wrote:
> Yes, I did that. Its working now. Sorry for inconvenience.
> Thanks for your help. Even though questions were stupid, still you were
> very helpful.
>
> -----Original Message-----
> From: Sean Owen [mailto:srowen@gmail.com]
> Sent: Tuesday, December 08, 2009 2:50 PM
> To: mahout-user@lucene.apache.org
> Subject: Re: Mahout on Hadoop
>
> Harjeet I think you need to read the documentation of the code, of the
> data you're using and of Hadoop. I think many of the questions you are
> asking are answerable from those resources.
>
> The userIdFile is not your ratings file. It is a file full of user IDs
> to recommend for.
>
> On Tue, Dec 8, 2009 at 7:05 AM, Rajpal, Harjeet Kumar
> <Ha...@honeywell.com> wrote:
>> I just read the code in Recommender and saw that userIdfile path is
>> given as input to mapred.input.dir. if I only give --userIdFile
> /input/
>> instead of --userIdFile /input/users.dat. then it still gives number
>> format error.
>>
>
RE: Mahout on Hadoop
Posted by "Rajpal, Harjeet Kumar" <Ha...@Honeywell.com>.
Yes, I did that. Its working now. Sorry for inconvenience.
Thanks for your help. Even though questions were stupid, still you were
very helpful.
-----Original Message-----
From: Sean Owen [mailto:srowen@gmail.com]
Sent: Tuesday, December 08, 2009 2:50 PM
To: mahout-user@lucene.apache.org
Subject: Re: Mahout on Hadoop
Harjeet I think you need to read the documentation of the code, of the
data you're using and of Hadoop. I think many of the questions you are
asking are answerable from those resources.
The userIdFile is not your ratings file. It is a file full of user IDs
to recommend for.
On Tue, Dec 8, 2009 at 7:05 AM, Rajpal, Harjeet Kumar
<Ha...@honeywell.com> wrote:
> I just read the code in Recommender and saw that userIdfile path is
> given as input to mapred.input.dir. if I only give --userIdFile
/input/
> instead of --userIdFile /input/users.dat. then it still gives number
> format error.
>
Re: Mahout on Hadoop
Posted by Sean Owen <sr...@gmail.com>.
Harjeet I think you need to read the documentation of the code, of the
data you're using and of Hadoop. I think many of the questions you are
asking are answerable from those resources.
The userIdFile is not your ratings file. It is a file full of user IDs
to recommend for.
On Tue, Dec 8, 2009 at 7:05 AM, Rajpal, Harjeet Kumar
<Ha...@honeywell.com> wrote:
> I just read the code in Recommender and saw that userIdfile path is
> given as input to mapred.input.dir. if I only give --userIdFile /input/
> instead of --userIdFile /input/users.dat. then it still gives number
> format error.
>
Re: Mahout on Hadoop
Posted by Sean Owen <sr...@gmail.com>.
This is not the ratings file, ua.base is the ratings file. Check the
README file in the data set.
I think you can see that because this file has 5 fields, not 3, and
has "F" -- what user or item ID would that be?
On Tue, Dec 8, 2009 at 6:24 AM, Rajpal, Harjeet Kumar
<Ha...@honeywell.com> wrote:
> This is format of users.dat file from group lens data.
>
> "1,F,1,10,48067" . format of rating file is right as userID,itemID,pref.
RE: Mahout on Hadoop
Posted by "Rajpal, Harjeet Kumar" <Ha...@Honeywell.com>.
I just read the code in Recommender and saw that userIdfile path is
given as input to mapred.input.dir. if I only give --userIdFile /input/
instead of --userIdFile /input/users.dat. then it still gives number
format error.
09/12/08 12:27:10 INFO mapred.JobClient: Running job:
job_200912072136_0008
09/12/08 12:27:11 INFO mapred.JobClient: map 0% reduce 0%
09/12/08 12:29:02 INFO mapred.JobClient: Task Id :
attempt_200912072136_0008_m_000000_0, Status : FAILED
java.lang.NumberFormatException: For input string: "23,3920,2"
at
java.lang.NumberFormatException.forInputString(NumberFormatException.jav
a:48)
at java.lang.Long.parseLong(Long.java:412)
at java.lang.Long.parseLong(Long.java:461)
at
org.apache.mahout.cf.taste.hadoop.RecommenderMapper.map(RecommenderMappe
r.java:104)
at
org.apache.mahout.cf.taste.hadoop.RecommenderMapper.map(RecommenderMappe
r.java:52)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
at
org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:358)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
at org.apache.hadoop.mapred.Child.main(Child.java:170)
09/12/08 12:29:02 INFO mapred.JobClient: Task Id :
attempt_200912072136_0008_m_000001_0, Status : FAILED
java.lang.NumberFormatException: For input string: "1,1193,5"
at
java.lang.NumberFormatException.forInputString(NumberFormatException.jav
a:48)
at java.lang.Long.parseLong(Long.java:412)
at java.lang.Long.parseLong(Long.java:461)
at
org.apache.mahout.cf.taste.hadoop.RecommenderMapper.map(RecommenderMappe
r.java:104)
at
org.apache.mahout.cf.taste.hadoop.RecommenderMapper.map(RecommenderMappe
r.java:52)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
at
org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:358)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
at org.apache.hadoop.mapred.Child.main(Child.java:170)
09/12/08 12:29:14 INFO mapred.JobClient: Task Id :
attempt_200912072136_0008_m_000001_1, Status : FAILED
java.lang.NumberFormatException: For input string: "1,1193,5"
at
java.lang.NumberFormatException.forInputString(NumberFormatException.jav
a:48)
at java.lang.Long.parseLong(Long.java:412)
at java.lang.Long.parseLong(Long.java:461)
at
org.apache.mahout.cf.taste.hadoop.RecommenderMapper.map(RecommenderMappe
r.java:104)
at
org.apache.mahout.cf.taste.hadoop.RecommenderMapper.map(RecommenderMappe
r.java:52)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
at
org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:358)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
at org.apache.hadoop.mapred.Child.main(Child.java:170)
09/12/08 12:29:20 INFO mapred.JobClient: Task Id :
attempt_200912072136_0008_m_000000_1, Status : FAILED
java.lang.NumberFormatException: For input string: "23,3920,2"
at
java.lang.NumberFormatException.forInputString(NumberFormatException.jav
a:48)
at java.lang.Long.parseLong(Long.java:412)
at java.lang.Long.parseLong(Long.java:461)
at
org.apache.mahout.cf.taste.hadoop.RecommenderMapper.map(RecommenderMappe
r.java:104)
at
org.apache.mahout.cf.taste.hadoop.RecommenderMapper.map(RecommenderMappe
r.java:52)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
at
org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:358)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
at org.apache.hadoop.mapred.Child.main(Child.java:170)
09/12/08 12:29:26 INFO mapred.JobClient: Task Id :
attempt_200912072136_0008_m_000000_2, Status : FAILED
java.lang.NumberFormatException: For input string: "23,3920,2"
at
java.lang.NumberFormatException.forInputString(NumberFormatException.jav
a:48)
at java.lang.Long.parseLong(Long.java:412)
at java.lang.Long.parseLong(Long.java:461)
at
org.apache.mahout.cf.taste.hadoop.RecommenderMapper.map(RecommenderMappe
r.java:104)
at
org.apache.mahout.cf.taste.hadoop.RecommenderMapper.map(RecommenderMappe
r.java:52)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
at
org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:358)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
at org.apache.hadoop.mapred.Child.main(Child.java:170)
09/12/08 12:29:26 INFO mapred.JobClient: Task Id :
attempt_200912072136_0008_m_000001_2, Status : FAILED
java.lang.NumberFormatException: For input string: "1,1193,5"
at
java.lang.NumberFormatException.forInputString(NumberFormatException.jav
a:48)
at java.lang.Long.parseLong(Long.java:412)
at java.lang.Long.parseLong(Long.java:461)
at
org.apache.mahout.cf.taste.hadoop.RecommenderMapper.map(RecommenderMappe
r.java:104)
at
org.apache.mahout.cf.taste.hadoop.RecommenderMapper.map(RecommenderMappe
r.java:52)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
at
org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:358)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
at org.apache.hadoop.mapred.Child.main(Child.java:170)
09/12/08 12:29:41 INFO mapred.JobClient: Job complete:
job_200912072136_0008
09/12/08 12:29:41 INFO mapred.JobClient: Counters: 3
09/12/08 12:29:41 INFO mapred.JobClient: Job Counters
09/12/08 12:29:41 INFO mapred.JobClient: Launched map tasks=8
09/12/08 12:29:41 INFO mapred.JobClient: Data-local map tasks=8
09/12/08 12:29:41 INFO mapred.JobClient: Failed map tasks=1
Exception in thread "main" java.io.IOException: Job failed!
at
org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1252)
at
org.apache.mahout.cf.taste.hadoop.RecommenderJob.main(RecommenderJob.jav
a:178)
Honestly, This is for me is confusing. I tried to figure out solution
but could not.:(
-----Original Message-----
From: Rajpal, Harjeet Kumar [mailto:Harjeet.Kumar@Honeywell.com]
Sent: Tuesday, December 08, 2009 12:02 PM
To: mahout-user@lucene.apache.org
Subject: RE: Mahout on Hadoop
I mean if I am right both ratings and userid file are needed as input to
recommenderjob.as per cfollowing command.
Bin/hadoop jar mahout-examples-0.3-SNAPSHOT.job
org.apache.mahout.cf.taste.hadoop.RecommenderJob --recommenderClassName
org.apache.mahout.cf.taste.example.grouplens.GroupLensRecommender
--userRec 5 --userIdFile /input/users.dat --dataModelFile
/input/ratings.dat --output reducerOutputDir --jarFile
mahout-examples-0.3-SNAPSHOT.job
The command in previous message was wrong. Sorry for that.
-----Original Message-----
From: Rajpal, Harjeet Kumar [mailto:Harjeet.Kumar@Honeywell.com]
Sent: Tuesday, December 08, 2009 11:55 AM
To: mahout-user@lucene.apache.org
Subject: RE: Mahout on Hadoop
This is format of users.dat file from group lens data.
"1,F,1,10,48067" . format of rating file is right as userID,itemID,pref.
Bin/hadoop jar mahout-examples-0.3-SNAPSHOT.job --recommenderClassName
org.apache.mahout.cf.taste.example.grouplens.GroupLensRecommender
--userRec 5 --userIdFile /input/users.dat --dataModelFile
/input/ratings.dat --output reducerOutputDir --jarFile
mahout-examples-0.3-SNAPSHOT.job
Best Regards,
Harjeet Kumar
-----Original Message-----
From: Sean Owen [mailto:srowen@gmail.com]
Sent: Monday, December 07, 2009 9:55 PM
To: mahout-user@lucene.apache.org
Subject: Re: Mahout on Hadoop
Yes, use the GroupLens 100K data set.
I don't know what data file you are passing, but this does not appear
to be a valid preference file, what is this? Should be
userID,itemID,pref. You may need to transform the input.
On Mon, Dec 7, 2009 at 4:14 PM, Rajpal, Harjeet Kumar
<Ha...@honeywell.com> wrote:
>
> Ya you are right. I corrected that mistake. Sorry for inconvenience.
> I tried this with Grouplens data. It gave java.lang.OutOfMemoryError.
> can you suggest any smaller amount of data for this. I tried to use a
> part of grouplens data. Then it gave number format exception.
> "1,F,1,10,48067" .
>
> If you can provide some demo data. That will be helpful.
> Best Regards,
> Harjeet Kumar
>
> -----Original Message-----
> From: Sean Owen [mailto:srowen@gmail.com]
> Sent: Monday, December 07, 2009 3:17 PM
> To: mahout-user@lucene.apache.org
> Subject: Re: Mahout on Hadoop
>
> Yes, that's right, you want to use the .job file as your ".jar" file.
> Are you passing it both as the argument to "hadoop jar" and the
> "--jarFile" argument? I suspect you still have some deployment problem
> since this all works for me.
>
> On an unrelated note, I am changing this part of the code dramatically
> right now, improving it and cleaning it up. You may want to update at
> some point but it will likely break what you are doing a little bit.
>
> On Mon, Dec 7, 2009 at 9:18 AM, Rajpal, Harjeet Kumar
> <Ha...@honeywell.com> wrote:
>> I have extracted the mahout-core-snapshot.job and checked the lib.
The
>> relevant uncommons-maths-1.2.jar is inside that.
>> I hope I am not making any stupid mistake here. If I am, please
> correct
>> me. Best Regards,
>> Harjeet Kumar Rajpal
>>
>> -----Original Message-----
>> From: Sean Owen [mailto:srowen@gmail.com]
>> Sent: Monday, December 07, 2009 1:02 PM
>> To: mahout-user@lucene.apache.org
>> Subject: Re: Mahout on Hadoop
>>
>> Caused by: java.lang.ClassNotFoundException:
>> org.uncommons.maths.random.MersenneTwisterRNG
>>
>> You aren't bundling the dependent class files with the .jar file used
>> with the Hadoop job.
>>
>> On Mon, Dec 7, 2009 at 7:01 AM, Rajpal, Harjeet Kumar
>> <Ha...@honeywell.com> wrote:
>>> Hi Sean,
>>>
>>> Sorry for Delay in Reply.
>>> I got the new code now. The error has changed.
>>>
>>
>
RE: Mahout on Hadoop
Posted by "Rajpal, Harjeet Kumar" <Ha...@Honeywell.com>.
I mean if I am right both ratings and userid file are needed as input to
recommenderjob.as per cfollowing command.
Bin/hadoop jar mahout-examples-0.3-SNAPSHOT.job
org.apache.mahout.cf.taste.hadoop.RecommenderJob --recommenderClassName
org.apache.mahout.cf.taste.example.grouplens.GroupLensRecommender
--userRec 5 --userIdFile /input/users.dat --dataModelFile
/input/ratings.dat --output reducerOutputDir --jarFile
mahout-examples-0.3-SNAPSHOT.job
The command in previous message was wrong. Sorry for that.
-----Original Message-----
From: Rajpal, Harjeet Kumar [mailto:Harjeet.Kumar@Honeywell.com]
Sent: Tuesday, December 08, 2009 11:55 AM
To: mahout-user@lucene.apache.org
Subject: RE: Mahout on Hadoop
This is format of users.dat file from group lens data.
"1,F,1,10,48067" . format of rating file is right as userID,itemID,pref.
Bin/hadoop jar mahout-examples-0.3-SNAPSHOT.job --recommenderClassName
org.apache.mahout.cf.taste.example.grouplens.GroupLensRecommender
--userRec 5 --userIdFile /input/users.dat --dataModelFile
/input/ratings.dat --output reducerOutputDir --jarFile
mahout-examples-0.3-SNAPSHOT.job
Best Regards,
Harjeet Kumar
-----Original Message-----
From: Sean Owen [mailto:srowen@gmail.com]
Sent: Monday, December 07, 2009 9:55 PM
To: mahout-user@lucene.apache.org
Subject: Re: Mahout on Hadoop
Yes, use the GroupLens 100K data set.
I don't know what data file you are passing, but this does not appear
to be a valid preference file, what is this? Should be
userID,itemID,pref. You may need to transform the input.
On Mon, Dec 7, 2009 at 4:14 PM, Rajpal, Harjeet Kumar
<Ha...@honeywell.com> wrote:
>
> Ya you are right. I corrected that mistake. Sorry for inconvenience.
> I tried this with Grouplens data. It gave java.lang.OutOfMemoryError.
> can you suggest any smaller amount of data for this. I tried to use a
> part of grouplens data. Then it gave number format exception.
> "1,F,1,10,48067" .
>
> If you can provide some demo data. That will be helpful.
> Best Regards,
> Harjeet Kumar
>
> -----Original Message-----
> From: Sean Owen [mailto:srowen@gmail.com]
> Sent: Monday, December 07, 2009 3:17 PM
> To: mahout-user@lucene.apache.org
> Subject: Re: Mahout on Hadoop
>
> Yes, that's right, you want to use the .job file as your ".jar" file.
> Are you passing it both as the argument to "hadoop jar" and the
> "--jarFile" argument? I suspect you still have some deployment problem
> since this all works for me.
>
> On an unrelated note, I am changing this part of the code dramatically
> right now, improving it and cleaning it up. You may want to update at
> some point but it will likely break what you are doing a little bit.
>
> On Mon, Dec 7, 2009 at 9:18 AM, Rajpal, Harjeet Kumar
> <Ha...@honeywell.com> wrote:
>> I have extracted the mahout-core-snapshot.job and checked the lib.
The
>> relevant uncommons-maths-1.2.jar is inside that.
>> I hope I am not making any stupid mistake here. If I am, please
> correct
>> me. Best Regards,
>> Harjeet Kumar Rajpal
>>
>> -----Original Message-----
>> From: Sean Owen [mailto:srowen@gmail.com]
>> Sent: Monday, December 07, 2009 1:02 PM
>> To: mahout-user@lucene.apache.org
>> Subject: Re: Mahout on Hadoop
>>
>> Caused by: java.lang.ClassNotFoundException:
>> org.uncommons.maths.random.MersenneTwisterRNG
>>
>> You aren't bundling the dependent class files with the .jar file used
>> with the Hadoop job.
>>
>> On Mon, Dec 7, 2009 at 7:01 AM, Rajpal, Harjeet Kumar
>> <Ha...@honeywell.com> wrote:
>>> Hi Sean,
>>>
>>> Sorry for Delay in Reply.
>>> I got the new code now. The error has changed.
>>>
>>
>
RE: Mahout on Hadoop
Posted by "Rajpal, Harjeet Kumar" <Ha...@Honeywell.com>.
This is format of users.dat file from group lens data.
"1,F,1,10,48067" . format of rating file is right as userID,itemID,pref.
Bin/hadoop jar mahout-examples-0.3-SNAPSHOT.job --recommenderClassName
org.apache.mahout.cf.taste.example.grouplens.GroupLensRecommender
--userRec 5 --userIdFile /input/users.dat --dataModelFile
/input/ratings.dat --output reducerOutputDir --jarFile
mahout-examples-0.3-SNAPSHOT.job
Best Regards,
Harjeet Kumar
-----Original Message-----
From: Sean Owen [mailto:srowen@gmail.com]
Sent: Monday, December 07, 2009 9:55 PM
To: mahout-user@lucene.apache.org
Subject: Re: Mahout on Hadoop
Yes, use the GroupLens 100K data set.
I don't know what data file you are passing, but this does not appear
to be a valid preference file, what is this? Should be
userID,itemID,pref. You may need to transform the input.
On Mon, Dec 7, 2009 at 4:14 PM, Rajpal, Harjeet Kumar
<Ha...@honeywell.com> wrote:
>
> Ya you are right. I corrected that mistake. Sorry for inconvenience.
> I tried this with Grouplens data. It gave java.lang.OutOfMemoryError.
> can you suggest any smaller amount of data for this. I tried to use a
> part of grouplens data. Then it gave number format exception.
> "1,F,1,10,48067" .
>
> If you can provide some demo data. That will be helpful.
> Best Regards,
> Harjeet Kumar
>
> -----Original Message-----
> From: Sean Owen [mailto:srowen@gmail.com]
> Sent: Monday, December 07, 2009 3:17 PM
> To: mahout-user@lucene.apache.org
> Subject: Re: Mahout on Hadoop
>
> Yes, that's right, you want to use the .job file as your ".jar" file.
> Are you passing it both as the argument to "hadoop jar" and the
> "--jarFile" argument? I suspect you still have some deployment problem
> since this all works for me.
>
> On an unrelated note, I am changing this part of the code dramatically
> right now, improving it and cleaning it up. You may want to update at
> some point but it will likely break what you are doing a little bit.
>
> On Mon, Dec 7, 2009 at 9:18 AM, Rajpal, Harjeet Kumar
> <Ha...@honeywell.com> wrote:
>> I have extracted the mahout-core-snapshot.job and checked the lib.
The
>> relevant uncommons-maths-1.2.jar is inside that.
>> I hope I am not making any stupid mistake here. If I am, please
> correct
>> me. Best Regards,
>> Harjeet Kumar Rajpal
>>
>> -----Original Message-----
>> From: Sean Owen [mailto:srowen@gmail.com]
>> Sent: Monday, December 07, 2009 1:02 PM
>> To: mahout-user@lucene.apache.org
>> Subject: Re: Mahout on Hadoop
>>
>> Caused by: java.lang.ClassNotFoundException:
>> org.uncommons.maths.random.MersenneTwisterRNG
>>
>> You aren't bundling the dependent class files with the .jar file used
>> with the Hadoop job.
>>
>> On Mon, Dec 7, 2009 at 7:01 AM, Rajpal, Harjeet Kumar
>> <Ha...@honeywell.com> wrote:
>>> Hi Sean,
>>>
>>> Sorry for Delay in Reply.
>>> I got the new code now. The error has changed.
>>>
>>
>
Re: Mahout on Hadoop
Posted by Sean Owen <sr...@gmail.com>.
Yes, use the GroupLens 100K data set.
I don't know what data file you are passing, but this does not appear
to be a valid preference file, what is this? Should be
userID,itemID,pref. You may need to transform the input.
On Mon, Dec 7, 2009 at 4:14 PM, Rajpal, Harjeet Kumar
<Ha...@honeywell.com> wrote:
>
> Ya you are right. I corrected that mistake. Sorry for inconvenience.
> I tried this with Grouplens data. It gave java.lang.OutOfMemoryError.
> can you suggest any smaller amount of data for this. I tried to use a
> part of grouplens data. Then it gave number format exception.
> "1,F,1,10,48067" .
>
> If you can provide some demo data. That will be helpful.
> Best Regards,
> Harjeet Kumar
>
> -----Original Message-----
> From: Sean Owen [mailto:srowen@gmail.com]
> Sent: Monday, December 07, 2009 3:17 PM
> To: mahout-user@lucene.apache.org
> Subject: Re: Mahout on Hadoop
>
> Yes, that's right, you want to use the .job file as your ".jar" file.
> Are you passing it both as the argument to "hadoop jar" and the
> "--jarFile" argument? I suspect you still have some deployment problem
> since this all works for me.
>
> On an unrelated note, I am changing this part of the code dramatically
> right now, improving it and cleaning it up. You may want to update at
> some point but it will likely break what you are doing a little bit.
>
> On Mon, Dec 7, 2009 at 9:18 AM, Rajpal, Harjeet Kumar
> <Ha...@honeywell.com> wrote:
>> I have extracted the mahout-core-snapshot.job and checked the lib. The
>> relevant uncommons-maths-1.2.jar is inside that.
>> I hope I am not making any stupid mistake here. If I am, please
> correct
>> me. Best Regards,
>> Harjeet Kumar Rajpal
>>
>> -----Original Message-----
>> From: Sean Owen [mailto:srowen@gmail.com]
>> Sent: Monday, December 07, 2009 1:02 PM
>> To: mahout-user@lucene.apache.org
>> Subject: Re: Mahout on Hadoop
>>
>> Caused by: java.lang.ClassNotFoundException:
>> org.uncommons.maths.random.MersenneTwisterRNG
>>
>> You aren't bundling the dependent class files with the .jar file used
>> with the Hadoop job.
>>
>> On Mon, Dec 7, 2009 at 7:01 AM, Rajpal, Harjeet Kumar
>> <Ha...@honeywell.com> wrote:
>>> Hi Sean,
>>>
>>> Sorry for Delay in Reply.
>>> I got the new code now. The error has changed.
>>>
>>
>
RE: Mahout on Hadoop
Posted by "Rajpal, Harjeet Kumar" <Ha...@Honeywell.com>.
Ya you are right. I corrected that mistake. Sorry for inconvenience.
I tried this with Grouplens data. It gave java.lang.OutOfMemoryError.
can you suggest any smaller amount of data for this. I tried to use a
part of grouplens data. Then it gave number format exception.
"1,F,1,10,48067" .
If you can provide some demo data. That will be helpful.
Best Regards,
Harjeet Kumar
-----Original Message-----
From: Sean Owen [mailto:srowen@gmail.com]
Sent: Monday, December 07, 2009 3:17 PM
To: mahout-user@lucene.apache.org
Subject: Re: Mahout on Hadoop
Yes, that's right, you want to use the .job file as your ".jar" file.
Are you passing it both as the argument to "hadoop jar" and the
"--jarFile" argument? I suspect you still have some deployment problem
since this all works for me.
On an unrelated note, I am changing this part of the code dramatically
right now, improving it and cleaning it up. You may want to update at
some point but it will likely break what you are doing a little bit.
On Mon, Dec 7, 2009 at 9:18 AM, Rajpal, Harjeet Kumar
<Ha...@honeywell.com> wrote:
> I have extracted the mahout-core-snapshot.job and checked the lib. The
> relevant uncommons-maths-1.2.jar is inside that.
> I hope I am not making any stupid mistake here. If I am, please
correct
> me. Best Regards,
> Harjeet Kumar Rajpal
>
> -----Original Message-----
> From: Sean Owen [mailto:srowen@gmail.com]
> Sent: Monday, December 07, 2009 1:02 PM
> To: mahout-user@lucene.apache.org
> Subject: Re: Mahout on Hadoop
>
> Caused by: java.lang.ClassNotFoundException:
> org.uncommons.maths.random.MersenneTwisterRNG
>
> You aren't bundling the dependent class files with the .jar file used
> with the Hadoop job.
>
> On Mon, Dec 7, 2009 at 7:01 AM, Rajpal, Harjeet Kumar
> <Ha...@honeywell.com> wrote:
>> Hi Sean,
>>
>> Sorry for Delay in Reply.
>> I got the new code now. The error has changed.
>>
>
Re: Mahout on Hadoop
Posted by Sean Owen <sr...@gmail.com>.
Yes, that's right, you want to use the .job file as your ".jar" file.
Are you passing it both as the argument to "hadoop jar" and the
"--jarFile" argument? I suspect you still have some deployment problem
since this all works for me.
On an unrelated note, I am changing this part of the code dramatically
right now, improving it and cleaning it up. You may want to update at
some point but it will likely break what you are doing a little bit.
On Mon, Dec 7, 2009 at 9:18 AM, Rajpal, Harjeet Kumar
<Ha...@honeywell.com> wrote:
> I have extracted the mahout-core-snapshot.job and checked the lib. The
> relevant uncommons-maths-1.2.jar is inside that.
> I hope I am not making any stupid mistake here. If I am, please correct
> me. Best Regards,
> Harjeet Kumar Rajpal
>
> -----Original Message-----
> From: Sean Owen [mailto:srowen@gmail.com]
> Sent: Monday, December 07, 2009 1:02 PM
> To: mahout-user@lucene.apache.org
> Subject: Re: Mahout on Hadoop
>
> Caused by: java.lang.ClassNotFoundException:
> org.uncommons.maths.random.MersenneTwisterRNG
>
> You aren't bundling the dependent class files with the .jar file used
> with the Hadoop job.
>
> On Mon, Dec 7, 2009 at 7:01 AM, Rajpal, Harjeet Kumar
> <Ha...@honeywell.com> wrote:
>> Hi Sean,
>>
>> Sorry for Delay in Reply.
>> I got the new code now. The error has changed.
>>
>
RE: Mahout on Hadoop
Posted by "Rajpal, Harjeet Kumar" <Ha...@Honeywell.com>.
I have extracted the mahout-core-snapshot.job and checked the lib. The
relevant uncommons-maths-1.2.jar is inside that.
I hope I am not making any stupid mistake here. If I am, please correct
me. Best Regards,
Harjeet Kumar Rajpal
-----Original Message-----
From: Sean Owen [mailto:srowen@gmail.com]
Sent: Monday, December 07, 2009 1:02 PM
To: mahout-user@lucene.apache.org
Subject: Re: Mahout on Hadoop
Caused by: java.lang.ClassNotFoundException:
org.uncommons.maths.random.MersenneTwisterRNG
You aren't bundling the dependent class files with the .jar file used
with the Hadoop job.
On Mon, Dec 7, 2009 at 7:01 AM, Rajpal, Harjeet Kumar
<Ha...@honeywell.com> wrote:
> Hi Sean,
>
> Sorry for Delay in Reply.
> I got the new code now. The error has changed.
>
Re: Mahout on Hadoop
Posted by Sean Owen <sr...@gmail.com>.
Caused by: java.lang.ClassNotFoundException:
org.uncommons.maths.random.MersenneTwisterRNG
You aren't bundling the dependent class files with the .jar file used
with the Hadoop job.
On Mon, Dec 7, 2009 at 7:01 AM, Rajpal, Harjeet Kumar
<Ha...@honeywell.com> wrote:
> Hi Sean,
>
> Sorry for Delay in Reply.
> I got the new code now. The error has changed.
>
RE: Mahout on Hadoop
Posted by "Rajpal, Harjeet Kumar" <Ha...@Honeywell.com>.
Hi Sean,
Sorry for Delay in Reply.
I got the new code now. The error has changed.
09/12/07 12:25:34 INFO mapred.FileInputFormat: Total input paths to
process : 1
09/12/07 12:25:37 INFO mapred.JobClient: Running job:
job_200912071126_0007
09/12/07 12:25:38 INFO mapred.JobClient: map 0% reduce 0%
09/12/07 12:25:58 INFO mapred.JobClient: Task Id :
attempt_200912071126_0007_m_000000_0, Status : FAILED
java.lang.RuntimeException: Error in configuring object
at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:9
3)
at
org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)
at
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:
117)
at
org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:354)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
at org.apache.hadoop.mapred.Child.main(Child.java:170)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.jav
a:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessor
Impl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:8
8)
... 5 more
Caused by: java.lang.RuntimeException: Error in configuring object
at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:9
3)
at
org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)
at
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:
117)
at
org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)
... 10 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.jav
a:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessor
Impl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:8
8)
... 13 more
Caused by: java.lang.IllegalStateException:
java.lang.NoClassDefFoundError:
org/uncommons/maths/random/MersenneTwisterRNG
at
org.apache.mahout.cf.taste.hadoop.RecommenderMapper.configure(Recommende
rMapper.java:93)
... 18 more
Caused by: java.lang.NoClassDefFoundError:
org/uncommons/maths/random/MersenneTwisterRNG
at
org.apache.mahout.cf.taste.impl.common.FastByIDMap.<init>(FastByIDMap.ja
va:80)
at
org.apache.mahout.cf.taste.impl.common.FastByIDMap.<init>(FastByIDMap.ja
va:54)
at
org.apache.mahout.cf.taste.impl.model.file.FileDataModel.buildModel(File
DataModel.java:152)
at
org.apache.mahout.cf.taste.impl.model.file.FileDataModel.reload(FileData
Model.java:131)
at
org.apache.mahout.cf.taste.impl.model.file.FileDataModel.checkLoaded(Fil
eDataModel.java:347)
at
org.apache.mahout.cf.taste.impl.model.file.FileDataModel.getNumItems(Fil
eDataModel.java:406)
at
org.apache.mahout.cf.taste.impl.recommender.slopeone.MemoryDiffStorage.<
init>(MemoryDiffStorage.java:95)
at
org.apache.mahout.cf.taste.impl.recommender.slopeone.SlopeOneRecommender
.<init>(SlopeOneRecommender.java:63)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorA
ccessorImpl.java:39)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingCons
tructorAccessorImpl.java:27)
at
java.lang.reflect.Constructor.newInstance(Constructor.java:513)
at
org.apache.mahout.cf.taste.hadoop.RecommenderMapper.configure(Recommende
rMapper.java:83)
... 18 more
Caused by: java.lang.ClassNotFoundException:
org.uncommons.maths.random.MersenneTwisterRNG
at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:252)
at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:320)
... 31 more
09/12/07 12:25:59 INFO mapred.JobClient: Task Id :
attempt_200912071126_0007_m_000001_0, Status : FAILED
java.lang.RuntimeException: Error in configuring object
at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:9
3)
at
org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)
at
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:
117)
at
org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:354)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
at org.apache.hadoop.mapred.Child.main(Child.java:170)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.jav
a:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessor
Impl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:8
8)
... 5 more
Caused by: java.lang.RuntimeException: Error in configuring object
at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:9
3)
at
org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)
at
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:
117)
at
org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)
... 10 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.jav
a:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessor
Impl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:8
8)
... 13 more
Caused by: java.lang.IllegalStateException:
java.lang.NoClassDefFoundError:
org/uncommons/maths/random/MersenneTwisterRNG
at
org.apache.mahout.cf.taste.hadoop.RecommenderMapper.configure(Recommende
rMapper.java:93)
... 18 more
Caused by: java.lang.NoClassDefFoundError:
org/uncommons/maths/random/MersenneTwisterRNG
at
org.apache.mahout.cf.taste.impl.common.FastByIDMap.<init>(FastByIDMap.ja
va:80)
at
org.apache.mahout.cf.taste.impl.common.FastByIDMap.<init>(FastByIDMap.ja
va:54)
at
org.apache.mahout.cf.taste.impl.model.file.FileDataModel.buildModel(File
DataModel.java:152)
at
org.apache.mahout.cf.taste.impl.model.file.FileDataModel.reload(FileData
Model.java:131)
at
org.apache.mahout.cf.taste.impl.model.file.FileDataModel.checkLoaded(Fil
eDataModel.java:347)
at
org.apache.mahout.cf.taste.impl.model.file.FileDataModel.getNumItems(Fil
eDataModel.java:406)
at
org.apache.mahout.cf.taste.impl.recommender.slopeone.MemoryDiffStorage.<
init>(MemoryDiffStorage.java:95)
at
org.apache.mahout.cf.taste.impl.recommender.slopeone.SlopeOneRecommender
.<init>(SlopeOneRecommender.java:63)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorA
ccessorImpl.java:39)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingCons
tructorAccessorImpl.java:27)
at
java.lang.reflect.Constructor.newInstance(Constructor.java:513)
at
org.apache.mahout.cf.taste.hadoop.RecommenderMapper.configure(Recommende
rMapper.java:83)
... 18 more
Caused by: java.lang.ClassNotFoundException:
org.uncommons.maths.random.MersenneTwisterRNG
at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:252)
at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:320)
... 31 more
09/12/07 12:26:08 INFO mapred.JobClient: Task Id :
attempt_200912071126_0007_m_000001_1, Status : FAILED
java.lang.RuntimeException: Error in configuring object
at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:9
3)
at
org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)
at
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:
117)
at
org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:354)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
at org.apache.hadoop.mapred.Child.main(Child.java:170)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.jav
a:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessor
Impl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:8
8)
... 5 more
Caused by: java.lang.RuntimeException: Error in configuring object
at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:9
3)
at
org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)
at
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:
117)
at
org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)
... 10 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.jav
a:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessor
Impl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:8
8)
... 13 more
Caused by: java.lang.IllegalStateException:
java.lang.NoClassDefFoundError:
org/uncommons/maths/random/MersenneTwisterRNG
at
org.apache.mahout.cf.taste.hadoop.RecommenderMapper.configure(Recommende
rMapper.java:93)
... 18 more
Caused by: java.lang.NoClassDefFoundError:
org/uncommons/maths/random/MersenneTwisterRNG
at
org.apache.mahout.cf.taste.impl.common.FastByIDMap.<init>(FastByIDMap.ja
va:80)
at
org.apache.mahout.cf.taste.impl.common.FastByIDMap.<init>(FastByIDMap.ja
va:54)
at
org.apache.mahout.cf.taste.impl.model.file.FileDataModel.buildModel(File
DataModel.java:152)
at
org.apache.mahout.cf.taste.impl.model.file.FileDataModel.reload(FileData
Model.java:131)
at
org.apache.mahout.cf.taste.impl.model.file.FileDataModel.checkLoaded(Fil
eDataModel.java:347)
at
org.apache.mahout.cf.taste.impl.model.file.FileDataModel.getNumItems(Fil
eDataModel.java:406)
at
org.apache.mahout.cf.taste.impl.recommender.slopeone.MemoryDiffStorage.<
init>(MemoryDiffStorage.java:95)
at
org.apache.mahout.cf.taste.impl.recommender.slopeone.SlopeOneRecommender
.<init>(SlopeOneRecommender.java:63)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorA
ccessorImpl.java:39)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingCons
tructorAccessorImpl.java:27)
at
java.lang.reflect.Constructor.newInstance(Constructor.java:513)
at
org.apache.mahout.cf.taste.hadoop.RecommenderMapper.configure(Recommende
rMapper.java:83)
... 18 more
Caused by: java.lang.ClassNotFoundException:
org.uncommons.maths.random.MersenneTwisterRNG
at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:252)
at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:320)
... 31 more
09/12/07 12:26:08 INFO mapred.JobClient: Task Id :
attempt_200912071126_0007_m_000000_1, Status : FAILED
java.lang.RuntimeException: Error in configuring object
at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:9
3)
at
org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)
at
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:
117)
at
org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:354)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
at org.apache.hadoop.mapred.Child.main(Child.java:170)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.jav
a:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessor
Impl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:8
8)
... 5 more
Caused by: java.lang.RuntimeException: Error in configuring object
at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:9
3)
at
org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)
at
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:
117)
at
org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)
... 10 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.jav
a:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessor
Impl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:8
8)
... 13 more
Caused by: java.lang.IllegalStateException:
java.lang.NoClassDefFoundError:
org/uncommons/maths/random/MersenneTwisterRNG
at
org.apache.mahout.cf.taste.hadoop.RecommenderMapper.configure(Recommende
rMapper.java:93)
... 18 more
Caused by: java.lang.NoClassDefFoundError:
org/uncommons/maths/random/MersenneTwisterRNG
at
org.apache.mahout.cf.taste.impl.common.FastByIDMap.<init>(FastByIDMap.ja
va:80)
at
org.apache.mahout.cf.taste.impl.common.FastByIDMap.<init>(FastByIDMap.ja
va:54)
at
org.apache.mahout.cf.taste.impl.model.file.FileDataModel.buildModel(File
DataModel.java:152)
at
org.apache.mahout.cf.taste.impl.model.file.FileDataModel.reload(FileData
Model.java:131)
at
org.apache.mahout.cf.taste.impl.model.file.FileDataModel.checkLoaded(Fil
eDataModel.java:347)
at
org.apache.mahout.cf.taste.impl.model.file.FileDataModel.getNumItems(Fil
eDataModel.java:406)
at
org.apache.mahout.cf.taste.impl.recommender.slopeone.MemoryDiffStorage.<
init>(MemoryDiffStorage.java:95)
at
org.apache.mahout.cf.taste.impl.recommender.slopeone.SlopeOneRecommender
.<init>(SlopeOneRecommender.java:63)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorA
ccessorImpl.java:39)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingCons
tructorAccessorImpl.java:27)
at
java.lang.reflect.Constructor.newInstance(Constructor.java:513)
at
org.apache.mahout.cf.taste.hadoop.RecommenderMapper.configure(Recommende
rMapper.java:83)
... 18 more
Caused by: java.lang.ClassNotFoundException:
org.uncommons.maths.random.MersenneTwisterRNG
at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:252)
at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:320)
... 31 more
09/12/07 12:26:16 INFO mapred.JobClient: Task Id :
attempt_200912071126_0007_m_000000_2, Status : FAILED
java.lang.RuntimeException: Error in configuring object
at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:9
3)
at
org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)
at
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:
117)
at
org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:354)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
at org.apache.hadoop.mapred.Child.main(Child.java:170)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.jav
a:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessor
Impl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:8
8)
... 5 more
Caused by: java.lang.RuntimeException: Error in configuring object
at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:9
3)
at
org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)
at
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:
117)
at
org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)
... 10 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.jav
a:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessor
Impl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:8
8)
... 13 more
Caused by: java.lang.IllegalStateException:
java.lang.NoClassDefFoundError:
org/uncommons/maths/random/MersenneTwisterRNG
at
org.apache.mahout.cf.taste.hadoop.RecommenderMapper.configure(Recommende
rMapper.java:93)
... 18 more
Caused by: java.lang.NoClassDefFoundError:
org/uncommons/maths/random/MersenneTwisterRNG
at
org.apache.mahout.cf.taste.impl.common.FastByIDMap.<init>(FastByIDMap.ja
va:80)
at
org.apache.mahout.cf.taste.impl.common.FastByIDMap.<init>(FastByIDMap.ja
va:54)
at
org.apache.mahout.cf.taste.impl.model.file.FileDataModel.buildModel(File
DataModel.java:152)
at
org.apache.mahout.cf.taste.impl.model.file.FileDataModel.reload(FileData
Model.java:131)
at
org.apache.mahout.cf.taste.impl.model.file.FileDataModel.checkLoaded(Fil
eDataModel.java:347)
at
org.apache.mahout.cf.taste.impl.model.file.FileDataModel.getNumItems(Fil
eDataModel.java:406)
at
org.apache.mahout.cf.taste.impl.recommender.slopeone.MemoryDiffStorage.<
init>(MemoryDiffStorage.java:95)
at
org.apache.mahout.cf.taste.impl.recommender.slopeone.SlopeOneRecommender
.<init>(SlopeOneRecommender.java:63)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorA
ccessorImpl.java:39)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingCons
tructorAccessorImpl.java:27)
at
java.lang.reflect.Constructor.newInstance(Constructor.java:513)
at
org.apache.mahout.cf.taste.hadoop.RecommenderMapper.configure(Recommende
rMapper.java:83)
... 18 more
Caused by: java.lang.ClassNotFoundException:
org.uncommons.maths.random.MersenneTwisterRNG
at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:252)
at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:320)
... 31 more
09/12/07 12:26:16 INFO mapred.JobClient: Task Id :
attempt_200912071126_0007_m_000001_2, Status : FAILED
java.lang.RuntimeException: Error in configuring object
at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:9
3)
at
org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)
at
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:
117)
at
org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:354)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
at org.apache.hadoop.mapred.Child.main(Child.java:170)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.jav
a:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessor
Impl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:8
8)
... 5 more
Caused by: java.lang.RuntimeException: Error in configuring object
at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:9
3)
at
org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)
at
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:
117)
at
org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)
... 10 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.jav
a:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessor
Impl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:8
8)
... 13 more
Caused by: java.lang.IllegalStateException:
java.lang.NoClassDefFoundError:
org/uncommons/maths/random/MersenneTwisterRNG
at
org.apache.mahout.cf.taste.hadoop.RecommenderMapper.configure(Recommende
rMapper.java:93)
... 18 more
Caused by: java.lang.NoClassDefFoundError:
org/uncommons/maths/random/MersenneTwisterRNG
at
org.apache.mahout.cf.taste.impl.common.FastByIDMap.<init>(FastByIDMap.ja
va:80)
at
org.apache.mahout.cf.taste.impl.common.FastByIDMap.<init>(FastByIDMap.ja
va:54)
at
org.apache.mahout.cf.taste.impl.model.file.FileDataModel.buildModel(File
DataModel.java:152)
at
org.apache.mahout.cf.taste.impl.model.file.FileDataModel.reload(FileData
Model.java:131)
at
org.apache.mahout.cf.taste.impl.model.file.FileDataModel.checkLoaded(Fil
eDataModel.java:347)
at
org.apache.mahout.cf.taste.impl.model.file.FileDataModel.getNumItems(Fil
eDataModel.java:406)
at
org.apache.mahout.cf.taste.impl.recommender.slopeone.MemoryDiffStorage.<
init>(MemoryDiffStorage.java:95)
at
org.apache.mahout.cf.taste.impl.recommender.slopeone.SlopeOneRecommender
.<init>(SlopeOneRecommender.java:63)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorA
ccessorImpl.java:39)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingCons
tructorAccessorImpl.java:27)
at
java.lang.reflect.Constructor.newInstance(Constructor.java:513)
at
org.apache.mahout.cf.taste.hadoop.RecommenderMapper.configure(Recommende
rMapper.java:83)
... 18 more
Caused by: java.lang.ClassNotFoundException:
org.uncommons.maths.random.MersenneTwisterRNG
at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:252)
at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:320)
... 31 more
09/12/07 12:26:29 INFO mapred.JobClient: Job complete:
job_200912071126_0007
09/12/07 12:26:29 INFO mapred.JobClient: Counters: 3
09/12/07 12:26:29 INFO mapred.JobClient: Job Counters
09/12/07 12:26:29 INFO mapred.JobClient: Launched map tasks=8
09/12/07 12:26:29 INFO mapred.JobClient: Data-local map tasks=8
09/12/07 12:26:29 INFO mapred.JobClient: Failed map tasks=1
Exception in thread "main" java.io.IOException: Job failed!
at
org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1252)
at
org.apache.mahout.cf.taste.hadoop.RecommenderJob.main(RecommenderJob.jav
a:178)
Best Regards,
Harjeet Kumar Rajpal
-----Original Message-----
From: Sean Owen [mailto:srowen@gmail.com]
Sent: Friday, December 04, 2009 1:38 PM
To: mahout-user@lucene.apache.org
Subject: Re: Mahout on Hadoop
Open RecommenderMapper.java. See that it contains "import
org.apache.hadoop.mapred.Mapper;" -- if it doesn't, then you don't
have the latest code. If it does, then it demonstrates that the Mapper
does implement the old APIs. The error you see says that the compiled
class does not implement the old API. that would suggest to me that
you do still somehow have the old compiled code somewhere. I can tell
you this works for me.
On Fri, Dec 4, 2009 at 6:06 AM, Rajpal, Harjeet Kumar
<Ha...@honeywell.com> wrote:
> Hi Sean,
> I checked out the whole code again from SVN but still the error is
same.
> What can be the possible other reason?
>
> Best Regards,
> Harjeet Kumar Rajpal
>
> -----Original Message-----
> From: Sean Owen [mailto:srowen@gmail.com]
> Sent: Thursday, December 03, 2009 9:52 PM
> To: mahout-user@lucene.apache.org
> Subject: Re: Mahout on Hadoop
>
> Oh wait, no, now I see the full stack trade. Yes that looks like the
> same error. Maybe you still have the old compiled code.
> RecommenderMapper now implements mapred.Mapper so that should not
> happen.
>
> On Thu, Dec 3, 2009 at 4:18 PM, Rajpal, Harjeet Kumar
> <Ha...@honeywell.com> wrote:
>> Yes, I am using Hadoop 0.20.1 . what can be reson for this error
>
Re: Mahout on Hadoop
Posted by Sean Owen <sr...@gmail.com>.
Open RecommenderMapper.java. See that it contains "import
org.apache.hadoop.mapred.Mapper;" -- if it doesn't, then you don't
have the latest code. If it does, then it demonstrates that the Mapper
does implement the old APIs. The error you see says that the compiled
class does not implement the old API. that would suggest to me that
you do still somehow have the old compiled code somewhere. I can tell
you this works for me.
On Fri, Dec 4, 2009 at 6:06 AM, Rajpal, Harjeet Kumar
<Ha...@honeywell.com> wrote:
> Hi Sean,
> I checked out the whole code again from SVN but still the error is same.
> What can be the possible other reason?
>
> Best Regards,
> Harjeet Kumar Rajpal
>
> -----Original Message-----
> From: Sean Owen [mailto:srowen@gmail.com]
> Sent: Thursday, December 03, 2009 9:52 PM
> To: mahout-user@lucene.apache.org
> Subject: Re: Mahout on Hadoop
>
> Oh wait, no, now I see the full stack trade. Yes that looks like the
> same error. Maybe you still have the old compiled code.
> RecommenderMapper now implements mapred.Mapper so that should not
> happen.
>
> On Thu, Dec 3, 2009 at 4:18 PM, Rajpal, Harjeet Kumar
> <Ha...@honeywell.com> wrote:
>> Yes, I am using Hadoop 0.20.1 . what can be reson for this error
>
RE: Mahout on Hadoop
Posted by "Rajpal, Harjeet Kumar" <Ha...@Honeywell.com>.
Hi Sean,
I checked out the whole code again from SVN but still the error is same.
What can be the possible other reason?
Best Regards,
Harjeet Kumar Rajpal
-----Original Message-----
From: Sean Owen [mailto:srowen@gmail.com]
Sent: Thursday, December 03, 2009 9:52 PM
To: mahout-user@lucene.apache.org
Subject: Re: Mahout on Hadoop
Oh wait, no, now I see the full stack trade. Yes that looks like the
same error. Maybe you still have the old compiled code.
RecommenderMapper now implements mapred.Mapper so that should not
happen.
On Thu, Dec 3, 2009 at 4:18 PM, Rajpal, Harjeet Kumar
<Ha...@honeywell.com> wrote:
> Yes, I am using Hadoop 0.20.1 . what can be reson for this error
Re: Mahout on Hadoop
Posted by Sean Owen <sr...@gmail.com>.
Oh wait, no, now I see the full stack trade. Yes that looks like the
same error. Maybe you still have the old compiled code.
RecommenderMapper now implements mapred.Mapper so that should not
happen.
On Thu, Dec 3, 2009 at 4:18 PM, Rajpal, Harjeet Kumar
<Ha...@honeywell.com> wrote:
> Yes, I am using Hadoop 0.20.1 . what can be reson for this error
RE: Mahout on Hadoop
Posted by "Rajpal, Harjeet Kumar" <Ha...@Honeywell.com>.
Yes, I am using Hadoop 0.20.1 . what can be reson for this error
-----Original Message-----
From: Sean Owen [mailto:srowen@gmail.com]
Sent: Thursday, December 03, 2009 9:37 PM
To: mahout-user@lucene.apache.org
Subject: Re: Mahout on Hadoop
That's not the same error. I don't see that. Are you using 0.20.1?
works for me at least.
On Thu, Dec 3, 2009 at 3:54 PM, Rajpal, Harjeet Kumar
<Ha...@honeywell.com> wrote:
> Hi Sean,
> I tried that.
> But the error is same as below
>
Re: Mahout on Hadoop
Posted by Sean Owen <sr...@gmail.com>.
That's not the same error. I don't see that. Are you using 0.20.1?
works for me at least.
On Thu, Dec 3, 2009 at 3:54 PM, Rajpal, Harjeet Kumar
<Ha...@honeywell.com> wrote:
> Hi Sean,
> I tried that.
> But the error is same as below
>
RE: Mahout on Hadoop
Posted by "Rajpal, Harjeet Kumar" <Ha...@Honeywell.com>.
Hi Sean,
I tried that.
But the error is same as below
09/12/03 21:21:36 INFO mapred.FileInputFormat: Total input paths to
process : 1
09/12/03 21:21:38 INFO mapred.JobClient: Running job:
job_200912031726_0037
09/12/03 21:21:39 INFO mapred.JobClient: map 0% reduce 0%
09/12/03 21:21:59 INFO mapred.JobClient: Task Id :
attempt_200912031726_0037_m_000000_0, Status : FAILED
java.lang.RuntimeException: Error in configuring object
at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:9
3)
at
org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)
at
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:
117)
at
org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:354)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
at org.apache.hadoop.mapred.Child.main(Child.java:170)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.jav
a:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessor
Impl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:8
8)
... 5 more
Caused by: java.lang.RuntimeException: java.lang.RuntimeException: class
org.apache.mahout.cf.taste.hadoop.RecommenderMapper not
org.apache.hadoop.mapred.Mapper
at
org.apache.hadoop.conf.Configuration.getClass(Configuration.java:840)
at
org.apache.hadoop.mapred.JobConf.getMapperClass(JobConf.java:771)
at
org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)
... 10 more
Caused by: java.lang.RuntimeException: class
org.apache.mahout.cf.taste.hadoop.RecommenderMapper not
org.apache.hadoop.mapred.Mapper
at
org.apache.hadoop.conf.Configuration.getClass(Configuration.java:834)
... 12 more
09/12/03 21:21:59 INFO mapred.JobClient: Task Id :
attempt_200912031726_0037_m_000001_0, Status : FAILED
java.lang.RuntimeException: Error in configuring object
at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:9
3)
at
org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)
at
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:
117)
at
org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:354)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
at org.apache.hadoop.mapred.Child.main(Child.java:170)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.jav
a:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessor
Impl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:8
8)
... 5 more
Caused by: java.lang.RuntimeException: java.lang.RuntimeException: class
org.apache.mahout.cf.taste.hadoop.RecommenderMapper not
org.apache.hadoop.mapred.Mapper
at
org.apache.hadoop.conf.Configuration.getClass(Configuration.java:840)
at
org.apache.hadoop.mapred.JobConf.getMapperClass(JobConf.java:771)
at
org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)
... 10 more
Caused by: java.lang.RuntimeException: class
org.apache.mahout.cf.taste.hadoop.RecommenderMapper not
org.apache.hadoop.mapred.Mapper
at
org.apache.hadoop.conf.Configuration.getClass(Configuration.java:834)
... 12 more
09/12/03 21:22:08 INFO mapred.JobClient: Task Id :
attempt_200912031726_0037_m_000001_1, Status : FAILED
java.lang.RuntimeException: Error in configuring object
at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:9
3)
at
org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)
at
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:
117)
at
org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:354)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
at org.apache.hadoop.mapred.Child.main(Child.java:170)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.jav
a:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessor
Impl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:8
8)
... 5 more
Caused by: java.lang.RuntimeException: java.lang.RuntimeException: class
org.apache.mahout.cf.taste.hadoop.RecommenderMapper not
org.apache.hadoop.mapred.Mapper
at
org.apache.hadoop.conf.Configuration.getClass(Configuration.java:840)
at
org.apache.hadoop.mapred.JobConf.getMapperClass(JobConf.java:771)
at
org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)
... 10 more
Caused by: java.lang.RuntimeException: class
org.apache.mahout.cf.taste.hadoop.RecommenderMapper not
org.apache.hadoop.mapred.Mapper
at
org.apache.hadoop.conf.Configuration.getClass(Configuration.java:834)
... 12 more
09/12/03 21:22:08 INFO mapred.JobClient: Task Id :
attempt_200912031726_0037_m_000000_1, Status : FAILED
java.lang.RuntimeException: Error in configuring object
at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:9
3)
at
org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)
at
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:
117)
at
org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:354)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
at org.apache.hadoop.mapred.Child.main(Child.java:170)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.jav
a:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessor
Impl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:8
8)
... 5 more
Caused by: java.lang.RuntimeException: java.lang.RuntimeException: class
org.apache.mahout.cf.taste.hadoop.RecommenderMapper not
org.apache.hadoop.mapred.Mapper
at
org.apache.hadoop.conf.Configuration.getClass(Configuration.java:840)
at
org.apache.hadoop.mapred.JobConf.getMapperClass(JobConf.java:771)
at
org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)
... 10 more
Caused by: java.lang.RuntimeException: class
org.apache.mahout.cf.taste.hadoop.RecommenderMapper not
org.apache.hadoop.mapred.Mapper
at
org.apache.hadoop.conf.Configuration.getClass(Configuration.java:834)
... 12 more
09/12/03 21:22:17 INFO mapred.JobClient: Task Id :
attempt_200912031726_0037_m_000000_2, Status : FAILED
java.lang.RuntimeException: Error in configuring object
at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:9
3)
at
org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)
at
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:
117)
at
org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:354)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
at org.apache.hadoop.mapred.Child.main(Child.java:170)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.jav
a:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessor
Impl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:8
8)
... 5 more
Caused by: java.lang.RuntimeException: java.lang.RuntimeException: class
org.apache.mahout.cf.taste.hadoop.RecommenderMapper not
org.apache.hadoop.mapred.Mapper
at
org.apache.hadoop.conf.Configuration.getClass(Configuration.java:840)
at
org.apache.hadoop.mapred.JobConf.getMapperClass(JobConf.java:771)
at
org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)
... 10 more
Caused by: java.lang.RuntimeException: class
org.apache.mahout.cf.taste.hadoop.RecommenderMapper not
org.apache.hadoop.mapred.Mapper
at
org.apache.hadoop.conf.Configuration.getClass(Configuration.java:834)
... 12 more
09/12/03 21:22:17 INFO mapred.JobClient: Task Id :
attempt_200912031726_0037_m_000001_2, Status : FAILED
java.lang.RuntimeException: Error in configuring object
at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:9
3)
at
org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)
at
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:
117)
at
org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:354)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
at org.apache.hadoop.mapred.Child.main(Child.java:170)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.jav
a:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessor
Impl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:8
8)
... 5 more
Caused by: java.lang.RuntimeException: java.lang.RuntimeException: class
org.apache.mahout.cf.taste.hadoop.RecommenderMapper not
org.apache.hadoop.mapred.Mapper
at
org.apache.hadoop.conf.Configuration.getClass(Configuration.java:840)
at
org.apache.hadoop.mapred.JobConf.getMapperClass(JobConf.java:771)
at
org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)
... 10 more
Caused by: java.lang.RuntimeException: class
org.apache.mahout.cf.taste.hadoop.RecommenderMapper not
org.apache.hadoop.mapred.Mapper
at
org.apache.hadoop.conf.Configuration.getClass(Configuration.java:834)
... 12 more
09/12/03 21:22:30 INFO mapred.JobClient: Job complete:
job_200912031726_0037
09/12/03 21:22:30 INFO mapred.JobClient: Counters: 3
09/12/03 21:22:30 INFO mapred.JobClient: Job Counters
09/12/03 21:22:30 INFO mapred.JobClient: Launched map tasks=8
09/12/03 21:22:30 INFO mapred.JobClient: Data-local map tasks=8
09/12/03 21:22:30 INFO mapred.JobClient: Failed map tasks=1
Exception in thread "main" java.io.IOException: Job failed!
at
org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1252)
at
org.apache.mahout.cf.taste.hadoop.RecommenderJob.main(RecommenderJob.jav
a:178)
Thanks and Regards,
Harjeet Kumar Rajpal
-----Original Message-----
From: Sean Owen [mailto:srowen@gmail.com]
Sent: Thursday, December 03, 2009 9:02 PM
To: mahout-user@lucene.apache.org
Subject: Re: Mahout on Hadoop
Sounds like a server problem, but I am able to access it now.
In any event, I'm just saying update the code from SVN. That's all.
On Thu, Dec 3, 2009 at 3:25 PM, Rajpal, Harjeet Kumar
<Ha...@honeywell.com> wrote:
> Hi Sean,
> I am unable to access https://issues.apache.org/jira/browse/MAHOUT-143
> It shows could not find server.
> Could you tell me some other way to access the corrected code.
> Thanks and regards,
> Harjeet Kumar
Re: Mahout on Hadoop
Posted by Sean Owen <sr...@gmail.com>.
Sounds like a server problem, but I am able to access it now.
In any event, I'm just saying update the code from SVN. That's all.
On Thu, Dec 3, 2009 at 3:25 PM, Rajpal, Harjeet Kumar
<Ha...@honeywell.com> wrote:
> Hi Sean,
> I am unable to access https://issues.apache.org/jira/browse/MAHOUT-143
> It shows could not find server.
> Could you tell me some other way to access the corrected code.
> Thanks and regards,
> Harjeet Kumar
RE: Mahout on Hadoop
Posted by "Rajpal, Harjeet Kumar" <Ha...@Honeywell.com>.
Hi Sean,
I am unable to access https://issues.apache.org/jira/browse/MAHOUT-143
It shows could not find server.
Could you tell me some other way to access the corrected code.
Thanks and regards,
Harjeet Kumar
-----Original Message-----
From: Sean Owen [mailto:srowen@gmail.com]
Sent: Thursday, December 03, 2009 8:17 PM
To: mahout-user@lucene.apache.org
Subject: Re: Mahout on Hadoop
You can try the code again. I updated it to use the old pre-0.20.x
APIs, and should be OK now. It works with 0.20.x though.
On Tue, Dec 1, 2009 at 7:16 AM, Sean Owen <sr...@gmail.com> wrote:
> Yes, same error, really. It expects an object that implements the old
> deprecated APIs, when using the new ones:
> org.apache.hadoop.mapred.Mapper. Unless I am missing something very
> big, yes, 0.20.x seems to have quite a critical problem.
>
> These jobs are the "canaries" from the project testing out using the
> new APIs, and at the moment they don't work. I am in the process of
> trying to resolve this, because Hadoop 0.21 evidently fixes a lot of
> this. You can follow my progress in MAHOUT-143:
>
> https://issues.apache.org/jira/browse/MAHOUT-143
>
> On Tue, Dec 1, 2009 at 6:50 AM, Rajpal, Harjeet Kumar
> <Ha...@honeywell.com> wrote:
>> Hello all,
>> This time when I reconfigured the Hadoop 0.20.1 the error has changed.
>> Now I am using Mahout 0.1. The error is:
>>
>> 09/12/01 12:07:01 INFO mapred.FileInputFormat: Total input paths to process : 1
>> 09/12/01 12:07:04 INFO mapred.JobClient: Running job: job_200912011202_0002
>> 09/12/01 12:07:05 INFO mapred.JobClient: map 0% reduce 0%
>> 09/12/01 12:07:25 INFO mapred.JobClient: Task Id : attempt_200912011202_0002_m_000000_0, Status : FAILED
>> java.lang.RuntimeException: Error in configuring object
>> at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93)
>> at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)
>> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
>> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:354)
>> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
>> at org.apache.hadoop.mapred.Child.main(Child.java:170)
>> Caused by: java.lang.reflect.InvocationTargetException
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>> at java.lang.reflect.Method.invoke(Method.java:597)
>> at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:88)
>> ... 5 more
>> Caused by: java.lang.RuntimeException: java.lang.RuntimeException: class org.apache.mahout.cf.taste.hadoop.RecommenderMapper not org.apache.hadoop.mapred.Mapper
>> at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:840)
>> at org.apache.hadoop.mapred.JobConf.getMapperClass(JobConf.java:771)
>> at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)
>> ... 10 more
>> Caused by: java.lang.RuntimeException: class org.apache.mahout.cf.taste.hadoop.RecommenderMapper not org.apache.hadoop.mapred.Mapper
>> at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:834)
>> ... 12 more
>>
>> 09/12/01 12:07:25 INFO mapred.JobClient: Task Id : attempt_200912011202_0002_m_000001_0, Status : FAILED
>> java.lang.RuntimeException: Error in configuring object
>> at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93)
>> at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)
>> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
>> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:354)
>> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
>> at org.apache.hadoop.mapred.Child.main(Child.java:170)
>>
>> I will Really appreciate if anybody can help.
>>
>> Best Regards,
>> Harjeet Kumar Rajpal
>
Re: Mahout on Hadoop
Posted by Sean Owen <sr...@gmail.com>.
You can try the code again. I updated it to use the old pre-0.20.x
APIs, and should be OK now. It works with 0.20.x though.
On Tue, Dec 1, 2009 at 7:16 AM, Sean Owen <sr...@gmail.com> wrote:
> Yes, same error, really. It expects an object that implements the old
> deprecated APIs, when using the new ones:
> org.apache.hadoop.mapred.Mapper. Unless I am missing something very
> big, yes, 0.20.x seems to have quite a critical problem.
>
> These jobs are the "canaries" from the project testing out using the
> new APIs, and at the moment they don't work. I am in the process of
> trying to resolve this, because Hadoop 0.21 evidently fixes a lot of
> this. You can follow my progress in MAHOUT-143:
>
> https://issues.apache.org/jira/browse/MAHOUT-143
>
> On Tue, Dec 1, 2009 at 6:50 AM, Rajpal, Harjeet Kumar
> <Ha...@honeywell.com> wrote:
>> Hello all,
>> This time when I reconfigured the Hadoop 0.20.1 the error has changed.
>> Now I am using Mahout 0.1. The error is:
>>
>> 09/12/01 12:07:01 INFO mapred.FileInputFormat: Total input paths to process : 1
>> 09/12/01 12:07:04 INFO mapred.JobClient: Running job: job_200912011202_0002
>> 09/12/01 12:07:05 INFO mapred.JobClient: map 0% reduce 0%
>> 09/12/01 12:07:25 INFO mapred.JobClient: Task Id : attempt_200912011202_0002_m_000000_0, Status : FAILED
>> java.lang.RuntimeException: Error in configuring object
>> at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93)
>> at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)
>> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
>> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:354)
>> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
>> at org.apache.hadoop.mapred.Child.main(Child.java:170)
>> Caused by: java.lang.reflect.InvocationTargetException
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>> at java.lang.reflect.Method.invoke(Method.java:597)
>> at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:88)
>> ... 5 more
>> Caused by: java.lang.RuntimeException: java.lang.RuntimeException: class org.apache.mahout.cf.taste.hadoop.RecommenderMapper not org.apache.hadoop.mapred.Mapper
>> at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:840)
>> at org.apache.hadoop.mapred.JobConf.getMapperClass(JobConf.java:771)
>> at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)
>> ... 10 more
>> Caused by: java.lang.RuntimeException: class org.apache.mahout.cf.taste.hadoop.RecommenderMapper not org.apache.hadoop.mapred.Mapper
>> at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:834)
>> ... 12 more
>>
>> 09/12/01 12:07:25 INFO mapred.JobClient: Task Id : attempt_200912011202_0002_m_000001_0, Status : FAILED
>> java.lang.RuntimeException: Error in configuring object
>> at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93)
>> at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)
>> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
>> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:354)
>> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
>> at org.apache.hadoop.mapred.Child.main(Child.java:170)
>>
>> I will Really appreciate if anybody can help.
>>
>> Best Regards,
>> Harjeet Kumar Rajpal
>
Re: Mahout on Hadoop
Posted by Sean Owen <sr...@gmail.com>.
Yes, same error, really. It expects an object that implements the old
deprecated APIs, when using the new ones:
org.apache.hadoop.mapred.Mapper. Unless I am missing something very
big, yes, 0.20.x seems to have quite a critical problem.
These jobs are the "canaries" from the project testing out using the
new APIs, and at the moment they don't work. I am in the process of
trying to resolve this, because Hadoop 0.21 evidently fixes a lot of
this. You can follow my progress in MAHOUT-143:
https://issues.apache.org/jira/browse/MAHOUT-143
On Tue, Dec 1, 2009 at 6:50 AM, Rajpal, Harjeet Kumar
<Ha...@honeywell.com> wrote:
> Hello all,
> This time when I reconfigured the Hadoop 0.20.1 the error has changed.
> Now I am using Mahout 0.1. The error is:
>
> 09/12/01 12:07:01 INFO mapred.FileInputFormat: Total input paths to process : 1
> 09/12/01 12:07:04 INFO mapred.JobClient: Running job: job_200912011202_0002
> 09/12/01 12:07:05 INFO mapred.JobClient: map 0% reduce 0%
> 09/12/01 12:07:25 INFO mapred.JobClient: Task Id : attempt_200912011202_0002_m_000000_0, Status : FAILED
> java.lang.RuntimeException: Error in configuring object
> at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93)
> at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:354)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
> at org.apache.hadoop.mapred.Child.main(Child.java:170)
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:88)
> ... 5 more
> Caused by: java.lang.RuntimeException: java.lang.RuntimeException: class org.apache.mahout.cf.taste.hadoop.RecommenderMapper not org.apache.hadoop.mapred.Mapper
> at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:840)
> at org.apache.hadoop.mapred.JobConf.getMapperClass(JobConf.java:771)
> at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)
> ... 10 more
> Caused by: java.lang.RuntimeException: class org.apache.mahout.cf.taste.hadoop.RecommenderMapper not org.apache.hadoop.mapred.Mapper
> at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:834)
> ... 12 more
>
> 09/12/01 12:07:25 INFO mapred.JobClient: Task Id : attempt_200912011202_0002_m_000001_0, Status : FAILED
> java.lang.RuntimeException: Error in configuring object
> at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93)
> at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:354)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
> at org.apache.hadoop.mapred.Child.main(Child.java:170)
>
> I will Really appreciate if anybody can help.
>
> Best Regards,
> Harjeet Kumar Rajpal