You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@mahout.apache.org by Parimi Rohit <ro...@gmail.com> on 2013/09/10 19:33:59 UTC
running mahout on Hadoop 2.0.0-cdh4.3.1
Hi All,
I am used to running mahout (mahout-core-0.9-SNAPSHOT-job.jar) in the
Apache Hadoop environment, however, we had to switch to Cloudera
distribution.
When I try to run the item based collaborative filtering job
(org.apache.mahout.cf.taste.hadoop.item.RecommenderJob) in the Cloudera
distribution, I get the following error,
Exception in thread "main" java.lang.IncompatibleClassChangeError: Found
interface org.apache.hadoop.mapreduce.JobContext, but class was expected
at
org.apache.mahout.common.HadoopUtil.getCustomJobName(HadoopUtil.java:174)
at
org.apache.mahout.common.AbstractJob.prepareJob(AbstractJob.java:614)
at
org.apache.mahout.cf.taste.hadoop.preparation.PreparePreferenceMatrixJob.run(PreparePreferenceMatrixJob.java:75)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at
org.apache.mahout.cf.taste.hadoop.item.RecommenderJob.run(RecommenderJob.java:158)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at
org.apache.mahout.cf.taste.hadoop.item.RecommenderJob.main(RecommenderJob.java:312)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.main(RunJar.java:208)
Is there a way to run Mahout in the Cloudera environment? I mean, a
download specific to Cloudera distribution of Hadoop?
Thanks in advance,
Rohit
Re: running mahout on Hadoop 2.0.0-cdh4.3.1
Posted by Parimi Rohit <ro...@gmail.com>.
Hi Barrie,
We were not able to solve this issue. Hence we decided to install Apache
hadoop 1.04 version.
Thanks,
Rohit
On Wed, Sep 18, 2013 at 8:32 AM, B Kersbergen <ke...@gmail.com> wrote:
> Hi Rohit
>
> I also run into the IncompatibleClassChangeError when using the streaming k
> means feature. Apparently this only applies to some parts of the code base
> because I have not run into this issue with other distributed
> functionalities of mahout.
> Were you able to solve this? If so, what were the specific parameters that
> you used to build mahout?
>
> I'm using CDH4.3.0-mr1 client and cluster.
> Kind regards,
> Barrie
>
>
> 2013/9/12 Gokhan Capan <gk...@gmail.com>
>
> > You might also need to build mahout using hadoop-0.23 profile with
> > hadoop.version parameter set to your hadoop version.
> >
> > Gokhan
> >
> >
> > On Tue, Sep 10, 2013 at 10:09 PM, Parimi Rohit <rohit.parimi@gmail.com
> > >wrote:
> >
> > > Thanks Sean. Will look into that.
> > >
> > > Rohit
> > >
> > >
> > > On Tue, Sep 10, 2013 at 1:32 PM, Sean Owen <sr...@gmail.com> wrote:
> > >
> > > > You are trying to run on Hadoop 2 and Mahout only works with Hadoop 1
> > and
> > > > related branches. This wont work.
> > > >
> > > > However the CDH distributions also come in an 'mr1' flavor that
> stands
> > a
> > > > much better chance of working with something that is built for Hadoop
> > 1.
> > > > Use 2.0.0-mr1-4.3.1 instead. (PS 4.3.2 and 4.4.0 are available now)
> > > >
> > > > You will likely still have to compile Mahout again with this
> different
> > > > dependency to get it to work but with any luck that's it.
> > > >
> > > > Sean
> > > > On Sep 10, 2013 6:34 PM, "Parimi Rohit" <ro...@gmail.com>
> > wrote:
> > > >
> > > > > Hi All,
> > > > >
> > > > > I am used to running mahout (mahout-core-0.9-SNAPSHOT-job.jar) in
> the
> > > > > Apache Hadoop environment, however, we had to switch to Cloudera
> > > > > distribution.
> > > > >
> > > > > When I try to run the item based collaborative filtering job
> > > > > (org.apache.mahout.cf.taste.hadoop.item.RecommenderJob) in the
> > Cloudera
> > > > > distribution, I get the following error,
> > > > >
> > > > > Exception in thread "main" java.lang.IncompatibleClassChangeError:
> > > Found
> > > > > interface org.apache.hadoop.mapreduce.JobContext, but class was
> > > expected
> > > > > at
> > > > >
> > >
> org.apache.mahout.common.HadoopUtil.getCustomJobName(HadoopUtil.java:174)
> > > > > at
> > > > >
> org.apache.mahout.common.AbstractJob.prepareJob(AbstractJob.java:614)
> > > > > at
> > > > >
> > > > >
> > > >
> > >
> >
> org.apache.mahout.cf.taste.hadoop.preparation.PreparePreferenceMatrixJob.run(PreparePreferenceMatrixJob.java:75)
> > > > > at
> org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> > > > > at
> > > > >
> > > > >
> > > >
> > >
> >
> org.apache.mahout.cf.taste.hadoop.item.RecommenderJob.run(RecommenderJob.java:158)
> > > > > at
> org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> > > > > at
> > > > >
> > > > >
> > > >
> > >
> >
> org.apache.mahout.cf.taste.hadoop.item.RecommenderJob.main(RecommenderJob.java:312)
> > > > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
> > Method)
> > > > > at
> > > > >
> > > > >
> > > >
> > >
> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> > > > > at
> > > > >
> > > > >
> > > >
> > >
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> > > > > at java.lang.reflect.Method.invoke(Method.java:606)
> > > > > at org.apache.hadoop.util.RunJar.main(RunJar.java:208)
> > > > >
> > > > > Is there a way to run Mahout in the Cloudera environment? I mean, a
> > > > > download specific to Cloudera distribution of Hadoop?
> > > > >
> > > > > Thanks in advance,
> > > > > Rohit
> > > > >
> > > >
> > >
> >
>
Re: running mahout on Hadoop 2.0.0-cdh4.3.1
Posted by B Kersbergen <ke...@gmail.com>.
Hi Rohit
I also run into the IncompatibleClassChangeError when using the streaming k
means feature. Apparently this only applies to some parts of the code base
because I have not run into this issue with other distributed
functionalities of mahout.
Were you able to solve this? If so, what were the specific parameters that
you used to build mahout?
I'm using CDH4.3.0-mr1 client and cluster.
Kind regards,
Barrie
2013/9/12 Gokhan Capan <gk...@gmail.com>
> You might also need to build mahout using hadoop-0.23 profile with
> hadoop.version parameter set to your hadoop version.
>
> Gokhan
>
>
> On Tue, Sep 10, 2013 at 10:09 PM, Parimi Rohit <rohit.parimi@gmail.com
> >wrote:
>
> > Thanks Sean. Will look into that.
> >
> > Rohit
> >
> >
> > On Tue, Sep 10, 2013 at 1:32 PM, Sean Owen <sr...@gmail.com> wrote:
> >
> > > You are trying to run on Hadoop 2 and Mahout only works with Hadoop 1
> and
> > > related branches. This wont work.
> > >
> > > However the CDH distributions also come in an 'mr1' flavor that stands
> a
> > > much better chance of working with something that is built for Hadoop
> 1.
> > > Use 2.0.0-mr1-4.3.1 instead. (PS 4.3.2 and 4.4.0 are available now)
> > >
> > > You will likely still have to compile Mahout again with this different
> > > dependency to get it to work but with any luck that's it.
> > >
> > > Sean
> > > On Sep 10, 2013 6:34 PM, "Parimi Rohit" <ro...@gmail.com>
> wrote:
> > >
> > > > Hi All,
> > > >
> > > > I am used to running mahout (mahout-core-0.9-SNAPSHOT-job.jar) in the
> > > > Apache Hadoop environment, however, we had to switch to Cloudera
> > > > distribution.
> > > >
> > > > When I try to run the item based collaborative filtering job
> > > > (org.apache.mahout.cf.taste.hadoop.item.RecommenderJob) in the
> Cloudera
> > > > distribution, I get the following error,
> > > >
> > > > Exception in thread "main" java.lang.IncompatibleClassChangeError:
> > Found
> > > > interface org.apache.hadoop.mapreduce.JobContext, but class was
> > expected
> > > > at
> > > >
> > org.apache.mahout.common.HadoopUtil.getCustomJobName(HadoopUtil.java:174)
> > > > at
> > > > org.apache.mahout.common.AbstractJob.prepareJob(AbstractJob.java:614)
> > > > at
> > > >
> > > >
> > >
> >
> org.apache.mahout.cf.taste.hadoop.preparation.PreparePreferenceMatrixJob.run(PreparePreferenceMatrixJob.java:75)
> > > > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> > > > at
> > > >
> > > >
> > >
> >
> org.apache.mahout.cf.taste.hadoop.item.RecommenderJob.run(RecommenderJob.java:158)
> > > > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> > > > at
> > > >
> > > >
> > >
> >
> org.apache.mahout.cf.taste.hadoop.item.RecommenderJob.main(RecommenderJob.java:312)
> > > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
> > > > at
> > > >
> > > >
> > >
> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> > > > at
> > > >
> > > >
> > >
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> > > > at java.lang.reflect.Method.invoke(Method.java:606)
> > > > at org.apache.hadoop.util.RunJar.main(RunJar.java:208)
> > > >
> > > > Is there a way to run Mahout in the Cloudera environment? I mean, a
> > > > download specific to Cloudera distribution of Hadoop?
> > > >
> > > > Thanks in advance,
> > > > Rohit
> > > >
> > >
> >
>
Re: running mahout on Hadoop 2.0.0-cdh4.3.1
Posted by Gokhan Capan <gk...@gmail.com>.
You might also need to build mahout using hadoop-0.23 profile with
hadoop.version parameter set to your hadoop version.
Gokhan
On Tue, Sep 10, 2013 at 10:09 PM, Parimi Rohit <ro...@gmail.com>wrote:
> Thanks Sean. Will look into that.
>
> Rohit
>
>
> On Tue, Sep 10, 2013 at 1:32 PM, Sean Owen <sr...@gmail.com> wrote:
>
> > You are trying to run on Hadoop 2 and Mahout only works with Hadoop 1 and
> > related branches. This wont work.
> >
> > However the CDH distributions also come in an 'mr1' flavor that stands a
> > much better chance of working with something that is built for Hadoop 1.
> > Use 2.0.0-mr1-4.3.1 instead. (PS 4.3.2 and 4.4.0 are available now)
> >
> > You will likely still have to compile Mahout again with this different
> > dependency to get it to work but with any luck that's it.
> >
> > Sean
> > On Sep 10, 2013 6:34 PM, "Parimi Rohit" <ro...@gmail.com> wrote:
> >
> > > Hi All,
> > >
> > > I am used to running mahout (mahout-core-0.9-SNAPSHOT-job.jar) in the
> > > Apache Hadoop environment, however, we had to switch to Cloudera
> > > distribution.
> > >
> > > When I try to run the item based collaborative filtering job
> > > (org.apache.mahout.cf.taste.hadoop.item.RecommenderJob) in the Cloudera
> > > distribution, I get the following error,
> > >
> > > Exception in thread "main" java.lang.IncompatibleClassChangeError:
> Found
> > > interface org.apache.hadoop.mapreduce.JobContext, but class was
> expected
> > > at
> > >
> org.apache.mahout.common.HadoopUtil.getCustomJobName(HadoopUtil.java:174)
> > > at
> > > org.apache.mahout.common.AbstractJob.prepareJob(AbstractJob.java:614)
> > > at
> > >
> > >
> >
> org.apache.mahout.cf.taste.hadoop.preparation.PreparePreferenceMatrixJob.run(PreparePreferenceMatrixJob.java:75)
> > > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> > > at
> > >
> > >
> >
> org.apache.mahout.cf.taste.hadoop.item.RecommenderJob.run(RecommenderJob.java:158)
> > > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> > > at
> > >
> > >
> >
> org.apache.mahout.cf.taste.hadoop.item.RecommenderJob.main(RecommenderJob.java:312)
> > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > > at
> > >
> > >
> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> > > at
> > >
> > >
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> > > at java.lang.reflect.Method.invoke(Method.java:606)
> > > at org.apache.hadoop.util.RunJar.main(RunJar.java:208)
> > >
> > > Is there a way to run Mahout in the Cloudera environment? I mean, a
> > > download specific to Cloudera distribution of Hadoop?
> > >
> > > Thanks in advance,
> > > Rohit
> > >
> >
>
Re: running mahout on Hadoop 2.0.0-cdh4.3.1
Posted by Parimi Rohit <ro...@gmail.com>.
Thanks Sean. Will look into that.
Rohit
On Tue, Sep 10, 2013 at 1:32 PM, Sean Owen <sr...@gmail.com> wrote:
> You are trying to run on Hadoop 2 and Mahout only works with Hadoop 1 and
> related branches. This wont work.
>
> However the CDH distributions also come in an 'mr1' flavor that stands a
> much better chance of working with something that is built for Hadoop 1.
> Use 2.0.0-mr1-4.3.1 instead. (PS 4.3.2 and 4.4.0 are available now)
>
> You will likely still have to compile Mahout again with this different
> dependency to get it to work but with any luck that's it.
>
> Sean
> On Sep 10, 2013 6:34 PM, "Parimi Rohit" <ro...@gmail.com> wrote:
>
> > Hi All,
> >
> > I am used to running mahout (mahout-core-0.9-SNAPSHOT-job.jar) in the
> > Apache Hadoop environment, however, we had to switch to Cloudera
> > distribution.
> >
> > When I try to run the item based collaborative filtering job
> > (org.apache.mahout.cf.taste.hadoop.item.RecommenderJob) in the Cloudera
> > distribution, I get the following error,
> >
> > Exception in thread "main" java.lang.IncompatibleClassChangeError: Found
> > interface org.apache.hadoop.mapreduce.JobContext, but class was expected
> > at
> > org.apache.mahout.common.HadoopUtil.getCustomJobName(HadoopUtil.java:174)
> > at
> > org.apache.mahout.common.AbstractJob.prepareJob(AbstractJob.java:614)
> > at
> >
> >
> org.apache.mahout.cf.taste.hadoop.preparation.PreparePreferenceMatrixJob.run(PreparePreferenceMatrixJob.java:75)
> > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> > at
> >
> >
> org.apache.mahout.cf.taste.hadoop.item.RecommenderJob.run(RecommenderJob.java:158)
> > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> > at
> >
> >
> org.apache.mahout.cf.taste.hadoop.item.RecommenderJob.main(RecommenderJob.java:312)
> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > at
> >
> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> > at
> >
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> > at java.lang.reflect.Method.invoke(Method.java:606)
> > at org.apache.hadoop.util.RunJar.main(RunJar.java:208)
> >
> > Is there a way to run Mahout in the Cloudera environment? I mean, a
> > download specific to Cloudera distribution of Hadoop?
> >
> > Thanks in advance,
> > Rohit
> >
>
Re: running mahout on Hadoop 2.0.0-cdh4.3.1
Posted by Sean Owen <sr...@gmail.com>.
You are trying to run on Hadoop 2 and Mahout only works with Hadoop 1 and
related branches. This wont work.
However the CDH distributions also come in an 'mr1' flavor that stands a
much better chance of working with something that is built for Hadoop 1.
Use 2.0.0-mr1-4.3.1 instead. (PS 4.3.2 and 4.4.0 are available now)
You will likely still have to compile Mahout again with this different
dependency to get it to work but with any luck that's it.
Sean
On Sep 10, 2013 6:34 PM, "Parimi Rohit" <ro...@gmail.com> wrote:
> Hi All,
>
> I am used to running mahout (mahout-core-0.9-SNAPSHOT-job.jar) in the
> Apache Hadoop environment, however, we had to switch to Cloudera
> distribution.
>
> When I try to run the item based collaborative filtering job
> (org.apache.mahout.cf.taste.hadoop.item.RecommenderJob) in the Cloudera
> distribution, I get the following error,
>
> Exception in thread "main" java.lang.IncompatibleClassChangeError: Found
> interface org.apache.hadoop.mapreduce.JobContext, but class was expected
> at
> org.apache.mahout.common.HadoopUtil.getCustomJobName(HadoopUtil.java:174)
> at
> org.apache.mahout.common.AbstractJob.prepareJob(AbstractJob.java:614)
> at
>
> org.apache.mahout.cf.taste.hadoop.preparation.PreparePreferenceMatrixJob.run(PreparePreferenceMatrixJob.java:75)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> at
>
> org.apache.mahout.cf.taste.hadoop.item.RecommenderJob.run(RecommenderJob.java:158)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> at
>
> org.apache.mahout.cf.taste.hadoop.item.RecommenderJob.main(RecommenderJob.java:312)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
>
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:208)
>
> Is there a way to run Mahout in the Cloudera environment? I mean, a
> download specific to Cloudera distribution of Hadoop?
>
> Thanks in advance,
> Rohit
>