You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@mahout.apache.org by Jian Fang <ji...@gmail.com> on 2013/03/13 20:58:25 UTC
Mahout and Hadoop 2
Hi,
Is there anyway to make mahout 0.7 or 0.8 work with Hadoop 2.0.2-alpha?
Seems Mahout builds against Hadoop 1.X by default in the pom.xml and it
also requires hadoop-core.jar, which only exists in Hadoop 1.x if I
remember correctly.
Thanks,
Jian
Re: Mahout and Hadoop 2
Posted by Jian Fang <ji...@gmail.com>.
Thanks. I commented out hadoop-core, but it threw errors such as
'dependencies.dependency.version' for org.apache.hadoop:hadoop-core:jar is
missing
Do you know where I can find the profile?
Thanks,
Jian
On Wed, Mar 13, 2013 at 1:13 PM, Sean Owen <sr...@gmail.com> wrote:
> I think someone submitted a different build profile that changes the
> dependencies for you. I believe the issue is using "hadoop-common" and not
> "hadoop-core" as well as changing versions. I think the rest is compile
> compatible and probably runtime compatible. But I've not tried.
>
>
> On Wed, Mar 13, 2013 at 7:58 PM, Jian Fang <jian.fang.subscribe@gmail.com
> >wrote:
>
> > Hi,
> >
> > Is there anyway to make mahout 0.7 or 0.8 work with Hadoop 2.0.2-alpha?
> >
> > Seems Mahout builds against Hadoop 1.X by default in the pom.xml and it
> > also requires hadoop-core.jar, which only exists in Hadoop 1.x if I
> > remember correctly.
> >
> > Thanks,
> >
> > Jian
> >
>
Re: Mahout and Hadoop 2
Posted by Sean Owen <sr...@gmail.com>.
I think someone submitted a different build profile that changes the
dependencies for you. I believe the issue is using "hadoop-common" and not
"hadoop-core" as well as changing versions. I think the rest is compile
compatible and probably runtime compatible. But I've not tried.
On Wed, Mar 13, 2013 at 7:58 PM, Jian Fang <ji...@gmail.com>wrote:
> Hi,
>
> Is there anyway to make mahout 0.7 or 0.8 work with Hadoop 2.0.2-alpha?
>
> Seems Mahout builds against Hadoop 1.X by default in the pom.xml and it
> also requires hadoop-core.jar, which only exists in Hadoop 1.x if I
> remember correctly.
>
> Thanks,
>
> Jian
>
Re: Mahout and Hadoop 2
Posted by Jian Fang <ji...@gmail.com>.
I used the following command and seems it worked.
mvn -Phadoop-0.23 clean install -DskipTests -Dhadoop.version=2.0.2-alpha
-Dmahout.skip.distribution=false
On Wed, Mar 13, 2013 at 1:38 PM, Jian Fang <ji...@gmail.com>wrote:
> No, it won't because of missing method in class.
>
> I guess the profile is hadoop-0.23 in core/pom.xml. Will try that.
>
> Thanks,
>
> Jian
>
>
> On Wed, Mar 13, 2013 at 1:35 PM, Dmitriy Lyubimov <dl...@gmail.com>wrote:
>
>> as far as i understand, you can rebuild as is then run it with 2.0.0 at
>> runtime, it should stay compatible. (unless it doesn't :)
>>
>>
>> On Wed, Mar 13, 2013 at 1:26 PM, Jian Fang <jian.fang.subscribe@gmail.com
>> >wrote:
>>
>> > Thanks., but we want to build from original Apache hadoop, not cdh.
>> >
>> > On Wed, Mar 13, 2013 at 1:23 PM, Zheyi RONG <ro...@gmail.com>
>> wrote:
>> >
>> > > Hi,
>> > >
>> > > I can run mahout 0.7 with both command line and Java Api, on
>> cloudera4,
>> > > i.e. hadoop 2.0.0-mr1-cdh4.1.2
>> > >
>> > > Zheyi
>> > >
>> > > On Wed, Mar 13, 2013 at 8:58 PM, Jian Fang <
>> > jian.fang.subscribe@gmail.com
>> > > >wrote:
>> > >
>> > > > Hi,
>> > > >
>> > > > Is there anyway to make mahout 0.7 or 0.8 work with Hadoop
>> 2.0.2-alpha?
>> > > >
>> > > > Seems Mahout builds against Hadoop 1.X by default in the pom.xml
>> and it
>> > > > also requires hadoop-core.jar, which only exists in Hadoop 1.x if I
>> > > > remember correctly.
>> > > >
>> > > > Thanks,
>> > > >
>> > > > Jian
>> > > >
>> > >
>> >
>>
>
>
Re: Mahout and Hadoop 2
Posted by Jian Fang <ji...@gmail.com>.
No, it won't because of missing method in class.
I guess the profile is hadoop-0.23 in core/pom.xml. Will try that.
Thanks,
Jian
On Wed, Mar 13, 2013 at 1:35 PM, Dmitriy Lyubimov <dl...@gmail.com> wrote:
> as far as i understand, you can rebuild as is then run it with 2.0.0 at
> runtime, it should stay compatible. (unless it doesn't :)
>
>
> On Wed, Mar 13, 2013 at 1:26 PM, Jian Fang <jian.fang.subscribe@gmail.com
> >wrote:
>
> > Thanks., but we want to build from original Apache hadoop, not cdh.
> >
> > On Wed, Mar 13, 2013 at 1:23 PM, Zheyi RONG <ro...@gmail.com> wrote:
> >
> > > Hi,
> > >
> > > I can run mahout 0.7 with both command line and Java Api, on cloudera4,
> > > i.e. hadoop 2.0.0-mr1-cdh4.1.2
> > >
> > > Zheyi
> > >
> > > On Wed, Mar 13, 2013 at 8:58 PM, Jian Fang <
> > jian.fang.subscribe@gmail.com
> > > >wrote:
> > >
> > > > Hi,
> > > >
> > > > Is there anyway to make mahout 0.7 or 0.8 work with Hadoop
> 2.0.2-alpha?
> > > >
> > > > Seems Mahout builds against Hadoop 1.X by default in the pom.xml and
> it
> > > > also requires hadoop-core.jar, which only exists in Hadoop 1.x if I
> > > > remember correctly.
> > > >
> > > > Thanks,
> > > >
> > > > Jian
> > > >
> > >
> >
>
Re: Mahout and Hadoop 2
Posted by Dmitriy Lyubimov <dl...@gmail.com>.
as far as i understand, you can rebuild as is then run it with 2.0.0 at
runtime, it should stay compatible. (unless it doesn't :)
On Wed, Mar 13, 2013 at 1:26 PM, Jian Fang <ji...@gmail.com>wrote:
> Thanks., but we want to build from original Apache hadoop, not cdh.
>
> On Wed, Mar 13, 2013 at 1:23 PM, Zheyi RONG <ro...@gmail.com> wrote:
>
> > Hi,
> >
> > I can run mahout 0.7 with both command line and Java Api, on cloudera4,
> > i.e. hadoop 2.0.0-mr1-cdh4.1.2
> >
> > Zheyi
> >
> > On Wed, Mar 13, 2013 at 8:58 PM, Jian Fang <
> jian.fang.subscribe@gmail.com
> > >wrote:
> >
> > > Hi,
> > >
> > > Is there anyway to make mahout 0.7 or 0.8 work with Hadoop 2.0.2-alpha?
> > >
> > > Seems Mahout builds against Hadoop 1.X by default in the pom.xml and it
> > > also requires hadoop-core.jar, which only exists in Hadoop 1.x if I
> > > remember correctly.
> > >
> > > Thanks,
> > >
> > > Jian
> > >
> >
>
Re: Mahout and Hadoop 2
Posted by Jian Fang <ji...@gmail.com>.
Thanks., but we want to build from original Apache hadoop, not cdh.
On Wed, Mar 13, 2013 at 1:23 PM, Zheyi RONG <ro...@gmail.com> wrote:
> Hi,
>
> I can run mahout 0.7 with both command line and Java Api, on cloudera4,
> i.e. hadoop 2.0.0-mr1-cdh4.1.2
>
> Zheyi
>
> On Wed, Mar 13, 2013 at 8:58 PM, Jian Fang <jian.fang.subscribe@gmail.com
> >wrote:
>
> > Hi,
> >
> > Is there anyway to make mahout 0.7 or 0.8 work with Hadoop 2.0.2-alpha?
> >
> > Seems Mahout builds against Hadoop 1.X by default in the pom.xml and it
> > also requires hadoop-core.jar, which only exists in Hadoop 1.x if I
> > remember correctly.
> >
> > Thanks,
> >
> > Jian
> >
>
Re: Mahout and Hadoop 2
Posted by Zheyi RONG <ro...@gmail.com>.
Hi,
I can run mahout 0.7 with both command line and Java Api, on cloudera4,
i.e. hadoop 2.0.0-mr1-cdh4.1.2
Zheyi
On Wed, Mar 13, 2013 at 8:58 PM, Jian Fang <ji...@gmail.com>wrote:
> Hi,
>
> Is there anyway to make mahout 0.7 or 0.8 work with Hadoop 2.0.2-alpha?
>
> Seems Mahout builds against Hadoop 1.X by default in the pom.xml and it
> also requires hadoop-core.jar, which only exists in Hadoop 1.x if I
> remember correctly.
>
> Thanks,
>
> Jian
>