You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@mahout.apache.org by gaurav singh <ga...@gmail.com> on 2012/01/21 18:54:28 UTC
Mahout and Hadoop on Windows
Hi Everyone,
I am fairly new to mahout and hadoop, I was trying to run sequential
algorithms of mahout from the source. I have checked out the code in
Eclipse on windows but it seems even the sequential algorithms in mahout
can not work without installing and setting up hadoop(I was trying to run
the sequential FPGrowth ). I don't have hadoop installed on windows and
would love to ask if its difficult to setup hadoop on windows than its on
Linux. I also had to include a lot of external JARs(packages) before the
mahout code became error free and stopped showing couldn't resolve this
class errors, like
hadoop-0.19.1-core.jar,com.google.common.source_1.0.0.201004262004.jar etc
to resolve many of the imported packages like
com.google.common.io.Closeable etc. Did I do the right thing? Please any
detailed light on its functioning would be great.
Thanks everyone!
--
regards
Gaurav Singh
RE: Mahout and Hadoop on Windows
Posted by "Sontakke, Sushilkumar" <Su...@bmc.com>.
I have hadoop-0.20.203 installed on ubuntu virtual machine
@ HADOOP_INSTALL directory /usr/local/hadoop1
My temp file log gets stored @ /app/hadoop/tmp directory
How do I configure eclipse Indigo to access this virtual machine
-----Original Message-----
From: Lance Norskog [mailto:goksron@gmail.com]
Sent: Sunday, January 22, 2012 4:53 AM
To: user@mahout.apache.org
Subject: Re: Mahout and Hadoop on Windows
1) You do not technically need a separate Hadoop installation. Mahout
has its own mini-installation that works fine.
2) Running Mahout on Windows requires a) Cygwin and b) a lot of Unix
sysadmin expertise.
You are much better off using Linux in a virtual machine.
On Sat, Jan 21, 2012 at 11:00 AM, Sontakke, Sushilkumar
<Su...@bmc.com> wrote:
> Hello everyone. EVEN I am working with same . I would really be grateful if someone gives a proper tutorial and do we need to setup the mapreduce environment/perspective in eclipse to work with mahout ?? The tutorial links on mahout site for hadoop installation are for old versions like 0.17.X and 0.19.1 and the requirement page of mahout contradicts by saying that the mahout works with hadoop's 0.20.x or higher version
>
> THANK YOU
>
> -----Original Message-----
> From: Raphael Cendrillon [mailto:cendrillon1978@gmail.com]
> Sent: Saturday, January 21, 2012 11:39 PM
> To: user@mahout.apache.org
> Subject: Re: Mahout and Hadoop on Windows
>
> Hi Gaurav,
>
> If you just want to run in standalone mode in Eclipse then it's quite straightforward.
>
> Install maven, then use 'mvn eclipse:eclipse' in the mahout trunk directory.
>
> It will then download the jars it needs and you can use 'import existing project' in Eclipse to import mahout-core, mahout examples etc.
>
> One issue is that maven will download hadoop version 0.20.204 which has issues with windows as it tries to call chmod. You can overcome this by downloading Hadoop 0.20.202 which doesn't have this bug.
>
> This may get you by for some lite development.
>
> Another option you could consider is EC2.
>
> On Jan 21, 2012, at 9:54 AM, gaurav singh <ga...@gmail.com> wrote:
>
>> Hi Everyone,
>>
>> I am fairly new to mahout and hadoop, I was trying to run sequential
>> algorithms of mahout from the source. I have checked out the code in
>> Eclipse on windows but it seems even the sequential algorithms in mahout
>> can not work without installing and setting up hadoop(I was trying to run
>> the sequential FPGrowth ). I don't have hadoop installed on windows and
>> would love to ask if its difficult to setup hadoop on windows than its on
>> Linux. I also had to include a lot of external JARs(packages) before the
>> mahout code became error free and stopped showing couldn't resolve this
>> class errors, like
>> hadoop-0.19.1-core.jar,com.google.common.source_1.0.0.201004262004.jar etc
>> to resolve many of the imported packages like
>> com.google.common.io.Closeable etc. Did I do the right thing? Please any
>> detailed light on its functioning would be great.
>>
>> Thanks everyone!
>>
>> --
>> regards
>> Gaurav Singh
>
--
Lance Norskog
goksron@gmail.com
Re: Mahout and Hadoop on Windows
Posted by Lance Norskog <go...@gmail.com>.
1) You do not technically need a separate Hadoop installation. Mahout
has its own mini-installation that works fine.
2) Running Mahout on Windows requires a) Cygwin and b) a lot of Unix
sysadmin expertise.
You are much better off using Linux in a virtual machine.
On Sat, Jan 21, 2012 at 11:00 AM, Sontakke, Sushilkumar
<Su...@bmc.com> wrote:
> Hello everyone. EVEN I am working with same . I would really be grateful if someone gives a proper tutorial and do we need to setup the mapreduce environment/perspective in eclipse to work with mahout ?? The tutorial links on mahout site for hadoop installation are for old versions like 0.17.X and 0.19.1 and the requirement page of mahout contradicts by saying that the mahout works with hadoop's 0.20.x or higher version
>
> THANK YOU
>
> -----Original Message-----
> From: Raphael Cendrillon [mailto:cendrillon1978@gmail.com]
> Sent: Saturday, January 21, 2012 11:39 PM
> To: user@mahout.apache.org
> Subject: Re: Mahout and Hadoop on Windows
>
> Hi Gaurav,
>
> If you just want to run in standalone mode in Eclipse then it's quite straightforward.
>
> Install maven, then use 'mvn eclipse:eclipse' in the mahout trunk directory.
>
> It will then download the jars it needs and you can use 'import existing project' in Eclipse to import mahout-core, mahout examples etc.
>
> One issue is that maven will download hadoop version 0.20.204 which has issues with windows as it tries to call chmod. You can overcome this by downloading Hadoop 0.20.202 which doesn't have this bug.
>
> This may get you by for some lite development.
>
> Another option you could consider is EC2.
>
> On Jan 21, 2012, at 9:54 AM, gaurav singh <ga...@gmail.com> wrote:
>
>> Hi Everyone,
>>
>> I am fairly new to mahout and hadoop, I was trying to run sequential
>> algorithms of mahout from the source. I have checked out the code in
>> Eclipse on windows but it seems even the sequential algorithms in mahout
>> can not work without installing and setting up hadoop(I was trying to run
>> the sequential FPGrowth ). I don't have hadoop installed on windows and
>> would love to ask if its difficult to setup hadoop on windows than its on
>> Linux. I also had to include a lot of external JARs(packages) before the
>> mahout code became error free and stopped showing couldn't resolve this
>> class errors, like
>> hadoop-0.19.1-core.jar,com.google.common.source_1.0.0.201004262004.jar etc
>> to resolve many of the imported packages like
>> com.google.common.io.Closeable etc. Did I do the right thing? Please any
>> detailed light on its functioning would be great.
>>
>> Thanks everyone!
>>
>> --
>> regards
>> Gaurav Singh
>
--
Lance Norskog
goksron@gmail.com
RE: Mahout and Hadoop on Windows
Posted by "Sontakke, Sushilkumar" <Su...@bmc.com>.
Hello everyone. EVEN I am working with same . I would really be grateful if someone gives a proper tutorial and do we need to setup the mapreduce environment/perspective in eclipse to work with mahout ?? The tutorial links on mahout site for hadoop installation are for old versions like 0.17.X and 0.19.1 and the requirement page of mahout contradicts by saying that the mahout works with hadoop's 0.20.x or higher version
THANK YOU
-----Original Message-----
From: Raphael Cendrillon [mailto:cendrillon1978@gmail.com]
Sent: Saturday, January 21, 2012 11:39 PM
To: user@mahout.apache.org
Subject: Re: Mahout and Hadoop on Windows
Hi Gaurav,
If you just want to run in standalone mode in Eclipse then it's quite straightforward.
Install maven, then use 'mvn eclipse:eclipse' in the mahout trunk directory.
It will then download the jars it needs and you can use 'import existing project' in Eclipse to import mahout-core, mahout examples etc.
One issue is that maven will download hadoop version 0.20.204 which has issues with windows as it tries to call chmod. You can overcome this by downloading Hadoop 0.20.202 which doesn't have this bug.
This may get you by for some lite development.
Another option you could consider is EC2.
On Jan 21, 2012, at 9:54 AM, gaurav singh <ga...@gmail.com> wrote:
> Hi Everyone,
>
> I am fairly new to mahout and hadoop, I was trying to run sequential
> algorithms of mahout from the source. I have checked out the code in
> Eclipse on windows but it seems even the sequential algorithms in mahout
> can not work without installing and setting up hadoop(I was trying to run
> the sequential FPGrowth ). I don't have hadoop installed on windows and
> would love to ask if its difficult to setup hadoop on windows than its on
> Linux. I also had to include a lot of external JARs(packages) before the
> mahout code became error free and stopped showing couldn't resolve this
> class errors, like
> hadoop-0.19.1-core.jar,com.google.common.source_1.0.0.201004262004.jar etc
> to resolve many of the imported packages like
> com.google.common.io.Closeable etc. Did I do the right thing? Please any
> detailed light on its functioning would be great.
>
> Thanks everyone!
>
> --
> regards
> Gaurav Singh
Re: Mahout and Hadoop on Windows
Posted by Raphael Cendrillon <ce...@gmail.com>.
Hi Gaurav,
If you just want to run in standalone mode in Eclipse then it's quite straightforward.
Install maven, then use 'mvn eclipse:eclipse' in the mahout trunk directory.
It will then download the jars it needs and you can use 'import existing project' in Eclipse to import mahout-core, mahout examples etc.
One issue is that maven will download hadoop version 0.20.204 which has issues with windows as it tries to call chmod. You can overcome this by downloading Hadoop 0.20.202 which doesn't have this bug.
This may get you by for some lite development.
Another option you could consider is EC2.
On Jan 21, 2012, at 9:54 AM, gaurav singh <ga...@gmail.com> wrote:
> Hi Everyone,
>
> I am fairly new to mahout and hadoop, I was trying to run sequential
> algorithms of mahout from the source. I have checked out the code in
> Eclipse on windows but it seems even the sequential algorithms in mahout
> can not work without installing and setting up hadoop(I was trying to run
> the sequential FPGrowth ). I don't have hadoop installed on windows and
> would love to ask if its difficult to setup hadoop on windows than its on
> Linux. I also had to include a lot of external JARs(packages) before the
> mahout code became error free and stopped showing couldn't resolve this
> class errors, like
> hadoop-0.19.1-core.jar,com.google.common.source_1.0.0.201004262004.jar etc
> to resolve many of the imported packages like
> com.google.common.io.Closeable etc. Did I do the right thing? Please any
> detailed light on its functioning would be great.
>
> Thanks everyone!
>
> --
> regards
> Gaurav Singh