hadoop mr cluster mode on my laptop?

[Pedro Costa <ps...@gmail.com>]

Hi,

1 - I would like to run Hadoop MR in my laptop, but with the cluster
mode configuration. I've put a slave file with the following content:
[code]
127.0.0.1
localhost
192.168.10.1
mylaptop
[/code]


the 192.168.10.1 is the IP of the machine and the "mylaptop" is the
logical name of the address. Is this a well configured slaves file to
do what I want?

2 - I was also thinking taking advantage of my CPU to put Hadoop MR as
cluster mode.
This is my CPU characteristics:

Essentials
Processor Number i5-460M
# of Cores 	        2
# of Threads 	4
Clock Speed      2.53 GHz
Max Turbo Frequency 2.8 GHz
Intel® Smart Cache 3 MB
Instruction Set   64-bit

I was thinking in running a Tasktracker in each Core, although I don't
know how to do it.

Any help to install Hadoop MR in cluster mode on my laptop?

Thanks,
-- 
Pedro

Re: hadoop mr cluster mode on my laptop?

[Niels Basjes <Ni...@basjes.nl>]

Hi,

You should be doing the setup for what is called "Pseudo-distributed" mode.
Have a look at this:
http://hadoop.apache.org/common/docs/r0.20.2/quickstart.html#PseudoDistributed

Niels

2011/4/18 Pedro Costa <ps...@gmail.com>:
> Hi,
>
> 1 - I would like to run Hadoop MR in my laptop, but with the cluster
> mode configuration. I've put a slave file with the following content:
> [code]
> 127.0.0.1
> localhost
> 192.168.10.1
> mylaptop
> [/code]
>
>
> the 192.168.10.1 is the IP of the machine and the "mylaptop" is the
> logical name of the address. Is this a well configured slaves file to
> do what I want?
>
> 2 - I was also thinking taking advantage of my CPU to put Hadoop MR as
> cluster mode.
> This is my CPU characteristics:
>
> Essentials
> Processor Number i5-460M
> # of Cores              2
> # of Threads    4
> Clock Speed      2.53 GHz
> Max Turbo Frequency 2.8 GHz
> Intel® Smart Cache 3 MB
> Instruction Set   64-bit
>
> I was thinking in running a Tasktracker in each Core, although I don't
> know how to do it.
>
> Any help to install Hadoop MR in cluster mode on my laptop?
>
> Thanks,
> --
> Pedro
>



-- 
Met vriendelijke groeten,

Niels Basjes

Re: hadoop mr cluster mode on my laptop?

[Harsh J <ha...@cloudera.com>]

Hello Pedro,

On Mon, Apr 18, 2011 at 6:39 PM, Pedro Costa <ps...@gmail.com> wrote:
> Hi,
>
> 1 - I would like to run Hadoop MR in my laptop, but with the cluster
> mode configuration. I've put a slave file with the following content:
> [code]
> 127.0.0.1
> localhost
> 192.168.10.1
> mylaptop
> [/code]
>
>
> the 192.168.10.1 is the IP of the machine and the "mylaptop" is the
> logical name of the address. Is this a well configured slaves file to
> do what I want?

Just the hostnames (or just the IPs) should suffice in the conf/slaves
list. Do not add multiple entries pointing to the same node.

> I was thinking in running a Tasktracker in each Core, although I don't
> know how to do it.

Set the value to 4 for the parameters:
* mapred.tasktracker.map.tasks.maximum (For 4 parallel mappers)
* mapred.tasktracker.reduce.tasks.maximum (For 4 parallel reducers)

-- 
Harsh J