Fwd: FW: How to run multiple data nodes and multiple task trackers on single server.

[Xiaobo Gu <gu...@gmail.com>]

Hi,
Can you have a look at this please,

Regards,

Xiaobo Gu


---------- Forwarded message ----------
From: Xiaobo Gu <gu...@gmail.com>
Date: Sun, Jul 3, 2011 at 10:07 PM
Subject: Re: FW: How to run multiple data nodes and multiple task
trackers on single server.
To: mapreduce-user@hadoop.apache.org


Hi Harsh
I have successfully running 2 data nodes in a single vitual machine,
and we will depoly 4 or 8 data nodes on our big SMP server, which has
32 CPU cores and 256G RAM,in order to take full advantage of all the
resources, do we need to configure more task trackers too, or can we
set mapred.tasktracker.map.tasks.maximum and
mapred.tasktracker.reduce.tasks.maximum to a larger number such as 8
or 16 to achieve the same purpose?

We have seen this
http://developer.yahoo.com/events/hadoopsummit2011/agenda.html#21, but
have not get any more details, we think multiple data node
configuration on big SMP servers is a good point to start with.

Regards,

Xiaobo Gu

On Sun, Jul 3, 2011 at 9:56 PM, Harsh J <ha...@cloudera.com> wrote:
> On Sun, Jul 3, 2011 at 9:41 AM, XiaoboGu <gu...@gmail.com> wrote:
>>> Hi,
>>>
>>> Do we have to run multiple task trackers when running multiple data nodes on a single
>>> computer?
>>>
>>> Regards,
>>>
>>> Xiaobo Gu
>>>
>
> Do we _have_ to? --> No, its a matter of your choice if you want
> MapReduce daemons running along. They are not coupled.
>
> Regd. your original question, what's the string of "$DN_CONF_OPTS" being passed?
>
> --
> Harsh J
>