RE: Is there a way to start namenode, 2 datanodes on one box?

[Devaraj Das <dd...@yahoo-inc.com>]

Moved this to hadoop-user...

My 2 cents: A datanode is identified by its storageID. If you have multiple
storage devices in your box (visible to the user as different directories)
then you can have multiple datanodes on your box (one for each directory).
The way to do that is to specify a set of comma separated entries in your
conf file for the dfs.data.dir conf value. These datanodes will bind to
different ports.

-----Original Message-----
From: yichao.zhang [mailto:yichao.zhang@gmail.com] 
Sent: Tuesday, July 25, 2006 2:12 PM
To: hadoop-dev@lucene.apache.org
Subject: Is there a way to start namenode, 2 datanodes on one box?

Hi
is there a way to start the name server and more than 1 data servers on just
one box. I have only one PC and want to test against dfs.
I have set up one name server and just one data server in psuedo-distributed
mode on my box in eclipse.
However it seems no way to add another data server even if I have 3 Ethernet
adapters, one is a real d-link card, the other 2 are Microsoft loop
adapters. so my box has three different ip. but the data server is
identified by machine name, so there is impossible to have 2 data servers on
a single box, is that right?  or is it possible to simply identify a data
node by its ip to solve this issue?
your 2 cents appreciated!

-Yichao

Re: Is there a way to start namenode, 2 datanodes on one box?

["yichao.zhang" <yi...@gmail.com>]

Thanks Devaraj
Setting multiple entries to fs.data.dir conf makes multiple datanode
threads, and works fine. However they are SubThreads, so only one connection
to namenode. It's some different from real cluster.
Now I start a datanode with one dfs.data.dir conf value and then start a
second datanode with another dfs.data.dir conf value, it works fine too and
there are 2 connections to the namenode now.



On 7/25/06, Devaraj Das <dd...@yahoo-inc.com> wrote:
>
> Moved this to hadoop-user...
>
> My 2 cents: A datanode is identified by its storageID. If you have
> multiple
> storage devices in your box (visible to the user as different directories)
> then you can have multiple datanodes on your box (one for each directory).
>
> The way to do that is to specify a set of comma separated entries in your
> conf file for the dfs.data.dir conf value. These datanodes will bind to
> different ports.
>
> -----Original Message-----
> From: yichao.zhang [mailto:yichao.zhang@gmail.com]
> Sent: Tuesday, July 25, 2006 2:12 PM
> To: hadoop-dev@lucene.apache.org
> Subject: Is there a way to start namenode, 2 datanodes on one box?
>
> Hi
> is there a way to start the name server and more than 1 data servers on
> just
> one box. I have only one PC and want to test against dfs.
> I have set up one name server and just one data server in
> psuedo-distributed
> mode on my box in eclipse.
> However it seems no way to add another data server even if I have 3
> Ethernet
> adapters, one is a real d-link card, the other 2 are Microsoft loop
> adapters. so my box has three different ip. but the data server is
> identified by machine name, so there is impossible to have 2 data servers
> on
> a single box, is that right?  or is it possible to simply identify a data
> node by its ip to solve this issue?
> your 2 cents appreciated!
>
> -Yichao
>
>