You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-user@hadoop.apache.org by navaz <na...@gmail.com> on 2013/12/08 20:14:54 UTC
set up a hadoop cluster
Hi All
I need one suggestion from his group. I am trying to set up a hadoop
cluster in my research lab. I would like to know how many physical VMs are
needed for this. . My main intrest to measure the shuffle phase network
traffic. What is the basic requirement like namenode and data node .
Thanks & Regards
*Abdul Navaz*
*Graduate Student*
*University of Houston*
*Houston, TX - 77204-4020*
Re: set up a hadoop cluster
Posted by Smarty Juice <sm...@gmail.com>.
hadoop default replication factor is 3 configured in
hdfs-default.xml<http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-hdfs/hdfs-default.xml>(1
master (namenode) and 2 slaves - data nodes )
useful resource
<http://www.michael-noll.com/tutorials/running-hadoop-on-ubuntu-linux-multi-node-cluster/>to
start hadoop mutli node cluster installation >
On Mon, Dec 9, 2013 at 4:50 PM, navaz <na...@gmail.com> wrote:
> second try...
>
>
> On Sun, Dec 8, 2013 at 1:14 PM, navaz <na...@gmail.com> wrote:
>
>> Hi All
>>
>> I need one suggestion from his group. I am trying to set up a hadoop
>> cluster in my research lab. I would like to know how many physical VMs are
>> needed for this. . My main intrest to measure the shuffle phase network
>> traffic. What is the basic requirement like namenode and data node .
>>
>>
>> Thanks & Regards
>> *Abdul Navaz*
>>
>> *Graduate Student *
>> *University of Houston*
>> *Houston, TX - 77204-4020*
>>
>>
>>
>
>
> --
> *Abdul Navaz*
> *Masters in Network Communications*
> *University of Houston*
> *Houston, TX - 77204-4020*
> *Ph - 281-685-0388 <281-685-0388>*
> *fabdulnavaz@uh.edu* <fa...@uh.edu>
>
>
Re: set up a hadoop cluster
Posted by Smarty Juice <sm...@gmail.com>.
Hadoop default replication factor is 3 and you can configure it in
hdfs-default.xml<http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-hdfs/hdfs-default.xml>,
youu should have one Master (NameNode and 2 Slaves (Data Nodes)
and please follow
http://www.michael-noll.com/tutorials/running-hadoop-on-ubuntu-linux-multi-node-cluster/
On Mon, Dec 9, 2013 at 4:50 PM, navaz <na...@gmail.com> wrote:
> second try...
>
>
> On Sun, Dec 8, 2013 at 1:14 PM, navaz <na...@gmail.com> wrote:
>
>> Hi All
>>
>> I need one suggestion from his group. I am trying to set up a hadoop
>> cluster in my research lab. I would like to know how many physical VMs are
>> needed for this. . My main intrest to measure the shuffle phase network
>> traffic. What is the basic requirement like namenode and data node .
>>
>>
>> Thanks & Regards
>> *Abdul Navaz*
>>
>> *Graduate Student *
>> *University of Houston*
>> *Houston, TX - 77204-4020*
>>
>>
>>
>
>
> --
> *Abdul Navaz*
> *Masters in Network Communications*
> *University of Houston*
> *Houston, TX - 77204-4020*
> *Ph - 281-685-0388 <281-685-0388>*
> *fabdulnavaz@uh.edu* <fa...@uh.edu>
>
>
Re: set up a hadoop cluster
Posted by Smarty Juice <sm...@gmail.com>.
hadoop default replication factor is 3 configured in
hdfs-default.xml<http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-hdfs/hdfs-default.xml>(1
master (namenode) and 2 slaves - data nodes )
useful resource
<http://www.michael-noll.com/tutorials/running-hadoop-on-ubuntu-linux-multi-node-cluster/>to
start hadoop mutli node cluster installation >
On Mon, Dec 9, 2013 at 4:50 PM, navaz <na...@gmail.com> wrote:
> second try...
>
>
> On Sun, Dec 8, 2013 at 1:14 PM, navaz <na...@gmail.com> wrote:
>
>> Hi All
>>
>> I need one suggestion from his group. I am trying to set up a hadoop
>> cluster in my research lab. I would like to know how many physical VMs are
>> needed for this. . My main intrest to measure the shuffle phase network
>> traffic. What is the basic requirement like namenode and data node .
>>
>>
>> Thanks & Regards
>> *Abdul Navaz*
>>
>> *Graduate Student *
>> *University of Houston*
>> *Houston, TX - 77204-4020*
>>
>>
>>
>
>
> --
> *Abdul Navaz*
> *Masters in Network Communications*
> *University of Houston*
> *Houston, TX - 77204-4020*
> *Ph - 281-685-0388 <281-685-0388>*
> *fabdulnavaz@uh.edu* <fa...@uh.edu>
>
>
Re: set up a hadoop cluster
Posted by Smarty Juice <sm...@gmail.com>.
hadoop default replication factor is 3 configured in
hdfs-default.xml<http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-hdfs/hdfs-default.xml>(1
master (namenode) and 2 slaves - data nodes )
useful resource
<http://www.michael-noll.com/tutorials/running-hadoop-on-ubuntu-linux-multi-node-cluster/>to
start hadoop mutli node cluster installation >
On Mon, Dec 9, 2013 at 4:50 PM, navaz <na...@gmail.com> wrote:
> second try...
>
>
> On Sun, Dec 8, 2013 at 1:14 PM, navaz <na...@gmail.com> wrote:
>
>> Hi All
>>
>> I need one suggestion from his group. I am trying to set up a hadoop
>> cluster in my research lab. I would like to know how many physical VMs are
>> needed for this. . My main intrest to measure the shuffle phase network
>> traffic. What is the basic requirement like namenode and data node .
>>
>>
>> Thanks & Regards
>> *Abdul Navaz*
>>
>> *Graduate Student *
>> *University of Houston*
>> *Houston, TX - 77204-4020*
>>
>>
>>
>
>
> --
> *Abdul Navaz*
> *Masters in Network Communications*
> *University of Houston*
> *Houston, TX - 77204-4020*
> *Ph - 281-685-0388 <281-685-0388>*
> *fabdulnavaz@uh.edu* <fa...@uh.edu>
>
>
Re: set up a hadoop cluster
Posted by Smarty Juice <sm...@gmail.com>.
Hadoop default replication factor is 3 and you can configure it in
hdfs-default.xml<http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-hdfs/hdfs-default.xml>,
youu should have one Master (NameNode and 2 Slaves (Data Nodes)
and please follow
http://www.michael-noll.com/tutorials/running-hadoop-on-ubuntu-linux-multi-node-cluster/
On Mon, Dec 9, 2013 at 4:50 PM, navaz <na...@gmail.com> wrote:
> second try...
>
>
> On Sun, Dec 8, 2013 at 1:14 PM, navaz <na...@gmail.com> wrote:
>
>> Hi All
>>
>> I need one suggestion from his group. I am trying to set up a hadoop
>> cluster in my research lab. I would like to know how many physical VMs are
>> needed for this. . My main intrest to measure the shuffle phase network
>> traffic. What is the basic requirement like namenode and data node .
>>
>>
>> Thanks & Regards
>> *Abdul Navaz*
>>
>> *Graduate Student *
>> *University of Houston*
>> *Houston, TX - 77204-4020*
>>
>>
>>
>
>
> --
> *Abdul Navaz*
> *Masters in Network Communications*
> *University of Houston*
> *Houston, TX - 77204-4020*
> *Ph - 281-685-0388 <281-685-0388>*
> *fabdulnavaz@uh.edu* <fa...@uh.edu>
>
>
Re: set up a hadoop cluster
Posted by Smarty Juice <sm...@gmail.com>.
Hadoop default replication factor is 3 and you can configure it in
hdfs-default.xml<http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-hdfs/hdfs-default.xml>,
youu should have one Master (NameNode and 2 Slaves (Data Nodes)
and please follow
http://www.michael-noll.com/tutorials/running-hadoop-on-ubuntu-linux-multi-node-cluster/
On Mon, Dec 9, 2013 at 4:50 PM, navaz <na...@gmail.com> wrote:
> second try...
>
>
> On Sun, Dec 8, 2013 at 1:14 PM, navaz <na...@gmail.com> wrote:
>
>> Hi All
>>
>> I need one suggestion from his group. I am trying to set up a hadoop
>> cluster in my research lab. I would like to know how many physical VMs are
>> needed for this. . My main intrest to measure the shuffle phase network
>> traffic. What is the basic requirement like namenode and data node .
>>
>>
>> Thanks & Regards
>> *Abdul Navaz*
>>
>> *Graduate Student *
>> *University of Houston*
>> *Houston, TX - 77204-4020*
>>
>>
>>
>
>
> --
> *Abdul Navaz*
> *Masters in Network Communications*
> *University of Houston*
> *Houston, TX - 77204-4020*
> *Ph - 281-685-0388 <281-685-0388>*
> *fabdulnavaz@uh.edu* <fa...@uh.edu>
>
>
Re: set up a hadoop cluster
Posted by Smarty Juice <sm...@gmail.com>.
hadoop default replication factor is 3 configured in
hdfs-default.xml<http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-hdfs/hdfs-default.xml>(1
master (namenode) and 2 slaves - data nodes )
useful resource
<http://www.michael-noll.com/tutorials/running-hadoop-on-ubuntu-linux-multi-node-cluster/>to
start hadoop mutli node cluster installation >
On Mon, Dec 9, 2013 at 4:50 PM, navaz <na...@gmail.com> wrote:
> second try...
>
>
> On Sun, Dec 8, 2013 at 1:14 PM, navaz <na...@gmail.com> wrote:
>
>> Hi All
>>
>> I need one suggestion from his group. I am trying to set up a hadoop
>> cluster in my research lab. I would like to know how many physical VMs are
>> needed for this. . My main intrest to measure the shuffle phase network
>> traffic. What is the basic requirement like namenode and data node .
>>
>>
>> Thanks & Regards
>> *Abdul Navaz*
>>
>> *Graduate Student *
>> *University of Houston*
>> *Houston, TX - 77204-4020*
>>
>>
>>
>
>
> --
> *Abdul Navaz*
> *Masters in Network Communications*
> *University of Houston*
> *Houston, TX - 77204-4020*
> *Ph - 281-685-0388 <281-685-0388>*
> *fabdulnavaz@uh.edu* <fa...@uh.edu>
>
>
Re: set up a hadoop cluster
Posted by Smarty Juice <sm...@gmail.com>.
Hadoop default replication factor is 3 and you can configure it in
hdfs-default.xml<http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-hdfs/hdfs-default.xml>,
youu should have one Master (NameNode and 2 Slaves (Data Nodes)
and please follow
http://www.michael-noll.com/tutorials/running-hadoop-on-ubuntu-linux-multi-node-cluster/
On Mon, Dec 9, 2013 at 4:50 PM, navaz <na...@gmail.com> wrote:
> second try...
>
>
> On Sun, Dec 8, 2013 at 1:14 PM, navaz <na...@gmail.com> wrote:
>
>> Hi All
>>
>> I need one suggestion from his group. I am trying to set up a hadoop
>> cluster in my research lab. I would like to know how many physical VMs are
>> needed for this. . My main intrest to measure the shuffle phase network
>> traffic. What is the basic requirement like namenode and data node .
>>
>>
>> Thanks & Regards
>> *Abdul Navaz*
>>
>> *Graduate Student *
>> *University of Houston*
>> *Houston, TX - 77204-4020*
>>
>>
>>
>
>
> --
> *Abdul Navaz*
> *Masters in Network Communications*
> *University of Houston*
> *Houston, TX - 77204-4020*
> *Ph - 281-685-0388 <281-685-0388>*
> *fabdulnavaz@uh.edu* <fa...@uh.edu>
>
>
Re: set up a hadoop cluster
Posted by navaz <na...@gmail.com>.
second try...
On Sun, Dec 8, 2013 at 1:14 PM, navaz <na...@gmail.com> wrote:
> Hi All
>
> I need one suggestion from his group. I am trying to set up a hadoop
> cluster in my research lab. I would like to know how many physical VMs are
> needed for this. . My main intrest to measure the shuffle phase network
> traffic. What is the basic requirement like namenode and data node .
>
>
> Thanks & Regards
> *Abdul Navaz*
>
> *Graduate Student *
> *University of Houston*
> *Houston, TX - 77204-4020*
>
>
>
--
*Abdul Navaz*
*Masters in Network Communications*
*University of Houston*
*Houston, TX - 77204-4020*
*Ph - 281-685-0388*
*fabdulnavaz@uh.edu* <fa...@uh.edu>
Re: set up a hadoop cluster
Posted by navaz <na...@gmail.com>.
second try...
On Sun, Dec 8, 2013 at 1:14 PM, navaz <na...@gmail.com> wrote:
> Hi All
>
> I need one suggestion from his group. I am trying to set up a hadoop
> cluster in my research lab. I would like to know how many physical VMs are
> needed for this. . My main intrest to measure the shuffle phase network
> traffic. What is the basic requirement like namenode and data node .
>
>
> Thanks & Regards
> *Abdul Navaz*
>
> *Graduate Student *
> *University of Houston*
> *Houston, TX - 77204-4020*
>
>
>
--
*Abdul Navaz*
*Masters in Network Communications*
*University of Houston*
*Houston, TX - 77204-4020*
*Ph - 281-685-0388*
*fabdulnavaz@uh.edu* <fa...@uh.edu>
Re: set up a hadoop cluster
Posted by navaz <na...@gmail.com>.
second try...
On Sun, Dec 8, 2013 at 1:14 PM, navaz <na...@gmail.com> wrote:
> Hi All
>
> I need one suggestion from his group. I am trying to set up a hadoop
> cluster in my research lab. I would like to know how many physical VMs are
> needed for this. . My main intrest to measure the shuffle phase network
> traffic. What is the basic requirement like namenode and data node .
>
>
> Thanks & Regards
> *Abdul Navaz*
>
> *Graduate Student *
> *University of Houston*
> *Houston, TX - 77204-4020*
>
>
>
--
*Abdul Navaz*
*Masters in Network Communications*
*University of Houston*
*Houston, TX - 77204-4020*
*Ph - 281-685-0388*
*fabdulnavaz@uh.edu* <fa...@uh.edu>
Re: set up a hadoop cluster
Posted by navaz <na...@gmail.com>.
second try...
On Sun, Dec 8, 2013 at 1:14 PM, navaz <na...@gmail.com> wrote:
> Hi All
>
> I need one suggestion from his group. I am trying to set up a hadoop
> cluster in my research lab. I would like to know how many physical VMs are
> needed for this. . My main intrest to measure the shuffle phase network
> traffic. What is the basic requirement like namenode and data node .
>
>
> Thanks & Regards
> *Abdul Navaz*
>
> *Graduate Student *
> *University of Houston*
> *Houston, TX - 77204-4020*
>
>
>
--
*Abdul Navaz*
*Masters in Network Communications*
*University of Houston*
*Houston, TX - 77204-4020*
*Ph - 281-685-0388*
*fabdulnavaz@uh.edu* <fa...@uh.edu>