You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by navaz <na...@gmail.com> on 2013/12/18 21:01:29 UTC

multinode hadoop cluster on vmware

Hi

I want to set up a multinode hadoop cluser on vmware. Can anybody suggest
some good material to do so. I have used these
https://www.dropbox.com/s/05aurcp42asuktp/Chiu%20Hadoop%20Pig%20Install%20Instructions.docx
instruction to setup single node hadoop cluster on vmware. Now can  anybody
help me in creating a multinode hadoop cluster ( 3 nodes) on vmware?




Thanks & Regards
*Abdul Navaz*
*Masters in Network Communications*
*University of Houston*
*Houston, TX - 77204-4020*
*Ph - 281-685-0388*
*fabdulnavaz@uh.edu* <fa...@uh.edu>

Re: multinode hadoop cluster on vmware

Posted by Hiran Chaudhuri <hi...@amadeus.com>.
If I understand correctly, Hadoop or BigData applicatins are highly I/O 
bound. So for performant processing you would definitely prefer many 
little physical machines.
Apart from that setting up Hadoop on VMWare should bring no questions that 
would not occur on physical installations.

Depending on your performance needs and your three virtual machines you 
would probably go with a dedicated master and two slave nodes.
The master should be running NameNode and ResourceManager, the slaves 
DataNode and NodeManager. Other nodes in the Hadoop setup are optional.

Hiran

Hiran Chaudhuri
System Support Programmer / Analyst
IT Service Assurance
Hosting & Regional Services (IH)
Amadeus Data Processing GmbH
Berghamer Strasse 6
85435 Erding
T: +49-8122-43x3662
hiran.chaudhuri@amadeus.com
http://www.amadeus.com





From:   navaz <na...@gmail.com>
To:     user <us...@hadoop.apache.org>
Date:   18-12-13 21:02
Subject:        multinode hadoop cluster on vmware



Hi

I want to set up a multinode hadoop cluser on vmware. Can anybody suggest 
some good material to do so. I have used these 
https://www.dropbox.com/s/05aurcp42asuktp/Chiu%20Hadoop%20Pig%20Install%20Instructions.docx
instruction to setup single node hadoop cluster on vmware. Now can 
 anybody help me in creating a multinode hadoop cluster ( 3 nodes) on 
vmware?




Thanks & Regards
Abdul Navaz
Masters in Network Communications
University of Houston
Houston, TX - 77204-4020
Ph - 281-685-0388
fabdulnavaz@uh.edu





IMPORTANT  -  CONFIDENTIALITY  NOTICE  - This e-mail is intended only for 
the use of the individual or entity shown above as addressees. It may 
contain information which is privileged, confidential or otherwise 
protected from disclosure under applicable laws.  If the reader of this 
transmission is not the intended recipient, you are hereby notified that 
any dissemination, printing, distribution, copying, disclosure or the 
taking of any action in reliance on the contents of this information is 
strictly prohibited.  If you have received this transmission in error, 
please immediately notify us by reply e-mail or using the address below 
and delete the message and any attachments from your system. 

Amadeus Data Processing GmbH 
Geschäftsführer: Dr. Wolfgang Krips
Sitz der Gesellschaft: Erding 
HR München 48 199 
Berghamer Strasse 6 
85435 Erding 
Germany

Re: multinode hadoop cluster on vmware

Posted by Hiran Chaudhuri <hi...@amadeus.com>.
If I understand correctly, Hadoop or BigData applicatins are highly I/O 
bound. So for performant processing you would definitely prefer many 
little physical machines.
Apart from that setting up Hadoop on VMWare should bring no questions that 
would not occur on physical installations.

Depending on your performance needs and your three virtual machines you 
would probably go with a dedicated master and two slave nodes.
The master should be running NameNode and ResourceManager, the slaves 
DataNode and NodeManager. Other nodes in the Hadoop setup are optional.

Hiran

Hiran Chaudhuri
System Support Programmer / Analyst
IT Service Assurance
Hosting & Regional Services (IH)
Amadeus Data Processing GmbH
Berghamer Strasse 6
85435 Erding
T: +49-8122-43x3662
hiran.chaudhuri@amadeus.com
http://www.amadeus.com





From:   navaz <na...@gmail.com>
To:     user <us...@hadoop.apache.org>
Date:   18-12-13 21:02
Subject:        multinode hadoop cluster on vmware



Hi

I want to set up a multinode hadoop cluser on vmware. Can anybody suggest 
some good material to do so. I have used these 
https://www.dropbox.com/s/05aurcp42asuktp/Chiu%20Hadoop%20Pig%20Install%20Instructions.docx
instruction to setup single node hadoop cluster on vmware. Now can 
 anybody help me in creating a multinode hadoop cluster ( 3 nodes) on 
vmware?




Thanks & Regards
Abdul Navaz
Masters in Network Communications
University of Houston
Houston, TX - 77204-4020
Ph - 281-685-0388
fabdulnavaz@uh.edu





IMPORTANT  -  CONFIDENTIALITY  NOTICE  - This e-mail is intended only for 
the use of the individual or entity shown above as addressees. It may 
contain information which is privileged, confidential or otherwise 
protected from disclosure under applicable laws.  If the reader of this 
transmission is not the intended recipient, you are hereby notified that 
any dissemination, printing, distribution, copying, disclosure or the 
taking of any action in reliance on the contents of this information is 
strictly prohibited.  If you have received this transmission in error, 
please immediately notify us by reply e-mail or using the address below 
and delete the message and any attachments from your system. 

Amadeus Data Processing GmbH 
Geschäftsführer: Dr. Wolfgang Krips
Sitz der Gesellschaft: Erding 
HR München 48 199 
Berghamer Strasse 6 
85435 Erding 
Germany

Re: multinode hadoop cluster on vmware

Posted by Hiran Chaudhuri <hi...@amadeus.com>.
If I understand correctly, Hadoop or BigData applicatins are highly I/O 
bound. So for performant processing you would definitely prefer many 
little physical machines.
Apart from that setting up Hadoop on VMWare should bring no questions that 
would not occur on physical installations.

Depending on your performance needs and your three virtual machines you 
would probably go with a dedicated master and two slave nodes.
The master should be running NameNode and ResourceManager, the slaves 
DataNode and NodeManager. Other nodes in the Hadoop setup are optional.

Hiran

Hiran Chaudhuri
System Support Programmer / Analyst
IT Service Assurance
Hosting & Regional Services (IH)
Amadeus Data Processing GmbH
Berghamer Strasse 6
85435 Erding
T: +49-8122-43x3662
hiran.chaudhuri@amadeus.com
http://www.amadeus.com





From:   navaz <na...@gmail.com>
To:     user <us...@hadoop.apache.org>
Date:   18-12-13 21:02
Subject:        multinode hadoop cluster on vmware



Hi

I want to set up a multinode hadoop cluser on vmware. Can anybody suggest 
some good material to do so. I have used these 
https://www.dropbox.com/s/05aurcp42asuktp/Chiu%20Hadoop%20Pig%20Install%20Instructions.docx
instruction to setup single node hadoop cluster on vmware. Now can 
 anybody help me in creating a multinode hadoop cluster ( 3 nodes) on 
vmware?




Thanks & Regards
Abdul Navaz
Masters in Network Communications
University of Houston
Houston, TX - 77204-4020
Ph - 281-685-0388
fabdulnavaz@uh.edu





IMPORTANT  -  CONFIDENTIALITY  NOTICE  - This e-mail is intended only for 
the use of the individual or entity shown above as addressees. It may 
contain information which is privileged, confidential or otherwise 
protected from disclosure under applicable laws.  If the reader of this 
transmission is not the intended recipient, you are hereby notified that 
any dissemination, printing, distribution, copying, disclosure or the 
taking of any action in reliance on the contents of this information is 
strictly prohibited.  If you have received this transmission in error, 
please immediately notify us by reply e-mail or using the address below 
and delete the message and any attachments from your system. 

Amadeus Data Processing GmbH 
Geschäftsführer: Dr. Wolfgang Krips
Sitz der Gesellschaft: Erding 
HR München 48 199 
Berghamer Strasse 6 
85435 Erding 
Germany

Re: multinode hadoop cluster on vmware

Posted by Hiran Chaudhuri <hi...@amadeus.com>.
If I understand correctly, Hadoop or BigData applicatins are highly I/O 
bound. So for performant processing you would definitely prefer many 
little physical machines.
Apart from that setting up Hadoop on VMWare should bring no questions that 
would not occur on physical installations.

Depending on your performance needs and your three virtual machines you 
would probably go with a dedicated master and two slave nodes.
The master should be running NameNode and ResourceManager, the slaves 
DataNode and NodeManager. Other nodes in the Hadoop setup are optional.

Hiran

Hiran Chaudhuri
System Support Programmer / Analyst
IT Service Assurance
Hosting & Regional Services (IH)
Amadeus Data Processing GmbH
Berghamer Strasse 6
85435 Erding
T: +49-8122-43x3662
hiran.chaudhuri@amadeus.com
http://www.amadeus.com





From:   navaz <na...@gmail.com>
To:     user <us...@hadoop.apache.org>
Date:   18-12-13 21:02
Subject:        multinode hadoop cluster on vmware



Hi

I want to set up a multinode hadoop cluser on vmware. Can anybody suggest 
some good material to do so. I have used these 
https://www.dropbox.com/s/05aurcp42asuktp/Chiu%20Hadoop%20Pig%20Install%20Instructions.docx
instruction to setup single node hadoop cluster on vmware. Now can 
 anybody help me in creating a multinode hadoop cluster ( 3 nodes) on 
vmware?




Thanks & Regards
Abdul Navaz
Masters in Network Communications
University of Houston
Houston, TX - 77204-4020
Ph - 281-685-0388
fabdulnavaz@uh.edu





IMPORTANT  -  CONFIDENTIALITY  NOTICE  - This e-mail is intended only for 
the use of the individual or entity shown above as addressees. It may 
contain information which is privileged, confidential or otherwise 
protected from disclosure under applicable laws.  If the reader of this 
transmission is not the intended recipient, you are hereby notified that 
any dissemination, printing, distribution, copying, disclosure or the 
taking of any action in reliance on the contents of this information is 
strictly prohibited.  If you have received this transmission in error, 
please immediately notify us by reply e-mail or using the address below 
and delete the message and any attachments from your system. 

Amadeus Data Processing GmbH 
Geschäftsführer: Dr. Wolfgang Krips
Sitz der Gesellschaft: Erding 
HR München 48 199 
Berghamer Strasse 6 
85435 Erding 
Germany

Re: multinode hadoop cluster on vmware

Posted by Adam Kawa <ka...@gmail.com>.
Maybe you can try Serengeti http://www.projectserengeti.org/ or Vagrant (
http://java.dzone.com/articles/setting-hadoop-virtual-cluster,
http://blog.cloudera.com/blog/2013/04/how-to-use-vagrant-to-set-up-a-virtual-hadoop-cluster/
)?


2013/12/18 navaz <na...@gmail.com>

> Hi
>
> I want to set up a multinode hadoop cluser on vmware. Can anybody suggest
> some good material to do so. I have used these
> https://www.dropbox.com/s/05aurcp42asuktp/Chiu%20Hadoop%20Pig%20Install%20Instructions.docx
> instruction to setup single node hadoop cluster on vmware. Now can
>  anybody help me in creating a multinode hadoop cluster ( 3 nodes) on
> vmware?
>
>
>
>
> Thanks & Regards
> *Abdul Navaz*
> *Masters in Network Communications*
> *University of Houston*
> *Houston, TX - 77204-4020 <77204-4020>*
> *Ph - 281-685-0388*
> *fabdulnavaz@uh.edu* <fa...@uh.edu>
>
>

Re: multinode hadoop cluster on vmware

Posted by Adam Kawa <ka...@gmail.com>.
Maybe you can try Serengeti http://www.projectserengeti.org/ or Vagrant (
http://java.dzone.com/articles/setting-hadoop-virtual-cluster,
http://blog.cloudera.com/blog/2013/04/how-to-use-vagrant-to-set-up-a-virtual-hadoop-cluster/
)?


2013/12/18 navaz <na...@gmail.com>

> Hi
>
> I want to set up a multinode hadoop cluser on vmware. Can anybody suggest
> some good material to do so. I have used these
> https://www.dropbox.com/s/05aurcp42asuktp/Chiu%20Hadoop%20Pig%20Install%20Instructions.docx
> instruction to setup single node hadoop cluster on vmware. Now can
>  anybody help me in creating a multinode hadoop cluster ( 3 nodes) on
> vmware?
>
>
>
>
> Thanks & Regards
> *Abdul Navaz*
> *Masters in Network Communications*
> *University of Houston*
> *Houston, TX - 77204-4020 <77204-4020>*
> *Ph - 281-685-0388*
> *fabdulnavaz@uh.edu* <fa...@uh.edu>
>
>

Re: multinode hadoop cluster on vmware

Posted by Adam Kawa <ka...@gmail.com>.
Maybe you can try Serengeti http://www.projectserengeti.org/ or Vagrant (
http://java.dzone.com/articles/setting-hadoop-virtual-cluster,
http://blog.cloudera.com/blog/2013/04/how-to-use-vagrant-to-set-up-a-virtual-hadoop-cluster/
)?


2013/12/18 navaz <na...@gmail.com>

> Hi
>
> I want to set up a multinode hadoop cluser on vmware. Can anybody suggest
> some good material to do so. I have used these
> https://www.dropbox.com/s/05aurcp42asuktp/Chiu%20Hadoop%20Pig%20Install%20Instructions.docx
> instruction to setup single node hadoop cluster on vmware. Now can
>  anybody help me in creating a multinode hadoop cluster ( 3 nodes) on
> vmware?
>
>
>
>
> Thanks & Regards
> *Abdul Navaz*
> *Masters in Network Communications*
> *University of Houston*
> *Houston, TX - 77204-4020 <77204-4020>*
> *Ph - 281-685-0388*
> *fabdulnavaz@uh.edu* <fa...@uh.edu>
>
>

Re: multinode hadoop cluster on vmware

Posted by Adam Kawa <ka...@gmail.com>.
Maybe you can try Serengeti http://www.projectserengeti.org/ or Vagrant (
http://java.dzone.com/articles/setting-hadoop-virtual-cluster,
http://blog.cloudera.com/blog/2013/04/how-to-use-vagrant-to-set-up-a-virtual-hadoop-cluster/
)?


2013/12/18 navaz <na...@gmail.com>

> Hi
>
> I want to set up a multinode hadoop cluser on vmware. Can anybody suggest
> some good material to do so. I have used these
> https://www.dropbox.com/s/05aurcp42asuktp/Chiu%20Hadoop%20Pig%20Install%20Instructions.docx
> instruction to setup single node hadoop cluster on vmware. Now can
>  anybody help me in creating a multinode hadoop cluster ( 3 nodes) on
> vmware?
>
>
>
>
> Thanks & Regards
> *Abdul Navaz*
> *Masters in Network Communications*
> *University of Houston*
> *Houston, TX - 77204-4020 <77204-4020>*
> *Ph - 281-685-0388*
> *fabdulnavaz@uh.edu* <fa...@uh.edu>
>
>