You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-user@hadoop.apache.org by Manar Elkady <m....@fci-cu.edu.eg> on 2014/07/06 05:27:45 UTC
Hadoop virtual machine
Hi,
I am a newcomer in using Hadoop, and I read many online tutorial to set
up Hadoop on Window by using virtual machines, but all of them link to
old versions of Hadoop virtual machines.
Could any one help me to find a Hadoop virtual machine, which include a
newer version of hadoop? Or should I do it myself from scratch?
Also, any well explained Hadoop installing tutorial and any other
helpful material are appreciated.
Manar,
--
Re: Hadoop virtual machine
Posted by Manar Elkady <m....@fci-cu.edu.eg>.
Thanks, for your help.
On 7/6/2014 5:38 AM, Gavin Yue wrote:
> http://hortonworks.com/products/hortonworks-sandbox/
>
> or
>
> CDH5
> http://www.cloudera.com/content/cloudera-content/cloudera-docs/DemoVMs/Cloudera-QuickStart-VM/cloudera_quickstart_vm.html
>
>
>
> On Sat, Jul 5, 2014 at 11:27 PM, Manar Elkady <m.elkady@fci-cu.edu.eg
> <ma...@fci-cu.edu.eg>> wrote:
>
> Hi,
>
> I am a newcomer in using Hadoop, and I read many online tutorial
> to set up Hadoop on Window by using virtual machines, but all of
> them link to old versions of Hadoop virtual machines.
> Could any one help me to find a Hadoop virtual machine, which
> include a newer version of hadoop? Or should I do it myself from
> scratch?
> Also, any well explained Hadoop installing tutorial and any other
> helpful material are appreciated.
>
>
> Manar,
>
>
> --
>
>
--
Re: Hadoop virtual machine
Posted by jay vyas <ja...@gmail.com>.
I really like the Cascading recipes above, thanks for sharing that !
Also we have *apache bigtop vagrant recipes* which we curate for this kind
of thing, which are really useful. You can spin up a 1 or multi node
cluster, just by running the startup.sh script.
Which are probably the most configurable and flexible. These are super
easy to use, and allow you maximal control over your environment.
1) git clone https://github.com/apache/bigtop
2) cd bigtop-deploy/vm/vagrant/vagrant-puppet
3) Follow the directions in the README to create your hadoop cluster
You can look into the provision script to see how you can customize exactly
which components come (hbase,mahout,pig,....) installed in your
distribution.
Feel free to drop a line on the bigtop mailing list if you need any help
with getting them up and running.
On Sun, Jul 6, 2014 at 12:47 PM, Andre Kelpe <ak...@concurrentinc.com>
wrote:
> We have a multi-vm or single-vm setup with apache hadoop, if you want to
> give that a spin:
> https://github.com/Cascading/vagrant-cascading-hadoop-cluster
>
> - André
>
>
> On Sun, Jul 6, 2014 at 9:05 AM, MrAsanjar . <af...@gmail.com> wrote:
>
>> For my hadoop development and testing I use LXC (linux container) instead
>> of VM, mainly due to its light weight resource consumption. As mater of
>> fact as I am typing, my ubuntu system is automatically building a 6 nodes
>> hadoop cluster on my 16G labtop.
>> If you have an Ubuntu system you could install a fully configurable
>> Hadoop 2.2.0 single node or multi-node cluster in less then 10 minutes.
>> Here what you need to do:
>> 1) Install and learn Ubuntu Juju (shouldn't take an hour)- instructions :
>> https://juju.ubuntu.com/docs/getting-started.html
>> 2) there are two types hadoop charms:
>> a) Single node for hadoop development :
>> https://jujucharms.com/?text=hadoop2-devel
>> b) multi-node for testing testing :
>> https://jujucharms.com/?text=hadoop
>> Let me know if you need more help
>>
>>
>> On Sun, Jul 6, 2014 at 7:59 AM, Marco Shaw <ma...@gmail.com> wrote:
>>
>>> Note that the CDH link is for Cloudera which only provides Hadoop for
>>> Linux.
>>>
>>> HDP has "pre-built VMs" for both Linux and Windows hosts.
>>>
>>> You can also search for "HDInsight emulator" which runs on Windows and
>>> is based on HDP.
>>>
>>> Marco
>>>
>>> On Jul 6, 2014, at 12:38 AM, Gavin Yue <yu...@gmail.com> wrote:
>>>
>>> http://hortonworks.com/products/hortonworks-sandbox/
>>>
>>> or
>>>
>>> CDH5
>>>
>>> http://www.cloudera.com/content/cloudera-content/cloudera-docs/DemoVMs/Cloudera-QuickStart-VM/cloudera_quickstart_vm.html
>>>
>>>
>>>
>>> On Sat, Jul 5, 2014 at 11:27 PM, Manar Elkady <m....@fci-cu.edu.eg>
>>> wrote:
>>>
>>>> Hi,
>>>>
>>>> I am a newcomer in using Hadoop, and I read many online tutorial to set
>>>> up Hadoop on Window by using virtual machines, but all of them link to old
>>>> versions of Hadoop virtual machines.
>>>> Could any one help me to find a Hadoop virtual machine, which include a
>>>> newer version of hadoop? Or should I do it myself from scratch?
>>>> Also, any well explained Hadoop installing tutorial and any other
>>>> helpful material are appreciated.
>>>>
>>>>
>>>> Manar,
>>>>
>>>>
>>>> --
>>>>
>>>>
>>>
>>
>
>
> --
> André Kelpe
> andre@concurrentinc.com
> http://concurrentinc.com
>
--
jay vyas
Re: Hadoop virtual machine
Posted by jay vyas <ja...@gmail.com>.
I really like the Cascading recipes above, thanks for sharing that !
Also we have *apache bigtop vagrant recipes* which we curate for this kind
of thing, which are really useful. You can spin up a 1 or multi node
cluster, just by running the startup.sh script.
Which are probably the most configurable and flexible. These are super
easy to use, and allow you maximal control over your environment.
1) git clone https://github.com/apache/bigtop
2) cd bigtop-deploy/vm/vagrant/vagrant-puppet
3) Follow the directions in the README to create your hadoop cluster
You can look into the provision script to see how you can customize exactly
which components come (hbase,mahout,pig,....) installed in your
distribution.
Feel free to drop a line on the bigtop mailing list if you need any help
with getting them up and running.
On Sun, Jul 6, 2014 at 12:47 PM, Andre Kelpe <ak...@concurrentinc.com>
wrote:
> We have a multi-vm or single-vm setup with apache hadoop, if you want to
> give that a spin:
> https://github.com/Cascading/vagrant-cascading-hadoop-cluster
>
> - André
>
>
> On Sun, Jul 6, 2014 at 9:05 AM, MrAsanjar . <af...@gmail.com> wrote:
>
>> For my hadoop development and testing I use LXC (linux container) instead
>> of VM, mainly due to its light weight resource consumption. As mater of
>> fact as I am typing, my ubuntu system is automatically building a 6 nodes
>> hadoop cluster on my 16G labtop.
>> If you have an Ubuntu system you could install a fully configurable
>> Hadoop 2.2.0 single node or multi-node cluster in less then 10 minutes.
>> Here what you need to do:
>> 1) Install and learn Ubuntu Juju (shouldn't take an hour)- instructions :
>> https://juju.ubuntu.com/docs/getting-started.html
>> 2) there are two types hadoop charms:
>> a) Single node for hadoop development :
>> https://jujucharms.com/?text=hadoop2-devel
>> b) multi-node for testing testing :
>> https://jujucharms.com/?text=hadoop
>> Let me know if you need more help
>>
>>
>> On Sun, Jul 6, 2014 at 7:59 AM, Marco Shaw <ma...@gmail.com> wrote:
>>
>>> Note that the CDH link is for Cloudera which only provides Hadoop for
>>> Linux.
>>>
>>> HDP has "pre-built VMs" for both Linux and Windows hosts.
>>>
>>> You can also search for "HDInsight emulator" which runs on Windows and
>>> is based on HDP.
>>>
>>> Marco
>>>
>>> On Jul 6, 2014, at 12:38 AM, Gavin Yue <yu...@gmail.com> wrote:
>>>
>>> http://hortonworks.com/products/hortonworks-sandbox/
>>>
>>> or
>>>
>>> CDH5
>>>
>>> http://www.cloudera.com/content/cloudera-content/cloudera-docs/DemoVMs/Cloudera-QuickStart-VM/cloudera_quickstart_vm.html
>>>
>>>
>>>
>>> On Sat, Jul 5, 2014 at 11:27 PM, Manar Elkady <m....@fci-cu.edu.eg>
>>> wrote:
>>>
>>>> Hi,
>>>>
>>>> I am a newcomer in using Hadoop, and I read many online tutorial to set
>>>> up Hadoop on Window by using virtual machines, but all of them link to old
>>>> versions of Hadoop virtual machines.
>>>> Could any one help me to find a Hadoop virtual machine, which include a
>>>> newer version of hadoop? Or should I do it myself from scratch?
>>>> Also, any well explained Hadoop installing tutorial and any other
>>>> helpful material are appreciated.
>>>>
>>>>
>>>> Manar,
>>>>
>>>>
>>>> --
>>>>
>>>>
>>>
>>
>
>
> --
> André Kelpe
> andre@concurrentinc.com
> http://concurrentinc.com
>
--
jay vyas
Re: Hadoop virtual machine
Posted by jay vyas <ja...@gmail.com>.
I really like the Cascading recipes above, thanks for sharing that !
Also we have *apache bigtop vagrant recipes* which we curate for this kind
of thing, which are really useful. You can spin up a 1 or multi node
cluster, just by running the startup.sh script.
Which are probably the most configurable and flexible. These are super
easy to use, and allow you maximal control over your environment.
1) git clone https://github.com/apache/bigtop
2) cd bigtop-deploy/vm/vagrant/vagrant-puppet
3) Follow the directions in the README to create your hadoop cluster
You can look into the provision script to see how you can customize exactly
which components come (hbase,mahout,pig,....) installed in your
distribution.
Feel free to drop a line on the bigtop mailing list if you need any help
with getting them up and running.
On Sun, Jul 6, 2014 at 12:47 PM, Andre Kelpe <ak...@concurrentinc.com>
wrote:
> We have a multi-vm or single-vm setup with apache hadoop, if you want to
> give that a spin:
> https://github.com/Cascading/vagrant-cascading-hadoop-cluster
>
> - André
>
>
> On Sun, Jul 6, 2014 at 9:05 AM, MrAsanjar . <af...@gmail.com> wrote:
>
>> For my hadoop development and testing I use LXC (linux container) instead
>> of VM, mainly due to its light weight resource consumption. As mater of
>> fact as I am typing, my ubuntu system is automatically building a 6 nodes
>> hadoop cluster on my 16G labtop.
>> If you have an Ubuntu system you could install a fully configurable
>> Hadoop 2.2.0 single node or multi-node cluster in less then 10 minutes.
>> Here what you need to do:
>> 1) Install and learn Ubuntu Juju (shouldn't take an hour)- instructions :
>> https://juju.ubuntu.com/docs/getting-started.html
>> 2) there are two types hadoop charms:
>> a) Single node for hadoop development :
>> https://jujucharms.com/?text=hadoop2-devel
>> b) multi-node for testing testing :
>> https://jujucharms.com/?text=hadoop
>> Let me know if you need more help
>>
>>
>> On Sun, Jul 6, 2014 at 7:59 AM, Marco Shaw <ma...@gmail.com> wrote:
>>
>>> Note that the CDH link is for Cloudera which only provides Hadoop for
>>> Linux.
>>>
>>> HDP has "pre-built VMs" for both Linux and Windows hosts.
>>>
>>> You can also search for "HDInsight emulator" which runs on Windows and
>>> is based on HDP.
>>>
>>> Marco
>>>
>>> On Jul 6, 2014, at 12:38 AM, Gavin Yue <yu...@gmail.com> wrote:
>>>
>>> http://hortonworks.com/products/hortonworks-sandbox/
>>>
>>> or
>>>
>>> CDH5
>>>
>>> http://www.cloudera.com/content/cloudera-content/cloudera-docs/DemoVMs/Cloudera-QuickStart-VM/cloudera_quickstart_vm.html
>>>
>>>
>>>
>>> On Sat, Jul 5, 2014 at 11:27 PM, Manar Elkady <m....@fci-cu.edu.eg>
>>> wrote:
>>>
>>>> Hi,
>>>>
>>>> I am a newcomer in using Hadoop, and I read many online tutorial to set
>>>> up Hadoop on Window by using virtual machines, but all of them link to old
>>>> versions of Hadoop virtual machines.
>>>> Could any one help me to find a Hadoop virtual machine, which include a
>>>> newer version of hadoop? Or should I do it myself from scratch?
>>>> Also, any well explained Hadoop installing tutorial and any other
>>>> helpful material are appreciated.
>>>>
>>>>
>>>> Manar,
>>>>
>>>>
>>>> --
>>>>
>>>>
>>>
>>
>
>
> --
> André Kelpe
> andre@concurrentinc.com
> http://concurrentinc.com
>
--
jay vyas
Re: Hadoop virtual machine
Posted by jay vyas <ja...@gmail.com>.
I really like the Cascading recipes above, thanks for sharing that !
Also we have *apache bigtop vagrant recipes* which we curate for this kind
of thing, which are really useful. You can spin up a 1 or multi node
cluster, just by running the startup.sh script.
Which are probably the most configurable and flexible. These are super
easy to use, and allow you maximal control over your environment.
1) git clone https://github.com/apache/bigtop
2) cd bigtop-deploy/vm/vagrant/vagrant-puppet
3) Follow the directions in the README to create your hadoop cluster
You can look into the provision script to see how you can customize exactly
which components come (hbase,mahout,pig,....) installed in your
distribution.
Feel free to drop a line on the bigtop mailing list if you need any help
with getting them up and running.
On Sun, Jul 6, 2014 at 12:47 PM, Andre Kelpe <ak...@concurrentinc.com>
wrote:
> We have a multi-vm or single-vm setup with apache hadoop, if you want to
> give that a spin:
> https://github.com/Cascading/vagrant-cascading-hadoop-cluster
>
> - André
>
>
> On Sun, Jul 6, 2014 at 9:05 AM, MrAsanjar . <af...@gmail.com> wrote:
>
>> For my hadoop development and testing I use LXC (linux container) instead
>> of VM, mainly due to its light weight resource consumption. As mater of
>> fact as I am typing, my ubuntu system is automatically building a 6 nodes
>> hadoop cluster on my 16G labtop.
>> If you have an Ubuntu system you could install a fully configurable
>> Hadoop 2.2.0 single node or multi-node cluster in less then 10 minutes.
>> Here what you need to do:
>> 1) Install and learn Ubuntu Juju (shouldn't take an hour)- instructions :
>> https://juju.ubuntu.com/docs/getting-started.html
>> 2) there are two types hadoop charms:
>> a) Single node for hadoop development :
>> https://jujucharms.com/?text=hadoop2-devel
>> b) multi-node for testing testing :
>> https://jujucharms.com/?text=hadoop
>> Let me know if you need more help
>>
>>
>> On Sun, Jul 6, 2014 at 7:59 AM, Marco Shaw <ma...@gmail.com> wrote:
>>
>>> Note that the CDH link is for Cloudera which only provides Hadoop for
>>> Linux.
>>>
>>> HDP has "pre-built VMs" for both Linux and Windows hosts.
>>>
>>> You can also search for "HDInsight emulator" which runs on Windows and
>>> is based on HDP.
>>>
>>> Marco
>>>
>>> On Jul 6, 2014, at 12:38 AM, Gavin Yue <yu...@gmail.com> wrote:
>>>
>>> http://hortonworks.com/products/hortonworks-sandbox/
>>>
>>> or
>>>
>>> CDH5
>>>
>>> http://www.cloudera.com/content/cloudera-content/cloudera-docs/DemoVMs/Cloudera-QuickStart-VM/cloudera_quickstart_vm.html
>>>
>>>
>>>
>>> On Sat, Jul 5, 2014 at 11:27 PM, Manar Elkady <m....@fci-cu.edu.eg>
>>> wrote:
>>>
>>>> Hi,
>>>>
>>>> I am a newcomer in using Hadoop, and I read many online tutorial to set
>>>> up Hadoop on Window by using virtual machines, but all of them link to old
>>>> versions of Hadoop virtual machines.
>>>> Could any one help me to find a Hadoop virtual machine, which include a
>>>> newer version of hadoop? Or should I do it myself from scratch?
>>>> Also, any well explained Hadoop installing tutorial and any other
>>>> helpful material are appreciated.
>>>>
>>>>
>>>> Manar,
>>>>
>>>>
>>>> --
>>>>
>>>>
>>>
>>
>
>
> --
> André Kelpe
> andre@concurrentinc.com
> http://concurrentinc.com
>
--
jay vyas
Re: Hadoop virtual machine
Posted by Andre Kelpe <ak...@concurrentinc.com>.
We have a multi-vm or single-vm setup with apache hadoop, if you want to
give that a spin:
https://github.com/Cascading/vagrant-cascading-hadoop-cluster
- André
On Sun, Jul 6, 2014 at 9:05 AM, MrAsanjar . <af...@gmail.com> wrote:
> For my hadoop development and testing I use LXC (linux container) instead
> of VM, mainly due to its light weight resource consumption. As mater of
> fact as I am typing, my ubuntu system is automatically building a 6 nodes
> hadoop cluster on my 16G labtop.
> If you have an Ubuntu system you could install a fully configurable Hadoop
> 2.2.0 single node or multi-node cluster in less then 10 minutes.
> Here what you need to do:
> 1) Install and learn Ubuntu Juju (shouldn't take an hour)- instructions :
> https://juju.ubuntu.com/docs/getting-started.html
> 2) there are two types hadoop charms:
> a) Single node for hadoop development :
> https://jujucharms.com/?text=hadoop2-devel
> b) multi-node for testing testing :
> https://jujucharms.com/?text=hadoop
> Let me know if you need more help
>
>
> On Sun, Jul 6, 2014 at 7:59 AM, Marco Shaw <ma...@gmail.com> wrote:
>
>> Note that the CDH link is for Cloudera which only provides Hadoop for
>> Linux.
>>
>> HDP has "pre-built VMs" for both Linux and Windows hosts.
>>
>> You can also search for "HDInsight emulator" which runs on Windows and is
>> based on HDP.
>>
>> Marco
>>
>> On Jul 6, 2014, at 12:38 AM, Gavin Yue <yu...@gmail.com> wrote:
>>
>> http://hortonworks.com/products/hortonworks-sandbox/
>>
>> or
>>
>> CDH5
>>
>> http://www.cloudera.com/content/cloudera-content/cloudera-docs/DemoVMs/Cloudera-QuickStart-VM/cloudera_quickstart_vm.html
>>
>>
>>
>> On Sat, Jul 5, 2014 at 11:27 PM, Manar Elkady <m....@fci-cu.edu.eg>
>> wrote:
>>
>>> Hi,
>>>
>>> I am a newcomer in using Hadoop, and I read many online tutorial to set
>>> up Hadoop on Window by using virtual machines, but all of them link to old
>>> versions of Hadoop virtual machines.
>>> Could any one help me to find a Hadoop virtual machine, which include a
>>> newer version of hadoop? Or should I do it myself from scratch?
>>> Also, any well explained Hadoop installing tutorial and any other
>>> helpful material are appreciated.
>>>
>>>
>>> Manar,
>>>
>>>
>>> --
>>>
>>>
>>
>
--
André Kelpe
andre@concurrentinc.com
http://concurrentinc.com
Re: Hadoop virtual machine
Posted by Andre Kelpe <ak...@concurrentinc.com>.
We have a multi-vm or single-vm setup with apache hadoop, if you want to
give that a spin:
https://github.com/Cascading/vagrant-cascading-hadoop-cluster
- André
On Sun, Jul 6, 2014 at 9:05 AM, MrAsanjar . <af...@gmail.com> wrote:
> For my hadoop development and testing I use LXC (linux container) instead
> of VM, mainly due to its light weight resource consumption. As mater of
> fact as I am typing, my ubuntu system is automatically building a 6 nodes
> hadoop cluster on my 16G labtop.
> If you have an Ubuntu system you could install a fully configurable Hadoop
> 2.2.0 single node or multi-node cluster in less then 10 minutes.
> Here what you need to do:
> 1) Install and learn Ubuntu Juju (shouldn't take an hour)- instructions :
> https://juju.ubuntu.com/docs/getting-started.html
> 2) there are two types hadoop charms:
> a) Single node for hadoop development :
> https://jujucharms.com/?text=hadoop2-devel
> b) multi-node for testing testing :
> https://jujucharms.com/?text=hadoop
> Let me know if you need more help
>
>
> On Sun, Jul 6, 2014 at 7:59 AM, Marco Shaw <ma...@gmail.com> wrote:
>
>> Note that the CDH link is for Cloudera which only provides Hadoop for
>> Linux.
>>
>> HDP has "pre-built VMs" for both Linux and Windows hosts.
>>
>> You can also search for "HDInsight emulator" which runs on Windows and is
>> based on HDP.
>>
>> Marco
>>
>> On Jul 6, 2014, at 12:38 AM, Gavin Yue <yu...@gmail.com> wrote:
>>
>> http://hortonworks.com/products/hortonworks-sandbox/
>>
>> or
>>
>> CDH5
>>
>> http://www.cloudera.com/content/cloudera-content/cloudera-docs/DemoVMs/Cloudera-QuickStart-VM/cloudera_quickstart_vm.html
>>
>>
>>
>> On Sat, Jul 5, 2014 at 11:27 PM, Manar Elkady <m....@fci-cu.edu.eg>
>> wrote:
>>
>>> Hi,
>>>
>>> I am a newcomer in using Hadoop, and I read many online tutorial to set
>>> up Hadoop on Window by using virtual machines, but all of them link to old
>>> versions of Hadoop virtual machines.
>>> Could any one help me to find a Hadoop virtual machine, which include a
>>> newer version of hadoop? Or should I do it myself from scratch?
>>> Also, any well explained Hadoop installing tutorial and any other
>>> helpful material are appreciated.
>>>
>>>
>>> Manar,
>>>
>>>
>>> --
>>>
>>>
>>
>
--
André Kelpe
andre@concurrentinc.com
http://concurrentinc.com
Re: Hadoop virtual machine
Posted by Andre Kelpe <ak...@concurrentinc.com>.
We have a multi-vm or single-vm setup with apache hadoop, if you want to
give that a spin:
https://github.com/Cascading/vagrant-cascading-hadoop-cluster
- André
On Sun, Jul 6, 2014 at 9:05 AM, MrAsanjar . <af...@gmail.com> wrote:
> For my hadoop development and testing I use LXC (linux container) instead
> of VM, mainly due to its light weight resource consumption. As mater of
> fact as I am typing, my ubuntu system is automatically building a 6 nodes
> hadoop cluster on my 16G labtop.
> If you have an Ubuntu system you could install a fully configurable Hadoop
> 2.2.0 single node or multi-node cluster in less then 10 minutes.
> Here what you need to do:
> 1) Install and learn Ubuntu Juju (shouldn't take an hour)- instructions :
> https://juju.ubuntu.com/docs/getting-started.html
> 2) there are two types hadoop charms:
> a) Single node for hadoop development :
> https://jujucharms.com/?text=hadoop2-devel
> b) multi-node for testing testing :
> https://jujucharms.com/?text=hadoop
> Let me know if you need more help
>
>
> On Sun, Jul 6, 2014 at 7:59 AM, Marco Shaw <ma...@gmail.com> wrote:
>
>> Note that the CDH link is for Cloudera which only provides Hadoop for
>> Linux.
>>
>> HDP has "pre-built VMs" for both Linux and Windows hosts.
>>
>> You can also search for "HDInsight emulator" which runs on Windows and is
>> based on HDP.
>>
>> Marco
>>
>> On Jul 6, 2014, at 12:38 AM, Gavin Yue <yu...@gmail.com> wrote:
>>
>> http://hortonworks.com/products/hortonworks-sandbox/
>>
>> or
>>
>> CDH5
>>
>> http://www.cloudera.com/content/cloudera-content/cloudera-docs/DemoVMs/Cloudera-QuickStart-VM/cloudera_quickstart_vm.html
>>
>>
>>
>> On Sat, Jul 5, 2014 at 11:27 PM, Manar Elkady <m....@fci-cu.edu.eg>
>> wrote:
>>
>>> Hi,
>>>
>>> I am a newcomer in using Hadoop, and I read many online tutorial to set
>>> up Hadoop on Window by using virtual machines, but all of them link to old
>>> versions of Hadoop virtual machines.
>>> Could any one help me to find a Hadoop virtual machine, which include a
>>> newer version of hadoop? Or should I do it myself from scratch?
>>> Also, any well explained Hadoop installing tutorial and any other
>>> helpful material are appreciated.
>>>
>>>
>>> Manar,
>>>
>>>
>>> --
>>>
>>>
>>
>
--
André Kelpe
andre@concurrentinc.com
http://concurrentinc.com
Re: Hadoop virtual machine
Posted by Andre Kelpe <ak...@concurrentinc.com>.
We have a multi-vm or single-vm setup with apache hadoop, if you want to
give that a spin:
https://github.com/Cascading/vagrant-cascading-hadoop-cluster
- André
On Sun, Jul 6, 2014 at 9:05 AM, MrAsanjar . <af...@gmail.com> wrote:
> For my hadoop development and testing I use LXC (linux container) instead
> of VM, mainly due to its light weight resource consumption. As mater of
> fact as I am typing, my ubuntu system is automatically building a 6 nodes
> hadoop cluster on my 16G labtop.
> If you have an Ubuntu system you could install a fully configurable Hadoop
> 2.2.0 single node or multi-node cluster in less then 10 minutes.
> Here what you need to do:
> 1) Install and learn Ubuntu Juju (shouldn't take an hour)- instructions :
> https://juju.ubuntu.com/docs/getting-started.html
> 2) there are two types hadoop charms:
> a) Single node for hadoop development :
> https://jujucharms.com/?text=hadoop2-devel
> b) multi-node for testing testing :
> https://jujucharms.com/?text=hadoop
> Let me know if you need more help
>
>
> On Sun, Jul 6, 2014 at 7:59 AM, Marco Shaw <ma...@gmail.com> wrote:
>
>> Note that the CDH link is for Cloudera which only provides Hadoop for
>> Linux.
>>
>> HDP has "pre-built VMs" for both Linux and Windows hosts.
>>
>> You can also search for "HDInsight emulator" which runs on Windows and is
>> based on HDP.
>>
>> Marco
>>
>> On Jul 6, 2014, at 12:38 AM, Gavin Yue <yu...@gmail.com> wrote:
>>
>> http://hortonworks.com/products/hortonworks-sandbox/
>>
>> or
>>
>> CDH5
>>
>> http://www.cloudera.com/content/cloudera-content/cloudera-docs/DemoVMs/Cloudera-QuickStart-VM/cloudera_quickstart_vm.html
>>
>>
>>
>> On Sat, Jul 5, 2014 at 11:27 PM, Manar Elkady <m....@fci-cu.edu.eg>
>> wrote:
>>
>>> Hi,
>>>
>>> I am a newcomer in using Hadoop, and I read many online tutorial to set
>>> up Hadoop on Window by using virtual machines, but all of them link to old
>>> versions of Hadoop virtual machines.
>>> Could any one help me to find a Hadoop virtual machine, which include a
>>> newer version of hadoop? Or should I do it myself from scratch?
>>> Also, any well explained Hadoop installing tutorial and any other
>>> helpful material are appreciated.
>>>
>>>
>>> Manar,
>>>
>>>
>>> --
>>>
>>>
>>
>
--
André Kelpe
andre@concurrentinc.com
http://concurrentinc.com
Re: Hadoop virtual machine
Posted by "MrAsanjar ." <af...@gmail.com>.
For my hadoop development and testing I use LXC (linux container) instead
of VM, mainly due to its light weight resource consumption. As mater of
fact as I am typing, my ubuntu system is automatically building a 6 nodes
hadoop cluster on my 16G labtop.
If you have an Ubuntu system you could install a fully configurable Hadoop
2.2.0 single node or multi-node cluster in less then 10 minutes.
Here what you need to do:
1) Install and learn Ubuntu Juju (shouldn't take an hour)- instructions :
https://juju.ubuntu.com/docs/getting-started.html
2) there are two types hadoop charms:
a) Single node for hadoop development :
https://jujucharms.com/?text=hadoop2-devel
b) multi-node for testing testing :
https://jujucharms.com/?text=hadoop
Let me know if you need more help
On Sun, Jul 6, 2014 at 7:59 AM, Marco Shaw <ma...@gmail.com> wrote:
> Note that the CDH link is for Cloudera which only provides Hadoop for
> Linux.
>
> HDP has "pre-built VMs" for both Linux and Windows hosts.
>
> You can also search for "HDInsight emulator" which runs on Windows and is
> based on HDP.
>
> Marco
>
> On Jul 6, 2014, at 12:38 AM, Gavin Yue <yu...@gmail.com> wrote:
>
> http://hortonworks.com/products/hortonworks-sandbox/
>
> or
>
> CDH5
>
> http://www.cloudera.com/content/cloudera-content/cloudera-docs/DemoVMs/Cloudera-QuickStart-VM/cloudera_quickstart_vm.html
>
>
>
> On Sat, Jul 5, 2014 at 11:27 PM, Manar Elkady <m....@fci-cu.edu.eg>
> wrote:
>
>> Hi,
>>
>> I am a newcomer in using Hadoop, and I read many online tutorial to set
>> up Hadoop on Window by using virtual machines, but all of them link to old
>> versions of Hadoop virtual machines.
>> Could any one help me to find a Hadoop virtual machine, which include a
>> newer version of hadoop? Or should I do it myself from scratch?
>> Also, any well explained Hadoop installing tutorial and any other helpful
>> material are appreciated.
>>
>>
>> Manar,
>>
>>
>> --
>>
>>
>
Re: Hadoop virtual machine
Posted by "MrAsanjar ." <af...@gmail.com>.
For my hadoop development and testing I use LXC (linux container) instead
of VM, mainly due to its light weight resource consumption. As mater of
fact as I am typing, my ubuntu system is automatically building a 6 nodes
hadoop cluster on my 16G labtop.
If you have an Ubuntu system you could install a fully configurable Hadoop
2.2.0 single node or multi-node cluster in less then 10 minutes.
Here what you need to do:
1) Install and learn Ubuntu Juju (shouldn't take an hour)- instructions :
https://juju.ubuntu.com/docs/getting-started.html
2) there are two types hadoop charms:
a) Single node for hadoop development :
https://jujucharms.com/?text=hadoop2-devel
b) multi-node for testing testing :
https://jujucharms.com/?text=hadoop
Let me know if you need more help
On Sun, Jul 6, 2014 at 7:59 AM, Marco Shaw <ma...@gmail.com> wrote:
> Note that the CDH link is for Cloudera which only provides Hadoop for
> Linux.
>
> HDP has "pre-built VMs" for both Linux and Windows hosts.
>
> You can also search for "HDInsight emulator" which runs on Windows and is
> based on HDP.
>
> Marco
>
> On Jul 6, 2014, at 12:38 AM, Gavin Yue <yu...@gmail.com> wrote:
>
> http://hortonworks.com/products/hortonworks-sandbox/
>
> or
>
> CDH5
>
> http://www.cloudera.com/content/cloudera-content/cloudera-docs/DemoVMs/Cloudera-QuickStart-VM/cloudera_quickstart_vm.html
>
>
>
> On Sat, Jul 5, 2014 at 11:27 PM, Manar Elkady <m....@fci-cu.edu.eg>
> wrote:
>
>> Hi,
>>
>> I am a newcomer in using Hadoop, and I read many online tutorial to set
>> up Hadoop on Window by using virtual machines, but all of them link to old
>> versions of Hadoop virtual machines.
>> Could any one help me to find a Hadoop virtual machine, which include a
>> newer version of hadoop? Or should I do it myself from scratch?
>> Also, any well explained Hadoop installing tutorial and any other helpful
>> material are appreciated.
>>
>>
>> Manar,
>>
>>
>> --
>>
>>
>
Re: Hadoop virtual machine
Posted by "MrAsanjar ." <af...@gmail.com>.
For my hadoop development and testing I use LXC (linux container) instead
of VM, mainly due to its light weight resource consumption. As mater of
fact as I am typing, my ubuntu system is automatically building a 6 nodes
hadoop cluster on my 16G labtop.
If you have an Ubuntu system you could install a fully configurable Hadoop
2.2.0 single node or multi-node cluster in less then 10 minutes.
Here what you need to do:
1) Install and learn Ubuntu Juju (shouldn't take an hour)- instructions :
https://juju.ubuntu.com/docs/getting-started.html
2) there are two types hadoop charms:
a) Single node for hadoop development :
https://jujucharms.com/?text=hadoop2-devel
b) multi-node for testing testing :
https://jujucharms.com/?text=hadoop
Let me know if you need more help
On Sun, Jul 6, 2014 at 7:59 AM, Marco Shaw <ma...@gmail.com> wrote:
> Note that the CDH link is for Cloudera which only provides Hadoop for
> Linux.
>
> HDP has "pre-built VMs" for both Linux and Windows hosts.
>
> You can also search for "HDInsight emulator" which runs on Windows and is
> based on HDP.
>
> Marco
>
> On Jul 6, 2014, at 12:38 AM, Gavin Yue <yu...@gmail.com> wrote:
>
> http://hortonworks.com/products/hortonworks-sandbox/
>
> or
>
> CDH5
>
> http://www.cloudera.com/content/cloudera-content/cloudera-docs/DemoVMs/Cloudera-QuickStart-VM/cloudera_quickstart_vm.html
>
>
>
> On Sat, Jul 5, 2014 at 11:27 PM, Manar Elkady <m....@fci-cu.edu.eg>
> wrote:
>
>> Hi,
>>
>> I am a newcomer in using Hadoop, and I read many online tutorial to set
>> up Hadoop on Window by using virtual machines, but all of them link to old
>> versions of Hadoop virtual machines.
>> Could any one help me to find a Hadoop virtual machine, which include a
>> newer version of hadoop? Or should I do it myself from scratch?
>> Also, any well explained Hadoop installing tutorial and any other helpful
>> material are appreciated.
>>
>>
>> Manar,
>>
>>
>> --
>>
>>
>
Re: Hadoop virtual machine
Posted by "MrAsanjar ." <af...@gmail.com>.
For my hadoop development and testing I use LXC (linux container) instead
of VM, mainly due to its light weight resource consumption. As mater of
fact as I am typing, my ubuntu system is automatically building a 6 nodes
hadoop cluster on my 16G labtop.
If you have an Ubuntu system you could install a fully configurable Hadoop
2.2.0 single node or multi-node cluster in less then 10 minutes.
Here what you need to do:
1) Install and learn Ubuntu Juju (shouldn't take an hour)- instructions :
https://juju.ubuntu.com/docs/getting-started.html
2) there are two types hadoop charms:
a) Single node for hadoop development :
https://jujucharms.com/?text=hadoop2-devel
b) multi-node for testing testing :
https://jujucharms.com/?text=hadoop
Let me know if you need more help
On Sun, Jul 6, 2014 at 7:59 AM, Marco Shaw <ma...@gmail.com> wrote:
> Note that the CDH link is for Cloudera which only provides Hadoop for
> Linux.
>
> HDP has "pre-built VMs" for both Linux and Windows hosts.
>
> You can also search for "HDInsight emulator" which runs on Windows and is
> based on HDP.
>
> Marco
>
> On Jul 6, 2014, at 12:38 AM, Gavin Yue <yu...@gmail.com> wrote:
>
> http://hortonworks.com/products/hortonworks-sandbox/
>
> or
>
> CDH5
>
> http://www.cloudera.com/content/cloudera-content/cloudera-docs/DemoVMs/Cloudera-QuickStart-VM/cloudera_quickstart_vm.html
>
>
>
> On Sat, Jul 5, 2014 at 11:27 PM, Manar Elkady <m....@fci-cu.edu.eg>
> wrote:
>
>> Hi,
>>
>> I am a newcomer in using Hadoop, and I read many online tutorial to set
>> up Hadoop on Window by using virtual machines, but all of them link to old
>> versions of Hadoop virtual machines.
>> Could any one help me to find a Hadoop virtual machine, which include a
>> newer version of hadoop? Or should I do it myself from scratch?
>> Also, any well explained Hadoop installing tutorial and any other helpful
>> material are appreciated.
>>
>>
>> Manar,
>>
>>
>> --
>>
>>
>
Re: Hadoop virtual machine
Posted by Marco Shaw <ma...@gmail.com>.
Note that the CDH link is for Cloudera which only provides Hadoop for Linux.
HDP has "pre-built VMs" for both Linux and Windows hosts.
You can also search for "HDInsight emulator" which runs on Windows and is based on HDP.
Marco
> On Jul 6, 2014, at 12:38 AM, Gavin Yue <yu...@gmail.com> wrote:
>
> http://hortonworks.com/products/hortonworks-sandbox/
>
> or
>
> CDH5
> http://www.cloudera.com/content/cloudera-content/cloudera-docs/DemoVMs/Cloudera-QuickStart-VM/cloudera_quickstart_vm.html
>
>
>
>> On Sat, Jul 5, 2014 at 11:27 PM, Manar Elkady <m....@fci-cu.edu.eg> wrote:
>> Hi,
>>
>> I am a newcomer in using Hadoop, and I read many online tutorial to set up Hadoop on Window by using virtual machines, but all of them link to old versions of Hadoop virtual machines.
>> Could any one help me to find a Hadoop virtual machine, which include a newer version of hadoop? Or should I do it myself from scratch?
>> Also, any well explained Hadoop installing tutorial and any other helpful material are appreciated.
>>
>>
>> Manar,
>>
>>
>> --
>
Re: Hadoop virtual machine
Posted by Marco Shaw <ma...@gmail.com>.
Note that the CDH link is for Cloudera which only provides Hadoop for Linux.
HDP has "pre-built VMs" for both Linux and Windows hosts.
You can also search for "HDInsight emulator" which runs on Windows and is based on HDP.
Marco
> On Jul 6, 2014, at 12:38 AM, Gavin Yue <yu...@gmail.com> wrote:
>
> http://hortonworks.com/products/hortonworks-sandbox/
>
> or
>
> CDH5
> http://www.cloudera.com/content/cloudera-content/cloudera-docs/DemoVMs/Cloudera-QuickStart-VM/cloudera_quickstart_vm.html
>
>
>
>> On Sat, Jul 5, 2014 at 11:27 PM, Manar Elkady <m....@fci-cu.edu.eg> wrote:
>> Hi,
>>
>> I am a newcomer in using Hadoop, and I read many online tutorial to set up Hadoop on Window by using virtual machines, but all of them link to old versions of Hadoop virtual machines.
>> Could any one help me to find a Hadoop virtual machine, which include a newer version of hadoop? Or should I do it myself from scratch?
>> Also, any well explained Hadoop installing tutorial and any other helpful material are appreciated.
>>
>>
>> Manar,
>>
>>
>> --
>
Re: Hadoop virtual machine
Posted by Manar Elkady <m....@fci-cu.edu.eg>.
Thanks, for your help.
On 7/6/2014 5:38 AM, Gavin Yue wrote:
> http://hortonworks.com/products/hortonworks-sandbox/
>
> or
>
> CDH5
> http://www.cloudera.com/content/cloudera-content/cloudera-docs/DemoVMs/Cloudera-QuickStart-VM/cloudera_quickstart_vm.html
>
>
>
> On Sat, Jul 5, 2014 at 11:27 PM, Manar Elkady <m.elkady@fci-cu.edu.eg
> <ma...@fci-cu.edu.eg>> wrote:
>
> Hi,
>
> I am a newcomer in using Hadoop, and I read many online tutorial
> to set up Hadoop on Window by using virtual machines, but all of
> them link to old versions of Hadoop virtual machines.
> Could any one help me to find a Hadoop virtual machine, which
> include a newer version of hadoop? Or should I do it myself from
> scratch?
> Also, any well explained Hadoop installing tutorial and any other
> helpful material are appreciated.
>
>
> Manar,
>
>
> --
>
>
--
Re: Hadoop virtual machine
Posted by Manar Elkady <m....@fci-cu.edu.eg>.
Thanks, for your help.
On 7/6/2014 5:38 AM, Gavin Yue wrote:
> http://hortonworks.com/products/hortonworks-sandbox/
>
> or
>
> CDH5
> http://www.cloudera.com/content/cloudera-content/cloudera-docs/DemoVMs/Cloudera-QuickStart-VM/cloudera_quickstart_vm.html
>
>
>
> On Sat, Jul 5, 2014 at 11:27 PM, Manar Elkady <m.elkady@fci-cu.edu.eg
> <ma...@fci-cu.edu.eg>> wrote:
>
> Hi,
>
> I am a newcomer in using Hadoop, and I read many online tutorial
> to set up Hadoop on Window by using virtual machines, but all of
> them link to old versions of Hadoop virtual machines.
> Could any one help me to find a Hadoop virtual machine, which
> include a newer version of hadoop? Or should I do it myself from
> scratch?
> Also, any well explained Hadoop installing tutorial and any other
> helpful material are appreciated.
>
>
> Manar,
>
>
> --
>
>
--
Re: Hadoop virtual machine
Posted by Manar Elkady <m....@fci-cu.edu.eg>.
Thanks, for your help.
On 7/6/2014 5:38 AM, Gavin Yue wrote:
> http://hortonworks.com/products/hortonworks-sandbox/
>
> or
>
> CDH5
> http://www.cloudera.com/content/cloudera-content/cloudera-docs/DemoVMs/Cloudera-QuickStart-VM/cloudera_quickstart_vm.html
>
>
>
> On Sat, Jul 5, 2014 at 11:27 PM, Manar Elkady <m.elkady@fci-cu.edu.eg
> <ma...@fci-cu.edu.eg>> wrote:
>
> Hi,
>
> I am a newcomer in using Hadoop, and I read many online tutorial
> to set up Hadoop on Window by using virtual machines, but all of
> them link to old versions of Hadoop virtual machines.
> Could any one help me to find a Hadoop virtual machine, which
> include a newer version of hadoop? Or should I do it myself from
> scratch?
> Also, any well explained Hadoop installing tutorial and any other
> helpful material are appreciated.
>
>
> Manar,
>
>
> --
>
>
--
Re: Hadoop virtual machine
Posted by Marco Shaw <ma...@gmail.com>.
Note that the CDH link is for Cloudera which only provides Hadoop for Linux.
HDP has "pre-built VMs" for both Linux and Windows hosts.
You can also search for "HDInsight emulator" which runs on Windows and is based on HDP.
Marco
> On Jul 6, 2014, at 12:38 AM, Gavin Yue <yu...@gmail.com> wrote:
>
> http://hortonworks.com/products/hortonworks-sandbox/
>
> or
>
> CDH5
> http://www.cloudera.com/content/cloudera-content/cloudera-docs/DemoVMs/Cloudera-QuickStart-VM/cloudera_quickstart_vm.html
>
>
>
>> On Sat, Jul 5, 2014 at 11:27 PM, Manar Elkady <m....@fci-cu.edu.eg> wrote:
>> Hi,
>>
>> I am a newcomer in using Hadoop, and I read many online tutorial to set up Hadoop on Window by using virtual machines, but all of them link to old versions of Hadoop virtual machines.
>> Could any one help me to find a Hadoop virtual machine, which include a newer version of hadoop? Or should I do it myself from scratch?
>> Also, any well explained Hadoop installing tutorial and any other helpful material are appreciated.
>>
>>
>> Manar,
>>
>>
>> --
>
Re: Hadoop virtual machine
Posted by Marco Shaw <ma...@gmail.com>.
Note that the CDH link is for Cloudera which only provides Hadoop for Linux.
HDP has "pre-built VMs" for both Linux and Windows hosts.
You can also search for "HDInsight emulator" which runs on Windows and is based on HDP.
Marco
> On Jul 6, 2014, at 12:38 AM, Gavin Yue <yu...@gmail.com> wrote:
>
> http://hortonworks.com/products/hortonworks-sandbox/
>
> or
>
> CDH5
> http://www.cloudera.com/content/cloudera-content/cloudera-docs/DemoVMs/Cloudera-QuickStart-VM/cloudera_quickstart_vm.html
>
>
>
>> On Sat, Jul 5, 2014 at 11:27 PM, Manar Elkady <m....@fci-cu.edu.eg> wrote:
>> Hi,
>>
>> I am a newcomer in using Hadoop, and I read many online tutorial to set up Hadoop on Window by using virtual machines, but all of them link to old versions of Hadoop virtual machines.
>> Could any one help me to find a Hadoop virtual machine, which include a newer version of hadoop? Or should I do it myself from scratch?
>> Also, any well explained Hadoop installing tutorial and any other helpful material are appreciated.
>>
>>
>> Manar,
>>
>>
>> --
>
Re: Hadoop virtual machine
Posted by Gavin Yue <yu...@gmail.com>.
http://hortonworks.com/products/hortonworks-sandbox/
or
CDH5
http://www.cloudera.com/content/cloudera-content/cloudera-docs/DemoVMs/Cloudera-QuickStart-VM/cloudera_quickstart_vm.html
On Sat, Jul 5, 2014 at 11:27 PM, Manar Elkady <m....@fci-cu.edu.eg>
wrote:
> Hi,
>
> I am a newcomer in using Hadoop, and I read many online tutorial to set up
> Hadoop on Window by using virtual machines, but all of them link to old
> versions of Hadoop virtual machines.
> Could any one help me to find a Hadoop virtual machine, which include a
> newer version of hadoop? Or should I do it myself from scratch?
> Also, any well explained Hadoop installing tutorial and any other helpful
> material are appreciated.
>
>
> Manar,
>
>
> --
>
>
Re: Hadoop virtual machine
Posted by Gavin Yue <yu...@gmail.com>.
http://hortonworks.com/products/hortonworks-sandbox/
or
CDH5
http://www.cloudera.com/content/cloudera-content/cloudera-docs/DemoVMs/Cloudera-QuickStart-VM/cloudera_quickstart_vm.html
On Sat, Jul 5, 2014 at 11:27 PM, Manar Elkady <m....@fci-cu.edu.eg>
wrote:
> Hi,
>
> I am a newcomer in using Hadoop, and I read many online tutorial to set up
> Hadoop on Window by using virtual machines, but all of them link to old
> versions of Hadoop virtual machines.
> Could any one help me to find a Hadoop virtual machine, which include a
> newer version of hadoop? Or should I do it myself from scratch?
> Also, any well explained Hadoop installing tutorial and any other helpful
> material are appreciated.
>
>
> Manar,
>
>
> --
>
>
Re: Hadoop virtual machine
Posted by Gavin Yue <yu...@gmail.com>.
http://hortonworks.com/products/hortonworks-sandbox/
or
CDH5
http://www.cloudera.com/content/cloudera-content/cloudera-docs/DemoVMs/Cloudera-QuickStart-VM/cloudera_quickstart_vm.html
On Sat, Jul 5, 2014 at 11:27 PM, Manar Elkady <m....@fci-cu.edu.eg>
wrote:
> Hi,
>
> I am a newcomer in using Hadoop, and I read many online tutorial to set up
> Hadoop on Window by using virtual machines, but all of them link to old
> versions of Hadoop virtual machines.
> Could any one help me to find a Hadoop virtual machine, which include a
> newer version of hadoop? Or should I do it myself from scratch?
> Also, any well explained Hadoop installing tutorial and any other helpful
> material are appreciated.
>
>
> Manar,
>
>
> --
>
>
Re: Hadoop virtual machine
Posted by Gavin Yue <yu...@gmail.com>.
http://hortonworks.com/products/hortonworks-sandbox/
or
CDH5
http://www.cloudera.com/content/cloudera-content/cloudera-docs/DemoVMs/Cloudera-QuickStart-VM/cloudera_quickstart_vm.html
On Sat, Jul 5, 2014 at 11:27 PM, Manar Elkady <m....@fci-cu.edu.eg>
wrote:
> Hi,
>
> I am a newcomer in using Hadoop, and I read many online tutorial to set up
> Hadoop on Window by using virtual machines, but all of them link to old
> versions of Hadoop virtual machines.
> Could any one help me to find a Hadoop virtual machine, which include a
> newer version of hadoop? Or should I do it myself from scratch?
> Also, any well explained Hadoop installing tutorial and any other helpful
> material are appreciated.
>
>
> Manar,
>
>
> --
>
>