You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-user@hadoop.apache.org by Amjad Syed <am...@gmail.com> on 2014/12/07 09:08:09 UTC
Hardware requirements for simple node hadoop cluster
Hello,
We are trying to do a proof of concept at our Data center with two node
hadoop cluster.
We have two (dual socket quad core ) HP proliant DL 380G6 servers we
want to utilize for this test.
Can any one please recommend the minimum HDD and RAM requirements for
both servers ?
Thanks
Re: Hardware requirements for simple node hadoop cluster
Posted by Alexander Pivovarov <ap...@gmail.com>.
for balanced conf you need (per core)
1-1.5 2 GB 7200 SATA hdd for hdfs (in JBOD mode, not RAID)
3-4 GB RAM ECC
reserve 4GB RAM for OS
better to use separate hdd or usb stick for OS
e.g. for 16 cores you can use
16-24 2GB hdds
64 GB RAM (if planing to use Apache Spark put 128 GB)
On Sun, Dec 7, 2014 at 12:08 AM, Amjad Syed <am...@gmail.com> wrote:
> Hello,
>
> We are trying to do a proof of concept at our Data center with two node
> hadoop cluster.
>
> We have two (dual socket quad core ) HP proliant DL 380G6 servers we
> want to utilize for this test.
>
> Can any one please recommend the minimum HDD and RAM requirements for
> both servers ?
>
> Thanks
>
Re: Hardware requirements for simple node hadoop cluster
Posted by Alexander Pivovarov <ap...@gmail.com>.
for balanced conf you need (per core)
1-1.5 2 GB 7200 SATA hdd for hdfs (in JBOD mode, not RAID)
3-4 GB RAM ECC
reserve 4GB RAM for OS
better to use separate hdd or usb stick for OS
e.g. for 16 cores you can use
16-24 2GB hdds
64 GB RAM (if planing to use Apache Spark put 128 GB)
On Sun, Dec 7, 2014 at 12:08 AM, Amjad Syed <am...@gmail.com> wrote:
> Hello,
>
> We are trying to do a proof of concept at our Data center with two node
> hadoop cluster.
>
> We have two (dual socket quad core ) HP proliant DL 380G6 servers we
> want to utilize for this test.
>
> Can any one please recommend the minimum HDD and RAM requirements for
> both servers ?
>
> Thanks
>
Re: Hardware requirements for simple node hadoop cluster
Posted by Alexander Pivovarov <ap...@gmail.com>.
for balanced conf you need (per core)
1-1.5 2 GB 7200 SATA hdd for hdfs (in JBOD mode, not RAID)
3-4 GB RAM ECC
reserve 4GB RAM for OS
better to use separate hdd or usb stick for OS
e.g. for 16 cores you can use
16-24 2GB hdds
64 GB RAM (if planing to use Apache Spark put 128 GB)
On Sun, Dec 7, 2014 at 12:08 AM, Amjad Syed <am...@gmail.com> wrote:
> Hello,
>
> We are trying to do a proof of concept at our Data center with two node
> hadoop cluster.
>
> We have two (dual socket quad core ) HP proliant DL 380G6 servers we
> want to utilize for this test.
>
> Can any one please recommend the minimum HDD and RAM requirements for
> both servers ?
>
> Thanks
>
Re: Hardware requirements for simple node hadoop cluster
Posted by Alexander Pivovarov <ap...@gmail.com>.
for balanced conf you need (per core)
1-1.5 2 GB 7200 SATA hdd for hdfs (in JBOD mode, not RAID)
3-4 GB RAM ECC
reserve 4GB RAM for OS
better to use separate hdd or usb stick for OS
e.g. for 16 cores you can use
16-24 2GB hdds
64 GB RAM (if planing to use Apache Spark put 128 GB)
On Sun, Dec 7, 2014 at 12:08 AM, Amjad Syed <am...@gmail.com> wrote:
> Hello,
>
> We are trying to do a proof of concept at our Data center with two node
> hadoop cluster.
>
> We have two (dual socket quad core ) HP proliant DL 380G6 servers we
> want to utilize for this test.
>
> Can any one please recommend the minimum HDD and RAM requirements for
> both servers ?
>
> Thanks
>