Hardware - please sanity check?

[tim robertson <ti...@gmail.com>]

[Sorry to the core-hadoop members for cross posting]

Hi all,

I am not a hardware guy but about to set up a 10 node cluster for some
processing of (mostly) tab files, generating various indexes and
researching HBase, Mahout, pig, hive etc.

Could someone please sanity check that these specs look sensible?
[I know 4 drives would be better but price is a factor (second hand
not an option, hosting is not either as there is very good bandwidth
provided)]

Something along the lines of:

Dell R200 (8GB is max memory)
Quad Core Intel® Xeon® X3360, 2.83GHz, 2x6MB Cache, 1333MHz FSB
8GB Memory, DDR2, 800MHz (4x2GB Dual Ranked DIMMs)
2x 500GB 7.200 rpm 3.5-inch SATA Hard Drive


Dell R300 (can be expanded to 24GB RAM)
Quad Core Intel® Xeon® X3363, 2.83GHz, 2x6M Cache, 1333MHz FS
8GB Memory, DDR2, 667MHz (2x4GB Dual Ranked DIMMs)
2x 500GB 7.200 rpm 3.5-inch SATA Hard Drive


If there is a major flaw please can you let me know.

Thanks,

Tim
(not a hardware guy ;o)

RE: Hardware - please sanity check?

[Jonathan Gray <jl...@streamy.com>]

Tim,

Looks good to me.  Our cluster runs on very similar hardware.

The only issue we've had is if you're going to be doing large numbers of
MapReduce jobs, especially CPU bound, you might consider a jump to 2 x quad
core.  Otherwise you run into starvation issues with just a couple tasks per
node.

Other than that, you should have no problem.  We've got bunches of R200's
here and have not had any problems.

JG

> -----Original Message-----
> From: tim robertson [mailto:timrobertson100@gmail.com]
> Sent: Thursday, April 02, 2009 6:20 AM
> To: hbase-user@hadoop.apache.org
> Subject: Hardware - please sanity check?
> 
> [Sorry to the core-hadoop members for cross posting]
> 
> Hi all,
> 
> I am not a hardware guy but about to set up a 10 node cluster for some
> processing of (mostly) tab files, generating various indexes and
> researching HBase, Mahout, pig, hive etc.
> 
> Could someone please sanity check that these specs look sensible?
> [I know 4 drives would be better but price is a factor (second hand
> not an option, hosting is not either as there is very good bandwidth
> provided)]
> 
> Something along the lines of:
> 
> Dell R200 (8GB is max memory)
> Quad Core IntelR XeonR X3360, 2.83GHz, 2x6MB Cache, 1333MHz FSB
> 8GB Memory, DDR2, 800MHz (4x2GB Dual Ranked DIMMs)
> 2x 500GB 7.200 rpm 3.5-inch SATA Hard Drive
> 
> 
> Dell R300 (can be expanded to 24GB RAM)
> Quad Core IntelR XeonR X3363, 2.83GHz, 2x6M Cache, 1333MHz FS
> 8GB Memory, DDR2, 667MHz (2x4GB Dual Ranked DIMMs)
> 2x 500GB 7.200 rpm 3.5-inch SATA Hard Drive
> 
> 
> If there is a major flaw please can you let me know.
> 
> Thanks,
> 
> Tim
> (not a hardware guy ;o)