How well does HBase handle heterogeneous clusters?

[Devin Bayer <lu...@doubly.so>]

Hi everyone,

We are planning to expand our Hadoop/HBase cluster and have been researching the consequences of adding more powerful nodes on HBase performance. As far as I have been able to determine, HBase balances load evenly across nodes, regardless of server performance. So this could result in the old systems being overloaded while the new ones site idle.

Are our assumptions correct or is there a load balancer that could take full advantage or newer, bigger hardware alongside our current systems?

Thanks a lot!
Devin Bayer
RIPE NCC

Re: How well does HBase handle heterogeneous clusters?

[Jean-Marc Spaggiari <je...@spaggiari.org>]

Hi Devin,

Your assumptions are right. However, the overall performances will
still increase since you will split the work a bit more between all
the nodes. But at the end, it will run as fast as the flowest node.

On my dev cluster I have 6 notes. The slowest one is a 500MB P4 2.5Mhz
and the fastest one is a 12G 8CPU 3.4Ghz....

When I run a MR job, the slowest one is always the last to finish. But
the more the regions are spread over the cluster, the less the slowest
node has to process, and the faster the MR job is.

Later, at some point, you might need to think about removing the
slowest node to increase the overall performances.

JM

2012/10/12, Devin Bayer <lu...@doubly.so>:
> Hi everyone,
>
> We are planning to expand our Hadoop/HBase cluster and have been researching
> the consequences of adding more powerful nodes on HBase performance. As far
> as I have been able to determine, HBase balances load evenly across nodes,
> regardless of server performance. So this could result in the old systems
> being overloaded while the new ones site idle.
>
> Are our assumptions correct or is there a load balancer that could take full
> advantage or newer, bigger hardware alongside our current systems?
>
> Thanks a lot!
> Devin Bayer
> RIPE NCC