hypertable routs hbase

[David Medinets <da...@gmail.com>]

http://highscalability.com/blog/2012/2/7/hypertable-routs-hbase-in-performance-test-hbase-overwhelmed.html

The test was run on a total of sixteen machines connected together
with gigabit Ethernet.. HBase failed to handle 41 and 167 billion
record inserts, overwhelmed by garbage collection

I haven't used Accumulo enough to know if the same garbage collection
issue would arise with it.

Re: hypertable routs hbase

[Billie J Rinaldi <bi...@ugov.gov>]

On Thursday, February 9, 2012 1:37:25 PM, "David Medinets" <da...@gmail.com>:
> http://highscalability.com/blog/2012/2/7/hypertable-routs-hbase-in-performance-test-hbase-overwhelmed.html
> 
> The test was run on a total of sixteen machines connected together
> with gigabit Ethernet.. HBase failed to handle 41 and 167 billion
> record inserts, overwhelmed by garbage collection
> 
> I haven't used Accumulo enough to know if the same garbage collection
> issue would arise with it.

It's unfortunate that they've tested the brand-new version of Hypertable against a 6-month-old version of HBase.
Accumulo deals with Java garbage collection issues by not storing the in-memory maps in Java managed memory, instead using a C++ STL map of maps.  This will be used as long as the native map library is built and available on the system; if it isn't available, Accumulo falls back to using Java managed memory.  So it is important when configuring a new instance to make sure to sort out the native map library.

Billie