performance regression after hbase restart

[Tao Xie <xi...@gmail.com>]

hi, I know regions will be reassigned when hbase cluster restarts. My
regionserver and my datanode sit on same physical node. So in my tests after
I restart hbase cluster, performance number drops, I guess this is due to
data locality problem. But in a further experiment, I increase the
replication factor to cluster size ( I have 8 nodes). After restart and
major compaction, the performance still cannot catch up with the best
scenario. I checked datanode log, most hdfs read has same src and dest node.

Anyone give me some hints on this? I wonder if anything happens during
region reassignment.

Thanks.

Tao

RE: performance regression after hbase restart

[Jonathan Gray <jg...@fb.com>]

In HBase 0.90.0 there is a new "retain assignment" configuration parameter that makes it so your cluster keeps the same region assignment between full cluster restarts.  It is ON by default.

JG

> -----Original Message-----
> From: Tao Xie [mailto:xietao.mailbox@gmail.com]
> Sent: Thursday, January 20, 2011 6:37 PM
> To: user@hbase.apache.org
> Subject: performance regression after hbase restart
> 
> hi, I know regions will be reassigned when hbase cluster restarts. My
> regionserver and my datanode sit on same physical node. So in my tests after
> I restart hbase cluster, performance number drops, I guess this is due to data
> locality problem. But in a further experiment, I increase the replication factor
> to cluster size ( I have 8 nodes). After restart and major compaction, the
> performance still cannot catch up with the best scenario. I checked datanode
> log, most hdfs read has same src and dest node.
> 
> Anyone give me some hints on this? I wonder if anything happens during
> region reassignment.
> 
> Thanks.
> 
> Tao