You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by kaveh minooie <ka...@plutoz.com> on 2013/04/23 00:58:52 UTC

number of zookeeper connections, how many is too many?

Hi

I was just wondering if what I am seeing in my cluster makes sense. I 
have a hadoop cluster with 10 nodes and I am running 10 regionserver on 
top them as well. in my zoo keeper configuration I choose to allow 
unlimited number of connection mostly to see how high it actually goes. 
now, I run 8 map task on each of my node to a total of 80 concurrent map 
tasks, and my hbase regionservers each have a bit short of 200 regions 
on each server for a total of 1838 (or something) all belonging to only 
one table.

right after bringing up the hbase or when no mapreduce (or anyother 
client )is using the hbase the number of connection is always 23. when i 
run a mapreduce job that basically goes over the entire talbe ( has 1800 
something map tasks). I see ( in the zk_dump on hbase master web 
interface ) that the number of connections goes up to about 390ish.

I am new to this, so my main question is first does this makes sense? or 
am i doing something wrong? cause I don't understand why each region 
server has to establish more than one connection.

thanks,