Scalability on multi-core machines

[Marcus Chou <ch...@gmail.com>]

Hi all,
As far as I know, one user account could have only one HBase instance,
either HMaster or HRegionServer, run on a single machine. My question is,
how many cores could HBase utilize, or rather could HBase scale on
multi-core machines?

Thanks in advance.

-- 
Best regards,
Mark

Re: Scalability on multi-core machines

[Lars George <la...@gmail.com>]

It sure does. You have a number of handler threads that serve multiple clients. As with Hadoop you can easily saturate on IO so that the core count is actually only helpful depending if you have a chance to use them properly. That is also why you must have if you can a spindle per core. Must run with less but you want to monitor your actual use and adjust hardware accordingly.

Lars

On Nov 28, 2010, at 19:47, Jack Levin <ma...@gmail.com> wrote:

> I run master, regionserver and zookeeper on one server.  Good question though - does regionserver run multi-threaded?
> 
> -Jack
> 
> 
> On Nov 28, 2010, at 4:44 AM, Marcus Chou <ch...@gmail.com> wrote:
> 
>> Hi all,
>> As far as I know, one user account could have only one HBase instance,
>> either HMaster or HRegionServer, run on a single machine. My question is,
>> how many cores could HBase utilize, or rather could HBase scale on
>> multi-core machines?
>> 
>> Thanks in advance.
>> 
>> -- 
>> Best regards,
>> Mark

RE: Scalability on multi-core machines

[Jonathan Gray <jg...@fb.com>]

The recommended setup if you want to put RS and ZK on the same node, is to ensure ZK has its own dedicated disk.

> -----Original Message-----
> From: Michael Segel [mailto:michael_segel@hotmail.com]
> Sent: Tuesday, November 30, 2010 5:35 AM
> To: user@hbase.apache.org
> Subject: RE: Scalability on multi-core machines
> 
> 
> 
> 
> > Subject: Re: Scalability on multi-core machines
> > From: magnito@gmail.com
> > Date: Sun, 28 Nov 2010 10:47:22 -0800
> > To: user@hbase.apache.org
> >
> > I run master, regionserver and zookeeper on one server.  Good question
> though - does regionserver run multi-threaded?
> >
> > -Jack
> >
> Don't.
> 
> Or rather don't run a Regionserver and Zookeeper on the same machine.
> From our experience... we're disk i/o bound before we're cpu bound and
> network bound.
> 
> We run hbase on machines that are also running m/r TT and DN.
> 

RE: Scalability on multi-core machines

[Michael Segel <mi...@hotmail.com>]

> Subject: Re: Scalability on multi-core machines
> From: magnito@gmail.com
> Date: Sun, 28 Nov 2010 10:47:22 -0800
> To: user@hbase.apache.org
> 
> I run master, regionserver and zookeeper on one server.  Good question though - does regionserver run multi-threaded?
> 
> -Jack
> 
Don't.

Or rather don't run a Regionserver and Zookeeper on the same machine.
>From our experience... we're disk i/o bound before we're cpu bound and network bound.

We run hbase on machines that are also running m/r TT and DN.
 		 	   		  

Re: Scalability on multi-core machines

[Jack Levin <ma...@gmail.com>]

I run master, regionserver and zookeeper on one server.  Good question though - does regionserver run multi-threaded?

-Jack


On Nov 28, 2010, at 4:44 AM, Marcus Chou <ch...@gmail.com> wrote:

> Hi all,
> As far as I know, one user account could have only one HBase instance,
> either HMaster or HRegionServer, run on a single machine. My question is,
> how many cores could HBase utilize, or rather could HBase scale on
> multi-core machines?
> 
> Thanks in advance.
> 
> -- 
> Best regards,
> Mark