You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@hbase.apache.org by Sam Seigal <se...@yahoo.com> on 2011/10/14 23:45:51 UTC

memory requirements for daemons

Hi All,

I have the Datanode, JobTracker and RegionServer daemons running on a
fleet of machines. Each of these machines have 8 G of memory and are
dedicated hardware for running HBase. How do you guys decided which %
of memory to assign to each ? What should this number be dependent on
?

Thank you,

Sam

Re: memory requirements for daemons

Posted by Sam Seigal <se...@yahoo.com>.

Sorry, I meant *TaskTracker. The main purpose of the HBase cluster it
to run Map-Reduce jobs. The random read load is fairly low. Currently,
I have allocates 5 G to the Region Server, and 500 MB to Task Tracker
and 500 MB to DataNode.

On Fri, Oct 14, 2011 at 2:51 PM, Arun C Murthy <ac...@hortonworks.com> wrote:
> You mean DataNode, _TaskTracker_ and RegionServer I assume.
>
> DN and TT should not need more than 1G each - even that is excessive. You want much of your resources free for actual work (map/reduce tasks if you are running Hadoop MR).
>
> I'll let HBase experts comment on RS.
>
> Arun
>
> On Oct 14, 2011, at 2:45 PM, Sam Seigal wrote:
>
>> Hi All,
>>
>> I have the Datanode, JobTracker and RegionServer daemons running on a
>> fleet of machines. Each of these machines have 8 G of memory and are
>> dedicated hardware for running HBase. How do you guys decided which %
>> of memory to assign to each ? What should this number be dependent on
>> ?
>>
>> Thank you,
>>
>> Sam
>
>

Re: memory requirements for daemons

Posted by Arun C Murthy <ac...@hortonworks.com>.

You mean DataNode, _TaskTracker_ and RegionServer I assume.

DN and TT should not need more than 1G each - even that is excessive. You want much of your resources free for actual work (map/reduce tasks if you are running Hadoop MR).

I'll let HBase experts comment on RS.

Arun

On Oct 14, 2011, at 2:45 PM, Sam Seigal wrote:

> Hi All,
> 
> I have the Datanode, JobTracker and RegionServer daemons running on a
> fleet of machines. Each of these machines have 8 G of memory and are
> dedicated hardware for running HBase. How do you guys decided which %
> of memory to assign to each ? What should this number be dependent on
> ?
> 
> Thank you,
> 
> Sam