You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@pig.apache.org by jagaran das <ja...@yahoo.co.in> on 2011/08/07 07:40:51 UTC

NameNode Profiling Tools

Hi,

Please suggest what would be the best way to profile NameNode?
Any specific tools.
We would streaming transaction data using around 2000 threads concurrently to NameNode continuously. Size is around 300 KB/transaction
I am using DataInputStream and writing continuously for through each 2000 connections for 5 mins and then closing and reopening again new 2000 connections.
 Any benchmarks on CPU and Mem utilization of NameNode ?

My NameNode Box Config:
1. HPDL360 G7 2X2.66GHz CPU's, 72 GB RAM, 8X300GB Drives.

Regards,
JD

Re: NameNode Profiling Tools

Posted by Adi <ad...@gmail.com>.

>>Please suggest what would be the best way to profile NameNode?
>>Any specific tools.

http://developer.yahoo.com/hadoop/tutorial/module7.html#monitoring

Ganglia and Nagios(or any other system monitoring you have in place)

-Adi

On Sun, Aug 7, 2011 at 1:40 AM, jagaran das <ja...@yahoo.co.in> wrote:

> Hi,
>
> Please suggest what would be the best way to profile NameNode?
> Any specific tools.
> We would streaming transaction data using around 2000 threads concurrently
> to NameNode continuously. Size is around 300 KB/transaction
> I am using DataInputStream and writing continuously for through each 2000
> connections for 5 mins and then closing and reopening again new 2000
> connections.
>  Any benchmarks on CPU and Mem utilization of NameNode ?
>
> My NameNode Box Config:
> 1. HPDL360 G7 2X2.66GHz CPU's, 72 GB RAM, 8X300GB Drives.
>
> Regards,
> JD