You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Raghu Angadi (JIRA)" <ji...@apache.org> on 2007/01/11 20:26:27 UTC
[jira] Commented: (HADOOP-885) Reduce CPU usage on namenode:
gettimeofday
[ https://issues.apache.org/jira/browse/HADOOP-885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12463993 ]
Raghu Angadi commented on HADOOP-885:
-------------------------------------
In addition, isn't RPC supposed to use persistent connections? From the stats above seem to using new connection for every RPC call.
> Reduce CPU usage on namenode: gettimeofday
> ------------------------------------------
>
> Key: HADOOP-885
> URL: https://issues.apache.org/jira/browse/HADOOP-885
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.10.1
> Reporter: dhruba borthakur
> Assigned To: dhruba borthakur
>
> On a 900 node idle cluster, the namenode spends about 20% of CPU. Most of this CPU is spent processing pure heartbeats. No jobs are running on this cluster and all nodes are alive and acting well.
> Of the total namenode CPU usage, about 12% is in usermode and about 70% is in kernel mode! The question that natually arises is why is heartbeat processing taking so much time in kernel mode?
> An strace of namenode reveals that a 20 second period has about 52000 syscalls with the following breakup:
> gettimeofday : 18000 calls
> accept : 2655 calls
> close : 2655 calls
> shutdown : 2655 calls
> fcntl : 7965 calls
> read : 7965 calls
> futex : 5295 calls
> poll : 4894 calls
> A code inspection reveals that the code is doing multiple (about 5) calls to System.currentTimeMillis() in processing a single request in the RPC.java and Server.java classes. This might mean that there is a possibility of optimization.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira