[jira] Commented: (HADOOP-1149) DFS Scalability: high cpu usage in addStoredBlock

["Raghu Angadi (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HADOOP-1149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12485015 ] 

Raghu Angadi commented on HADOOP-1149:
--------------------------------------

Each addStoredBlock() invokes processOverReplicatedBlock(), which can be made a no-op in common case easily. Currently it does some allocation and iterations. But it does not seem more than rest of addStoredBlock(). Did you find processOverReplicatedBlock() in stack trace most of the time when you observed this?

Let me know when you suspect this again, and we can check the trace. 


> DFS Scalability: high cpu usage in addStoredBlock
> -------------------------------------------------
>
>                 Key: HADOOP-1149
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1149
>             Project: Hadoop
>          Issue Type: Bug
>          Components: dfs
>            Reporter: dhruba borthakur
>         Assigned To: Raghu Angadi
>
> I have seen addStoredBlock() consume lots of CPU. One possible cause is that it invokes proccessOverReplicatedBlock. The logic to find and purge over-replicated blocks can be done much less often.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.