You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Suresh Srinivas (JIRA)" <ji...@apache.org> on 2009/05/22 19:20:45 UTC

[jira] Created: (HADOOP-5897) Add more Metrics to Namenode to capture heap usage

Add more Metrics to Namenode to capture heap usage
--------------------------------------------------

                 Key: HADOOP-5897
                 URL: https://issues.apache.org/jira/browse/HADOOP-5897
             Project: Hadoop Core
          Issue Type: Improvement
            Reporter: Suresh Srinivas
            Assignee: Suresh Srinivas


Recently we had GC issues, where Namenode used more heap than usual. There was no growth indicated by the data in current Metrics to justify the heap usage. Adding more stats such as:
- Counter to track blocks that are pending deletion
- BlocksMap hashmap capacity
- Counter to track excess number of blocks 


-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (HADOOP-5897) Add more Metrics to Namenode to capture heap usage

Posted by "Suresh Srinivas (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HADOOP-5897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12719646#action_12719646 ] 

Suresh Srinivas commented on HADOOP-5897:
-----------------------------------------

The failed tests are unrelated to this patch:
* TestJobHistory - tracked in HADOOP-5920
* TestHdfsProxy - tracked in HADOOP-5837 

> Add more Metrics to Namenode to capture heap usage
> --------------------------------------------------
>
>                 Key: HADOOP-5897
>                 URL: https://issues.apache.org/jira/browse/HADOOP-5897
>             Project: Hadoop Core
>          Issue Type: Improvement
>            Reporter: Suresh Srinivas
>            Assignee: Suresh Srinivas
>         Attachments: stats.1.patch, stats.patch, stats.patch
>
>
> Recently we had GC issues, where Namenode used more heap than usual. There was no growth indicated by the data in current Metrics to justify the heap usage. Adding more stats such as:
> - Counter to track blocks that are pending deletion
> - BlocksMap hashmap capacity
> - Counter to track excess number of blocks 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (HADOOP-5897) Add more Metrics to Namenode to capture heap usage

Posted by "Suresh Srinivas (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HADOOP-5897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Suresh Srinivas updated HADOOP-5897:
------------------------------------

    Status: Patch Available  (was: Open)

> Add more Metrics to Namenode to capture heap usage
> --------------------------------------------------
>
>                 Key: HADOOP-5897
>                 URL: https://issues.apache.org/jira/browse/HADOOP-5897
>             Project: Hadoop Core
>          Issue Type: Improvement
>            Reporter: Suresh Srinivas
>            Assignee: Suresh Srinivas
>         Attachments: stats.1.patch, stats.patch, stats.patch
>
>
> Recently we had GC issues, where Namenode used more heap than usual. There was no growth indicated by the data in current Metrics to justify the heap usage. Adding more stats such as:
> - Counter to track blocks that are pending deletion
> - BlocksMap hashmap capacity
> - Counter to track excess number of blocks 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (HADOOP-5897) Add more Metrics to Namenode to capture heap usage

Posted by "Suresh Srinivas (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HADOOP-5897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Suresh Srinivas updated HADOOP-5897:
------------------------------------

    Attachment: stats.patch

# Moved {{CorruptBlocks}} metrics from NameNodeMetrics to FSNamesystemMetrics, where other related metrics are maintained.
# Added the following metrics:
#* PendingDeletionBlocks - tracks the number of block pending deletion 
#* ExcessBlocks - blocks that have more replicas than the required replication factor defined for the file
#* BlockCapacity - this is the capacity of the hashmap where the blocks are stored
# Added testcases for testing FSNamesystemMetrics

The new metrics indicates the data structures that can grow to large size and will help in corelating the heap growth indicated in GC logs.


> Add more Metrics to Namenode to capture heap usage
> --------------------------------------------------
>
>                 Key: HADOOP-5897
>                 URL: https://issues.apache.org/jira/browse/HADOOP-5897
>             Project: Hadoop Core
>          Issue Type: Improvement
>            Reporter: Suresh Srinivas
>            Assignee: Suresh Srinivas
>         Attachments: stats.patch
>
>
> Recently we had GC issues, where Namenode used more heap than usual. There was no growth indicated by the data in current Metrics to justify the heap usage. Adding more stats such as:
> - Counter to track blocks that are pending deletion
> - BlocksMap hashmap capacity
> - Counter to track excess number of blocks 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (HADOOP-5897) Add more Metrics to Namenode to capture heap usage

Posted by "Suresh Srinivas (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HADOOP-5897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Suresh Srinivas updated HADOOP-5897:
------------------------------------

    Attachment: 5897.rel20.patch

Attaching a back porting patch as this change is required on release 20.

> Add more Metrics to Namenode to capture heap usage
> --------------------------------------------------
>
>                 Key: HADOOP-5897
>                 URL: https://issues.apache.org/jira/browse/HADOOP-5897
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: dfs, metrics
>            Reporter: Suresh Srinivas
>            Assignee: Suresh Srinivas
>             Fix For: 0.21.0
>
>         Attachments: 5897.rel20.patch, stats.1.patch, stats.patch, stats.patch
>
>
> Recently we had GC issues, where Namenode used more heap than usual. There was no growth indicated by the data in current Metrics to justify the heap usage. Adding more stats such as:
> - Counter to track blocks that are pending deletion
> - BlocksMap hashmap capacity
> - Counter to track excess number of blocks 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (HADOOP-5897) Add more Metrics to Namenode to capture heap usage

Posted by "Suresh Srinivas (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HADOOP-5897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12717482#action_12717482 ] 

Suresh Srinivas commented on HADOOP-5897:
-----------------------------------------

bq. These are HashMap parameters and can be retrieved directly from the HashMap.
Unfortunately HashMap access to capacity and loadFactor is package private. Beats me why that was the choice made. The hashmap implementation is closely tied to capacity being multiple of two and the code added that calculates capacity should be future proof.

I will make the other counters volatile.

> Add more Metrics to Namenode to capture heap usage
> --------------------------------------------------
>
>                 Key: HADOOP-5897
>                 URL: https://issues.apache.org/jira/browse/HADOOP-5897
>             Project: Hadoop Core
>          Issue Type: Improvement
>            Reporter: Suresh Srinivas
>            Assignee: Suresh Srinivas
>         Attachments: stats.patch, stats.patch
>
>
> Recently we had GC issues, where Namenode used more heap than usual. There was no growth indicated by the data in current Metrics to justify the heap usage. Adding more stats such as:
> - Counter to track blocks that are pending deletion
> - BlocksMap hashmap capacity
> - Counter to track excess number of blocks 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (HADOOP-5897) Add more Metrics to Namenode to capture heap usage

Posted by "Konstantin Shvachko (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HADOOP-5897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12717357#action_12717357 ] 

Konstantin Shvachko commented on HADOOP-5897:
---------------------------------------------

# Why do we need to store {{capacity}} and {{loadFactor}} in {{BlocksMap}}?
These are {{HashMap}} parameters and can be retrieved directly from the {{HashMap}}.
Or do I miss something here?
# Should we make {{pendingReplicationBlocksCount}} and other block counts volatile?

> Add more Metrics to Namenode to capture heap usage
> --------------------------------------------------
>
>                 Key: HADOOP-5897
>                 URL: https://issues.apache.org/jira/browse/HADOOP-5897
>             Project: Hadoop Core
>          Issue Type: Improvement
>            Reporter: Suresh Srinivas
>            Assignee: Suresh Srinivas
>         Attachments: stats.patch, stats.patch
>
>
> Recently we had GC issues, where Namenode used more heap than usual. There was no growth indicated by the data in current Metrics to justify the heap usage. Adding more stats such as:
> - Counter to track blocks that are pending deletion
> - BlocksMap hashmap capacity
> - Counter to track excess number of blocks 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (HADOOP-5897) Add more Metrics to Namenode to capture heap usage

Posted by "Hudson (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HADOOP-5897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12724840#action_12724840 ] 

Hudson commented on HADOOP-5897:
--------------------------------

Integrated in Hadoop-Common-trunk #9 (See [http://hudson.zones.apache.org/hudson/job/Hadoop-Common-trunk/9/])
    . Promote new name-node metrics to  branch 0.20.


> Add more Metrics to Namenode to capture heap usage
> --------------------------------------------------
>
>                 Key: HADOOP-5897
>                 URL: https://issues.apache.org/jira/browse/HADOOP-5897
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: dfs, metrics
>            Reporter: Suresh Srinivas
>            Assignee: Suresh Srinivas
>             Fix For: 0.21.0
>
>         Attachments: 5897.rel20.patch, stats.1.patch, stats.patch, stats.patch
>
>
> Recently we had GC issues, where Namenode used more heap than usual. There was no growth indicated by the data in current Metrics to justify the heap usage. Adding more stats such as:
> - Counter to track blocks that are pending deletion
> - BlocksMap hashmap capacity
> - Counter to track excess number of blocks 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (HADOOP-5897) Add more Metrics to Namenode to capture heap usage

Posted by "gary murry (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HADOOP-5897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12717359#action_12717359 ] 

gary murry commented on HADOOP-5897:
------------------------------------

I will be out of office June 12.

-Gary


> Add more Metrics to Namenode to capture heap usage
> --------------------------------------------------
>
>                 Key: HADOOP-5897
>                 URL: https://issues.apache.org/jira/browse/HADOOP-5897
>             Project: Hadoop Core
>          Issue Type: Improvement
>            Reporter: Suresh Srinivas
>            Assignee: Suresh Srinivas
>         Attachments: stats.patch, stats.patch
>
>
> Recently we had GC issues, where Namenode used more heap than usual. There was no growth indicated by the data in current Metrics to justify the heap usage. Adding more stats such as:
> - Counter to track blocks that are pending deletion
> - BlocksMap hashmap capacity
> - Counter to track excess number of blocks 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (HADOOP-5897) Add more Metrics to Namenode to capture heap usage

Posted by "Suresh Srinivas (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HADOOP-5897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12724633#action_12724633 ] 

Suresh Srinivas commented on HADOOP-5897:
-----------------------------------------

Patch passes all the unit tests.

> Add more Metrics to Namenode to capture heap usage
> --------------------------------------------------
>
>                 Key: HADOOP-5897
>                 URL: https://issues.apache.org/jira/browse/HADOOP-5897
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: dfs, metrics
>            Reporter: Suresh Srinivas
>            Assignee: Suresh Srinivas
>             Fix For: 0.21.0
>
>         Attachments: 5897.rel20.patch, stats.1.patch, stats.patch, stats.patch
>
>
> Recently we had GC issues, where Namenode used more heap than usual. There was no growth indicated by the data in current Metrics to justify the heap usage. Adding more stats such as:
> - Counter to track blocks that are pending deletion
> - BlocksMap hashmap capacity
> - Counter to track excess number of blocks 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (HADOOP-5897) Add more Metrics to Namenode to capture heap usage

Posted by "Konstantin Shvachko (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HADOOP-5897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12724755#action_12724755 ] 

Konstantin Shvachko commented on HADOOP-5897:
---------------------------------------------

+1
Committed to branch 0.20.

> Add more Metrics to Namenode to capture heap usage
> --------------------------------------------------
>
>                 Key: HADOOP-5897
>                 URL: https://issues.apache.org/jira/browse/HADOOP-5897
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: dfs, metrics
>            Reporter: Suresh Srinivas
>            Assignee: Suresh Srinivas
>             Fix For: 0.21.0
>
>         Attachments: 5897.rel20.patch, stats.1.patch, stats.patch, stats.patch
>
>
> Recently we had GC issues, where Namenode used more heap than usual. There was no growth indicated by the data in current Metrics to justify the heap usage. Adding more stats such as:
> - Counter to track blocks that are pending deletion
> - BlocksMap hashmap capacity
> - Counter to track excess number of blocks 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (HADOOP-5897) Add more Metrics to Namenode to capture heap usage

Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HADOOP-5897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12719095#action_12719095 ] 

Hadoop QA commented on HADOOP-5897:
-----------------------------------

-1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12410190/stats.1.patch
  against trunk revision 784318.

    +1 @author.  The patch does not contain any @author tags.

    +1 tests included.  The patch appears to include 3 new or modified tests.

    +1 javadoc.  The javadoc tool did not generate any warning messages.

    +1 javac.  The applied patch does not increase the total number of javac compiler warnings.

    +1 findbugs.  The patch does not introduce any new Findbugs warnings.

    +1 Eclipse classpath. The patch retains Eclipse classpath integrity.

    +1 release audit.  The applied patch does not increase the total number of release audit warnings.

    -1 core tests.  The patch failed core unit tests.

    -1 contrib tests.  The patch failed contrib unit tests.

Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/497/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/497/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/497/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/497/console

This message is automatically generated.

> Add more Metrics to Namenode to capture heap usage
> --------------------------------------------------
>
>                 Key: HADOOP-5897
>                 URL: https://issues.apache.org/jira/browse/HADOOP-5897
>             Project: Hadoop Core
>          Issue Type: Improvement
>            Reporter: Suresh Srinivas
>            Assignee: Suresh Srinivas
>         Attachments: stats.1.patch, stats.patch, stats.patch
>
>
> Recently we had GC issues, where Namenode used more heap than usual. There was no growth indicated by the data in current Metrics to justify the heap usage. Adding more stats such as:
> - Counter to track blocks that are pending deletion
> - BlocksMap hashmap capacity
> - Counter to track excess number of blocks 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (HADOOP-5897) Add more Metrics to Namenode to capture heap usage

Posted by "Suresh Srinivas (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HADOOP-5897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Suresh Srinivas updated HADOOP-5897:
------------------------------------

    Attachment: stats.patch

Uploading a newer patch. Previous patch the unit test does not compile.

> Add more Metrics to Namenode to capture heap usage
> --------------------------------------------------
>
>                 Key: HADOOP-5897
>                 URL: https://issues.apache.org/jira/browse/HADOOP-5897
>             Project: Hadoop Core
>          Issue Type: Improvement
>            Reporter: Suresh Srinivas
>            Assignee: Suresh Srinivas
>         Attachments: stats.patch, stats.patch
>
>
> Recently we had GC issues, where Namenode used more heap than usual. There was no growth indicated by the data in current Metrics to justify the heap usage. Adding more stats such as:
> - Counter to track blocks that are pending deletion
> - BlocksMap hashmap capacity
> - Counter to track excess number of blocks 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (HADOOP-5897) Add more Metrics to Namenode to capture heap usage

Posted by "Konstantin Shvachko (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HADOOP-5897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Konstantin Shvachko updated HADOOP-5897:
----------------------------------------

       Resolution: Fixed
    Fix Version/s: 0.21.0
     Hadoop Flags: [Reviewed]
           Status: Resolved  (was: Patch Available)

I just committed this. Thank you Suresh.

> Add more Metrics to Namenode to capture heap usage
> --------------------------------------------------
>
>                 Key: HADOOP-5897
>                 URL: https://issues.apache.org/jira/browse/HADOOP-5897
>             Project: Hadoop Core
>          Issue Type: Improvement
>            Reporter: Suresh Srinivas
>            Assignee: Suresh Srinivas
>             Fix For: 0.21.0
>
>         Attachments: stats.1.patch, stats.patch, stats.patch
>
>
> Recently we had GC issues, where Namenode used more heap than usual. There was no growth indicated by the data in current Metrics to justify the heap usage. Adding more stats such as:
> - Counter to track blocks that are pending deletion
> - BlocksMap hashmap capacity
> - Counter to track excess number of blocks 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (HADOOP-5897) Add more Metrics to Namenode to capture heap usage

Posted by "Suresh Srinivas (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HADOOP-5897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Suresh Srinivas updated HADOOP-5897:
------------------------------------

    Attachment: stats.1.patch

Uploaded patch incorporates changes suggested

> Add more Metrics to Namenode to capture heap usage
> --------------------------------------------------
>
>                 Key: HADOOP-5897
>                 URL: https://issues.apache.org/jira/browse/HADOOP-5897
>             Project: Hadoop Core
>          Issue Type: Improvement
>            Reporter: Suresh Srinivas
>            Assignee: Suresh Srinivas
>         Attachments: stats.1.patch, stats.patch, stats.patch
>
>
> Recently we had GC issues, where Namenode used more heap than usual. There was no growth indicated by the data in current Metrics to justify the heap usage. Adding more stats such as:
> - Counter to track blocks that are pending deletion
> - BlocksMap hashmap capacity
> - Counter to track excess number of blocks 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (HADOOP-5897) Add more Metrics to Namenode to capture heap usage

Posted by "Hudson (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HADOOP-5897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12720840#action_12720840 ] 

Hudson commented on HADOOP-5897:
--------------------------------

Integrated in Hadoop-trunk #869 (See [http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/869/])
    

> Add more Metrics to Namenode to capture heap usage
> --------------------------------------------------
>
>                 Key: HADOOP-5897
>                 URL: https://issues.apache.org/jira/browse/HADOOP-5897
>             Project: Hadoop Core
>          Issue Type: Improvement
>          Components: dfs, metrics
>            Reporter: Suresh Srinivas
>            Assignee: Suresh Srinivas
>             Fix For: 0.21.0
>
>         Attachments: stats.1.patch, stats.patch, stats.patch
>
>
> Recently we had GC issues, where Namenode used more heap than usual. There was no growth indicated by the data in current Metrics to justify the heap usage. Adding more stats such as:
> - Counter to track blocks that are pending deletion
> - BlocksMap hashmap capacity
> - Counter to track excess number of blocks 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (HADOOP-5897) Add more Metrics to Namenode to capture heap usage

Posted by "Konstantin Shvachko (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HADOOP-5897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12717771#action_12717771 ] 

Konstantin Shvachko commented on HADOOP-5897:
---------------------------------------------

I see, the methods are not accessible in HashMap, so although ugly there is no other way around but to dub them in our code.
+1

> Add more Metrics to Namenode to capture heap usage
> --------------------------------------------------
>
>                 Key: HADOOP-5897
>                 URL: https://issues.apache.org/jira/browse/HADOOP-5897
>             Project: Hadoop Core
>          Issue Type: Improvement
>            Reporter: Suresh Srinivas
>            Assignee: Suresh Srinivas
>         Attachments: stats.1.patch, stats.patch, stats.patch
>
>
> Recently we had GC issues, where Namenode used more heap than usual. There was no growth indicated by the data in current Metrics to justify the heap usage. Adding more stats such as:
> - Counter to track blocks that are pending deletion
> - BlocksMap hashmap capacity
> - Counter to track excess number of blocks 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (HADOOP-5897) Add more Metrics to Namenode to capture heap usage

Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HADOOP-5897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Tsz Wo (Nicholas), SZE updated HADOOP-5897:
-------------------------------------------

    Component/s: metrics
                 dfs

> Add more Metrics to Namenode to capture heap usage
> --------------------------------------------------
>
>                 Key: HADOOP-5897
>                 URL: https://issues.apache.org/jira/browse/HADOOP-5897
>             Project: Hadoop Core
>          Issue Type: Improvement
>          Components: dfs, metrics
>            Reporter: Suresh Srinivas
>            Assignee: Suresh Srinivas
>             Fix For: 0.21.0
>
>         Attachments: stats.1.patch, stats.patch, stats.patch
>
>
> Recently we had GC issues, where Namenode used more heap than usual. There was no growth indicated by the data in current Metrics to justify the heap usage. Adding more stats such as:
> - Counter to track blocks that are pending deletion
> - BlocksMap hashmap capacity
> - Counter to track excess number of blocks 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.