[jira] [Commented] (HBASE-20914) Trim Master memory usage

["stack (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HBASE-20914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16586952#comment-16586952 ] 

stack commented on HBASE-20914:
-------------------------------

Link to a follow-on issue looking at Master memory usage.

> Trim Master memory usage
> ------------------------
>
>                 Key: HBASE-20914
>                 URL: https://issues.apache.org/jira/browse/HBASE-20914
>             Project: HBase
>          Issue Type: Sub-task
>          Components: master
>            Reporter: stack
>            Assignee: stack
>            Priority: Major
>             Fix For: 3.0.0, 2.0.2, 2.2.0, 2.1.1
>
>         Attachments: HBASE-20914.branch-2.0.001.patch, Screen Shot 2018-07-19 at 1.20.23 PM.png, Screen Shot 2018-07-19 at 1.20.23 PM.png, Screen Shot 2018-07-19 at 1.38.56 PM.png, Screen Shot 2018-07-19 at 1.38.56 PM.png, Screen Shot 2018-07-19 at 2.22.42 PM.png, Screen Shot 2018-07-19 at 2.22.42 PM.png, Screen Shot 2018-07-19 at 2.43.42 PM.png, Screen Shot 2018-07-19 at 2.43.42 PM.png
>
>
> While working on the parent issue, looking at a heap from a Master tha was running ~650 servers and > 300k regions, I tripped over some silly items in the heap:
> 1. Balancer has a regions x server matrix which takes up 18% of the Master heap according to jxray and 40% according to eclipse. Looks like the matrix should be regions x racks which would be much smaller (Issue came in with HBASE-18164 Fast locality computation in balancer  -Added new LocalityCostFunction and LocalityCandidateGenerator ..). See  !Screen Shot 2018-07-19 at 1.20.23 PM.png!  !Screen Shot 2018-07-19 at 1.38.56 PM.png! 
> 2. Duplicate Strings make up ~5% of the Master heap. Of these, ServerName seems to be the font. Interesting is report that there 54k instances of ServerName in this heap though there are only 650 Servers. See  !Screen Shot 2018-07-19 at 2.22.42 PM.png! 
> 3. ArrayDequeue initializes its internal elements array with 16 elements. We use this in a few places. In Procedures, of which there are many in this heap, we near never make use of this array.  See  !Screen Shot 2018-07-19 at 2.43.42 PM.png! 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)