[jira] [Commented] (KUDU-3097) whether master load deleted entries into memory could be configuable

["YifanZhang (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KUDU-3097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17458956#comment-17458956 ] 

YifanZhang commented on KUDU-3097:
----------------------------------

We also notice that sometimes the memory usage of kudu master could be very huge. I collected sampled heap usage and it shows that the master allocated too much memory for storing `SysTabletEntryPB`.   [^set-09.svg] 

The number of deleted tablets keeps growing in a typical usage scenario where we keep only the latest partition and then delete the historical tablets. Maybe it is not so necessary to keep all tablets(including deleted ones) in memory?

> whether master load deleted entries into memory could be configuable
> --------------------------------------------------------------------
>
>                 Key: KUDU-3097
>                 URL: https://issues.apache.org/jira/browse/KUDU-3097
>             Project: Kudu
>          Issue Type: New Feature
>            Reporter: wangningito
>            Assignee: wangningito
>            Priority: Major
>         Attachments: image-2020-05-28-19-41-05-485.png, screenshot-1.png, set-09.svg
>
>
> The tablet of master is not under control of MVCC.
> The deleted entries like table structure, deleted tablet ids would be load into memory.
> For those who use the massive columns or lots of tablets and frequently switch table, it may result in some unnecessary memory usage. 
> By the way, the memory usage is different between leader and follower in master. It may result in imbalance among master cluster.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)