[jira] [Commented] (KUDU-3197) Tablet keeps all history schemas in memory may result in high memory consumption

["shenxingwuying (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KUDU-3197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17458898#comment-17458898 ] 

shenxingwuying commented on KUDU-3197:
--------------------------------------

As mentioned above by [~wangning]'s, alter schema will lead to tserver's memory too large, just like memory leak.

[~wangning] has proposed a simple method at [https://gerrit.cloudera.org/c/16508/.|https://gerrit.cloudera.org/c/16508/,]

 

I continue the work's thought, and the thought looks like simple, but implements is a little complex, for Schema has changed to scoped_refptr<Schema>, and which lead to some writes not correct, I must have to repaire a lot of unit tests.

> Tablet keeps all history schemas in memory may result in high memory consumption
> --------------------------------------------------------------------------------
>
>                 Key: KUDU-3197
>                 URL: https://issues.apache.org/jira/browse/KUDU-3197
>             Project: Kudu
>          Issue Type: Improvement
>          Components: tablet
>    Affects Versions: 1.12.0
>            Reporter: wangningito
>            Assignee: wangningito
>            Priority: Minor
>         Attachments: image-2020-09-25-14-45-33-402.png, image-2020-09-25-14-49-30-913.png, image-2020-09-25-15-05-44-948.png, image-2020-12-02-19-59-46-733.png, screenshot-1.png
>
>
> In case of high frequency of updating table, memory consumption of kudu-tserver may be very high, and the memory in not tracked in the memory page. 
> This is the memory usage of a tablet, the memory consumption of tablet-xxx‘s peak is 3.6G， but none of its' childrens' memory can reach.
> !image-2020-09-25-14-45-33-402.png!
> So I use pprof to get the heap sampling. The tserver started for long but the memory is still consuming by TabletBootstrap:PlayAlterSchemaRequest. 
> !image-2020-09-25-14-49-30-913.png!
> I change the `old_schemas_` in tablet_metadata.h to a fixed size vector, 
>     // Previous values of 'schema_'.
>     // These are currently kept alive forever, under the assumption that
>     // a given tablet won't have thousands of "alter table" calls.
>     // They are kept alive so that callers of schema() don't need to
>     // worry about reference counting or locking.
>     std::vector<Schema*> old_schemas_;
> The heap sampling then becomes
>  !image-2020-09-25-15-05-44-948.png! 
> So, to make application layer more flexible, it could be better to make the size of the old_schemas configurable.
>  



--
This message was sent by Atlassian Jira
(v8.20.1#820001)