You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@kudu.apache.org by "Todd Lipcon (JIRA)" <ji...@apache.org> on 2018/01/19 00:41:13 UTC
[jira] [Commented] (KUDU-2226) Tablets with too many DRSs will
cause a huge DMS memory overhead
[ https://issues.apache.org/jira/browse/KUDU-2226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16331513#comment-16331513 ]
Todd Lipcon commented on KUDU-2226:
-----------------------------------
[~zhquake@gmail.com] can you try upgrading to 1.4 or later? There were a bunch of patches in 1.4 which substantially reduce the memory consumption of DRS/DMS. eg 286de539213602f502f0a25fddde27f97ab9665b , 9adbd90d170a0a525e8aef05939ba211ca6c3a41 , c3953ad01e91bfd5b10d2851c7db5cdf424eec8e, 02cbf8835d72157c348d8b4c1e34611b0bc05f44 and some others.
> Tablets with too many DRSs will cause a huge DMS memory overhead
> ----------------------------------------------------------------
>
> Key: KUDU-2226
> URL: https://issues.apache.org/jira/browse/KUDU-2226
> Project: Kudu
> Issue Type: Improvement
> Affects Versions: 1.3.0
> Environment: CentOS6.5 Linux 2.6.32-431
> Kudu1.3.0
> GitCommit 00813f96b9cb
> Reporter: ZhangZhen
> Priority: Major
>
> I have a table with 10M rows in total and has been hash partitioned to 16 buckets. Each tablet has about 100MB on disk size according to the /tablets Web UI. Everyday 50K new rows will be inserted into this table, and about 5M rows of this table will be updated, that's about half of rows in total, each row will be updated only once.
> Then I found something strange, from the /mem-trackers UI of TS, I found every tablet of this table occupied about 900MB memory, mainly occupied by DeltaMemStore, the peak memory consumption is about 1.8G.
> I don't understand why the DeltaMemStore will cost so much memory, 900MB DMS vs 100MB on disk size, that seems strange to me. What's more, I found these DMS will be flushed very slowly, so for a long time these memory are occupied, which cause "Soft memory limit exceeded" in the TS, and in result cause "Rejecting consensus request".
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)