[jira] [Commented] (CASSANDRA-6078) Wrong memtable size estimation: liveRatio is not honored in edge cases

["Jonathan Ellis (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/CASSANDRA-6078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13774607#comment-13774607 ] 

Jonathan Ellis commented on CASSANDRA-6078:
-------------------------------------------

This isn't 100% correct, either.  Consider the case where we've allocated a single 1-byte column from a 1MB slab: multiplying the 1MB by liveRatio is clearly incorrect.

The intent (CASSANDRA-5497) is to bound our error when liveRatio is incorrect.  I think where we get into trouble is with the upper bound, and I don't see a good way to fix that.  (On the bright side, overestimating the size is a lot less dangerous than underestimating it.)

Patch attached to remove the upper bound.
                
> Wrong memtable size estimation: liveRatio is not honored in edge cases
> ----------------------------------------------------------------------
>
>                 Key: CASSANDRA-6078
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-6078
>             Project: Cassandra
>          Issue Type: Bug
>            Reporter: Oleg Anastasyev
>            Priority: Minor
>             Fix For: 1.2.11, 2.0.2
>
>         Attachments: 6078-v2.txt, Memtable-getLiveSize.diff
>
>
> Memtable.getLiveSize does not honours liveRatio the correct way: 
> allocator.get**Size() return sizes allocated only by name and columns data (i.e. no liveRatio applied); but conditions, which cap estimated size, compare it with estimatedSize, already multiplied by liveRatio.
> If liveRatio is big enough (i've seen >11 on our dataset), this leads to huge estimation errors and even to OutOfMemory, because MeteredFlusher  underestimates memtables sizes.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira