[jira] [Commented] (CASSANDRA-7401) Memtable.maybeUpdateLiveRatio goes into an endless loop when currentOperations is zero

["Christian Spriegel (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/CASSANDRA-7401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14060404#comment-14060404 ] 

Christian Spriegel commented on CASSANDRA-7401:
-----------------------------------------------

I just found my log message in our logfiles (where my custom-build is running):

{quote}
WARN [MutationStage:43] 2014-07-13 18:06:11,365 Memtable.java (line 184) TEST - This would be a crash: Memtable-ResponsePortal@908155793(0/0 serialized/live bytes, 0 ops)

 WARN [MutationStage:46] 2014-07-13 18:06:11,365 Memtable.java (line 184) TEST - This would be a crash: Memtable-ResponsePortal@908155793(0/0 serialized/live bytes, 0 ops)
{quote}

It shows that the Memtable can be empty. Why that is, I have no idea.

Nevertheless, the fix works.

> Memtable.maybeUpdateLiveRatio goes into an endless loop when currentOperations is zero
> --------------------------------------------------------------------------------------
>
>                 Key: CASSANDRA-7401
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-7401
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>            Reporter: Christian Spriegel
>            Assignee: Christian Spriegel
>             Fix For: 2.0.9
>
>         Attachments: MemtableFixV1.patch
>
>
> Hi,
> I was describing an error the other day on the mailing list, where the MemoryMeter would go into an endless loop. This happened multiple times last week, unfortunetaly I cannot reproduce it at the moment.
> The whole cassandra server got unresponsive and logged about 7000k messages per second into the log:
> {quote}
> ...
>  INFO [MemoryMeter:1] 2014-06-14 19:24:09,488 Memtable.java (line 481) CFS(Keyspace='MDS', ColumnFamily='ResponsePortal') liveRatio is 64.0 (just-counted was 64.0).  calculation took 0ms for 0 cells
> ...
> {quote}
> The cause for this seems to be Memtable.maybeUpdateLiveRatio(), which cannot handle currentOperations (and liveRatioComputedAt) to be zero. The loop will iterate endlessly:
> {code}
>             ...
>             if (operations < 2 * last) // does never break when zero: 0 < 0 is not true
>                 break;
>             ...
> {code}
> One thing I cannot explain: How can the operationcount be zero when maybeUpdateLiveRatio() gets called?
> is it possible that addAndGet in resolve() increases by 0  in some cases?
> {code}
> currentOperations.addAndGet(cf.getColumnCount() + (cf.isMarkedForDelete() ? 1 : 0) + cf.deletionInfo().rangeCount()); // can this be zero? 
> {code}
> Nevertheless, the attached patch fixes the endless loop. Feel free to reassign this ticket or create a followup ticket if currentOperations should not be zero.
> kind regards,
> Christian



--
This message was sent by Atlassian JIRA
(v6.2#6252)