You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Stefania (JIRA)" <ji...@apache.org> on 2015/12/14 16:50:46 UTC

[jira] [Commented] (CASSANDRA-9472) Reintroduce off heap memtables

    [ https://issues.apache.org/jira/browse/CASSANDRA-9472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15056156#comment-15056156 ] 

Stefania commented on CASSANDRA-9472:
-------------------------------------

I've finally have a reworked patch and the first stable CI results:

|[patch|https://github.com/stef1927/cassandra/commits/9472]|[testall|http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-9472-testall/]|[dtest|http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-9472-dtest/]|

I still have 3 dtests and 2 utests to check but the core issues should be fixed now. 

The initial patch was correct but there were three more problems to address, the most significant of which is by far the last one:

* Segmentation faults: the instances with {{peer = 0}} for {{NativeCell}} and {{NativeClustering}} used to measure empty object sizes were causing segmentation faults, at least in the debugger, because of base class methods such as {{toString()}} calling methods using a null {{peer}}. I've added guards for {{peer == 0}} in all methods but perhaps we ought to come up with a better way to estimate empty object sizes.

* Wrong byte ordering: by default hollow byte buffers were given native byte ordering (little endian on my box) but if the original byte buffers were big endian then the correct order is actually big endian. Since we only deal with big endian buffers, to the best of my knowledge, instead of wasting space and time storing the initial encoding of each byte buffer I've added assertions and enforced big endian ordering when creating hollow byte buffers.

* Released native memory: this was by far the hardest thing to debug and fix. After a memtable is flushed, the native memory is released and the native objects, or the byte buffers returned by them, are no longer valid. The existing mechanism of copying such objects on the heap had become inadequate due to the refactoring of 8099. The strategy I've adopted is to copy any native row or decorated key that is returned by {{AbstractBTreePartition}} or its row iterators. As far as I understood the code, this should be sufficient provided the partition and row iterators are guarded by a read op barrier normally via an execution controller.

> Reintroduce off heap memtables
> ------------------------------
>
>                 Key: CASSANDRA-9472
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-9472
>             Project: Cassandra
>          Issue Type: Improvement
>            Reporter: Benedict
>            Assignee: Stefania
>             Fix For: 3.2
>
>
> CASSANDRA-8099 removes off heap memtables. We should reintroduce them ASAP.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)