You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by Michał Łowicki <ml...@gmail.com> on 2015/06/17 18:53:14 UTC

Garbage collector launched on all nodes at once

Hi,

Two datacenters with 6 nodes (2.1.6) each. In each DC garbage collection is
launched at the same time on each node (See [1] for total GC duration per 5
seconds). RF is set to 3. Any ideas?

[1]
https://www.dropbox.com/s/bsbyew1jxbe3dgo/Screenshot%202015-06-17%2018.49.48.png?dl=0

-- 
BR,
Michał Łowicki

Re: Garbage collector launched on all nodes at once

Posted by Jonathan Haddad <jo...@jonhaddad.com>.
How much memory do you have?  Recently people have been seeing really great
performance using G1GC with heaps > 8GB and offheap memtable objects.

On Thu, Jun 18, 2015 at 1:31 AM Jason Wee <pe...@gmail.com> wrote:

> okay, iirc memtable has been removed off heap, google and got this
> http://www.datastax.com/dev/blog/off-heap-memtables-in-Cassandra-2-1
>  apparently, there are still some reference on heap.
>
> On Thu, Jun 18, 2015 at 1:11 PM, Marcus Eriksson <kr...@gmail.com>
> wrote:
>
>> It is probably this: https://issues.apache.org/jira/browse/CASSANDRA-9549
>>
>> On Wed, Jun 17, 2015 at 7:37 PM, Michał Łowicki <ml...@gmail.com>
>> wrote:
>>
>>> Looks that memtable heap size is growing on some nodes rapidly (
>>> https://www.dropbox.com/s/3brloiy3fqang1r/Screenshot%202015-06-17%2019.21.49.png?dl=0).
>>> Drops are the places when nodes have been restarted.
>>>
>>> On Wed, Jun 17, 2015 at 6:53 PM, Michał Łowicki <ml...@gmail.com>
>>> wrote:
>>>
>>>> Hi,
>>>>
>>>> Two datacenters with 6 nodes (2.1.6) each. In each DC garbage
>>>> collection is launched at the same time on each node (See [1] for total GC
>>>> duration per 5 seconds). RF is set to 3. Any ideas?
>>>>
>>>> [1]
>>>> https://www.dropbox.com/s/bsbyew1jxbe3dgo/Screenshot%202015-06-17%2018.49.48.png?dl=0
>>>>
>>>> --
>>>> BR,
>>>> Michał Łowicki
>>>>
>>>
>>>
>>>
>>> --
>>> BR,
>>> Michał Łowicki
>>>
>>
>>
>

Re: Garbage collector launched on all nodes at once

Posted by Jason Wee <pe...@gmail.com>.
okay, iirc memtable has been removed off heap, google and got this
http://www.datastax.com/dev/blog/off-heap-memtables-in-Cassandra-2-1
 apparently, there are still some reference on heap.

On Thu, Jun 18, 2015 at 1:11 PM, Marcus Eriksson <kr...@gmail.com> wrote:

> It is probably this: https://issues.apache.org/jira/browse/CASSANDRA-9549
>
> On Wed, Jun 17, 2015 at 7:37 PM, Michał Łowicki <ml...@gmail.com>
> wrote:
>
>> Looks that memtable heap size is growing on some nodes rapidly (
>> https://www.dropbox.com/s/3brloiy3fqang1r/Screenshot%202015-06-17%2019.21.49.png?dl=0).
>> Drops are the places when nodes have been restarted.
>>
>> On Wed, Jun 17, 2015 at 6:53 PM, Michał Łowicki <ml...@gmail.com>
>> wrote:
>>
>>> Hi,
>>>
>>> Two datacenters with 6 nodes (2.1.6) each. In each DC garbage collection
>>> is launched at the same time on each node (See [1] for total GC duration
>>> per 5 seconds). RF is set to 3. Any ideas?
>>>
>>> [1]
>>> https://www.dropbox.com/s/bsbyew1jxbe3dgo/Screenshot%202015-06-17%2018.49.48.png?dl=0
>>>
>>> --
>>> BR,
>>> Michał Łowicki
>>>
>>
>>
>>
>> --
>> BR,
>> Michał Łowicki
>>
>
>

Re: Garbage collector launched on all nodes at once

Posted by Marcus Eriksson <kr...@gmail.com>.
It is probably this: https://issues.apache.org/jira/browse/CASSANDRA-9549

On Wed, Jun 17, 2015 at 7:37 PM, Michał Łowicki <ml...@gmail.com> wrote:

> Looks that memtable heap size is growing on some nodes rapidly (
> https://www.dropbox.com/s/3brloiy3fqang1r/Screenshot%202015-06-17%2019.21.49.png?dl=0).
> Drops are the places when nodes have been restarted.
>
> On Wed, Jun 17, 2015 at 6:53 PM, Michał Łowicki <ml...@gmail.com>
> wrote:
>
>> Hi,
>>
>> Two datacenters with 6 nodes (2.1.6) each. In each DC garbage collection
>> is launched at the same time on each node (See [1] for total GC duration
>> per 5 seconds). RF is set to 3. Any ideas?
>>
>> [1]
>> https://www.dropbox.com/s/bsbyew1jxbe3dgo/Screenshot%202015-06-17%2018.49.48.png?dl=0
>>
>> --
>> BR,
>> Michał Łowicki
>>
>
>
>
> --
> BR,
> Michał Łowicki
>

Re: Garbage collector launched on all nodes at once

Posted by Michał Łowicki <ml...@gmail.com>.
Looks that memtable heap size is growing on some nodes rapidly (
https://www.dropbox.com/s/3brloiy3fqang1r/Screenshot%202015-06-17%2019.21.49.png?dl=0).
Drops are the places when nodes have been restarted.

On Wed, Jun 17, 2015 at 6:53 PM, Michał Łowicki <ml...@gmail.com> wrote:

> Hi,
>
> Two datacenters with 6 nodes (2.1.6) each. In each DC garbage collection
> is launched at the same time on each node (See [1] for total GC duration
> per 5 seconds). RF is set to 3. Any ideas?
>
> [1]
> https://www.dropbox.com/s/bsbyew1jxbe3dgo/Screenshot%202015-06-17%2018.49.48.png?dl=0
>
> --
> BR,
> Michał Łowicki
>



-- 
BR,
Michał Łowicki