You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by Pranay akula <pr...@gmail.com> on 2017/05/02 02:01:20 UTC
Re: what is MemtableReclaimMemory mean ??
Hi Alain,
when "*MemtableReclaimMemory*" Pending Tasks increasing, its slowly
backing up reads and writes mostly writes. yes i am seeing bit high GC
pressure, currently we are using 24Gb Heap and G1GC collection. I tried
changing Memtable flush threshold it did helped a little but not much. I am
not seeing any Errors in the Logs.
Thanks
Pranay.
On Thu, Apr 27, 2017 at 6:08 AM, Alain RODRIGUEZ <ar...@gmail.com> wrote:
> Hi Pranay,
>
> According to http://docs.datastax.com/en/cassandra/3.0/cassandra/tools/
> toolsTPstats.html, "*MemtableReclaimMemory*" is the thread pool used for
> "Making unused memory available". I don't know much about it since it was
> never an issue for me. Neither did I heard much about it.
>
>
> - Are pending tasks staying high for a long period? `watch -d nodetool
> tpstats`
> - What are your GC settings?
> - Any other threads pending, blocked or dropped?
> - Do you have errors or warnings in your logs?
> - Any GC pressure? (monitored through charts or logs at INFO level, or
> WARN on recent versions)
>
>
> C*heers,
> -----------------------
> Alain Rodriguez - @arodream - alain@thelastpickle.com
> France
>
> The Last Pickle - Apache Cassandra Consulting
> http://www.thelastpickle.com
>
>
>
> 2017-04-16 16:04 GMT+02:00 Pranay akula <pr...@gmail.com>:
>
>> Hi,
>>
>> what is *MemtableReclaimMemory* mean in nodetooltpstats ?? does this
>> mean trying to flushing memtable from memory to SStables.
>>
>> I can see sometimes increase in pending tasks of MemtableReclaimMemory
>> in nodetool tpstats, at that time i can see increase in load on those nodes.
>>
>> Does decreasing memtable_cleanup_threshold will help ??
>>
>> Thanks
>> Pranay.
>>
>
>
Re: what is MemtableReclaimMemory mean ??
Posted by Chris Lohfink <cl...@gmail.com>.
Question though, how many tables do you have? If you have more than a few
hundreds it could be bottlenecking the flushing if it is flushing very
frequently.
On Mon, May 1, 2017 at 9:32 PM, Chris Lohfink <cl...@gmail.com> wrote:
> Theres a read barrier to stop reclaiming a memtable when there are
> requests actively reading it. The *MemtableReclaimMemory* pool offloads
> that wait instead of blocking the caller. It in itself is not going to use
> any cpu or increase load. It will however block the releasing of the
> memtable resources which might cause additional heap allocation pressure.
> Its more likely a symptom of GCs or reads being slow than the cause of the
> issue however.
>
> Chris
>
> On Mon, May 1, 2017 at 9:01 PM, Pranay akula <pr...@gmail.com>
> wrote:
>
>> Hi Alain,
>>
>> when "*MemtableReclaimMemory*" Pending Tasks increasing, its slowly
>> backing up reads and writes mostly writes. yes i am seeing bit high GC
>> pressure, currently we are using 24Gb Heap and G1GC collection. I tried
>> changing Memtable flush threshold it did helped a little but not much. I am
>> not seeing any Errors in the Logs.
>>
>>
>> Thanks
>> Pranay.
>>
>> On Thu, Apr 27, 2017 at 6:08 AM, Alain RODRIGUEZ <ar...@gmail.com>
>> wrote:
>>
>>> Hi Pranay,
>>>
>>> According to http://docs.datastax.com/en/ca
>>> ssandra/3.0/cassandra/tools/toolsTPstats.html, "*MemtableReclaimMemory*"
>>> is the thread pool used for "Making unused memory available". I don't know
>>> much about it since it was never an issue for me. Neither did I heard much
>>> about it.
>>>
>>>
>>> - Are pending tasks staying high for a long period? `watch -d
>>> nodetool tpstats`
>>> - What are your GC settings?
>>> - Any other threads pending, blocked or dropped?
>>> - Do you have errors or warnings in your logs?
>>> - Any GC pressure? (monitored through charts or logs at INFO level,
>>> or WARN on recent versions)
>>>
>>>
>>> C*heers,
>>> -----------------------
>>> Alain Rodriguez - @arodream - alain@thelastpickle.com
>>> France
>>>
>>> The Last Pickle - Apache Cassandra Consulting
>>> http://www.thelastpickle.com
>>>
>>>
>>>
>>> 2017-04-16 16:04 GMT+02:00 Pranay akula <pr...@gmail.com>:
>>>
>>>> Hi,
>>>>
>>>> what is *MemtableReclaimMemory* mean in nodetooltpstats ?? does this
>>>> mean trying to flushing memtable from memory to SStables.
>>>>
>>>> I can see sometimes increase in pending tasks of MemtableReclaimMemory
>>>> in nodetool tpstats, at that time i can see increase in load on those nodes.
>>>>
>>>> Does decreasing memtable_cleanup_threshold will help ??
>>>>
>>>> Thanks
>>>> Pranay.
>>>>
>>>
>>>
>>
>
Re: what is MemtableReclaimMemory mean ??
Posted by Chris Lohfink <cl...@gmail.com>.
Theres a read barrier to stop reclaiming a memtable when there are requests
actively reading it. The *MemtableReclaimMemory* pool offloads that wait
instead of blocking the caller. It in itself is not going to use any cpu or
increase load. It will however block the releasing of the memtable
resources which might cause additional heap allocation pressure. Its more
likely a symptom of GCs or reads being slow than the cause of the issue
however.
Chris
On Mon, May 1, 2017 at 9:01 PM, Pranay akula <pr...@gmail.com>
wrote:
> Hi Alain,
>
> when "*MemtableReclaimMemory*" Pending Tasks increasing, its slowly
> backing up reads and writes mostly writes. yes i am seeing bit high GC
> pressure, currently we are using 24Gb Heap and G1GC collection. I tried
> changing Memtable flush threshold it did helped a little but not much. I am
> not seeing any Errors in the Logs.
>
>
> Thanks
> Pranay.
>
> On Thu, Apr 27, 2017 at 6:08 AM, Alain RODRIGUEZ <ar...@gmail.com>
> wrote:
>
>> Hi Pranay,
>>
>> According to http://docs.datastax.com/en/cassandra/3.0/cassandra/tools/to
>> olsTPstats.html, "*MemtableReclaimMemory*" is the thread pool used for
>> "Making unused memory available". I don't know much about it since it was
>> never an issue for me. Neither did I heard much about it.
>>
>>
>> - Are pending tasks staying high for a long period? `watch -d
>> nodetool tpstats`
>> - What are your GC settings?
>> - Any other threads pending, blocked or dropped?
>> - Do you have errors or warnings in your logs?
>> - Any GC pressure? (monitored through charts or logs at INFO level,
>> or WARN on recent versions)
>>
>>
>> C*heers,
>> -----------------------
>> Alain Rodriguez - @arodream - alain@thelastpickle.com
>> France
>>
>> The Last Pickle - Apache Cassandra Consulting
>> http://www.thelastpickle.com
>>
>>
>>
>> 2017-04-16 16:04 GMT+02:00 Pranay akula <pr...@gmail.com>:
>>
>>> Hi,
>>>
>>> what is *MemtableReclaimMemory* mean in nodetooltpstats ?? does this
>>> mean trying to flushing memtable from memory to SStables.
>>>
>>> I can see sometimes increase in pending tasks of MemtableReclaimMemory
>>> in nodetool tpstats, at that time i can see increase in load on those nodes.
>>>
>>> Does decreasing memtable_cleanup_threshold will help ??
>>>
>>> Thanks
>>> Pranay.
>>>
>>
>>
>