You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by Gurpreet Singh <gu...@gmail.com> on 2012/06/26 02:50:06 UTC
key_cache_size_in_mb
Hi,
I have a question about cassandra 1.1
Just wanted to confirm if key_cache_size_in_mb is the maximum amount of
memory that key cache will use in memory? If not, what is it?
My observations:
With key cache disabled, I started cassandra. I invoked Full GC through
jconsole a couple of times just to make sure that memory is freed up.
Total heap memory in use is 2.2 gigs. I have almost 550 million keys, so
this looked reasonable as the bloom filter itself is around 1 gig.
I enabled key cache again by making key_cache_size_in_mb to a value 800.
This would lead cassandra to load a previously saved key cache file of size
98 mb on disk. I restarted cassandra again. This time the heap size
increased to 4.2 gigs. I checked the cache sizing in jconsole.
KeyCacheCapacityInMb: 799
KeyCacheSize: 410,316,048
This says that the key cache size is 410 mb. However the increase in the
heap is 2 gigs. Doesnt add up!!
Can anyone explain this?
Thanks
Gurpreet
Re: key_cache_size_in_mb
Posted by Vijay <vi...@gmail.com>.
You might want to show us the GC log's... if there are promotion failures,
count on the objects which you can get by inspecting the heapdumps etc, I
am kind of assuming stuff :)
Regards,
</VJ>
On Thu, Jun 28, 2012 at 11:01 PM, Gurpreet Singh
<gu...@gmail.com>wrote:
> Vijay,
> Thanks for the suggestion.
> Row Key size is 8 bytes (its a long).
> Is the issue here incorrect calculation really?
> The server already had the key cache in memory when i stopped it, and it
> was using abt 6 gigs of heap total. How is it tht it cannot startup again
> with the same keycache?
>
> /G
>
>
> On Thu, Jun 28, 2012 at 4:25 PM, Vijay <vi...@gmail.com> wrote:
>
>> > It has 13.7million keys
>> How big is your Row Keys? Current calculation (in 1.1) assumes that the
>> average size of the Entry<RowKey, Long> in the keycache is 16 bytes.
>>
>> One thing which you can do to get out this issue is to reduce the
>> Keycache to a low enough to get the server started or increase the Heap to
>> hold all the objects. (If you are using the visualVM try to see the object
>> counts).
>>
>> Regards,
>> </VJ>
>>
>>
>>
>>
>> On Thu, Jun 28, 2012 at 3:40 PM, Gurpreet Singh <gurpreet.singh@gmail.com
>> > wrote:
>>
>>> Vijay,
>>> I am running into an issue right now, not sure if its related to this
>>> one.
>>>
>>> My cassandra 1.1 was running fine for a week, and the key cache size (as
>>> checked by jconsole) increased to almost 650 mb. It has 13.7million keys. I
>>> have set key_cache_size_in_mb to 800 mb. I checked the heap size, and it
>>> was at 6 gigs. All good!
>>> However, when i try to restart cassandra, it is unable to start now.
>>> Before i shut it down, it seemed fine at 6 gigs, which means its able to
>>> handle the memory requirements just fine. However it is not able to start
>>> up anymore with the same key cache. jconsole says that the heap is full.
>>>
>>> Thanks
>>> /G
>>>
>>>
>>> On Thu, Jun 28, 2012 at 12:22 PM, Vijay <vi...@gmail.com> wrote:
>>>
>>>> in 1.1 we dont calculate the Key size accurately, hence we have the fix
>>>> in https://issues.apache.org/jira/browse/CASSANDRA-4315
>>>>
>>>> Regards,
>>>> </VJ>
>>>>
>>>>
>>>>
>>>>
>>>> On Thu, Jun 28, 2012 at 11:25 AM, Gurpreet Singh <
>>>> gurpreet.singh@gmail.com> wrote:
>>>>
>>>>> anyone have an explanation for this?
>>>>> This kinda screws up memory calculations.
>>>>>
>>>>> /G
>>>>>
>>>>> On Mon, Jun 25, 2012 at 5:50 PM, Gurpreet Singh <
>>>>> gurpreet.singh@gmail.com> wrote:
>>>>>
>>>>>> Hi,
>>>>>> I have a question about cassandra 1.1
>>>>>>
>>>>>> Just wanted to confirm if key_cache_size_in_mb is the maximum amount
>>>>>> of memory that key cache will use in memory? If not, what is it?
>>>>>>
>>>>>> My observations:
>>>>>> With key cache disabled, I started cassandra. I invoked Full GC
>>>>>> through jconsole a couple of times just to make sure that memory is freed
>>>>>> up.
>>>>>> Total heap memory in use is 2.2 gigs. I have almost 550 million keys,
>>>>>> so this looked reasonable as the bloom filter itself is around 1 gig.
>>>>>>
>>>>>> I enabled key cache again by making key_cache_size_in_mb to a value
>>>>>> 800. This would lead cassandra to load a previously saved key cache file of
>>>>>> size 98 mb on disk. I restarted cassandra again. This time the heap size
>>>>>> increased to 4.2 gigs. I checked the cache sizing in jconsole.
>>>>>>
>>>>>> KeyCacheCapacityInMb: 799
>>>>>> KeyCacheSize: 410,316,048
>>>>>>
>>>>>> This says that the key cache size is 410 mb. However the increase in
>>>>>> the heap is 2 gigs. Doesnt add up!!
>>>>>>
>>>>>> Can anyone explain this?
>>>>>> Thanks
>>>>>> Gurpreet
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
>
Re: key_cache_size_in_mb
Posted by Gurpreet Singh <gu...@gmail.com>.
Vijay,
Thanks for the suggestion.
Row Key size is 8 bytes (its a long).
Is the issue here incorrect calculation really?
The server already had the key cache in memory when i stopped it, and it
was using abt 6 gigs of heap total. How is it tht it cannot startup again
with the same keycache?
/G
On Thu, Jun 28, 2012 at 4:25 PM, Vijay <vi...@gmail.com> wrote:
> > It has 13.7million keys
> How big is your Row Keys? Current calculation (in 1.1) assumes that the
> average size of the Entry<RowKey, Long> in the keycache is 16 bytes.
>
> One thing which you can do to get out this issue is to reduce the Keycache
> to a low enough to get the server started or increase the Heap to hold all
> the objects. (If you are using the visualVM try to see the object counts).
>
> Regards,
> </VJ>
>
>
>
>
> On Thu, Jun 28, 2012 at 3:40 PM, Gurpreet Singh <gu...@gmail.com>wrote:
>
>> Vijay,
>> I am running into an issue right now, not sure if its related to this one.
>>
>> My cassandra 1.1 was running fine for a week, and the key cache size (as
>> checked by jconsole) increased to almost 650 mb. It has 13.7million keys. I
>> have set key_cache_size_in_mb to 800 mb. I checked the heap size, and it
>> was at 6 gigs. All good!
>> However, when i try to restart cassandra, it is unable to start now.
>> Before i shut it down, it seemed fine at 6 gigs, which means its able to
>> handle the memory requirements just fine. However it is not able to start
>> up anymore with the same key cache. jconsole says that the heap is full.
>>
>> Thanks
>> /G
>>
>>
>> On Thu, Jun 28, 2012 at 12:22 PM, Vijay <vi...@gmail.com> wrote:
>>
>>> in 1.1 we dont calculate the Key size accurately, hence we have the fix
>>> in https://issues.apache.org/jira/browse/CASSANDRA-4315
>>>
>>> Regards,
>>> </VJ>
>>>
>>>
>>>
>>>
>>> On Thu, Jun 28, 2012 at 11:25 AM, Gurpreet Singh <
>>> gurpreet.singh@gmail.com> wrote:
>>>
>>>> anyone have an explanation for this?
>>>> This kinda screws up memory calculations.
>>>>
>>>> /G
>>>>
>>>> On Mon, Jun 25, 2012 at 5:50 PM, Gurpreet Singh <
>>>> gurpreet.singh@gmail.com> wrote:
>>>>
>>>>> Hi,
>>>>> I have a question about cassandra 1.1
>>>>>
>>>>> Just wanted to confirm if key_cache_size_in_mb is the maximum amount
>>>>> of memory that key cache will use in memory? If not, what is it?
>>>>>
>>>>> My observations:
>>>>> With key cache disabled, I started cassandra. I invoked Full GC
>>>>> through jconsole a couple of times just to make sure that memory is freed
>>>>> up.
>>>>> Total heap memory in use is 2.2 gigs. I have almost 550 million keys,
>>>>> so this looked reasonable as the bloom filter itself is around 1 gig.
>>>>>
>>>>> I enabled key cache again by making key_cache_size_in_mb to a value
>>>>> 800. This would lead cassandra to load a previously saved key cache file of
>>>>> size 98 mb on disk. I restarted cassandra again. This time the heap size
>>>>> increased to 4.2 gigs. I checked the cache sizing in jconsole.
>>>>>
>>>>> KeyCacheCapacityInMb: 799
>>>>> KeyCacheSize: 410,316,048
>>>>>
>>>>> This says that the key cache size is 410 mb. However the increase in
>>>>> the heap is 2 gigs. Doesnt add up!!
>>>>>
>>>>> Can anyone explain this?
>>>>> Thanks
>>>>> Gurpreet
>>>>>
>>>>
>>>>
>>>
>>
>
Re: key_cache_size_in_mb
Posted by Vijay <vi...@gmail.com>.
> It has 13.7million keys
How big is your Row Keys? Current calculation (in 1.1) assumes that the
average size of the Entry<RowKey, Long> in the keycache is 16 bytes.
One thing which you can do to get out this issue is to reduce the Keycache
to a low enough to get the server started or increase the Heap to hold all
the objects. (If you are using the visualVM try to see the object counts).
Regards,
</VJ>
On Thu, Jun 28, 2012 at 3:40 PM, Gurpreet Singh <gu...@gmail.com>wrote:
> Vijay,
> I am running into an issue right now, not sure if its related to this one.
>
> My cassandra 1.1 was running fine for a week, and the key cache size (as
> checked by jconsole) increased to almost 650 mb. It has 13.7million keys. I
> have set key_cache_size_in_mb to 800 mb. I checked the heap size, and it
> was at 6 gigs. All good!
> However, when i try to restart cassandra, it is unable to start now.
> Before i shut it down, it seemed fine at 6 gigs, which means its able to
> handle the memory requirements just fine. However it is not able to start
> up anymore with the same key cache. jconsole says that the heap is full.
>
> Thanks
> /G
>
>
> On Thu, Jun 28, 2012 at 12:22 PM, Vijay <vi...@gmail.com> wrote:
>
>> in 1.1 we dont calculate the Key size accurately, hence we have the fix
>> in https://issues.apache.org/jira/browse/CASSANDRA-4315
>>
>> Regards,
>> </VJ>
>>
>>
>>
>>
>> On Thu, Jun 28, 2012 at 11:25 AM, Gurpreet Singh <
>> gurpreet.singh@gmail.com> wrote:
>>
>>> anyone have an explanation for this?
>>> This kinda screws up memory calculations.
>>>
>>> /G
>>>
>>> On Mon, Jun 25, 2012 at 5:50 PM, Gurpreet Singh <
>>> gurpreet.singh@gmail.com> wrote:
>>>
>>>> Hi,
>>>> I have a question about cassandra 1.1
>>>>
>>>> Just wanted to confirm if key_cache_size_in_mb is the maximum amount of
>>>> memory that key cache will use in memory? If not, what is it?
>>>>
>>>> My observations:
>>>> With key cache disabled, I started cassandra. I invoked Full GC through
>>>> jconsole a couple of times just to make sure that memory is freed up.
>>>> Total heap memory in use is 2.2 gigs. I have almost 550 million keys,
>>>> so this looked reasonable as the bloom filter itself is around 1 gig.
>>>>
>>>> I enabled key cache again by making key_cache_size_in_mb to a value
>>>> 800. This would lead cassandra to load a previously saved key cache file of
>>>> size 98 mb on disk. I restarted cassandra again. This time the heap size
>>>> increased to 4.2 gigs. I checked the cache sizing in jconsole.
>>>>
>>>> KeyCacheCapacityInMb: 799
>>>> KeyCacheSize: 410,316,048
>>>>
>>>> This says that the key cache size is 410 mb. However the increase in
>>>> the heap is 2 gigs. Doesnt add up!!
>>>>
>>>> Can anyone explain this?
>>>> Thanks
>>>> Gurpreet
>>>>
>>>
>>>
>>
>
Re: key_cache_size_in_mb
Posted by Gurpreet Singh <gu...@gmail.com>.
Vijay,
I am running into an issue right now, not sure if its related to this one.
My cassandra 1.1 was running fine for a week, and the key cache size (as
checked by jconsole) increased to almost 650 mb. It has 13.7million keys. I
have set key_cache_size_in_mb to 800 mb. I checked the heap size, and it
was at 6 gigs. All good!
However, when i try to restart cassandra, it is unable to start now. Before
i shut it down, it seemed fine at 6 gigs, which means its able to handle
the memory requirements just fine. However it is not able to start up
anymore with the same key cache. jconsole says that the heap is full.
Thanks
/G
On Thu, Jun 28, 2012 at 12:22 PM, Vijay <vi...@gmail.com> wrote:
> in 1.1 we dont calculate the Key size accurately, hence we have the fix in
> https://issues.apache.org/jira/browse/CASSANDRA-4315
>
> Regards,
> </VJ>
>
>
>
>
> On Thu, Jun 28, 2012 at 11:25 AM, Gurpreet Singh <gurpreet.singh@gmail.com
> > wrote:
>
>> anyone have an explanation for this?
>> This kinda screws up memory calculations.
>>
>> /G
>>
>> On Mon, Jun 25, 2012 at 5:50 PM, Gurpreet Singh <gurpreet.singh@gmail.com
>> > wrote:
>>
>>> Hi,
>>> I have a question about cassandra 1.1
>>>
>>> Just wanted to confirm if key_cache_size_in_mb is the maximum amount of
>>> memory that key cache will use in memory? If not, what is it?
>>>
>>> My observations:
>>> With key cache disabled, I started cassandra. I invoked Full GC through
>>> jconsole a couple of times just to make sure that memory is freed up.
>>> Total heap memory in use is 2.2 gigs. I have almost 550 million keys, so
>>> this looked reasonable as the bloom filter itself is around 1 gig.
>>>
>>> I enabled key cache again by making key_cache_size_in_mb to a value 800.
>>> This would lead cassandra to load a previously saved key cache file of size
>>> 98 mb on disk. I restarted cassandra again. This time the heap size
>>> increased to 4.2 gigs. I checked the cache sizing in jconsole.
>>>
>>> KeyCacheCapacityInMb: 799
>>> KeyCacheSize: 410,316,048
>>>
>>> This says that the key cache size is 410 mb. However the increase in the
>>> heap is 2 gigs. Doesnt add up!!
>>>
>>> Can anyone explain this?
>>> Thanks
>>> Gurpreet
>>>
>>
>>
>
Re: key_cache_size_in_mb
Posted by Vijay <vi...@gmail.com>.
in 1.1 we dont calculate the Key size accurately, hence we have the fix in
https://issues.apache.org/jira/browse/CASSANDRA-4315
Regards,
</VJ>
On Thu, Jun 28, 2012 at 11:25 AM, Gurpreet Singh
<gu...@gmail.com>wrote:
> anyone have an explanation for this?
> This kinda screws up memory calculations.
>
> /G
>
> On Mon, Jun 25, 2012 at 5:50 PM, Gurpreet Singh <gu...@gmail.com>wrote:
>
>> Hi,
>> I have a question about cassandra 1.1
>>
>> Just wanted to confirm if key_cache_size_in_mb is the maximum amount of
>> memory that key cache will use in memory? If not, what is it?
>>
>> My observations:
>> With key cache disabled, I started cassandra. I invoked Full GC through
>> jconsole a couple of times just to make sure that memory is freed up.
>> Total heap memory in use is 2.2 gigs. I have almost 550 million keys, so
>> this looked reasonable as the bloom filter itself is around 1 gig.
>>
>> I enabled key cache again by making key_cache_size_in_mb to a value 800.
>> This would lead cassandra to load a previously saved key cache file of size
>> 98 mb on disk. I restarted cassandra again. This time the heap size
>> increased to 4.2 gigs. I checked the cache sizing in jconsole.
>>
>> KeyCacheCapacityInMb: 799
>> KeyCacheSize: 410,316,048
>>
>> This says that the key cache size is 410 mb. However the increase in the
>> heap is 2 gigs. Doesnt add up!!
>>
>> Can anyone explain this?
>> Thanks
>> Gurpreet
>>
>
>
Re: key_cache_size_in_mb
Posted by Gurpreet Singh <gu...@gmail.com>.
anyone have an explanation for this?
This kinda screws up memory calculations.
/G
On Mon, Jun 25, 2012 at 5:50 PM, Gurpreet Singh <gu...@gmail.com>wrote:
> Hi,
> I have a question about cassandra 1.1
>
> Just wanted to confirm if key_cache_size_in_mb is the maximum amount of
> memory that key cache will use in memory? If not, what is it?
>
> My observations:
> With key cache disabled, I started cassandra. I invoked Full GC through
> jconsole a couple of times just to make sure that memory is freed up.
> Total heap memory in use is 2.2 gigs. I have almost 550 million keys, so
> this looked reasonable as the bloom filter itself is around 1 gig.
>
> I enabled key cache again by making key_cache_size_in_mb to a value 800.
> This would lead cassandra to load a previously saved key cache file of size
> 98 mb on disk. I restarted cassandra again. This time the heap size
> increased to 4.2 gigs. I checked the cache sizing in jconsole.
>
> KeyCacheCapacityInMb: 799
> KeyCacheSize: 410,316,048
>
> This says that the key cache size is 410 mb. However the increase in the
> heap is 2 gigs. Doesnt add up!!
>
> Can anyone explain this?
> Thanks
> Gurpreet
>