You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by David Dabbs <dm...@gmail.com> on 2011/01/17 01:01:09 UTC
Cassandra and -XX:+UseLargePages
Hello.
Can anyone comment on the performance impact (positive or negative)
of running Cassandra configured to use large pages under Linux?
Yes, YMMV applies, but I thought I'd ask before enlisting sysadmin Fu, etc.
Thanks!
David
Re: RE: Cassandra and -XX:+UseLargePages
Posted by Aaron Morton <aa...@thelastpickle.com>.
I pretty much had the same thoughts as you. I think setting xms to the same as xmx helps the JVM allocate all the memory up front.
See the comments in conf/cassandra-env.sh.
I was confident in defeating the imaginary bats, but was a little concerned that they should turn to dragons if I ignored them long enough.
A
On 17 Jan, 2011,at 02:37 PM, David Dabbs <dm...@gmail.com> wrote:
-----Original Message-----
From: Aaron Morton [mailto:aaron@thelastpickle.com]
Sent: Sunday, January 16, 2011 7:07 PM
To: user@cassandra.apache.org
Subject: Re: Cassandra and -XX:+UseLargePages
Chris, could you provide some more info on you experience? Were you using
mmapped files? Using row or key caches? What were the memtabe thresholds?
Using mlockall() ?
There are a couple of issues listed in the first paragraphs here that at
first glance may cause issues
http://www.oracle.com/technetwork/java/javase/tech/largememory-jsp-137182.ht
ml
cheers
Aaron
Aaron, are you referring to:
However please note sometimes using large page memory can negatively
affect system performance. For example, when a large amount of memory
is pinned by an application, it may create a shortage of regular memory
and cause excessive paging in other applications and slow down the entire
system. Also please note for a system that has been up for a long time,
excessive fragmentation can make it impossible to reserve enough large
page memory. When it happens, either the OS or JVM will revert to using
regular pages.
As I read this (and various posts about UseLargePages), you may need to
reboot the box and start your app quickly so memory is not fragmented
and that reserving lots of memory may starve others. Not really
issues if you're dedicating a box/VM to Cassandra, which I'm guessing
most folks are doing.
Or did you see something else that says "there be dragons" for Cassandra?
I didn't inquire before about experiences with the recent transparent
hugepages kernel patch (we're not on that fresh a kernel) but anyone's
experiences +/- would be of interest.
David Dabbs
On 17/01/2011, at 1:45 PM, Chris Goffinet <cg...@chrisgoffinet.com> wrote:
> I've seen about a 13% improvement in practice.
>
> -Chris
>
> On Jan 16, 2011, at 4:01 PM, David Dabbs wrote:
>
>> Hello.
>>
>> Can anyone comment on the performance impact (positive or negative)
>> of running Cassandra configured to use large pages under Linux?
>> Yes, YMMV applies, but I thought I'd ask before enlisting sysadmin Fu,
etc.
>>
>>
>> Thanks!
>>
>> David
>>
>>
>
RE: Cassandra and -XX:+UseLargePages
Posted by David Dabbs <dm...@gmail.com>.
-----Original Message-----
From: Aaron Morton [mailto:aaron@thelastpickle.com]
Sent: Sunday, January 16, 2011 7:07 PM
To: user@cassandra.apache.org
Subject: Re: Cassandra and -XX:+UseLargePages
Chris, could you provide some more info on you experience? Were you using
mmapped files? Using row or key caches? What were the memtabe thresholds?
Using mlockall() ?
There are a couple of issues listed in the first paragraphs here that at
first glance may cause issues
http://www.oracle.com/technetwork/java/javase/tech/largememory-jsp-137182.ht
ml
cheers
Aaron
Aaron, are you referring to:
However please note sometimes using large page memory can negatively
affect system performance. For example, when a large amount of memory
is pinned by an application, it may create a shortage of regular memory
and cause excessive paging in other applications and slow down the entire
system. Also please note for a system that has been up for a long time,
excessive fragmentation can make it impossible to reserve enough large
page memory. When it happens, either the OS or JVM will revert to using
regular pages.
As I read this (and various posts about UseLargePages), you may need to
reboot the box and start your app quickly so memory is not fragmented
and that reserving lots of memory may starve others. Not really
issues if you're dedicating a box/VM to Cassandra, which I'm guessing
most folks are doing.
Or did you see something else that says "there be dragons" for Cassandra?
I didn't inquire before about experiences with the recent transparent
hugepages kernel patch (we're not on that fresh a kernel) but anyone's
experiences +/- would be of interest.
David Dabbs
On 17/01/2011, at 1:45 PM, Chris Goffinet <cg...@chrisgoffinet.com> wrote:
> I've seen about a 13% improvement in practice.
>
> -Chris
>
> On Jan 16, 2011, at 4:01 PM, David Dabbs wrote:
>
>> Hello.
>>
>> Can anyone comment on the performance impact (positive or negative)
>> of running Cassandra configured to use large pages under Linux?
>> Yes, YMMV applies, but I thought I'd ask before enlisting sysadmin Fu,
etc.
>>
>>
>> Thanks!
>>
>> David
>>
>>
>
Re: Cassandra and -XX:+UseLargePages
Posted by Chris Goffinet <cg...@chrisgoffinet.com>.
DiskAccessMode mmap
mlockall() used
No row caches
Default keycache
Default memtable thresholds
-Chris
On Jan 16, 2011, at 5:06 PM, Aaron Morton wrote:
> Chris, could you provide some more info on you experience? Were you using mmapped files? Using row or key caches? What were the memtabe thresholds? Using mlockall() ?
>
> There are a couple of issues listed in the first paragraphs here that at first glance may cause issues http://www.oracle.com/technetwork/java/javase/tech/largememory-jsp-137182.html
>
> cheers
> Aaron
>
> On 17/01/2011, at 1:45 PM, Chris Goffinet <cg...@chrisgoffinet.com> wrote:
>
>> I've seen about a 13% improvement in practice.
>>
>> -Chris
>>
>> On Jan 16, 2011, at 4:01 PM, David Dabbs wrote:
>>
>>> Hello.
>>>
>>> Can anyone comment on the performance impact (positive or negative)
>>> of running Cassandra configured to use large pages under Linux?
>>> Yes, YMMV applies, but I thought I'd ask before enlisting sysadmin Fu, etc.
>>>
>>>
>>> Thanks!
>>>
>>> David
>>>
>>>
>>
Re: Cassandra and -XX:+UseLargePages
Posted by Aaron Morton <aa...@thelastpickle.com>.
Chris, could you provide some more info on you experience? Were you using mmapped files? Using row or key caches? What were the memtabe thresholds? Using mlockall() ?
There are a couple of issues listed in the first paragraphs here that at first glance may cause issues http://www.oracle.com/technetwork/java/javase/tech/largememory-jsp-137182.html
cheers
Aaron
On 17/01/2011, at 1:45 PM, Chris Goffinet <cg...@chrisgoffinet.com> wrote:
> I've seen about a 13% improvement in practice.
>
> -Chris
>
> On Jan 16, 2011, at 4:01 PM, David Dabbs wrote:
>
>> Hello.
>>
>> Can anyone comment on the performance impact (positive or negative)
>> of running Cassandra configured to use large pages under Linux?
>> Yes, YMMV applies, but I thought I'd ask before enlisting sysadmin Fu, etc.
>>
>>
>> Thanks!
>>
>> David
>>
>>
>
Re: Cassandra and -XX:+UseLargePages
Posted by Chris Goffinet <cg...@chrisgoffinet.com>.
I've seen about a 13% improvement in practice.
-Chris
On Jan 16, 2011, at 4:01 PM, David Dabbs wrote:
> Hello.
>
> Can anyone comment on the performance impact (positive or negative)
> of running Cassandra configured to use large pages under Linux?
> Yes, YMMV applies, but I thought I'd ask before enlisting sysadmin Fu, etc.
>
>
> Thanks!
>
> David
>
>