You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by Ramesh Natarajan <ra...@gmail.com> on 2011/10/04 04:47:08 UTC
Cassandra JVM heap size
I was reading an article @
http://www.acunu.com/products/choosing-cassandra/and it mentions
cassandra cannot benefit from more than 8GB allocated to JVM
heap. Is this true? Are these cassandra installations with larger heap
sizes? We are planning to have a cluster of 6 nodes with each node running
with about 100 GB or so RAM. Will this be a problem?
thanks
Ramesh
from http://www.acunu.com/products/choosing-cassandra/
Memory Ceiling
Cassandra typically cannot benefit from more than 8GB of RAM allocated to
the Java heap, imposing a hard limit on data size. Taking advantage of big
servers with lots of memory or many disks is no problem for Acunu. Thereʼs
no memory ceiling for Acunu and as a result, no data ceiling either. Need to
use larger servers? Go ahead.
Re: Cassandra JVM heap size
Posted by Yi Yang <i...@iyyang.com>.
Someone has just talked about the heap size in this mail list, says that bigger heap size will result into a longer GC phase, that could probably be one of the reason not using larger heap size.
But I have really heard of some others using Cassandra with some 60 gigabytes of heap size.
從我的 BlackBerry® 無線裝置
-----Original Message-----
From: Ramesh Natarajan <ra...@gmail.com>
Date: Mon, 3 Oct 2011 21:47:08
To: <us...@cassandra.apache.org>
Reply-To: user@cassandra.apache.org
Subject: Cassandra JVM heap size
I was reading an article @
http://www.acunu.com/products/choosing-cassandra/and it mentions
cassandra cannot benefit from more than 8GB allocated to JVM
heap. Is this true? Are these cassandra installations with larger heap
sizes? We are planning to have a cluster of 6 nodes with each node running
with about 100 GB or so RAM. Will this be a problem?
thanks
Ramesh
from http://www.acunu.com/products/choosing-cassandra/
Memory Ceiling
Cassandra typically cannot benefit from more than 8GB of RAM allocated to
the Java heap, imposing a hard limit on data size. Taking advantage of big
servers with lots of memory or many disks is no problem for Acunu. Thereʼs
no memory ceiling for Acunu and as a result, no data ceiling either. Need to
use larger servers? Go ahead.
Re: Cassandra JVM heap size
Posted by Jonathan Ellis <jb...@gmail.com>.
Sure, other things being equal.
Of course, other things are not truly equal and in practice I think
dual-quad-core, 32GB servers are at a good sweet spot for a lot of
applications.
As a rule of thumb, inserts will be cpu-bound and reads will be ram/io bound.
On Mon, Oct 3, 2011 at 11:10 PM, Ramesh Natarajan <ra...@gmail.com> wrote:
> Thanks. We are not planning to use row cache because we don't anticipate
> requests for the same row coming in often and we would better let the OS do
> the caching.. So does this mean in my case instead of running 6 servers
> with 100 GB each, I can run 75 servers with 8 GB RAM and set the Xms/Xmx to
> 4GB.
> thanks
> Ramesh
>
> On Mon, Oct 3, 2011 at 10:25 PM, Jonathan Ellis <jb...@gmail.com> wrote:
>>
>> That's misleading, because you don't necessarily need to give the
>> memory to the JVM for Cassandra to make use of it. (See, for example,
>>
>> http://www.datastax.com/dev/blog/whats-new-in-cassandra-1-0-improved-memory-and-disk-space-management.)
>> In fact it's counterproductive to increase heap size past the point
>> where it can handle the bloom filters + memtables for your data set.
>>
>> I suspect that the vast majority of deployments will not benefit from
>> heaps larger than 4GB, and there is a ticket open to make this the
>> default for 1.0: https://issues.apache.org/jira/browse/CASSANDRA-3295
>>
>> That said, if you have the choice it's generally better to choose
>> more, smaller servers than fewer, larger ones, primarily because it's
>> easier to deal with failures. If you had 12 nodes half as expensive,
>> for instance, losing one would be 1/12 of your capacity instead of
>> 1/6.
>>
>> On Mon, Oct 3, 2011 at 9:47 PM, Ramesh Natarajan <ra...@gmail.com>
>> wrote:
>> > I was reading an article
>> > @ http://www.acunu.com/products/choosing-cassandra/
>> > and it mentions cassandra cannot benefit from more than 8GB allocated to
>> > JVM
>> > heap. Is this true? Are these cassandra installations with larger heap
>> > sizes? We are planning to have a cluster of 6 nodes with each node
>> > running
>> > with about 100 GB or so RAM. Will this be a problem?
>> > thanks
>> > Ramesh
>> >
>> > from http://www.acunu.com/products/choosing-cassandra/
>> > Memory Ceiling
>> >
>> > Cassandra typically cannot benefit from more than 8GB of RAM allocated
>> > to
>> > the Java heap, imposing a hard limit on data size. Taking advantage of
>> > big
>> > servers with lots of memory or many disks is no problem for Acunu.
>> > Thereʼs
>> > no memory ceiling for Acunu and as a result, no data ceiling either.
>> > Need to
>> > use larger servers? Go ahead.
>>
>>
>>
>> --
>> Jonathan Ellis
>> Project Chair, Apache Cassandra
>> co-founder of DataStax, the source for professional Cassandra support
>> http://www.datastax.com
>
>
--
Jonathan Ellis
Project Chair, Apache Cassandra
co-founder of DataStax, the source for professional Cassandra support
http://www.datastax.com
Re: Cassandra JVM heap size
Posted by Ramesh Natarajan <ra...@gmail.com>.
Thanks. We are not planning to use row cache because we don't anticipate
requests for the same row coming in often and we would better let the OS do
the caching.. So does this mean in my case instead of running 6 servers
with 100 GB each, I can run 75 servers with 8 GB RAM and set the Xms/Xmx to
4GB.
thanks
Ramesh
On Mon, Oct 3, 2011 at 10:25 PM, Jonathan Ellis <jb...@gmail.com> wrote:
> That's misleading, because you don't necessarily need to give the
> memory to the JVM for Cassandra to make use of it. (See, for example,
>
> http://www.datastax.com/dev/blog/whats-new-in-cassandra-1-0-improved-memory-and-disk-space-management
> .)
> In fact it's counterproductive to increase heap size past the point
> where it can handle the bloom filters + memtables for your data set.
>
> I suspect that the vast majority of deployments will not benefit from
> heaps larger than 4GB, and there is a ticket open to make this the
> default for 1.0: https://issues.apache.org/jira/browse/CASSANDRA-3295
>
> That said, if you have the choice it's generally better to choose
> more, smaller servers than fewer, larger ones, primarily because it's
> easier to deal with failures. If you had 12 nodes half as expensive,
> for instance, losing one would be 1/12 of your capacity instead of
> 1/6.
>
> On Mon, Oct 3, 2011 at 9:47 PM, Ramesh Natarajan <ra...@gmail.com>
> wrote:
> > I was reading an article @
> http://www.acunu.com/products/choosing-cassandra/
> > and it mentions cassandra cannot benefit from more than 8GB allocated to
> JVM
> > heap. Is this true? Are these cassandra installations with larger heap
> > sizes? We are planning to have a cluster of 6 nodes with each node
> running
> > with about 100 GB or so RAM. Will this be a problem?
> > thanks
> > Ramesh
> >
> > from http://www.acunu.com/products/choosing-cassandra/
> > Memory Ceiling
> >
> > Cassandra typically cannot benefit from more than 8GB of RAM allocated to
> > the Java heap, imposing a hard limit on data size. Taking advantage of
> big
> > servers with lots of memory or many disks is no problem for Acunu.
> Thereʼs
> > no memory ceiling for Acunu and as a result, no data ceiling either. Need
> to
> > use larger servers? Go ahead.
>
>
>
> --
> Jonathan Ellis
> Project Chair, Apache Cassandra
> co-founder of DataStax, the source for professional Cassandra support
> http://www.datastax.com
>
Re: Cassandra JVM heap size
Posted by Jonathan Ellis <jb...@gmail.com>.
That's misleading, because you don't necessarily need to give the
memory to the JVM for Cassandra to make use of it. (See, for example,
http://www.datastax.com/dev/blog/whats-new-in-cassandra-1-0-improved-memory-and-disk-space-management.)
In fact it's counterproductive to increase heap size past the point
where it can handle the bloom filters + memtables for your data set.
I suspect that the vast majority of deployments will not benefit from
heaps larger than 4GB, and there is a ticket open to make this the
default for 1.0: https://issues.apache.org/jira/browse/CASSANDRA-3295
That said, if you have the choice it's generally better to choose
more, smaller servers than fewer, larger ones, primarily because it's
easier to deal with failures. If you had 12 nodes half as expensive,
for instance, losing one would be 1/12 of your capacity instead of
1/6.
On Mon, Oct 3, 2011 at 9:47 PM, Ramesh Natarajan <ra...@gmail.com> wrote:
> I was reading an article @ http://www.acunu.com/products/choosing-cassandra/
> and it mentions cassandra cannot benefit from more than 8GB allocated to JVM
> heap. Is this true? Are these cassandra installations with larger heap
> sizes? We are planning to have a cluster of 6 nodes with each node running
> with about 100 GB or so RAM. Will this be a problem?
> thanks
> Ramesh
>
> from http://www.acunu.com/products/choosing-cassandra/
> Memory Ceiling
>
> Cassandra typically cannot benefit from more than 8GB of RAM allocated to
> the Java heap, imposing a hard limit on data size. Taking advantage of big
> servers with lots of memory or many disks is no problem for Acunu. Thereʼs
> no memory ceiling for Acunu and as a result, no data ceiling either. Need to
> use larger servers? Go ahead.
--
Jonathan Ellis
Project Chair, Apache Cassandra
co-founder of DataStax, the source for professional Cassandra support
http://www.datastax.com