You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by Nicolas Labrot <ni...@gmail.com> on 2010/04/21 14:10:44 UTC
Cassandra tuning for running test on a desktop
Hello,
For my first message I will first thanks Cassandra contributors for their
great works.
I have a parameter issue with Cassandra (I hope it's just a parameter
issue). I'm using Cassandra 6.0.1 with Hector client on my desktop. It's a
simple dual core with 4GB of RAM on WinXP. I have keep the default JVM
option inside cassandra.bat (Xmx1G)
I'm trying to insert 3 millions of SC with 6 Columns each inside 1 CF (named
Super1). The insertion go to 1 millions of SC (without slowdown) and
Cassandra crash because of an OOM. (I store an average of 100 bytes per SC
with a max of 10kB).
I have aggressively decreased all the memories parameters without any
respect to the consistency (My config is here [1]), the cache is turn off
but Cassandra still go to OOM. I have joined the last line of the Cassandra
life [2].
What can I do to fix my issue ? Is there another solution than increasing
the Xmx ?
Thanks for your help,
Nicolas
[1]
<Keyspaces>
<Keyspace Name="Keyspace1">
<ColumnFamily Name="Super1"
ColumnType="Super"
CompareWith="BytesType"
CompareSubcolumnsWith="BytesType" />
<ReplicaPlacementStrategy>org.apache.cassandra.locator.RackUnawareStrategy</ReplicaPlacementStrategy>
<ReplicationFactor>1</ReplicationFactor>
<EndPointSnitch>org.apache.cassandra.locator.EndPointSnitch</EndPointSnitch>
</Keyspace>
</Keyspaces>
<CommitLogRotationThresholdInMB>32</CommitLogRotationThresholdInMB>
<DiskAccessMode>auto</DiskAccessMode>
<RowWarningThresholdInMB>64</RowWarningThresholdInMB>
<SlicedBufferSizeInKB>64</SlicedBufferSizeInKB>
<FlushDataBufferSizeInMB>16</FlushDataBufferSizeInMB>
<FlushIndexBufferSizeInMB>4</FlushIndexBufferSizeInMB>
<ColumnIndexSizeInKB>64</ColumnIndexSizeInKB>
<MemtableThroughputInMB>16</MemtableThroughputInMB>
<BinaryMemtableThroughputInMB>32</BinaryMemtableThroughputInMB>
<MemtableOperationsInMillions>0.01</MemtableOperationsInMillions>
<MemtableObjectCountInMillions>0.01</MemtableObjectCountInMillions>
<MemtableFlushAfterMinutes>60</MemtableFlushAfterMinutes>
<ConcurrentReads>4</ConcurrentReads>
<ConcurrentWrites>8</ConcurrentWrites>
</Storage>
[2]
INFO 13:36:41,062 Super1 has reached its threshold; switching in a fresh
Memtable at
CommitLogContext(file='d:/cassandra/commitlog\CommitLog-1271849783703.log',
position=5417524)
INFO 13:36:41,062 Enqueuing flush of Memtable(Super1)@15385755
INFO 13:36:41,062 Writing Memtable(Super1)@15385755
INFO 13:36:42,062 Completed flushing
d:\cassandra\data\Keyspace1\Super1-711-Data.db
INFO 13:36:45,781 Super1 has reached its threshold; switching in a fresh
Memtable at
CommitLogContext(file='d:/cassandra/commitlog\CommitLog-1271849783703.log',
position=6065637)
INFO 13:36:45,781 Enqueuing flush of Memtable(Super1)@15578910
INFO 13:36:45,796 Writing Memtable(Super1)@15578910
INFO 13:36:46,109 Completed flushing
d:\cassandra\data\Keyspace1\Super1-712-Data.db
INFO 13:36:54,296 GC for ConcurrentMarkSweep: 7149 ms, 58337240 reclaimed
leaving 922392600 used; max is 1174208512
INFO 13:36:54,593 Super1 has reached its threshold; switching in a fresh
Memtable at
CommitLogContext(file='d:/cassandra/commitlog\CommitLog-1271849783703.log',
position=6722241)
INFO 13:36:54,593 Enqueuing flush of Memtable(Super1)@24468872
INFO 13:36:54,593 Writing Memtable(Super1)@24468872
INFO 13:36:55,421 Completed flushing
d:\cassandra\data\Keyspace1\Super1-713-Data.dbjava.lang.OutOfMemoryError:
Java heap space
INFO 13:37:08,281 GC for ConcurrentMarkSweep: 5561 ms, 9432 reclaimed
leaving 971904520 used; max is 1174208512
Re: At what point does the cluster get faster than the individual
nodes?
Posted by "Jim R. Wilson" <wi...@gmail.com>.
Hi Mark,
I'm a relative newcomer to Cassandra, but I believe the common
experience is that you start seeing gains after 5 nodes in a
column-oriented data store. It may also depend on your usage pattern.
Others may know better - hope this helps!
-- Jim R. Wilson (jimbojw)
On Wed, Apr 21, 2010 at 11:28 AM, Mark Jones <MJ...@imagehawk.com> wrote:
> I’m seeing a cluster of 4 (replication factor=2) to be about as slow overall
> as the barely faster than the slowest node in the group. When I run the 4
> nodes individually, I see:
>
>
>
> For inserts:
>
> Two nodes @ 12000/second
>
> 1 node @ 9000/second
>
> 1 node @ 7000/second
>
>
>
> For reads:
>
> Abysmal, less than 1000/second (not range slices, individual lookups) Disk
> util @ 88+%
>
>
>
>
>
> How many nodes are required before you see a net positive gain on inserts
> and reads (QUORUM consistency on both)?
>
> When I use my 2 fastest nodes as a pair, the thruput is around 9000
> inserts/second.
>
>
>
> What is a good to excellent hardware config for Cassandra? I have separate
> drives for data and commit log and 8GB in 3 machines (all dual core). My
> fastest insert node has 4GB and a triple core processor.
>
>
>
> I’ve run py_stress, and my C++ code beats it by several 1000 inserts/second
> toward the end of the runs, so I don’t think it is my app, and I’ve removed
> the super columns per some suggestions yesterday.
>
>
>
> When Cassandra is working, it performs well, the problem is that is
> frequently slows down to < 50% of its peaks and occasionally slows down to 0
> inserts/second which greatly reduces aggregate thruput.
Re: Row deletion and get_range_slices (cassandra 0.6.1)
Posted by Jonathan Ellis <jb...@gmail.com>.
On Fri, Apr 23, 2010 at 3:53 AM, David Harrison
<da...@gmail.com> wrote:
> So I'm guessing that means compaction doesn't include purging of
> tombstone-d keys ?
Incorrect.
http://wiki.apache.org/cassandra/DistributedDeletes
http://wiki.apache.org/cassandra/MemtableSSTable
Re: Row deletion and get_range_slices (cassandra 0.6.1)
Posted by David Harrison <da...@gmail.com>.
So I'm guessing that means compaction doesn't include purging of
tombstone-d keys ? Is there any situation or maintenance process that
does ? (or are keys forever?)
On 23 April 2010 17:44, Ryan King <ry...@twitter.com> wrote:
> On Thu, Apr 22, 2010 at 8:24 PM, David Harrison
> <da...@gmail.com> wrote:
>> Do those tombstone-d keys ever get purged completely ? I've tried
>> shortening the GCGraceSeconds right down but they still don't get
>> cleaned up.
>
> The GCGraceSeconds will only apply when you compact data.
>
> -ryan
>
Re: Row deletion and get_range_slices (cassandra 0.6.1)
Posted by Ryan King <ry...@twitter.com>.
On Thu, Apr 22, 2010 at 8:24 PM, David Harrison
<da...@gmail.com> wrote:
> Do those tombstone-d keys ever get purged completely ? I've tried
> shortening the GCGraceSeconds right down but they still don't get
> cleaned up.
The GCGraceSeconds will only apply when you compact data.
-ryan
Re: Row deletion and get_range_slices (cassandra 0.6.1)
Posted by David Harrison <da...@gmail.com>.
Do those tombstone-d keys ever get purged completely ? I've tried
shortening the GCGraceSeconds right down but they still don't get
cleaned up.
On 23 April 2010 08:57, Jonathan Ellis <jb...@gmail.com> wrote:
> http://wiki.apache.org/cassandra/FAQ#range_ghosts
>
> On Thu, Apr 22, 2010 at 5:29 PM, Carlos Sanchez
> <ca...@riskmetrics.com> wrote:
>> I have a curious question..
>>
>> I am doing some testing where I insert 500 rows to a super column family and then delete one row, I make sure the row was indeed deleted (NotFoundException in the get call) and then I ran a get_range_slices and the row indeed returned. The shutdown Cassandra and restarted it. I repeated the test (with inserting the rows) and even though I get the NotFoundException for that row, the get_rance_slices still returns it. Is this the expected behavior? How long should I wait before I don't see the row in the get_range_slices? Do I have to force a flush or change consistency level?
>>
>> Thanks,
>>
>> Carlos
>>
>> This email message and any attachments are for the sole use of the intended recipients and may contain proprietary and/or confidential information which may be privileged or otherwise protected from disclosure. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not an intended recipient, please contact the sender by reply email and destroy the original message and any copies of the message as well as any attachments to the original message.
>>
>
Re: Row deletion and get_range_slices (cassandra 0.6.1)
Posted by Jonathan Ellis <jb...@gmail.com>.
http://wiki.apache.org/cassandra/FAQ#range_ghosts
On Thu, Apr 22, 2010 at 5:29 PM, Carlos Sanchez
<ca...@riskmetrics.com> wrote:
> I have a curious question..
>
> I am doing some testing where I insert 500 rows to a super column family and then delete one row, I make sure the row was indeed deleted (NotFoundException in the get call) and then I ran a get_range_slices and the row indeed returned. The shutdown Cassandra and restarted it. I repeated the test (with inserting the rows) and even though I get the NotFoundException for that row, the get_rance_slices still returns it. Is this the expected behavior? How long should I wait before I don't see the row in the get_range_slices? Do I have to force a flush or change consistency level?
>
> Thanks,
>
> Carlos
>
> This email message and any attachments are for the sole use of the intended recipients and may contain proprietary and/or confidential information which may be privileged or otherwise protected from disclosure. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not an intended recipient, please contact the sender by reply email and destroy the original message and any copies of the message as well as any attachments to the original message.
>
Row deletion and get_range_slices (cassandra 0.6.1)
Posted by Carlos Sanchez <ca...@riskmetrics.com>.
I have a curious question..
I am doing some testing where I insert 500 rows to a super column family and then delete one row, I make sure the row was indeed deleted (NotFoundException in the get call) and then I ran a get_range_slices and the row indeed returned. The shutdown Cassandra and restarted it. I repeated the test (with inserting the rows) and even though I get the NotFoundException for that row, the get_rance_slices still returns it. Is this the expected behavior? How long should I wait before I don't see the row in the get_range_slices? Do I have to force a flush or change consistency level?
Thanks,
Carlos
This email message and any attachments are for the sole use of the intended recipients and may contain proprietary and/or confidential information which may be privileged or otherwise protected from disclosure. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not an intended recipient, please contact the sender by reply email and destroy the original message and any copies of the message as well as any attachments to the original message.
Re: At what point does the cluster get faster than the individual
nodes?
Posted by Jonathan Ellis <jb...@gmail.com>.
fyi,
https://issues.apache.org/jira/browse/CASSANDRA-930
https://issues.apache.org/jira/browse/CASSANDRA-982
On Thu, Apr 22, 2010 at 11:11 AM, Mike Malone <mi...@simplegeo.com> wrote:
> On Wed, Apr 21, 2010 at 9:50 AM, Mark Greene <gr...@gmail.com> wrote:
>>
>> Right it's a similar concept to DB sharding where you spread the write
>> load around to different DB servers but won't necessarily increase the
>> throughput of an one DB server but rather collectively.
>
> Except with Cassandra, read-repair causes every read to go to every replica
> for a piece of data.
> Mike
Re: At what point does the cluster get faster than the individual
nodes?
Posted by Mike Malone <mi...@simplegeo.com>.
On Wed, Apr 21, 2010 at 9:50 AM, Mark Greene <gr...@gmail.com> wrote:
> Right it's a similar concept to DB sharding where you spread the write load
> around to different DB servers but won't necessarily increase the throughput
> of an one DB server but rather collectively.
Except with Cassandra, read-repair causes every read to go to every replica
for a piece of data.
Mike
Re: At what point does the cluster get faster than the individual
nodes?
Posted by Mark Greene <gr...@gmail.com>.
Right it's a similar concept to DB sharding where you spread the write load
around to different DB servers but won't necessarily increase the throughput
of an one DB server but rather collectively.
On Wed, Apr 21, 2010 at 12:16 PM, Mike Gallamore <
mike.e.gallamore@googlemail.com> wrote:
> Some people might be able to answer this better than me. However: with
> quorum consistency you have to communicate with n/2 + 1 where n is the
> replication factor nodes. So unless you are disk bound your real expense is
> going to be all those extra network latencies. I'd expect that you'll see a
> relatively flat throughput per thread once you reach the point that you
> aren't disk or CPU bound. That said the extra nodes mean if you should be
> able to handle more threads/connections at the same throughput on each
> thread/connection. So bigger cluster doesn't mean a single job goes faster
> necessarily, just that you can handle more jobs at the same time.
>
> On 04/21/2010 08:28 AM, Mark Jones wrote:
>
> I’m seeing a cluster of 4 (replication factor=2) to be about as slow
> overall as the barely faster than the slowest node in the group. When I run
> the 4 nodes individually, I see:
>
>
>
> For inserts:
>
> Two nodes @ 12000/second
>
> 1 node @ 9000/second
>
> 1 node @ 7000/second
>
>
>
> For reads:
>
> Abysmal, less than 1000/second (not range slices, individual lookups) Disk
> util @ 88+%
>
>
>
>
>
> How many nodes are required before you see a net positive gain on inserts
> and reads (QUORUM consistency on both)?
>
> When I use my 2 fastest nodes as a pair, the thruput is around 9000
> inserts/second.
>
>
>
> What is a good to excellent hardware config for Cassandra? I have separate
> drives for data and commit log and 8GB in 3 machines (all dual core). My
> fastest insert node has 4GB and a triple core processor.
>
>
>
> I’ve run py_stress, and my C++ code beats it by several 1000 inserts/second
> toward the end of the runs, so I don’t think it is my app, and I’ve removed
> the super columns per some suggestions yesterday.
>
>
>
> When Cassandra is working, it performs well, the problem is that is
> frequently slows down to < 50% of its peaks and occasionally slows down to 0
> inserts/second which greatly reduces aggregate thruput.
>
>
>
Re: At what point does the cluster get faster than the individual
nodes?
Posted by Mike Gallamore <mi...@googlemail.com>.
Some people might be able to answer this better than me. However: with
quorum consistency you have to communicate with n/2 + 1 where n is the
replication factor nodes. So unless you are disk bound your real expense
is going to be all those extra network latencies. I'd expect that you'll
see a relatively flat throughput per thread once you reach the point
that you aren't disk or CPU bound. That said the extra nodes mean if you
should be able to handle more threads/connections at the same throughput
on each thread/connection. So bigger cluster doesn't mean a single job
goes faster necessarily, just that you can handle more jobs at the same
time.
On 04/21/2010 08:28 AM, Mark Jones wrote:
>
> I'm seeing a cluster of 4 (replication factor=2) to be about as slow
> overall as the barely faster than the slowest node in the group. When
> I run the 4 nodes individually, I see:
>
> For inserts:
>
> Two nodes @ 12000/second
>
> 1 node @ 9000/second
>
> 1 node @ 7000/second
>
> For reads:
>
> Abysmal, less than 1000/second (not range slices, individual lookups)
> Disk util @ 88+%
>
> How many nodes are required before you see a net positive gain on
> inserts and reads (QUORUM consistency on both)?
>
> When I use my 2 fastest nodes as a pair, the thruput is around 9000
> inserts/second.
>
> What is a good to excellent hardware config for Cassandra? I have
> separate drives for data and commit log and 8GB in 3 machines (all
> dual core). My fastest insert node has 4GB and a triple core processor.
>
> I've run py_stress, and my C++ code beats it by several 1000
> inserts/second toward the end of the runs, so I don't think it is my
> app, and I've removed the super columns per some suggestions yesterday.
>
> When Cassandra is working, it performs well, the problem is that is
> frequently slows down to < 50% of its peaks and occasionally slows
> down to 0 inserts/second which greatly reduces aggregate thruput.
>
At what point does the cluster get faster than the individual nodes?
Posted by Mark Jones <MJ...@imagehawk.com>.
I'm seeing a cluster of 4 (replication factor=2) to be about as slow overall as the barely faster than the slowest node in the group. When I run the 4 nodes individually, I see:
For inserts:
Two nodes @ 12000/second
1 node @ 9000/second
1 node @ 7000/second
For reads:
Abysmal, less than 1000/second (not range slices, individual lookups) Disk util @ 88+%
How many nodes are required before you see a net positive gain on inserts and reads (QUORUM consistency on both)?
When I use my 2 fastest nodes as a pair, the thruput is around 9000 inserts/second.
What is a good to excellent hardware config for Cassandra? I have separate drives for data and commit log and 8GB in 3 machines (all dual core). My fastest insert node has 4GB and a triple core processor.
I've run py_stress, and my C++ code beats it by several 1000 inserts/second toward the end of the runs, so I don't think it is my app, and I've removed the super columns per some suggestions yesterday.
When Cassandra is working, it performs well, the problem is that is frequently slows down to < 50% of its peaks and occasionally slows down to 0 inserts/second which greatly reduces aggregate thruput.
Re: Cassandra tuning for running test on a desktop
Posted by Nicolas Labrot <ni...@gmail.com>.
Yes I think. I have read this wiki entry and the JIRA. I will use different
row key until it will be fixed
Thanks,
Nicolas
On Thu, Apr 22, 2010 at 4:47 AM, Stu Hood <st...@rackspace.com> wrote:
> Nicolas,
>
> Were all of those super column writes going to the same row?
> http://wiki.apache.org/cassandra/CassandraLimitations
>
> Thanks,
> Stu
>
> -----Original Message-----
> From: "Nicolas Labrot" <ni...@gmail.com>
> Sent: Wednesday, April 21, 2010 11:54am
> To: user@cassandra.apache.org
> Subject: Re: Cassandra tuning for running test on a desktop
>
> I donnot have a website ;)
>
> I'm testing the viability of Cassandra to store XML documents and make fast
> search queries. 4000 XML files (80MB of XML) create with my datamodel (one
> SC per XML node) 1000000 SC which make Cassandra go OOM with Xmx 1GB. On
> the
> contrary an xml DB like eXist handles 4000 XML doc without any problem with
> an acceptable amount of memories.
>
> What I like with Cassandra is his simplicity and his scalability. eXist is
> not able to scale with data, the only viable solution his marklogic which
> cost an harm and a feet... :)
>
> I will install linux and buy some memories to continue my test.
>
> Could a Cassandra developper give me the technical reason of this OOM ?
>
>
>
>
>
> On Wed, Apr 21, 2010 at 5:13 PM, Mark Greene <gr...@gmail.com> wrote:
>
> > Maybe, maybe not. Presumably if you are running a RDMS with any
> reasonable
> > amount of traffic now a days, it's sitting on a machine with 4-8G of
> memory
> > at least.
> >
> >
> > On Wed, Apr 21, 2010 at 10:48 AM, Nicolas Labrot <nithril@gmail.com
> >wrote:
> >
> >> Thanks Mark.
> >>
> >> Cassandra is maybe too much for my need ;)
> >>
> >>
> >>
> >> On Wed, Apr 21, 2010 at 4:45 PM, Mark Greene <gr...@gmail.com>
> wrote:
> >>
> >>> Hit send to early....
> >>>
> >>> That being said a lot of people running Cassandra in production are
> using
> >>> 4-6GB max heaps on 8GB machines, don't know if that helps but hopefully
> >>> gives you some perspective.
> >>>
> >>>
> >>> On Wed, Apr 21, 2010 at 10:39 AM, Mark Greene <greenemj@gmail.com
> >wrote:
> >>>
> >>>> RAM doesn't necessarily need to be proportional but I would say the
> >>>> number of nodes does. You can't just throw a bazillion inserts at one
> node.
> >>>> This is the main benefit of Cassandra is that if you start hitting
> your
> >>>> capacity, you add more machines and distribute the keys across more
> >>>> machines.
> >>>>
> >>>>
> >>>> On Wed, Apr 21, 2010 at 9:07 AM, Nicolas Labrot <nithril@gmail.com
> >wrote:
> >>>>
> >>>>> So does it means the RAM needed is proportionnal with the data
> handled
> >>>>> ?
> >>>>>
> >>>>> Or Cassandra need a minimum amount or RAM when dataset is big?
> >>>>>
> >>>>> I must confess this OOM behaviour is strange.
> >>>>>
> >>>>>
> >>>>> On Wed, Apr 21, 2010 at 2:54 PM, Mark Jones <MJones@imagehawk.com
> >wrote:
> >>>>>
> >>>>>> On my 4GB machine I’m giving it 3GB and having no trouble with 60+
> >>>>>> million 500 byte columns
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> *From:* Nicolas Labrot [mailto:nithril@gmail.com]
> >>>>>> *Sent:* Wednesday, April 21, 2010 7:47 AM
> >>>>>> *To:* user@cassandra.apache.org
> >>>>>> *Subject:* Re: Cassandra tuning for running test on a desktop
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> I have try 1400M, and Cassandra OOM too.
> >>>>>>
> >>>>>> Is there another solution ? My data isn't very big.
> >>>>>>
> >>>>>> It seems that is the merge of the db
> >>>>>>
> >>>>>> On Wed, Apr 21, 2010 at 2:14 PM, Mark Greene <gr...@gmail.com>
> >>>>>> wrote:
> >>>>>>
> >>>>>> Trying increasing Xmx. 1G is probably not enough for the amount of
> >>>>>> inserts you are doing.
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> On Wed, Apr 21, 2010 at 8:10 AM, Nicolas Labrot <ni...@gmail.com>
> >>>>>> wrote:
> >>>>>>
> >>>>>> Hello,
> >>>>>>
> >>>>>> For my first message I will first thanks Cassandra contributors for
> >>>>>> their great works.
> >>>>>>
> >>>>>> I have a parameter issue with Cassandra (I hope it's just a
> parameter
> >>>>>> issue). I'm using Cassandra 6.0.1 with Hector client on my desktop.
> It's a
> >>>>>> simple dual core with 4GB of RAM on WinXP. I have keep the default
> JVM
> >>>>>> option inside cassandra.bat (Xmx1G)
> >>>>>>
> >>>>>> I'm trying to insert 3 millions of SC with 6 Columns each inside 1
> CF
> >>>>>> (named Super1). The insertion go to 1 millions of SC (without
> slowdown) and
> >>>>>> Cassandra crash because of an OOM. (I store an average of 100 bytes
> per SC
> >>>>>> with a max of 10kB).
> >>>>>> I have aggressively decreased all the memories parameters without
> any
> >>>>>> respect to the consistency (My config is here [1]), the cache is
> turn off
> >>>>>> but Cassandra still go to OOM. I have joined the last line of the
> Cassandra
> >>>>>> life [2].
> >>>>>>
> >>>>>> What can I do to fix my issue ? Is there another solution than
> >>>>>> increasing the Xmx ?
> >>>>>>
> >>>>>> Thanks for your help,
> >>>>>>
> >>>>>> Nicolas
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> [1]
> >>>>>> <Keyspaces>
> >>>>>> <Keyspace Name="Keyspace1">
> >>>>>> <ColumnFamily Name="Super1"
> >>>>>> ColumnType="Super"
> >>>>>> CompareWith="BytesType"
> >>>>>> CompareSubcolumnsWith="BytesType" />
> >>>>>>
> >>>>>>
> <ReplicaPlacementStrategy>org.apache.cassandra.locator.RackUnawareStrategy</ReplicaPlacementStrategy>
> >>>>>> <ReplicationFactor>1</ReplicationFactor>
> >>>>>>
> >>>>>>
> <EndPointSnitch>org.apache.cassandra.locator.EndPointSnitch</EndPointSnitch>
> >>>>>> </Keyspace>
> >>>>>> </Keyspaces>
> >>>>>>
> <CommitLogRotationThresholdInMB>32</CommitLogRotationThresholdInMB>
> >>>>>>
> >>>>>> <DiskAccessMode>auto</DiskAccessMode>
> >>>>>> <RowWarningThresholdInMB>64</RowWarningThresholdInMB>
> >>>>>> <SlicedBufferSizeInKB>64</SlicedBufferSizeInKB>
> >>>>>> <FlushDataBufferSizeInMB>16</FlushDataBufferSizeInMB>
> >>>>>> <FlushIndexBufferSizeInMB>4</FlushIndexBufferSizeInMB>
> >>>>>> <ColumnIndexSizeInKB>64</ColumnIndexSizeInKB>
> >>>>>>
> >>>>>> <MemtableThroughputInMB>16</MemtableThroughputInMB>
> >>>>>> <BinaryMemtableThroughputInMB>32</BinaryMemtableThroughputInMB>
> >>>>>> <MemtableOperationsInMillions>0.01</MemtableOperationsInMillions>
> >>>>>>
> <MemtableObjectCountInMillions>0.01</MemtableObjectCountInMillions>
> >>>>>> <MemtableFlushAfterMinutes>60</MemtableFlushAfterMinutes>
> >>>>>> <ConcurrentReads>4</ConcurrentReads>
> >>>>>> <ConcurrentWrites>8</ConcurrentWrites>
> >>>>>> </Storage>
> >>>>>>
> >>>>>>
> >>>>>> [2]
> >>>>>> INFO 13:36:41,062 Super1 has reached its threshold; switching in a
> >>>>>> fresh Memtable at
> >>>>>>
> CommitLogContext(file='d:/cassandra/commitlog\CommitLog-1271849783703.log',
> >>>>>> position=5417524)
> >>>>>> INFO 13:36:41,062 Enqueuing flush of Memtable(Super1)@15385755
> >>>>>> INFO 13:36:41,062 Writing Memtable(Super1)@15385755
> >>>>>> INFO 13:36:42,062 Completed flushing
> >>>>>> d:\cassandra\data\Keyspace1\Super1-711-Data.db
> >>>>>> INFO 13:36:45,781 Super1 has reached its threshold; switching in a
> >>>>>> fresh Memtable at
> >>>>>>
> CommitLogContext(file='d:/cassandra/commitlog\CommitLog-1271849783703.log',
> >>>>>> position=6065637)
> >>>>>> INFO 13:36:45,781 Enqueuing flush of Memtable(Super1)@15578910
> >>>>>> INFO 13:36:45,796 Writing Memtable(Super1)@15578910
> >>>>>> INFO 13:36:46,109 Completed flushing
> >>>>>> d:\cassandra\data\Keyspace1\Super1-712-Data.db
> >>>>>> INFO 13:36:54,296 GC for ConcurrentMarkSweep: 7149 ms, 58337240
> >>>>>> reclaimed leaving 922392600 used; max is 1174208512
> >>>>>> INFO 13:36:54,593 Super1 has reached its threshold; switching in a
> >>>>>> fresh Memtable at
> >>>>>>
> CommitLogContext(file='d:/cassandra/commitlog\CommitLog-1271849783703.log',
> >>>>>> position=6722241)
> >>>>>> INFO 13:36:54,593 Enqueuing flush of Memtable(Super1)@24468872
> >>>>>> INFO 13:36:54,593 Writing Memtable(Super1)@24468872
> >>>>>> INFO 13:36:55,421 Completed flushing
> >>>>>>
> d:\cassandra\data\Keyspace1\Super1-713-Data.dbjava.lang.OutOfMemoryError:
> >>>>>> Java heap space
> >>>>>> INFO 13:37:08,281 GC for ConcurrentMarkSweep: 5561 ms, 9432
> reclaimed
> >>>>>> leaving 971904520 used; max is 1174208512
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>
> >>>>>
> >>>>
> >>>
> >>
> >
>
>
>
Re: Cassandra tuning for running test on a desktop
Posted by Stu Hood <st...@rackspace.com>.
Nicolas,
Were all of those super column writes going to the same row? http://wiki.apache.org/cassandra/CassandraLimitations
Thanks,
Stu
-----Original Message-----
From: "Nicolas Labrot" <ni...@gmail.com>
Sent: Wednesday, April 21, 2010 11:54am
To: user@cassandra.apache.org
Subject: Re: Cassandra tuning for running test on a desktop
I donnot have a website ;)
I'm testing the viability of Cassandra to store XML documents and make fast
search queries. 4000 XML files (80MB of XML) create with my datamodel (one
SC per XML node) 1000000 SC which make Cassandra go OOM with Xmx 1GB. On the
contrary an xml DB like eXist handles 4000 XML doc without any problem with
an acceptable amount of memories.
What I like with Cassandra is his simplicity and his scalability. eXist is
not able to scale with data, the only viable solution his marklogic which
cost an harm and a feet... :)
I will install linux and buy some memories to continue my test.
Could a Cassandra developper give me the technical reason of this OOM ?
On Wed, Apr 21, 2010 at 5:13 PM, Mark Greene <gr...@gmail.com> wrote:
> Maybe, maybe not. Presumably if you are running a RDMS with any reasonable
> amount of traffic now a days, it's sitting on a machine with 4-8G of memory
> at least.
>
>
> On Wed, Apr 21, 2010 at 10:48 AM, Nicolas Labrot <ni...@gmail.com>wrote:
>
>> Thanks Mark.
>>
>> Cassandra is maybe too much for my need ;)
>>
>>
>>
>> On Wed, Apr 21, 2010 at 4:45 PM, Mark Greene <gr...@gmail.com> wrote:
>>
>>> Hit send to early....
>>>
>>> That being said a lot of people running Cassandra in production are using
>>> 4-6GB max heaps on 8GB machines, don't know if that helps but hopefully
>>> gives you some perspective.
>>>
>>>
>>> On Wed, Apr 21, 2010 at 10:39 AM, Mark Greene <gr...@gmail.com>wrote:
>>>
>>>> RAM doesn't necessarily need to be proportional but I would say the
>>>> number of nodes does. You can't just throw a bazillion inserts at one node.
>>>> This is the main benefit of Cassandra is that if you start hitting your
>>>> capacity, you add more machines and distribute the keys across more
>>>> machines.
>>>>
>>>>
>>>> On Wed, Apr 21, 2010 at 9:07 AM, Nicolas Labrot <ni...@gmail.com>wrote:
>>>>
>>>>> So does it means the RAM needed is proportionnal with the data handled
>>>>> ?
>>>>>
>>>>> Or Cassandra need a minimum amount or RAM when dataset is big?
>>>>>
>>>>> I must confess this OOM behaviour is strange.
>>>>>
>>>>>
>>>>> On Wed, Apr 21, 2010 at 2:54 PM, Mark Jones <MJ...@imagehawk.com>wrote:
>>>>>
>>>>>> On my 4GB machine I’m giving it 3GB and having no trouble with 60+
>>>>>> million 500 byte columns
>>>>>>
>>>>>>
>>>>>>
>>>>>> *From:* Nicolas Labrot [mailto:nithril@gmail.com]
>>>>>> *Sent:* Wednesday, April 21, 2010 7:47 AM
>>>>>> *To:* user@cassandra.apache.org
>>>>>> *Subject:* Re: Cassandra tuning for running test on a desktop
>>>>>>
>>>>>>
>>>>>>
>>>>>> I have try 1400M, and Cassandra OOM too.
>>>>>>
>>>>>> Is there another solution ? My data isn't very big.
>>>>>>
>>>>>> It seems that is the merge of the db
>>>>>>
>>>>>> On Wed, Apr 21, 2010 at 2:14 PM, Mark Greene <gr...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>> Trying increasing Xmx. 1G is probably not enough for the amount of
>>>>>> inserts you are doing.
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Wed, Apr 21, 2010 at 8:10 AM, Nicolas Labrot <ni...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>> Hello,
>>>>>>
>>>>>> For my first message I will first thanks Cassandra contributors for
>>>>>> their great works.
>>>>>>
>>>>>> I have a parameter issue with Cassandra (I hope it's just a parameter
>>>>>> issue). I'm using Cassandra 6.0.1 with Hector client on my desktop. It's a
>>>>>> simple dual core with 4GB of RAM on WinXP. I have keep the default JVM
>>>>>> option inside cassandra.bat (Xmx1G)
>>>>>>
>>>>>> I'm trying to insert 3 millions of SC with 6 Columns each inside 1 CF
>>>>>> (named Super1). The insertion go to 1 millions of SC (without slowdown) and
>>>>>> Cassandra crash because of an OOM. (I store an average of 100 bytes per SC
>>>>>> with a max of 10kB).
>>>>>> I have aggressively decreased all the memories parameters without any
>>>>>> respect to the consistency (My config is here [1]), the cache is turn off
>>>>>> but Cassandra still go to OOM. I have joined the last line of the Cassandra
>>>>>> life [2].
>>>>>>
>>>>>> What can I do to fix my issue ? Is there another solution than
>>>>>> increasing the Xmx ?
>>>>>>
>>>>>> Thanks for your help,
>>>>>>
>>>>>> Nicolas
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> [1]
>>>>>> <Keyspaces>
>>>>>> <Keyspace Name="Keyspace1">
>>>>>> <ColumnFamily Name="Super1"
>>>>>> ColumnType="Super"
>>>>>> CompareWith="BytesType"
>>>>>> CompareSubcolumnsWith="BytesType" />
>>>>>>
>>>>>> <ReplicaPlacementStrategy>org.apache.cassandra.locator.RackUnawareStrategy</ReplicaPlacementStrategy>
>>>>>> <ReplicationFactor>1</ReplicationFactor>
>>>>>>
>>>>>> <EndPointSnitch>org.apache.cassandra.locator.EndPointSnitch</EndPointSnitch>
>>>>>> </Keyspace>
>>>>>> </Keyspaces>
>>>>>> <CommitLogRotationThresholdInMB>32</CommitLogRotationThresholdInMB>
>>>>>>
>>>>>> <DiskAccessMode>auto</DiskAccessMode>
>>>>>> <RowWarningThresholdInMB>64</RowWarningThresholdInMB>
>>>>>> <SlicedBufferSizeInKB>64</SlicedBufferSizeInKB>
>>>>>> <FlushDataBufferSizeInMB>16</FlushDataBufferSizeInMB>
>>>>>> <FlushIndexBufferSizeInMB>4</FlushIndexBufferSizeInMB>
>>>>>> <ColumnIndexSizeInKB>64</ColumnIndexSizeInKB>
>>>>>>
>>>>>> <MemtableThroughputInMB>16</MemtableThroughputInMB>
>>>>>> <BinaryMemtableThroughputInMB>32</BinaryMemtableThroughputInMB>
>>>>>> <MemtableOperationsInMillions>0.01</MemtableOperationsInMillions>
>>>>>> <MemtableObjectCountInMillions>0.01</MemtableObjectCountInMillions>
>>>>>> <MemtableFlushAfterMinutes>60</MemtableFlushAfterMinutes>
>>>>>> <ConcurrentReads>4</ConcurrentReads>
>>>>>> <ConcurrentWrites>8</ConcurrentWrites>
>>>>>> </Storage>
>>>>>>
>>>>>>
>>>>>> [2]
>>>>>> INFO 13:36:41,062 Super1 has reached its threshold; switching in a
>>>>>> fresh Memtable at
>>>>>> CommitLogContext(file='d:/cassandra/commitlog\CommitLog-1271849783703.log',
>>>>>> position=5417524)
>>>>>> INFO 13:36:41,062 Enqueuing flush of Memtable(Super1)@15385755
>>>>>> INFO 13:36:41,062 Writing Memtable(Super1)@15385755
>>>>>> INFO 13:36:42,062 Completed flushing
>>>>>> d:\cassandra\data\Keyspace1\Super1-711-Data.db
>>>>>> INFO 13:36:45,781 Super1 has reached its threshold; switching in a
>>>>>> fresh Memtable at
>>>>>> CommitLogContext(file='d:/cassandra/commitlog\CommitLog-1271849783703.log',
>>>>>> position=6065637)
>>>>>> INFO 13:36:45,781 Enqueuing flush of Memtable(Super1)@15578910
>>>>>> INFO 13:36:45,796 Writing Memtable(Super1)@15578910
>>>>>> INFO 13:36:46,109 Completed flushing
>>>>>> d:\cassandra\data\Keyspace1\Super1-712-Data.db
>>>>>> INFO 13:36:54,296 GC for ConcurrentMarkSweep: 7149 ms, 58337240
>>>>>> reclaimed leaving 922392600 used; max is 1174208512
>>>>>> INFO 13:36:54,593 Super1 has reached its threshold; switching in a
>>>>>> fresh Memtable at
>>>>>> CommitLogContext(file='d:/cassandra/commitlog\CommitLog-1271849783703.log',
>>>>>> position=6722241)
>>>>>> INFO 13:36:54,593 Enqueuing flush of Memtable(Super1)@24468872
>>>>>> INFO 13:36:54,593 Writing Memtable(Super1)@24468872
>>>>>> INFO 13:36:55,421 Completed flushing
>>>>>> d:\cassandra\data\Keyspace1\Super1-713-Data.dbjava.lang.OutOfMemoryError:
>>>>>> Java heap space
>>>>>> INFO 13:37:08,281 GC for ConcurrentMarkSweep: 5561 ms, 9432 reclaimed
>>>>>> leaving 971904520 used; max is 1174208512
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
>
Re: Cassandra tuning for running test on a desktop
Posted by Nicolas Labrot <ni...@gmail.com>.
I donnot have a website ;)
I'm testing the viability of Cassandra to store XML documents and make fast
search queries. 4000 XML files (80MB of XML) create with my datamodel (one
SC per XML node) 1000000 SC which make Cassandra go OOM with Xmx 1GB. On the
contrary an xml DB like eXist handles 4000 XML doc without any problem with
an acceptable amount of memories.
What I like with Cassandra is his simplicity and his scalability. eXist is
not able to scale with data, the only viable solution his marklogic which
cost an harm and a feet... :)
I will install linux and buy some memories to continue my test.
Could a Cassandra developper give me the technical reason of this OOM ?
On Wed, Apr 21, 2010 at 5:13 PM, Mark Greene <gr...@gmail.com> wrote:
> Maybe, maybe not. Presumably if you are running a RDMS with any reasonable
> amount of traffic now a days, it's sitting on a machine with 4-8G of memory
> at least.
>
>
> On Wed, Apr 21, 2010 at 10:48 AM, Nicolas Labrot <ni...@gmail.com>wrote:
>
>> Thanks Mark.
>>
>> Cassandra is maybe too much for my need ;)
>>
>>
>>
>> On Wed, Apr 21, 2010 at 4:45 PM, Mark Greene <gr...@gmail.com> wrote:
>>
>>> Hit send to early....
>>>
>>> That being said a lot of people running Cassandra in production are using
>>> 4-6GB max heaps on 8GB machines, don't know if that helps but hopefully
>>> gives you some perspective.
>>>
>>>
>>> On Wed, Apr 21, 2010 at 10:39 AM, Mark Greene <gr...@gmail.com>wrote:
>>>
>>>> RAM doesn't necessarily need to be proportional but I would say the
>>>> number of nodes does. You can't just throw a bazillion inserts at one node.
>>>> This is the main benefit of Cassandra is that if you start hitting your
>>>> capacity, you add more machines and distribute the keys across more
>>>> machines.
>>>>
>>>>
>>>> On Wed, Apr 21, 2010 at 9:07 AM, Nicolas Labrot <ni...@gmail.com>wrote:
>>>>
>>>>> So does it means the RAM needed is proportionnal with the data handled
>>>>> ?
>>>>>
>>>>> Or Cassandra need a minimum amount or RAM when dataset is big?
>>>>>
>>>>> I must confess this OOM behaviour is strange.
>>>>>
>>>>>
>>>>> On Wed, Apr 21, 2010 at 2:54 PM, Mark Jones <MJ...@imagehawk.com>wrote:
>>>>>
>>>>>> On my 4GB machine I’m giving it 3GB and having no trouble with 60+
>>>>>> million 500 byte columns
>>>>>>
>>>>>>
>>>>>>
>>>>>> *From:* Nicolas Labrot [mailto:nithril@gmail.com]
>>>>>> *Sent:* Wednesday, April 21, 2010 7:47 AM
>>>>>> *To:* user@cassandra.apache.org
>>>>>> *Subject:* Re: Cassandra tuning for running test on a desktop
>>>>>>
>>>>>>
>>>>>>
>>>>>> I have try 1400M, and Cassandra OOM too.
>>>>>>
>>>>>> Is there another solution ? My data isn't very big.
>>>>>>
>>>>>> It seems that is the merge of the db
>>>>>>
>>>>>> On Wed, Apr 21, 2010 at 2:14 PM, Mark Greene <gr...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>> Trying increasing Xmx. 1G is probably not enough for the amount of
>>>>>> inserts you are doing.
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Wed, Apr 21, 2010 at 8:10 AM, Nicolas Labrot <ni...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>> Hello,
>>>>>>
>>>>>> For my first message I will first thanks Cassandra contributors for
>>>>>> their great works.
>>>>>>
>>>>>> I have a parameter issue with Cassandra (I hope it's just a parameter
>>>>>> issue). I'm using Cassandra 6.0.1 with Hector client on my desktop. It's a
>>>>>> simple dual core with 4GB of RAM on WinXP. I have keep the default JVM
>>>>>> option inside cassandra.bat (Xmx1G)
>>>>>>
>>>>>> I'm trying to insert 3 millions of SC with 6 Columns each inside 1 CF
>>>>>> (named Super1). The insertion go to 1 millions of SC (without slowdown) and
>>>>>> Cassandra crash because of an OOM. (I store an average of 100 bytes per SC
>>>>>> with a max of 10kB).
>>>>>> I have aggressively decreased all the memories parameters without any
>>>>>> respect to the consistency (My config is here [1]), the cache is turn off
>>>>>> but Cassandra still go to OOM. I have joined the last line of the Cassandra
>>>>>> life [2].
>>>>>>
>>>>>> What can I do to fix my issue ? Is there another solution than
>>>>>> increasing the Xmx ?
>>>>>>
>>>>>> Thanks for your help,
>>>>>>
>>>>>> Nicolas
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> [1]
>>>>>> <Keyspaces>
>>>>>> <Keyspace Name="Keyspace1">
>>>>>> <ColumnFamily Name="Super1"
>>>>>> ColumnType="Super"
>>>>>> CompareWith="BytesType"
>>>>>> CompareSubcolumnsWith="BytesType" />
>>>>>>
>>>>>> <ReplicaPlacementStrategy>org.apache.cassandra.locator.RackUnawareStrategy</ReplicaPlacementStrategy>
>>>>>> <ReplicationFactor>1</ReplicationFactor>
>>>>>>
>>>>>> <EndPointSnitch>org.apache.cassandra.locator.EndPointSnitch</EndPointSnitch>
>>>>>> </Keyspace>
>>>>>> </Keyspaces>
>>>>>> <CommitLogRotationThresholdInMB>32</CommitLogRotationThresholdInMB>
>>>>>>
>>>>>> <DiskAccessMode>auto</DiskAccessMode>
>>>>>> <RowWarningThresholdInMB>64</RowWarningThresholdInMB>
>>>>>> <SlicedBufferSizeInKB>64</SlicedBufferSizeInKB>
>>>>>> <FlushDataBufferSizeInMB>16</FlushDataBufferSizeInMB>
>>>>>> <FlushIndexBufferSizeInMB>4</FlushIndexBufferSizeInMB>
>>>>>> <ColumnIndexSizeInKB>64</ColumnIndexSizeInKB>
>>>>>>
>>>>>> <MemtableThroughputInMB>16</MemtableThroughputInMB>
>>>>>> <BinaryMemtableThroughputInMB>32</BinaryMemtableThroughputInMB>
>>>>>> <MemtableOperationsInMillions>0.01</MemtableOperationsInMillions>
>>>>>> <MemtableObjectCountInMillions>0.01</MemtableObjectCountInMillions>
>>>>>> <MemtableFlushAfterMinutes>60</MemtableFlushAfterMinutes>
>>>>>> <ConcurrentReads>4</ConcurrentReads>
>>>>>> <ConcurrentWrites>8</ConcurrentWrites>
>>>>>> </Storage>
>>>>>>
>>>>>>
>>>>>> [2]
>>>>>> INFO 13:36:41,062 Super1 has reached its threshold; switching in a
>>>>>> fresh Memtable at
>>>>>> CommitLogContext(file='d:/cassandra/commitlog\CommitLog-1271849783703.log',
>>>>>> position=5417524)
>>>>>> INFO 13:36:41,062 Enqueuing flush of Memtable(Super1)@15385755
>>>>>> INFO 13:36:41,062 Writing Memtable(Super1)@15385755
>>>>>> INFO 13:36:42,062 Completed flushing
>>>>>> d:\cassandra\data\Keyspace1\Super1-711-Data.db
>>>>>> INFO 13:36:45,781 Super1 has reached its threshold; switching in a
>>>>>> fresh Memtable at
>>>>>> CommitLogContext(file='d:/cassandra/commitlog\CommitLog-1271849783703.log',
>>>>>> position=6065637)
>>>>>> INFO 13:36:45,781 Enqueuing flush of Memtable(Super1)@15578910
>>>>>> INFO 13:36:45,796 Writing Memtable(Super1)@15578910
>>>>>> INFO 13:36:46,109 Completed flushing
>>>>>> d:\cassandra\data\Keyspace1\Super1-712-Data.db
>>>>>> INFO 13:36:54,296 GC for ConcurrentMarkSweep: 7149 ms, 58337240
>>>>>> reclaimed leaving 922392600 used; max is 1174208512
>>>>>> INFO 13:36:54,593 Super1 has reached its threshold; switching in a
>>>>>> fresh Memtable at
>>>>>> CommitLogContext(file='d:/cassandra/commitlog\CommitLog-1271849783703.log',
>>>>>> position=6722241)
>>>>>> INFO 13:36:54,593 Enqueuing flush of Memtable(Super1)@24468872
>>>>>> INFO 13:36:54,593 Writing Memtable(Super1)@24468872
>>>>>> INFO 13:36:55,421 Completed flushing
>>>>>> d:\cassandra\data\Keyspace1\Super1-713-Data.dbjava.lang.OutOfMemoryError:
>>>>>> Java heap space
>>>>>> INFO 13:37:08,281 GC for ConcurrentMarkSweep: 5561 ms, 9432 reclaimed
>>>>>> leaving 971904520 used; max is 1174208512
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
>
Re: Cassandra tuning for running test on a desktop
Posted by Mark Greene <gr...@gmail.com>.
Maybe, maybe not. Presumably if you are running a RDMS with any reasonable
amount of traffic now a days, it's sitting on a machine with 4-8G of memory
at least.
On Wed, Apr 21, 2010 at 10:48 AM, Nicolas Labrot <ni...@gmail.com> wrote:
> Thanks Mark.
>
> Cassandra is maybe too much for my need ;)
>
>
>
> On Wed, Apr 21, 2010 at 4:45 PM, Mark Greene <gr...@gmail.com> wrote:
>
>> Hit send to early....
>>
>> That being said a lot of people running Cassandra in production are using
>> 4-6GB max heaps on 8GB machines, don't know if that helps but hopefully
>> gives you some perspective.
>>
>>
>> On Wed, Apr 21, 2010 at 10:39 AM, Mark Greene <gr...@gmail.com> wrote:
>>
>>> RAM doesn't necessarily need to be proportional but I would say the
>>> number of nodes does. You can't just throw a bazillion inserts at one node.
>>> This is the main benefit of Cassandra is that if you start hitting your
>>> capacity, you add more machines and distribute the keys across more
>>> machines.
>>>
>>>
>>> On Wed, Apr 21, 2010 at 9:07 AM, Nicolas Labrot <ni...@gmail.com>wrote:
>>>
>>>> So does it means the RAM needed is proportionnal with the data handled ?
>>>>
>>>> Or Cassandra need a minimum amount or RAM when dataset is big?
>>>>
>>>> I must confess this OOM behaviour is strange.
>>>>
>>>>
>>>> On Wed, Apr 21, 2010 at 2:54 PM, Mark Jones <MJ...@imagehawk.com>wrote:
>>>>
>>>>> On my 4GB machine I’m giving it 3GB and having no trouble with 60+
>>>>> million 500 byte columns
>>>>>
>>>>>
>>>>>
>>>>> *From:* Nicolas Labrot [mailto:nithril@gmail.com]
>>>>> *Sent:* Wednesday, April 21, 2010 7:47 AM
>>>>> *To:* user@cassandra.apache.org
>>>>> *Subject:* Re: Cassandra tuning for running test on a desktop
>>>>>
>>>>>
>>>>>
>>>>> I have try 1400M, and Cassandra OOM too.
>>>>>
>>>>> Is there another solution ? My data isn't very big.
>>>>>
>>>>> It seems that is the merge of the db
>>>>>
>>>>> On Wed, Apr 21, 2010 at 2:14 PM, Mark Greene <gr...@gmail.com>
>>>>> wrote:
>>>>>
>>>>> Trying increasing Xmx. 1G is probably not enough for the amount of
>>>>> inserts you are doing.
>>>>>
>>>>>
>>>>>
>>>>> On Wed, Apr 21, 2010 at 8:10 AM, Nicolas Labrot <ni...@gmail.com>
>>>>> wrote:
>>>>>
>>>>> Hello,
>>>>>
>>>>> For my first message I will first thanks Cassandra contributors for
>>>>> their great works.
>>>>>
>>>>> I have a parameter issue with Cassandra (I hope it's just a parameter
>>>>> issue). I'm using Cassandra 6.0.1 with Hector client on my desktop. It's a
>>>>> simple dual core with 4GB of RAM on WinXP. I have keep the default JVM
>>>>> option inside cassandra.bat (Xmx1G)
>>>>>
>>>>> I'm trying to insert 3 millions of SC with 6 Columns each inside 1 CF
>>>>> (named Super1). The insertion go to 1 millions of SC (without slowdown) and
>>>>> Cassandra crash because of an OOM. (I store an average of 100 bytes per SC
>>>>> with a max of 10kB).
>>>>> I have aggressively decreased all the memories parameters without any
>>>>> respect to the consistency (My config is here [1]), the cache is turn off
>>>>> but Cassandra still go to OOM. I have joined the last line of the Cassandra
>>>>> life [2].
>>>>>
>>>>> What can I do to fix my issue ? Is there another solution than
>>>>> increasing the Xmx ?
>>>>>
>>>>> Thanks for your help,
>>>>>
>>>>> Nicolas
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> [1]
>>>>> <Keyspaces>
>>>>> <Keyspace Name="Keyspace1">
>>>>> <ColumnFamily Name="Super1"
>>>>> ColumnType="Super"
>>>>> CompareWith="BytesType"
>>>>> CompareSubcolumnsWith="BytesType" />
>>>>>
>>>>> <ReplicaPlacementStrategy>org.apache.cassandra.locator.RackUnawareStrategy</ReplicaPlacementStrategy>
>>>>> <ReplicationFactor>1</ReplicationFactor>
>>>>>
>>>>> <EndPointSnitch>org.apache.cassandra.locator.EndPointSnitch</EndPointSnitch>
>>>>> </Keyspace>
>>>>> </Keyspaces>
>>>>> <CommitLogRotationThresholdInMB>32</CommitLogRotationThresholdInMB>
>>>>>
>>>>> <DiskAccessMode>auto</DiskAccessMode>
>>>>> <RowWarningThresholdInMB>64</RowWarningThresholdInMB>
>>>>> <SlicedBufferSizeInKB>64</SlicedBufferSizeInKB>
>>>>> <FlushDataBufferSizeInMB>16</FlushDataBufferSizeInMB>
>>>>> <FlushIndexBufferSizeInMB>4</FlushIndexBufferSizeInMB>
>>>>> <ColumnIndexSizeInKB>64</ColumnIndexSizeInKB>
>>>>>
>>>>> <MemtableThroughputInMB>16</MemtableThroughputInMB>
>>>>> <BinaryMemtableThroughputInMB>32</BinaryMemtableThroughputInMB>
>>>>> <MemtableOperationsInMillions>0.01</MemtableOperationsInMillions>
>>>>> <MemtableObjectCountInMillions>0.01</MemtableObjectCountInMillions>
>>>>> <MemtableFlushAfterMinutes>60</MemtableFlushAfterMinutes>
>>>>> <ConcurrentReads>4</ConcurrentReads>
>>>>> <ConcurrentWrites>8</ConcurrentWrites>
>>>>> </Storage>
>>>>>
>>>>>
>>>>> [2]
>>>>> INFO 13:36:41,062 Super1 has reached its threshold; switching in a
>>>>> fresh Memtable at
>>>>> CommitLogContext(file='d:/cassandra/commitlog\CommitLog-1271849783703.log',
>>>>> position=5417524)
>>>>> INFO 13:36:41,062 Enqueuing flush of Memtable(Super1)@15385755
>>>>> INFO 13:36:41,062 Writing Memtable(Super1)@15385755
>>>>> INFO 13:36:42,062 Completed flushing
>>>>> d:\cassandra\data\Keyspace1\Super1-711-Data.db
>>>>> INFO 13:36:45,781 Super1 has reached its threshold; switching in a
>>>>> fresh Memtable at
>>>>> CommitLogContext(file='d:/cassandra/commitlog\CommitLog-1271849783703.log',
>>>>> position=6065637)
>>>>> INFO 13:36:45,781 Enqueuing flush of Memtable(Super1)@15578910
>>>>> INFO 13:36:45,796 Writing Memtable(Super1)@15578910
>>>>> INFO 13:36:46,109 Completed flushing
>>>>> d:\cassandra\data\Keyspace1\Super1-712-Data.db
>>>>> INFO 13:36:54,296 GC for ConcurrentMarkSweep: 7149 ms, 58337240
>>>>> reclaimed leaving 922392600 used; max is 1174208512
>>>>> INFO 13:36:54,593 Super1 has reached its threshold; switching in a
>>>>> fresh Memtable at
>>>>> CommitLogContext(file='d:/cassandra/commitlog\CommitLog-1271849783703.log',
>>>>> position=6722241)
>>>>> INFO 13:36:54,593 Enqueuing flush of Memtable(Super1)@24468872
>>>>> INFO 13:36:54,593 Writing Memtable(Super1)@24468872
>>>>> INFO 13:36:55,421 Completed flushing
>>>>> d:\cassandra\data\Keyspace1\Super1-713-Data.dbjava.lang.OutOfMemoryError:
>>>>> Java heap space
>>>>> INFO 13:37:08,281 GC for ConcurrentMarkSweep: 5561 ms, 9432 reclaimed
>>>>> leaving 971904520 used; max is 1174208512
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>
>>
>
Re: Cassandra tuning for running test on a desktop
Posted by Nicolas Labrot <ni...@gmail.com>.
Thanks Mark.
Cassandra is maybe too much for my need ;)
On Wed, Apr 21, 2010 at 4:45 PM, Mark Greene <gr...@gmail.com> wrote:
> Hit send to early....
>
> That being said a lot of people running Cassandra in production are using
> 4-6GB max heaps on 8GB machines, don't know if that helps but hopefully
> gives you some perspective.
>
>
> On Wed, Apr 21, 2010 at 10:39 AM, Mark Greene <gr...@gmail.com> wrote:
>
>> RAM doesn't necessarily need to be proportional but I would say the number
>> of nodes does. You can't just throw a bazillion inserts at one node. This is
>> the main benefit of Cassandra is that if you start hitting your capacity,
>> you add more machines and distribute the keys across more machines.
>>
>>
>> On Wed, Apr 21, 2010 at 9:07 AM, Nicolas Labrot <ni...@gmail.com>wrote:
>>
>>> So does it means the RAM needed is proportionnal with the data handled ?
>>>
>>> Or Cassandra need a minimum amount or RAM when dataset is big?
>>>
>>> I must confess this OOM behaviour is strange.
>>>
>>>
>>> On Wed, Apr 21, 2010 at 2:54 PM, Mark Jones <MJ...@imagehawk.com>wrote:
>>>
>>>> On my 4GB machine I’m giving it 3GB and having no trouble with 60+
>>>> million 500 byte columns
>>>>
>>>>
>>>>
>>>> *From:* Nicolas Labrot [mailto:nithril@gmail.com]
>>>> *Sent:* Wednesday, April 21, 2010 7:47 AM
>>>> *To:* user@cassandra.apache.org
>>>> *Subject:* Re: Cassandra tuning for running test on a desktop
>>>>
>>>>
>>>>
>>>> I have try 1400M, and Cassandra OOM too.
>>>>
>>>> Is there another solution ? My data isn't very big.
>>>>
>>>> It seems that is the merge of the db
>>>>
>>>> On Wed, Apr 21, 2010 at 2:14 PM, Mark Greene <gr...@gmail.com>
>>>> wrote:
>>>>
>>>> Trying increasing Xmx. 1G is probably not enough for the amount of
>>>> inserts you are doing.
>>>>
>>>>
>>>>
>>>> On Wed, Apr 21, 2010 at 8:10 AM, Nicolas Labrot <ni...@gmail.com>
>>>> wrote:
>>>>
>>>> Hello,
>>>>
>>>> For my first message I will first thanks Cassandra contributors for
>>>> their great works.
>>>>
>>>> I have a parameter issue with Cassandra (I hope it's just a parameter
>>>> issue). I'm using Cassandra 6.0.1 with Hector client on my desktop. It's a
>>>> simple dual core with 4GB of RAM on WinXP. I have keep the default JVM
>>>> option inside cassandra.bat (Xmx1G)
>>>>
>>>> I'm trying to insert 3 millions of SC with 6 Columns each inside 1 CF
>>>> (named Super1). The insertion go to 1 millions of SC (without slowdown) and
>>>> Cassandra crash because of an OOM. (I store an average of 100 bytes per SC
>>>> with a max of 10kB).
>>>> I have aggressively decreased all the memories parameters without any
>>>> respect to the consistency (My config is here [1]), the cache is turn off
>>>> but Cassandra still go to OOM. I have joined the last line of the Cassandra
>>>> life [2].
>>>>
>>>> What can I do to fix my issue ? Is there another solution than
>>>> increasing the Xmx ?
>>>>
>>>> Thanks for your help,
>>>>
>>>> Nicolas
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> [1]
>>>> <Keyspaces>
>>>> <Keyspace Name="Keyspace1">
>>>> <ColumnFamily Name="Super1"
>>>> ColumnType="Super"
>>>> CompareWith="BytesType"
>>>> CompareSubcolumnsWith="BytesType" />
>>>>
>>>> <ReplicaPlacementStrategy>org.apache.cassandra.locator.RackUnawareStrategy</ReplicaPlacementStrategy>
>>>> <ReplicationFactor>1</ReplicationFactor>
>>>>
>>>> <EndPointSnitch>org.apache.cassandra.locator.EndPointSnitch</EndPointSnitch>
>>>> </Keyspace>
>>>> </Keyspaces>
>>>> <CommitLogRotationThresholdInMB>32</CommitLogRotationThresholdInMB>
>>>>
>>>> <DiskAccessMode>auto</DiskAccessMode>
>>>> <RowWarningThresholdInMB>64</RowWarningThresholdInMB>
>>>> <SlicedBufferSizeInKB>64</SlicedBufferSizeInKB>
>>>> <FlushDataBufferSizeInMB>16</FlushDataBufferSizeInMB>
>>>> <FlushIndexBufferSizeInMB>4</FlushIndexBufferSizeInMB>
>>>> <ColumnIndexSizeInKB>64</ColumnIndexSizeInKB>
>>>>
>>>> <MemtableThroughputInMB>16</MemtableThroughputInMB>
>>>> <BinaryMemtableThroughputInMB>32</BinaryMemtableThroughputInMB>
>>>> <MemtableOperationsInMillions>0.01</MemtableOperationsInMillions>
>>>> <MemtableObjectCountInMillions>0.01</MemtableObjectCountInMillions>
>>>> <MemtableFlushAfterMinutes>60</MemtableFlushAfterMinutes>
>>>> <ConcurrentReads>4</ConcurrentReads>
>>>> <ConcurrentWrites>8</ConcurrentWrites>
>>>> </Storage>
>>>>
>>>>
>>>> [2]
>>>> INFO 13:36:41,062 Super1 has reached its threshold; switching in a
>>>> fresh Memtable at
>>>> CommitLogContext(file='d:/cassandra/commitlog\CommitLog-1271849783703.log',
>>>> position=5417524)
>>>> INFO 13:36:41,062 Enqueuing flush of Memtable(Super1)@15385755
>>>> INFO 13:36:41,062 Writing Memtable(Super1)@15385755
>>>> INFO 13:36:42,062 Completed flushing
>>>> d:\cassandra\data\Keyspace1\Super1-711-Data.db
>>>> INFO 13:36:45,781 Super1 has reached its threshold; switching in a
>>>> fresh Memtable at
>>>> CommitLogContext(file='d:/cassandra/commitlog\CommitLog-1271849783703.log',
>>>> position=6065637)
>>>> INFO 13:36:45,781 Enqueuing flush of Memtable(Super1)@15578910
>>>> INFO 13:36:45,796 Writing Memtable(Super1)@15578910
>>>> INFO 13:36:46,109 Completed flushing
>>>> d:\cassandra\data\Keyspace1\Super1-712-Data.db
>>>> INFO 13:36:54,296 GC for ConcurrentMarkSweep: 7149 ms, 58337240
>>>> reclaimed leaving 922392600 used; max is 1174208512
>>>> INFO 13:36:54,593 Super1 has reached its threshold; switching in a
>>>> fresh Memtable at
>>>> CommitLogContext(file='d:/cassandra/commitlog\CommitLog-1271849783703.log',
>>>> position=6722241)
>>>> INFO 13:36:54,593 Enqueuing flush of Memtable(Super1)@24468872
>>>> INFO 13:36:54,593 Writing Memtable(Super1)@24468872
>>>> INFO 13:36:55,421 Completed flushing
>>>> d:\cassandra\data\Keyspace1\Super1-713-Data.dbjava.lang.OutOfMemoryError:
>>>> Java heap space
>>>> INFO 13:37:08,281 GC for ConcurrentMarkSweep: 5561 ms, 9432 reclaimed
>>>> leaving 971904520 used; max is 1174208512
>>>>
>>>>
>>>>
>>>>
>>>>
>>>
>>>
>>
>
Re: Cassandra tuning for running test on a desktop
Posted by Mark Greene <gr...@gmail.com>.
Hit send to early....
That being said a lot of people running Cassandra in production are using
4-6GB max heaps on 8GB machines, don't know if that helps but hopefully
gives you some perspective.
On Wed, Apr 21, 2010 at 10:39 AM, Mark Greene <gr...@gmail.com> wrote:
> RAM doesn't necessarily need to be proportional but I would say the number
> of nodes does. You can't just throw a bazillion inserts at one node. This is
> the main benefit of Cassandra is that if you start hitting your capacity,
> you add more machines and distribute the keys across more machines.
>
>
> On Wed, Apr 21, 2010 at 9:07 AM, Nicolas Labrot <ni...@gmail.com> wrote:
>
>> So does it means the RAM needed is proportionnal with the data handled ?
>>
>> Or Cassandra need a minimum amount or RAM when dataset is big?
>>
>> I must confess this OOM behaviour is strange.
>>
>>
>> On Wed, Apr 21, 2010 at 2:54 PM, Mark Jones <MJ...@imagehawk.com> wrote:
>>
>>> On my 4GB machine I’m giving it 3GB and having no trouble with 60+
>>> million 500 byte columns
>>>
>>>
>>>
>>> *From:* Nicolas Labrot [mailto:nithril@gmail.com]
>>> *Sent:* Wednesday, April 21, 2010 7:47 AM
>>> *To:* user@cassandra.apache.org
>>> *Subject:* Re: Cassandra tuning for running test on a desktop
>>>
>>>
>>>
>>> I have try 1400M, and Cassandra OOM too.
>>>
>>> Is there another solution ? My data isn't very big.
>>>
>>> It seems that is the merge of the db
>>>
>>> On Wed, Apr 21, 2010 at 2:14 PM, Mark Greene <gr...@gmail.com>
>>> wrote:
>>>
>>> Trying increasing Xmx. 1G is probably not enough for the amount of
>>> inserts you are doing.
>>>
>>>
>>>
>>> On Wed, Apr 21, 2010 at 8:10 AM, Nicolas Labrot <ni...@gmail.com>
>>> wrote:
>>>
>>> Hello,
>>>
>>> For my first message I will first thanks Cassandra contributors for their
>>> great works.
>>>
>>> I have a parameter issue with Cassandra (I hope it's just a parameter
>>> issue). I'm using Cassandra 6.0.1 with Hector client on my desktop. It's a
>>> simple dual core with 4GB of RAM on WinXP. I have keep the default JVM
>>> option inside cassandra.bat (Xmx1G)
>>>
>>> I'm trying to insert 3 millions of SC with 6 Columns each inside 1 CF
>>> (named Super1). The insertion go to 1 millions of SC (without slowdown) and
>>> Cassandra crash because of an OOM. (I store an average of 100 bytes per SC
>>> with a max of 10kB).
>>> I have aggressively decreased all the memories parameters without any
>>> respect to the consistency (My config is here [1]), the cache is turn off
>>> but Cassandra still go to OOM. I have joined the last line of the Cassandra
>>> life [2].
>>>
>>> What can I do to fix my issue ? Is there another solution than
>>> increasing the Xmx ?
>>>
>>> Thanks for your help,
>>>
>>> Nicolas
>>>
>>>
>>>
>>>
>>>
>>> [1]
>>> <Keyspaces>
>>> <Keyspace Name="Keyspace1">
>>> <ColumnFamily Name="Super1"
>>> ColumnType="Super"
>>> CompareWith="BytesType"
>>> CompareSubcolumnsWith="BytesType" />
>>>
>>> <ReplicaPlacementStrategy>org.apache.cassandra.locator.RackUnawareStrategy</ReplicaPlacementStrategy>
>>> <ReplicationFactor>1</ReplicationFactor>
>>>
>>> <EndPointSnitch>org.apache.cassandra.locator.EndPointSnitch</EndPointSnitch>
>>> </Keyspace>
>>> </Keyspaces>
>>> <CommitLogRotationThresholdInMB>32</CommitLogRotationThresholdInMB>
>>>
>>> <DiskAccessMode>auto</DiskAccessMode>
>>> <RowWarningThresholdInMB>64</RowWarningThresholdInMB>
>>> <SlicedBufferSizeInKB>64</SlicedBufferSizeInKB>
>>> <FlushDataBufferSizeInMB>16</FlushDataBufferSizeInMB>
>>> <FlushIndexBufferSizeInMB>4</FlushIndexBufferSizeInMB>
>>> <ColumnIndexSizeInKB>64</ColumnIndexSizeInKB>
>>>
>>> <MemtableThroughputInMB>16</MemtableThroughputInMB>
>>> <BinaryMemtableThroughputInMB>32</BinaryMemtableThroughputInMB>
>>> <MemtableOperationsInMillions>0.01</MemtableOperationsInMillions>
>>> <MemtableObjectCountInMillions>0.01</MemtableObjectCountInMillions>
>>> <MemtableFlushAfterMinutes>60</MemtableFlushAfterMinutes>
>>> <ConcurrentReads>4</ConcurrentReads>
>>> <ConcurrentWrites>8</ConcurrentWrites>
>>> </Storage>
>>>
>>>
>>> [2]
>>> INFO 13:36:41,062 Super1 has reached its threshold; switching in a fresh
>>> Memtable at
>>> CommitLogContext(file='d:/cassandra/commitlog\CommitLog-1271849783703.log',
>>> position=5417524)
>>> INFO 13:36:41,062 Enqueuing flush of Memtable(Super1)@15385755
>>> INFO 13:36:41,062 Writing Memtable(Super1)@15385755
>>> INFO 13:36:42,062 Completed flushing
>>> d:\cassandra\data\Keyspace1\Super1-711-Data.db
>>> INFO 13:36:45,781 Super1 has reached its threshold; switching in a fresh
>>> Memtable at
>>> CommitLogContext(file='d:/cassandra/commitlog\CommitLog-1271849783703.log',
>>> position=6065637)
>>> INFO 13:36:45,781 Enqueuing flush of Memtable(Super1)@15578910
>>> INFO 13:36:45,796 Writing Memtable(Super1)@15578910
>>> INFO 13:36:46,109 Completed flushing
>>> d:\cassandra\data\Keyspace1\Super1-712-Data.db
>>> INFO 13:36:54,296 GC for ConcurrentMarkSweep: 7149 ms, 58337240
>>> reclaimed leaving 922392600 used; max is 1174208512
>>> INFO 13:36:54,593 Super1 has reached its threshold; switching in a fresh
>>> Memtable at
>>> CommitLogContext(file='d:/cassandra/commitlog\CommitLog-1271849783703.log',
>>> position=6722241)
>>> INFO 13:36:54,593 Enqueuing flush of Memtable(Super1)@24468872
>>> INFO 13:36:54,593 Writing Memtable(Super1)@24468872
>>> INFO 13:36:55,421 Completed flushing
>>> d:\cassandra\data\Keyspace1\Super1-713-Data.dbjava.lang.OutOfMemoryError:
>>> Java heap space
>>> INFO 13:37:08,281 GC for ConcurrentMarkSweep: 5561 ms, 9432 reclaimed
>>> leaving 971904520 used; max is 1174208512
>>>
>>>
>>>
>>>
>>>
>>
>>
>
Re: Cassandra tuning for running test on a desktop
Posted by Mark Greene <gr...@gmail.com>.
RAM doesn't necessarily need to be proportional but I would say the number
of nodes does. You can't just throw a bazillion inserts at one node. This is
the main benefit of Cassandra is that if you start hitting your capacity,
you add more machines and distribute the keys across more machines.
On Wed, Apr 21, 2010 at 9:07 AM, Nicolas Labrot <ni...@gmail.com> wrote:
> So does it means the RAM needed is proportionnal with the data handled ?
>
> Or Cassandra need a minimum amount or RAM when dataset is big?
>
> I must confess this OOM behaviour is strange.
>
>
> On Wed, Apr 21, 2010 at 2:54 PM, Mark Jones <MJ...@imagehawk.com> wrote:
>
>> On my 4GB machine I’m giving it 3GB and having no trouble with 60+
>> million 500 byte columns
>>
>>
>>
>> *From:* Nicolas Labrot [mailto:nithril@gmail.com]
>> *Sent:* Wednesday, April 21, 2010 7:47 AM
>> *To:* user@cassandra.apache.org
>> *Subject:* Re: Cassandra tuning for running test on a desktop
>>
>>
>>
>> I have try 1400M, and Cassandra OOM too.
>>
>> Is there another solution ? My data isn't very big.
>>
>> It seems that is the merge of the db
>>
>> On Wed, Apr 21, 2010 at 2:14 PM, Mark Greene <gr...@gmail.com> wrote:
>>
>> Trying increasing Xmx. 1G is probably not enough for the amount of inserts
>> you are doing.
>>
>>
>>
>> On Wed, Apr 21, 2010 at 8:10 AM, Nicolas Labrot <ni...@gmail.com>
>> wrote:
>>
>> Hello,
>>
>> For my first message I will first thanks Cassandra contributors for their
>> great works.
>>
>> I have a parameter issue with Cassandra (I hope it's just a parameter
>> issue). I'm using Cassandra 6.0.1 with Hector client on my desktop. It's a
>> simple dual core with 4GB of RAM on WinXP. I have keep the default JVM
>> option inside cassandra.bat (Xmx1G)
>>
>> I'm trying to insert 3 millions of SC with 6 Columns each inside 1 CF
>> (named Super1). The insertion go to 1 millions of SC (without slowdown) and
>> Cassandra crash because of an OOM. (I store an average of 100 bytes per SC
>> with a max of 10kB).
>> I have aggressively decreased all the memories parameters without any
>> respect to the consistency (My config is here [1]), the cache is turn off
>> but Cassandra still go to OOM. I have joined the last line of the Cassandra
>> life [2].
>>
>> What can I do to fix my issue ? Is there another solution than increasing
>> the Xmx ?
>>
>> Thanks for your help,
>>
>> Nicolas
>>
>>
>>
>>
>>
>> [1]
>> <Keyspaces>
>> <Keyspace Name="Keyspace1">
>> <ColumnFamily Name="Super1"
>> ColumnType="Super"
>> CompareWith="BytesType"
>> CompareSubcolumnsWith="BytesType" />
>>
>> <ReplicaPlacementStrategy>org.apache.cassandra.locator.RackUnawareStrategy</ReplicaPlacementStrategy>
>> <ReplicationFactor>1</ReplicationFactor>
>>
>> <EndPointSnitch>org.apache.cassandra.locator.EndPointSnitch</EndPointSnitch>
>> </Keyspace>
>> </Keyspaces>
>> <CommitLogRotationThresholdInMB>32</CommitLogRotationThresholdInMB>
>>
>> <DiskAccessMode>auto</DiskAccessMode>
>> <RowWarningThresholdInMB>64</RowWarningThresholdInMB>
>> <SlicedBufferSizeInKB>64</SlicedBufferSizeInKB>
>> <FlushDataBufferSizeInMB>16</FlushDataBufferSizeInMB>
>> <FlushIndexBufferSizeInMB>4</FlushIndexBufferSizeInMB>
>> <ColumnIndexSizeInKB>64</ColumnIndexSizeInKB>
>>
>> <MemtableThroughputInMB>16</MemtableThroughputInMB>
>> <BinaryMemtableThroughputInMB>32</BinaryMemtableThroughputInMB>
>> <MemtableOperationsInMillions>0.01</MemtableOperationsInMillions>
>> <MemtableObjectCountInMillions>0.01</MemtableObjectCountInMillions>
>> <MemtableFlushAfterMinutes>60</MemtableFlushAfterMinutes>
>> <ConcurrentReads>4</ConcurrentReads>
>> <ConcurrentWrites>8</ConcurrentWrites>
>> </Storage>
>>
>>
>> [2]
>> INFO 13:36:41,062 Super1 has reached its threshold; switching in a fresh
>> Memtable at
>> CommitLogContext(file='d:/cassandra/commitlog\CommitLog-1271849783703.log',
>> position=5417524)
>> INFO 13:36:41,062 Enqueuing flush of Memtable(Super1)@15385755
>> INFO 13:36:41,062 Writing Memtable(Super1)@15385755
>> INFO 13:36:42,062 Completed flushing
>> d:\cassandra\data\Keyspace1\Super1-711-Data.db
>> INFO 13:36:45,781 Super1 has reached its threshold; switching in a fresh
>> Memtable at
>> CommitLogContext(file='d:/cassandra/commitlog\CommitLog-1271849783703.log',
>> position=6065637)
>> INFO 13:36:45,781 Enqueuing flush of Memtable(Super1)@15578910
>> INFO 13:36:45,796 Writing Memtable(Super1)@15578910
>> INFO 13:36:46,109 Completed flushing
>> d:\cassandra\data\Keyspace1\Super1-712-Data.db
>> INFO 13:36:54,296 GC for ConcurrentMarkSweep: 7149 ms, 58337240 reclaimed
>> leaving 922392600 used; max is 1174208512
>> INFO 13:36:54,593 Super1 has reached its threshold; switching in a fresh
>> Memtable at
>> CommitLogContext(file='d:/cassandra/commitlog\CommitLog-1271849783703.log',
>> position=6722241)
>> INFO 13:36:54,593 Enqueuing flush of Memtable(Super1)@24468872
>> INFO 13:36:54,593 Writing Memtable(Super1)@24468872
>> INFO 13:36:55,421 Completed flushing
>> d:\cassandra\data\Keyspace1\Super1-713-Data.dbjava.lang.OutOfMemoryError:
>> Java heap space
>> INFO 13:37:08,281 GC for ConcurrentMarkSweep: 5561 ms, 9432 reclaimed
>> leaving 971904520 used; max is 1174208512
>>
>>
>>
>>
>>
>
>
Re: Cassandra tuning for running test on a desktop
Posted by Nicolas Labrot <ni...@gmail.com>.
So does it means the RAM needed is proportionnal with the data handled ?
Or Cassandra need a minimum amount or RAM when dataset is big?
I must confess this OOM behaviour is strange.
On Wed, Apr 21, 2010 at 2:54 PM, Mark Jones <MJ...@imagehawk.com> wrote:
> On my 4GB machine I’m giving it 3GB and having no trouble with 60+
> million 500 byte columns
>
>
>
> *From:* Nicolas Labrot [mailto:nithril@gmail.com]
> *Sent:* Wednesday, April 21, 2010 7:47 AM
> *To:* user@cassandra.apache.org
> *Subject:* Re: Cassandra tuning for running test on a desktop
>
>
>
> I have try 1400M, and Cassandra OOM too.
>
> Is there another solution ? My data isn't very big.
>
> It seems that is the merge of the db
>
> On Wed, Apr 21, 2010 at 2:14 PM, Mark Greene <gr...@gmail.com> wrote:
>
> Trying increasing Xmx. 1G is probably not enough for the amount of inserts
> you are doing.
>
>
>
> On Wed, Apr 21, 2010 at 8:10 AM, Nicolas Labrot <ni...@gmail.com> wrote:
>
> Hello,
>
> For my first message I will first thanks Cassandra contributors for their
> great works.
>
> I have a parameter issue with Cassandra (I hope it's just a parameter
> issue). I'm using Cassandra 6.0.1 with Hector client on my desktop. It's a
> simple dual core with 4GB of RAM on WinXP. I have keep the default JVM
> option inside cassandra.bat (Xmx1G)
>
> I'm trying to insert 3 millions of SC with 6 Columns each inside 1 CF
> (named Super1). The insertion go to 1 millions of SC (without slowdown) and
> Cassandra crash because of an OOM. (I store an average of 100 bytes per SC
> with a max of 10kB).
> I have aggressively decreased all the memories parameters without any
> respect to the consistency (My config is here [1]), the cache is turn off
> but Cassandra still go to OOM. I have joined the last line of the Cassandra
> life [2].
>
> What can I do to fix my issue ? Is there another solution than increasing
> the Xmx ?
>
> Thanks for your help,
>
> Nicolas
>
>
>
>
>
> [1]
> <Keyspaces>
> <Keyspace Name="Keyspace1">
> <ColumnFamily Name="Super1"
> ColumnType="Super"
> CompareWith="BytesType"
> CompareSubcolumnsWith="BytesType" />
>
> <ReplicaPlacementStrategy>org.apache.cassandra.locator.RackUnawareStrategy</ReplicaPlacementStrategy>
> <ReplicationFactor>1</ReplicationFactor>
>
> <EndPointSnitch>org.apache.cassandra.locator.EndPointSnitch</EndPointSnitch>
> </Keyspace>
> </Keyspaces>
> <CommitLogRotationThresholdInMB>32</CommitLogRotationThresholdInMB>
>
> <DiskAccessMode>auto</DiskAccessMode>
> <RowWarningThresholdInMB>64</RowWarningThresholdInMB>
> <SlicedBufferSizeInKB>64</SlicedBufferSizeInKB>
> <FlushDataBufferSizeInMB>16</FlushDataBufferSizeInMB>
> <FlushIndexBufferSizeInMB>4</FlushIndexBufferSizeInMB>
> <ColumnIndexSizeInKB>64</ColumnIndexSizeInKB>
>
> <MemtableThroughputInMB>16</MemtableThroughputInMB>
> <BinaryMemtableThroughputInMB>32</BinaryMemtableThroughputInMB>
> <MemtableOperationsInMillions>0.01</MemtableOperationsInMillions>
> <MemtableObjectCountInMillions>0.01</MemtableObjectCountInMillions>
> <MemtableFlushAfterMinutes>60</MemtableFlushAfterMinutes>
> <ConcurrentReads>4</ConcurrentReads>
> <ConcurrentWrites>8</ConcurrentWrites>
> </Storage>
>
>
> [2]
> INFO 13:36:41,062 Super1 has reached its threshold; switching in a fresh
> Memtable at
> CommitLogContext(file='d:/cassandra/commitlog\CommitLog-1271849783703.log',
> position=5417524)
> INFO 13:36:41,062 Enqueuing flush of Memtable(Super1)@15385755
> INFO 13:36:41,062 Writing Memtable(Super1)@15385755
> INFO 13:36:42,062 Completed flushing
> d:\cassandra\data\Keyspace1\Super1-711-Data.db
> INFO 13:36:45,781 Super1 has reached its threshold; switching in a fresh
> Memtable at
> CommitLogContext(file='d:/cassandra/commitlog\CommitLog-1271849783703.log',
> position=6065637)
> INFO 13:36:45,781 Enqueuing flush of Memtable(Super1)@15578910
> INFO 13:36:45,796 Writing Memtable(Super1)@15578910
> INFO 13:36:46,109 Completed flushing
> d:\cassandra\data\Keyspace1\Super1-712-Data.db
> INFO 13:36:54,296 GC for ConcurrentMarkSweep: 7149 ms, 58337240 reclaimed
> leaving 922392600 used; max is 1174208512
> INFO 13:36:54,593 Super1 has reached its threshold; switching in a fresh
> Memtable at
> CommitLogContext(file='d:/cassandra/commitlog\CommitLog-1271849783703.log',
> position=6722241)
> INFO 13:36:54,593 Enqueuing flush of Memtable(Super1)@24468872
> INFO 13:36:54,593 Writing Memtable(Super1)@24468872
> INFO 13:36:55,421 Completed flushing
> d:\cassandra\data\Keyspace1\Super1-713-Data.dbjava.lang.OutOfMemoryError:
> Java heap space
> INFO 13:37:08,281 GC for ConcurrentMarkSweep: 5561 ms, 9432 reclaimed
> leaving 971904520 used; max is 1174208512
>
>
>
>
>
RE: Cassandra tuning for running test on a desktop
Posted by Mark Jones <MJ...@imagehawk.com>.
On my 4GB machine I'm giving it 3GB and having no trouble with 60+ million 500 byte columns
From: Nicolas Labrot [mailto:nithril@gmail.com]
Sent: Wednesday, April 21, 2010 7:47 AM
To: user@cassandra.apache.org
Subject: Re: Cassandra tuning for running test on a desktop
I have try 1400M, and Cassandra OOM too.
Is there another solution ? My data isn't very big.
It seems that is the merge of the db
On Wed, Apr 21, 2010 at 2:14 PM, Mark Greene <gr...@gmail.com>> wrote:
Trying increasing Xmx. 1G is probably not enough for the amount of inserts you are doing.
On Wed, Apr 21, 2010 at 8:10 AM, Nicolas Labrot <ni...@gmail.com>> wrote:
Hello,
For my first message I will first thanks Cassandra contributors for their great works.
I have a parameter issue with Cassandra (I hope it's just a parameter issue). I'm using Cassandra 6.0.1 with Hector client on my desktop. It's a simple dual core with 4GB of RAM on WinXP. I have keep the default JVM option inside cassandra.bat (Xmx1G)
I'm trying to insert 3 millions of SC with 6 Columns each inside 1 CF (named Super1). The insertion go to 1 millions of SC (without slowdown) and Cassandra crash because of an OOM. (I store an average of 100 bytes per SC with a max of 10kB).
I have aggressively decreased all the memories parameters without any respect to the consistency (My config is here [1]), the cache is turn off but Cassandra still go to OOM. I have joined the last line of the Cassandra life [2].
What can I do to fix my issue ? Is there another solution than increasing the Xmx ?
Thanks for your help,
Nicolas
[1]
<Keyspaces>
<Keyspace Name="Keyspace1">
<ColumnFamily Name="Super1"
ColumnType="Super"
CompareWith="BytesType"
CompareSubcolumnsWith="BytesType" />
<ReplicaPlacementStrategy>org.apache.cassandra.locator.RackUnawareStrategy</ReplicaPlacementStrategy>
<ReplicationFactor>1</ReplicationFactor>
<EndPointSnitch>org.apache.cassandra.locator.EndPointSnitch</EndPointSnitch>
</Keyspace>
</Keyspaces>
<CommitLogRotationThresholdInMB>32</CommitLogRotationThresholdInMB>
<DiskAccessMode>auto</DiskAccessMode>
<RowWarningThresholdInMB>64</RowWarningThresholdInMB>
<SlicedBufferSizeInKB>64</SlicedBufferSizeInKB>
<FlushDataBufferSizeInMB>16</FlushDataBufferSizeInMB>
<FlushIndexBufferSizeInMB>4</FlushIndexBufferSizeInMB>
<ColumnIndexSizeInKB>64</ColumnIndexSizeInKB>
<MemtableThroughputInMB>16</MemtableThroughputInMB>
<BinaryMemtableThroughputInMB>32</BinaryMemtableThroughputInMB>
<MemtableOperationsInMillions>0.01</MemtableOperationsInMillions>
<MemtableObjectCountInMillions>0.01</MemtableObjectCountInMillions>
<MemtableFlushAfterMinutes>60</MemtableFlushAfterMinutes>
<ConcurrentReads>4</ConcurrentReads>
<ConcurrentWrites>8</ConcurrentWrites>
</Storage>
[2]
INFO 13:36:41,062 Super1 has reached its threshold; switching in a fresh Memtable at CommitLogContext(file='d:/cassandra/commitlog\CommitLog-1271849783703.log', position=5417524)
INFO 13:36:41,062 Enqueuing flush of Memtable(Super1)@15385755
INFO 13:36:41,062 Writing Memtable(Super1)@15385755
INFO 13:36:42,062 Completed flushing d:\cassandra\data\Keyspace1\Super1-711-Data.db
INFO 13:36:45,781 Super1 has reached its threshold; switching in a fresh Memtable at CommitLogContext(file='d:/cassandra/commitlog\CommitLog-1271849783703.log', position=6065637)
INFO 13:36:45,781 Enqueuing flush of Memtable(Super1)@15578910
INFO 13:36:45,796 Writing Memtable(Super1)@15578910
INFO 13:36:46,109 Completed flushing d:\cassandra\data\Keyspace1\Super1-712-Data.db
INFO 13:36:54,296 GC for ConcurrentMarkSweep: 7149 ms, 58337240 reclaimed leaving 922392600 used; max is 1174208512
INFO 13:36:54,593 Super1 has reached its threshold; switching in a fresh Memtable at CommitLogContext(file='d:/cassandra/commitlog\CommitLog-1271849783703.log', position=6722241)
INFO 13:36:54,593 Enqueuing flush of Memtable(Super1)@24468872
INFO 13:36:54,593 Writing Memtable(Super1)@24468872
INFO 13:36:55,421 Completed flushing d:\cassandra\data\Keyspace1\Super1-713-Data.dbjava.lang.OutOfMemoryError: Java heap space
INFO 13:37:08,281 GC for ConcurrentMarkSweep: 5561 ms, 9432 reclaimed leaving 971904520 used; max is 1174208512
Re: Cassandra tuning for running test on a desktop
Posted by Nicolas Labrot <ni...@gmail.com>.
I have try 1400M, and Cassandra OOM too.
Is there another solution ? My data isn't very big.
It seems that is the merge of the db
On Wed, Apr 21, 2010 at 2:14 PM, Mark Greene <gr...@gmail.com> wrote:
> Trying increasing Xmx. 1G is probably not enough for the amount of inserts
> you are doing.
>
>
> On Wed, Apr 21, 2010 at 8:10 AM, Nicolas Labrot <ni...@gmail.com> wrote:
>
>> Hello,
>>
>> For my first message I will first thanks Cassandra contributors for their
>> great works.
>>
>> I have a parameter issue with Cassandra (I hope it's just a parameter
>> issue). I'm using Cassandra 6.0.1 with Hector client on my desktop. It's a
>> simple dual core with 4GB of RAM on WinXP. I have keep the default JVM
>> option inside cassandra.bat (Xmx1G)
>>
>> I'm trying to insert 3 millions of SC with 6 Columns each inside 1 CF
>> (named Super1). The insertion go to 1 millions of SC (without slowdown) and
>> Cassandra crash because of an OOM. (I store an average of 100 bytes per SC
>> with a max of 10kB).
>> I have aggressively decreased all the memories parameters without any
>> respect to the consistency (My config is here [1]), the cache is turn off
>> but Cassandra still go to OOM. I have joined the last line of the Cassandra
>> life [2].
>>
>> What can I do to fix my issue ? Is there another solution than increasing
>> the Xmx ?
>>
>> Thanks for your help,
>>
>> Nicolas
>>
>>
>>
>>
>>
>> [1]
>> <Keyspaces>
>> <Keyspace Name="Keyspace1">
>> <ColumnFamily Name="Super1"
>> ColumnType="Super"
>> CompareWith="BytesType"
>> CompareSubcolumnsWith="BytesType" />
>>
>> <ReplicaPlacementStrategy>org.apache.cassandra.locator.RackUnawareStrategy</ReplicaPlacementStrategy>
>> <ReplicationFactor>1</ReplicationFactor>
>>
>> <EndPointSnitch>org.apache.cassandra.locator.EndPointSnitch</EndPointSnitch>
>> </Keyspace>
>> </Keyspaces>
>> <CommitLogRotationThresholdInMB>32</CommitLogRotationThresholdInMB>
>>
>> <DiskAccessMode>auto</DiskAccessMode>
>> <RowWarningThresholdInMB>64</RowWarningThresholdInMB>
>> <SlicedBufferSizeInKB>64</SlicedBufferSizeInKB>
>> <FlushDataBufferSizeInMB>16</FlushDataBufferSizeInMB>
>> <FlushIndexBufferSizeInMB>4</FlushIndexBufferSizeInMB>
>> <ColumnIndexSizeInKB>64</ColumnIndexSizeInKB>
>>
>> <MemtableThroughputInMB>16</MemtableThroughputInMB>
>> <BinaryMemtableThroughputInMB>32</BinaryMemtableThroughputInMB>
>> <MemtableOperationsInMillions>0.01</MemtableOperationsInMillions>
>> <MemtableObjectCountInMillions>0.01</MemtableObjectCountInMillions>
>> <MemtableFlushAfterMinutes>60</MemtableFlushAfterMinutes>
>> <ConcurrentReads>4</ConcurrentReads>
>> <ConcurrentWrites>8</ConcurrentWrites>
>> </Storage>
>>
>>
>> [2]
>> INFO 13:36:41,062 Super1 has reached its threshold; switching in a fresh
>> Memtable at
>> CommitLogContext(file='d:/cassandra/commitlog\CommitLog-1271849783703.log',
>> position=5417524)
>> INFO 13:36:41,062 Enqueuing flush of Memtable(Super1)@15385755
>> INFO 13:36:41,062 Writing Memtable(Super1)@15385755
>> INFO 13:36:42,062 Completed flushing
>> d:\cassandra\data\Keyspace1\Super1-711-Data.db
>> INFO 13:36:45,781 Super1 has reached its threshold; switching in a fresh
>> Memtable at
>> CommitLogContext(file='d:/cassandra/commitlog\CommitLog-1271849783703.log',
>> position=6065637)
>> INFO 13:36:45,781 Enqueuing flush of Memtable(Super1)@15578910
>> INFO 13:36:45,796 Writing Memtable(Super1)@15578910
>> INFO 13:36:46,109 Completed flushing
>> d:\cassandra\data\Keyspace1\Super1-712-Data.db
>> INFO 13:36:54,296 GC for ConcurrentMarkSweep: 7149 ms, 58337240 reclaimed
>> leaving 922392600 used; max is 1174208512
>> INFO 13:36:54,593 Super1 has reached its threshold; switching in a fresh
>> Memtable at
>> CommitLogContext(file='d:/cassandra/commitlog\CommitLog-1271849783703.log',
>> position=6722241)
>> INFO 13:36:54,593 Enqueuing flush of Memtable(Super1)@24468872
>> INFO 13:36:54,593 Writing Memtable(Super1)@24468872
>> INFO 13:36:55,421 Completed flushing
>> d:\cassandra\data\Keyspace1\Super1-713-Data.dbjava.lang.OutOfMemoryError:
>> Java heap space
>> INFO 13:37:08,281 GC for ConcurrentMarkSweep: 5561 ms, 9432 reclaimed
>> leaving 971904520 used; max is 1174208512
>>
>>
>
Re: Cassandra tuning for running test on a desktop
Posted by Mark Greene <gr...@gmail.com>.
Trying increasing Xmx. 1G is probably not enough for the amount of inserts
you are doing.
On Wed, Apr 21, 2010 at 8:10 AM, Nicolas Labrot <ni...@gmail.com> wrote:
> Hello,
>
> For my first message I will first thanks Cassandra contributors for their
> great works.
>
> I have a parameter issue with Cassandra (I hope it's just a parameter
> issue). I'm using Cassandra 6.0.1 with Hector client on my desktop. It's a
> simple dual core with 4GB of RAM on WinXP. I have keep the default JVM
> option inside cassandra.bat (Xmx1G)
>
> I'm trying to insert 3 millions of SC with 6 Columns each inside 1 CF
> (named Super1). The insertion go to 1 millions of SC (without slowdown) and
> Cassandra crash because of an OOM. (I store an average of 100 bytes per SC
> with a max of 10kB).
> I have aggressively decreased all the memories parameters without any
> respect to the consistency (My config is here [1]), the cache is turn off
> but Cassandra still go to OOM. I have joined the last line of the Cassandra
> life [2].
>
> What can I do to fix my issue ? Is there another solution than increasing
> the Xmx ?
>
> Thanks for your help,
>
> Nicolas
>
>
>
>
>
> [1]
> <Keyspaces>
> <Keyspace Name="Keyspace1">
> <ColumnFamily Name="Super1"
> ColumnType="Super"
> CompareWith="BytesType"
> CompareSubcolumnsWith="BytesType" />
>
> <ReplicaPlacementStrategy>org.apache.cassandra.locator.RackUnawareStrategy</ReplicaPlacementStrategy>
> <ReplicationFactor>1</ReplicationFactor>
>
> <EndPointSnitch>org.apache.cassandra.locator.EndPointSnitch</EndPointSnitch>
> </Keyspace>
> </Keyspaces>
> <CommitLogRotationThresholdInMB>32</CommitLogRotationThresholdInMB>
>
> <DiskAccessMode>auto</DiskAccessMode>
> <RowWarningThresholdInMB>64</RowWarningThresholdInMB>
> <SlicedBufferSizeInKB>64</SlicedBufferSizeInKB>
> <FlushDataBufferSizeInMB>16</FlushDataBufferSizeInMB>
> <FlushIndexBufferSizeInMB>4</FlushIndexBufferSizeInMB>
> <ColumnIndexSizeInKB>64</ColumnIndexSizeInKB>
>
> <MemtableThroughputInMB>16</MemtableThroughputInMB>
> <BinaryMemtableThroughputInMB>32</BinaryMemtableThroughputInMB>
> <MemtableOperationsInMillions>0.01</MemtableOperationsInMillions>
> <MemtableObjectCountInMillions>0.01</MemtableObjectCountInMillions>
> <MemtableFlushAfterMinutes>60</MemtableFlushAfterMinutes>
> <ConcurrentReads>4</ConcurrentReads>
> <ConcurrentWrites>8</ConcurrentWrites>
> </Storage>
>
>
> [2]
> INFO 13:36:41,062 Super1 has reached its threshold; switching in a fresh
> Memtable at
> CommitLogContext(file='d:/cassandra/commitlog\CommitLog-1271849783703.log',
> position=5417524)
> INFO 13:36:41,062 Enqueuing flush of Memtable(Super1)@15385755
> INFO 13:36:41,062 Writing Memtable(Super1)@15385755
> INFO 13:36:42,062 Completed flushing
> d:\cassandra\data\Keyspace1\Super1-711-Data.db
> INFO 13:36:45,781 Super1 has reached its threshold; switching in a fresh
> Memtable at
> CommitLogContext(file='d:/cassandra/commitlog\CommitLog-1271849783703.log',
> position=6065637)
> INFO 13:36:45,781 Enqueuing flush of Memtable(Super1)@15578910
> INFO 13:36:45,796 Writing Memtable(Super1)@15578910
> INFO 13:36:46,109 Completed flushing
> d:\cassandra\data\Keyspace1\Super1-712-Data.db
> INFO 13:36:54,296 GC for ConcurrentMarkSweep: 7149 ms, 58337240 reclaimed
> leaving 922392600 used; max is 1174208512
> INFO 13:36:54,593 Super1 has reached its threshold; switching in a fresh
> Memtable at
> CommitLogContext(file='d:/cassandra/commitlog\CommitLog-1271849783703.log',
> position=6722241)
> INFO 13:36:54,593 Enqueuing flush of Memtable(Super1)@24468872
> INFO 13:36:54,593 Writing Memtable(Super1)@24468872
> INFO 13:36:55,421 Completed flushing
> d:\cassandra\data\Keyspace1\Super1-713-Data.dbjava.lang.OutOfMemoryError:
> Java heap space
> INFO 13:37:08,281 GC for ConcurrentMarkSweep: 5561 ms, 9432 reclaimed
> leaving 971904520 used; max is 1174208512
>
>