You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by Sorin Julean <so...@gmail.com> on 2011/09/12 13:55:55 UTC
memtable flush thresholds
Hi,
I've checked the memtable flush (cassandra 0.8.4) and it seams to me it
hapens sooner then the threshold is reached.
Here's the threshould's (the default ones calculated for a heap size of
-Xmx1980M):
ColumnFamily: idx_graphable (Super)
Key Validation Class: org.apache.cassandra.db.marshal.BytesType
Default column value validator:
org.apache.cassandra.db.marshal.BytesType
Columns sorted by:
org.apache.cassandra.db.marshal.UTF8Type/org.apache.cassandra.db.marshal.UTF8Type
Row cache size / save period in seconds: 0.0/0
Key cache size / save period in seconds: 200000.0/14400
*Memtable thresholds: 0.5671875/1440/121 (millions of ops/minutes/MB)*
GC grace seconds: 864000
Compaction min/max thresholds: 4/32
Read repair chance: 1.0
Replicate on write: true
In the logs it seams to me none oh the thresold is reached ( definitively
minutes threshold is not reached ).
9-08 20:12:30,136 MeteredFlusher.java (line 62) flushing high-traffic column
family ColumnFamilyStore(table='graph', columnFamily='idx_graphable')
INFO [NonPeriodicTasks:1] 2011-09-08 20:12:30,144 ColumnFamilyStore.java
(line 1036) Enqueuing flush of
Memtable-idx_graphable@915643571*(4671498/96780112
serialized/live bytes, 59891 ops)*
INFO [FlushWriter:111] 2011-09-08 20:12:30,145 Memtable.java (line 237)
Writing Memtable-idx_graphable@915643571(4671498/96780112 serialized/live
bytes, 59891 ops)
INFO [FlushWriter:111] 2011-09-08 20:12:30,348 Memtable.java (line 254)
Completed flushing [...]/cassandra/data/graph/idx_graphable-g-23-Data.db
(4673905 bytes)
Could someone clarify it for me ?
"high-traffic column family" has a special meaning ?
Many thanks,
Sorin
Re: memtable flush thresholds
Posted by Sorin Julean <so...@gmail.com>.
Thanks Jonathan !
memtable_total_space_in_mb is the threshold that is reached.
Kind regards,
Sorin
On Mon, Sep 12, 2011 at 3:16 PM, Jonathan Ellis <jb...@gmail.com> wrote:
> see memtable_total_space_in_mb at
> http://thelastpickle.com/2011/05/04/How-are-Memtables-measured/
>
> On Mon, Sep 12, 2011 at 6:55 AM, Sorin Julean <so...@gmail.com>
> wrote:
> > Hi,
> >
> > I've checked the memtable flush (cassandra 0.8.4) and it seams to me it
> > hapens sooner then the threshold is reached.
> >
> > Here's the threshould's (the default ones calculated for a heap size of
> > -Xmx1980M):
> > ColumnFamily: idx_graphable (Super)
> > Key Validation Class: org.apache.cassandra.db.marshal.BytesType
> > Default column value validator:
> > org.apache.cassandra.db.marshal.BytesType
> > Columns sorted by:
> >
> org.apache.cassandra.db.marshal.UTF8Type/org.apache.cassandra.db.marshal.UTF8Type
> > Row cache size / save period in seconds: 0.0/0
> > Key cache size / save period in seconds: 200000.0/14400
> > Memtable thresholds: 0.5671875/1440/121 (millions of
> ops/minutes/MB)
> > GC grace seconds: 864000
> > Compaction min/max thresholds: 4/32
> > Read repair chance: 1.0
> > Replicate on write: true
> >
> > In the logs it seams to me none oh the thresold is reached ( definitively
> > minutes threshold is not reached ).
> >
> > 9-08 20:12:30,136 MeteredFlusher.java (line 62) flushing high-traffic
> column
> > family ColumnFamilyStore(table='graph', columnFamily='idx_graphable')
> > INFO [NonPeriodicTasks:1] 2011-09-08 20:12:30,144 ColumnFamilyStore.java
> > (line 1036) Enqueuing flush of
> > Memtable-idx_graphable@915643571(4671498/96780112 serialized/live bytes,
> > 59891 ops)
> > INFO [FlushWriter:111] 2011-09-08 20:12:30,145 Memtable.java (line 237)
> > Writing Memtable-idx_graphable@915643571(4671498/96780112
> serialized/live
> > bytes, 59891 ops)
> > INFO [FlushWriter:111] 2011-09-08 20:12:30,348 Memtable.java (line 254)
> > Completed flushing [...]/cassandra/data/graph/idx_graphable-g-23-Data.db
> > (4673905 bytes)
> >
> >
> > Could someone clarify it for me ?
> > "high-traffic column family" has a special meaning ?
> >
> > Many thanks,
> > Sorin
> >
>
>
>
> --
> Jonathan Ellis
> Project Chair, Apache Cassandra
> co-founder of DataStax, the source for professional Cassandra support
> http://www.datastax.com
>
Re: memtable flush thresholds
Posted by Jonathan Ellis <jb...@gmail.com>.
see memtable_total_space_in_mb at
http://thelastpickle.com/2011/05/04/How-are-Memtables-measured/
On Mon, Sep 12, 2011 at 6:55 AM, Sorin Julean <so...@gmail.com> wrote:
> Hi,
>
> I've checked the memtable flush (cassandra 0.8.4) and it seams to me it
> hapens sooner then the threshold is reached.
>
> Here's the threshould's (the default ones calculated for a heap size of
> -Xmx1980M):
> ColumnFamily: idx_graphable (Super)
> Key Validation Class: org.apache.cassandra.db.marshal.BytesType
> Default column value validator:
> org.apache.cassandra.db.marshal.BytesType
> Columns sorted by:
> org.apache.cassandra.db.marshal.UTF8Type/org.apache.cassandra.db.marshal.UTF8Type
> Row cache size / save period in seconds: 0.0/0
> Key cache size / save period in seconds: 200000.0/14400
> Memtable thresholds: 0.5671875/1440/121 (millions of ops/minutes/MB)
> GC grace seconds: 864000
> Compaction min/max thresholds: 4/32
> Read repair chance: 1.0
> Replicate on write: true
>
> In the logs it seams to me none oh the thresold is reached ( definitively
> minutes threshold is not reached ).
>
> 9-08 20:12:30,136 MeteredFlusher.java (line 62) flushing high-traffic column
> family ColumnFamilyStore(table='graph', columnFamily='idx_graphable')
> INFO [NonPeriodicTasks:1] 2011-09-08 20:12:30,144 ColumnFamilyStore.java
> (line 1036) Enqueuing flush of
> Memtable-idx_graphable@915643571(4671498/96780112 serialized/live bytes,
> 59891 ops)
> INFO [FlushWriter:111] 2011-09-08 20:12:30,145 Memtable.java (line 237)
> Writing Memtable-idx_graphable@915643571(4671498/96780112 serialized/live
> bytes, 59891 ops)
> INFO [FlushWriter:111] 2011-09-08 20:12:30,348 Memtable.java (line 254)
> Completed flushing [...]/cassandra/data/graph/idx_graphable-g-23-Data.db
> (4673905 bytes)
>
>
> Could someone clarify it for me ?
> "high-traffic column family" has a special meaning ?
>
> Many thanks,
> Sorin
>
--
Jonathan Ellis
Project Chair, Apache Cassandra
co-founder of DataStax, the source for professional Cassandra support
http://www.datastax.com
Re: memtable flush thresholds
Posted by Jonathan Ellis <jb...@gmail.com>.
Because for small columns the Java "container" objects are larger than
the actual column data.
On Tue, Sep 13, 2011 at 9:38 AM, Radim Kolar <hs...@sendmail.cz> wrote:
>> flush of Memtable-idx_graphable@915643571
> (4671498/96780112 serialized/live bytes, 59891 ops)
> can somebody explain why difference between live and serialized is so big?
> Serialized bytes are actual bytes written to sstable?
>
>
--
Jonathan Ellis
Project Chair, Apache Cassandra
co-founder of DataStax, the source for professional Cassandra support
http://www.datastax.com
Re: memtable flush thresholds
Posted by Radim Kolar <hs...@sendmail.cz>.
> flush of Memtable-idx_graphable@915643571*(4671498/96780112
serialized/live bytes, 59891 ops)
can somebody explain why difference between live and serialized is so
big? Serialized bytes are actual bytes written to sstable?
*