Load Vs Disk Space Usage

[Brayton Thompson <th...@grnoc.iu.edu>]

We're playing around with Cassandra trying to get a feel for it. Can someone please explain the difference between load (from nodetool) and whats actually stored on disk? Sometimes these number mirror each other and sometimes the disk usage is up to 2x the load reported. as you can see below...

Cassandra 1	Keyspace Data Dir. Size (MB)	System Data Dir Size (MB)	Commit Log Data Dir Size (MB)	Load (reported by ring) (MB)
Initial	0	0.11	0.03	0.04
Run 1	254	0.11	344	253.23
Run 2	634	0.11	302	632.84
Run 3	1500	0.11	260	1012.46
Run 4	1400	0.11	219	1360
Run 5	2200	0.11	305	1610
Run 6	4000	0.11	264	1980
Run 7	2400	0.11	350	2350
Run 8	2800	0.11	309	2720
Run 9	3600	0.11	267	3090
Run 10	3400	0.11	354	3340
Run 11	3800	0.11	312	3710
Run 12	6100	0.11	271	4080
Run 3,5,6,9, and 12 don't seem to match up well. Can someone explain this please?

Thank you.

Re: Load Vs Disk Space Usage

[Peter Schuller <pe...@infidyne.com>]

> We're playing around with Cassandra trying to get a feel for it. Can someone please explain the difference between load (from nodetool) and whats actually stored on disk? Sometimes these number mirror each other and sometimes the disk usage is up to 2x the load reported. as you can see below...
[snip]
> Run 3,5,6,9, and 12 don't seem to match up well. Can someone explain this please?

Probably there are obsolete sstables that have not yet been removed.
Removal of sstables is somewhat delayed because it relies on GC to
avoid synchronization complexities in the implementation. See:

   http://wiki.apache.org/cassandra/MemtableSSTable

I believe sstables that are obsolete will not count towards load.

You can either trigger the GC, restart the cassandra nodes, or just
wait until they disappear (generating enough activity to trigger a CMS
sweep of the heap should be enough, assuming you use CMS).

--
/ Peter Schuller