You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by Jenny <aw...@163.com> on 2011/09/04 16:05:42 UTC
what did actually happen?
Hi:
I created a new super column family and insert 1000 super columns into it in a short time. Cassandra 0.8 was running on my laptop. When inserting, the system was continuing flushing to files and compaction. I use the default configuration. I just can't figure out why the size of data file is like that.
public Integer binary_memtable_throughput_in_mb = 256;
/* if the size of columns or super-columns are more than this, indexing will kick in */
public Integer column_index_size_in_kb = 64;
public Integer in_memory_compaction_limit_in_mb = 256;
public Integer concurrent_compactors = Runtime.getRuntime().availableProcessors();
public Integer compaction_throughput_mb_per_sec = 16;
///////The compacted file become bigger and bigger, is there any limit? 256 mb? Why there are three files, not one file? When to stop compaction?
-rw-r--r-- 1 wy staff 0 9 4 21:07 0-g-29-Compacted
-rw-r--r-- 2 wy staff 26712582 9 4 21:07 0-g-29-Data.db
-rw-r--r-- 2 wy staff 1936 9 4 21:07 0-g-29-Filter.db
-rw-r--r-- 2 wy staff 14 9 4 21:07 0-g-29-Index.db
-rw-r--r-- 2 wy staff 4276 9 4 21:07 0-g-29-Statistics.db
-rw-r--r-- 1 wy staff 0 9 4 21:07 0-g-30-Compacted
-rw-r--r-- 3 wy staff 1206800 9 4 21:07 0-g-30-Data.db
-rw-r--r-- 3 wy staff 16 9 4 21:07 0-g-30-Filter.db
-rw-r--r-- 3 wy staff 14 9 4 21:07 0-g-30-Index.db
-rw-r--r-- 3 wy staff 4276 9 4 21:07 0-g-30-Statistics.db
-rw-r--r-- 1 wy staff 0 9 4 21:07 0-g-31-Compacted
-rw-r--r-- 3 wy staff 1238050 9 4 21:07 0-g-31-Data.db
-rw-r--r-- 3 wy staff 16 9 4 21:07 0-g-31-Filter.db
-rw-r--r-- 3 wy staff 14 9 4 21:07 0-g-31-Index.db
-rw-r--r-- 3 wy staff 4276 9 4 21:07 0-g-31-Statistics.db
//////under "backups" folder
///// these are original data files, with almost same size 1.2 mb. It seems that the thread-hood size of data in memory flushed to disk is 1.2 mb??
conf.memtable_total_space_in_mb = (int) (Runtime.getRuntime().maxMemory() / (3 * 1048576));
if -Xmx256M ,
conf.memtable_total_space_in_mb = 83mb. Then, how to figure out 1.2mb?
-rw-r--r-- 1 wy staff 1221766 9 4 21:00 0-g-1-Data.db
-rw-r--r-- 1 wy staff 16 9 4 21:00 0-g-1-Filter.db
-rw-r--r-- 1 wy staff 14 9 4 21:00 0-g-1-Index.db
-rw-r--r-- 1 wy staff 4276 9 4 21:00 0-g-1-Statistics.db
-rw-r--r-- 1 wy staff 1210346 9 4 21:02 0-g-10-Data.db
-rw-r--r-- 1 wy staff 16 9 4 21:02 0-g-10-Filter.db
-rw-r--r-- 1 wy staff 14 9 4 21:02 0-g-10-Index.db
-rw-r--r-- 1 wy staff 4276 9 4 21:02 0-g-10-Statistics.db
-rw-r--r-- 1 wy staff 1225068 9 4 21:02 0-g-11-Data.db
-rw-r--r-- 1 wy staff 16 9 4 21:02 0-g-11-Filter.db
-rw-r--r-- 1 wy staff 14 9 4 21:02 0-g-11-Index.db
-rw-r--r-- 1 wy staff 4276 9 4 21:02 0-g-11-Statistics.db
-rw-r--r-- 1 wy staff 1223328 9 4 21:02 0-g-12-Data.db
-rw-r--r-- 1 wy staff 16 9 4 21:02 0-g-12-Filter.db
-rw-r--r-- 1 wy staff 14 9 4 21:02 0-g-12-Index.db
-rw-r--r-- 1 wy staff 4276 9 4 21:02 0-g-12-Statistics.db
-rw-r--r-- 1 wy staff 1211216 9 4 21:03 0-g-14-Data.db
-rw-r--r-- 1 wy staff 16 9 4 21:03 0-g-14-Filter.db
-rw-r--r-- 1 wy staff 14 9 4 21:03 0-g-14-Index.db
-rw-r--r-- 1 wy staff 4276 9 4 21:03 0-g-14-Statistics.db
-rw-r--r-- 1 wy staff 1256036 9 4 21:03 0-g-15-Data.db
-rw-r--r-- 1 wy staff 16 9 4 21:03 0-g-15-Filter.db
-rw-r--r-- 1 wy staff 14 9 4 21:03 0-g-15-Index.db
-rw-r--r-- 1 wy staff 4276 9 4 21:03 0-g-15-Statistics.db
-rw-r--r-- 1 wy staff 1201208 9 4 21:03 0-g-16-Data.db
-rw-r--r-- 1 wy staff 16 9 4 21:03 0-g-16-Filter.db
-rw-r--r-- 1 wy staff 14 9 4 21:03 0-g-16-Index.db
-rw-r--r-- 1 wy staff 4276 9 4 21:03 0-g-16-Statistics.db
-rw-r--r-- 1 wy staff 1208896 9 4 21:04 0-g-18-Data.db
-rw-r--r-- 1 wy staff 16 9 4 21:04 0-g-18-Filter.db
-rw-r--r-- 1 wy staff 14 9 4 21:04 0-g-18-Index.db
-rw-r--r-- 1 wy staff 4276 9 4 21:04 0-g-18-Statistics.db
-rw-r--r-- 1 wy staff 1251322 9 4 21:04 0-g-19-Data.db
-rw-r--r-- 1 wy staff 16 9 4 21:04 0-g-19-Filter.db
-rw-r--r-- 1 wy staff 14 9 4 21:04 0-g-19-Index.db
-rw-r--r-- 1 wy staff 4276 9 4 21:04 0-g-19-Statistics.db
-rw-r--r-- 1 wy staff 1211504 9 4 21:00 0-g-2-Data.db
-rw-r--r-- 1 wy staff 16 9 4 21:00 0-g-2-Filter.db
-rw-r--r-- 1 wy staff 14 9 4 21:00 0-g-2-Index.db
-rw-r--r-- 1 wy staff 4276 9 4 21:00 0-g-2-Statistics.db
-rw-r--r-- 1 wy staff 1249872 9 4 21:04 0-g-20-Data.db
-rw-r--r-- 1 wy staff 16 9 4 21:04 0-g-20-Filter.db
-rw-r--r-- 1 wy staff 14 9 4 21:04 0-g-20-Index.db
-rw-r--r-- 1 wy staff 4276 9 4 21:04 0-g-20-Statistics.db
-rw-r--r-- 1 wy staff 1218034 9 4 21:05 0-g-22-Data.db
-rw-r--r-- 1 wy staff 16 9 4 21:05 0-g-22-Filter.db
-rw-r--r-- 1 wy staff 14 9 4 21:05 0-g-22-Index.db
-rw-r--r-- 1 wy staff 4276 9 4 21:05 0-g-22-Statistics.db
-rw-r--r-- 1 wy staff 1244578 9 4 21:05 0-g-23-Data.db
-rw-r--r-- 1 wy staff 16 9 4 21:05 0-g-23-Filter.db
-rw-r--r-- 1 wy staff 14 9 4 21:05 0-g-23-Index.db
-rw-r--r-- 1 wy staff 4276 9 4 21:05 0-g-23-Statistics.db
-rw-r--r-- 1 wy staff 1213320 9 4 21:05 0-g-24-Data.db
-rw-r--r-- 1 wy staff 16 9 4 21:05 0-g-24-Filter.db
-rw-r--r-- 1 wy staff 14 9 4 21:05 0-g-24-Index.db
-rw-r--r-- 1 wy staff 4276 9 4 21:05 0-g-24-Statistics.db
-rw-r--r-- 1 wy staff 1196204 9 4 21:06 0-g-26-Data.db
-rw-r--r-- 1 wy staff 16 9 4 21:06 0-g-26-Filter.db
-rw-r--r-- 1 wy staff 14 9 4 21:06 0-g-26-Index.db
-rw-r--r-- 1 wy staff 4276 9 4 21:06 0-g-26-Statistics.db
-rw-r--r-- 1 wy staff 1236890 9 4 21:06 0-g-27-Data.db
-rw-r--r-- 1 wy staff 16 9 4 21:06 0-g-27-Filter.db
-rw-r--r-- 1 wy staff 14 9 4 21:06 0-g-27-Index.db
-rw-r--r-- 1 wy staff 4276 9 4 21:06 0-g-27-Statistics.db
-rw-r--r-- 1 wy staff 1174382 9 4 21:06 0-g-28-Data.db
-rw-r--r-- 1 wy staff 16 9 4 21:06 0-g-28-Filter.db
-rw-r--r-- 1 wy staff 14 9 4 21:06 0-g-28-Index.db
-rw-r--r-- 1 wy staff 4276 9 4 21:06 0-g-28-Statistics.db
-rw-r--r-- 1 wy staff 1196784 9 4 21:01 0-g-3-Data.db
-rw-r--r-- 1 wy staff 16 9 4 21:01 0-g-3-Filter.db
-rw-r--r-- 1 wy staff 14 9 4 21:01 0-g-3-Index.db
-rw-r--r-- 1 wy staff 4276 9 4 21:01 0-g-3-Statistics.db
-rw-r--r-- 3 wy staff 1206800 9 4 21:07 0-g-30-Data.db
-rw-r--r-- 3 wy staff 16 9 4 21:07 0-g-30-Filter.db
-rw-r--r-- 3 wy staff 14 9 4 21:07 0-g-30-Index.db
-rw-r--r-- 3 wy staff 4276 9 4 21:07 0-g-30-Statistics.db
-rw-r--r-- 3 wy staff 1238050 9 4 21:07 0-g-31-Data.db
-rw-r--r-- 3 wy staff 16 9 4 21:07 0-g-31-Filter.db
-rw-r--r-- 3 wy staff 14 9 4 21:07 0-g-31-Index.db
-rw-r--r-- 3 wy staff 4276 9 4 21:07 0-g-31-Statistics.db
-rw-r--r-- 1 wy staff 1195044 9 4 21:01 0-g-4-Data.db
-rw-r--r-- 1 wy staff 16 9 4 21:01 0-g-4-Filter.db
-rw-r--r-- 1 wy staff 14 9 4 21:01 0-g-4-Index.db
-rw-r--r-- 1 wy staff 4276 9 4 21:01 0-g-4-Statistics.db
-rw-r--r-- 1 wy staff 1170538 9 4 21:01 0-g-6-Data.db
-rw-r--r-- 1 wy staff 16 9 4 21:01 0-g-6-Filter.db
-rw-r--r-- 1 wy staff 14 9 4 21:01 0-g-6-Index.db
-rw-r--r-- 1 wy staff 4276 9 4 21:01 0-g-6-Statistics.db
-rw-r--r-- 1 wy staff 1199468 9 4 21:01 0-g-7-Data.db
-rw-r--r-- 1 wy staff 16 9 4 21:01 0-g-7-Filter.db
-rw-r--r-- 1 wy staff 14 9 4 21:01 0-g-7-Index.db
-rw-r--r-- 1 wy staff 4276 9 4 21:01 0-g-7-Statistics.db
-rw-r--r-- 1 wy staff 1198532 9 4 21:02 0-g-8-Data.db
-rw-r--r-- 1 wy staff 16 9 4 21:02 0-g-8-Filter.db
-rw-r--r-- 1 wy staff 14 9 4 21:02 0-g-8-Index.db
-rw-r--r-- 1 wy staff 4276 9 4 21:02 0-g-8-Statistics.db
/////Is there any optimization for extremely large row size?
Best Regards!
Yi Wang(Jenny)
Debug & Timeout
Posted by Jenny <aw...@163.com>.
Hi
I've already set Long rpc_timeout_in_ms = Long.MAX_VALUE;
How can i avoid time out exception, when debugging the cassandra server.
Best Regards!
Yi Wang(Jenny)