You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@cassandra.apache.org by Renato Baceolar da Silveira <re...@indabamobile.co.za> on 2011/10/20 10:59:23 UTC

Column Family not able to be replicated between 2 nodes

Hi All and greetings from South Africa, a country with a/*sad Rugby team 
:(*/

I am processing a batch operation, that inserts a number of column families.

This batch operation is executed in a cluster with 2 nodes and 
replication of 2.

EACH NODE is installed in a  VMWARE ESSI virtual box, in a Ubuntu 9
server 64 bit. Both Nodes live in the same physical server, and 
communication between
them is through shared hardware.

  The virtual machines have 4gb of ram, and plenty hdd
space.

The version of Cassandra is 0.8.5.

Both nodes start empty before this batch process commences.

At the insertion of the first Column Family I receive this error in the 
log, which
tells me that for a number of times the specific column family is Replayed
for insertion, but it seems not to succeed. Without the insertion of this
first Column Family our validation process fails the batch process.

Can any one perhaps shed some light on this matter?


Kind Regards

Renato da Silveira

===============
error log - Applicaion Process
===============

me.prettyprint.hector.api.
exceptions.HectorException: SchemaDisagreementException()
     at 
me.prettyprint.cassandra.service.ExceptionsTranslatorImpl.translate(ExceptionsTranslatorImpl.java:68)
     at 
me.prettyprint.cassandra.service.ThriftCluster$4.execute(ThriftCluster.java:82)
     at 
me.prettyprint.cassandra.service.ThriftCluster$4.execute(ThriftCluster.java:76)
     at 
me.prettyprint.cassandra.service.Operation.executeAndSetResult(Operation.java:101)
     at 
me.prettyprint.cassandra.connection.HConnectionManager.operateWithFailover(HConnectionManager.java:219)
     at 
me.prettyprint.cassandra.service.ThriftCluster.addColumnFamily(ThriftCluster.java:86)
     at 
com.xxx.xxx.hector.HectorManager.addColumnFamily(HectorManager.java:132)
     at 
com.xxx.xxx.facade.CassandraManager.addColumnFamily(CassandraManager.java:117)
     at com.xxx.xxx.model.xxx.createRegistryContainer(xxx.java:63)
     at com.xxx.xxx.model.xxx.buildRegistry(xxx.java:120)
     at com.xxx.xxx.facade.Process.articleLoaderDoInstance(Process.java:117)
     at com.xxx.xxx.facade.Process.articleLoader(Process.java:58)
Caused by: SchemaDisagreementException()
     at 
org.apache.cassandra.thrift.Cassandra$system_add_column_family_result.read(Cassandra.java:26119)
     at 
org.apache.cassandra.thrift.Cassandra$Client.recv_system_add_column_family(Cassandra.java:1455)
     at 
org.apache.cassandra.thrift.Cassandra$Client.system_add_column_family(Cassandra.java:1430)
     at 
me.prettyprint.cassandra.service.ThriftCluster$4.execute(ThriftCluster.java:80)
     ... 10 more


===============
error log - Cassandra Log
===============
INFO [MigrationStage:1] 2011-10-20 10:03:58,944 Migration.java (line 
116) Applying migration 0aadb220-faf2-11e0-0000-1584a17080bf Add 
keyspace: MyKeyspace, rep strategy:SimpleStrategy{}, durable_writes: true
DEBUG [MigrationStage:1] 2011-10-20 10:03:58,945 Table.java (line 398) 
applying mutation of row 4c617374204d6967726174696f6e
DEBUG [MigrationStage:1] 2011-10-20 10:03:58,945 ColumnFamilyStore.java 
(line 765) flush position is ReplayPosition(segmentId=1319097494650, 
position=10486)
  INFO [MigrationStage:1] 2011-10-20 10:03:58,945 ColumnFamilyStore.java 
(line 1128) Enqueuing flush of Memtable-Migrations@539124954(6675/8343 
serialized/live bytes, 1 ops)
DEBUG [MigrationStage:1] 2011-10-20 10:03:58,946 ColumnFamilyStore.java 
(line 765) flush position is ReplayPosition(segmentId=1319097494650, 
position=10486)
  INFO [FlushWriter:3] 2011-10-20 10:03:58,946 Memtable.java (line 237) 
Writing Memtable-Migrations@539124954(6675/8343 serialized/live bytes, 1 
ops)
  INFO [MigrationStage:1] 2011-10-20 10:03:58,946 ColumnFamilyStore.java 
(line 1128) Enqueuing flush of Memtable-Schema@1225252094(2696/3370 
serialized/live bytes, 3 ops)
DEBUG [FlushWriter:3] 2011-10-20 10:03:58,947 DatabaseDescriptor.java 
(line 821) expected data files size is 8043; largest free partition has 
88471957504 bytes free
  INFO [FlushWriter:3] 2011-10-20 10:03:58,991 Memtable.java (line 254) 
Completed flushing /var/lib/cassandra/data/system/Migrations-g-1-Data.db 
(6739 bytes)
DEBUG [FlushWriter:3] 2011-10-20 10:03:58,991 DataTracker.java (line 
264) adding /var/lib/cassandra/data/system/Migrations-g-1 to list of 
files tracked for system.Migrations
DEBUG [COMMIT-LOG-WRITER] 2011-10-20 10:03:58,992 CommitLog.java (line 
446) discard completed log segments for 
ReplayPosition(segmentId=1319097494650, position=10486), column family 2.
  INFO [FlushWriter:3] 2011-10-20 10:03:58,992 Memtable.java (line 237) 
Writing Memtable-Schema@1225252094(2696/3370 serialized/live bytes, 3 ops)
DEBUG [COMMIT-LOG-WRITER] 2011-10-20 10:03:58,992 CommitLog.java (line 
486) Not safe to delete commit log 
CommitLogSegment(/var/lib/cassandra/commitlog/CommitLog-1319097494650.log); 
dirty is Schema (3), ; hasNext: false
DEBUG [CompactionExecutor:4] 2011-10-20 10:03:58,992 
CompactionManager.java (line 124) Checking to see if compaction of 
Migrations would be useful
DEBUG [FlushWriter:3] 2011-10-20 10:03:58,993 DatabaseDescriptor.java 
(line 821) expected data files size is 3355; largest free partition has 
88471932928 bytes free
  INFO [FlushWriter:3] 2011-10-20 10:03:59,026 Memtable.java (line 254) 
Completed flushing /var/lib/cassandra/data/system/Schema-g-1-Data.db 
(2846 bytes)
DEBUG [FlushWriter:3] 2011-10-20 10:03:59,026 DataTracker.java (line 
264) adding /var/lib/cassandra/data/system/Schema-g-1 to list of files 
tracked for system.Schema
DEBUG [COMMIT-LOG-WRITER] 2011-10-20 10:03:59,027 CommitLog.java (line 
446) discard completed log segments for 
ReplayPosition(segmentId=1319097494650, position=10486), column family 3.
DEBUG [CompactionExecutor:5] 2011-10-20 10:03:59,027 
CompactionManager.java (line 124) Checking to see if compaction of 
Schema would be useful
DEBUG [COMMIT-LOG-WRITER] 2011-10-20 10:03:59,027 CommitLog.java (line 
486) Not safe to delete commit log 
CommitLogSegment(/var/lib/cassandra/commitlog/CommitLog-1319097494650.log); 
dirty is ; hasNext: false
DEBUG [MigrationStage:1] 2011-10-20 10:03:59,028 MigrationManager.java 
(line 137) Gossiping my schema version 0aadb220-faf2-11e0-0000-1584a17080bf
DEBUG [MigrationStage:1] 2011-10-20 10:03:59,029 
SchemaCheckVerbHandler.java (line 36) Received schema check request.
DEBUG [ScheduledTasks:1] 2011-10-20 10:04:16,797 
StorageLoadBalancer.java (line 336) Disseminating load info ...
DEBUG [MigrationStage:1] 2011-10-20 10:04:37,373 
SchemaCheckVerbHandler.java (line 36) Received schema check request.
*DEBUG [MigrationStage:1] 2011-10-20 10:04:37,444 
DefinitionsUpdateVerbHandler.java (line 70) Applying AddColumnFamily 
from /192.168.1.30 <http://192.168.1.30>*
DEBUG [MigrationStage:1] 2011-10-20 10:04:37,444 Table.java (line 398) 
applying mutation of row 
32316535633164302d666166322d313165302d303030302d313538346131373038306266
DEBUG [MigrationStage:1] 2011-10-20 10:04:37,447 Table.java (line 398) 
applying mutation of row 4d6967726174696f6e73204b6579
  INFO [MigrationStage:1] 2011-10-20 10:04:37,456 Migration.java (line 
116)*Applying migration 21e5c1d0-faf2-11e0-0000-1584a17080bf Add column 
family: 
org.apache.cassandra.config.CFMetaData@7f1fc4b8[cfId=1000,ksName=MyKeyspace,cfName=a1307,cfType=Standard,comparator=org.apache.cassandra.db.marshal.AsciiType,subcolumncomparator=<null>,comment=,rowCacheSize=0.0,keyCacheSize=200000.0,readRepairChance=1.0,replicateOnWrite=true,gcGraceSeconds=864000,defaultValidator=org.apache.cassandra.db.marshal.BytesType,keyValidator=org.apache.cassandra.db.marshal.BytesType,minCompactionThreshold=4,maxCompactionThreshold=32,rowCacheSavePeriodInSeconds=0,keyCacheSavePeriodInSeconds=14400,memtableFlushAfterMins=1440,memtableThroughputInMb=55,memtableOperationsInMillions=0.2578125,mergeShardsChance=0.1,keyAlias=<null>,column_metadata={}]*
DEBUG [MigrationStage:1] 2011-10-20 10:04:37,456 Table.java (line 398) 
applying mutation of row 4c617374204d6967726174696f6e
*DEBUG [MigrationStage:1] 2011-10-20 10:04:37,456 ColumnFamilyStore.java 
(line 765) flush position is ReplayPosition(segmentId=1319097494650, 
position=20900)*
  INFO [MigrationStage:1] 2011-10-20 10:04:37,457 ColumnFamilyStore.java 
(line 1128) Enqueuing flush of Memtable-Migrations@2125221179(7183/8978 
serialized/live bytes, 1 ops)
DEBUG [MigrationStage:1] 2011-10-20 10:04:37,457 ColumnFamilyStore.java 
(line 765) flush position is ReplayPosition(segmentId=1319097494650, 
position=20900)
  INFO [FlushWriter:3] 2011-10-20 10:04:37,457 Memtable.java (line 237) 
Writing Memtable-Migrations@2125221179(7183/8978 serialized/live bytes, 
1 ops)
  INFO [MigrationStage:1] 2011-10-20 10:04:37,458 ColumnFamilyStore.java 
(line 1128) Enqueuing flush of Memtable-Schema@1644338549(2990/3737 
serialized/live bytes, 3 ops)
DEBUG [FlushWriter:3] 2011-10-20 10:04:37,459 DatabaseDescriptor.java 
(line 821) expected data files size is 8653; largest free partition has 
88471896064 bytes free
  INFO [FlushWriter:3] 2011-10-20 10:04:37,513 Memtable.java (line 254) 
Completed flushing /var/lib/cassandra/data/system/Migrations-g-2-Data.db 
(7247 bytes)
DEBUG [FlushWriter:3] 2011-10-20 10:04:37,514 DataTracker.java (line 
264) adding /var/lib/cassandra/data/system/Migrations-g-2 to list of 
files tracked for system.Migrations
DEBUG [FlushWriter:3] 2011-10-20 10:04:37,514 AutoSavingCache.java (line 
175) KeyCache capacity for Migrations is 2
DEBUG [COMMIT-LOG-WRITER] 2011-10-20 10:04:37,514 CommitLog.java (line 
446) discard completed log segments for 
ReplayPosition(segmentId=1319097494650, position=20900), column family 2.
  INFO [FlushWriter:3] 2011-10-20 10:04:37,515 Memtable.java (line 237) 
Writing Memtable-Schema@1644338549(2990/3737 serialized/live bytes, 3 ops)
DEBUG [CompactionExecutor:6] 2011-10-20 10:04:37,515 
CompactionManager.java (line 124) Checking to see if compaction of 
Migrations would be useful
DEBUG [COMMIT-LOG-WRITER] 2011-10-20 10:04:37,515 CommitLog.java (line 
486) Not safe to delete commit log 
CommitLogSegment(/var/lib/cassandra/commitlog/CommitLog-1319097494650.log); 
dirty is Schema (3), ; hasNext: false
  INFO [FlushWriter:3] 2011-10-20 10:04:37,515 Memtable.java (line 237) 
Writing Memtable-Schema@1644338549(2990/3737 serialized/live bytes, 3 ops)
DEBUG [CompactionExecutor:6] 2011-10-20 10:04:37,515 
CompactionManager.java (line 124) Checking to see if compaction of 
Migrations would be useful
DEBUG [COMMIT-LOG-WRITER] 2011-10-20 10:04:37,515 CommitLog.java (line 
486) Not safe to delete commit log 
CommitLogSegment(/var/lib/cassandra/commitlog/CommitLog-1319097494650.log); 
dirty is Schema (3), ; hasNext: false
DEBUG [FlushWriter:3] 2011-10-20 10:04:37,515 DatabaseDescriptor.java 
(line 821) expected data files size is 3708; largest free partition has 
88471871488 bytes free
ERROR [MutationStage:33] 2011-10-20 10:04:37,530 
RowMutationVerbHandler.java (line 82) Error in row mutation
org.apache.cassandra.db.UnserializableColumnFamilyException: Couldn't 
find cfId=1000
     at 
org.apache.cassandra.db.ColumnFamilySerializer.deserialize(ColumnFamilySerializer.java:123)
     at 
org.apache.cassandra.db.RowMutation$RowMutationSerializer.deserialize(RowMutation.java:392)
     at 
org.apache.cassandra.db.RowMutation$RowMutationSerializer.deserialize(RowMutation.java:400)
     at org.apache.cassandra.db.RowMutation.fromBytes(RowMutation.java:329)
     at 
org.apache.cassandra.db.RowMutationVerbHandler.doVerb(RowMutationVerbHandler.java:46)
     at 
org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:59)
     at 
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
     at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
     at java.lang.Thread.run(Thread.java:662)
  INFO [FlushWriter:3] 2011-10-20 10:04:37,553 Memtable.java (line 254) 
Completed flushing /var/lib/cassandra/data/system/Schema-g-2-Data.db 
(3140 bytes)
DEBUG [FlushWriter:3] 2011-10-20 10:04:37,553 DataTracker.java (line 
264) adding /var/lib/cassandra/data/system/Schema-g-2 to list of files 
tracked for system.Schema
DEBUG [FlushWriter:3] 2011-10-20 10:04:37,554 AutoSavingCache.java (line 
175) KeyCache capacity for Schema is 2
DEBUG [COMMIT-LOG-WRITER] 2011-10-20 10:04:37,555 CommitLog.java (line 
446) discard completed log segments for 
ReplayPosition(segmentId=1319097494650, position=20900), column family 3.
DEBUG [CompactionExecutor:7] 2011-10-20 10:04:37,555 
CompactionManager.java (line 124) Checking to see if compaction of 
Schema would be useful
DEBUG [COMMIT-LOG-WRITER] 2011-10-20 10:04:37,555 CommitLog.java (line 
486) Not safe to delete commit log 
CommitLogSegment(/var/lib/cassandra/commitlog/CommitLog-1319097494650.log); 
dirty is ; hasNext: false
DEBUG [MigrationStage:1] 2011-10-20 10:04:37,556 ColumnFamilyStore.java 
(line 264) Starting CFS a1307
DEBUG [MigrationStage:1] 2011-10-20 10:04:37,557 AutoSavingCache.java 
(line 175) KeyCache capacity for a1307 is 200000
DEBUG [MigrationStage:1] 2011-10-20 10:04:37,557 MigrationManager.java 
(line 137) Gossiping my schema version 21e5c1d0-faf2-11e0-0000-1584a17080bf
DEBUG [MutationStage:34] 2011-10-20 10:04:47,503 
RowMutationVerbHandler.java (line 48) Applying 
RowMutation(keyspace='MyKeyspace', key='6131333034', 
modifications=[ColumnFamily(a1307 [User:false:3@1319097867943000,])])
DEBUG [MutationStage:34] 2011-10-20 10:04:47,504 Table.java (line 398) 
applying mutation of row 6131333034
DEBUG [MutationStage:34] 2011-10-20 10:04:47,506 
RowMutationVerbHandler.java (line 77) RowMutation(keyspace='MyKeyspace', 
key='6131333034', modifications=[ColumnFamily(a1307 
[User:false:3@1319097867943000,])]) applied.  Sending response to 
1490@/192.168.1.30 <http://192.168.1.30>
DEBUG [MutationStage:35] 2011-10-20 10:04:47,544 
RowMutationVerbHandler.java (line 48) Applying 
RowMutation(keyspace='MyKeyspace', key='6131333034', 
modifications=[ColumnFamily(a1307 [Group:false:3@1319097878019000,])])
DEBUG [MutationStage:35] 2011-10-20 10:04:47,546 Table.java (line 398) 
applying mutation of row 6131333034
DEBUG [MutationStage:35] 2011-10-20 10:04:47,546 
RowMutationVerbHandler.java (line 77) RowMutation(keyspace='MyKeyspace', 
key='6131333034', modifications=[ColumnFamily(a1307 
[Group:false:3@1319097878019000,])]) applied.  Sending response to 
1491@/192.168.1.30 <http://192.168.1.30>
DEBUG [MutationStage:36] 2011-10-20 10:04:47,553 
RowMutationVerbHandler.java (line 48) Applying 
RowMutation(keyspace='MyKeyspace', key='6131333035', 
modifications=[ColumnFamily(a1307 [RegContent:false:5@1319097878059000,])])
DEBUG [MutationStage:36] 2011-10-20 10:04:47,554 Table.java (line 398) 
applying mutation of row 6131333035
DEBUG [MutationStage:36] 2011-10-20 10:04:47,554 
RowMutationVerbHandler.java (line 77) RowMutation(keyspace='MyKeyspace', 
key='6131333035', modifications=[ColumnFamily(a1307 
[RegContent:false:5@1319097878059000,])]) applied.  Sending response to 
1492@/192.168.1.30 <http://192.168.1.30>
DEBUG [MutationStage:37] 2011-10-20 10:04:47,558 
RowMutationVerbHandler.java (line 48) Applying 
RowMutation(keyspace='MyKeyspace', key='6131333035', 
modifications=[ColumnFamily(a1307 [RegGroup:false:5@1319097878065000,])])
DEBUG [MutationStage:37] 2011-10-20 10:04:47,558 Table.java (line 398) 
applying mutation of row 6131333035
DEBUG [MutationStage:37] 2011-10-20 10:04:47,559 
RowMutationVerbHandler.java (line 77) RowMutation(keyspace='MyKeyspace', 
key='6131333035', modifications=[ColumnFamily(a1307 
[RegGroup:false:5@1319097878065000,])]) applied.  Sending response to 
1493@/192.168.1.30 <http://192.168.1.30>
DEBUG [MutationStage:38] 2011-10-20 10:04:47,563 
RowMutationVerbHandler.java (line 48) Applying 
RowMutation(keyspace='MyKeyspace', key='6131333035', 
modifications=[ColumnFamily(a1307 [RegUser:false:5@1319097878070000,])])
DEBUG [MutationStage:38] 2011-10-20 10:04:47,563 Table.java (line 398) 
applying mutation of row 6131333035
DEBUG [MutationStage:38] 2011-10-20 10:04:47,563 
RowMutationVerbHandler.java (line 77) RowMutation(keyspace='MyKeyspace', 
key='6131333035', modifications=[ColumnFamily(a1307 
[RegUser:false:5@1319097878070000,])]) applied.  Sending response to 
1494@/192.168.1.30 <http://192.168.1.30>
DEBUG [MutationStage:39] 2011-10-20 10:04:47,567 
RowMutationVerbHandler.java (line 48) Applying 
RowMutation(keyspace='MyKeyspace', key='6131333035', 
modifications=[ColumnFamily(a1307 
[RegContentSched:false:5@1319097878074000,])])
DEBUG [MutationStage:39] 2011-10-20 10:04:47,567 Table.java (line 398) 
applying mutation of row 6131333035
DEBUG [MutationStage:39] 2011-10-20 10:04:47,567 
RowMutationVerbHandler.java (line 77) RowMutation(keyspace='MyKeyspace', 
key='6131333035', modifications=[ColumnFamily(a1307 
[RegContentSched:false:5@1319097878074000,])]) applied.  Sending 
response to 1495@/192.168.1.30 <http://192.168.1.30>
DEBUG [MutationStage:40] 2011-10-20 10:04:47,571 
RowMutationVerbHandler.java (line 48) Applying 
RowMutation(keyspace='MyKeyspace', key='6131333035', 
modifications=[ColumnFamily(a1307 
[ActiveEntity:false:5@1319097878078000,])])
DEBUG [MutationStage:40] 2011-10-20 10:04:47,571 Table.java (line 398) 
applying mutation of row 6131333035
DEBUG [MutationStage:40] 2011-10-20 10:04:47,571 
RowMutationVerbHandler.java (line 77) RowMutation(keyspace='MyKeyspace', 
key='6131333035', modifications=[ColumnFamily(a1307 
[ActiveEntity:false:5@1319097878078000,])]) applied.  Sending response 
to 1496@/192.168.1.30 <http://192.168.1.30>
DEBUG [MutationStage:41] 2011-10-20 10:04:47,574 
RowMutationVerbHandler.java (line 48) Applying 
RowMutation(keyspace='MyKeyspace', key='6131333035', 
modifications=[ColumnFamily(a1307 
[InactiveEntity:false:5@1319097878082000,])])
DEBUG [MutationStage:41] 2011-10-20 10:04:47,575 Table.java (line 398) 
applying mutation of row 6131333035
DEBUG [MutationStage:41] 2011-10-20 10:04:47,575 
RowMutationVerbHandler.java (line 77) RowMutation(keyspace='MyKeyspace', 
key='6131333035', modifications=[ColumnFamily(a1307 
[InactiveEntity:false:5@1319097878082000,])]) applied.  Sending response 
to 1497@/192.168.1.30 <http://192.168.1.30>
DEBUG [MutationStage:42] 2011-10-20 10:04:47,579 
RowMutationVerbHandler.java (line 48) Applying 
RowMutation(keyspace='MyKeyspace', key='6131333036', 
modifications=[ColumnFamily(a1307 [Friend List:false:5@1319097878087000,])])
DEBUG [MutationStage:42] 2011-10-20 10:04:47,579 Table.java (line 398) 
applying mutation of row 6131333036
DEBUG [MutationStage:42] 2011-10-20 10:04:47,580 
RowMutationVerbHandler.java (line 77) RowMutation(keyspace='MyKeyspace', 
key='6131333036', modifications=[ColumnFamily(a1307 [Friend 
List:false:5@1319097878087000,])]) applied.  Sending response to 
1498@/192.168.1.30 <http://192.168.1.30>
DEBUG [MutationStage:43] 2011-10-20 10:04:47,582 
RowMutationVerbHandler.java (line 48) Applying 
RowMutation(keyspace='MyKeyspace', key='6131333036', 
modifications=[ColumnFamily(a1307 [Profile 
Items:false:5@1319097878090000,])])
DEBUG [MutationStage:43] 2011-10-20 10:04:47,583 Table.java (line 398) 
applying mutation of row 6131333036
DEBUG [MutationStage:43] 2011-10-20 10:04:47,583 
RowMutationVerbHandler.java (line 77) RowMutation(keyspace='MyKeyspace', 
key='6131333036', modifications=[ColumnFamily(a1307 [Profile 
Items:false:5@1319097878090000,])]) applied.  Sending response to 
1499@/192.168.1.30 <http://192.168.1.30>
DEBUG [MutationStage:44] 2011-10-20 10:04:47,586 
RowMutationVerbHandler.java (line 48) Applying 
RowMutation(keyspace='MyKeyspace', key='6131333036', 
modifications=[ColumnFamily(a1307 [Likes:false:5@1319097878094000,])])
DEBUG [MutationStage:44] 2011-10-20 10:04:47,586 Table.java (line 398) 
applying mutation of row 6131333036
DEBUG [MutationStage:44] 2011-10-20 10:04:47,587 
RowMutationVerbHandler.java (line 77) RowMutation(keyspace='MyKeyspace', 
key='6131333036', modifications=[ColumnFamily(a1307 
[Likes:false:5@1319097878094000,])]) applied.  Sending response to 
1500@/192.168.1.30 <http://192.168.1.30>
DEBUG [MutationStage:45] 2011-10-20 10:04:47,590 
RowMutationVerbHandler.java (line 48) Applying 
RowMutation(keyspace='MyKeyspace', key='6131333036', 
modifications=[ColumnFamily(a1307 [Groups:false:5@1319097878097000,])])
DEBUG [MutationStage:45] 2011-10-20 10:04:47,590 Table.java (line 398) 
applying mutation of row 6131333036
DEBUG [MutationStage:45] 2011-10-20 10:04:47,591 
RowMutationVerbHandler.java (line 77) RowMutation(keyspace='MyKeyspace', 
key='6131333036', modifications=[ColumnFamily(a1307 
[Groups:false:5@1319097878097000,])]) applied.  Sending response to 
1501@/192.168.1.30 <http://192.168.1.30>
DEBUG [MutationStage:46] 2011-10-20 10:04:47,594 
RowMutationVerbHandler.java (line 48) Applying 
RowMutation(keyspace='MyKeyspace', key='6131333036', 
modifications=[ColumnFamily(a1307 [Content:false:5@1319097878102000,])])
DEBUG [MutationStage:46] 2011-10-20 10:04:47,595 Table.java (line 398) 
applying mutation of row 6131333036
DEBUG [MutationStage:46] 2011-10-20 10:04:47,595 
RowMutationVerbHandler.java (line 77) RowMutation(keyspace='MyKeyspace', 
key='6131333036', modifications=[ColumnFamily(a1307 
[Content:false:5@1319097878102000,])]) applied.  Sending response to 
1502@/192.168.1.30 <http://192.168.1.30>
DEBUG [MutationStage:47] 2011-10-20 10:04:47,598 
RowMutationVerbHandler.java (line 48) Applying 
RowMutation(keyspace='MyKeyspace', key='6131333036', 
modifications=[ColumnFamily(a1307 [Events:false:5@1319097878106000,])])
DEBUG [MutationStage:47] 2011-10-20 10:04:47,599 Table.java (line 398) 
applying mutation of row 6131333036
DEBUG [MutationStage:47] 2011-10-20 10:04:47,599 
RowMutationVerbHandler.java (line 77) RowMutation(keyspace='MyKeyspace', 
key='6131333036', modifications=[ColumnFamily(a1307 
[Events:false:5@1319097878106000,])]) applied.  Sending response to 
1503@/192.168.1.30 <http://192.168.1.30>
DEBUG [MutationStage:48] 2011-10-20 10:04:47,602 
RowMutationVerbHandler.java (line 48) Applying 
RowMutation(keyspace='MyKeyspace', key='6131333036', 
modifications=[ColumnFamily(a1307 [Bespoke 
Containers:false:5@1319097878110000,])])
DEBUG [MutationStage:48] 2011-10-20 10:04:47,602 Table.java (line 398) 
applying mutation of row 6131333036
DEBUG [MutationStage:48] 2011-10-20 10:04:47,603 
RowMutationVerbHandler.java (line 77) RowMutation(keyspace='MyKeyspace', 
key='6131333036', modifications=[ColumnFamily(a1307 [Bespoke 
Containers:false:5@1319097878110000,])]) applied.  Sending response to 
1504@/192.168.1.30 <http://192.168.1.30>
DEBUG [MutationStage:49] 2011-10-20 10:04:47,605 
RowMutationVerbHandler.java (line 48) Applying 
RowMutation(keyspace='MyKeyspace', key='6131323231', 
modifications=[ColumnFamily(a1307 [Entity ID 
Field:false:5@1319097878113000,])])
DEBUG [MutationStage:49] 2011-10-20 10:04:47,606 Table.java (line 398) 
applying mutation of row 6131323231
DEBUG [MutationStage:49] 2011-10-20 10:04:47,606 
RowMutationVerbHandler.java (line 77) RowMutation(keyspace='MyKeyspace', 
key='6131323231', modifications=[ColumnFamily(a1307 [Entity ID 
Field:false:5@1319097878113000,])]) applied.  Sending response to 
1505@/192.168.1.30 <http://192.168.1.30>
DEBUG [MutationStage:50] 2011-10-20 10:04:47,609 
RowMutationVerbHandler.java (line 48) Applying 
RowMutation(keyspace='MyKeyspace', key='6131323231', 
modifications=[ColumnFamily(a1307 [Entity Type 
Field:false:5@1319097878117000,])])
DEBUG [MutationStage:50] 2011-10-20 10:04:47,609 Table.java (line 398) 
applying mutation of row 6131323231
DEBUG [MutationStage:50] 2011-10-20 10:04:47,610 
RowMutationVerbHandler.java (line 77) RowMutation(keyspace='MyKeyspace', 
key='6131323231', modifications=[ColumnFamily(a1307 [Entity Type 
Field:false:5@1319097878117000,])]) applied.  Sending response to 
1506@/192.168.1.30 <http://192.168.1.30>
DEBUG [MigrationStage:1] 2011-10-20 10:04:47,612 
SchemaCheckVerbHandler.java (line 36) Received schema check request.
DEBUG [MigrationStage:1] 2011-10-20 10:04:47,655 
SchemaCheckVerbHandler.java (line 36) Received schema check request.
DEBUG [MigrationStage:1] 2011-10-20 10:04:47,671 
DefinitionsUpdateVerbHandler.java (line 70) Applying AddColumnFamily 
from /192.168.1.30 <http://192.168.1.30>
DEBUG [MigrationStage:1] 2011-10-20 10:04:47,671 Table.java (line 398) 
applying mutation of row 
32376666636361302d666166322d313165302d303030302d313538346131373038306266
DEBUG [MigrationStage:1] 2011-10-20 10:04:47,673 Table.java (line 398) 
applying mutation of row 4d6967726174696f6e73204b6579
  I*NFO [MigrationStage:1] 2011-10-20 10:04:47,673 Migration.java (line 
116) Applying migration 27ffcca0-faf2-11e0-0000-1584a17080bf Add column 
family: 
org.apache.cassandra.config.CFMetaData@33a998c7[cfId=1001,ksName=MyKeyspace,cfName=a1101,cfType=Standard,comparator=org.apache.cassandra.db.marshal.AsciiType,subcolumncomparator=<null>,comment=,rowCacheSize=0.0,keyCacheSize=200000.0,readRepairChance=1.0,replicateOnWrite=true,gcGraceSeconds=864000,defaultValidator=org.apache.cassandra.db.marshal.BytesType,keyValidator=org.apache.cassandra.db.marshal.BytesType,minCompactionThreshold=4,maxCompactionThreshold=32,rowCacheSavePeriodInSeconds=0,keyCacheSavePeriodInSeconds=14400,memtableFlushAfterMins=1440,memtableThroughputInMb=55,memtableOperationsInMillions=0.2578125,mergeShardsChance=0.1,keyAlias=<null>,column_metadata={}]*
DEBUG [MigrationStage:1] 2011-10-20 10:04:47,673 Table.java (line 398) 
applying mutation of row 4c617374204d6967726174696f6e
DEBUG [MigrationStage:1] 2011-10-20 10:04:47,674 ColumnFamilyStore.java 
(line 765) flush position is ReplayPosition(segmentId=1319097494650, 
position=33582)
  INFO [MigrationStage:1] 2011-10-20 10:04:47,674 ColumnFamilyStore.java 
(line 1128) Enqueuing flush of Memtable-Migrations@1131236176(7476/9345 
serialized/live bytes, 1 ops)
DEBUG [MigrationStage:1] 2011-10-20 10:04:47,674 ColumnFamilyStore.java 
(line 765) flush position is ReplayPosition(segmentId=1319097494650, 
position=33582)
  INFO [FlushWriter:3] 2011-10-20 10:04:47,675 Memtable.java (line 237) 
Writing Memtable-Migrations@1131236176(7476/9345 serialized/live bytes, 
1 ops)
  INFO [MigrationStage:1] 2011-10-20 10:04:47,676 ColumnFamilyStore.java 
(line 1128) Enqueuing flush of Memtable-Schema@1013683141(3283/4103 
serialized/live bytes, 3 ops)
DEBUG [FlushWriter:3] 2011-10-20 10:04:47,676 DatabaseDescriptor.java 
(line 821) expected data files size is 9004; largest free partition has 
88471822336 bytes free
  INFO [FlushWriter:3] 2011-10-20 10:04:47,725 Memtable.java (line 254) 
Completed flushing /var/lib/cassandra/data/system/Migrations-g-3-Data.db 
(7540 bytes)
DEBUG [FlushWriter:3] 2011-10-20 10:04:47,725 DataTracker.java (line 
264) adding /var/lib/cassandra/data/system/Migrations-g-3 to list of 
files tracked for system.Migrations
DEBUG [FlushWriter:3] 2011-10-20 10:04:47,726 AutoSavingCache.java (line 
175) KeyCache capacity for Migrations is 3
DEBUG [CompactionExecutor:4] 2011-10-20 10:04:47,726 
CompactionManager.java (line 124) Checking to see if compaction of 
Migrations would be useful
  INFO [FlushWriter:3] 2011-10-20 10:04:47,726 Memtable.java (line 237) 
Writing Memtable-Schema@1013683141(3283/4103 serialized/live bytes, 3 ops)
DEBUG [COMMIT-LOG-WRITER] 2011-10-20 10:04:47,726 CommitLog.java (line 
446) discard completed log segments for 
ReplayPosition(segmentId=1319097494650, position=33582), column family 2.
DEBUG [COMMIT-LOG-WRITER] 2011-10-20 10:04:47,727 CommitLog.java (line 
486) Not safe to delete commit log 
CommitLogSegment(/var/lib/cassandra/commitlog/CommitLog-1319097494650.log); 
dirty is a1307 (1000), Schema (3), ; hasNext: false
DEBUG [FlushWriter:3] 2011-10-20 10:04:47,727 DatabaseDescriptor.java 
(line 821) expected data files size is 4059; largest free partition has 
88471797760 bytes free
  INFO [FlushWriter:3] 2011-10-20 10:04:47,751 Memtable.java (line 254) 
Completed flushing /var/lib/cassandra/data/system/Schema-g-3-Data.db 
(3433 bytes)
DEBUG [FlushWriter:3] 2011-10-20 10:04:47,752 DataTracker.java (line 
264) adding /var/lib/cassandra/data/system/Schema-g-3 to list of files 
tracked for system.Schema
DEBUG [FlushWriter:3] 2011-10-20 10:04:47,752 AutoSavingCache.java (line 
175) KeyCache capacity for Schema is 3
DEBUG [CompactionExecutor:5] 2011-10-20 10:04:47,752 
CompactionManager.java (line 124) Checking to see if compaction of 
Schema would be useful
DEBUG [COMMIT-LOG-WRITER] 2011-10-20 10:04:47,752 CommitLog.java (line 
446) discard completed log segments for 
ReplayPosition(segmentId=1319097494650, position=33582), column family 3.
DEBUG [COMMIT-LOG-WRITER] 2011-10-20 10:04:47,753 CommitLog.java (line 
486) Not safe to delete commit log 
CommitLogSegment(/var/lib/cassandra/commitlog/CommitLog-1319097494650.log); 
dirty is a1307 (1000), ; hasNext: false


-- 
Renato Bacelar da Silveira
Software Engineer (Java J2SE J2EE)

Re: Column Family not able to be replicated between 2 nodes

Posted by aaron morton <aa...@thelastpickle.com>.

Also if you are still playing around now is a good time to get up to date and using the latest 0.8 release. 

Cheers
(Temporary Kiwi (ex Australia, Ex England))

-----------------
Aaron Morton
Freelance Developer
@aaronmorton
http://www.thelastpickle.com

On 20/10/2011, at 10:45 PM, Jérémy SEVELLEC wrote:

> no problem
> 
> and sorry for your sad rugby team
> 
> Jérémy (french :-))
> 
> 2011/10/20 Renato Baceolar da Silveira <re...@indabamobile.co.za>
> Thank You Jeremy
> 
> Will report back to you with any changes!
> 
> Regards
> 
> 
> On 20/10/2011 11:15, Jérémy SEVELLEC wrote:
>> 
>> Hi,
>> 
>> I think it's because you create a new columnFamily with hector and you don't wait the propagation of this modification in the cluster before writing.
>> 
>> Hector provide a method witjh a boolean to say "blockUntilComplete" when you create a columnFamily :
>> cluster.addColumnFamily(<your cf definition>, true);
>> 
>> Here is the hector javadoc about that : http://rantav.github.com/hector//source/content/API/core/0.8.0-2/me/prettyprint/hector/api/Cluster.html#addColumnFamily(me.prettyprint.hector.api.ddl.ColumnFamilyDefinition, boolean)
>> 
>> Regards,
>> 
>> Jérémy
>> 
>> 2011/10/20 Renato Baceolar da Silveira <re...@indabamobile.co.za>
>> Hi All and greetings from South Africa, a country with a sad Rugby team :(
>> 
>> I am processing a batch operation, that inserts a number of column families.
>> 
>> This batch operation is executed in a cluster with 2 nodes and replication of 2.
>> 
>> EACH NODE is installed in a  VMWARE ESSI virtual box, in a Ubuntu 9
>> server 64 bit. Both Nodes live in the same physical server, and communication between
>> them is through shared hardware.
>> 
>>  The virtual machines have 4gb of ram, and plenty hdd 
>> space. 
>> 
>> The version of Cassandra is 0.8.5.
>> 
>> Both nodes start empty before this batch process commences.
>> 
>> At the insertion of the first Column Family I receive this error in the log, which
>> tells me that for a number of times the specific column family is Replayed
>> for insertion, but it seems not to succeed. Without the insertion of this
>> first Column Family our validation process fails the batch process.
>> 
>> Can any one perhaps shed some light on this matter?
>> 
>> 
>> Kind Regards
>> 
>> Renato da Silveira
>> 
>> ===============
>> error log - Applicaion Process
>> ===============
>> 
>> me.prettyprint.hector.api.
>> exceptions.HectorException: SchemaDisagreementException()
>>     at me.prettyprint.cassandra.service.ExceptionsTranslatorImpl.translate(ExceptionsTranslatorImpl.java:68)
>>     at me.prettyprint.cassandra.service.ThriftCluster$4.execute(ThriftCluster.java:82)
>>     at me.prettyprint.cassandra.service.ThriftCluster$4.execute(ThriftCluster.java:76)
>>     at me.prettyprint.cassandra.service.Operation.executeAndSetResult(Operation.java:101)
>>     at me.prettyprint.cassandra.connection.HConnectionManager.operateWithFailover(HConnectionManager.java:219)
>>     at me.prettyprint.cassandra.service.ThriftCluster.addColumnFamily(ThriftCluster.java:86)
>>     at com.xxx.xxx.hector.HectorManager.addColumnFamily(HectorManager.java:132)
>>     at com.xxx.xxx.facade.CassandraManager.addColumnFamily(CassandraManager.java:117)
>>     at com.xxx.xxx.model.xxx.createRegistryContainer(xxx.java:63)
>>     at com.xxx.xxx.model.xxx.buildRegistry(xxx.java:120)
>>     at com.xxx.xxx.facade.Process.articleLoaderDoInstance(Process.java:117)
>>     at com.xxx.xxx.facade.Process.articleLoader(Process.java:58)
>> Caused by: SchemaDisagreementException()
>>     at org.apache.cassandra.thrift.Cassandra$system_add_column_family_result.read(Cassandra.java:26119)
>>     at org.apache.cassandra.thrift.Cassandra$Client.recv_system_add_column_family(Cassandra.java:1455)
>>     at org.apache.cassandra.thrift.Cassandra$Client.system_add_column_family(Cassandra.java:1430)
>>     at me.prettyprint.cassandra.service.ThriftCluster$4.execute(ThriftCluster.java:80)
>>     ... 10 more
>> 
>> 
>> ===============
>> error log - Cassandra Log
>> ===============
>> INFO [MigrationStage:1] 2011-10-20 10:03:58,944 Migration.java (line 116) Applying migration 0aadb220-faf2-11e0-0000-1584a17080bf Add keyspace: MyKeyspace, rep strategy:SimpleStrategy{}, durable_writes: true
>> DEBUG [MigrationStage:1] 2011-10-20 10:03:58,945 Table.java (line 398) applying mutation of row 4c617374204d6967726174696f6e
>> DEBUG [MigrationStage:1] 2011-10-20 10:03:58,945 ColumnFamilyStore.java (line 765) flush position is ReplayPosition(segmentId=1319097494650, position=10486)
>>  INFO [MigrationStage:1] 2011-10-20 10:03:58,945 ColumnFamilyStore.java (line 1128) Enqueuing flush of Memtable-Migrations@539124954(6675/8343 serialized/live bytes, 1 ops)
>> DEBUG [MigrationStage:1] 2011-10-20 10:03:58,946 ColumnFamilyStore.java (line 765) flush position is ReplayPosition(segmentId=1319097494650, position=10486)
>>  INFO [FlushWriter:3] 2011-10-20 10:03:58,946 Memtable.java (line 237) Writing Memtable-Migrations@539124954(6675/8343 serialized/live bytes, 1 ops)
>>  INFO [MigrationStage:1] 2011-10-20 10:03:58,946 ColumnFamilyStore.java (line 1128) Enqueuing flush of Memtable-Schema@1225252094(2696/3370 serialized/live bytes, 3 ops)
>> DEBUG [FlushWriter:3] 2011-10-20 10:03:58,947 DatabaseDescriptor.java (line 821) expected data files size is 8043; largest free partition has 88471957504 bytes free
>>  INFO [FlushWriter:3] 2011-10-20 10:03:58,991 Memtable.java (line 254) Completed flushing /var/lib/cassandra/data/system/Migrations-g-1-Data.db (6739 bytes)
>> DEBUG [FlushWriter:3] 2011-10-20 10:03:58,991 DataTracker.java (line 264) adding /var/lib/cassandra/data/system/Migrations-g-1 to list of files tracked for system.Migrations
>> DEBUG [COMMIT-LOG-WRITER] 2011-10-20 10:03:58,992 CommitLog.java (line 446) discard completed log segments for ReplayPosition(segmentId=1319097494650, position=10486), column family 2.
>>  INFO [FlushWriter:3] 2011-10-20 10:03:58,992 Memtable.java (line 237) Writing Memtable-Schema@1225252094(2696/3370 serialized/live bytes, 3 ops)
>> DEBUG [COMMIT-LOG-WRITER] 2011-10-20 10:03:58,992 CommitLog.java (line 486) Not safe to delete commit log CommitLogSegment(/var/lib/cassandra/commitlog/CommitLog-1319097494650.log); dirty is Schema (3), ; hasNext: false
>> DEBUG [CompactionExecutor:4] 2011-10-20 10:03:58,992 CompactionManager.java (line 124) Checking to see if compaction of Migrations would be useful
>> DEBUG [FlushWriter:3] 2011-10-20 10:03:58,993 DatabaseDescriptor.java (line 821) expected data files size is 3355; largest free partition has 88471932928 bytes free
>>  INFO [FlushWriter:3] 2011-10-20 10:03:59,026 Memtable.java (line 254) Completed flushing /var/lib/cassandra/data/system/Schema-g-1-Data.db (2846 bytes)
>> DEBUG [FlushWriter:3] 2011-10-20 10:03:59,026 DataTracker.java (line 264) adding /var/lib/cassandra/data/system/Schema-g-1 to list of files tracked for system.Schema
>> DEBUG [COMMIT-LOG-WRITER] 2011-10-20 10:03:59,027 CommitLog.java (line 446) discard completed log segments for ReplayPosition(segmentId=1319097494650, position=10486), column family 3.
>> DEBUG [CompactionExecutor:5] 2011-10-20 10:03:59,027 CompactionManager.java (line 124) Checking to see if compaction of Schema would be useful
>> DEBUG [COMMIT-LOG-WRITER] 2011-10-20 10:03:59,027 CommitLog.java (line 486) Not safe to delete commit log CommitLogSegment(/var/lib/cassandra/commitlog/CommitLog-1319097494650.log); dirty is ; hasNext: false
>> DEBUG [MigrationStage:1] 2011-10-20 10:03:59,028 MigrationManager.java (line 137) Gossiping my schema version 0aadb220-faf2-11e0-0000-1584a17080bf
>> DEBUG [MigrationStage:1] 2011-10-20 10:03:59,029 SchemaCheckVerbHandler.java (line 36) Received schema check request.
>> DEBUG [ScheduledTasks:1] 2011-10-20 10:04:16,797 StorageLoadBalancer.java (line 336) Disseminating load info ...
>> DEBUG [MigrationStage:1] 2011-10-20 10:04:37,373 SchemaCheckVerbHandler.java (line 36) Received schema check request.
>> DEBUG [MigrationStage:1] 2011-10-20 10:04:37,444 DefinitionsUpdateVerbHandler.java (line 70) Applying AddColumnFamily from /192.168.1.30
>> DEBUG [MigrationStage:1] 2011-10-20 10:04:37,444 Table.java (line 398) applying mutation of row 32316535633164302d666166322d313165302d303030302d313538346131373038306266
>> DEBUG [MigrationStage:1] 2011-10-20 10:04:37,447 Table.java (line 398) applying mutation of row 4d6967726174696f6e73204b6579
>>  INFO [MigrationStage:1] 2011-10-20 10:04:37,456 Migration.java (line 116) Applying migration 21e5c1d0-faf2-11e0-0000-1584a17080bf Add column family: org.apache.cassandra.config.CFMetaData@7f1fc4b8[cfId=1000,ksName=MyKeyspace,cfName=a1307,cfType=Standard,comparator=org.apache.cassandra.db.marshal.AsciiType,subcolumncomparator=<null>,comment=,rowCacheSize=0.0,keyCacheSize=200000.0,readRepairChance=1.0,replicateOnWrite=true,gcGraceSeconds=864000,defaultValidator=org.apache.cassandra.db.marshal.BytesType,keyValidator=org.apache.cassandra.db.marshal.BytesType,minCompactionThreshold=4,maxCompactionThreshold=32,rowCacheSavePeriodInSeconds=0,keyCacheSavePeriodInSeconds=14400,memtableFlushAfterMins=1440,memtableThroughputInMb=55,memtableOperationsInMillions=0.2578125,mergeShardsChance=0.1,keyAlias=<null>,column_metadata={}]
>> DEBUG [MigrationStage:1] 2011-10-20 10:04:37,456 Table.java (line 398) applying mutation of row 4c617374204d6967726174696f6e
>> DEBUG [MigrationStage:1] 2011-10-20 10:04:37,456 ColumnFamilyStore.java (line 765) flush position is ReplayPosition(segmentId=1319097494650, position=20900)
>>  INFO [MigrationStage:1] 2011-10-20 10:04:37,457 ColumnFamilyStore.java (line 1128) Enqueuing flush of Memtable-Migrations@2125221179(7183/8978 serialized/live bytes, 1 ops)
>> DEBUG [MigrationStage:1] 2011-10-20 10:04:37,457 ColumnFamilyStore.java (line 765) flush position is ReplayPosition(segmentId=1319097494650, position=20900)
>>  INFO [FlushWriter:3] 2011-10-20 10:04:37,457 Memtable.java (line 237) Writing Memtable-Migrations@2125221179(7183/8978 serialized/live bytes, 1 ops)
>>  INFO [MigrationStage:1] 2011-10-20 10:04:37,458 ColumnFamilyStore.java (line 1128) Enqueuing flush of Memtable-Schema@1644338549(2990/3737 serialized/live bytes, 3 ops)
>> DEBUG [FlushWriter:3] 2011-10-20 10:04:37,459 DatabaseDescriptor.java (line 821) expected data files size is 8653; largest free partition has 88471896064 bytes free
>>  INFO [FlushWriter:3] 2011-10-20 10:04:37,513 Memtable.java (line 254) Completed flushing /var/lib/cassandra/data/system/Migrations-g-2-Data.db (7247 bytes)
>> DEBUG [FlushWriter:3] 2011-10-20 10:04:37,514 DataTracker.java (line 264) adding /var/lib/cassandra/data/system/Migrations-g-2 to list of files tracked for system.Migrations
>> DEBUG [FlushWriter:3] 2011-10-20 10:04:37,514 AutoSavingCache.java (line 175) KeyCache capacity for Migrations is 2
>> DEBUG [COMMIT-LOG-WRITER] 2011-10-20 10:04:37,514 CommitLog.java (line 446) discard completed log segments for ReplayPosition(segmentId=1319097494650, position=20900), column family 2.
>>  INFO [FlushWriter:3] 2011-10-20 10:04:37,515 Memtable.java (line 237) Writing Memtable-Schema@1644338549(2990/3737 serialized/live bytes, 3 ops)
>> DEBUG [CompactionExecutor:6] 2011-10-20 10:04:37,515 CompactionManager.java (line 124) Checking to see if compaction of Migrations would be useful
>> DEBUG [COMMIT-LOG-WRITER] 2011-10-20 10:04:37,515 CommitLog.java (line 486) Not safe to delete commit log CommitLogSegment(/var/lib/cassandra/commitlog/CommitLog-1319097494650.log); dirty is Schema (3), ; hasNext: false
>>  INFO [FlushWriter:3] 2011-10-20 10:04:37,515 Memtable.java (line 237) Writing Memtable-Schema@1644338549(2990/3737 serialized/live bytes, 3 ops)
>> DEBUG [CompactionExecutor:6] 2011-10-20 10:04:37,515 CompactionManager.java (line 124) Checking to see if compaction of Migrations would be useful
>> DEBUG [COMMIT-LOG-WRITER] 2011-10-20 10:04:37,515 CommitLog.java (line 486) Not safe to delete commit log CommitLogSegment(/var/lib/cassandra/commitlog/CommitLog-1319097494650.log); dirty is Schema (3), ; hasNext: false
>> DEBUG [FlushWriter:3] 2011-10-20 10:04:37,515 DatabaseDescriptor.java (line 821) expected data files size is 3708; largest free partition has 88471871488 bytes free
>> ERROR [MutationStage:33] 2011-10-20 10:04:37,530 RowMutationVerbHandler.java (line 82) Error in row mutation
>> org.apache.cassandra.db.UnserializableColumnFamilyException: Couldn't find cfId=1000
>>     at org.apache.cassandra.db.ColumnFamilySerializer.deserialize(ColumnFamilySerializer.java:123)
>>     at org.apache.cassandra.db.RowMutation$RowMutationSerializer.deserialize(RowMutation.java:392)
>>     at org.apache.cassandra.db.RowMutation$RowMutationSerializer.deserialize(RowMutation.java:400)
>>     at org.apache.cassandra.db.RowMutation.fromBytes(RowMutation.java:329)
>>     at org.apache.cassandra.db.RowMutationVerbHandler.doVerb(RowMutationVerbHandler.java:46)
>>     at org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:59)
>>     at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>     at java.lang.Thread.run(Thread.java:662)
>>  INFO [FlushWriter:3] 2011-10-20 10:04:37,553 Memtable.java (line 254) Completed flushing /var/lib/cassandra/data/system/Schema-g-2-Data.db (3140 bytes)
>> DEBUG [FlushWriter:3] 2011-10-20 10:04:37,553 DataTracker.java (line 264) adding /var/lib/cassandra/data/system/Schema-g-2 to list of files tracked for system.Schema
>> DEBUG [FlushWriter:3] 2011-10-20 10:04:37,554 AutoSavingCache.java (line 175) KeyCache capacity for Schema is 2
>> DEBUG [COMMIT-LOG-WRITER] 2011-10-20 10:04:37,555 CommitLog.java (line 446) discard completed log segments for ReplayPosition(segmentId=1319097494650, position=20900), column family 3.
>> DEBUG [CompactionExecutor:7] 2011-10-20 10:04:37,555 CompactionManager.java (line 124) Checking to see if compaction of Schema would be useful
>> DEBUG [COMMIT-LOG-WRITER] 2011-10-20 10:04:37,555 CommitLog.java (line 486) Not safe to delete commit log CommitLogSegment(/var/lib/cassandra/commitlog/CommitLog-1319097494650.log); dirty is ; hasNext: false
>> DEBUG [MigrationStage:1] 2011-10-20 10:04:37,556 ColumnFamilyStore.java (line 264) Starting CFS a1307
>> DEBUG [MigrationStage:1] 2011-10-20 10:04:37,557 AutoSavingCache.java (line 175) KeyCache capacity for a1307 is 200000
>> DEBUG [MigrationStage:1] 2011-10-20 10:04:37,557 MigrationManager.java (line 137) Gossiping my schema version 21e5c1d0-faf2-11e0-0000-1584a17080bf
>> DEBUG [MutationStage:34] 2011-10-20 10:04:47,503 RowMutationVerbHandler.java (line 48) Applying RowMutation(keyspace='MyKeyspace', key='6131333034', modifications=[ColumnFamily(a1307 [User:false:3@1319097867943000,])])
>> DEBUG [MutationStage:34] 2011-10-20 10:04:47,504 Table.java (line 398) applying mutation of row 6131333034
>> DEBUG [MutationStage:34] 2011-10-20 10:04:47,506 RowMutationVerbHandler.java (line 77) RowMutation(keyspace='MyKeyspace', key='6131333034', modifications=[ColumnFamily(a1307 [User:false:3@1319097867943000,])]) applied.  Sending response to 1490@/192.168.1.30
>> DEBUG [MutationStage:35] 2011-10-20 10:04:47,544 RowMutationVerbHandler.java (line 48) Applying RowMutation(keyspace='MyKeyspace', key='6131333034', modifications=[ColumnFamily(a1307 [Group:false:3@1319097878019000,])])
>> DEBUG [MutationStage:35] 2011-10-20 10:04:47,546 Table.java (line 398) applying mutation of row 6131333034
>> DEBUG [MutationStage:35] 2011-10-20 10:04:47,546 RowMutationVerbHandler.java (line 77) RowMutation(keyspace='MyKeyspace', key='6131333034', modifications=[ColumnFamily(a1307 [Group:false:3@1319097878019000,])]) applied.  Sending response to 1491@/192.168.1.30
>> DEBUG [MutationStage:36] 2011-10-20 10:04:47,553 RowMutationVerbHandler.java (line 48) Applying RowMutation(keyspace='MyKeyspace', key='6131333035', modifications=[ColumnFamily(a1307 [RegContent:false:5@1319097878059000,])])
>> DEBUG [MutationStage:36] 2011-10-20 10:04:47,554 Table.java (line 398) applying mutation of row 6131333035
>> DEBUG [MutationStage:36] 2011-10-20 10:04:47,554 RowMutationVerbHandler.java (line 77) RowMutation(keyspace='MyKeyspace', key='6131333035', modifications=[ColumnFamily(a1307 [RegContent:false:5@1319097878059000,])]) applied.  Sending response to 1492@/192.168.1.30
>> DEBUG [MutationStage:37] 2011-10-20 10:04:47,558 RowMutationVerbHandler.java (line 48) Applying RowMutation(keyspace='MyKeyspace', key='6131333035', modifications=[ColumnFamily(a1307 [RegGroup:false:5@1319097878065000,])])
>> DEBUG [MutationStage:37] 2011-10-20 10:04:47,558 Table.java (line 398) applying mutation of row 6131333035
>> DEBUG [MutationStage:37] 2011-10-20 10:04:47,559 RowMutationVerbHandler.java (line 77) RowMutation(keyspace='MyKeyspace', key='6131333035', modifications=[ColumnFamily(a1307 [RegGroup:false:5@1319097878065000,])]) applied.  Sending response to 1493@/192.168.1.30
>> DEBUG [MutationStage:38] 2011-10-20 10:04:47,563 RowMutationVerbHandler.java (line 48) Applying RowMutation(keyspace='MyKeyspace', key='6131333035', modifications=[ColumnFamily(a1307 [RegUser:false:5@1319097878070000,])])
>> DEBUG [MutationStage:38] 2011-10-20 10:04:47,563 Table.java (line 398) applying mutation of row 6131333035
>> DEBUG [MutationStage:38] 2011-10-20 10:04:47,563 RowMutationVerbHandler.java (line 77) RowMutation(keyspace='MyKeyspace', key='6131333035', modifications=[ColumnFamily(a1307 [RegUser:false:5@1319097878070000,])]) applied.  Sending response to 1494@/192.168.1.30
>> DEBUG [MutationStage:39] 2011-10-20 10:04:47,567 RowMutationVerbHandler.java (line 48) Applying RowMutation(keyspace='MyKeyspace', key='6131333035', modifications=[ColumnFamily(a1307 [RegContentSched:false:5@1319097878074000,])])
>> DEBUG [MutationStage:39] 2011-10-20 10:04:47,567 Table.java (line 398) applying mutation of row 6131333035
>> DEBUG [MutationStage:39] 2011-10-20 10:04:47,567 RowMutationVerbHandler.java (line 77) RowMutation(keyspace='MyKeyspace', key='6131333035', modifications=[ColumnFamily(a1307 [RegContentSched:false:5@1319097878074000,])]) applied.  Sending response to 1495@/192.168.1.30
>> DEBUG [MutationStage:40] 2011-10-20 10:04:47,571 RowMutationVerbHandler.java (line 48) Applying RowMutation(keyspace='MyKeyspace', key='6131333035', modifications=[ColumnFamily(a1307 [ActiveEntity:false:5@1319097878078000,])])
>> DEBUG [MutationStage:40] 2011-10-20 10:04:47,571 Table.java (line 398) applying mutation of row 6131333035
>> DEBUG [MutationStage:40] 2011-10-20 10:04:47,571 RowMutationVerbHandler.java (line 77) RowMutation(keyspace='MyKeyspace', key='6131333035', modifications=[ColumnFamily(a1307 [ActiveEntity:false:5@1319097878078000,])]) applied.  Sending response to 1496@/192.168.1.30
>> DEBUG [MutationStage:41] 2011-10-20 10:04:47,574 RowMutationVerbHandler.java (line 48) Applying RowMutation(keyspace='MyKeyspace', key='6131333035', modifications=[ColumnFamily(a1307 [InactiveEntity:false:5@1319097878082000,])])
>> DEBUG [MutationStage:41] 2011-10-20 10:04:47,575 Table.java (line 398) applying mutation of row 6131333035
>> DEBUG [MutationStage:41] 2011-10-20 10:04:47,575 RowMutationVerbHandler.java (line 77) RowMutation(keyspace='MyKeyspace', key='6131333035', modifications=[ColumnFamily(a1307 [InactiveEntity:false:5@1319097878082000,])]) applied.  Sending response to 1497@/192.168.1.30
>> DEBUG [MutationStage:42] 2011-10-20 10:04:47,579 RowMutationVerbHandler.java (line 48) Applying RowMutation(keyspace='MyKeyspace', key='6131333036', modifications=[ColumnFamily(a1307 [Friend List:false:5@1319097878087000,])])
>> DEBUG [MutationStage:42] 2011-10-20 10:04:47,579 Table.java (line 398) applying mutation of row 6131333036
>> DEBUG [MutationStage:42] 2011-10-20 10:04:47,580 RowMutationVerbHandler.java (line 77) RowMutation(keyspace='MyKeyspace', key='6131333036', modifications=[ColumnFamily(a1307 [Friend List:false:5@1319097878087000,])]) applied.  Sending response to 1498@/192.168.1.30
>> DEBUG [MutationStage:43] 2011-10-20 10:04:47,582 RowMutationVerbHandler.java (line 48) Applying RowMutation(keyspace='MyKeyspace', key='6131333036', modifications=[ColumnFamily(a1307 [Profile Items:false:5@1319097878090000,])])
>> DEBUG [MutationStage:43] 2011-10-20 10:04:47,583 Table.java (line 398) applying mutation of row 6131333036
>> DEBUG [MutationStage:43] 2011-10-20 10:04:47,583 RowMutationVerbHandler.java (line 77) RowMutation(keyspace='MyKeyspace', key='6131333036', modifications=[ColumnFamily(a1307 [Profile Items:false:5@1319097878090000,])]) applied.  Sending response to 1499@/192.168.1.30
>> DEBUG [MutationStage:44] 2011-10-20 10:04:47,586 RowMutationVerbHandler.java (line 48) Applying RowMutation(keyspace='MyKeyspace', key='6131333036', modifications=[ColumnFamily(a1307 [Likes:false:5@1319097878094000,])])
>> DEBUG [MutationStage:44] 2011-10-20 10:04:47,586 Table.java (line 398) applying mutation of row 6131333036
>> DEBUG [MutationStage:44] 2011-10-20 10:04:47,587 RowMutationVerbHandler.java (line 77) RowMutation(keyspace='MyKeyspace', key='6131333036', modifications=[ColumnFamily(a1307 [Likes:false:5@1319097878094000,])]) applied.  Sending response to 1500@/192.168.1.30
>> DEBUG [MutationStage:45] 2011-10-20 10:04:47,590 RowMutationVerbHandler.java (line 48) Applying RowMutation(keyspace='MyKeyspace', key='6131333036', modifications=[ColumnFamily(a1307 [Groups:false:5@1319097878097000,])])
>> DEBUG [MutationStage:45] 2011-10-20 10:04:47,590 Table.java (line 398) applying mutation of row 6131333036
>> DEBUG [MutationStage:45] 2011-10-20 10:04:47,591 RowMutationVerbHandler.java (line 77) RowMutation(keyspace='MyKeyspace', key='6131333036', modifications=[ColumnFamily(a1307 [Groups:false:5@1319097878097000,])]) applied.  Sending response to 1501@/192.168.1.30
>> DEBUG [MutationStage:46] 2011-10-20 10:04:47,594 RowMutationVerbHandler.java (line 48) Applying RowMutation(keyspace='MyKeyspace', key='6131333036', modifications=[ColumnFamily(a1307 [Content:false:5@1319097878102000,])])
>> DEBUG [MutationStage:46] 2011-10-20 10:04:47,595 Table.java (line 398) applying mutation of row 6131333036
>> DEBUG [MutationStage:46] 2011-10-20 10:04:47,595 RowMutationVerbHandler.java (line 77) RowMutation(keyspace='MyKeyspace', key='6131333036', modifications=[ColumnFamily(a1307 [Content:false:5@1319097878102000,])]) applied.  Sending response to 1502@/192.168.1.30
>> DEBUG [MutationStage:47] 2011-10-20 10:04:47,598 RowMutationVerbHandler.java (line 48) Applying RowMutation(keyspace='MyKeyspace', key='6131333036', modifications=[ColumnFamily(a1307 [Events:false:5@1319097878106000,])])
>> DEBUG [MutationStage:47] 2011-10-20 10:04:47,599 Table.java (line 398) applying mutation of row 6131333036
>> DEBUG [MutationStage:47] 2011-10-20 10:04:47,599 RowMutationVerbHandler.java (line 77) RowMutation(keyspace='MyKeyspace', key='6131333036', modifications=[ColumnFamily(a1307 [Events:false:5@1319097878106000,])]) applied.  Sending response to 1503@/192.168.1.30
>> DEBUG [MutationStage:48] 2011-10-20 10:04:47,602 RowMutationVerbHandler.java (line 48) Applying RowMutation(keyspace='MyKeyspace', key='6131333036', modifications=[ColumnFamily(a1307 [Bespoke Containers:false:5@1319097878110000,])])
>> DEBUG [MutationStage:48] 2011-10-20 10:04:47,602 Table.java (line 398) applying mutation of row 6131333036
>> DEBUG [MutationStage:48] 2011-10-20 10:04:47,603 RowMutationVerbHandler.java (line 77) RowMutation(keyspace='MyKeyspace', key='6131333036', modifications=[ColumnFamily(a1307 [Bespoke Containers:false:5@1319097878110000,])]) applied.  Sending response to 1504@/192.168.1.30
>> DEBUG [MutationStage:49] 2011-10-20 10:04:47,605 RowMutationVerbHandler.java (line 48) Applying RowMutation(keyspace='MyKeyspace', key='6131323231', modifications=[ColumnFamily(a1307 [Entity ID Field:false:5@1319097878113000,])])
>> DEBUG [MutationStage:49] 2011-10-20 10:04:47,606 Table.java (line 398) applying mutation of row 6131323231
>> DEBUG [MutationStage:49] 2011-10-20 10:04:47,606 RowMutationVerbHandler.java (line 77) RowMutation(keyspace='MyKeyspace', key='6131323231', modifications=[ColumnFamily(a1307 [Entity ID Field:false:5@1319097878113000,])]) applied.  Sending response to 1505@/192.168.1.30
>> DEBUG [MutationStage:50] 2011-10-20 10:04:47,609 RowMutationVerbHandler.java (line 48) Applying RowMutation(keyspace='MyKeyspace', key='6131323231', modifications=[ColumnFamily(a1307 [Entity Type Field:false:5@1319097878117000,])])
>> DEBUG [MutationStage:50] 2011-10-20 10:04:47,609 Table.java (line 398) applying mutation of row 6131323231
>> DEBUG [MutationStage:50] 2011-10-20 10:04:47,610 RowMutationVerbHandler.java (line 77) RowMutation(keyspace='MyKeyspace', key='6131323231', modifications=[ColumnFamily(a1307 [Entity Type Field:false:5@1319097878117000,])]) applied.  Sending response to 1506@/192.168.1.30
>> DEBUG [MigrationStage:1] 2011-10-20 10:04:47,612 SchemaCheckVerbHandler.java (line 36) Received schema check request.
>> DEBUG [MigrationStage:1] 2011-10-20 10:04:47,655 SchemaCheckVerbHandler.java (line 36) Received schema check request.
>> DEBUG [MigrationStage:1] 2011-10-20 10:04:47,671 DefinitionsUpdateVerbHandler.java (line 70) Applying AddColumnFamily from /192.168.1.30
>> DEBUG [MigrationStage:1] 2011-10-20 10:04:47,671 Table.java (line 398) applying mutation of row 32376666636361302d666166322d313165302d303030302d313538346131373038306266
>> DEBUG [MigrationStage:1] 2011-10-20 10:04:47,673 Table.java (line 398) applying mutation of row 4d6967726174696f6e73204b6579
>>  INFO [MigrationStage:1] 2011-10-20 10:04:47,673 Migration.java (line 116) Applying migration 27ffcca0-faf2-11e0-0000-1584a17080bf Add column family: org.apache.cassandra.config.CFMetaData@33a998c7[cfId=1001,ksName=MyKeyspace,cfName=a1101,cfType=Standard,comparator=org.apache.cassandra.db.marshal.AsciiType,subcolumncomparator=<null>,comment=,rowCacheSize=0.0,keyCacheSize=200000.0,readRepairChance=1.0,replicateOnWrite=true,gcGraceSeconds=864000,defaultValidator=org.apache.cassandra.db.marshal.BytesType,keyValidator=org.apache.cassandra.db.marshal.BytesType,minCompactionThreshold=4,maxCompactionThreshold=32,rowCacheSavePeriodInSeconds=0,keyCacheSavePeriodInSeconds=14400,memtableFlushAfterMins=1440,memtableThroughputInMb=55,memtableOperationsInMillions=0.2578125,mergeShardsChance=0.1,keyAlias=<null>,column_metadata={}]
>> DEBUG [MigrationStage:1] 2011-10-20 10:04:47,673 Table.java (line 398) applying mutation of row 4c617374204d6967726174696f6e
>> DEBUG [MigrationStage:1] 2011-10-20 10:04:47,674 ColumnFamilyStore.java (line 765) flush position is ReplayPosition(segmentId=1319097494650, position=33582)
>>  INFO [MigrationStage:1] 2011-10-20 10:04:47,674 ColumnFamilyStore.java (line 1128) Enqueuing flush of Memtable-Migrations@1131236176(7476/9345 serialized/live bytes, 1 ops)
>> DEBUG [MigrationStage:1] 2011-10-20 10:04:47,674 ColumnFamilyStore.java (line 765) flush position is ReplayPosition(segmentId=1319097494650, position=33582)
>>  INFO [FlushWriter:3] 2011-10-20 10:04:47,675 Memtable.java (line 237) Writing Memtable-Migrations@1131236176(7476/9345 serialized/live bytes, 1 ops)
>>  INFO [MigrationStage:1] 2011-10-20 10:04:47,676 ColumnFamilyStore.java (line 1128) Enqueuing flush of Memtable-Schema@1013683141(3283/4103 serialized/live bytes, 3 ops)
>> DEBUG [FlushWriter:3] 2011-10-20 10:04:47,676 DatabaseDescriptor.java (line 821) expected data files size is 9004; largest free partition has 88471822336 bytes free
>>  INFO [FlushWriter:3] 2011-10-20 10:04:47,725 Memtable.java (line 254) Completed flushing /var/lib/cassandra/data/system/Migrations-g-3-Data.db (7540 bytes)
>> DEBUG [FlushWriter:3] 2011-10-20 10:04:47,725 DataTracker.java (line 264) adding /var/lib/cassandra/data/system/Migrations-g-3 to list of files tracked for system.Migrations
>> DEBUG [FlushWriter:3] 2011-10-20 10:04:47,726 AutoSavingCache.java (line 175) KeyCache capacity for Migrations is 3
>> DEBUG [CompactionExecutor:4] 2011-10-20 10:04:47,726 CompactionManager.java (line 124) Checking to see if compaction of Migrations would be useful
>>  INFO [FlushWriter:3] 2011-10-20 10:04:47,726 Memtable.java (line 237) Writing Memtable-Schema@1013683141(3283/4103 serialized/live bytes, 3 ops)
>> DEBUG [COMMIT-LOG-WRITER] 2011-10-20 10:04:47,726 CommitLog.java (line 446) discard completed log segments for ReplayPosition(segmentId=1319097494650, position=33582), column family 2.
>> DEBUG [COMMIT-LOG-WRITER] 2011-10-20 10:04:47,727 CommitLog.java (line 486) Not safe to delete commit log CommitLogSegment(/var/lib/cassandra/commitlog/CommitLog-1319097494650.log); dirty is a1307 (1000), Schema (3), ; hasNext: false
>> DEBUG [FlushWriter:3] 2011-10-20 10:04:47,727 DatabaseDescriptor.java (line 821) expected data files size is 4059; largest free partition has 88471797760 bytes free
>>  INFO [FlushWriter:3] 2011-10-20 10:04:47,751 Memtable.java (line 254) Completed flushing /var/lib/cassandra/data/system/Schema-g-3-Data.db (3433 bytes)
>> DEBUG [FlushWriter:3] 2011-10-20 10:04:47,752 DataTracker.java (line 264) adding /var/lib/cassandra/data/system/Schema-g-3 to list of files tracked for system.Schema
>> DEBUG [FlushWriter:3] 2011-10-20 10:04:47,752 AutoSavingCache.java (line 175) KeyCache capacity for Schema is 3
>> DEBUG [CompactionExecutor:5] 2011-10-20 10:04:47,752 CompactionManager.java (line 124) Checking to see if compaction of Schema would be useful
>> DEBUG [COMMIT-LOG-WRITER] 2011-10-20 10:04:47,752 CommitLog.java (line 446) discard completed log segments for ReplayPosition(segmentId=1319097494650, position=33582), column family 3.
>> DEBUG [COMMIT-LOG-WRITER] 2011-10-20 10:04:47,753 CommitLog.java (line 486) Not safe to delete commit log CommitLogSegment(/var/lib/cassandra/commitlog/CommitLog-1319097494650.log); dirty is a1307 (1000), ; hasNext: false
>> 
>> 
>> -- 
>> Renato Bacelar da Silveira
>> Software Engineer (Java J2SE J2EE)
>> 
>> 
>> 
>> -- 
>> Jérémy
> 
> 
> 
> 
> -- 
> Jérémy

Re: Column Family not able to be replicated between 2 nodes

Posted by Jérémy SEVELLEC <js...@gmail.com>.

no problem

and sorry for your sad rugby team

Jérémy (french :-))

2011/10/20 Renato Baceolar da Silveira <re...@indabamobile.co.za>

>  Thank You Jeremy
>
> Will report back to you with any changes!
>
> Regards
>
>
> On 20/10/2011 11:15, Jérémy SEVELLEC wrote:
>
> Hi,
>
>  I think it's because you create a new columnFamily with hector and you
> don't wait the propagation of this modification in the cluster before
> writing.
>
>  Hector provide a method witjh a boolean to say "blockUntilComplete" when
> you create a columnFamily :
> cluster.addColumnFamily(<your cf definition>, true);
>
>  Here is the hector javadoc about that : http://rantav.github.com/hector//source/content/API/core/0.8.0-2/me/prettyprint/hector/api/Cluster.html#addColumnFamily(me.prettyprint.hector.api.ddl.ColumnFamilyDefinition,
> boolean)<http://rantav.github.com/hector//source/content/API/core/0.8.0-2/me/prettyprint/hector/api/Cluster.html#addColumnFamily%28me.prettyprint.hector.api.ddl.ColumnFamilyDefinition,%20boolean%29>
>
>  Regards,
>
>  Jérémy
>
> 2011/10/20 Renato Baceolar da Silveira <re...@indabamobile.co.za>
>
>>  Hi All and greetings from South Africa, a country with a* sad Rugby team
>> :(*
>>
>> I am processing a batch operation, that inserts a number of column
>> families.
>>
>> This batch operation is executed in a cluster with 2 nodes and replication
>> of 2.
>>
>> EACH NODE is installed in a  VMWARE ESSI virtual box, in a Ubuntu 9
>> server 64 bit. Both Nodes live in the same physical server, and
>> communication between
>> them is through shared hardware.
>>
>>  The virtual machines have 4gb of ram, and plenty hdd
>> space.
>>
>> The version of Cassandra is 0.8.5.
>>
>> Both nodes start empty before this batch process commences.
>>
>> At the insertion of the first Column Family I receive this error in the
>> log, which
>> tells me that for a number of times the specific column family is Replayed
>> for insertion, but it seems not to succeed. Without the insertion of this
>> first Column Family our validation process fails the batch process.
>>
>> Can any one perhaps shed some light on this matter?
>>
>>
>> Kind Regards
>>
>> Renato da Silveira
>>
>> ===============
>> error log - Applicaion Process
>> ===============
>>
>> me.prettyprint.hector.api.
>> exceptions.HectorException: SchemaDisagreementException()
>>     at
>> me.prettyprint.cassandra.service.ExceptionsTranslatorImpl.translate(ExceptionsTranslatorImpl.java:68)
>>     at
>> me.prettyprint.cassandra.service.ThriftCluster$4.execute(ThriftCluster.java:82)
>>     at
>> me.prettyprint.cassandra.service.ThriftCluster$4.execute(ThriftCluster.java:76)
>>     at
>> me.prettyprint.cassandra.service.Operation.executeAndSetResult(Operation.java:101)
>>     at
>> me.prettyprint.cassandra.connection.HConnectionManager.operateWithFailover(HConnectionManager.java:219)
>>     at
>> me.prettyprint.cassandra.service.ThriftCluster.addColumnFamily(ThriftCluster.java:86)
>>     at
>> com.xxx.xxx.hector.HectorManager.addColumnFamily(HectorManager.java:132)
>>     at
>> com.xxx.xxx.facade.CassandraManager.addColumnFamily(CassandraManager.java:117)
>>     at com.xxx.xxx.model.xxx.createRegistryContainer(xxx.java:63)
>>     at com.xxx.xxx.model.xxx.buildRegistry(xxx.java:120)
>>     at
>> com.xxx.xxx.facade.Process.articleLoaderDoInstance(Process.java:117)
>>     at com.xxx.xxx.facade.Process.articleLoader(Process.java:58)
>> Caused by: SchemaDisagreementException()
>>     at
>> org.apache.cassandra.thrift.Cassandra$system_add_column_family_result.read(Cassandra.java:26119)
>>     at
>> org.apache.cassandra.thrift.Cassandra$Client.recv_system_add_column_family(Cassandra.java:1455)
>>     at
>> org.apache.cassandra.thrift.Cassandra$Client.system_add_column_family(Cassandra.java:1430)
>>     at
>> me.prettyprint.cassandra.service.ThriftCluster$4.execute(ThriftCluster.java:80)
>>     ... 10 more
>>
>>
>> ===============
>> error log - Cassandra Log
>> ===============
>> INFO [MigrationStage:1] 2011-10-20 10:03:58,944 Migration.java (line 116)
>> Applying migration 0aadb220-faf2-11e0-0000-1584a17080bf Add keyspace:
>> MyKeyspace, rep strategy:SimpleStrategy{}, durable_writes: true
>> DEBUG [MigrationStage:1] 2011-10-20 10:03:58,945 Table.java (line 398)
>> applying mutation of row 4c617374204d6967726174696f6e
>> DEBUG [MigrationStage:1] 2011-10-20 10:03:58,945 ColumnFamilyStore.java
>> (line 765) flush position is ReplayPosition(segmentId=1319097494650,
>> position=10486)
>>  INFO [MigrationStage:1] 2011-10-20 10:03:58,945 ColumnFamilyStore.java
>> (line 1128) Enqueuing flush of Memtable-Migrations@539124954(6675/8343
>> serialized/live bytes, 1 ops)
>> DEBUG [MigrationStage:1] 2011-10-20 10:03:58,946 ColumnFamilyStore.java
>> (line 765) flush position is ReplayPosition(segmentId=1319097494650,
>> position=10486)
>>  INFO [FlushWriter:3] 2011-10-20 10:03:58,946 Memtable.java (line 237)
>> Writing Memtable-Migrations@539124954(6675/8343 serialized/live bytes, 1
>> ops)
>>  INFO [MigrationStage:1] 2011-10-20 10:03:58,946 ColumnFamilyStore.java
>> (line 1128) Enqueuing flush of Memtable-Schema@1225252094(2696/3370
>> serialized/live bytes, 3 ops)
>> DEBUG [FlushWriter:3] 2011-10-20 10:03:58,947 DatabaseDescriptor.java
>> (line 821) expected data files size is 8043; largest free partition has
>> 88471957504 bytes free
>>  INFO [FlushWriter:3] 2011-10-20 10:03:58,991 Memtable.java (line 254)
>> Completed flushing /var/lib/cassandra/data/system/Migrations-g-1-Data.db
>> (6739 bytes)
>> DEBUG [FlushWriter:3] 2011-10-20 10:03:58,991 DataTracker.java (line 264)
>> adding /var/lib/cassandra/data/system/Migrations-g-1 to list of files
>> tracked for system.Migrations
>> DEBUG [COMMIT-LOG-WRITER] 2011-10-20 10:03:58,992 CommitLog.java (line
>> 446) discard completed log segments for
>> ReplayPosition(segmentId=1319097494650, position=10486), column family 2.
>>  INFO [FlushWriter:3] 2011-10-20 10:03:58,992 Memtable.java (line 237)
>> Writing Memtable-Schema@1225252094(2696/3370 serialized/live bytes, 3
>> ops)
>> DEBUG [COMMIT-LOG-WRITER] 2011-10-20 10:03:58,992 CommitLog.java (line
>> 486) Not safe to delete commit log
>> CommitLogSegment(/var/lib/cassandra/commitlog/CommitLog-1319097494650.log);
>> dirty is Schema (3), ; hasNext: false
>> DEBUG [CompactionExecutor:4] 2011-10-20 10:03:58,992
>> CompactionManager.java (line 124) Checking to see if compaction of
>> Migrations would be useful
>> DEBUG [FlushWriter:3] 2011-10-20 10:03:58,993 DatabaseDescriptor.java
>> (line 821) expected data files size is 3355; largest free partition has
>> 88471932928 bytes free
>>  INFO [FlushWriter:3] 2011-10-20 10:03:59,026 Memtable.java (line 254)
>> Completed flushing /var/lib/cassandra/data/system/Schema-g-1-Data.db (2846
>> bytes)
>> DEBUG [FlushWriter:3] 2011-10-20 10:03:59,026 DataTracker.java (line 264)
>> adding /var/lib/cassandra/data/system/Schema-g-1 to list of files tracked
>> for system.Schema
>> DEBUG [COMMIT-LOG-WRITER] 2011-10-20 10:03:59,027 CommitLog.java (line
>> 446) discard completed log segments for
>> ReplayPosition(segmentId=1319097494650, position=10486), column family 3.
>> DEBUG [CompactionExecutor:5] 2011-10-20 10:03:59,027
>> CompactionManager.java (line 124) Checking to see if compaction of Schema
>> would be useful
>> DEBUG [COMMIT-LOG-WRITER] 2011-10-20 10:03:59,027 CommitLog.java (line
>> 486) Not safe to delete commit log
>> CommitLogSegment(/var/lib/cassandra/commitlog/CommitLog-1319097494650.log);
>> dirty is ; hasNext: false
>> DEBUG [MigrationStage:1] 2011-10-20 10:03:59,028 MigrationManager.java
>> (line 137) Gossiping my schema version 0aadb220-faf2-11e0-0000-1584a17080bf
>> DEBUG [MigrationStage:1] 2011-10-20 10:03:59,029
>> SchemaCheckVerbHandler.java (line 36) Received schema check request.
>> DEBUG [ScheduledTasks:1] 2011-10-20 10:04:16,797 StorageLoadBalancer.java
>> (line 336) Disseminating load info ...
>> DEBUG [MigrationStage:1] 2011-10-20 10:04:37,373
>> SchemaCheckVerbHandler.java (line 36) Received schema check request.
>> *DEBUG [MigrationStage:1] 2011-10-20 10:04:37,444
>> DefinitionsUpdateVerbHandler.java (line 70) Applying AddColumnFamily from /
>> 192.168.1.30*
>> DEBUG [MigrationStage:1] 2011-10-20 10:04:37,444 Table.java (line 398)
>> applying mutation of row
>> 32316535633164302d666166322d313165302d303030302d313538346131373038306266
>> DEBUG [MigrationStage:1] 2011-10-20 10:04:37,447 Table.java (line 398)
>> applying mutation of row 4d6967726174696f6e73204b6579
>>  INFO [MigrationStage:1] 2011-10-20 10:04:37,456 Migration.java (line
>> 116)* Applying migration 21e5c1d0-faf2-11e0-0000-1584a17080bf Add column
>> family: org.apache.cassandra.config.CFMetaData@7f1fc4b8
>> [cfId=1000,ksName=MyKeyspace,cfName=a1307,cfType=Standard,comparator=org.apache.cassandra.db.marshal.AsciiType,subcolumncomparator=<null>,comment=,rowCacheSize=0.0,keyCacheSize=200000.0,readRepairChance=1.0,replicateOnWrite=true,gcGraceSeconds=864000,defaultValidator=org.apache.cassandra.db.marshal.BytesType,keyValidator=org.apache.cassandra.db.marshal.BytesType,minCompactionThreshold=4,maxCompactionThreshold=32,rowCacheSavePeriodInSeconds=0,keyCacheSavePeriodInSeconds=14400,memtableFlushAfterMins=1440,memtableThroughputInMb=55,memtableOperationsInMillions=0.2578125,mergeShardsChance=0.1,keyAlias=<null>,column_metadata={}]
>> *
>> DEBUG [MigrationStage:1] 2011-10-20 10:04:37,456 Table.java (line 398)
>> applying mutation of row 4c617374204d6967726174696f6e
>> *DEBUG [MigrationStage:1] 2011-10-20 10:04:37,456 ColumnFamilyStore.java
>> (line 765) flush position is ReplayPosition(segmentId=1319097494650,
>> position=20900)*
>>  INFO [MigrationStage:1] 2011-10-20 10:04:37,457 ColumnFamilyStore.java
>> (line 1128) Enqueuing flush of Memtable-Migrations@2125221179(7183/8978
>> serialized/live bytes, 1 ops)
>> DEBUG [MigrationStage:1] 2011-10-20 10:04:37,457 ColumnFamilyStore.java
>> (line 765) flush position is ReplayPosition(segmentId=1319097494650,
>> position=20900)
>>  INFO [FlushWriter:3] 2011-10-20 10:04:37,457 Memtable.java (line 237)
>> Writing Memtable-Migrations@2125221179(7183/8978 serialized/live bytes, 1
>> ops)
>>  INFO [MigrationStage:1] 2011-10-20 10:04:37,458 ColumnFamilyStore.java
>> (line 1128) Enqueuing flush of Memtable-Schema@1644338549(2990/3737
>> serialized/live bytes, 3 ops)
>> DEBUG [FlushWriter:3] 2011-10-20 10:04:37,459 DatabaseDescriptor.java
>> (line 821) expected data files size is 8653; largest free partition has
>> 88471896064 bytes free
>>  INFO [FlushWriter:3] 2011-10-20 10:04:37,513 Memtable.java (line 254)
>> Completed flushing /var/lib/cassandra/data/system/Migrations-g-2-Data.db
>> (7247 bytes)
>> DEBUG [FlushWriter:3] 2011-10-20 10:04:37,514 DataTracker.java (line 264)
>> adding /var/lib/cassandra/data/system/Migrations-g-2 to list of files
>> tracked for system.Migrations
>> DEBUG [FlushWriter:3] 2011-10-20 10:04:37,514 AutoSavingCache.java (line
>> 175) KeyCache capacity for Migrations is 2
>> DEBUG [COMMIT-LOG-WRITER] 2011-10-20 10:04:37,514 CommitLog.java (line
>> 446) discard completed log segments for
>> ReplayPosition(segmentId=1319097494650, position=20900), column family 2.
>>  INFO [FlushWriter:3] 2011-10-20 10:04:37,515 Memtable.java (line 237)
>> Writing Memtable-Schema@1644338549(2990/3737 serialized/live bytes, 3
>> ops)
>> DEBUG [CompactionExecutor:6] 2011-10-20 10:04:37,515
>> CompactionManager.java (line 124) Checking to see if compaction of
>> Migrations would be useful
>> DEBUG [COMMIT-LOG-WRITER] 2011-10-20 10:04:37,515 CommitLog.java (line
>> 486) Not safe to delete commit log
>> CommitLogSegment(/var/lib/cassandra/commitlog/CommitLog-1319097494650.log);
>> dirty is Schema (3), ; hasNext: false
>>  INFO [FlushWriter:3] 2011-10-20 10:04:37,515 Memtable.java (line 237)
>> Writing Memtable-Schema@1644338549(2990/3737 serialized/live bytes, 3
>> ops)
>> DEBUG [CompactionExecutor:6] 2011-10-20 10:04:37,515
>> CompactionManager.java (line 124) Checking to see if compaction of
>> Migrations would be useful
>> DEBUG [COMMIT-LOG-WRITER] 2011-10-20 10:04:37,515 CommitLog.java (line
>> 486) Not safe to delete commit log
>> CommitLogSegment(/var/lib/cassandra/commitlog/CommitLog-1319097494650.log);
>> dirty is Schema (3), ; hasNext: false
>> DEBUG [FlushWriter:3] 2011-10-20 10:04:37,515 DatabaseDescriptor.java
>> (line 821) expected data files size is 3708; largest free partition has
>> 88471871488 bytes free
>> ERROR [MutationStage:33] 2011-10-20 10:04:37,530
>> RowMutationVerbHandler.java (line 82) Error in row mutation
>> org.apache.cassandra.db.UnserializableColumnFamilyException: Couldn't find
>> cfId=1000
>>     at
>> org.apache.cassandra.db.ColumnFamilySerializer.deserialize(ColumnFamilySerializer.java:123)
>>     at
>> org.apache.cassandra.db.RowMutation$RowMutationSerializer.deserialize(RowMutation.java:392)
>>     at
>> org.apache.cassandra.db.RowMutation$RowMutationSerializer.deserialize(RowMutation.java:400)
>>     at org.apache.cassandra.db.RowMutation.fromBytes(RowMutation.java:329)
>>     at
>> org.apache.cassandra.db.RowMutationVerbHandler.doVerb(RowMutationVerbHandler.java:46)
>>     at org.apache.cassandra.net
>> .MessageDeliveryTask.run(MessageDeliveryTask.java:59)
>>     at
>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>     at
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>     at java.lang.Thread.run(Thread.java:662)
>>  INFO [FlushWriter:3] 2011-10-20 10:04:37,553 Memtable.java (line 254)
>> Completed flushing /var/lib/cassandra/data/system/Schema-g-2-Data.db (3140
>> bytes)
>> DEBUG [FlushWriter:3] 2011-10-20 10:04:37,553 DataTracker.java (line 264)
>> adding /var/lib/cassandra/data/system/Schema-g-2 to list of files tracked
>> for system.Schema
>> DEBUG [FlushWriter:3] 2011-10-20 10:04:37,554 AutoSavingCache.java (line
>> 175) KeyCache capacity for Schema is 2
>> DEBUG [COMMIT-LOG-WRITER] 2011-10-20 10:04:37,555 CommitLog.java (line
>> 446) discard completed log segments for
>> ReplayPosition(segmentId=1319097494650, position=20900), column family 3.
>> DEBUG [CompactionExecutor:7] 2011-10-20 10:04:37,555
>> CompactionManager.java (line 124) Checking to see if compaction of Schema
>> would be useful
>> DEBUG [COMMIT-LOG-WRITER] 2011-10-20 10:04:37,555 CommitLog.java (line
>> 486) Not safe to delete commit log
>> CommitLogSegment(/var/lib/cassandra/commitlog/CommitLog-1319097494650.log);
>> dirty is ; hasNext: false
>> DEBUG [MigrationStage:1] 2011-10-20 10:04:37,556 ColumnFamilyStore.java
>> (line 264) Starting CFS a1307
>> DEBUG [MigrationStage:1] 2011-10-20 10:04:37,557 AutoSavingCache.java
>> (line 175) KeyCache capacity for a1307 is 200000
>> DEBUG [MigrationStage:1] 2011-10-20 10:04:37,557 MigrationManager.java
>> (line 137) Gossiping my schema version 21e5c1d0-faf2-11e0-0000-1584a17080bf
>> DEBUG [MutationStage:34] 2011-10-20 10:04:47,503
>> RowMutationVerbHandler.java (line 48) Applying
>> RowMutation(keyspace='MyKeyspace', key='6131333034',
>> modifications=[ColumnFamily(a1307 [User:false:3@1319097867943000,])])
>> DEBUG [MutationStage:34] 2011-10-20 10:04:47,504 Table.java (line 398)
>> applying mutation of row 6131333034
>> DEBUG [MutationStage:34] 2011-10-20 10:04:47,506
>> RowMutationVerbHandler.java (line 77) RowMutation(keyspace='MyKeyspace',
>> key='6131333034', modifications=[ColumnFamily(a1307
>> [User:false:3@1319097867943000,])]) applied.  Sending response to 1490@/
>> 192.168.1.30
>> DEBUG [MutationStage:35] 2011-10-20 10:04:47,544
>> RowMutationVerbHandler.java (line 48) Applying
>> RowMutation(keyspace='MyKeyspace', key='6131333034',
>> modifications=[ColumnFamily(a1307 [Group:false:3@1319097878019000,])])
>> DEBUG [MutationStage:35] 2011-10-20 10:04:47,546 Table.java (line 398)
>> applying mutation of row 6131333034
>> DEBUG [MutationStage:35] 2011-10-20 10:04:47,546
>> RowMutationVerbHandler.java (line 77) RowMutation(keyspace='MyKeyspace',
>> key='6131333034', modifications=[ColumnFamily(a1307
>> [Group:false:3@1319097878019000,])]) applied.  Sending response to 1491@/
>> 192.168.1.30
>> DEBUG [MutationStage:36] 2011-10-20 10:04:47,553
>> RowMutationVerbHandler.java (line 48) Applying
>> RowMutation(keyspace='MyKeyspace', key='6131333035',
>> modifications=[ColumnFamily(a1307 [RegContent:false:5@1319097878059000
>> ,])])
>> DEBUG [MutationStage:36] 2011-10-20 10:04:47,554 Table.java (line 398)
>> applying mutation of row 6131333035
>> DEBUG [MutationStage:36] 2011-10-20 10:04:47,554
>> RowMutationVerbHandler.java (line 77) RowMutation(keyspace='MyKeyspace',
>> key='6131333035', modifications=[ColumnFamily(a1307
>> [RegContent:false:5@1319097878059000,])]) applied.  Sending response to
>> 1492@/192.168.1.30
>> DEBUG [MutationStage:37] 2011-10-20 10:04:47,558
>> RowMutationVerbHandler.java (line 48) Applying
>> RowMutation(keyspace='MyKeyspace', key='6131333035',
>> modifications=[ColumnFamily(a1307 [RegGroup:false:5@1319097878065000,])])
>> DEBUG [MutationStage:37] 2011-10-20 10:04:47,558 Table.java (line 398)
>> applying mutation of row 6131333035
>> DEBUG [MutationStage:37] 2011-10-20 10:04:47,559
>> RowMutationVerbHandler.java (line 77) RowMutation(keyspace='MyKeyspace',
>> key='6131333035', modifications=[ColumnFamily(a1307
>> [RegGroup:false:5@1319097878065000,])]) applied.  Sending response to
>> 1493@/192.168.1.30
>> DEBUG [MutationStage:38] 2011-10-20 10:04:47,563
>> RowMutationVerbHandler.java (line 48) Applying
>> RowMutation(keyspace='MyKeyspace', key='6131333035',
>> modifications=[ColumnFamily(a1307 [RegUser:false:5@1319097878070000,])])
>> DEBUG [MutationStage:38] 2011-10-20 10:04:47,563 Table.java (line 398)
>> applying mutation of row 6131333035
>> DEBUG [MutationStage:38] 2011-10-20 10:04:47,563
>> RowMutationVerbHandler.java (line 77) RowMutation(keyspace='MyKeyspace',
>> key='6131333035', modifications=[ColumnFamily(a1307
>> [RegUser:false:5@1319097878070000,])]) applied.  Sending response to
>> 1494@/192.168.1.30
>> DEBUG [MutationStage:39] 2011-10-20 10:04:47,567
>> RowMutationVerbHandler.java (line 48) Applying
>> RowMutation(keyspace='MyKeyspace', key='6131333035',
>> modifications=[ColumnFamily(a1307 [RegContentSched:false:5@1319097878074000
>> ,])])
>> DEBUG [MutationStage:39] 2011-10-20 10:04:47,567 Table.java (line 398)
>> applying mutation of row 6131333035
>> DEBUG [MutationStage:39] 2011-10-20 10:04:47,567
>> RowMutationVerbHandler.java (line 77) RowMutation(keyspace='MyKeyspace',
>> key='6131333035', modifications=[ColumnFamily(a1307
>> [RegContentSched:false:5@1319097878074000,])]) applied.  Sending response
>> to 1495@/192.168.1.30
>> DEBUG [MutationStage:40] 2011-10-20 10:04:47,571
>> RowMutationVerbHandler.java (line 48) Applying
>> RowMutation(keyspace='MyKeyspace', key='6131333035',
>> modifications=[ColumnFamily(a1307 [ActiveEntity:false:5@1319097878078000
>> ,])])
>> DEBUG [MutationStage:40] 2011-10-20 10:04:47,571 Table.java (line 398)
>> applying mutation of row 6131333035
>> DEBUG [MutationStage:40] 2011-10-20 10:04:47,571
>> RowMutationVerbHandler.java (line 77) RowMutation(keyspace='MyKeyspace',
>> key='6131333035', modifications=[ColumnFamily(a1307
>> [ActiveEntity:false:5@1319097878078000,])]) applied.  Sending response to
>> 1496@/192.168.1.30
>> DEBUG [MutationStage:41] 2011-10-20 10:04:47,574
>> RowMutationVerbHandler.java (line 48) Applying
>> RowMutation(keyspace='MyKeyspace', key='6131333035',
>> modifications=[ColumnFamily(a1307 [InactiveEntity:false:5@1319097878082000
>> ,])])
>> DEBUG [MutationStage:41] 2011-10-20 10:04:47,575 Table.java (line 398)
>> applying mutation of row 6131333035
>> DEBUG [MutationStage:41] 2011-10-20 10:04:47,575
>> RowMutationVerbHandler.java (line 77) RowMutation(keyspace='MyKeyspace',
>> key='6131333035', modifications=[ColumnFamily(a1307
>> [InactiveEntity:false:5@1319097878082000,])]) applied.  Sending response
>> to 1497@/192.168.1.30
>> DEBUG [MutationStage:42] 2011-10-20 10:04:47,579
>> RowMutationVerbHandler.java (line 48) Applying
>> RowMutation(keyspace='MyKeyspace', key='6131333036',
>> modifications=[ColumnFamily(a1307 [Friend List:false:5@1319097878087000
>> ,])])
>> DEBUG [MutationStage:42] 2011-10-20 10:04:47,579 Table.java (line 398)
>> applying mutation of row 6131333036
>> DEBUG [MutationStage:42] 2011-10-20 10:04:47,580
>> RowMutationVerbHandler.java (line 77) RowMutation(keyspace='MyKeyspace',
>> key='6131333036', modifications=[ColumnFamily(a1307 [Friend
>> List:false:5@1319097878087000,])]) applied.  Sending response to 1498@/
>> 192.168.1.30
>> DEBUG [MutationStage:43] 2011-10-20 10:04:47,582
>> RowMutationVerbHandler.java (line 48) Applying
>> RowMutation(keyspace='MyKeyspace', key='6131333036',
>> modifications=[ColumnFamily(a1307 [Profile Items:false:5@1319097878090000
>> ,])])
>> DEBUG [MutationStage:43] 2011-10-20 10:04:47,583 Table.java (line 398)
>> applying mutation of row 6131333036
>> DEBUG [MutationStage:43] 2011-10-20 10:04:47,583
>> RowMutationVerbHandler.java (line 77) RowMutation(keyspace='MyKeyspace',
>> key='6131333036', modifications=[ColumnFamily(a1307 [Profile
>> Items:false:5@1319097878090000,])]) applied.  Sending response to 1499@/
>> 192.168.1.30
>> DEBUG [MutationStage:44] 2011-10-20 10:04:47,586
>> RowMutationVerbHandler.java (line 48) Applying
>> RowMutation(keyspace='MyKeyspace', key='6131333036',
>> modifications=[ColumnFamily(a1307 [Likes:false:5@1319097878094000,])])
>> DEBUG [MutationStage:44] 2011-10-20 10:04:47,586 Table.java (line 398)
>> applying mutation of row 6131333036
>> DEBUG [MutationStage:44] 2011-10-20 10:04:47,587
>> RowMutationVerbHandler.java (line 77) RowMutation(keyspace='MyKeyspace',
>> key='6131333036', modifications=[ColumnFamily(a1307
>> [Likes:false:5@1319097878094000,])]) applied.  Sending response to 1500@/
>> 192.168.1.30
>> DEBUG [MutationStage:45] 2011-10-20 10:04:47,590
>> RowMutationVerbHandler.java (line 48) Applying
>> RowMutation(keyspace='MyKeyspace', key='6131333036',
>> modifications=[ColumnFamily(a1307 [Groups:false:5@1319097878097000,])])
>> DEBUG [MutationStage:45] 2011-10-20 10:04:47,590 Table.java (line 398)
>> applying mutation of row 6131333036
>> DEBUG [MutationStage:45] 2011-10-20 10:04:47,591
>> RowMutationVerbHandler.java (line 77) RowMutation(keyspace='MyKeyspace',
>> key='6131333036', modifications=[ColumnFamily(a1307
>> [Groups:false:5@1319097878097000,])]) applied.  Sending response to 1501@
>> /192.168.1.30
>> DEBUG [MutationStage:46] 2011-10-20 10:04:47,594
>> RowMutationVerbHandler.java (line 48) Applying
>> RowMutation(keyspace='MyKeyspace', key='6131333036',
>> modifications=[ColumnFamily(a1307 [Content:false:5@1319097878102000,])])
>> DEBUG [MutationStage:46] 2011-10-20 10:04:47,595 Table.java (line 398)
>> applying mutation of row 6131333036
>> DEBUG [MutationStage:46] 2011-10-20 10:04:47,595
>> RowMutationVerbHandler.java (line 77) RowMutation(keyspace='MyKeyspace',
>> key='6131333036', modifications=[ColumnFamily(a1307
>> [Content:false:5@1319097878102000,])]) applied.  Sending response to
>> 1502@/192.168.1.30
>> DEBUG [MutationStage:47] 2011-10-20 10:04:47,598
>> RowMutationVerbHandler.java (line 48) Applying
>> RowMutation(keyspace='MyKeyspace', key='6131333036',
>> modifications=[ColumnFamily(a1307 [Events:false:5@1319097878106000,])])
>> DEBUG [MutationStage:47] 2011-10-20 10:04:47,599 Table.java (line 398)
>> applying mutation of row 6131333036
>> DEBUG [MutationStage:47] 2011-10-20 10:04:47,599
>> RowMutationVerbHandler.java (line 77) RowMutation(keyspace='MyKeyspace',
>> key='6131333036', modifications=[ColumnFamily(a1307
>> [Events:false:5@1319097878106000,])]) applied.  Sending response to 1503@
>> /192.168.1.30
>> DEBUG [MutationStage:48] 2011-10-20 10:04:47,602
>> RowMutationVerbHandler.java (line 48) Applying
>> RowMutation(keyspace='MyKeyspace', key='6131333036',
>> modifications=[ColumnFamily(a1307 [Bespoke
>> Containers:false:5@1319097878110000,])])
>> DEBUG [MutationStage:48] 2011-10-20 10:04:47,602 Table.java (line 398)
>> applying mutation of row 6131333036
>> DEBUG [MutationStage:48] 2011-10-20 10:04:47,603
>> RowMutationVerbHandler.java (line 77) RowMutation(keyspace='MyKeyspace',
>> key='6131333036', modifications=[ColumnFamily(a1307 [Bespoke
>> Containers:false:5@1319097878110000,])]) applied.  Sending response to
>> 1504@/192.168.1.30
>> DEBUG [MutationStage:49] 2011-10-20 10:04:47,605
>> RowMutationVerbHandler.java (line 48) Applying
>> RowMutation(keyspace='MyKeyspace', key='6131323231',
>> modifications=[ColumnFamily(a1307 [Entity ID Field:false:5@1319097878113000
>> ,])])
>> DEBUG [MutationStage:49] 2011-10-20 10:04:47,606 Table.java (line 398)
>> applying mutation of row 6131323231
>> DEBUG [MutationStage:49] 2011-10-20 10:04:47,606
>> RowMutationVerbHandler.java (line 77) RowMutation(keyspace='MyKeyspace',
>> key='6131323231', modifications=[ColumnFamily(a1307 [Entity ID
>> Field:false:5@1319097878113000,])]) applied.  Sending response to 1505@/
>> 192.168.1.30
>> DEBUG [MutationStage:50] 2011-10-20 10:04:47,609
>> RowMutationVerbHandler.java (line 48) Applying
>> RowMutation(keyspace='MyKeyspace', key='6131323231',
>> modifications=[ColumnFamily(a1307 [Entity Type
>> Field:false:5@1319097878117000,])])
>> DEBUG [MutationStage:50] 2011-10-20 10:04:47,609 Table.java (line 398)
>> applying mutation of row 6131323231
>> DEBUG [MutationStage:50] 2011-10-20 10:04:47,610
>> RowMutationVerbHandler.java (line 77) RowMutation(keyspace='MyKeyspace',
>> key='6131323231', modifications=[ColumnFamily(a1307 [Entity Type
>> Field:false:5@1319097878117000,])]) applied.  Sending response to 1506@/
>> 192.168.1.30
>> DEBUG [MigrationStage:1] 2011-10-20 10:04:47,612
>> SchemaCheckVerbHandler.java (line 36) Received schema check request.
>> DEBUG [MigrationStage:1] 2011-10-20 10:04:47,655
>> SchemaCheckVerbHandler.java (line 36) Received schema check request.
>> DEBUG [MigrationStage:1] 2011-10-20 10:04:47,671
>> DefinitionsUpdateVerbHandler.java (line 70) Applying AddColumnFamily from /
>> 192.168.1.30
>> DEBUG [MigrationStage:1] 2011-10-20 10:04:47,671 Table.java (line 398)
>> applying mutation of row
>> 32376666636361302d666166322d313165302d303030302d313538346131373038306266
>> DEBUG [MigrationStage:1] 2011-10-20 10:04:47,673 Table.java (line 398)
>> applying mutation of row 4d6967726174696f6e73204b6579
>>  I*NFO [MigrationStage:1] 2011-10-20 <1%5D%202011-10-20> 10:04:47,673
>> Migration.java (line 116) Applying migration
>> 27ffcca0-faf2-11e0-0000-1584a17080bf Add column family:
>> org.apache.cassandra.config.CFMetaData@33a998c7
>> [cfId=1001,ksName=MyKeyspace,cfName=a1101,cfType=Standard,comparator=org.apache.cassandra.db.marshal.AsciiType,subcolumncomparator=<null>,comment=,rowCacheSize=0.0,keyCacheSize=200000.0,readRepairChance=1.0,replicateOnWrite=true,gcGraceSeconds=864000,defaultValidator=org.apache.cassandra.db.marshal.BytesType,keyValidator=org.apache.cassandra.db.marshal.BytesType,minCompactionThreshold=4,maxCompactionThreshold=32,rowCacheSavePeriodInSeconds=0,keyCacheSavePeriodInSeconds=14400,memtableFlushAfterMins=1440,memtableThroughputInMb=55,memtableOperationsInMillions=0.2578125,mergeShardsChance=0.1,keyAlias=<null>,column_metadata={}]
>> *
>> DEBUG [MigrationStage:1] 2011-10-20 10:04:47,673 Table.java (line 398)
>> applying mutation of row 4c617374204d6967726174696f6e
>> DEBUG [MigrationStage:1] 2011-10-20 10:04:47,674 ColumnFamilyStore.java
>> (line 765) flush position is ReplayPosition(segmentId=1319097494650,
>> position=33582)
>>  INFO [MigrationStage:1] 2011-10-20 10:04:47,674 ColumnFamilyStore.java
>> (line 1128) Enqueuing flush of Memtable-Migrations@1131236176(7476/9345
>> serialized/live bytes, 1 ops)
>> DEBUG [MigrationStage:1] 2011-10-20 10:04:47,674 ColumnFamilyStore.java
>> (line 765) flush position is ReplayPosition(segmentId=1319097494650,
>> position=33582)
>>  INFO [FlushWriter:3] 2011-10-20 10:04:47,675 Memtable.java (line 237)
>> Writing Memtable-Migrations@1131236176(7476/9345 serialized/live bytes, 1
>> ops)
>>  INFO [MigrationStage:1] 2011-10-20 10:04:47,676 ColumnFamilyStore.java
>> (line 1128) Enqueuing flush of Memtable-Schema@1013683141(3283/4103
>> serialized/live bytes, 3 ops)
>> DEBUG [FlushWriter:3] 2011-10-20 10:04:47,676 DatabaseDescriptor.java
>> (line 821) expected data files size is 9004; largest free partition has
>> 88471822336 bytes free
>>  INFO [FlushWriter:3] 2011-10-20 10:04:47,725 Memtable.java (line 254)
>> Completed flushing /var/lib/cassandra/data/system/Migrations-g-3-Data.db
>> (7540 bytes)
>> DEBUG [FlushWriter:3] 2011-10-20 10:04:47,725 DataTracker.java (line 264)
>> adding /var/lib/cassandra/data/system/Migrations-g-3 to list of files
>> tracked for system.Migrations
>> DEBUG [FlushWriter:3] 2011-10-20 10:04:47,726 AutoSavingCache.java (line
>> 175) KeyCache capacity for Migrations is 3
>> DEBUG [CompactionExecutor:4] 2011-10-20 10:04:47,726
>> CompactionManager.java (line 124) Checking to see if compaction of
>> Migrations would be useful
>>  INFO [FlushWriter:3] 2011-10-20 10:04:47,726 Memtable.java (line 237)
>> Writing Memtable-Schema@1013683141(3283/4103 serialized/live bytes, 3
>> ops)
>> DEBUG [COMMIT-LOG-WRITER] 2011-10-20 10:04:47,726 CommitLog.java (line
>> 446) discard completed log segments for
>> ReplayPosition(segmentId=1319097494650, position=33582), column family 2.
>> DEBUG [COMMIT-LOG-WRITER] 2011-10-20 10:04:47,727 CommitLog.java (line
>> 486) Not safe to delete commit log
>> CommitLogSegment(/var/lib/cassandra/commitlog/CommitLog-1319097494650.log);
>> dirty is a1307 (1000), Schema (3), ; hasNext: false
>> DEBUG [FlushWriter:3] 2011-10-20 10:04:47,727 DatabaseDescriptor.java
>> (line 821) expected data files size is 4059; largest free partition has
>> 88471797760 bytes free
>>  INFO [FlushWriter:3] 2011-10-20 10:04:47,751 Memtable.java (line 254)
>> Completed flushing /var/lib/cassandra/data/system/Schema-g-3-Data.db (3433
>> bytes)
>> DEBUG [FlushWriter:3] 2011-10-20 10:04:47,752 DataTracker.java (line 264)
>> adding /var/lib/cassandra/data/system/Schema-g-3 to list of files tracked
>> for system.Schema
>> DEBUG [FlushWriter:3] 2011-10-20 10:04:47,752 AutoSavingCache.java (line
>> 175) KeyCache capacity for Schema is 3
>> DEBUG [CompactionExecutor:5] 2011-10-20 10:04:47,752
>> CompactionManager.java (line 124) Checking to see if compaction of Schema
>> would be useful
>> DEBUG [COMMIT-LOG-WRITER] 2011-10-20 10:04:47,752 CommitLog.java (line
>> 446) discard completed log segments for
>> ReplayPosition(segmentId=1319097494650, position=33582), column family 3.
>> DEBUG [COMMIT-LOG-WRITER] 2011-10-20 10:04:47,753 CommitLog.java (line
>> 486) Not safe to delete commit log
>> CommitLogSegment(/var/lib/cassandra/commitlog/CommitLog-1319097494650.log);
>> dirty is a1307 (1000), ; hasNext: false
>>
>>
>> --
>> Renato Bacelar da Silveira
>> Software Engineer (Java J2SE J2EE)
>>
>
>
>
>  --
> Jérémy
>
>
>


-- 
Jérémy

Re: Column Family not able to be replicated between 2 nodes

Posted by Renato Baceolar da Silveira <re...@indabamobile.co.za>.

Thank You Jeremy

Will report back to you with any changes!

Regards

On 20/10/2011 11:15, Jérémy SEVELLEC wrote:
> Hi,
>
> I think it's because you create a new columnFamily with hector and you 
> don't wait the propagation of this modification in the cluster before 
> writing.
>
> Hector provide a method witjh a boolean to say "blockUntilComplete" 
> when you create a columnFamily :
> cluster.addColumnFamily(<your cf definition>, true);
>
> Here is the hector javadoc about that : 
> http://rantav.github.com/hector//source/content/API/core/0.8.0-2/me/prettyprint/hector/api/Cluster.html#addColumnFamily(me.prettyprint.hector.api.ddl.ColumnFamilyDefinition, 
> boolean) 
> <http://rantav.github.com/hector//source/content/API/core/0.8.0-2/me/prettyprint/hector/api/Cluster.html#addColumnFamily%28me.prettyprint.hector.api.ddl.ColumnFamilyDefinition,%20boolean%29>
>
> Regards,
>
> Jérémy
>
> 2011/10/20 Renato Baceolar da Silveira <renatods@indabamobile.co.za 
> <ma...@indabamobile.co.za>>
>
>     Hi All and greetings from South Africa, a country with a/*sad
>     Rugby team :(*/
>
>     I am processing a batch operation, that inserts a number of column
>     families.
>
>     This batch operation is executed in a cluster with 2 nodes and
>     replication of 2.
>
>     EACH NODE is installed in a  VMWARE ESSI virtual box, in a Ubuntu 9
>     server 64 bit. Both Nodes live in the same physical server, and
>     communication between
>     them is through shared hardware.
>
>      The virtual machines have 4gb of ram, and plenty hdd
>     space.
>
>     The version of Cassandra is 0.8.5.
>
>     Both nodes start empty before this batch process commences.
>
>     At the insertion of the first Column Family I receive this error
>     in the log, which
>     tells me that for a number of times the specific column family is
>     Replayed
>     for insertion, but it seems not to succeed. Without the insertion
>     of this
>     first Column Family our validation process fails the batch process.
>
>     Can any one perhaps shed some light on this matter?
>
>
>     Kind Regards
>
>     Renato da Silveira
>
>     ===============
>     error log - Applicaion Process
>     ===============
>
>     me.prettyprint.hector.api.
>     exceptions.HectorException: SchemaDisagreementException()
>         at
>     me.prettyprint.cassandra.service.ExceptionsTranslatorImpl.translate(ExceptionsTranslatorImpl.java:68)
>         at
>     me.prettyprint.cassandra.service.ThriftCluster$4.execute(ThriftCluster.java:82)
>         at
>     me.prettyprint.cassandra.service.ThriftCluster$4.execute(ThriftCluster.java:76)
>         at
>     me.prettyprint.cassandra.service.Operation.executeAndSetResult(Operation.java:101)
>         at
>     me.prettyprint.cassandra.connection.HConnectionManager.operateWithFailover(HConnectionManager.java:219)
>         at
>     me.prettyprint.cassandra.service.ThriftCluster.addColumnFamily(ThriftCluster.java:86)
>         at
>     com.xxx.xxx.hector.HectorManager.addColumnFamily(HectorManager.java:132)
>         at
>     com.xxx.xxx.facade.CassandraManager.addColumnFamily(CassandraManager.java:117)
>         at com.xxx.xxx.model.xxx.createRegistryContainer(xxx.java:63)
>         at com.xxx.xxx.model.xxx.buildRegistry(xxx.java:120)
>         at
>     com.xxx.xxx.facade.Process.articleLoaderDoInstance(Process.java:117)
>         at com.xxx.xxx.facade.Process.articleLoader(Process.java:58)
>     Caused by: SchemaDisagreementException()
>         at
>     org.apache.cassandra.thrift.Cassandra$system_add_column_family_result.read(Cassandra.java:26119)
>         at
>     org.apache.cassandra.thrift.Cassandra$Client.recv_system_add_column_family(Cassandra.java:1455)
>         at
>     org.apache.cassandra.thrift.Cassandra$Client.system_add_column_family(Cassandra.java:1430)
>         at
>     me.prettyprint.cassandra.service.ThriftCluster$4.execute(ThriftCluster.java:80)
>         ... 10 more
>
>
>     ===============
>     error log - Cassandra Log
>     ===============
>     INFO [MigrationStage:1] 2011-10-20 <tel:1%5D%202011-10-20>
>     10:03:58,944 Migration.java (line 116) Applying migration
>     0aadb220-faf2-11e0-0000-1584a17080bf Add keyspace: MyKeyspace, rep
>     strategy:SimpleStrategy{}, durable_writes: true
>     DEBUG [MigrationStage:1] 2011-10-20 <tel:1%5D%202011-10-20>
>     10:03:58,945 Table.java (line 398) applying mutation of row
>     4c617374204d6967726174696f6e
>     DEBUG [MigrationStage:1] 2011-10-20 <tel:1%5D%202011-10-20>
>     10:03:58,945 ColumnFamilyStore.java (line 765) flush position is
>     ReplayPosition(segmentId=1319097494650, position=10486)
>      INFO [MigrationStage:1] 2011-10-20 <tel:1%5D%202011-10-20>
>     10:03:58,945 ColumnFamilyStore.java (line 1128) Enqueuing flush of
>     Memtable-Migrations@539124954 <tel:539124954>(6675/8343
>     serialized/live bytes, 1 ops)
>     DEBUG [MigrationStage:1] 2011-10-20 <tel:1%5D%202011-10-20>
>     10:03:58,946 ColumnFamilyStore.java (line 765) flush position is
>     ReplayPosition(segmentId=1319097494650, position=10486)
>      INFO [FlushWriter:3] 2011-10-20 <tel:3%5D%202011-10-20>
>     10:03:58,946 Memtable.java (line 237) Writing
>     Memtable-Migrations@539124954 <tel:539124954>(6675/8343
>     serialized/live bytes, 1 ops)
>      INFO [MigrationStage:1] 2011-10-20 <tel:1%5D%202011-10-20>
>     10:03:58,946 ColumnFamilyStore.java (line 1128) Enqueuing flush of
>     Memtable-Schema@1225252094(2696/3370 serialized/live bytes, 3 ops)
>     DEBUG [FlushWriter:3] 2011-10-20 <tel:3%5D%202011-10-20>
>     10:03:58,947 DatabaseDescriptor.java (line 821) expected data
>     files size is 8043; largest free partition has 88471957504 bytes free
>      INFO [FlushWriter:3] 2011-10-20 <tel:3%5D%202011-10-20>
>     10:03:58,991 Memtable.java (line 254) Completed flushing
>     /var/lib/cassandra/data/system/Migrations-g-1-Data.db (6739 bytes)
>     DEBUG [FlushWriter:3] 2011-10-20 <tel:3%5D%202011-10-20>
>     10:03:58,991 DataTracker.java (line 264) adding
>     /var/lib/cassandra/data/system/Migrations-g-1 to list of files
>     tracked for system.Migrations
>     DEBUG [COMMIT-LOG-WRITER] 2011-10-20 10:03:58,992 CommitLog.java
>     (line 446) discard completed log segments for
>     ReplayPosition(segmentId=1319097494650, position=10486), column
>     family 2.
>      INFO [FlushWriter:3] 2011-10-20 <tel:3%5D%202011-10-20>
>     10:03:58,992 Memtable.java (line 237) Writing
>     Memtable-Schema@1225252094(2696/3370 serialized/live bytes, 3 ops)
>     DEBUG [COMMIT-LOG-WRITER] 2011-10-20 10:03:58,992 CommitLog.java
>     (line 486) Not safe to delete commit log
>     CommitLogSegment(/var/lib/cassandra/commitlog/CommitLog-1319097494650.log);
>     dirty is Schema (3), ; hasNext: false
>     DEBUG [CompactionExecutor:4] 2011-10-20 <tel:4%5D%202011-10-20>
>     10:03:58,992 CompactionManager.java (line 124) Checking to see if
>     compaction of Migrations would be useful
>     DEBUG [FlushWriter:3] 2011-10-20 <tel:3%5D%202011-10-20>
>     10:03:58,993 DatabaseDescriptor.java (line 821) expected data
>     files size is 3355; largest free partition has 88471932928 bytes free
>      INFO [FlushWriter:3] 2011-10-20 <tel:3%5D%202011-10-20>
>     10:03:59,026 Memtable.java (line 254) Completed flushing
>     /var/lib/cassandra/data/system/Schema-g-1-Data.db (2846 bytes)
>     DEBUG [FlushWriter:3] 2011-10-20 <tel:3%5D%202011-10-20>
>     10:03:59,026 DataTracker.java (line 264) adding
>     /var/lib/cassandra/data/system/Schema-g-1 to list of files tracked
>     for system.Schema
>     DEBUG [COMMIT-LOG-WRITER] 2011-10-20 10:03:59,027 CommitLog.java
>     (line 446) discard completed log segments for
>     ReplayPosition(segmentId=1319097494650, position=10486), column
>     family 3.
>     DEBUG [CompactionExecutor:5] 2011-10-20 <tel:5%5D%202011-10-20>
>     10:03:59,027 CompactionManager.java (line 124) Checking to see if
>     compaction of Schema would be useful
>     DEBUG [COMMIT-LOG-WRITER] 2011-10-20 10:03:59,027 CommitLog.java
>     (line 486) Not safe to delete commit log
>     CommitLogSegment(/var/lib/cassandra/commitlog/CommitLog-1319097494650.log);
>     dirty is ; hasNext: false
>     DEBUG [MigrationStage:1] 2011-10-20 <tel:1%5D%202011-10-20>
>     10:03:59,028 MigrationManager.java (line 137) Gossiping my schema
>     version 0aadb220-faf2-11e0-0000-1584a17080bf
>     DEBUG [MigrationStage:1] 2011-10-20 <tel:1%5D%202011-10-20>
>     10:03:59,029 SchemaCheckVerbHandler.java (line 36) Received schema
>     check request.
>     DEBUG [ScheduledTasks:1] 2011-10-20 <tel:1%5D%202011-10-20>
>     10:04:16,797 StorageLoadBalancer.java (line 336) Disseminating
>     load info ...
>     DEBUG [MigrationStage:1] 2011-10-20 <tel:1%5D%202011-10-20>
>     10:04:37,373 SchemaCheckVerbHandler.java (line 36) Received schema
>     check request.
>     *DEBUG [MigrationStage:1] 2011-10-20 <tel:1%5D%202011-10-20>
>     10:04:37,444 DefinitionsUpdateVerbHandler.java (line 70) Applying
>     AddColumnFamily from /192.168.1.30 <http://192.168.1.30>*
>     DEBUG [MigrationStage:1] 2011-10-20 <tel:1%5D%202011-10-20>
>     10:04:37,444 Table.java (line 398) applying mutation of row
>     32316535633164302d666166322d313165302d303030302d313538346131373038306266
>     DEBUG [MigrationStage:1] 2011-10-20 <tel:1%5D%202011-10-20>
>     10:04:37,447 Table.java (line 398) applying mutation of row
>     4d6967726174696f6e73204b6579
>      INFO [MigrationStage:1] 2011-10-20 <tel:1%5D%202011-10-20>
>     10:04:37,456 Migration.java (line 116)*Applying migration
>     21e5c1d0-faf2-11e0-0000-1584a17080bf Add column family:
>     org.apache.cassandra.config.CFMetaData@7f1fc4b8[cfId=1000,ksName=MyKeyspace,cfName=a1307,cfType=Standard,comparator=org.apache.cassandra.db.marshal.AsciiType,subcolumncomparator=<null>,comment=,rowCacheSize=0.0,keyCacheSize=200000.0,readRepairChance=1.0,replicateOnWrite=true,gcGraceSeconds=864000,defaultValidator=org.apache.cassandra.db.marshal.BytesType,keyValidator=org.apache.cassandra.db.marshal.BytesType,minCompactionThreshold=4,maxCompactionThreshold=32,rowCacheSavePeriodInSeconds=0,keyCacheSavePeriodInSeconds=14400,memtableFlushAfterMins=1440,memtableThroughputInMb=55,memtableOperationsInMillions=0.2578125,mergeShardsChance=0.1,keyAlias=<null>,column_metadata={}]*
>     DEBUG [MigrationStage:1] 2011-10-20 <tel:1%5D%202011-10-20>
>     10:04:37,456 Table.java (line 398) applying mutation of row
>     4c617374204d6967726174696f6e
>     *DEBUG [MigrationStage:1] 2011-10-20 <tel:1%5D%202011-10-20>
>     10:04:37,456 ColumnFamilyStore.java (line 765) flush position is
>     ReplayPosition(segmentId=1319097494650, position=20900)*
>      INFO [MigrationStage:1] 2011-10-20 <tel:1%5D%202011-10-20>
>     10:04:37,457 ColumnFamilyStore.java (line 1128) Enqueuing flush of
>     Memtable-Migrations@2125221179(7183/8978 serialized/live bytes, 1 ops)
>     DEBUG [MigrationStage:1] 2011-10-20 <tel:1%5D%202011-10-20>
>     10:04:37,457 ColumnFamilyStore.java (line 765) flush position is
>     ReplayPosition(segmentId=1319097494650, position=20900)
>      INFO [FlushWriter:3] 2011-10-20 <tel:3%5D%202011-10-20>
>     10:04:37,457 Memtable.java (line 237) Writing
>     Memtable-Migrations@2125221179(7183/8978 serialized/live bytes, 1 ops)
>      INFO [MigrationStage:1] 2011-10-20 <tel:1%5D%202011-10-20>
>     10:04:37,458 ColumnFamilyStore.java (line 1128) Enqueuing flush of
>     Memtable-Schema@1644338549(2990/3737 serialized/live bytes, 3 ops)
>     DEBUG [FlushWriter:3] 2011-10-20 <tel:3%5D%202011-10-20>
>     10:04:37,459 DatabaseDescriptor.java (line 821) expected data
>     files size is 8653; largest free partition has 88471896064 bytes free
>      INFO [FlushWriter:3] 2011-10-20 <tel:3%5D%202011-10-20>
>     10:04:37,513 Memtable.java (line 254) Completed flushing
>     /var/lib/cassandra/data/system/Migrations-g-2-Data.db (7247 bytes)
>     DEBUG [FlushWriter:3] 2011-10-20 <tel:3%5D%202011-10-20>
>     10:04:37,514 DataTracker.java (line 264) adding
>     /var/lib/cassandra/data/system/Migrations-g-2 to list of files
>     tracked for system.Migrations
>     DEBUG [FlushWriter:3] 2011-10-20 <tel:3%5D%202011-10-20>
>     10:04:37,514 AutoSavingCache.java (line 175) KeyCache capacity for
>     Migrations is 2
>     DEBUG [COMMIT-LOG-WRITER] 2011-10-20 10:04:37,514 CommitLog.java
>     (line 446) discard completed log segments for
>     ReplayPosition(segmentId=1319097494650, position=20900), column
>     family 2.
>      INFO [FlushWriter:3] 2011-10-20 <tel:3%5D%202011-10-20>
>     10:04:37,515 Memtable.java (line 237) Writing
>     Memtable-Schema@1644338549(2990/3737 serialized/live bytes, 3 ops)
>     DEBUG [CompactionExecutor:6] 2011-10-20 <tel:6%5D%202011-10-20>
>     10:04:37,515 CompactionManager.java (line 124) Checking to see if
>     compaction of Migrations would be useful
>     DEBUG [COMMIT-LOG-WRITER] 2011-10-20 10:04:37,515 CommitLog.java
>     (line 486) Not safe to delete commit log
>     CommitLogSegment(/var/lib/cassandra/commitlog/CommitLog-1319097494650.log);
>     dirty is Schema (3), ; hasNext: false
>      INFO [FlushWriter:3] 2011-10-20 <tel:3%5D%202011-10-20>
>     10:04:37,515 Memtable.java (line 237) Writing
>     Memtable-Schema@1644338549(2990/3737 serialized/live bytes, 3 ops)
>     DEBUG [CompactionExecutor:6] 2011-10-20 <tel:6%5D%202011-10-20>
>     10:04:37,515 CompactionManager.java (line 124) Checking to see if
>     compaction of Migrations would be useful
>     DEBUG [COMMIT-LOG-WRITER] 2011-10-20 10:04:37,515 CommitLog.java
>     (line 486) Not safe to delete commit log
>     CommitLogSegment(/var/lib/cassandra/commitlog/CommitLog-1319097494650.log);
>     dirty is Schema (3), ; hasNext: false
>     DEBUG [FlushWriter:3] 2011-10-20 <tel:3%5D%202011-10-20>
>     10:04:37,515 DatabaseDescriptor.java (line 821) expected data
>     files size is 3708; largest free partition has 88471871488 bytes free
>     ERROR [MutationStage:33] 2011-10-20 10:04:37,530
>     RowMutationVerbHandler.java (line 82) Error in row mutation
>     org.apache.cassandra.db.UnserializableColumnFamilyException:
>     Couldn't find cfId=1000
>         at
>     org.apache.cassandra.db.ColumnFamilySerializer.deserialize(ColumnFamilySerializer.java:123)
>         at
>     org.apache.cassandra.db.RowMutation$RowMutationSerializer.deserialize(RowMutation.java:392)
>         at
>     org.apache.cassandra.db.RowMutation$RowMutationSerializer.deserialize(RowMutation.java:400)
>         at
>     org.apache.cassandra.db.RowMutation.fromBytes(RowMutation.java:329)
>         at
>     org.apache.cassandra.db.RowMutationVerbHandler.doVerb(RowMutationVerbHandler.java:46)
>         at org.apache.cassandra.net
>     <http://org.apache.cassandra.net>.MessageDeliveryTask.run(MessageDeliveryTask.java:59)
>         at
>     java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>         at
>     java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>         at java.lang.Thread.run(Thread.java:662)
>      INFO [FlushWriter:3] 2011-10-20 <tel:3%5D%202011-10-20>
>     10:04:37,553 Memtable.java (line 254) Completed flushing
>     /var/lib/cassandra/data/system/Schema-g-2-Data.db (3140 bytes)
>     DEBUG [FlushWriter:3] 2011-10-20 <tel:3%5D%202011-10-20>
>     10:04:37,553 DataTracker.java (line 264) adding
>     /var/lib/cassandra/data/system/Schema-g-2 to list of files tracked
>     for system.Schema
>     DEBUG [FlushWriter:3] 2011-10-20 <tel:3%5D%202011-10-20>
>     10:04:37,554 AutoSavingCache.java (line 175) KeyCache capacity for
>     Schema is 2
>     DEBUG [COMMIT-LOG-WRITER] 2011-10-20 10:04:37,555 CommitLog.java
>     (line 446) discard completed log segments for
>     ReplayPosition(segmentId=1319097494650, position=20900), column
>     family 3.
>     DEBUG [CompactionExecutor:7] 2011-10-20 10:04:37,555
>     CompactionManager.java (line 124) Checking to see if compaction of
>     Schema would be useful
>     DEBUG [COMMIT-LOG-WRITER] 2011-10-20 10:04:37,555 CommitLog.java
>     (line 486) Not safe to delete commit log
>     CommitLogSegment(/var/lib/cassandra/commitlog/CommitLog-1319097494650.log);
>     dirty is ; hasNext: false
>     DEBUG [MigrationStage:1] 2011-10-20 <tel:1%5D%202011-10-20>
>     10:04:37,556 ColumnFamilyStore.java (line 264) Starting CFS a1307
>     DEBUG [MigrationStage:1] 2011-10-20 <tel:1%5D%202011-10-20>
>     10:04:37,557 AutoSavingCache.java (line 175) KeyCache capacity for
>     a1307 is 200000
>     DEBUG [MigrationStage:1] 2011-10-20 <tel:1%5D%202011-10-20>
>     10:04:37,557 MigrationManager.java (line 137) Gossiping my schema
>     version 21e5c1d0-faf2-11e0-0000-1584a17080bf
>     DEBUG [MutationStage:34] 2011-10-20 10:04:47,503
>     RowMutationVerbHandler.java (line 48) Applying
>     RowMutation(keyspace='MyKeyspace', key='6131333034',
>     modifications=[ColumnFamily(a1307 [User:false:3@1319097867943000,])])
>     DEBUG [MutationStage:34] 2011-10-20 10:04:47,504 Table.java (line
>     398) applying mutation of row 6131333034
>     DEBUG [MutationStage:34] 2011-10-20 10:04:47,506
>     RowMutationVerbHandler.java (line 77)
>     RowMutation(keyspace='MyKeyspace', key='6131333034',
>     modifications=[ColumnFamily(a1307
>     [User:false:3@1319097867943000,])]) applied.  Sending response to
>     1490@/192.168.1.30 <http://192.168.1.30>
>     DEBUG [MutationStage:35] 2011-10-20 10:04:47,544
>     RowMutationVerbHandler.java (line 48) Applying
>     RowMutation(keyspace='MyKeyspace', key='6131333034',
>     modifications=[ColumnFamily(a1307 [Group:false:3@1319097878019000,])])
>     DEBUG [MutationStage:35] 2011-10-20 10:04:47,546 Table.java (line
>     398) applying mutation of row 6131333034
>     DEBUG [MutationStage:35] 2011-10-20 10:04:47,546
>     RowMutationVerbHandler.java (line 77)
>     RowMutation(keyspace='MyKeyspace', key='6131333034',
>     modifications=[ColumnFamily(a1307
>     [Group:false:3@1319097878019000,])]) applied.  Sending response to
>     1491@/192.168.1.30 <http://192.168.1.30>
>     DEBUG [MutationStage:36] 2011-10-20 10:04:47,553
>     RowMutationVerbHandler.java (line 48) Applying
>     RowMutation(keyspace='MyKeyspace', key='6131333035',
>     modifications=[ColumnFamily(a1307
>     [RegContent:false:5@1319097878059000,])])
>     DEBUG [MutationStage:36] 2011-10-20 10:04:47,554 Table.java (line
>     398) applying mutation of row 6131333035
>     DEBUG [MutationStage:36] 2011-10-20 10:04:47,554
>     RowMutationVerbHandler.java (line 77)
>     RowMutation(keyspace='MyKeyspace', key='6131333035',
>     modifications=[ColumnFamily(a1307
>     [RegContent:false:5@1319097878059000,])]) applied.  Sending
>     response to 1492@/192.168.1.30 <http://192.168.1.30>
>     DEBUG [MutationStage:37] 2011-10-20 10:04:47,558
>     RowMutationVerbHandler.java (line 48) Applying
>     RowMutation(keyspace='MyKeyspace', key='6131333035',
>     modifications=[ColumnFamily(a1307
>     [RegGroup:false:5@1319097878065000,])])
>     DEBUG [MutationStage:37] 2011-10-20 10:04:47,558 Table.java (line
>     398) applying mutation of row 6131333035
>     DEBUG [MutationStage:37] 2011-10-20 10:04:47,559
>     RowMutationVerbHandler.java (line 77)
>     RowMutation(keyspace='MyKeyspace', key='6131333035',
>     modifications=[ColumnFamily(a1307
>     [RegGroup:false:5@1319097878065000,])]) applied.  Sending response
>     to 1493@/192.168.1.30 <http://192.168.1.30>
>     DEBUG [MutationStage:38] 2011-10-20 10:04:47,563
>     RowMutationVerbHandler.java (line 48) Applying
>     RowMutation(keyspace='MyKeyspace', key='6131333035',
>     modifications=[ColumnFamily(a1307
>     [RegUser:false:5@1319097878070000,])])
>     DEBUG [MutationStage:38] 2011-10-20 10:04:47,563 Table.java (line
>     398) applying mutation of row 6131333035
>     DEBUG [MutationStage:38] 2011-10-20 10:04:47,563
>     RowMutationVerbHandler.java (line 77)
>     RowMutation(keyspace='MyKeyspace', key='6131333035',
>     modifications=[ColumnFamily(a1307
>     [RegUser:false:5@1319097878070000,])]) applied.  Sending response
>     to 1494@/192.168.1.30 <http://192.168.1.30>
>     DEBUG [MutationStage:39] 2011-10-20 10:04:47,567
>     RowMutationVerbHandler.java (line 48) Applying
>     RowMutation(keyspace='MyKeyspace', key='6131333035',
>     modifications=[ColumnFamily(a1307
>     [RegContentSched:false:5@1319097878074000,])])
>     DEBUG [MutationStage:39] 2011-10-20 10:04:47,567 Table.java (line
>     398) applying mutation of row 6131333035
>     DEBUG [MutationStage:39] 2011-10-20 10:04:47,567
>     RowMutationVerbHandler.java (line 77)
>     RowMutation(keyspace='MyKeyspace', key='6131333035',
>     modifications=[ColumnFamily(a1307
>     [RegContentSched:false:5@1319097878074000,])]) applied.  Sending
>     response to 1495@/192.168.1.30 <http://192.168.1.30>
>     DEBUG [MutationStage:40] 2011-10-20 10:04:47,571
>     RowMutationVerbHandler.java (line 48) Applying
>     RowMutation(keyspace='MyKeyspace', key='6131333035',
>     modifications=[ColumnFamily(a1307
>     [ActiveEntity:false:5@1319097878078000,])])
>     DEBUG [MutationStage:40] 2011-10-20 10:04:47,571 Table.java (line
>     398) applying mutation of row 6131333035
>     DEBUG [MutationStage:40] 2011-10-20 10:04:47,571
>     RowMutationVerbHandler.java (line 77)
>     RowMutation(keyspace='MyKeyspace', key='6131333035',
>     modifications=[ColumnFamily(a1307
>     [ActiveEntity:false:5@1319097878078000,])]) applied.  Sending
>     response to 1496@/192.168.1.30 <http://192.168.1.30>
>     DEBUG [MutationStage:41] 2011-10-20 10:04:47,574
>     RowMutationVerbHandler.java (line 48) Applying
>     RowMutation(keyspace='MyKeyspace', key='6131333035',
>     modifications=[ColumnFamily(a1307
>     [InactiveEntity:false:5@1319097878082000,])])
>     DEBUG [MutationStage:41] 2011-10-20 10:04:47,575 Table.java (line
>     398) applying mutation of row 6131333035
>     DEBUG [MutationStage:41] 2011-10-20 10:04:47,575
>     RowMutationVerbHandler.java (line 77)
>     RowMutation(keyspace='MyKeyspace', key='6131333035',
>     modifications=[ColumnFamily(a1307
>     [InactiveEntity:false:5@1319097878082000,])]) applied.  Sending
>     response to 1497@/192.168.1.30 <http://192.168.1.30>
>     DEBUG [MutationStage:42] 2011-10-20 10:04:47,579
>     RowMutationVerbHandler.java (line 48) Applying
>     RowMutation(keyspace='MyKeyspace', key='6131333036',
>     modifications=[ColumnFamily(a1307 [Friend
>     List:false:5@1319097878087000,])])
>     DEBUG [MutationStage:42] 2011-10-20 10:04:47,579 Table.java (line
>     398) applying mutation of row 6131333036
>     DEBUG [MutationStage:42] 2011-10-20 10:04:47,580
>     RowMutationVerbHandler.java (line 77)
>     RowMutation(keyspace='MyKeyspace', key='6131333036',
>     modifications=[ColumnFamily(a1307 [Friend
>     List:false:5@1319097878087000,])]) applied.  Sending response to
>     1498@/192.168.1.30 <http://192.168.1.30>
>     DEBUG [MutationStage:43] 2011-10-20 10:04:47,582
>     RowMutationVerbHandler.java (line 48) Applying
>     RowMutation(keyspace='MyKeyspace', key='6131333036',
>     modifications=[ColumnFamily(a1307 [Profile
>     Items:false:5@1319097878090000,])])
>     DEBUG [MutationStage:43] 2011-10-20 10:04:47,583 Table.java (line
>     398) applying mutation of row 6131333036
>     DEBUG [MutationStage:43] 2011-10-20 10:04:47,583
>     RowMutationVerbHandler.java (line 77)
>     RowMutation(keyspace='MyKeyspace', key='6131333036',
>     modifications=[ColumnFamily(a1307 [Profile
>     Items:false:5@1319097878090000,])]) applied.  Sending response to
>     1499@/192.168.1.30 <http://192.168.1.30>
>     DEBUG [MutationStage:44] 2011-10-20 10:04:47,586
>     RowMutationVerbHandler.java (line 48) Applying
>     RowMutation(keyspace='MyKeyspace', key='6131333036',
>     modifications=[ColumnFamily(a1307 [Likes:false:5@1319097878094000,])])
>     DEBUG [MutationStage:44] 2011-10-20 10:04:47,586 Table.java (line
>     398) applying mutation of row 6131333036
>     DEBUG [MutationStage:44] 2011-10-20 10:04:47,587
>     RowMutationVerbHandler.java (line 77)
>     RowMutation(keyspace='MyKeyspace', key='6131333036',
>     modifications=[ColumnFamily(a1307
>     [Likes:false:5@1319097878094000,])]) applied.  Sending response to
>     1500@/192.168.1.30 <http://192.168.1.30>
>     DEBUG [MutationStage:45] 2011-10-20 10:04:47,590
>     RowMutationVerbHandler.java (line 48) Applying
>     RowMutation(keyspace='MyKeyspace', key='6131333036',
>     modifications=[ColumnFamily(a1307
>     [Groups:false:5@1319097878097000,])])
>     DEBUG [MutationStage:45] 2011-10-20 10:04:47,590 Table.java (line
>     398) applying mutation of row 6131333036
>     DEBUG [MutationStage:45] 2011-10-20 10:04:47,591
>     RowMutationVerbHandler.java (line 77)
>     RowMutation(keyspace='MyKeyspace', key='6131333036',
>     modifications=[ColumnFamily(a1307
>     [Groups:false:5@1319097878097000,])]) applied.  Sending response
>     to 1501@/192.168.1.30 <http://192.168.1.30>
>     DEBUG [MutationStage:46] 2011-10-20 10:04:47,594
>     RowMutationVerbHandler.java (line 48) Applying
>     RowMutation(keyspace='MyKeyspace', key='6131333036',
>     modifications=[ColumnFamily(a1307
>     [Content:false:5@1319097878102000,])])
>     DEBUG [MutationStage:46] 2011-10-20 10:04:47,595 Table.java (line
>     398) applying mutation of row 6131333036
>     DEBUG [MutationStage:46] 2011-10-20 10:04:47,595
>     RowMutationVerbHandler.java (line 77)
>     RowMutation(keyspace='MyKeyspace', key='6131333036',
>     modifications=[ColumnFamily(a1307
>     [Content:false:5@1319097878102000,])]) applied.  Sending response
>     to 1502@/192.168.1.30 <http://192.168.1.30>
>     DEBUG [MutationStage:47] 2011-10-20 10:04:47,598
>     RowMutationVerbHandler.java (line 48) Applying
>     RowMutation(keyspace='MyKeyspace', key='6131333036',
>     modifications=[ColumnFamily(a1307
>     [Events:false:5@1319097878106000,])])
>     DEBUG [MutationStage:47] 2011-10-20 10:04:47,599 Table.java (line
>     398) applying mutation of row 6131333036
>     DEBUG [MutationStage:47] 2011-10-20 10:04:47,599
>     RowMutationVerbHandler.java (line 77)
>     RowMutation(keyspace='MyKeyspace', key='6131333036',
>     modifications=[ColumnFamily(a1307
>     [Events:false:5@1319097878106000,])]) applied.  Sending response
>     to 1503@/192.168.1.30 <http://192.168.1.30>
>     DEBUG [MutationStage:48] 2011-10-20 10:04:47,602
>     RowMutationVerbHandler.java (line 48) Applying
>     RowMutation(keyspace='MyKeyspace', key='6131333036',
>     modifications=[ColumnFamily(a1307 [Bespoke
>     Containers:false:5@1319097878110000,])])
>     DEBUG [MutationStage:48] 2011-10-20 10:04:47,602 Table.java (line
>     398) applying mutation of row 6131333036
>     DEBUG [MutationStage:48] 2011-10-20 10:04:47,603
>     RowMutationVerbHandler.java (line 77)
>     RowMutation(keyspace='MyKeyspace', key='6131333036',
>     modifications=[ColumnFamily(a1307 [Bespoke
>     Containers:false:5@1319097878110000,])]) applied.  Sending
>     response to 1504@/192.168.1.30 <http://192.168.1.30>
>     DEBUG [MutationStage:49] 2011-10-20 10:04:47,605
>     RowMutationVerbHandler.java (line 48) Applying
>     RowMutation(keyspace='MyKeyspace', key='6131323231',
>     modifications=[ColumnFamily(a1307 [Entity ID
>     Field:false:5@1319097878113000,])])
>     DEBUG [MutationStage:49] 2011-10-20 10:04:47,606 Table.java (line
>     398) applying mutation of row 6131323231
>     DEBUG [MutationStage:49] 2011-10-20 10:04:47,606
>     RowMutationVerbHandler.java (line 77)
>     RowMutation(keyspace='MyKeyspace', key='6131323231',
>     modifications=[ColumnFamily(a1307 [Entity ID
>     Field:false:5@1319097878113000,])]) applied.  Sending response to
>     1505@/192.168.1.30 <http://192.168.1.30>
>     DEBUG [MutationStage:50] 2011-10-20 10:04:47,609
>     RowMutationVerbHandler.java (line 48) Applying
>     RowMutation(keyspace='MyKeyspace', key='6131323231',
>     modifications=[ColumnFamily(a1307 [Entity Type
>     Field:false:5@1319097878117000,])])
>     DEBUG [MutationStage:50] 2011-10-20 10:04:47,609 Table.java (line
>     398) applying mutation of row 6131323231
>     DEBUG [MutationStage:50] 2011-10-20 10:04:47,610
>     RowMutationVerbHandler.java (line 77)
>     RowMutation(keyspace='MyKeyspace', key='6131323231',
>     modifications=[ColumnFamily(a1307 [Entity Type
>     Field:false:5@1319097878117000,])]) applied.  Sending response to
>     1506@/192.168.1.30 <http://192.168.1.30>
>     DEBUG [MigrationStage:1] 2011-10-20 <tel:1%5D%202011-10-20>
>     10:04:47,612 SchemaCheckVerbHandler.java (line 36) Received schema
>     check request.
>     DEBUG [MigrationStage:1] 2011-10-20 <tel:1%5D%202011-10-20>
>     10:04:47,655 SchemaCheckVerbHandler.java (line 36) Received schema
>     check request.
>     DEBUG [MigrationStage:1] 2011-10-20 <tel:1%5D%202011-10-20>
>     10:04:47,671 DefinitionsUpdateVerbHandler.java (line 70) Applying
>     AddColumnFamily from /192.168.1.30 <http://192.168.1.30>
>     DEBUG [MigrationStage:1] 2011-10-20 <tel:1%5D%202011-10-20>
>     10:04:47,671 Table.java (line 398) applying mutation of row
>     32376666636361302d666166322d313165302d303030302d313538346131373038306266
>     DEBUG [MigrationStage:1] 2011-10-20 <tel:1%5D%202011-10-20>
>     10:04:47,673 Table.java (line 398) applying mutation of row
>     4d6967726174696f6e73204b6579
>      I*NFO [MigrationStage:1] 2011-10-20 <tel:1%5D%202011-10-20>
>     10:04:47,673 Migration.java (line 116) Applying migration
>     27ffcca0-faf2-11e0-0000-1584a17080bf Add column family:
>     org.apache.cassandra.config.CFMetaData@33a998c7[cfId=1001,ksName=MyKeyspace,cfName=a1101,cfType=Standard,comparator=org.apache.cassandra.db.marshal.AsciiType,subcolumncomparator=<null>,comment=,rowCacheSize=0.0,keyCacheSize=200000.0,readRepairChance=1.0,replicateOnWrite=true,gcGraceSeconds=864000,defaultValidator=org.apache.cassandra.db.marshal.BytesType,keyValidator=org.apache.cassandra.db.marshal.BytesType,minCompactionThreshold=4,maxCompactionThreshold=32,rowCacheSavePeriodInSeconds=0,keyCacheSavePeriodInSeconds=14400,memtableFlushAfterMins=1440,memtableThroughputInMb=55,memtableOperationsInMillions=0.2578125,mergeShardsChance=0.1,keyAlias=<null>,column_metadata={}]*
>     DEBUG [MigrationStage:1] 2011-10-20 <tel:1%5D%202011-10-20>
>     10:04:47,673 Table.java (line 398) applying mutation of row
>     4c617374204d6967726174696f6e
>     DEBUG [MigrationStage:1] 2011-10-20 <tel:1%5D%202011-10-20>
>     10:04:47,674 ColumnFamilyStore.java (line 765) flush position is
>     ReplayPosition(segmentId=1319097494650, position=33582)
>      INFO [MigrationStage:1] 2011-10-20 <tel:1%5D%202011-10-20>
>     10:04:47,674 ColumnFamilyStore.java (line 1128) Enqueuing flush of
>     Memtable-Migrations@1131236176(7476/9345 serialized/live bytes, 1 ops)
>     DEBUG [MigrationStage:1] 2011-10-20 <tel:1%5D%202011-10-20>
>     10:04:47,674 ColumnFamilyStore.java (line 765) flush position is
>     ReplayPosition(segmentId=1319097494650, position=33582)
>      INFO [FlushWriter:3] 2011-10-20 <tel:3%5D%202011-10-20>
>     10:04:47,675 Memtable.java (line 237) Writing
>     Memtable-Migrations@1131236176(7476/9345 serialized/live bytes, 1 ops)
>      INFO [MigrationStage:1] 2011-10-20 <tel:1%5D%202011-10-20>
>     10:04:47,676 ColumnFamilyStore.java (line 1128) Enqueuing flush of
>     Memtable-Schema@1013683141(3283/4103 serialized/live bytes, 3 ops)
>     DEBUG [FlushWriter:3] 2011-10-20 <tel:3%5D%202011-10-20>
>     10:04:47,676 DatabaseDescriptor.java (line 821) expected data
>     files size is 9004; largest free partition has 88471822336 bytes free
>      INFO [FlushWriter:3] 2011-10-20 <tel:3%5D%202011-10-20>
>     10:04:47,725 Memtable.java (line 254) Completed flushing
>     /var/lib/cassandra/data/system/Migrations-g-3-Data.db (7540 bytes)
>     DEBUG [FlushWriter:3] 2011-10-20 <tel:3%5D%202011-10-20>
>     10:04:47,725 DataTracker.java (line 264) adding
>     /var/lib/cassandra/data/system/Migrations-g-3 to list of files
>     tracked for system.Migrations
>     DEBUG [FlushWriter:3] 2011-10-20 <tel:3%5D%202011-10-20>
>     10:04:47,726 AutoSavingCache.java (line 175) KeyCache capacity for
>     Migrations is 3
>     DEBUG [CompactionExecutor:4] 2011-10-20 <tel:4%5D%202011-10-20>
>     10:04:47,726 CompactionManager.java (line 124) Checking to see if
>     compaction of Migrations would be useful
>      INFO [FlushWriter:3] 2011-10-20 <tel:3%5D%202011-10-20>
>     10:04:47,726 Memtable.java (line 237) Writing
>     Memtable-Schema@1013683141(3283/4103 serialized/live bytes, 3 ops)
>     DEBUG [COMMIT-LOG-WRITER] 2011-10-20 10:04:47,726 CommitLog.java
>     (line 446) discard completed log segments for
>     ReplayPosition(segmentId=1319097494650, position=33582), column
>     family 2.
>     DEBUG [COMMIT-LOG-WRITER] 2011-10-20 10:04:47,727 CommitLog.java
>     (line 486) Not safe to delete commit log
>     CommitLogSegment(/var/lib/cassandra/commitlog/CommitLog-1319097494650.log);
>     dirty is a1307 (1000), Schema (3), ; hasNext: false
>     DEBUG [FlushWriter:3] 2011-10-20 <tel:3%5D%202011-10-20>
>     10:04:47,727 DatabaseDescriptor.java (line 821) expected data
>     files size is 4059; largest free partition has 88471797760 bytes free
>      INFO [FlushWriter:3] 2011-10-20 <tel:3%5D%202011-10-20>
>     10:04:47,751 Memtable.java (line 254) Completed flushing
>     /var/lib/cassandra/data/system/Schema-g-3-Data.db (3433 bytes)
>     DEBUG [FlushWriter:3] 2011-10-20 <tel:3%5D%202011-10-20>
>     10:04:47,752 DataTracker.java (line 264) adding
>     /var/lib/cassandra/data/system/Schema-g-3 to list of files tracked
>     for system.Schema
>     DEBUG [FlushWriter:3] 2011-10-20 <tel:3%5D%202011-10-20>
>     10:04:47,752 AutoSavingCache.java (line 175) KeyCache capacity for
>     Schema is 3
>     DEBUG [CompactionExecutor:5] 2011-10-20 <tel:5%5D%202011-10-20>
>     10:04:47,752 CompactionManager.java (line 124) Checking to see if
>     compaction of Schema would be useful
>     DEBUG [COMMIT-LOG-WRITER] 2011-10-20 10:04:47,752 CommitLog.java
>     (line 446) discard completed log segments for
>     ReplayPosition(segmentId=1319097494650, position=33582), column
>     family 3.
>     DEBUG [COMMIT-LOG-WRITER] 2011-10-20 10:04:47,753 CommitLog.java
>     (line 486) Not safe to delete commit log
>     CommitLogSegment(/var/lib/cassandra/commitlog/CommitLog-1319097494650.log);
>     dirty is a1307 (1000), ; hasNext: false
>
>
>     -- 
>     Renato Bacelar da Silveira
>     Software Engineer (Java J2SE J2EE)
>
>
>
>
> -- 
> Jérémy

Re: Column Family not able to be replicated between 2 nodes

Posted by Jérémy SEVELLEC <js...@gmail.com>.

Hi,

I think it's because you create a new columnFamily with hector and you don't
wait the propagation of this modification in the cluster before writing.

Hector provide a method witjh a boolean to say "blockUntilComplete" when you
create a columnFamily :
cluster.addColumnFamily(<your cf definition>, true);

Here is the hector javadoc about that :
http://rantav.github.com/hector//source/content/API/core/0.8.0-2/me/prettyprint/hector/api/Cluster.html#addColumnFamily(me.prettyprint.hector.api.ddl.ColumnFamilyDefinition,
boolean)

Regards,

Jérémy

2011/10/20 Renato Baceolar da Silveira <re...@indabamobile.co.za>

>  Hi All and greetings from South Africa, a country with a* sad Rugby team
> :(*
>
> I am processing a batch operation, that inserts a number of column
> families.
>
> This batch operation is executed in a cluster with 2 nodes and replication
> of 2.
>
> EACH NODE is installed in a  VMWARE ESSI virtual box, in a Ubuntu 9
> server 64 bit. Both Nodes live in the same physical server, and
> communication between
> them is through shared hardware.
>
>  The virtual machines have 4gb of ram, and plenty hdd
> space.
>
> The version of Cassandra is 0.8.5.
>
> Both nodes start empty before this batch process commences.
>
> At the insertion of the first Column Family I receive this error in the
> log, which
> tells me that for a number of times the specific column family is Replayed
> for insertion, but it seems not to succeed. Without the insertion of this
> first Column Family our validation process fails the batch process.
>
> Can any one perhaps shed some light on this matter?
>
>
> Kind Regards
>
> Renato da Silveira
>
> ===============
> error log - Applicaion Process
> ===============
>
> me.prettyprint.hector.api.
> **exceptions.HectorException: SchemaDisagreementException()
>     at me.prettyprint.cassandra.**service.**ExceptionsTranslatorImpl.**
> translate(**ExceptionsTranslatorImpl.java:**68)
>     at me.prettyprint.cassandra.**service.ThriftCluster$4.**
> execute(ThriftCluster.java:82)
>     at me.prettyprint.cassandra.**service.ThriftCluster$4.**
> execute(ThriftCluster.java:76)
>     at me.prettyprint.cassandra.**service.Operation.**
> executeAndSetResult(Operation.**java:101)
>     at me.prettyprint.cassandra.**connection.HConnectionManager.**
> operateWithFailover(**HConnectionManager.java:219)
>     at me.prettyprint.cassandra.**service.ThriftCluster.**
> addColumnFamily(ThriftCluster.**java:86)
>     at com.xxx.xxx.hector.**HectorManager.addColumnFamily(**
> HectorManager.java:132)
>     at com.xxx.xxx.facade.**CassandraManager.**addColumnFamily(**
> CassandraManager.java:117)
>     at com.xxx.xxx.model.xxx.**createRegistryContainer(xxx.**java:63)
>     at com.xxx.xxx.model.xxx.**buildRegistry(xxx.java:120)
>     at com.xxx.xxx.facade.Process.**articleLoaderDoInstance(**
> Process.java:117)
>     at com.xxx.xxx.facade.Process.**articleLoader(Process.java:58)
> Caused by: SchemaDisagreementException()
>     at org.apache.cassandra.thrift.**Cassandra$system_add_column_**
> family_result.read(Cassandra.**java:26119)
>     at org.apache.cassandra.thrift.**Cassandra$Client.recv_system_**
> add_column_family(Cassandra.**java:1455)
>     at org.apache.cassandra.thrift.**Cassandra$Client.system_add_**
> column_family(Cassandra.java:**1430)
>     at me.prettyprint.cassandra.**service.ThriftCluster$4.**
> execute(ThriftCluster.java:80)
>     ... 10 more
>
>
> ===============
> error log - Cassandra Log
> ===============
> INFO [MigrationStage:1] 2011-10-20 10:03:58,944 Migration.java (line 116)
> Applying migration 0aadb220-faf2-11e0-0000-**1584a17080bf Add keyspace:
> MyKeyspace, rep strategy:SimpleStrategy{}, durable_writes: true
> DEBUG [MigrationStage:1] 2011-10-20 10:03:58,945 Table.java (line 398)
> applying mutation of row 4c617374204d6967726174696f6e
> DEBUG [MigrationStage:1] 2011-10-20 10:03:58,945 ColumnFamilyStore.java
> (line 765) flush position is ReplayPosition(segmentId=**1319097494650,
> position=10486)
>  INFO [MigrationStage:1] 2011-10-20 10:03:58,945 ColumnFamilyStore.java
> (line 1128) Enqueuing flush of Memtable-Migrations@539124954(**6675/8343
> serialized/live bytes, 1 ops)
> DEBUG [MigrationStage:1] 2011-10-20 10:03:58,946 ColumnFamilyStore.java
> (line 765) flush position is ReplayPosition(segmentId=**1319097494650,
> position=10486)
>  INFO [FlushWriter:3] 2011-10-20 10:03:58,946 Memtable.java (line 237)
> Writing Memtable-Migrations@539124954(**6675/8343 serialized/live bytes, 1
> ops)
>  INFO [MigrationStage:1] 2011-10-20 10:03:58,946 ColumnFamilyStore.java
> (line 1128) Enqueuing flush of Memtable-Schema@1225252094(**2696/3370
> serialized/live bytes, 3 ops)
> DEBUG [FlushWriter:3] 2011-10-20 10:03:58,947 DatabaseDescriptor.java
> (line 821) expected data files size is 8043; largest free partition has
> 88471957504 bytes free
>  INFO [FlushWriter:3] 2011-10-20 10:03:58,991 Memtable.java (line 254)
> Completed flushing /var/lib/cassandra/data/**system/Migrations-g-1-Data.db
> (6739 bytes)
> DEBUG [FlushWriter:3] 2011-10-20 10:03:58,991 DataTracker.java (line 264)
> adding /var/lib/cassandra/data/**system/Migrations-g-1 to list of files
> tracked for system.Migrations
> DEBUG [COMMIT-LOG-WRITER] 2011-10-20 10:03:58,992 CommitLog.java (line 446)
> discard completed log segments for ReplayPosition(segmentId=**1319097494650,
> position=10486), column family 2.
>  INFO [FlushWriter:3] 2011-10-20 10:03:58,992 Memtable.java (line 237)
> Writing Memtable-Schema@1225252094(**2696/3370 serialized/live bytes, 3
> ops)
> DEBUG [COMMIT-LOG-WRITER] 2011-10-20 10:03:58,992 CommitLog.java (line 486)
> Not safe to delete commit log CommitLogSegment(/var/lib/**
> cassandra/commitlog/CommitLog-**1319097494650.log); dirty is Schema (3), ;
> hasNext: false
> DEBUG [CompactionExecutor:4] 2011-10-20 10:03:58,992
> CompactionManager.java (line 124) Checking to see if compaction of
> Migrations would be useful
> DEBUG [FlushWriter:3] 2011-10-20 10:03:58,993 DatabaseDescriptor.java
> (line 821) expected data files size is 3355; largest free partition has
> 88471932928 bytes free
>  INFO [FlushWriter:3] 2011-10-20 10:03:59,026 Memtable.java (line 254)
> Completed flushing /var/lib/cassandra/data/**system/Schema-g-1-Data.db
> (2846 bytes)
> DEBUG [FlushWriter:3] 2011-10-20 10:03:59,026 DataTracker.java (line 264)
> adding /var/lib/cassandra/data/**system/Schema-g-1 to list of files
> tracked for system.Schema
> DEBUG [COMMIT-LOG-WRITER] 2011-10-20 10:03:59,027 CommitLog.java (line 446)
> discard completed log segments for ReplayPosition(segmentId=**1319097494650,
> position=10486), column family 3.
> DEBUG [CompactionExecutor:5] 2011-10-20 10:03:59,027
> CompactionManager.java (line 124) Checking to see if compaction of Schema
> would be useful
> DEBUG [COMMIT-LOG-WRITER] 2011-10-20 10:03:59,027 CommitLog.java (line 486)
> Not safe to delete commit log CommitLogSegment(/var/lib/**
> cassandra/commitlog/CommitLog-**1319097494650.log); dirty is ; hasNext:
> false
> DEBUG [MigrationStage:1] 2011-10-20 10:03:59,028 MigrationManager.java
> (line 137) Gossiping my schema version 0aadb220-faf2-11e0-0000-**
> 1584a17080bf
> DEBUG [MigrationStage:1] 2011-10-20 10:03:59,029
> SchemaCheckVerbHandler.java (line 36) Received schema check request.
> DEBUG [ScheduledTasks:1] 2011-10-20 10:04:16,797 StorageLoadBalancer.java
> (line 336) Disseminating load info ...
> DEBUG [MigrationStage:1] 2011-10-20 10:04:37,373
> SchemaCheckVerbHandler.java (line 36) Received schema check request.
> *DEBUG [MigrationStage:1] 2011-10-20 10:04:37,444
> DefinitionsUpdateVerbHandler.java (line 70) Applying AddColumnFamily from
> /192.168.1.30*
> DEBUG [MigrationStage:1] 2011-10-20 10:04:37,444 Table.java (line 398)
> applying mutation of row 32316535633164302d666166322d31**
> 3165302d303030302d313538346131**373038306266
> DEBUG [MigrationStage:1] 2011-10-20 10:04:37,447 Table.java (line 398)
> applying mutation of row 4d6967726174696f6e73204b6579
>  INFO [MigrationStage:1] 2011-10-20 10:04:37,456 Migration.java (line 116)
> * Applying migration 21e5c1d0-faf2-11e0-0000-1584a17080bf Add column
> family: org.apache.cassandra.config.CFMetaData@7f1fc4b8[cfId=1000,
> ksName=MyKeyspace,cfName=a1307,cfType=Standard,comparator=org.apache.
> cassandra.db.marshal.AsciiType,subcolumncomparator=
> <null>,comment=,rowCacheSize=0.0,keyCacheSize=200000.0,
> readRepairChance=1.0,replicateOnWrite=true,gcGraceSeconds=864000,
> defaultValidator=org.apache.cassandra.db.marshal.
> BytesType,keyValidator=org.apache.cassandra.db.marshal.BytesType,
> minCompactionThreshold=4,maxCompactionThreshold=32,
> rowCacheSavePeriodInSeconds=0,keyCacheSavePeriodInSeconds=
> 14400,memtableFlushAfterMins=1440,memtableThroughputInMb=55,
> memtableOperationsInMillions=0.2578125,mergeShardsChance=0.
> 1,keyAlias=<null>,column_metadata={}]*
> DEBUG [MigrationStage:1] 2011-10-20 10:04:37,456 Table.java (line 398)
> applying mutation of row 4c617374204d6967726174696f6e
> *DEBUG [MigrationStage:1] 2011-10-20 10:04:37,456 ColumnFamilyStore.java
> (line 765) flush position is ReplayPosition(segmentId=1319097494650,
> position=20900)*
>  INFO [MigrationStage:1] 2011-10-20 10:04:37,457 ColumnFamilyStore.java
> (line 1128) Enqueuing flush of Memtable-Migrations@2125221179**(7183/8978
> serialized/live bytes, 1 ops)
> DEBUG [MigrationStage:1] 2011-10-20 10:04:37,457 ColumnFamilyStore.java
> (line 765) flush position is ReplayPosition(segmentId=**1319097494650,
> position=20900)
>  INFO [FlushWriter:3] 2011-10-20 10:04:37,457 Memtable.java (line 237)
> Writing Memtable-Migrations@2125221179**(7183/8978 serialized/live bytes,
> 1 ops)
>  INFO [MigrationStage:1] 2011-10-20 10:04:37,458 ColumnFamilyStore.java
> (line 1128) Enqueuing flush of Memtable-Schema@1644338549(**2990/3737
> serialized/live bytes, 3 ops)
> DEBUG [FlushWriter:3] 2011-10-20 10:04:37,459 DatabaseDescriptor.java
> (line 821) expected data files size is 8653; largest free partition has
> 88471896064 bytes free
>  INFO [FlushWriter:3] 2011-10-20 10:04:37,513 Memtable.java (line 254)
> Completed flushing /var/lib/cassandra/data/**system/Migrations-g-2-Data.db
> (7247 bytes)
> DEBUG [FlushWriter:3] 2011-10-20 10:04:37,514 DataTracker.java (line 264)
> adding /var/lib/cassandra/data/**system/Migrations-g-2 to list of files
> tracked for system.Migrations
> DEBUG [FlushWriter:3] 2011-10-20 10:04:37,514 AutoSavingCache.java (line
> 175) KeyCache capacity for Migrations is 2
> DEBUG [COMMIT-LOG-WRITER] 2011-10-20 10:04:37,514 CommitLog.java (line 446)
> discard completed log segments for ReplayPosition(segmentId=**1319097494650,
> position=20900), column family 2.
>  INFO [FlushWriter:3] 2011-10-20 10:04:37,515 Memtable.java (line 237)
> Writing Memtable-Schema@1644338549(**2990/3737 serialized/live bytes, 3
> ops)
> DEBUG [CompactionExecutor:6] 2011-10-20 10:04:37,515
> CompactionManager.java (line 124) Checking to see if compaction of
> Migrations would be useful
> DEBUG [COMMIT-LOG-WRITER] 2011-10-20 10:04:37,515 CommitLog.java (line 486)
> Not safe to delete commit log CommitLogSegment(/var/lib/**
> cassandra/commitlog/CommitLog-**1319097494650.log); dirty is Schema (3), ;
> hasNext: false
>  INFO [FlushWriter:3] 2011-10-20 10:04:37,515 Memtable.java (line 237)
> Writing Memtable-Schema@1644338549(**2990/3737 serialized/live bytes, 3
> ops)
> DEBUG [CompactionExecutor:6] 2011-10-20 10:04:37,515
> CompactionManager.java (line 124) Checking to see if compaction of
> Migrations would be useful
> DEBUG [COMMIT-LOG-WRITER] 2011-10-20 10:04:37,515 CommitLog.java (line 486)
> Not safe to delete commit log CommitLogSegment(/var/lib/**
> cassandra/commitlog/CommitLog-**1319097494650.log); dirty is Schema (3), ;
> hasNext: false
> DEBUG [FlushWriter:3] 2011-10-20 10:04:37,515 DatabaseDescriptor.java
> (line 821) expected data files size is 3708; largest free partition has
> 88471871488 bytes free
> ERROR [MutationStage:33] 2011-10-20 10:04:37,530
> RowMutationVerbHandler.java (line 82) Error in row mutation
> org.apache.cassandra.db.**UnserializableColumnFamilyExce**ption: Couldn't
> find cfId=1000
>     at org.apache.cassandra.db.**ColumnFamilySerializer.**deserialize(**
> ColumnFamilySerializer.java:**123)
>     at org.apache.cassandra.db.**RowMutation$**RowMutationSerializer.**
> deserialize(RowMutation.java:**392)
>     at org.apache.cassandra.db.**RowMutation$**RowMutationSerializer.**
> deserialize(RowMutation.java:**400)
>     at org.apache.cassandra.db.**RowMutation.fromBytes(**
> RowMutation.java:329)
>     at org.apache.cassandra.db.**RowMutationVerbHandler.doVerb(**
> RowMutationVerbHandler.java:**46)
>     at org.apache.cassandra.net.**MessageDeliveryTask.run(**
> MessageDeliveryTask.java:59)
>     at java.util.concurrent.**ThreadPoolExecutor$Worker.**
> runTask(ThreadPoolExecutor.**java:886)
>     at java.util.concurrent.**ThreadPoolExecutor$Worker.run(**
> ThreadPoolExecutor.java:908)
>     at java.lang.Thread.run(Thread.**java:662)
>  INFO [FlushWriter:3] 2011-10-20 10:04:37,553 Memtable.java (line 254)
> Completed flushing /var/lib/cassandra/data/**system/Schema-g-2-Data.db
> (3140 bytes)
> DEBUG [FlushWriter:3] 2011-10-20 10:04:37,553 DataTracker.java (line 264)
> adding /var/lib/cassandra/data/**system/Schema-g-2 to list of files
> tracked for system.Schema
> DEBUG [FlushWriter:3] 2011-10-20 10:04:37,554 AutoSavingCache.java (line
> 175) KeyCache capacity for Schema is 2
> DEBUG [COMMIT-LOG-WRITER] 2011-10-20 10:04:37,555 CommitLog.java (line 446)
> discard completed log segments for ReplayPosition(segmentId=**1319097494650,
> position=20900), column family 3.
> DEBUG [CompactionExecutor:7] 2011-10-20 10:04:37,555 CompactionManager.java
> (line 124) Checking to see if compaction of Schema would be useful
> DEBUG [COMMIT-LOG-WRITER] 2011-10-20 10:04:37,555 CommitLog.java (line 486)
> Not safe to delete commit log CommitLogSegment(/var/lib/**
> cassandra/commitlog/CommitLog-**1319097494650.log); dirty is ; hasNext:
> false
> DEBUG [MigrationStage:1] 2011-10-20 10:04:37,556 ColumnFamilyStore.java
> (line 264) Starting CFS a1307
> DEBUG [MigrationStage:1] 2011-10-20 10:04:37,557 AutoSavingCache.java
> (line 175) KeyCache capacity for a1307 is 200000
> DEBUG [MigrationStage:1] 2011-10-20 10:04:37,557 MigrationManager.java
> (line 137) Gossiping my schema version 21e5c1d0-faf2-11e0-0000-**
> 1584a17080bf
> DEBUG [MutationStage:34] 2011-10-20 10:04:47,503
> RowMutationVerbHandler.java (line 48) Applying RowMutation(keyspace='**MyKeyspace',
> key='6131333034', modifications=[ColumnFamily(**a1307
> [User:false:3@1319097867943000**,])])
> DEBUG [MutationStage:34] 2011-10-20 10:04:47,504 Table.java (line 398)
> applying mutation of row 6131333034
> DEBUG [MutationStage:34] 2011-10-20 10:04:47,506
> RowMutationVerbHandler.java (line 77) RowMutation(keyspace='**MyKeyspace',
> key='6131333034', modifications=[ColumnFamily(**a1307
> [User:false:3@1319097867943000**,])]) applied.  Sending response to 1490@/
> 192.168.1.30
> DEBUG [MutationStage:35] 2011-10-20 10:04:47,544
> RowMutationVerbHandler.java (line 48) Applying RowMutation(keyspace='**MyKeyspace',
> key='6131333034', modifications=[ColumnFamily(**a1307 [Group:false:3@**
> 1319097878019000,])])
> DEBUG [MutationStage:35] 2011-10-20 10:04:47,546 Table.java (line 398)
> applying mutation of row 6131333034
> DEBUG [MutationStage:35] 2011-10-20 10:04:47,546
> RowMutationVerbHandler.java (line 77) RowMutation(keyspace='**MyKeyspace',
> key='6131333034', modifications=[ColumnFamily(**a1307 [Group:false:3@**1319097878019000,])])
> applied.  Sending response to 1491@/192.168.1.30
> DEBUG [MutationStage:36] 2011-10-20 10:04:47,553
> RowMutationVerbHandler.java (line 48) Applying RowMutation(keyspace='**MyKeyspace',
> key='6131333035', modifications=[ColumnFamily(**a1307 [RegContent:false:5@
> **1319097878059000,])])
> DEBUG [MutationStage:36] 2011-10-20 10:04:47,554 Table.java (line 398)
> applying mutation of row 6131333035
> DEBUG [MutationStage:36] 2011-10-20 10:04:47,554
> RowMutationVerbHandler.java (line 77) RowMutation(keyspace='**MyKeyspace',
> key='6131333035', modifications=[ColumnFamily(**a1307 [RegContent:false:5@
> **1319097878059000,])]) applied.  Sending response to 1492@/192.168.1.30
> DEBUG [MutationStage:37] 2011-10-20 10:04:47,558
> RowMutationVerbHandler.java (line 48) Applying RowMutation(keyspace='**MyKeyspace',
> key='6131333035', modifications=[ColumnFamily(**a1307 [RegGroup:false:5@**
> 1319097878065000,])])
> DEBUG [MutationStage:37] 2011-10-20 10:04:47,558 Table.java (line 398)
> applying mutation of row 6131333035
> DEBUG [MutationStage:37] 2011-10-20 10:04:47,559
> RowMutationVerbHandler.java (line 77) RowMutation(keyspace='**MyKeyspace',
> key='6131333035', modifications=[ColumnFamily(**a1307 [RegGroup:false:5@**1319097878065000,])])
> applied.  Sending response to 1493@/192.168.1.30
> DEBUG [MutationStage:38] 2011-10-20 10:04:47,563
> RowMutationVerbHandler.java (line 48) Applying RowMutation(keyspace='**MyKeyspace',
> key='6131333035', modifications=[ColumnFamily(**a1307 [RegUser:false:5@**
> 1319097878070000,])])
> DEBUG [MutationStage:38] 2011-10-20 10:04:47,563 Table.java (line 398)
> applying mutation of row 6131333035
> DEBUG [MutationStage:38] 2011-10-20 10:04:47,563
> RowMutationVerbHandler.java (line 77) RowMutation(keyspace='**MyKeyspace',
> key='6131333035', modifications=[ColumnFamily(**a1307 [RegUser:false:5@**1319097878070000,])])
> applied.  Sending response to 1494@/192.168.1.30
> DEBUG [MutationStage:39] 2011-10-20 10:04:47,567
> RowMutationVerbHandler.java (line 48) Applying RowMutation(keyspace='**MyKeyspace',
> key='6131333035', modifications=[ColumnFamily(**a1307
> [RegContentSched:false:5@**1319097878074000,])])
> DEBUG [MutationStage:39] 2011-10-20 10:04:47,567 Table.java (line 398)
> applying mutation of row 6131333035
> DEBUG [MutationStage:39] 2011-10-20 10:04:47,567
> RowMutationVerbHandler.java (line 77) RowMutation(keyspace='**MyKeyspace',
> key='6131333035', modifications=[ColumnFamily(**a1307
> [RegContentSched:false:5@**1319097878074000,])]) applied.  Sending
> response to 1495@/192.168.1.30
> DEBUG [MutationStage:40] 2011-10-20 10:04:47,571
> RowMutationVerbHandler.java (line 48) Applying RowMutation(keyspace='**MyKeyspace',
> key='6131333035', modifications=[ColumnFamily(**a1307
> [ActiveEntity:false:5@**1319097878078000,])])
> DEBUG [MutationStage:40] 2011-10-20 10:04:47,571 Table.java (line 398)
> applying mutation of row 6131333035
> DEBUG [MutationStage:40] 2011-10-20 10:04:47,571
> RowMutationVerbHandler.java (line 77) RowMutation(keyspace='**MyKeyspace',
> key='6131333035', modifications=[ColumnFamily(**a1307
> [ActiveEntity:false:5@**1319097878078000,])]) applied.  Sending response
> to 1496@/192.168.1.30
> DEBUG [MutationStage:41] 2011-10-20 10:04:47,574
> RowMutationVerbHandler.java (line 48) Applying RowMutation(keyspace='**MyKeyspace',
> key='6131333035', modifications=[ColumnFamily(**a1307
> [InactiveEntity:false:5@**1319097878082000,])])
> DEBUG [MutationStage:41] 2011-10-20 10:04:47,575 Table.java (line 398)
> applying mutation of row 6131333035
> DEBUG [MutationStage:41] 2011-10-20 10:04:47,575
> RowMutationVerbHandler.java (line 77) RowMutation(keyspace='**MyKeyspace',
> key='6131333035', modifications=[ColumnFamily(**a1307
> [InactiveEntity:false:5@**1319097878082000,])]) applied.  Sending response
> to 1497@/192.168.1.30
> DEBUG [MutationStage:42] 2011-10-20 10:04:47,579
> RowMutationVerbHandler.java (line 48) Applying RowMutation(keyspace='**MyKeyspace',
> key='6131333036', modifications=[ColumnFamily(**a1307 [Friend
> List:false:5@1319097878087000,**])])
> DEBUG [MutationStage:42] 2011-10-20 10:04:47,579 Table.java (line 398)
> applying mutation of row 6131333036
> DEBUG [MutationStage:42] 2011-10-20 10:04:47,580
> RowMutationVerbHandler.java (line 77) RowMutation(keyspace='**MyKeyspace',
> key='6131333036', modifications=[ColumnFamily(**a1307 [Friend
> List:false:5@1319097878087000,**])]) applied.  Sending response to 1498@/
> 192.168.1.30
> DEBUG [MutationStage:43] 2011-10-20 10:04:47,582
> RowMutationVerbHandler.java (line 48) Applying RowMutation(keyspace='**MyKeyspace',
> key='6131333036', modifications=[ColumnFamily(**a1307 [Profile
> Items:false:5@1319097878090000**,])])
> DEBUG [MutationStage:43] 2011-10-20 10:04:47,583 Table.java (line 398)
> applying mutation of row 6131333036
> DEBUG [MutationStage:43] 2011-10-20 10:04:47,583
> RowMutationVerbHandler.java (line 77) RowMutation(keyspace='**MyKeyspace',
> key='6131333036', modifications=[ColumnFamily(**a1307 [Profile
> Items:false:5@1319097878090000**,])]) applied.  Sending response to 1499@/
> 192.168.1.30
> DEBUG [MutationStage:44] 2011-10-20 10:04:47,586
> RowMutationVerbHandler.java (line 48) Applying RowMutation(keyspace='**MyKeyspace',
> key='6131333036', modifications=[ColumnFamily(**a1307 [Likes:false:5@**
> 1319097878094000,])])
> DEBUG [MutationStage:44] 2011-10-20 10:04:47,586 Table.java (line 398)
> applying mutation of row 6131333036
> DEBUG [MutationStage:44] 2011-10-20 10:04:47,587
> RowMutationVerbHandler.java (line 77) RowMutation(keyspace='**MyKeyspace',
> key='6131333036', modifications=[ColumnFamily(**a1307 [Likes:false:5@**1319097878094000,])])
> applied.  Sending response to 1500@/192.168.1.30
> DEBUG [MutationStage:45] 2011-10-20 10:04:47,590
> RowMutationVerbHandler.java (line 48) Applying RowMutation(keyspace='**MyKeyspace',
> key='6131333036', modifications=[ColumnFamily(**a1307 [Groups:false:5@**
> 1319097878097000,])])
> DEBUG [MutationStage:45] 2011-10-20 10:04:47,590 Table.java (line 398)
> applying mutation of row 6131333036
> DEBUG [MutationStage:45] 2011-10-20 10:04:47,591
> RowMutationVerbHandler.java (line 77) RowMutation(keyspace='**MyKeyspace',
> key='6131333036', modifications=[ColumnFamily(**a1307 [Groups:false:5@**1319097878097000,])])
> applied.  Sending response to 1501@/192.168.1.30
> DEBUG [MutationStage:46] 2011-10-20 10:04:47,594
> RowMutationVerbHandler.java (line 48) Applying RowMutation(keyspace='**MyKeyspace',
> key='6131333036', modifications=[ColumnFamily(**a1307 [Content:false:5@**
> 1319097878102000,])])
> DEBUG [MutationStage:46] 2011-10-20 10:04:47,595 Table.java (line 398)
> applying mutation of row 6131333036
> DEBUG [MutationStage:46] 2011-10-20 10:04:47,595
> RowMutationVerbHandler.java (line 77) RowMutation(keyspace='**MyKeyspace',
> key='6131333036', modifications=[ColumnFamily(**a1307 [Content:false:5@**1319097878102000,])])
> applied.  Sending response to 1502@/192.168.1.30
> DEBUG [MutationStage:47] 2011-10-20 10:04:47,598
> RowMutationVerbHandler.java (line 48) Applying RowMutation(keyspace='**MyKeyspace',
> key='6131333036', modifications=[ColumnFamily(**a1307 [Events:false:5@**
> 1319097878106000,])])
> DEBUG [MutationStage:47] 2011-10-20 10:04:47,599 Table.java (line 398)
> applying mutation of row 6131333036
> DEBUG [MutationStage:47] 2011-10-20 10:04:47,599
> RowMutationVerbHandler.java (line 77) RowMutation(keyspace='**MyKeyspace',
> key='6131333036', modifications=[ColumnFamily(**a1307 [Events:false:5@**1319097878106000,])])
> applied.  Sending response to 1503@/192.168.1.30
> DEBUG [MutationStage:48] 2011-10-20 10:04:47,602
> RowMutationVerbHandler.java (line 48) Applying RowMutation(keyspace='**MyKeyspace',
> key='6131333036', modifications=[ColumnFamily(**a1307 [Bespoke
> Containers:false:5@**1319097878110000,])])
> DEBUG [MutationStage:48] 2011-10-20 10:04:47,602 Table.java (line 398)
> applying mutation of row 6131333036
> DEBUG [MutationStage:48] 2011-10-20 10:04:47,603
> RowMutationVerbHandler.java (line 77) RowMutation(keyspace='**MyKeyspace',
> key='6131333036', modifications=[ColumnFamily(**a1307 [Bespoke
> Containers:false:5@**1319097878110000,])]) applied.  Sending response to
> 1504@/192.168.1.30
> DEBUG [MutationStage:49] 2011-10-20 10:04:47,605
> RowMutationVerbHandler.java (line 48) Applying RowMutation(keyspace='**MyKeyspace',
> key='6131323231', modifications=[ColumnFamily(**a1307 [Entity ID
> Field:false:5@1319097878113000**,])])
> DEBUG [MutationStage:49] 2011-10-20 10:04:47,606 Table.java (line 398)
> applying mutation of row 6131323231
> DEBUG [MutationStage:49] 2011-10-20 10:04:47,606
> RowMutationVerbHandler.java (line 77) RowMutation(keyspace='**MyKeyspace',
> key='6131323231', modifications=[ColumnFamily(**a1307 [Entity ID
> Field:false:5@1319097878113000**,])]) applied.  Sending response to 1505@/
> 192.168.1.30
> DEBUG [MutationStage:50] 2011-10-20 10:04:47,609
> RowMutationVerbHandler.java (line 48) Applying RowMutation(keyspace='**MyKeyspace',
> key='6131323231', modifications=[ColumnFamily(**a1307 [Entity Type
> Field:false:5@1319097878117000**,])])
> DEBUG [MutationStage:50] 2011-10-20 10:04:47,609 Table.java (line 398)
> applying mutation of row 6131323231
> DEBUG [MutationStage:50] 2011-10-20 10:04:47,610
> RowMutationVerbHandler.java (line 77) RowMutation(keyspace='**MyKeyspace',
> key='6131323231', modifications=[ColumnFamily(**a1307 [Entity Type
> Field:false:5@1319097878117000**,])]) applied.  Sending response to 1506@/
> 192.168.1.30
> DEBUG [MigrationStage:1] 2011-10-20 10:04:47,612
> SchemaCheckVerbHandler.java (line 36) Received schema check request.
> DEBUG [MigrationStage:1] 2011-10-20 10:04:47,655
> SchemaCheckVerbHandler.java (line 36) Received schema check request.
> DEBUG [MigrationStage:1] 2011-10-20 10:04:47,671
> DefinitionsUpdateVerbHandler.**java (line 70) Applying AddColumnFamily
> from /192.168.1.30
> DEBUG [MigrationStage:1] 2011-10-20 10:04:47,671 Table.java (line 398)
> applying mutation of row 32376666636361302d666166322d31**
> 3165302d303030302d313538346131**373038306266
> DEBUG [MigrationStage:1] 2011-10-20 10:04:47,673 Table.java (line 398)
> applying mutation of row 4d6967726174696f6e73204b6579
>  I*NFO [MigrationStage:1] 2011-10-20 10:04:47,673 Migration.java (line
> 116) Applying migration 27ffcca0-faf2-11e0-0000-1584a17080bf Add column
> family: org.apache.cassandra.config.CFMetaData@33a998c7[cfId=1001,
> ksName=MyKeyspace,cfName=a1101,cfType=Standard,comparator=org.apache.
> cassandra.db.marshal.AsciiType,subcolumncomparator=
> <null>,comment=,rowCacheSize=0.0,keyCacheSize=200000.0,
> readRepairChance=1.0,replicateOnWrite=true,gcGraceSeconds=864000,
> defaultValidator=org.apache.cassandra.db.marshal.
> BytesType,keyValidator=org.apache.cassandra.db.marshal.BytesType,
> minCompactionThreshold=4,maxCompactionThreshold=32,
> rowCacheSavePeriodInSeconds=0,keyCacheSavePeriodInSeconds=
> 14400,memtableFlushAfterMins=1440,memtableThroughputInMb=55,
> memtableOperationsInMillions=0.2578125,mergeShardsChance=0.
> 1,keyAlias=<null>,column_metadata={}]*
> DEBUG [MigrationStage:1] 2011-10-20 10:04:47,673 Table.java (line 398)
> applying mutation of row 4c617374204d6967726174696f6e
> DEBUG [MigrationStage:1] 2011-10-20 10:04:47,674 ColumnFamilyStore.java
> (line 765) flush position is ReplayPosition(segmentId=**1319097494650,
> position=33582)
>  INFO [MigrationStage:1] 2011-10-20 10:04:47,674 ColumnFamilyStore.java
> (line 1128) Enqueuing flush of Memtable-Migrations@1131236176**(7476/9345
> serialized/live bytes, 1 ops)
> DEBUG [MigrationStage:1] 2011-10-20 10:04:47,674 ColumnFamilyStore.java
> (line 765) flush position is ReplayPosition(segmentId=**1319097494650,
> position=33582)
>  INFO [FlushWriter:3] 2011-10-20 10:04:47,675 Memtable.java (line 237)
> Writing Memtable-Migrations@1131236176**(7476/9345 serialized/live bytes,
> 1 ops)
>  INFO [MigrationStage:1] 2011-10-20 10:04:47,676 ColumnFamilyStore.java
> (line 1128) Enqueuing flush of Memtable-Schema@1013683141(**3283/4103
> serialized/live bytes, 3 ops)
> DEBUG [FlushWriter:3] 2011-10-20 10:04:47,676 DatabaseDescriptor.java
> (line 821) expected data files size is 9004; largest free partition has
> 88471822336 bytes free
>  INFO [FlushWriter:3] 2011-10-20 10:04:47,725 Memtable.java (line 254)
> Completed flushing /var/lib/cassandra/data/**system/Migrations-g-3-Data.db
> (7540 bytes)
> DEBUG [FlushWriter:3] 2011-10-20 10:04:47,725 DataTracker.java (line 264)
> adding /var/lib/cassandra/data/**system/Migrations-g-3 to list of files
> tracked for system.Migrations
> DEBUG [FlushWriter:3] 2011-10-20 10:04:47,726 AutoSavingCache.java (line
> 175) KeyCache capacity for Migrations is 3
> DEBUG [CompactionExecutor:4] 2011-10-20 10:04:47,726
> CompactionManager.java (line 124) Checking to see if compaction of
> Migrations would be useful
>  INFO [FlushWriter:3] 2011-10-20 10:04:47,726 Memtable.java (line 237)
> Writing Memtable-Schema@1013683141(**3283/4103 serialized/live bytes, 3
> ops)
> DEBUG [COMMIT-LOG-WRITER] 2011-10-20 10:04:47,726 CommitLog.java (line 446)
> discard completed log segments for ReplayPosition(segmentId=**1319097494650,
> position=33582), column family 2.
> DEBUG [COMMIT-LOG-WRITER] 2011-10-20 10:04:47,727 CommitLog.java (line 486)
> Not safe to delete commit log CommitLogSegment(/var/lib/**
> cassandra/commitlog/CommitLog-**1319097494650.log); dirty is a1307 (1000),
> Schema (3), ; hasNext: false
> DEBUG [FlushWriter:3] 2011-10-20 10:04:47,727 DatabaseDescriptor.java
> (line 821) expected data files size is 4059; largest free partition has
> 88471797760 bytes free
>  INFO [FlushWriter:3] 2011-10-20 10:04:47,751 Memtable.java (line 254)
> Completed flushing /var/lib/cassandra/data/**system/Schema-g-3-Data.db
> (3433 bytes)
> DEBUG [FlushWriter:3] 2011-10-20 10:04:47,752 DataTracker.java (line 264)
> adding /var/lib/cassandra/data/**system/Schema-g-3 to list of files
> tracked for system.Schema
> DEBUG [FlushWriter:3] 2011-10-20 10:04:47,752 AutoSavingCache.java (line
> 175) KeyCache capacity for Schema is 3
> DEBUG [CompactionExecutor:5] 2011-10-20 10:04:47,752
> CompactionManager.java (line 124) Checking to see if compaction of Schema
> would be useful
> DEBUG [COMMIT-LOG-WRITER] 2011-10-20 10:04:47,752 CommitLog.java (line 446)
> discard completed log segments for ReplayPosition(segmentId=**1319097494650,
> position=33582), column family 3.
> DEBUG [COMMIT-LOG-WRITER] 2011-10-20 10:04:47,753 CommitLog.java (line 486)
> Not safe to delete commit log CommitLogSegment(/var/lib/**
> cassandra/commitlog/CommitLog-**1319097494650.log); dirty is a1307 (1000),
> ; hasNext: false
>
>
> --
> Renato Bacelar da Silveira
> Software Engineer (Java J2SE J2EE)
>



-- 
Jérémy