You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Jake Farrell (JIRA)" <ji...@apache.org> on 2010/11/02 15:38:25 UTC

[jira] Created: (CASSANDRA-1695) Bootstrapping new node Error in row mutation UnserializableColumnFamilyException Couldn't find cfId

Bootstrapping new node Error in row mutation UnserializableColumnFamilyException Couldn't find cfId
---------------------------------------------------------------------------------------------------

                 Key: CASSANDRA-1695
                 URL: https://issues.apache.org/jira/browse/CASSANDRA-1695
             Project: Cassandra
          Issue Type: Bug
    Affects Versions: 0.7 beta 3
            Reporter: Jake Farrell


When auto_bootstraping a new node into an existing cluster with data (auto_bootstrap: true) the new node throws UnserializableColumnFamilyException: Couldn't find cfId. Started with beta3 and then tried latest trunk r1029823.  Created a cluster of 5 nodes all using default settings other than ip addresses, OP partitioner, and data dir locations. started writing data to the cluster. On 6th node changed auto_bootstrap setting to true and started 6th node. Received the cfid row mutation error after gossiper finishes and schema is being shared with new node. 


INFO [main] 2010-11-02 08:56:09,150 AbstractCassandraDaemon.java (line 72) Heap size: 10719985664/10719985664
 INFO [main] 2010-11-02 08:56:09,160 CLibrary.java (line 43) JNA not found. Native methods will be disabled.
 INFO [main] 2010-11-02 08:56:09,167 DatabaseDescriptor.java (line 122) Loading settings from file:/opt/cassandra/0.7.0-beta3/conf/cassandra.yaml
 INFO [main] 2010-11-02 08:56:09,262 DatabaseDescriptor.java (line 173) DiskAccessMode 'auto' determined to be mmap, indexAccessMode is mmap
 INFO [main] 2010-11-02 08:56:09,321 CommitLogSegment.java (line 50) Creating new commitlog segment /var/log/cassandra/commitlog/CommitLog-1288706169321.log
 INFO [main] 2010-11-02 08:56:09,406 ColumnFamilyStore.java (line 228) read 0 from saved key cache
 INFO [main] 2010-11-02 08:56:09,408 ColumnFamilyStore.java (line 228) read 0 from saved key cache
 INFO [main] 2010-11-02 08:56:09,409 ColumnFamilyStore.java (line 228) read 0 from saved key cache
 INFO [main] 2010-11-02 08:56:09,410 ColumnFamilyStore.java (line 228) read 0 from saved key cache
 INFO [main] 2010-11-02 08:56:09,411 ColumnFamilyStore.java (line 228) read 0 from saved key cache
 INFO [main] 2010-11-02 08:56:09,413 ColumnFamilyStore.java (line 504) loading row cache for LocationInfo of system
 INFO [main] 2010-11-02 08:56:09,414 ColumnFamilyStore.java (line 509) completed loading (1 ms; 0 keys)  row cache for LocationInfo of system
 INFO [main] 2010-11-02 08:56:09,414 ColumnFamilyStore.java (line 504) loading row cache for HintsColumnFamily of system
 INFO [main] 2010-11-02 08:56:09,414 ColumnFamilyStore.java (line 509) completed loading (0 ms; 0 keys)  row cache for HintsColumnFamily of system
 INFO [main] 2010-11-02 08:56:09,415 ColumnFamilyStore.java (line 504) loading row cache for Migrations of system
 INFO [main] 2010-11-02 08:56:09,415 ColumnFamilyStore.java (line 509) completed loading (0 ms; 0 keys)  row cache for Migrations of system
 INFO [main] 2010-11-02 08:56:09,417 ColumnFamilyStore.java (line 504) loading row cache for Schema of system
 INFO [main] 2010-11-02 08:56:09,417 ColumnFamilyStore.java (line 509) completed loading (0 ms; 0 keys)  row cache for Schema of system
 INFO [main] 2010-11-02 08:56:09,418 ColumnFamilyStore.java (line 504) loading row cache for IndexInfo of system
 INFO [main] 2010-11-02 08:56:09,418 ColumnFamilyStore.java (line 509) completed loading (1 ms; 0 keys)  row cache for IndexInfo of system
 INFO [main] 2010-11-02 08:56:09,442 DatabaseDescriptor.java (line 408) Couldn't detect any schema definitions in local storage.
 INFO [main] 2010-11-02 08:56:09,442 DatabaseDescriptor.java (line 434) Found table data in data directories. Consider using JMX to call org.apache.cassandra.service.StorageService.loadSchemaFromYaml().
 INFO [main] 2010-11-02 08:56:09,445 CommitLog.java (line 187) No commitlog files found; skipping replay
 INFO [main] 2010-11-02 08:56:09,461 SystemTable.java (line 80) Upgrading to 0.7. Purging hints if there are any. Old hints will be snapshotted.
 INFO [main] 2010-11-02 08:56:09,465 StorageService.java (line 347) Cassandra version: 0.7.0-beta3
 INFO [main] 2010-11-02 08:56:09,465 StorageService.java (line 348) Thrift API version: 19.4.0
 INFO [main] 2010-11-02 08:56:09,472 StorageService.java (line 370) Loading persisted ring state
 INFO [main] 2010-11-02 08:56:09,473 StorageService.java (line 378) Starting up server gossip
 INFO [main] 2010-11-02 08:56:09,478 ColumnFamilyStore.java (line 631) switching in a fresh Memtable for LocationInfo at CommitLogContext(file='/var/log/cassandra/commitlog/CommitLog-1288706169321.log', position=700)
 INFO [main] 2010-11-02 08:56:09,479 ColumnFamilyStore.java (line 934) Enqueuing flush of Memtable-LocationInfo@648734830(236 bytes, 4 operations)
 INFO [FlushWriter:1] 2010-11-02 08:56:09,480 Memtable.java (line 167) Writing Memtable-LocationInfo@648734830(236 bytes, 4 operations)
 INFO [FlushWriter:1] 2010-11-02 08:56:09,705 Memtable.java (line 174) Completed flushing /var/lib/cassandra/data1/system/LocationInfo-e-1-Data.db
 INFO [main] 2010-11-02 08:56:09,735 StorageService.java (line 394) This node will not auto bootstrap because it is configured to be a seed node.
 WARN [main] 2010-11-02 08:56:09,740 StorageService.java (line 434) Generated random token 4buaxINos8oCXqym. Random tokens will result in an unbalanced ring; see http://wiki.apache.org/cassandra/Operations
 INFO [main] 2010-11-02 08:56:09,740 ColumnFamilyStore.java (line 631) switching in a fresh Memtable for LocationInfo at CommitLogContext(file='/var/log/cassandra/commitlog/CommitLog-1288706169321.log', position=848)
 INFO [main] 2010-11-02 08:56:09,741 ColumnFamilyStore.java (line 934) Enqueuing flush of Memtable-LocationInfo@1159466336(36 bytes, 1 operations)
 INFO [FlushWriter:1] 2010-11-02 08:56:09,741 Memtable.java (line 167) Writing Memtable-LocationInfo@1159466336(36 bytes, 1 operations)
 INFO [FlushWriter:1] 2010-11-02 08:56:09,979 Memtable.java (line 174) Completed flushing /var/lib/cassandra/data2/system/LocationInfo-e-2-Data.db
 INFO [MigrationStage:1] 2010-11-02 08:56:09,999 ColumnFamilyStore.java (line 631) switching in a fresh Memtable for Migrations at CommitLogContext(file='/var/log/cassandra/commitlog/CommitLog-1288706169321.log', position=11469)
 INFO [MigrationStage:1] 2010-11-02 08:56:09,999 ColumnFamilyStore.java (line 934) Enqueuing flush of Memtable-Migrations@1518579727(6094 bytes, 1 operations)
 INFO [MigrationStage:1] 2010-11-02 08:56:10,000 ColumnFamilyStore.java (line 631) switching in a fresh Memtable for Schema at CommitLogContext(file='/var/log/cassandra/commitlog/CommitLog-1288706169321.log', position=11469)
 INFO [MigrationStage:1] 2010-11-02 08:56:10,000 ColumnFamilyStore.java (line 934) Enqueuing flush of Memtable-Schema@1386432387(2192 bytes, 3 operations)
 INFO [FlushWriter:1] 2010-11-02 08:56:10,000 Memtable.java (line 167) Writing Memtable-Migrations@1518579727(6094 bytes, 1 operations)
 INFO [main] 2010-11-02 08:56:10,005 Mx4jTool.java (line 73) Will not load MX4J, mx4j-tools.jar is not in the classpath
 INFO [main] 2010-11-02 08:56:10,055 CassandraDaemon.java (line 77) Binding thrift service to /192.168.1.94:9160
 INFO [main] 2010-11-02 08:56:10,057 CassandraDaemon.java (line 91) Using TFramedTransport with a max frame size of 15728640 bytes.
 INFO [main] 2010-11-02 08:56:10,061 CassandraDaemon.java (line 119) Listening for thrift clients...
 INFO [FlushWriter:1] 2010-11-02 08:56:10,168 Memtable.java (line 174) Completed flushing /var/lib/cassandra/data1/system/Migrations-e-1-Data.db
 INFO [FlushWriter:1] 2010-11-02 08:56:10,169 Memtable.java (line 167) Writing Memtable-Schema@1386432387(2192 bytes, 3 operations)
 INFO [FlushWriter:1] 2010-11-02 08:56:10,404 Memtable.java (line 174) Completed flushing /var/lib/cassandra/data2/system/Schema-e-1-Data.db
 INFO [MigrationStage:1] 2010-11-02 08:56:10,444 ColumnFamilyStore.java (line 631) switching in a fresh Memtable for Migrations at CommitLogContext(file='/var/log/cassandra/commitlog/CommitLog-1288706169321.log', position=22551)
 INFO [MigrationStage:1] 2010-11-02 08:56:10,445 ColumnFamilyStore.java (line 934) Enqueuing flush of Memtable-Migrations@668531077(6407 bytes, 1 operations)
 INFO [MigrationStage:1] 2010-11-02 08:56:10,445 ColumnFamilyStore.java (line 631) switching in a fresh Memtable for Schema at CommitLogContext(file='/var/log/cassandra/commitlog/CommitLog-1288706169321.log', position=22551)
 INFO [MigrationStage:1] 2010-11-02 08:56:10,445 ColumnFamilyStore.java (line 934) Enqueuing flush of Memtable-Schema@1709782292(2375 bytes, 3 operations)
 INFO [FlushWriter:1] 2010-11-02 08:56:10,450 Memtable.java (line 167) Writing Memtable-Migrations@668531077(6407 bytes, 1 operations)
 INFO [FlushWriter:1] 2010-11-02 08:56:10,680 Memtable.java (line 174) Completed flushing /var/lib/cassandra/data2/system/Migrations-e-2-Data.db
 INFO [FlushWriter:1] 2010-11-02 08:56:10,681 Memtable.java (line 167) Writing Memtable-Schema@1709782292(2375 bytes, 3 operations)
 INFO [FlushWriter:1] 2010-11-02 08:56:10,851 Memtable.java (line 174) Completed flushing /var/lib/cassandra/data1/system/Schema-e-2-Data.db
 INFO [MigrationStage:1] 2010-11-02 08:56:10,853 ColumnFamilyStore.java (line 228) read 0 from saved key cache
 INFO [MigrationStage:1] 2010-11-02 08:56:10,855 ColumnFamilyStore.java (line 631) switching in a fresh Memtable for Migrations at CommitLogContext(file='/var/log/cassandra/commitlog/CommitLog-1288706169321.log', position=33669)
 INFO [MigrationStage:1] 2010-11-02 08:56:10,855 ColumnFamilyStore.java (line 934) Enqueuing flush of Memtable-Migrations@1180026905(6591 bytes, 1 operations)
 INFO [MigrationStage:1] 2010-11-02 08:56:10,856 ColumnFamilyStore.java (line 631) switching in a fresh Memtable for Schema at CommitLogContext(file='/var/log/cassandra/commitlog/CommitLog-1288706169321.log', position=33669)
 INFO [FlushWriter:1] 2010-11-02 08:56:10,856 Memtable.java (line 167) Writing Memtable-Migrations@1180026905(6591 bytes, 1 operations)
 INFO [MigrationStage:1] 2010-11-02 08:56:10,856 ColumnFamilyStore.java (line 934) Enqueuing flush of Memtable-Schema@700712290(2558 bytes, 3 operations)
 INFO [FlushWriter:1] 2010-11-02 08:56:11,010 Memtable.java (line 174) Completed flushing /var/lib/cassandra/data1/system/Migrations-e-3-Data.db
 INFO [FlushWriter:1] 2010-11-02 08:56:11,011 Memtable.java (line 167) Writing Memtable-Schema@700712290(2558 bytes, 3 operations)
 INFO [FlushWriter:1] 2010-11-02 08:56:11,206 Memtable.java (line 174) Completed flushing /var/lib/cassandra/data2/system/Schema-e-3-Data.db
 INFO [MigrationStage:1] 2010-11-02 08:56:11,208 ColumnFamilyStore.java (line 228) read 0 from saved key cache
 INFO [MigrationStage:1] 2010-11-02 08:56:11,210 ColumnFamilyStore.java (line 631) switching in a fresh Memtable for Migrations at CommitLogContext(file='/var/log/cassandra/commitlog/CommitLog-1288706169321.log', position=44966)
 INFO [MigrationStage:1] 2010-11-02 08:56:11,210 ColumnFamilyStore.java (line 934) Enqueuing flush of Memtable-Migrations@1711957716(6770 bytes, 1 operations)
 INFO [MigrationStage:1] 2010-11-02 08:56:11,210 ColumnFamilyStore.java (line 631) switching in a fresh Memtable for Schema at CommitLogContext(file='/var/log/cassandra/commitlog/CommitLog-1288706169321.log', position=44966)
 INFO [FlushWriter:1] 2010-11-02 08:56:11,210 Memtable.java (line 167) Writing Memtable-Migrations@1711957716(6770 bytes, 1 operations)
 INFO [MigrationStage:1] 2010-11-02 08:56:11,211 ColumnFamilyStore.java (line 934) Enqueuing flush of Memtable-Schema@887430306(2739 bytes, 3 operations)
 INFO [GossipStage:1] 2010-11-02 08:56:11,329 Gossiper.java (line 577) Node /192.168.1.217 is now part of the cluster
 INFO [GossipStage:1] 2010-11-02 08:56:11,337 Gossiper.java (line 577) Node /192.168.1.216 is now part of the cluster
 INFO [GossipStage:1] 2010-11-02 08:56:11,338 Gossiper.java (line 577) Node /192.168.1.219 is now part of the cluster
 INFO [GossipStage:1] 2010-11-02 08:56:11,338 Gossiper.java (line 577) Node /192.168.1.218 is now part of the cluster
 INFO [GossipStage:1] 2010-11-02 08:56:11,338 Gossiper.java (line 577) Node /192.168.1.215 is now part of the cluster
 INFO [FlushWriter:1] 2010-11-02 08:56:11,414 Memtable.java (line 174) Completed flushing /var/lib/cassandra/data2/system/Migrations-e-4-Data.db
 INFO [FlushWriter:1] 2010-11-02 08:56:11,415 Memtable.java (line 167) Writing Memtable-Schema@887430306(2739 bytes, 3 operations)
 INFO [CompactionExecutor:1] 2010-11-02 08:56:11,420 CompactionManager.java (line 224) Compacting [org.apache.cassandra.io.sstable.SSTableReader(path='/var/lib/cassandra/data1/system/Migrations-e-1-Data.db'),org.apache.cassandra.io.sstable.SSTableReader(path='/var/lib/cassandra/data2/system/Migrations-e-2-Data.db'),org.apache.cassandra.io.sstable.SSTableReader(path='/var/lib/cassandra/data1/system/Migrations-e-3-Data.db'),org.apache.cassandra.io.sstable.SSTableReader(path='/var/lib/cassandra/data2/system/Migrations-e-4-Data.db')]
 INFO [GossipStage:1] 2010-11-02 08:56:11,727 Gossiper.java (line 569) InetAddress /192.168.1.216 is now UP
 INFO [HintedHandoff:1] 2010-11-02 08:56:11,727 HintedHandOffManager.java (line 190) Started hinted handoff for endpoint /192.168.1.216
 INFO [FlushWriter:1] 2010-11-02 08:56:11,727 Memtable.java (line 174) Completed flushing /var/lib/cassandra/data1/system/Schema-e-4-Data.db
 INFO [GossipStage:1] 2010-11-02 08:56:11,728 Gossiper.java (line 569) InetAddress /192.168.1.215 is now UP
 INFO [HintedHandoff:1] 2010-11-02 08:56:11,729 HintedHandOffManager.java (line 246) Finished hinted handoff of 0 rows to endpoint /192.168.1.216
 INFO [HintedHandoff:1] 2010-11-02 08:56:11,729 HintedHandOffManager.java (line 190) Started hinted handoff for endpoint /192.168.1.215
 INFO [MigrationStage:1] 2010-11-02 08:56:11,729 ColumnFamilyStore.java (line 228) read 0 from saved key cache
 INFO [HintedHandoff:1] 2010-11-02 08:56:11,729 HintedHandOffManager.java (line 246) Finished hinted handoff of 0 rows to endpoint /192.168.1.215
 INFO [MigrationStage:1] 2010-11-02 08:56:11,731 ColumnFamilyStore.java (line 631) switching in a fresh Memtable for Migrations at CommitLogContext(file='/var/log/cassandra/commitlog/CommitLog-1288706169321.log', position=57186)
 INFO [MigrationStage:1] 2010-11-02 08:56:11,732 ColumnFamilyStore.java (line 934) Enqueuing flush of Memtable-Migrations@1100275801(6953 bytes, 1 operations)
 INFO [MigrationStage:1] 2010-11-02 08:56:11,732 ColumnFamilyStore.java (line 631) switching in a fresh Memtable for Schema at CommitLogContext(file='/var/log/cassandra/commitlog/CommitLog-1288706169321.log', position=57186)
 INFO [FlushWriter:1] 2010-11-02 08:56:11,732 Memtable.java (line 167) Writing Memtable-Migrations@1100275801(6953 bytes, 1 operations)
 INFO [MigrationStage:1] 2010-11-02 08:56:11,732 ColumnFamilyStore.java (line 934) Enqueuing flush of Memtable-Schema@353703090(2921 bytes, 3 operations)
 INFO [CompactionExecutor:1] 2010-11-02 08:56:11,745 CompactionManager.java (line 303) Compacted to /var/lib/cassandra/data1/system/Migrations-tmp-e-5-Data.db.  26,394 to 25,995 (~98% of original) bytes for 1 keys.  Time: 315ms.
 INFO [CompactionExecutor:1] 2010-11-02 08:56:11,746 CompactionManager.java (line 224) Compacting [org.apache.cassandra.io.sstable.SSTableReader(path='/var/lib/cassandra/data2/system/Schema-e-1-Data.db'),org.apache.cassandra.io.sstable.SSTableReader(path='/var/lib/cassandra/data1/system/Schema-e-2-Data.db'),org.apache.cassandra.io.sstable.SSTableReader(path='/var/lib/cassandra/data2/system/Schema-e-3-Data.db'),org.apache.cassandra.io.sstable.SSTableReader(path='/var/lib/cassandra/data1/system/Schema-e-4-Data.db')]
 INFO [GossipStage:1] 2010-11-02 08:56:11,938 Gossiper.java (line 569) InetAddress /192.168.1.219 is now UP
 INFO [HintedHandoff:1] 2010-11-02 08:56:11,938 HintedHandOffManager.java (line 190) Started hinted handoff for endpoint /192.168.1.219
 INFO [HintedHandoff:1] 2010-11-02 08:56:11,939 HintedHandOffManager.java (line 246) Finished hinted handoff of 0 rows to endpoint /192.168.1.219
 INFO [FlushWriter:1] 2010-11-02 08:56:12,056 Memtable.java (line 174) Completed flushing /var/lib/cassandra/data2/system/Migrations-e-6-Data.db
 INFO [FlushWriter:1] 2010-11-02 08:56:12,057 Memtable.java (line 167) Writing Memtable-Schema@353703090(2921 bytes, 3 operations)
ERROR [MutationStage:1] 2010-11-02 08:56:12,135 RowMutationVerbHandler.java (line 83) Error in row mutation
org.apache.cassandra.db.UnserializableColumnFamilyException: Couldn't find cfId=1016
        at org.apache.cassandra.db.ColumnFamilySerializer.deserialize(ColumnFamilySerializer.java:112)
        at org.apache.cassandra.db.RowMutationSerializer.defreezeTheMaps(RowMutation.java:368)
        at org.apache.cassandra.db.RowMutationSerializer.deserialize(RowMutation.java:378)
        at org.apache.cassandra.db.RowMutationSerializer.deserialize(RowMutation.java:336)
        at org.apache.cassandra.db.RowMutationVerbHandler.doVerb(RowMutationVerbHandler.java:52)
        at org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:62)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
        at java.lang.Thread.run(Thread.java:619)
 INFO [CompactionExecutor:1] 2010-11-02 08:56:12,140 CompactionManager.java (line 303) Compacted to /var/lib/cassandra/data2/system/Schema-tmp-e-5-Data.db.  11,016 to 10,482 (~95% of original) bytes for 5 keys.  Time: 391ms.
 INFO [FlushWriter:1] 2010-11-02 08:56:12,269 Memtable.java (line 174) Completed flushing /var/lib/cassandra/data1/system/Schema-e-6-Data.db
 INFO [MigrationStage:1] 2010-11-02 08:56:12,270 ColumnFamilyStore.java (line 228) read 0 from saved key cache
 INFO [MigrationStage:1] 2010-11-02 08:56:12,272 ColumnFamilyStore.java (line 631) switching in a fresh Memtable for Migrations at CommitLogContext(file='/var/log/cassandra/commitlog/CommitLog-1288706169321.log', position=68838)
 INFO [MigrationStage:1] 2010-11-02 08:56:12,273 ColumnFamilyStore.java (line 934) Enqueuing flush of Memtable-Migrations@1869820356(7125 bytes, 1 operations)
 INFO [MigrationStage:1] 2010-11-02 08:56:12,273 ColumnFamilyStore.java (line 631) switching in a fresh Memtable for Schema at CommitLogContext(file='/var/log/cassandra/commitlog/CommitLog-1288706169321.log', position=68838)
 INFO [FlushWriter:1] 2010-11-02 08:56:12,273 Memtable.java (line 167) Writing Memtable-Migrations@1869820356(7125 bytes, 1 operations)
 INFO [MigrationStage:1] 2010-11-02 08:56:12,273 ColumnFamilyStore.java (line 934) Enqueuing flush of Memtable-Schema@1942516741(3098 bytes, 3 operations)
 INFO [GossipStage:1] 2010-11-02 08:56:12,330 Gossiper.java (line 569) InetAddress /192.168.1.218 is now UP
 INFO [HintedHandoff:1] 2010-11-02 08:56:12,330 HintedHandOffManager.java (line 190) Started hinted handoff for endpoint /192.168.1.218
 INFO [HintedHandoff:1] 2010-11-02 08:56:12,331 HintedHandOffManager.java (line 246) Finished hinted handoff of 0 rows to endpoint /192.168.1.218
 INFO [FlushWriter:1] 2010-11-02 08:56:12,473 Memtable.java (line 174) Completed flushing /var/lib/cassandra/data2/system/Migrations-e-7-Data.db
 INFO [FlushWriter:1] 2010-11-02 08:56:12,474 Memtable.java (line 167) Writing Memtable-Schema@1942516741(3098 bytes, 3 operations)
ERROR [MutationStage:2] 2010-11-02 08:56:12,584 RowMutationVerbHandler.java (line 83) Error in row mutation
org.apache.cassandra.db.UnserializableColumnFamilyException: Couldn't find cfId=1004
        at org.apache.cassandra.db.ColumnFamilySerializer.deserialize(ColumnFamilySerializer.java:112)
        at org.apache.cassandra.db.RowMutationSerializer.defreezeTheMaps(RowMutation.java:368)
        at org.apache.cassandra.db.RowMutationSerializer.deserialize(RowMutation.java:378)
        at org.apache.cassandra.db.RowMutationSerializer.deserialize(RowMutation.java:336)
        at org.apache.cassandra.db.RowMutationVerbHandler.doVerb(RowMutationVerbHandler.java:52)
        at org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:62)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
        at java.lang.Thread.run(Thread.java:619)
 INFO [FlushWriter:1] 2010-11-02 08:56:12,619 Memtable.java (line 174) Completed flushing /var/lib/cassandra/data1/system/Schema-e-7-Data.db
 INFO [MigrationStage:1] 2010-11-02 08:56:12,621 ColumnFamilyStore.java (line 228) read 0 from saved key cache
 INFO [MigrationStage:1] 2010-11-02 08:56:12,622 ColumnFamilyStore.java (line 631) switching in a fresh Memtable for Migrations at CommitLogContext(file='/var/log/cassandra/commitlog/CommitLog-1288706169321.log', position=80679)
 INFO [MigrationStage:1] 2010-11-02 08:56:12,623 ColumnFamilyStore.java (line 934) Enqueuing flush of Memtable-Migrations@461456971(7314 bytes, 1 operations)
 INFO [MigrationStage:1] 2010-11-02 08:56:12,623 ColumnFamilyStore.java (line 631) switching in a fresh Memtable for Schema at CommitLogContext(file='/var/log/cassandra/commitlog/CommitLog-1288706169321.log', position=80679)
 INFO [FlushWriter:1] 2010-11-02 08:56:12,623 Memtable.java (line 167) Writing Memtable-Migrations@461456971(7314 bytes, 1 operations)
 INFO [MigrationStage:1] 2010-11-02 08:56:12,623 ColumnFamilyStore.java (line 934) Enqueuing flush of Memtable-Schema@1143862280(3281 bytes, 3 operations)
 INFO [FlushWriter:1] 2010-11-02 08:56:12,815 Memtable.java (line 174) Completed flushing /var/lib/cassandra/data2/system/Migrations-e-8-Data.db
 INFO [CompactionExecutor:1] 2010-11-02 08:56:12,816 CompactionManager.java (line 224) Compacting [org.apache.cassandra.io.sstable.SSTableReader(path='/var/lib/cassandra/data1/system/Migrations-e-5-Data.db'),org.apache.cassandra.io.sstable.SSTableReader(path='/var/lib/cassandra/data2/system/Migrations-e-6-Data.db'),org.apache.cassandra.io.sstable.SSTableReader(path='/var/lib/cassandra/data2/system/Migrations-e-7-Data.db'),org.apache.cassandra.io.sstable.SSTableReader(path='/var/lib/cassandra/data2/system/Migrations-e-8-Data.db')]

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Resolved: (CASSANDRA-1695) Bootstrapping new node Error in row mutation UnserializableColumnFamilyException Couldn't find cfId

Posted by "Gary Dusbabek (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/CASSANDRA-1695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Gary Dusbabek resolved CASSANDRA-1695.
--------------------------------------

       Resolution: Invalid
    Fix Version/s: 0.7 beta 3

The bootstrapping node was configured as a seed, which means it won't really bootstrap, and in effect, is treated like a regular node with autobootstrap=false.

> Bootstrapping new node Error in row mutation UnserializableColumnFamilyException Couldn't find cfId
> ---------------------------------------------------------------------------------------------------
>
>                 Key: CASSANDRA-1695
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-1695
>             Project: Cassandra
>          Issue Type: Bug
>    Affects Versions: 0.7 beta 3
>            Reporter: Jake Farrell
>             Fix For: 0.7 beta 3
>
>
> When auto_bootstraping a new node into an existing cluster with data (auto_bootstrap: true) the new node throws UnserializableColumnFamilyException: Couldn't find cfId. Started with beta3 and then tried latest trunk r1029823.  Created a cluster of 5 nodes all using default settings other than ip addresses, OP partitioner, and data dir locations. started writing data to the cluster. On 6th node changed auto_bootstrap setting to true and started 6th node. Received the cfid row mutation error after gossiper finishes and schema is being shared with new node. 
> INFO [main] 2010-11-02 08:56:09,150 AbstractCassandraDaemon.java (line 72) Heap size: 10719985664/10719985664
>  INFO [main] 2010-11-02 08:56:09,160 CLibrary.java (line 43) JNA not found. Native methods will be disabled.
>  INFO [main] 2010-11-02 08:56:09,167 DatabaseDescriptor.java (line 122) Loading settings from file:/opt/cassandra/0.7.0-beta3/conf/cassandra.yaml
>  INFO [main] 2010-11-02 08:56:09,262 DatabaseDescriptor.java (line 173) DiskAccessMode 'auto' determined to be mmap, indexAccessMode is mmap
>  INFO [main] 2010-11-02 08:56:09,321 CommitLogSegment.java (line 50) Creating new commitlog segment /var/log/cassandra/commitlog/CommitLog-1288706169321.log
>  INFO [main] 2010-11-02 08:56:09,406 ColumnFamilyStore.java (line 228) read 0 from saved key cache
>  INFO [main] 2010-11-02 08:56:09,408 ColumnFamilyStore.java (line 228) read 0 from saved key cache
>  INFO [main] 2010-11-02 08:56:09,409 ColumnFamilyStore.java (line 228) read 0 from saved key cache
>  INFO [main] 2010-11-02 08:56:09,410 ColumnFamilyStore.java (line 228) read 0 from saved key cache
>  INFO [main] 2010-11-02 08:56:09,411 ColumnFamilyStore.java (line 228) read 0 from saved key cache
>  INFO [main] 2010-11-02 08:56:09,413 ColumnFamilyStore.java (line 504) loading row cache for LocationInfo of system
>  INFO [main] 2010-11-02 08:56:09,414 ColumnFamilyStore.java (line 509) completed loading (1 ms; 0 keys)  row cache for LocationInfo of system
>  INFO [main] 2010-11-02 08:56:09,414 ColumnFamilyStore.java (line 504) loading row cache for HintsColumnFamily of system
>  INFO [main] 2010-11-02 08:56:09,414 ColumnFamilyStore.java (line 509) completed loading (0 ms; 0 keys)  row cache for HintsColumnFamily of system
>  INFO [main] 2010-11-02 08:56:09,415 ColumnFamilyStore.java (line 504) loading row cache for Migrations of system
>  INFO [main] 2010-11-02 08:56:09,415 ColumnFamilyStore.java (line 509) completed loading (0 ms; 0 keys)  row cache for Migrations of system
>  INFO [main] 2010-11-02 08:56:09,417 ColumnFamilyStore.java (line 504) loading row cache for Schema of system
>  INFO [main] 2010-11-02 08:56:09,417 ColumnFamilyStore.java (line 509) completed loading (0 ms; 0 keys)  row cache for Schema of system
>  INFO [main] 2010-11-02 08:56:09,418 ColumnFamilyStore.java (line 504) loading row cache for IndexInfo of system
>  INFO [main] 2010-11-02 08:56:09,418 ColumnFamilyStore.java (line 509) completed loading (1 ms; 0 keys)  row cache for IndexInfo of system
>  INFO [main] 2010-11-02 08:56:09,442 DatabaseDescriptor.java (line 408) Couldn't detect any schema definitions in local storage.
>  INFO [main] 2010-11-02 08:56:09,442 DatabaseDescriptor.java (line 434) Found table data in data directories. Consider using JMX to call org.apache.cassandra.service.StorageService.loadSchemaFromYaml().
>  INFO [main] 2010-11-02 08:56:09,445 CommitLog.java (line 187) No commitlog files found; skipping replay
>  INFO [main] 2010-11-02 08:56:09,461 SystemTable.java (line 80) Upgrading to 0.7. Purging hints if there are any. Old hints will be snapshotted.
>  INFO [main] 2010-11-02 08:56:09,465 StorageService.java (line 347) Cassandra version: 0.7.0-beta3
>  INFO [main] 2010-11-02 08:56:09,465 StorageService.java (line 348) Thrift API version: 19.4.0
>  INFO [main] 2010-11-02 08:56:09,472 StorageService.java (line 370) Loading persisted ring state
>  INFO [main] 2010-11-02 08:56:09,473 StorageService.java (line 378) Starting up server gossip
>  INFO [main] 2010-11-02 08:56:09,478 ColumnFamilyStore.java (line 631) switching in a fresh Memtable for LocationInfo at CommitLogContext(file='/var/log/cassandra/commitlog/CommitLog-1288706169321.log', position=700)
>  INFO [main] 2010-11-02 08:56:09,479 ColumnFamilyStore.java (line 934) Enqueuing flush of Memtable-LocationInfo@648734830(236 bytes, 4 operations)
>  INFO [FlushWriter:1] 2010-11-02 08:56:09,480 Memtable.java (line 167) Writing Memtable-LocationInfo@648734830(236 bytes, 4 operations)
>  INFO [FlushWriter:1] 2010-11-02 08:56:09,705 Memtable.java (line 174) Completed flushing /var/lib/cassandra/data1/system/LocationInfo-e-1-Data.db
>  INFO [main] 2010-11-02 08:56:09,735 StorageService.java (line 394) This node will not auto bootstrap because it is configured to be a seed node.
>  WARN [main] 2010-11-02 08:56:09,740 StorageService.java (line 434) Generated random token 4buaxINos8oCXqym. Random tokens will result in an unbalanced ring; see http://wiki.apache.org/cassandra/Operations
>  INFO [main] 2010-11-02 08:56:09,740 ColumnFamilyStore.java (line 631) switching in a fresh Memtable for LocationInfo at CommitLogContext(file='/var/log/cassandra/commitlog/CommitLog-1288706169321.log', position=848)
>  INFO [main] 2010-11-02 08:56:09,741 ColumnFamilyStore.java (line 934) Enqueuing flush of Memtable-LocationInfo@1159466336(36 bytes, 1 operations)
>  INFO [FlushWriter:1] 2010-11-02 08:56:09,741 Memtable.java (line 167) Writing Memtable-LocationInfo@1159466336(36 bytes, 1 operations)
>  INFO [FlushWriter:1] 2010-11-02 08:56:09,979 Memtable.java (line 174) Completed flushing /var/lib/cassandra/data2/system/LocationInfo-e-2-Data.db
>  INFO [MigrationStage:1] 2010-11-02 08:56:09,999 ColumnFamilyStore.java (line 631) switching in a fresh Memtable for Migrations at CommitLogContext(file='/var/log/cassandra/commitlog/CommitLog-1288706169321.log', position=11469)
>  INFO [MigrationStage:1] 2010-11-02 08:56:09,999 ColumnFamilyStore.java (line 934) Enqueuing flush of Memtable-Migrations@1518579727(6094 bytes, 1 operations)
>  INFO [MigrationStage:1] 2010-11-02 08:56:10,000 ColumnFamilyStore.java (line 631) switching in a fresh Memtable for Schema at CommitLogContext(file='/var/log/cassandra/commitlog/CommitLog-1288706169321.log', position=11469)
>  INFO [MigrationStage:1] 2010-11-02 08:56:10,000 ColumnFamilyStore.java (line 934) Enqueuing flush of Memtable-Schema@1386432387(2192 bytes, 3 operations)
>  INFO [FlushWriter:1] 2010-11-02 08:56:10,000 Memtable.java (line 167) Writing Memtable-Migrations@1518579727(6094 bytes, 1 operations)
>  INFO [main] 2010-11-02 08:56:10,005 Mx4jTool.java (line 73) Will not load MX4J, mx4j-tools.jar is not in the classpath
>  INFO [main] 2010-11-02 08:56:10,055 CassandraDaemon.java (line 77) Binding thrift service to /192.168.1.94:9160
>  INFO [main] 2010-11-02 08:56:10,057 CassandraDaemon.java (line 91) Using TFramedTransport with a max frame size of 15728640 bytes.
>  INFO [main] 2010-11-02 08:56:10,061 CassandraDaemon.java (line 119) Listening for thrift clients...
>  INFO [FlushWriter:1] 2010-11-02 08:56:10,168 Memtable.java (line 174) Completed flushing /var/lib/cassandra/data1/system/Migrations-e-1-Data.db
>  INFO [FlushWriter:1] 2010-11-02 08:56:10,169 Memtable.java (line 167) Writing Memtable-Schema@1386432387(2192 bytes, 3 operations)
>  INFO [FlushWriter:1] 2010-11-02 08:56:10,404 Memtable.java (line 174) Completed flushing /var/lib/cassandra/data2/system/Schema-e-1-Data.db
>  INFO [MigrationStage:1] 2010-11-02 08:56:10,444 ColumnFamilyStore.java (line 631) switching in a fresh Memtable for Migrations at CommitLogContext(file='/var/log/cassandra/commitlog/CommitLog-1288706169321.log', position=22551)
>  INFO [MigrationStage:1] 2010-11-02 08:56:10,445 ColumnFamilyStore.java (line 934) Enqueuing flush of Memtable-Migrations@668531077(6407 bytes, 1 operations)
>  INFO [MigrationStage:1] 2010-11-02 08:56:10,445 ColumnFamilyStore.java (line 631) switching in a fresh Memtable for Schema at CommitLogContext(file='/var/log/cassandra/commitlog/CommitLog-1288706169321.log', position=22551)
>  INFO [MigrationStage:1] 2010-11-02 08:56:10,445 ColumnFamilyStore.java (line 934) Enqueuing flush of Memtable-Schema@1709782292(2375 bytes, 3 operations)
>  INFO [FlushWriter:1] 2010-11-02 08:56:10,450 Memtable.java (line 167) Writing Memtable-Migrations@668531077(6407 bytes, 1 operations)
>  INFO [FlushWriter:1] 2010-11-02 08:56:10,680 Memtable.java (line 174) Completed flushing /var/lib/cassandra/data2/system/Migrations-e-2-Data.db
>  INFO [FlushWriter:1] 2010-11-02 08:56:10,681 Memtable.java (line 167) Writing Memtable-Schema@1709782292(2375 bytes, 3 operations)
>  INFO [FlushWriter:1] 2010-11-02 08:56:10,851 Memtable.java (line 174) Completed flushing /var/lib/cassandra/data1/system/Schema-e-2-Data.db
>  INFO [MigrationStage:1] 2010-11-02 08:56:10,853 ColumnFamilyStore.java (line 228) read 0 from saved key cache
>  INFO [MigrationStage:1] 2010-11-02 08:56:10,855 ColumnFamilyStore.java (line 631) switching in a fresh Memtable for Migrations at CommitLogContext(file='/var/log/cassandra/commitlog/CommitLog-1288706169321.log', position=33669)
>  INFO [MigrationStage:1] 2010-11-02 08:56:10,855 ColumnFamilyStore.java (line 934) Enqueuing flush of Memtable-Migrations@1180026905(6591 bytes, 1 operations)
>  INFO [MigrationStage:1] 2010-11-02 08:56:10,856 ColumnFamilyStore.java (line 631) switching in a fresh Memtable for Schema at CommitLogContext(file='/var/log/cassandra/commitlog/CommitLog-1288706169321.log', position=33669)
>  INFO [FlushWriter:1] 2010-11-02 08:56:10,856 Memtable.java (line 167) Writing Memtable-Migrations@1180026905(6591 bytes, 1 operations)
>  INFO [MigrationStage:1] 2010-11-02 08:56:10,856 ColumnFamilyStore.java (line 934) Enqueuing flush of Memtable-Schema@700712290(2558 bytes, 3 operations)
>  INFO [FlushWriter:1] 2010-11-02 08:56:11,010 Memtable.java (line 174) Completed flushing /var/lib/cassandra/data1/system/Migrations-e-3-Data.db
>  INFO [FlushWriter:1] 2010-11-02 08:56:11,011 Memtable.java (line 167) Writing Memtable-Schema@700712290(2558 bytes, 3 operations)
>  INFO [FlushWriter:1] 2010-11-02 08:56:11,206 Memtable.java (line 174) Completed flushing /var/lib/cassandra/data2/system/Schema-e-3-Data.db
>  INFO [MigrationStage:1] 2010-11-02 08:56:11,208 ColumnFamilyStore.java (line 228) read 0 from saved key cache
>  INFO [MigrationStage:1] 2010-11-02 08:56:11,210 ColumnFamilyStore.java (line 631) switching in a fresh Memtable for Migrations at CommitLogContext(file='/var/log/cassandra/commitlog/CommitLog-1288706169321.log', position=44966)
>  INFO [MigrationStage:1] 2010-11-02 08:56:11,210 ColumnFamilyStore.java (line 934) Enqueuing flush of Memtable-Migrations@1711957716(6770 bytes, 1 operations)
>  INFO [MigrationStage:1] 2010-11-02 08:56:11,210 ColumnFamilyStore.java (line 631) switching in a fresh Memtable for Schema at CommitLogContext(file='/var/log/cassandra/commitlog/CommitLog-1288706169321.log', position=44966)
>  INFO [FlushWriter:1] 2010-11-02 08:56:11,210 Memtable.java (line 167) Writing Memtable-Migrations@1711957716(6770 bytes, 1 operations)
>  INFO [MigrationStage:1] 2010-11-02 08:56:11,211 ColumnFamilyStore.java (line 934) Enqueuing flush of Memtable-Schema@887430306(2739 bytes, 3 operations)
>  INFO [GossipStage:1] 2010-11-02 08:56:11,329 Gossiper.java (line 577) Node /192.168.1.217 is now part of the cluster
>  INFO [GossipStage:1] 2010-11-02 08:56:11,337 Gossiper.java (line 577) Node /192.168.1.216 is now part of the cluster
>  INFO [GossipStage:1] 2010-11-02 08:56:11,338 Gossiper.java (line 577) Node /192.168.1.219 is now part of the cluster
>  INFO [GossipStage:1] 2010-11-02 08:56:11,338 Gossiper.java (line 577) Node /192.168.1.218 is now part of the cluster
>  INFO [GossipStage:1] 2010-11-02 08:56:11,338 Gossiper.java (line 577) Node /192.168.1.215 is now part of the cluster
>  INFO [FlushWriter:1] 2010-11-02 08:56:11,414 Memtable.java (line 174) Completed flushing /var/lib/cassandra/data2/system/Migrations-e-4-Data.db
>  INFO [FlushWriter:1] 2010-11-02 08:56:11,415 Memtable.java (line 167) Writing Memtable-Schema@887430306(2739 bytes, 3 operations)
>  INFO [CompactionExecutor:1] 2010-11-02 08:56:11,420 CompactionManager.java (line 224) Compacting [org.apache.cassandra.io.sstable.SSTableReader(path='/var/lib/cassandra/data1/system/Migrations-e-1-Data.db'),org.apache.cassandra.io.sstable.SSTableReader(path='/var/lib/cassandra/data2/system/Migrations-e-2-Data.db'),org.apache.cassandra.io.sstable.SSTableReader(path='/var/lib/cassandra/data1/system/Migrations-e-3-Data.db'),org.apache.cassandra.io.sstable.SSTableReader(path='/var/lib/cassandra/data2/system/Migrations-e-4-Data.db')]
>  INFO [GossipStage:1] 2010-11-02 08:56:11,727 Gossiper.java (line 569) InetAddress /192.168.1.216 is now UP
>  INFO [HintedHandoff:1] 2010-11-02 08:56:11,727 HintedHandOffManager.java (line 190) Started hinted handoff for endpoint /192.168.1.216
>  INFO [FlushWriter:1] 2010-11-02 08:56:11,727 Memtable.java (line 174) Completed flushing /var/lib/cassandra/data1/system/Schema-e-4-Data.db
>  INFO [GossipStage:1] 2010-11-02 08:56:11,728 Gossiper.java (line 569) InetAddress /192.168.1.215 is now UP
>  INFO [HintedHandoff:1] 2010-11-02 08:56:11,729 HintedHandOffManager.java (line 246) Finished hinted handoff of 0 rows to endpoint /192.168.1.216
>  INFO [HintedHandoff:1] 2010-11-02 08:56:11,729 HintedHandOffManager.java (line 190) Started hinted handoff for endpoint /192.168.1.215
>  INFO [MigrationStage:1] 2010-11-02 08:56:11,729 ColumnFamilyStore.java (line 228) read 0 from saved key cache
>  INFO [HintedHandoff:1] 2010-11-02 08:56:11,729 HintedHandOffManager.java (line 246) Finished hinted handoff of 0 rows to endpoint /192.168.1.215
>  INFO [MigrationStage:1] 2010-11-02 08:56:11,731 ColumnFamilyStore.java (line 631) switching in a fresh Memtable for Migrations at CommitLogContext(file='/var/log/cassandra/commitlog/CommitLog-1288706169321.log', position=57186)
>  INFO [MigrationStage:1] 2010-11-02 08:56:11,732 ColumnFamilyStore.java (line 934) Enqueuing flush of Memtable-Migrations@1100275801(6953 bytes, 1 operations)
>  INFO [MigrationStage:1] 2010-11-02 08:56:11,732 ColumnFamilyStore.java (line 631) switching in a fresh Memtable for Schema at CommitLogContext(file='/var/log/cassandra/commitlog/CommitLog-1288706169321.log', position=57186)
>  INFO [FlushWriter:1] 2010-11-02 08:56:11,732 Memtable.java (line 167) Writing Memtable-Migrations@1100275801(6953 bytes, 1 operations)
>  INFO [MigrationStage:1] 2010-11-02 08:56:11,732 ColumnFamilyStore.java (line 934) Enqueuing flush of Memtable-Schema@353703090(2921 bytes, 3 operations)
>  INFO [CompactionExecutor:1] 2010-11-02 08:56:11,745 CompactionManager.java (line 303) Compacted to /var/lib/cassandra/data1/system/Migrations-tmp-e-5-Data.db.  26,394 to 25,995 (~98% of original) bytes for 1 keys.  Time: 315ms.
>  INFO [CompactionExecutor:1] 2010-11-02 08:56:11,746 CompactionManager.java (line 224) Compacting [org.apache.cassandra.io.sstable.SSTableReader(path='/var/lib/cassandra/data2/system/Schema-e-1-Data.db'),org.apache.cassandra.io.sstable.SSTableReader(path='/var/lib/cassandra/data1/system/Schema-e-2-Data.db'),org.apache.cassandra.io.sstable.SSTableReader(path='/var/lib/cassandra/data2/system/Schema-e-3-Data.db'),org.apache.cassandra.io.sstable.SSTableReader(path='/var/lib/cassandra/data1/system/Schema-e-4-Data.db')]
>  INFO [GossipStage:1] 2010-11-02 08:56:11,938 Gossiper.java (line 569) InetAddress /192.168.1.219 is now UP
>  INFO [HintedHandoff:1] 2010-11-02 08:56:11,938 HintedHandOffManager.java (line 190) Started hinted handoff for endpoint /192.168.1.219
>  INFO [HintedHandoff:1] 2010-11-02 08:56:11,939 HintedHandOffManager.java (line 246) Finished hinted handoff of 0 rows to endpoint /192.168.1.219
>  INFO [FlushWriter:1] 2010-11-02 08:56:12,056 Memtable.java (line 174) Completed flushing /var/lib/cassandra/data2/system/Migrations-e-6-Data.db
>  INFO [FlushWriter:1] 2010-11-02 08:56:12,057 Memtable.java (line 167) Writing Memtable-Schema@353703090(2921 bytes, 3 operations)
> ERROR [MutationStage:1] 2010-11-02 08:56:12,135 RowMutationVerbHandler.java (line 83) Error in row mutation
> org.apache.cassandra.db.UnserializableColumnFamilyException: Couldn't find cfId=1016
>         at org.apache.cassandra.db.ColumnFamilySerializer.deserialize(ColumnFamilySerializer.java:112)
>         at org.apache.cassandra.db.RowMutationSerializer.defreezeTheMaps(RowMutation.java:368)
>         at org.apache.cassandra.db.RowMutationSerializer.deserialize(RowMutation.java:378)
>         at org.apache.cassandra.db.RowMutationSerializer.deserialize(RowMutation.java:336)
>         at org.apache.cassandra.db.RowMutationVerbHandler.doVerb(RowMutationVerbHandler.java:52)
>         at org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:62)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>         at java.lang.Thread.run(Thread.java:619)
>  INFO [CompactionExecutor:1] 2010-11-02 08:56:12,140 CompactionManager.java (line 303) Compacted to /var/lib/cassandra/data2/system/Schema-tmp-e-5-Data.db.  11,016 to 10,482 (~95% of original) bytes for 5 keys.  Time: 391ms.
>  INFO [FlushWriter:1] 2010-11-02 08:56:12,269 Memtable.java (line 174) Completed flushing /var/lib/cassandra/data1/system/Schema-e-6-Data.db
>  INFO [MigrationStage:1] 2010-11-02 08:56:12,270 ColumnFamilyStore.java (line 228) read 0 from saved key cache
>  INFO [MigrationStage:1] 2010-11-02 08:56:12,272 ColumnFamilyStore.java (line 631) switching in a fresh Memtable for Migrations at CommitLogContext(file='/var/log/cassandra/commitlog/CommitLog-1288706169321.log', position=68838)
>  INFO [MigrationStage:1] 2010-11-02 08:56:12,273 ColumnFamilyStore.java (line 934) Enqueuing flush of Memtable-Migrations@1869820356(7125 bytes, 1 operations)
>  INFO [MigrationStage:1] 2010-11-02 08:56:12,273 ColumnFamilyStore.java (line 631) switching in a fresh Memtable for Schema at CommitLogContext(file='/var/log/cassandra/commitlog/CommitLog-1288706169321.log', position=68838)
>  INFO [FlushWriter:1] 2010-11-02 08:56:12,273 Memtable.java (line 167) Writing Memtable-Migrations@1869820356(7125 bytes, 1 operations)
>  INFO [MigrationStage:1] 2010-11-02 08:56:12,273 ColumnFamilyStore.java (line 934) Enqueuing flush of Memtable-Schema@1942516741(3098 bytes, 3 operations)
>  INFO [GossipStage:1] 2010-11-02 08:56:12,330 Gossiper.java (line 569) InetAddress /192.168.1.218 is now UP
>  INFO [HintedHandoff:1] 2010-11-02 08:56:12,330 HintedHandOffManager.java (line 190) Started hinted handoff for endpoint /192.168.1.218
>  INFO [HintedHandoff:1] 2010-11-02 08:56:12,331 HintedHandOffManager.java (line 246) Finished hinted handoff of 0 rows to endpoint /192.168.1.218
>  INFO [FlushWriter:1] 2010-11-02 08:56:12,473 Memtable.java (line 174) Completed flushing /var/lib/cassandra/data2/system/Migrations-e-7-Data.db
>  INFO [FlushWriter:1] 2010-11-02 08:56:12,474 Memtable.java (line 167) Writing Memtable-Schema@1942516741(3098 bytes, 3 operations)
> ERROR [MutationStage:2] 2010-11-02 08:56:12,584 RowMutationVerbHandler.java (line 83) Error in row mutation
> org.apache.cassandra.db.UnserializableColumnFamilyException: Couldn't find cfId=1004
>         at org.apache.cassandra.db.ColumnFamilySerializer.deserialize(ColumnFamilySerializer.java:112)
>         at org.apache.cassandra.db.RowMutationSerializer.defreezeTheMaps(RowMutation.java:368)
>         at org.apache.cassandra.db.RowMutationSerializer.deserialize(RowMutation.java:378)
>         at org.apache.cassandra.db.RowMutationSerializer.deserialize(RowMutation.java:336)
>         at org.apache.cassandra.db.RowMutationVerbHandler.doVerb(RowMutationVerbHandler.java:52)
>         at org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:62)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>         at java.lang.Thread.run(Thread.java:619)
>  INFO [FlushWriter:1] 2010-11-02 08:56:12,619 Memtable.java (line 174) Completed flushing /var/lib/cassandra/data1/system/Schema-e-7-Data.db
>  INFO [MigrationStage:1] 2010-11-02 08:56:12,621 ColumnFamilyStore.java (line 228) read 0 from saved key cache
>  INFO [MigrationStage:1] 2010-11-02 08:56:12,622 ColumnFamilyStore.java (line 631) switching in a fresh Memtable for Migrations at CommitLogContext(file='/var/log/cassandra/commitlog/CommitLog-1288706169321.log', position=80679)
>  INFO [MigrationStage:1] 2010-11-02 08:56:12,623 ColumnFamilyStore.java (line 934) Enqueuing flush of Memtable-Migrations@461456971(7314 bytes, 1 operations)
>  INFO [MigrationStage:1] 2010-11-02 08:56:12,623 ColumnFamilyStore.java (line 631) switching in a fresh Memtable for Schema at CommitLogContext(file='/var/log/cassandra/commitlog/CommitLog-1288706169321.log', position=80679)
>  INFO [FlushWriter:1] 2010-11-02 08:56:12,623 Memtable.java (line 167) Writing Memtable-Migrations@461456971(7314 bytes, 1 operations)
>  INFO [MigrationStage:1] 2010-11-02 08:56:12,623 ColumnFamilyStore.java (line 934) Enqueuing flush of Memtable-Schema@1143862280(3281 bytes, 3 operations)
>  INFO [FlushWriter:1] 2010-11-02 08:56:12,815 Memtable.java (line 174) Completed flushing /var/lib/cassandra/data2/system/Migrations-e-8-Data.db
>  INFO [CompactionExecutor:1] 2010-11-02 08:56:12,816 CompactionManager.java (line 224) Compacting [org.apache.cassandra.io.sstable.SSTableReader(path='/var/lib/cassandra/data1/system/Migrations-e-5-Data.db'),org.apache.cassandra.io.sstable.SSTableReader(path='/var/lib/cassandra/data2/system/Migrations-e-6-Data.db'),org.apache.cassandra.io.sstable.SSTableReader(path='/var/lib/cassandra/data2/system/Migrations-e-7-Data.db'),org.apache.cassandra.io.sstable.SSTableReader(path='/var/lib/cassandra/data2/system/Migrations-e-8-Data.db')]

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.