You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by Ramesh Natarajan <ra...@gmail.com> on 2011/11/22 17:35:41 UTC
Assertion error during bootstraping cassandra 1.0.2
Hi,
I have a 3 node cassandra cluster. I have RF set to 3 and do reads
and writes using QUORUM.
Here is my initial ring configuration
[root@CAP4-CNode1 ~]# /root/cassandra/bin/nodetool -h localhost ring
Address DC Rack Status State Load
Owns Token
113427455640312821154458202477256070484
10.19.104.11 datacenter1 rack1 Up Normal 1.66 GB
33.33% 0
10.19.104.12 datacenter1 rack1 Up Normal 1.06 GB
33.33% 56713727820156410577229101238628035242
10.19.104.13 datacenter1 rack1 Up Normal 1.61 GB
33.33% 113427455640312821154458202477256070484
I want to add 10.19.104.14 to the cluster.
I edited the 10.19.104.14 cassandra.yaml file and set the token to
127605887595351923798765477786913079296 and set auto_bootstrap to
true.
When I started cassandra I am getting Assertion Error. Can someone
point to me if i am doing anything wrong here?
thanks
Ramesh
[root@CAP4-CNode4 cassandra]# INFO 10:29:46,093 Logging initialized
INFO 10:29:46,099 JVM vendor/version: Java HotSpot(TM) 64-Bit Server
VM/1.6.0_25
INFO 10:29:46,100 Heap size: 8304721920/8304721920
INFO 10:29:46,100 Classpath:
bin/../conf:bin/../build/classes/main:bin/../build/classes/thrift:bin/../lib/antlr-3.2.jar:bin/../lib/apache-cassandra-1.0.2.jar:bin/../lib/apache-cassandra-clientutil-1.0.2.jar:bin/../lib/apache-cassandra-thrift-1.0.2.jar:bin/../lib/avro-1.4.0-fixes.jar:bin/../lib/avro-1.4.0-sources-fixes.jar:bin/../lib/commons-cli-1.1.jar:bin/../lib/commons-codec-1.2.jar:bin/../lib/commons-lang-2.4.jar:bin/../lib/compress-lzf-0.8.4.jar:bin/../lib/concurrentlinkedhashmap-lru-1.2.jar:bin/../lib/guava-r08.jar:bin/../lib/high-scale-lib-1.1.2.jar:bin/../lib/jackson-core-asl-1.4.0.jar:bin/../lib/jackson-mapper-asl-1.4.0.jar:bin/../lib/jamm-0.2.5.jar:bin/../lib/jline-0.9.94.jar:bin/../lib/jna.jar:bin/../lib/json-simple-1.1.jar:bin/../lib/libthrift-0.6.jar:bin/../lib/log4j-1.2.16.jar:bin/../lib/mx4j-examples.jar:bin/../lib/mx4j-impl.jar:bin/../lib/mx4j.jar:bin/../lib/mx4j-jmx.jar:bin/../lib/mx4j-remote.jar:bin/../lib/mx4j-rimpl.jar:bin/../lib/mx4j-rjmx.jar:bin/../lib/mx4j-tools.jar:bin/../lib/servlet-api-2.5-20081211.jar:bin/../lib/slf4j-api-1.6.1.jar:bin/../lib/slf4j-log4j12-1.6.1.jar:bin/../lib/snakeyaml-1.6.jar:bin/../lib/snappy-java-1.0.4.1.jar:bin/../lib/jamm-0.2.5.jar
INFO 10:29:48,713 JNA mlockall successful
INFO 10:29:48,726 Loading settings from
file:/root/apache-cassandra-1.0.2/conf/cassandra.yaml
INFO 10:29:48,883 DiskAccessMode 'auto' determined to be mmap,
indexAccessMode is mmap
INFO 10:29:48,898 Global memtable threshold is enabled at 2640MB
INFO 10:29:49,203 Couldn't detect any schema definitions in local storage.
INFO 10:29:49,204 Found table data in data directories. Consider
using the CLI to define your schema.
INFO 10:29:49,220 Creating new commitlog segment
/var/lib/cassandra/commitlog/CommitLog-1321979389220.log
INFO 10:29:49,227 No commitlog files found; skipping replay
INFO 10:29:49,230 Cassandra version: 1.0.2
INFO 10:29:49,230 Thrift API version: 19.18.0
INFO 10:29:49,230 Loading persisted ring state
INFO 10:29:49,235 Starting up server gossip
INFO 10:29:49,259 Enqueuing flush of
Memtable-LocationInfo@122130810(192/240 serialized/live bytes, 4 ops)
INFO 10:29:49,260 Writing Memtable-LocationInfo@122130810(192/240
serialized/live bytes, 4 ops)
INFO 10:29:49,317 Completed flushing
/var/lib/cassandra/data/system/LocationInfo-h-1-Data.db (300 bytes)
INFO 10:29:49,340 Starting Messaging Service on port 7000
INFO 10:29:49,349 JOINING: waiting for ring and schema information
INFO 10:29:50,759 Applying migration
4b0e20f0-1511-11e1-0000-c11bc95834d7 Add keyspace: MSA, rep
strategy:SimpleStrategy{}, durable_writes: true
INFO 10:29:50,761 Enqueuing flush of
Memtable-Migrations@1507565381(6744/8430 serialized/live bytes, 1 ops)
INFO 10:29:50,761 Writing Memtable-Migrations@1507565381(6744/8430
serialized/live bytes, 1 ops)
INFO 10:29:50,761 Enqueuing flush of
Memtable-Schema@1498835564(2889/3611 serialized/live bytes, 3 ops)
INFO 10:29:50,776 Completed flushing
/var/lib/cassandra/data/system/Migrations-h-1-Data.db (6808 bytes)
INFO 10:29:50,777 Writing Memtable-Schema@1498835564(2889/3611
serialized/live bytes, 3 ops)
INFO 10:29:50,797 Completed flushing
/var/lib/cassandra/data/system/Schema-h-1-Data.db (3039 bytes)
INFO 10:29:50,814 Applying migration
4b6f2cb0-1511-11e1-0000-c11bc95834d7 Add column family:
org.apache.cassandra.config.CFMetaData@1639d811[cfId=1000,ksName=MSA,cfName=modseq,cfType=Standard,comparator=org.apache.cassandra.db.marshal.ReversedType(org.apache.cassandra.db.marshal.BytesType),subcolumncomparator=<null>,comment=,rowCacheSize=0.0,keyCacheSize=5000000.0,readRepairChance=1.0,replicateOnWrite=true,gcGraceSeconds=3600,defaultValidator=org.apache.cassandra.db.marshal.BytesType,keyValidator=org.apache.cassandra.db.marshal.BytesType,minCompactionThreshold=4,maxCompactionThreshold=32,rowCacheSavePeriodInSeconds=0,keyCacheSavePeriodInSeconds=14400,rowCacheKeysToSave=2147483647,rowCacheProvider=org.apache.cassandra.cache.SerializingCacheProvider@2f984f7d,mergeShardsChance=0.1,keyAlias=<null>,column_metadata={},compactionStrategyClass=class
org.apache.cassandra.db.compaction.LeveledCompactionStrategy,compactionStrategyOptions={sstable_size_in_mb=10},compressionOptions={}]
INFO 10:29:50,815 Enqueuing flush of
Memtable-Migrations@948613108(7482/9352 serialized/live bytes, 1 ops)
INFO 10:29:50,816 Writing Memtable-Migrations@948613108(7482/9352
serialized/live bytes, 1 ops)
INFO 10:29:50,816 Enqueuing flush of
Memtable-Schema@421910828(3294/4117 serialized/live bytes, 3 ops)
INFO 10:29:50,831 Completed flushing
/var/lib/cassandra/data/system/Migrations-h-2-Data.db (7546 bytes)
INFO 10:29:50,832 Writing Memtable-Schema@421910828(3294/4117
serialized/live bytes, 3 ops)
INFO 10:29:50,846 Completed flushing
/var/lib/cassandra/data/system/Schema-h-2-Data.db (3444 bytes)
INFO 10:29:50,854 Applying migration
4b8c9fc0-1511-11e1-0000-c11bc95834d7 Add column family:
org.apache.cassandra.config.CFMetaData@1bd97d0d[cfId=1001,ksName=MSA,cfName=msgid,cfType=Standard,comparator=org.apache.cassandra.db.marshal.BytesType,subcolumncomparator=<null>,comment=,rowCacheSize=0.0,keyCacheSize=1000000.0,readRepairChance=1.0,replicateOnWrite=true,gcGraceSeconds=3600,defaultValidator=org.apache.cassandra.db.marshal.BytesType,keyValidator=org.apache.cassandra.db.marshal.BytesType,minCompactionThreshold=4,maxCompactionThreshold=32,rowCacheSavePeriodInSeconds=0,keyCacheSavePeriodInSeconds=14400,rowCacheKeysToSave=2147483647,rowCacheProvider=org.apache.cassandra.cache.SerializingCacheProvider@63a0eec3,mergeShardsChance=0.1,keyAlias=<null>,column_metadata={},compactionStrategyClass=class
org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy,compactionStrategyOptions={},compressionOptions={}]
INFO 10:29:50,855 Enqueuing flush of
Memtable-Migrations@1520138062(7750/9687 serialized/live bytes, 1 ops)
INFO 10:29:50,856 Writing Memtable-Migrations@1520138062(7750/9687
serialized/live bytes, 1 ops)
INFO 10:29:50,856 Enqueuing flush of
Memtable-Schema@347459675(3630/4537 serialized/live bytes, 3 ops)
INFO 10:29:50,878 Completed flushing
/var/lib/cassandra/data/system/Migrations-h-3-Data.db (7814 bytes)
INFO 10:29:50,879 Writing Memtable-Schema@347459675(3630/4537
serialized/live bytes, 3 ops)
INFO 10:29:50,894 Completed flushing
/var/lib/cassandra/data/system/Schema-h-3-Data.db (3780 bytes)
INFO 10:29:50,900 Applying migration
4ba1ae60-1511-11e1-0000-c11bc95834d7 Add column family:
org.apache.cassandra.config.CFMetaData@6a095b8a[cfId=1002,ksName=MSA,cfName=participants,cfType=Standard,comparator=org.apache.cassandra.db.marshal.ReversedType(org.apache.cassandra.db.marshal.BytesType),subcolumncomparator=<null>,comment=,rowCacheSize=0.0,keyCacheSize=1000000.0,readRepairChance=1.0,replicateOnWrite=true,gcGraceSeconds=3600,defaultValidator=org.apache.cassandra.db.marshal.BytesType,keyValidator=org.apache.cassandra.db.marshal.BytesType,minCompactionThreshold=4,maxCompactionThreshold=32,rowCacheSavePeriodInSeconds=0,keyCacheSavePeriodInSeconds=14400,rowCacheKeysToSave=2147483647,rowCacheProvider=org.apache.cassandra.cache.SerializingCacheProvider@c58f769,mergeShardsChance=0.1,keyAlias=<null>,column_metadata={},compactionStrategyClass=class
org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy,compactionStrategyOptions={},compressionOptions={}]
INFO 10:29:50,900 Enqueuing flush of
Memtable-Migrations@618337492(8194/10242 serialized/live bytes, 1 ops)
INFO 10:29:50,901 Writing Memtable-Migrations@618337492(8194/10242
serialized/live bytes, 1 ops)
INFO 10:29:50,902 Enqueuing flush of
Memtable-Schema@724860211(4020/5025 serialized/live bytes, 3 ops)
INFO 10:29:50,917 Completed flushing
/var/lib/cassandra/data/system/Migrations-h-4-Data.db (8258 bytes)
INFO 10:29:50,918 Writing Memtable-Schema@724860211(4020/5025
serialized/live bytes, 3 ops)
INFO 10:29:50,925 Compacting
[SSTableReader(path='/var/lib/cassandra/data/system/Migrations-h-1-Data.db'),
SSTableReader(path='/var/lib/cassandra/data/system/Migrations-h-2-Data.db'),
SSTableReader(path='/var/lib/cassandra/data/system/Migrations-h-4-Data.db'),
SSTableReader(path='/var/lib/cassandra/data/system/Migrations-h-3-Data.db')]
INFO 10:29:50,934 Completed flushing
/var/lib/cassandra/data/system/Schema-h-4-Data.db (4170 bytes)
INFO 10:29:50,935 Compacting
[SSTableReader(path='/var/lib/cassandra/data/system/Schema-h-2-Data.db'),
SSTableReader(path='/var/lib/cassandra/data/system/Schema-h-1-Data.db'),
SSTableReader(path='/var/lib/cassandra/data/system/Schema-h-4-Data.db'),
SSTableReader(path='/var/lib/cassandra/data/system/Schema-h-3-Data.db')]
INFO 10:29:50,940 Applying migration
4bb4e840-1511-11e1-0000-c11bc95834d7 Add column family:
org.apache.cassandra.config.CFMetaData@318c69a9[cfId=1003,ksName=MSA,cfName=subinfo,cfType=Standard,comparator=org.apache.cassandra.db.marshal.ReversedType(org.apache.cassandra.db.marshal.BytesType),subcolumncomparator=<null>,comment=,rowCacheSize=5000.0,keyCacheSize=5000000.0,readRepairChance=1.0,replicateOnWrite=true,gcGraceSeconds=3600,defaultValidator=org.apache.cassandra.db.marshal.BytesType,keyValidator=org.apache.cassandra.db.marshal.BytesType,minCompactionThreshold=4,maxCompactionThreshold=32,rowCacheSavePeriodInSeconds=14400,keyCacheSavePeriodInSeconds=14400,rowCacheKeysToSave=2147483647,rowCacheProvider=org.apache.cassandra.cache.SerializingCacheProvider@796cefa8,mergeShardsChance=0.1,keyAlias=<null>,column_metadata={},compactionStrategyClass=class
org.apache.cassandra.db.compaction.LeveledCompactionStrategy,compactionStrategyOptions={sstable_size_in_mb=10},compressionOptions={}]
INFO 10:29:50,941 Enqueuing flush of
Memtable-Migrations@1682081063(8618/10772 serialized/live bytes, 1
ops)
INFO 10:29:50,941 Writing Memtable-Migrations@1682081063(8618/10772
serialized/live bytes, 1 ops)
INFO 10:29:50,941 Enqueuing flush of
Memtable-Schema@1083461053(4427/5533 serialized/live bytes, 3 ops)
INFO 10:29:50,977 Completed flushing
/var/lib/cassandra/data/system/Migrations-h-5-Data.db (8682 bytes)
INFO 10:29:50,978 Writing Memtable-Schema@1083461053(4427/5533
serialized/live bytes, 3 ops)
INFO 10:29:50,991 Compacted to
[/var/lib/cassandra/data/system/Schema-h-5-Data.db,]. 14,433 to
14,106 (~97% of original) bytes for 5 keys at 0.269051MB/s. Time:
50ms.
INFO 10:29:50,995 Completed flushing
/var/lib/cassandra/data/system/Schema-h-7-Data.db (4577 bytes)
INFO 10:29:51,000 Applying migration
4bc6e9a0-1511-11e1-0000-c11bc95834d7 Add column family:
org.apache.cassandra.config.CFMetaData@20b00ec2[cfId=1004,ksName=MSA,cfName=transactions,cfType=Standard,comparator=org.apache.cassandra.db.marshal.ReversedType(org.apache.cassandra.db.marshal.BytesType),subcolumncomparator=<null>,comment=,rowCacheSize=0.0,keyCacheSize=0.0,readRepairChance=1.0,replicateOnWrite=true,gcGraceSeconds=3600,defaultValidator=org.apache.cassandra.db.marshal.BytesType,keyValidator=org.apache.cassandra.db.marshal.BytesType,minCompactionThreshold=4,maxCompactionThreshold=32,rowCacheSavePeriodInSeconds=0,keyCacheSavePeriodInSeconds=0,rowCacheKeysToSave=2147483647,rowCacheProvider=org.apache.cassandra.cache.SerializingCacheProvider@698f352,mergeShardsChance=0.1,keyAlias=<null>,column_metadata={},compactionStrategyClass=class
org.apache.cassandra.db.compaction.LeveledCompactionStrategy,compactionStrategyOptions={sstable_size_in_mb=10},compressionOptions={}]
INFO 10:29:51,001 Enqueuing flush of
Memtable-Migrations@596545504(9027/11283 serialized/live bytes, 1 ops)
INFO 10:29:51,002 Writing Memtable-Migrations@596545504(9027/11283
serialized/live bytes, 1 ops)
INFO 10:29:51,003 Enqueuing flush of
Memtable-Schema@1686621532(4835/6043 serialized/live bytes, 3 ops)
INFO 10:29:51,029 Completed flushing
/var/lib/cassandra/data/system/Migrations-h-7-Data.db (9091 bytes)
INFO 10:29:51,029 Writing Memtable-Schema@1686621532(4835/6043
serialized/live bytes, 3 ops)
INFO 10:29:51,031 Compacted to
[/var/lib/cassandra/data/system/Migrations-h-6-Data.db,]. 30,426 to
30,234 (~99% of original) bytes for 1 keys at 0.272013MB/s. Time:
106ms.
INFO 10:29:51,044 Completed flushing
/var/lib/cassandra/data/system/Schema-h-8-Data.db (4985 bytes)
INFO 10:29:51,049 Applying migration
4bd76460-1511-11e1-0000-c11bc95834d7 Add column family:
org.apache.cassandra.config.CFMetaData@4ab4faeb[cfId=1005,ksName=MSA,cfName=uid,cfType=Standard,comparator=org.apache.cassandra.db.marshal.ReversedType(org.apache.cassandra.db.marshal.BytesType),subcolumncomparator=<null>,comment=,rowCacheSize=0.0,keyCacheSize=1500000.0,readRepairChance=1.0,replicateOnWrite=true,gcGraceSeconds=3600,defaultValidator=org.apache.cassandra.db.marshal.BytesType,keyValidator=org.apache.cassandra.db.marshal.BytesType,minCompactionThreshold=4,maxCompactionThreshold=32,rowCacheSavePeriodInSeconds=0,keyCacheSavePeriodInSeconds=14400,rowCacheKeysToSave=2147483647,rowCacheProvider=org.apache.cassandra.cache.SerializingCacheProvider@2fc5809e,mergeShardsChance=0.1,keyAlias=<null>,column_metadata={},compactionStrategyClass=class
org.apache.cassandra.db.compaction.LeveledCompactionStrategy,compactionStrategyOptions={sstable_size_in_mb=10},compressionOptions={}]
INFO 10:29:51,050 Enqueuing flush of
Memtable-Migrations@1333730706(9421/11776 serialized/live bytes, 1
ops)
INFO 10:29:51,050 Writing Memtable-Migrations@1333730706(9421/11776
serialized/live bytes, 1 ops)
INFO 10:29:51,051 Enqueuing flush of
Memtable-Schema@577668356(5236/6545 serialized/live bytes, 3 ops)
INFO 10:29:51,065 Completed flushing
/var/lib/cassandra/data/system/Migrations-h-9-Data.db (9485 bytes)
INFO 10:29:51,066 Compacting
[SSTableReader(path='/var/lib/cassandra/data/system/Migrations-h-6-Data.db'),
SSTableReader(path='/var/lib/cassandra/data/system/Migrations-h-9-Data.db'),
SSTableReader(path='/var/lib/cassandra/data/system/Migrations-h-7-Data.db'),
SSTableReader(path='/var/lib/cassandra/data/system/Migrations-h-5-Data.db')]
INFO 10:29:51,066 Writing Memtable-Schema@577668356(5236/6545
serialized/live bytes, 3 ops)
INFO 10:29:51,081 Completed flushing
/var/lib/cassandra/data/system/Schema-h-9-Data.db (5386 bytes)
INFO 10:29:51,083 Compacting
[SSTableReader(path='/var/lib/cassandra/data/system/Schema-h-5-Data.db'),
SSTableReader(path='/var/lib/cassandra/data/system/Schema-h-9-Data.db'),
SSTableReader(path='/var/lib/cassandra/data/system/Schema-h-8-Data.db'),
SSTableReader(path='/var/lib/cassandra/data/system/Schema-h-7-Data.db')]
INFO 10:29:51,114 Compacted to
[/var/lib/cassandra/data/system/Schema-h-10-Data.db,]. 29,054 to
28,727 (~98% of original) bytes for 8 keys at 0.913207MB/s. Time:
30ms.
INFO 10:29:51,144 Compacted to
[/var/lib/cassandra/data/system/Migrations-h-10-Data.db,]. 57,492 to
57,300 (~99% of original) bytes for 1 keys at 0.700584MB/s. Time:
78ms.
INFO 10:29:51,410 Node /10.19.104.13 is now part of the cluster
INFO 10:29:51,412 InetAddress /10.19.104.13 is now UP
INFO 10:29:51,414 Enqueuing flush of
Memtable-LocationInfo@709342045(35/43 serialized/live bytes, 1 ops)
INFO 10:29:51,415 Writing Memtable-LocationInfo@709342045(35/43
serialized/live bytes, 1 ops)
INFO 10:29:51,428 Completed flushing
/var/lib/cassandra/data/system/LocationInfo-h-2-Data.db (89 bytes)
INFO 10:29:51,439 Node /10.19.104.12 is now part of the cluster
INFO 10:29:51,439 InetAddress /10.19.104.12 is now UP
INFO 10:29:51,441 Enqueuing flush of
Memtable-LocationInfo@1292444743(35/43 serialized/live bytes, 1 ops)
INFO 10:29:51,441 Writing Memtable-LocationInfo@1292444743(35/43
serialized/live bytes, 1 ops)
INFO 10:29:51,455 Completed flushing
/var/lib/cassandra/data/system/LocationInfo-h-3-Data.db (89 bytes)
INFO 10:29:51,456 Node /10.19.104.11 is now part of the cluster
INFO 10:29:51,457 InetAddress /10.19.104.11 is now UP
INFO 10:29:51,459 Enqueuing flush of
Memtable-LocationInfo@1891328597(20/25 serialized/live bytes, 1 ops)
INFO 10:29:51,459 Writing Memtable-LocationInfo@1891328597(20/25
serialized/live bytes, 1 ops)
INFO 10:29:51,471 Completed flushing
/var/lib/cassandra/data/system/LocationInfo-h-4-Data.db (74 bytes)
INFO 10:29:51,473 Compacting
[SSTableReader(path='/var/lib/cassandra/data/system/LocationInfo-h-2-Data.db'),
SSTableReader(path='/var/lib/cassandra/data/system/LocationInfo-h-4-Data.db'),
SSTableReader(path='/var/lib/cassandra/data/system/LocationInfo-h-1-Data.db'),
SSTableReader(path='/var/lib/cassandra/data/system/LocationInfo-h-3-Data.db')]
INFO 10:29:51,497 Compacted to
[/var/lib/cassandra/data/system/LocationInfo-h-5-Data.db,]. 552 to
444 (~80% of original) bytes for 3 keys at 0.018410MB/s. Time: 23ms.
INFO 10:30:19,349 JOINING: getting bootstrap token
INFO 10:30:19,352 Enqueuing flush of
Memtable-LocationInfo@225265367(36/45 serialized/live bytes, 1 ops)
INFO 10:30:19,353 Writing Memtable-LocationInfo@225265367(36/45
serialized/live bytes, 1 ops)
INFO 10:30:19,364 Completed flushing
/var/lib/cassandra/data/system/LocationInfo-h-7-Data.db (87 bytes)
INFO 10:30:19,374 JOINING: sleeping 30000 ms for pending range setup
INFO 10:30:49,375 JOINING: Starting to bootstrap...
ERROR 10:31:13,444 Fatal exception in thread Thread[Thread-49,5,main]
java.lang.AssertionError
at org.apache.cassandra.db.compaction.LeveledManifest.promote(LeveledManifest.java:178)
at org.apache.cassandra.db.compaction.LeveledCompactionStrategy.handleNotification(LeveledCompactionStrategy.java:141)
at org.apache.cassandra.db.DataTracker.notifySSTablesChanged(DataTracker.java:466)
at org.apache.cassandra.db.DataTracker.replace(DataTracker.java:275)
at org.apache.cassandra.db.DataTracker.addSSTables(DataTracker.java:237)
at org.apache.cassandra.db.DataTracker.addStreamedSSTable(DataTracker.java:242)
at org.apache.cassandra.db.ColumnFamilyStore.addSSTable(ColumnFamilyStore.java:922)
at org.apache.cassandra.streaming.StreamInSession.closeIfFinished(StreamInSession.java:141)
at org.apache.cassandra.streaming.IncomingStreamReader.read(IncomingStreamReader.java:102)
at org.apache.cassandra.net.IncomingTcpConnection.stream(IncomingTcpConnection.java:184)
at org.apache.cassandra.net.IncomingTcpConnection.run(IncomingTcpConnection.java:81)
Re: Assertion error during bootstraping cassandra 1.0.2
Posted by Ramesh Natarajan <ra...@gmail.com>.
https://issues.apache.org/jira/browse/CASSANDRA-3536
thanks
Ramesh
On Mon, Nov 28, 2011 at 5:59 AM, Sylvain Lebresne <sy...@datastax.com> wrote:
> I don't think you're doing anything wrong. Would you mind opening a ticket
> on JIRA (https://issues.apache.org/jira/browse/CASSANDRA) ?
>
> --
> Sylvain
>
> On Tue, Nov 22, 2011 at 5:35 PM, Ramesh Natarajan <ra...@gmail.com> wrote:
>> Hi,
>>
>> I have a 3 node cassandra cluster. I have RF set to 3 and do reads
>> and writes using QUORUM.
>>
>> Here is my initial ring configuration
>>
>> [root@CAP4-CNode1 ~]# /root/cassandra/bin/nodetool -h localhost ring
>> Address DC Rack Status State Load
>> Owns Token
>>
>> 113427455640312821154458202477256070484
>> 10.19.104.11 datacenter1 rack1 Up Normal 1.66 GB
>> 33.33% 0
>> 10.19.104.12 datacenter1 rack1 Up Normal 1.06 GB
>> 33.33% 56713727820156410577229101238628035242
>> 10.19.104.13 datacenter1 rack1 Up Normal 1.61 GB
>> 33.33% 113427455640312821154458202477256070484
>>
>> I want to add 10.19.104.14 to the cluster.
>>
>> I edited the 10.19.104.14 cassandra.yaml file and set the token to
>> 127605887595351923798765477786913079296 and set auto_bootstrap to
>> true.
>>
>> When I started cassandra I am getting Assertion Error. Can someone
>> point to me if i am doing anything wrong here?
>>
>> thanks
>> Ramesh
>>
>>
>>
>>
>> [root@CAP4-CNode4 cassandra]# INFO 10:29:46,093 Logging initialized
>> INFO 10:29:46,099 JVM vendor/version: Java HotSpot(TM) 64-Bit Server
>> VM/1.6.0_25
>> INFO 10:29:46,100 Heap size: 8304721920/8304721920
>> INFO 10:29:46,100 Classpath:
>> bin/../conf:bin/../build/classes/main:bin/../build/classes/thrift:bin/../lib/antlr-3.2.jar:bin/../lib/apache-cassandra-1.0.2.jar:bin/../lib/apache-cassandra-clientutil-1.0.2.jar:bin/../lib/apache-cassandra-thrift-1.0.2.jar:bin/../lib/avro-1.4.0-fixes.jar:bin/../lib/avro-1.4.0-sources-fixes.jar:bin/../lib/commons-cli-1.1.jar:bin/../lib/commons-codec-1.2.jar:bin/../lib/commons-lang-2.4.jar:bin/../lib/compress-lzf-0.8.4.jar:bin/../lib/concurrentlinkedhashmap-lru-1.2.jar:bin/../lib/guava-r08.jar:bin/../lib/high-scale-lib-1.1.2.jar:bin/../lib/jackson-core-asl-1.4.0.jar:bin/../lib/jackson-mapper-asl-1.4.0.jar:bin/../lib/jamm-0.2.5.jar:bin/../lib/jline-0.9.94.jar:bin/../lib/jna.jar:bin/../lib/json-simple-1.1.jar:bin/../lib/libthrift-0.6.jar:bin/../lib/log4j-1.2.16.jar:bin/../lib/mx4j-examples.jar:bin/../lib/mx4j-impl.jar:bin/../lib/mx4j.jar:bin/../lib/mx4j-jmx.jar:bin/../lib/mx4j-remote.jar:bin/../lib/mx4j-rimpl.jar:bin/../lib/mx4j-rjmx.jar:bin/../lib/mx4j-tools.jar:bin/../lib/servlet-api-2.5-20081211.jar:bin/../lib/slf4j-api-1.6.1.jar:bin/../lib/slf4j-log4j12-1.6.1.jar:bin/../lib/snakeyaml-1.6.jar:bin/../lib/snappy-java-1.0.4.1.jar:bin/../lib/jamm-0.2.5.jar
>> INFO 10:29:48,713 JNA mlockall successful
>> INFO 10:29:48,726 Loading settings from
>> file:/root/apache-cassandra-1.0.2/conf/cassandra.yaml
>> INFO 10:29:48,883 DiskAccessMode 'auto' determined to be mmap,
>> indexAccessMode is mmap
>> INFO 10:29:48,898 Global memtable threshold is enabled at 2640MB
>> INFO 10:29:49,203 Couldn't detect any schema definitions in local storage.
>> INFO 10:29:49,204 Found table data in data directories. Consider
>> using the CLI to define your schema.
>> INFO 10:29:49,220 Creating new commitlog segment
>> /var/lib/cassandra/commitlog/CommitLog-1321979389220.log
>> INFO 10:29:49,227 No commitlog files found; skipping replay
>> INFO 10:29:49,230 Cassandra version: 1.0.2
>> INFO 10:29:49,230 Thrift API version: 19.18.0
>> INFO 10:29:49,230 Loading persisted ring state
>> INFO 10:29:49,235 Starting up server gossip
>> INFO 10:29:49,259 Enqueuing flush of
>> Memtable-LocationInfo@122130810(192/240 serialized/live bytes, 4 ops)
>> INFO 10:29:49,260 Writing Memtable-LocationInfo@122130810(192/240
>> serialized/live bytes, 4 ops)
>> INFO 10:29:49,317 Completed flushing
>> /var/lib/cassandra/data/system/LocationInfo-h-1-Data.db (300 bytes)
>> INFO 10:29:49,340 Starting Messaging Service on port 7000
>> INFO 10:29:49,349 JOINING: waiting for ring and schema information
>> INFO 10:29:50,759 Applying migration
>> 4b0e20f0-1511-11e1-0000-c11bc95834d7 Add keyspace: MSA, rep
>> strategy:SimpleStrategy{}, durable_writes: true
>> INFO 10:29:50,761 Enqueuing flush of
>> Memtable-Migrations@1507565381(6744/8430 serialized/live bytes, 1 ops)
>> INFO 10:29:50,761 Writing Memtable-Migrations@1507565381(6744/8430
>> serialized/live bytes, 1 ops)
>> INFO 10:29:50,761 Enqueuing flush of
>> Memtable-Schema@1498835564(2889/3611 serialized/live bytes, 3 ops)
>> INFO 10:29:50,776 Completed flushing
>> /var/lib/cassandra/data/system/Migrations-h-1-Data.db (6808 bytes)
>> INFO 10:29:50,777 Writing Memtable-Schema@1498835564(2889/3611
>> serialized/live bytes, 3 ops)
>> INFO 10:29:50,797 Completed flushing
>> /var/lib/cassandra/data/system/Schema-h-1-Data.db (3039 bytes)
>> INFO 10:29:50,814 Applying migration
>> 4b6f2cb0-1511-11e1-0000-c11bc95834d7 Add column family:
>> org.apache.cassandra.config.CFMetaData@1639d811[cfId=1000,ksName=MSA,cfName=modseq,cfType=Standard,comparator=org.apache.cassandra.db.marshal.ReversedType(org.apache.cassandra.db.marshal.BytesType),subcolumncomparator=<null>,comment=,rowCacheSize=0.0,keyCacheSize=5000000.0,readRepairChance=1.0,replicateOnWrite=true,gcGraceSeconds=3600,defaultValidator=org.apache.cassandra.db.marshal.BytesType,keyValidator=org.apache.cassandra.db.marshal.BytesType,minCompactionThreshold=4,maxCompactionThreshold=32,rowCacheSavePeriodInSeconds=0,keyCacheSavePeriodInSeconds=14400,rowCacheKeysToSave=2147483647,rowCacheProvider=org.apache.cassandra.cache.SerializingCacheProvider@2f984f7d,mergeShardsChance=0.1,keyAlias=<null>,column_metadata={},compactionStrategyClass=class
>> org.apache.cassandra.db.compaction.LeveledCompactionStrategy,compactionStrategyOptions={sstable_size_in_mb=10},compressionOptions={}]
>> INFO 10:29:50,815 Enqueuing flush of
>> Memtable-Migrations@948613108(7482/9352 serialized/live bytes, 1 ops)
>> INFO 10:29:50,816 Writing Memtable-Migrations@948613108(7482/9352
>> serialized/live bytes, 1 ops)
>> INFO 10:29:50,816 Enqueuing flush of
>> Memtable-Schema@421910828(3294/4117 serialized/live bytes, 3 ops)
>> INFO 10:29:50,831 Completed flushing
>> /var/lib/cassandra/data/system/Migrations-h-2-Data.db (7546 bytes)
>> INFO 10:29:50,832 Writing Memtable-Schema@421910828(3294/4117
>> serialized/live bytes, 3 ops)
>> INFO 10:29:50,846 Completed flushing
>> /var/lib/cassandra/data/system/Schema-h-2-Data.db (3444 bytes)
>> INFO 10:29:50,854 Applying migration
>> 4b8c9fc0-1511-11e1-0000-c11bc95834d7 Add column family:
>> org.apache.cassandra.config.CFMetaData@1bd97d0d[cfId=1001,ksName=MSA,cfName=msgid,cfType=Standard,comparator=org.apache.cassandra.db.marshal.BytesType,subcolumncomparator=<null>,comment=,rowCacheSize=0.0,keyCacheSize=1000000.0,readRepairChance=1.0,replicateOnWrite=true,gcGraceSeconds=3600,defaultValidator=org.apache.cassandra.db.marshal.BytesType,keyValidator=org.apache.cassandra.db.marshal.BytesType,minCompactionThreshold=4,maxCompactionThreshold=32,rowCacheSavePeriodInSeconds=0,keyCacheSavePeriodInSeconds=14400,rowCacheKeysToSave=2147483647,rowCacheProvider=org.apache.cassandra.cache.SerializingCacheProvider@63a0eec3,mergeShardsChance=0.1,keyAlias=<null>,column_metadata={},compactionStrategyClass=class
>> org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy,compactionStrategyOptions={},compressionOptions={}]
>> INFO 10:29:50,855 Enqueuing flush of
>> Memtable-Migrations@1520138062(7750/9687 serialized/live bytes, 1 ops)
>> INFO 10:29:50,856 Writing Memtable-Migrations@1520138062(7750/9687
>> serialized/live bytes, 1 ops)
>> INFO 10:29:50,856 Enqueuing flush of
>> Memtable-Schema@347459675(3630/4537 serialized/live bytes, 3 ops)
>> INFO 10:29:50,878 Completed flushing
>> /var/lib/cassandra/data/system/Migrations-h-3-Data.db (7814 bytes)
>> INFO 10:29:50,879 Writing Memtable-Schema@347459675(3630/4537
>> serialized/live bytes, 3 ops)
>> INFO 10:29:50,894 Completed flushing
>> /var/lib/cassandra/data/system/Schema-h-3-Data.db (3780 bytes)
>> INFO 10:29:50,900 Applying migration
>> 4ba1ae60-1511-11e1-0000-c11bc95834d7 Add column family:
>> org.apache.cassandra.config.CFMetaData@6a095b8a[cfId=1002,ksName=MSA,cfName=participants,cfType=Standard,comparator=org.apache.cassandra.db.marshal.ReversedType(org.apache.cassandra.db.marshal.BytesType),subcolumncomparator=<null>,comment=,rowCacheSize=0.0,keyCacheSize=1000000.0,readRepairChance=1.0,replicateOnWrite=true,gcGraceSeconds=3600,defaultValidator=org.apache.cassandra.db.marshal.BytesType,keyValidator=org.apache.cassandra.db.marshal.BytesType,minCompactionThreshold=4,maxCompactionThreshold=32,rowCacheSavePeriodInSeconds=0,keyCacheSavePeriodInSeconds=14400,rowCacheKeysToSave=2147483647,rowCacheProvider=org.apache.cassandra.cache.SerializingCacheProvider@c58f769,mergeShardsChance=0.1,keyAlias=<null>,column_metadata={},compactionStrategyClass=class
>> org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy,compactionStrategyOptions={},compressionOptions={}]
>> INFO 10:29:50,900 Enqueuing flush of
>> Memtable-Migrations@618337492(8194/10242 serialized/live bytes, 1 ops)
>> INFO 10:29:50,901 Writing Memtable-Migrations@618337492(8194/10242
>> serialized/live bytes, 1 ops)
>> INFO 10:29:50,902 Enqueuing flush of
>> Memtable-Schema@724860211(4020/5025 serialized/live bytes, 3 ops)
>> INFO 10:29:50,917 Completed flushing
>> /var/lib/cassandra/data/system/Migrations-h-4-Data.db (8258 bytes)
>> INFO 10:29:50,918 Writing Memtable-Schema@724860211(4020/5025
>> serialized/live bytes, 3 ops)
>> INFO 10:29:50,925 Compacting
>> [SSTableReader(path='/var/lib/cassandra/data/system/Migrations-h-1-Data.db'),
>> SSTableReader(path='/var/lib/cassandra/data/system/Migrations-h-2-Data.db'),
>> SSTableReader(path='/var/lib/cassandra/data/system/Migrations-h-4-Data.db'),
>> SSTableReader(path='/var/lib/cassandra/data/system/Migrations-h-3-Data.db')]
>> INFO 10:29:50,934 Completed flushing
>> /var/lib/cassandra/data/system/Schema-h-4-Data.db (4170 bytes)
>> INFO 10:29:50,935 Compacting
>> [SSTableReader(path='/var/lib/cassandra/data/system/Schema-h-2-Data.db'),
>> SSTableReader(path='/var/lib/cassandra/data/system/Schema-h-1-Data.db'),
>> SSTableReader(path='/var/lib/cassandra/data/system/Schema-h-4-Data.db'),
>> SSTableReader(path='/var/lib/cassandra/data/system/Schema-h-3-Data.db')]
>> INFO 10:29:50,940 Applying migration
>> 4bb4e840-1511-11e1-0000-c11bc95834d7 Add column family:
>> org.apache.cassandra.config.CFMetaData@318c69a9[cfId=1003,ksName=MSA,cfName=subinfo,cfType=Standard,comparator=org.apache.cassandra.db.marshal.ReversedType(org.apache.cassandra.db.marshal.BytesType),subcolumncomparator=<null>,comment=,rowCacheSize=5000.0,keyCacheSize=5000000.0,readRepairChance=1.0,replicateOnWrite=true,gcGraceSeconds=3600,defaultValidator=org.apache.cassandra.db.marshal.BytesType,keyValidator=org.apache.cassandra.db.marshal.BytesType,minCompactionThreshold=4,maxCompactionThreshold=32,rowCacheSavePeriodInSeconds=14400,keyCacheSavePeriodInSeconds=14400,rowCacheKeysToSave=2147483647,rowCacheProvider=org.apache.cassandra.cache.SerializingCacheProvider@796cefa8,mergeShardsChance=0.1,keyAlias=<null>,column_metadata={},compactionStrategyClass=class
>> org.apache.cassandra.db.compaction.LeveledCompactionStrategy,compactionStrategyOptions={sstable_size_in_mb=10},compressionOptions={}]
>> INFO 10:29:50,941 Enqueuing flush of
>> Memtable-Migrations@1682081063(8618/10772 serialized/live bytes, 1
>> ops)
>> INFO 10:29:50,941 Writing Memtable-Migrations@1682081063(8618/10772
>> serialized/live bytes, 1 ops)
>> INFO 10:29:50,941 Enqueuing flush of
>> Memtable-Schema@1083461053(4427/5533 serialized/live bytes, 3 ops)
>> INFO 10:29:50,977 Completed flushing
>> /var/lib/cassandra/data/system/Migrations-h-5-Data.db (8682 bytes)
>> INFO 10:29:50,978 Writing Memtable-Schema@1083461053(4427/5533
>> serialized/live bytes, 3 ops)
>> INFO 10:29:50,991 Compacted to
>> [/var/lib/cassandra/data/system/Schema-h-5-Data.db,]. 14,433 to
>> 14,106 (~97% of original) bytes for 5 keys at 0.269051MB/s. Time:
>> 50ms.
>> INFO 10:29:50,995 Completed flushing
>> /var/lib/cassandra/data/system/Schema-h-7-Data.db (4577 bytes)
>> INFO 10:29:51,000 Applying migration
>> 4bc6e9a0-1511-11e1-0000-c11bc95834d7 Add column family:
>> org.apache.cassandra.config.CFMetaData@20b00ec2[cfId=1004,ksName=MSA,cfName=transactions,cfType=Standard,comparator=org.apache.cassandra.db.marshal.ReversedType(org.apache.cassandra.db.marshal.BytesType),subcolumncomparator=<null>,comment=,rowCacheSize=0.0,keyCacheSize=0.0,readRepairChance=1.0,replicateOnWrite=true,gcGraceSeconds=3600,defaultValidator=org.apache.cassandra.db.marshal.BytesType,keyValidator=org.apache.cassandra.db.marshal.BytesType,minCompactionThreshold=4,maxCompactionThreshold=32,rowCacheSavePeriodInSeconds=0,keyCacheSavePeriodInSeconds=0,rowCacheKeysToSave=2147483647,rowCacheProvider=org.apache.cassandra.cache.SerializingCacheProvider@698f352,mergeShardsChance=0.1,keyAlias=<null>,column_metadata={},compactionStrategyClass=class
>> org.apache.cassandra.db.compaction.LeveledCompactionStrategy,compactionStrategyOptions={sstable_size_in_mb=10},compressionOptions={}]
>> INFO 10:29:51,001 Enqueuing flush of
>> Memtable-Migrations@596545504(9027/11283 serialized/live bytes, 1 ops)
>> INFO 10:29:51,002 Writing Memtable-Migrations@596545504(9027/11283
>> serialized/live bytes, 1 ops)
>> INFO 10:29:51,003 Enqueuing flush of
>> Memtable-Schema@1686621532(4835/6043 serialized/live bytes, 3 ops)
>> INFO 10:29:51,029 Completed flushing
>> /var/lib/cassandra/data/system/Migrations-h-7-Data.db (9091 bytes)
>> INFO 10:29:51,029 Writing Memtable-Schema@1686621532(4835/6043
>> serialized/live bytes, 3 ops)
>> INFO 10:29:51,031 Compacted to
>> [/var/lib/cassandra/data/system/Migrations-h-6-Data.db,]. 30,426 to
>> 30,234 (~99% of original) bytes for 1 keys at 0.272013MB/s. Time:
>> 106ms.
>> INFO 10:29:51,044 Completed flushing
>> /var/lib/cassandra/data/system/Schema-h-8-Data.db (4985 bytes)
>> INFO 10:29:51,049 Applying migration
>> 4bd76460-1511-11e1-0000-c11bc95834d7 Add column family:
>> org.apache.cassandra.config.CFMetaData@4ab4faeb[cfId=1005,ksName=MSA,cfName=uid,cfType=Standard,comparator=org.apache.cassandra.db.marshal.ReversedType(org.apache.cassandra.db.marshal.BytesType),subcolumncomparator=<null>,comment=,rowCacheSize=0.0,keyCacheSize=1500000.0,readRepairChance=1.0,replicateOnWrite=true,gcGraceSeconds=3600,defaultValidator=org.apache.cassandra.db.marshal.BytesType,keyValidator=org.apache.cassandra.db.marshal.BytesType,minCompactionThreshold=4,maxCompactionThreshold=32,rowCacheSavePeriodInSeconds=0,keyCacheSavePeriodInSeconds=14400,rowCacheKeysToSave=2147483647,rowCacheProvider=org.apache.cassandra.cache.SerializingCacheProvider@2fc5809e,mergeShardsChance=0.1,keyAlias=<null>,column_metadata={},compactionStrategyClass=class
>> org.apache.cassandra.db.compaction.LeveledCompactionStrategy,compactionStrategyOptions={sstable_size_in_mb=10},compressionOptions={}]
>> INFO 10:29:51,050 Enqueuing flush of
>> Memtable-Migrations@1333730706(9421/11776 serialized/live bytes, 1
>> ops)
>> INFO 10:29:51,050 Writing Memtable-Migrations@1333730706(9421/11776
>> serialized/live bytes, 1 ops)
>> INFO 10:29:51,051 Enqueuing flush of
>> Memtable-Schema@577668356(5236/6545 serialized/live bytes, 3 ops)
>> INFO 10:29:51,065 Completed flushing
>> /var/lib/cassandra/data/system/Migrations-h-9-Data.db (9485 bytes)
>> INFO 10:29:51,066 Compacting
>> [SSTableReader(path='/var/lib/cassandra/data/system/Migrations-h-6-Data.db'),
>> SSTableReader(path='/var/lib/cassandra/data/system/Migrations-h-9-Data.db'),
>> SSTableReader(path='/var/lib/cassandra/data/system/Migrations-h-7-Data.db'),
>> SSTableReader(path='/var/lib/cassandra/data/system/Migrations-h-5-Data.db')]
>> INFO 10:29:51,066 Writing Memtable-Schema@577668356(5236/6545
>> serialized/live bytes, 3 ops)
>> INFO 10:29:51,081 Completed flushing
>> /var/lib/cassandra/data/system/Schema-h-9-Data.db (5386 bytes)
>> INFO 10:29:51,083 Compacting
>> [SSTableReader(path='/var/lib/cassandra/data/system/Schema-h-5-Data.db'),
>> SSTableReader(path='/var/lib/cassandra/data/system/Schema-h-9-Data.db'),
>> SSTableReader(path='/var/lib/cassandra/data/system/Schema-h-8-Data.db'),
>> SSTableReader(path='/var/lib/cassandra/data/system/Schema-h-7-Data.db')]
>> INFO 10:29:51,114 Compacted to
>> [/var/lib/cassandra/data/system/Schema-h-10-Data.db,]. 29,054 to
>> 28,727 (~98% of original) bytes for 8 keys at 0.913207MB/s. Time:
>> 30ms.
>> INFO 10:29:51,144 Compacted to
>> [/var/lib/cassandra/data/system/Migrations-h-10-Data.db,]. 57,492 to
>> 57,300 (~99% of original) bytes for 1 keys at 0.700584MB/s. Time:
>> 78ms.
>> INFO 10:29:51,410 Node /10.19.104.13 is now part of the cluster
>> INFO 10:29:51,412 InetAddress /10.19.104.13 is now UP
>> INFO 10:29:51,414 Enqueuing flush of
>> Memtable-LocationInfo@709342045(35/43 serialized/live bytes, 1 ops)
>> INFO 10:29:51,415 Writing Memtable-LocationInfo@709342045(35/43
>> serialized/live bytes, 1 ops)
>> INFO 10:29:51,428 Completed flushing
>> /var/lib/cassandra/data/system/LocationInfo-h-2-Data.db (89 bytes)
>> INFO 10:29:51,439 Node /10.19.104.12 is now part of the cluster
>> INFO 10:29:51,439 InetAddress /10.19.104.12 is now UP
>> INFO 10:29:51,441 Enqueuing flush of
>> Memtable-LocationInfo@1292444743(35/43 serialized/live bytes, 1 ops)
>> INFO 10:29:51,441 Writing Memtable-LocationInfo@1292444743(35/43
>> serialized/live bytes, 1 ops)
>> INFO 10:29:51,455 Completed flushing
>> /var/lib/cassandra/data/system/LocationInfo-h-3-Data.db (89 bytes)
>> INFO 10:29:51,456 Node /10.19.104.11 is now part of the cluster
>> INFO 10:29:51,457 InetAddress /10.19.104.11 is now UP
>> INFO 10:29:51,459 Enqueuing flush of
>> Memtable-LocationInfo@1891328597(20/25 serialized/live bytes, 1 ops)
>> INFO 10:29:51,459 Writing Memtable-LocationInfo@1891328597(20/25
>> serialized/live bytes, 1 ops)
>> INFO 10:29:51,471 Completed flushing
>> /var/lib/cassandra/data/system/LocationInfo-h-4-Data.db (74 bytes)
>> INFO 10:29:51,473 Compacting
>> [SSTableReader(path='/var/lib/cassandra/data/system/LocationInfo-h-2-Data.db'),
>> SSTableReader(path='/var/lib/cassandra/data/system/LocationInfo-h-4-Data.db'),
>> SSTableReader(path='/var/lib/cassandra/data/system/LocationInfo-h-1-Data.db'),
>> SSTableReader(path='/var/lib/cassandra/data/system/LocationInfo-h-3-Data.db')]
>> INFO 10:29:51,497 Compacted to
>> [/var/lib/cassandra/data/system/LocationInfo-h-5-Data.db,]. 552 to
>> 444 (~80% of original) bytes for 3 keys at 0.018410MB/s. Time: 23ms.
>> INFO 10:30:19,349 JOINING: getting bootstrap token
>> INFO 10:30:19,352 Enqueuing flush of
>> Memtable-LocationInfo@225265367(36/45 serialized/live bytes, 1 ops)
>> INFO 10:30:19,353 Writing Memtable-LocationInfo@225265367(36/45
>> serialized/live bytes, 1 ops)
>> INFO 10:30:19,364 Completed flushing
>> /var/lib/cassandra/data/system/LocationInfo-h-7-Data.db (87 bytes)
>> INFO 10:30:19,374 JOINING: sleeping 30000 ms for pending range setup
>> INFO 10:30:49,375 JOINING: Starting to bootstrap...
>> ERROR 10:31:13,444 Fatal exception in thread Thread[Thread-49,5,main]
>> java.lang.AssertionError
>> at org.apache.cassandra.db.compaction.LeveledManifest.promote(LeveledManifest.java:178)
>> at org.apache.cassandra.db.compaction.LeveledCompactionStrategy.handleNotification(LeveledCompactionStrategy.java:141)
>> at org.apache.cassandra.db.DataTracker.notifySSTablesChanged(DataTracker.java:466)
>> at org.apache.cassandra.db.DataTracker.replace(DataTracker.java:275)
>> at org.apache.cassandra.db.DataTracker.addSSTables(DataTracker.java:237)
>> at org.apache.cassandra.db.DataTracker.addStreamedSSTable(DataTracker.java:242)
>> at org.apache.cassandra.db.ColumnFamilyStore.addSSTable(ColumnFamilyStore.java:922)
>> at org.apache.cassandra.streaming.StreamInSession.closeIfFinished(StreamInSession.java:141)
>> at org.apache.cassandra.streaming.IncomingStreamReader.read(IncomingStreamReader.java:102)
>> at org.apache.cassandra.net.IncomingTcpConnection.stream(IncomingTcpConnection.java:184)
>> at org.apache.cassandra.net.IncomingTcpConnection.run(IncomingTcpConnection.java:81)
>>
>
Re: Assertion error during bootstraping cassandra 1.0.2
Posted by Sylvain Lebresne <sy...@datastax.com>.
I don't think you're doing anything wrong. Would you mind opening a ticket
on JIRA (https://issues.apache.org/jira/browse/CASSANDRA) ?
--
Sylvain
On Tue, Nov 22, 2011 at 5:35 PM, Ramesh Natarajan <ra...@gmail.com> wrote:
> Hi,
>
> I have a 3 node cassandra cluster. I have RF set to 3 and do reads
> and writes using QUORUM.
>
> Here is my initial ring configuration
>
> [root@CAP4-CNode1 ~]# /root/cassandra/bin/nodetool -h localhost ring
> Address DC Rack Status State Load
> Owns Token
>
> 113427455640312821154458202477256070484
> 10.19.104.11 datacenter1 rack1 Up Normal 1.66 GB
> 33.33% 0
> 10.19.104.12 datacenter1 rack1 Up Normal 1.06 GB
> 33.33% 56713727820156410577229101238628035242
> 10.19.104.13 datacenter1 rack1 Up Normal 1.61 GB
> 33.33% 113427455640312821154458202477256070484
>
> I want to add 10.19.104.14 to the cluster.
>
> I edited the 10.19.104.14 cassandra.yaml file and set the token to
> 127605887595351923798765477786913079296 and set auto_bootstrap to
> true.
>
> When I started cassandra I am getting Assertion Error. Can someone
> point to me if i am doing anything wrong here?
>
> thanks
> Ramesh
>
>
>
>
> [root@CAP4-CNode4 cassandra]# INFO 10:29:46,093 Logging initialized
> INFO 10:29:46,099 JVM vendor/version: Java HotSpot(TM) 64-Bit Server
> VM/1.6.0_25
> INFO 10:29:46,100 Heap size: 8304721920/8304721920
> INFO 10:29:46,100 Classpath:
> bin/../conf:bin/../build/classes/main:bin/../build/classes/thrift:bin/../lib/antlr-3.2.jar:bin/../lib/apache-cassandra-1.0.2.jar:bin/../lib/apache-cassandra-clientutil-1.0.2.jar:bin/../lib/apache-cassandra-thrift-1.0.2.jar:bin/../lib/avro-1.4.0-fixes.jar:bin/../lib/avro-1.4.0-sources-fixes.jar:bin/../lib/commons-cli-1.1.jar:bin/../lib/commons-codec-1.2.jar:bin/../lib/commons-lang-2.4.jar:bin/../lib/compress-lzf-0.8.4.jar:bin/../lib/concurrentlinkedhashmap-lru-1.2.jar:bin/../lib/guava-r08.jar:bin/../lib/high-scale-lib-1.1.2.jar:bin/../lib/jackson-core-asl-1.4.0.jar:bin/../lib/jackson-mapper-asl-1.4.0.jar:bin/../lib/jamm-0.2.5.jar:bin/../lib/jline-0.9.94.jar:bin/../lib/jna.jar:bin/../lib/json-simple-1.1.jar:bin/../lib/libthrift-0.6.jar:bin/../lib/log4j-1.2.16.jar:bin/../lib/mx4j-examples.jar:bin/../lib/mx4j-impl.jar:bin/../lib/mx4j.jar:bin/../lib/mx4j-jmx.jar:bin/../lib/mx4j-remote.jar:bin/../lib/mx4j-rimpl.jar:bin/../lib/mx4j-rjmx.jar:bin/../lib/mx4j-tools.jar:bin/../lib/servlet-api-2.5-20081211.jar:bin/../lib/slf4j-api-1.6.1.jar:bin/../lib/slf4j-log4j12-1.6.1.jar:bin/../lib/snakeyaml-1.6.jar:bin/../lib/snappy-java-1.0.4.1.jar:bin/../lib/jamm-0.2.5.jar
> INFO 10:29:48,713 JNA mlockall successful
> INFO 10:29:48,726 Loading settings from
> file:/root/apache-cassandra-1.0.2/conf/cassandra.yaml
> INFO 10:29:48,883 DiskAccessMode 'auto' determined to be mmap,
> indexAccessMode is mmap
> INFO 10:29:48,898 Global memtable threshold is enabled at 2640MB
> INFO 10:29:49,203 Couldn't detect any schema definitions in local storage.
> INFO 10:29:49,204 Found table data in data directories. Consider
> using the CLI to define your schema.
> INFO 10:29:49,220 Creating new commitlog segment
> /var/lib/cassandra/commitlog/CommitLog-1321979389220.log
> INFO 10:29:49,227 No commitlog files found; skipping replay
> INFO 10:29:49,230 Cassandra version: 1.0.2
> INFO 10:29:49,230 Thrift API version: 19.18.0
> INFO 10:29:49,230 Loading persisted ring state
> INFO 10:29:49,235 Starting up server gossip
> INFO 10:29:49,259 Enqueuing flush of
> Memtable-LocationInfo@122130810(192/240 serialized/live bytes, 4 ops)
> INFO 10:29:49,260 Writing Memtable-LocationInfo@122130810(192/240
> serialized/live bytes, 4 ops)
> INFO 10:29:49,317 Completed flushing
> /var/lib/cassandra/data/system/LocationInfo-h-1-Data.db (300 bytes)
> INFO 10:29:49,340 Starting Messaging Service on port 7000
> INFO 10:29:49,349 JOINING: waiting for ring and schema information
> INFO 10:29:50,759 Applying migration
> 4b0e20f0-1511-11e1-0000-c11bc95834d7 Add keyspace: MSA, rep
> strategy:SimpleStrategy{}, durable_writes: true
> INFO 10:29:50,761 Enqueuing flush of
> Memtable-Migrations@1507565381(6744/8430 serialized/live bytes, 1 ops)
> INFO 10:29:50,761 Writing Memtable-Migrations@1507565381(6744/8430
> serialized/live bytes, 1 ops)
> INFO 10:29:50,761 Enqueuing flush of
> Memtable-Schema@1498835564(2889/3611 serialized/live bytes, 3 ops)
> INFO 10:29:50,776 Completed flushing
> /var/lib/cassandra/data/system/Migrations-h-1-Data.db (6808 bytes)
> INFO 10:29:50,777 Writing Memtable-Schema@1498835564(2889/3611
> serialized/live bytes, 3 ops)
> INFO 10:29:50,797 Completed flushing
> /var/lib/cassandra/data/system/Schema-h-1-Data.db (3039 bytes)
> INFO 10:29:50,814 Applying migration
> 4b6f2cb0-1511-11e1-0000-c11bc95834d7 Add column family:
> org.apache.cassandra.config.CFMetaData@1639d811[cfId=1000,ksName=MSA,cfName=modseq,cfType=Standard,comparator=org.apache.cassandra.db.marshal.ReversedType(org.apache.cassandra.db.marshal.BytesType),subcolumncomparator=<null>,comment=,rowCacheSize=0.0,keyCacheSize=5000000.0,readRepairChance=1.0,replicateOnWrite=true,gcGraceSeconds=3600,defaultValidator=org.apache.cassandra.db.marshal.BytesType,keyValidator=org.apache.cassandra.db.marshal.BytesType,minCompactionThreshold=4,maxCompactionThreshold=32,rowCacheSavePeriodInSeconds=0,keyCacheSavePeriodInSeconds=14400,rowCacheKeysToSave=2147483647,rowCacheProvider=org.apache.cassandra.cache.SerializingCacheProvider@2f984f7d,mergeShardsChance=0.1,keyAlias=<null>,column_metadata={},compactionStrategyClass=class
> org.apache.cassandra.db.compaction.LeveledCompactionStrategy,compactionStrategyOptions={sstable_size_in_mb=10},compressionOptions={}]
> INFO 10:29:50,815 Enqueuing flush of
> Memtable-Migrations@948613108(7482/9352 serialized/live bytes, 1 ops)
> INFO 10:29:50,816 Writing Memtable-Migrations@948613108(7482/9352
> serialized/live bytes, 1 ops)
> INFO 10:29:50,816 Enqueuing flush of
> Memtable-Schema@421910828(3294/4117 serialized/live bytes, 3 ops)
> INFO 10:29:50,831 Completed flushing
> /var/lib/cassandra/data/system/Migrations-h-2-Data.db (7546 bytes)
> INFO 10:29:50,832 Writing Memtable-Schema@421910828(3294/4117
> serialized/live bytes, 3 ops)
> INFO 10:29:50,846 Completed flushing
> /var/lib/cassandra/data/system/Schema-h-2-Data.db (3444 bytes)
> INFO 10:29:50,854 Applying migration
> 4b8c9fc0-1511-11e1-0000-c11bc95834d7 Add column family:
> org.apache.cassandra.config.CFMetaData@1bd97d0d[cfId=1001,ksName=MSA,cfName=msgid,cfType=Standard,comparator=org.apache.cassandra.db.marshal.BytesType,subcolumncomparator=<null>,comment=,rowCacheSize=0.0,keyCacheSize=1000000.0,readRepairChance=1.0,replicateOnWrite=true,gcGraceSeconds=3600,defaultValidator=org.apache.cassandra.db.marshal.BytesType,keyValidator=org.apache.cassandra.db.marshal.BytesType,minCompactionThreshold=4,maxCompactionThreshold=32,rowCacheSavePeriodInSeconds=0,keyCacheSavePeriodInSeconds=14400,rowCacheKeysToSave=2147483647,rowCacheProvider=org.apache.cassandra.cache.SerializingCacheProvider@63a0eec3,mergeShardsChance=0.1,keyAlias=<null>,column_metadata={},compactionStrategyClass=class
> org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy,compactionStrategyOptions={},compressionOptions={}]
> INFO 10:29:50,855 Enqueuing flush of
> Memtable-Migrations@1520138062(7750/9687 serialized/live bytes, 1 ops)
> INFO 10:29:50,856 Writing Memtable-Migrations@1520138062(7750/9687
> serialized/live bytes, 1 ops)
> INFO 10:29:50,856 Enqueuing flush of
> Memtable-Schema@347459675(3630/4537 serialized/live bytes, 3 ops)
> INFO 10:29:50,878 Completed flushing
> /var/lib/cassandra/data/system/Migrations-h-3-Data.db (7814 bytes)
> INFO 10:29:50,879 Writing Memtable-Schema@347459675(3630/4537
> serialized/live bytes, 3 ops)
> INFO 10:29:50,894 Completed flushing
> /var/lib/cassandra/data/system/Schema-h-3-Data.db (3780 bytes)
> INFO 10:29:50,900 Applying migration
> 4ba1ae60-1511-11e1-0000-c11bc95834d7 Add column family:
> org.apache.cassandra.config.CFMetaData@6a095b8a[cfId=1002,ksName=MSA,cfName=participants,cfType=Standard,comparator=org.apache.cassandra.db.marshal.ReversedType(org.apache.cassandra.db.marshal.BytesType),subcolumncomparator=<null>,comment=,rowCacheSize=0.0,keyCacheSize=1000000.0,readRepairChance=1.0,replicateOnWrite=true,gcGraceSeconds=3600,defaultValidator=org.apache.cassandra.db.marshal.BytesType,keyValidator=org.apache.cassandra.db.marshal.BytesType,minCompactionThreshold=4,maxCompactionThreshold=32,rowCacheSavePeriodInSeconds=0,keyCacheSavePeriodInSeconds=14400,rowCacheKeysToSave=2147483647,rowCacheProvider=org.apache.cassandra.cache.SerializingCacheProvider@c58f769,mergeShardsChance=0.1,keyAlias=<null>,column_metadata={},compactionStrategyClass=class
> org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy,compactionStrategyOptions={},compressionOptions={}]
> INFO 10:29:50,900 Enqueuing flush of
> Memtable-Migrations@618337492(8194/10242 serialized/live bytes, 1 ops)
> INFO 10:29:50,901 Writing Memtable-Migrations@618337492(8194/10242
> serialized/live bytes, 1 ops)
> INFO 10:29:50,902 Enqueuing flush of
> Memtable-Schema@724860211(4020/5025 serialized/live bytes, 3 ops)
> INFO 10:29:50,917 Completed flushing
> /var/lib/cassandra/data/system/Migrations-h-4-Data.db (8258 bytes)
> INFO 10:29:50,918 Writing Memtable-Schema@724860211(4020/5025
> serialized/live bytes, 3 ops)
> INFO 10:29:50,925 Compacting
> [SSTableReader(path='/var/lib/cassandra/data/system/Migrations-h-1-Data.db'),
> SSTableReader(path='/var/lib/cassandra/data/system/Migrations-h-2-Data.db'),
> SSTableReader(path='/var/lib/cassandra/data/system/Migrations-h-4-Data.db'),
> SSTableReader(path='/var/lib/cassandra/data/system/Migrations-h-3-Data.db')]
> INFO 10:29:50,934 Completed flushing
> /var/lib/cassandra/data/system/Schema-h-4-Data.db (4170 bytes)
> INFO 10:29:50,935 Compacting
> [SSTableReader(path='/var/lib/cassandra/data/system/Schema-h-2-Data.db'),
> SSTableReader(path='/var/lib/cassandra/data/system/Schema-h-1-Data.db'),
> SSTableReader(path='/var/lib/cassandra/data/system/Schema-h-4-Data.db'),
> SSTableReader(path='/var/lib/cassandra/data/system/Schema-h-3-Data.db')]
> INFO 10:29:50,940 Applying migration
> 4bb4e840-1511-11e1-0000-c11bc95834d7 Add column family:
> org.apache.cassandra.config.CFMetaData@318c69a9[cfId=1003,ksName=MSA,cfName=subinfo,cfType=Standard,comparator=org.apache.cassandra.db.marshal.ReversedType(org.apache.cassandra.db.marshal.BytesType),subcolumncomparator=<null>,comment=,rowCacheSize=5000.0,keyCacheSize=5000000.0,readRepairChance=1.0,replicateOnWrite=true,gcGraceSeconds=3600,defaultValidator=org.apache.cassandra.db.marshal.BytesType,keyValidator=org.apache.cassandra.db.marshal.BytesType,minCompactionThreshold=4,maxCompactionThreshold=32,rowCacheSavePeriodInSeconds=14400,keyCacheSavePeriodInSeconds=14400,rowCacheKeysToSave=2147483647,rowCacheProvider=org.apache.cassandra.cache.SerializingCacheProvider@796cefa8,mergeShardsChance=0.1,keyAlias=<null>,column_metadata={},compactionStrategyClass=class
> org.apache.cassandra.db.compaction.LeveledCompactionStrategy,compactionStrategyOptions={sstable_size_in_mb=10},compressionOptions={}]
> INFO 10:29:50,941 Enqueuing flush of
> Memtable-Migrations@1682081063(8618/10772 serialized/live bytes, 1
> ops)
> INFO 10:29:50,941 Writing Memtable-Migrations@1682081063(8618/10772
> serialized/live bytes, 1 ops)
> INFO 10:29:50,941 Enqueuing flush of
> Memtable-Schema@1083461053(4427/5533 serialized/live bytes, 3 ops)
> INFO 10:29:50,977 Completed flushing
> /var/lib/cassandra/data/system/Migrations-h-5-Data.db (8682 bytes)
> INFO 10:29:50,978 Writing Memtable-Schema@1083461053(4427/5533
> serialized/live bytes, 3 ops)
> INFO 10:29:50,991 Compacted to
> [/var/lib/cassandra/data/system/Schema-h-5-Data.db,]. 14,433 to
> 14,106 (~97% of original) bytes for 5 keys at 0.269051MB/s. Time:
> 50ms.
> INFO 10:29:50,995 Completed flushing
> /var/lib/cassandra/data/system/Schema-h-7-Data.db (4577 bytes)
> INFO 10:29:51,000 Applying migration
> 4bc6e9a0-1511-11e1-0000-c11bc95834d7 Add column family:
> org.apache.cassandra.config.CFMetaData@20b00ec2[cfId=1004,ksName=MSA,cfName=transactions,cfType=Standard,comparator=org.apache.cassandra.db.marshal.ReversedType(org.apache.cassandra.db.marshal.BytesType),subcolumncomparator=<null>,comment=,rowCacheSize=0.0,keyCacheSize=0.0,readRepairChance=1.0,replicateOnWrite=true,gcGraceSeconds=3600,defaultValidator=org.apache.cassandra.db.marshal.BytesType,keyValidator=org.apache.cassandra.db.marshal.BytesType,minCompactionThreshold=4,maxCompactionThreshold=32,rowCacheSavePeriodInSeconds=0,keyCacheSavePeriodInSeconds=0,rowCacheKeysToSave=2147483647,rowCacheProvider=org.apache.cassandra.cache.SerializingCacheProvider@698f352,mergeShardsChance=0.1,keyAlias=<null>,column_metadata={},compactionStrategyClass=class
> org.apache.cassandra.db.compaction.LeveledCompactionStrategy,compactionStrategyOptions={sstable_size_in_mb=10},compressionOptions={}]
> INFO 10:29:51,001 Enqueuing flush of
> Memtable-Migrations@596545504(9027/11283 serialized/live bytes, 1 ops)
> INFO 10:29:51,002 Writing Memtable-Migrations@596545504(9027/11283
> serialized/live bytes, 1 ops)
> INFO 10:29:51,003 Enqueuing flush of
> Memtable-Schema@1686621532(4835/6043 serialized/live bytes, 3 ops)
> INFO 10:29:51,029 Completed flushing
> /var/lib/cassandra/data/system/Migrations-h-7-Data.db (9091 bytes)
> INFO 10:29:51,029 Writing Memtable-Schema@1686621532(4835/6043
> serialized/live bytes, 3 ops)
> INFO 10:29:51,031 Compacted to
> [/var/lib/cassandra/data/system/Migrations-h-6-Data.db,]. 30,426 to
> 30,234 (~99% of original) bytes for 1 keys at 0.272013MB/s. Time:
> 106ms.
> INFO 10:29:51,044 Completed flushing
> /var/lib/cassandra/data/system/Schema-h-8-Data.db (4985 bytes)
> INFO 10:29:51,049 Applying migration
> 4bd76460-1511-11e1-0000-c11bc95834d7 Add column family:
> org.apache.cassandra.config.CFMetaData@4ab4faeb[cfId=1005,ksName=MSA,cfName=uid,cfType=Standard,comparator=org.apache.cassandra.db.marshal.ReversedType(org.apache.cassandra.db.marshal.BytesType),subcolumncomparator=<null>,comment=,rowCacheSize=0.0,keyCacheSize=1500000.0,readRepairChance=1.0,replicateOnWrite=true,gcGraceSeconds=3600,defaultValidator=org.apache.cassandra.db.marshal.BytesType,keyValidator=org.apache.cassandra.db.marshal.BytesType,minCompactionThreshold=4,maxCompactionThreshold=32,rowCacheSavePeriodInSeconds=0,keyCacheSavePeriodInSeconds=14400,rowCacheKeysToSave=2147483647,rowCacheProvider=org.apache.cassandra.cache.SerializingCacheProvider@2fc5809e,mergeShardsChance=0.1,keyAlias=<null>,column_metadata={},compactionStrategyClass=class
> org.apache.cassandra.db.compaction.LeveledCompactionStrategy,compactionStrategyOptions={sstable_size_in_mb=10},compressionOptions={}]
> INFO 10:29:51,050 Enqueuing flush of
> Memtable-Migrations@1333730706(9421/11776 serialized/live bytes, 1
> ops)
> INFO 10:29:51,050 Writing Memtable-Migrations@1333730706(9421/11776
> serialized/live bytes, 1 ops)
> INFO 10:29:51,051 Enqueuing flush of
> Memtable-Schema@577668356(5236/6545 serialized/live bytes, 3 ops)
> INFO 10:29:51,065 Completed flushing
> /var/lib/cassandra/data/system/Migrations-h-9-Data.db (9485 bytes)
> INFO 10:29:51,066 Compacting
> [SSTableReader(path='/var/lib/cassandra/data/system/Migrations-h-6-Data.db'),
> SSTableReader(path='/var/lib/cassandra/data/system/Migrations-h-9-Data.db'),
> SSTableReader(path='/var/lib/cassandra/data/system/Migrations-h-7-Data.db'),
> SSTableReader(path='/var/lib/cassandra/data/system/Migrations-h-5-Data.db')]
> INFO 10:29:51,066 Writing Memtable-Schema@577668356(5236/6545
> serialized/live bytes, 3 ops)
> INFO 10:29:51,081 Completed flushing
> /var/lib/cassandra/data/system/Schema-h-9-Data.db (5386 bytes)
> INFO 10:29:51,083 Compacting
> [SSTableReader(path='/var/lib/cassandra/data/system/Schema-h-5-Data.db'),
> SSTableReader(path='/var/lib/cassandra/data/system/Schema-h-9-Data.db'),
> SSTableReader(path='/var/lib/cassandra/data/system/Schema-h-8-Data.db'),
> SSTableReader(path='/var/lib/cassandra/data/system/Schema-h-7-Data.db')]
> INFO 10:29:51,114 Compacted to
> [/var/lib/cassandra/data/system/Schema-h-10-Data.db,]. 29,054 to
> 28,727 (~98% of original) bytes for 8 keys at 0.913207MB/s. Time:
> 30ms.
> INFO 10:29:51,144 Compacted to
> [/var/lib/cassandra/data/system/Migrations-h-10-Data.db,]. 57,492 to
> 57,300 (~99% of original) bytes for 1 keys at 0.700584MB/s. Time:
> 78ms.
> INFO 10:29:51,410 Node /10.19.104.13 is now part of the cluster
> INFO 10:29:51,412 InetAddress /10.19.104.13 is now UP
> INFO 10:29:51,414 Enqueuing flush of
> Memtable-LocationInfo@709342045(35/43 serialized/live bytes, 1 ops)
> INFO 10:29:51,415 Writing Memtable-LocationInfo@709342045(35/43
> serialized/live bytes, 1 ops)
> INFO 10:29:51,428 Completed flushing
> /var/lib/cassandra/data/system/LocationInfo-h-2-Data.db (89 bytes)
> INFO 10:29:51,439 Node /10.19.104.12 is now part of the cluster
> INFO 10:29:51,439 InetAddress /10.19.104.12 is now UP
> INFO 10:29:51,441 Enqueuing flush of
> Memtable-LocationInfo@1292444743(35/43 serialized/live bytes, 1 ops)
> INFO 10:29:51,441 Writing Memtable-LocationInfo@1292444743(35/43
> serialized/live bytes, 1 ops)
> INFO 10:29:51,455 Completed flushing
> /var/lib/cassandra/data/system/LocationInfo-h-3-Data.db (89 bytes)
> INFO 10:29:51,456 Node /10.19.104.11 is now part of the cluster
> INFO 10:29:51,457 InetAddress /10.19.104.11 is now UP
> INFO 10:29:51,459 Enqueuing flush of
> Memtable-LocationInfo@1891328597(20/25 serialized/live bytes, 1 ops)
> INFO 10:29:51,459 Writing Memtable-LocationInfo@1891328597(20/25
> serialized/live bytes, 1 ops)
> INFO 10:29:51,471 Completed flushing
> /var/lib/cassandra/data/system/LocationInfo-h-4-Data.db (74 bytes)
> INFO 10:29:51,473 Compacting
> [SSTableReader(path='/var/lib/cassandra/data/system/LocationInfo-h-2-Data.db'),
> SSTableReader(path='/var/lib/cassandra/data/system/LocationInfo-h-4-Data.db'),
> SSTableReader(path='/var/lib/cassandra/data/system/LocationInfo-h-1-Data.db'),
> SSTableReader(path='/var/lib/cassandra/data/system/LocationInfo-h-3-Data.db')]
> INFO 10:29:51,497 Compacted to
> [/var/lib/cassandra/data/system/LocationInfo-h-5-Data.db,]. 552 to
> 444 (~80% of original) bytes for 3 keys at 0.018410MB/s. Time: 23ms.
> INFO 10:30:19,349 JOINING: getting bootstrap token
> INFO 10:30:19,352 Enqueuing flush of
> Memtable-LocationInfo@225265367(36/45 serialized/live bytes, 1 ops)
> INFO 10:30:19,353 Writing Memtable-LocationInfo@225265367(36/45
> serialized/live bytes, 1 ops)
> INFO 10:30:19,364 Completed flushing
> /var/lib/cassandra/data/system/LocationInfo-h-7-Data.db (87 bytes)
> INFO 10:30:19,374 JOINING: sleeping 30000 ms for pending range setup
> INFO 10:30:49,375 JOINING: Starting to bootstrap...
> ERROR 10:31:13,444 Fatal exception in thread Thread[Thread-49,5,main]
> java.lang.AssertionError
> at org.apache.cassandra.db.compaction.LeveledManifest.promote(LeveledManifest.java:178)
> at org.apache.cassandra.db.compaction.LeveledCompactionStrategy.handleNotification(LeveledCompactionStrategy.java:141)
> at org.apache.cassandra.db.DataTracker.notifySSTablesChanged(DataTracker.java:466)
> at org.apache.cassandra.db.DataTracker.replace(DataTracker.java:275)
> at org.apache.cassandra.db.DataTracker.addSSTables(DataTracker.java:237)
> at org.apache.cassandra.db.DataTracker.addStreamedSSTable(DataTracker.java:242)
> at org.apache.cassandra.db.ColumnFamilyStore.addSSTable(ColumnFamilyStore.java:922)
> at org.apache.cassandra.streaming.StreamInSession.closeIfFinished(StreamInSession.java:141)
> at org.apache.cassandra.streaming.IncomingStreamReader.read(IncomingStreamReader.java:102)
> at org.apache.cassandra.net.IncomingTcpConnection.stream(IncomingTcpConnection.java:184)
> at org.apache.cassandra.net.IncomingTcpConnection.run(IncomingTcpConnection.java:81)
>